Executive Committee Meeting on 27 September 2006

The weekly progress report round included

1. Pete Bernardin’s continuing to work on concurrent courses for Fall 2006
2. Serenity King’s still trying to round up another 150+ syllabi for the term and working on a newsletter
3. Simon Kane’s having launched some mods for the online assessment tool for core courses, having launched the online upward evaluation tool, having updated and locked down namebook, listbook and coursebook, and beginning to work on the compliance reporting tool
4. Abby Kratz’s decision to alter her strategy for fall credentialing
5. JoyLynn Reed’s planning to revise the assessment manual to reflect more closely the current process and sequence
6. Michael Coleman’s having verified the extreme differences among regional accrediting agencies across the country
7. Larry Redlinger’s planning to expend graduation rate analysis across the coming year
8. Metta Alsobrook’s continuing efforts to monitor the progress of all aspects of the committee’s projects.

We also discussed a modified approach to numbering the principles for the purposes of data collection.  As there continues to be talk that the Commission may change the principles (as proposed earlier this year), we are hesitant to collect data based on the old numbering system or the new numbering system, as either could become the system we will use during our visit - or we could wind up with a new set of principles as yet unseen.  Thus, we’ve built a “cross walk” table of the principles using both the old and new numbering scheme - and we’ve added a local id as well.  That local id will be used to label a sequence of “folders” on a server for committees to work and store materials.  That id will also be used in the database for the storage and eventual display of all compliance documents.  We’ll simply substitute the “new” number once that’s decided.

We also agreed to move the next Steering Committee meeting to 11 October - a lunch meeting with President Daniel.

Okay, so what does all that mean exactly?  We’re still pushing hard to move toward improving our own internal processes as well as our interactions with all the committee members across campus.  There’s so much work to do, and we’re trying to do as much as we can so others don’t have to do quite as much.

On another note, just because I’m already writing anyway, it seems that some folks aren’t exactly thrilled with using online tools for some of the assessments - and this includes both our own home-grown apps and the subscribed WEAVEonline.  Hmmm.  Let me throw exactly one argument for using the online tools:  once you hit that submit button, it’s no longer your problem if the file is damaged, lost, or stolen (?).  If you email another file to us, you’re still on the hook.

And Congratulations! to our friends at UT Arlington.  Your site looks really good, and all that hard work has paid off.  We’re impressed!

Yesterday, Reena Schellenberg mentioned that one of her students had seen a “QEP” sign on campus and said something like, “Oh, yeah, I’ve heard of that.”  Nice! Without benefit of an official kick-off or some big celebration, the QEP process is already making headway.

John Sibert has been beating the bushes for the last couple of months, seeking input, ideas, and suggestions for not only QEP topics but also ideas about just what’s good or bad about the education here and about what could make things better.  Student Government was particularly interested and has, as a result, starting pushing the whole idea of student involvement with signs across campus.

As one of those old campus radical types (okay, maybe just campus liberal types) from the late sixties and early seventies, I am so glad to see some signs of life from the student body here.  Students have, at least from my perspective, been remarkably passive.  Now they may have found a way to gear up their own level of activism (okay, maybe it’s just more active participation) - but it’s a start.

So, if you’re interested in joining the conversation, take a stroll over to the QEP blog.

QEP blog in a new window

After Wednesday’s meeting, it became clear that we are struggling with priorities and the difference between “the forest and the trees.” We seem to spend so much time on the little details: WEAVEonline, common core course assessment, program assessment, syllabi collection, tagging and indexing and filing and so on and so on and so on.

Okay, it’s an easy trap.  After all, it’s hard to conceptualize the “forest” without all those trees.”  As a result, we occasionally overlook the forest because we are stuck on one tree or another. A case in point:  While it is true that the program and course assessments, and all the other detailed work, provide a strong case for what we do, it is the “what we do” that is more important.  As such, we are now re-focusing on the forest:  The Principles themselves.

With the recent changes in accreditation procedures, it is the compliance certification that forms the basis of our real review (or so it seems).  The QEP, also, plays a major role in our success or failure in this endeavor.  We’re getting “back to the basics” by focusing on the certification documents themselves.  All the other stuff, so far, is the supporting documentation for what we write about our compliance with the Principles.

So, for now, we’re re-thinking deadlines for compliance certification text - and letting the supporting documentation (kinda, sorta) take care of itself.  Will this work?  Maybe.  After all, the point of all the assessment and measurement and analysis is continuous improvement.  If we can’t at least focus on the real issue, the degree to which we are currently complying with the Principles, how can we truly analyze the assessment data and know what it means?

Just to add one more trite expression to this:  We’re now putting the cart behind the horse, instead of the other way around.

(My sincere apologies to those of you who got the point quickly.)

Executive Team Meeting scheduled for today at 9:00 a.m. Team members include Julie Allen, Metta Alsobrook, Pete Barnardin, Michael Coleman, Ben Gaddis, Diane Griffith, Richard Huckaba, Simon Kane, Serenity King, Abby Kratz, Iris Kwong, Felicity Lenes, Justin Mausel, Robert Nelsen, Larry Redlinger, JoyLynn Reed, John Sibert, Andrea Stigon.

Whew.

Today’s Agenda:

1. The Death of Friday Meetings (there will be no tears)
2. Progress Reports
3. Core Curriculum Course Assessment Tool Launch
4. Assessment of Lab Sections
5. Deadlines for Compliance Principles
6. Self Assessment

Another busy, busy week, and even the QEP guy (we’re not kicking off yet, so keep that under your hat) has been busy speaking with another group or two (and one may have even been at 10 p.m.).  Everyone seems to be cranking at full speed, and no one seems to think we’re making a lot of headway right now.  It’s still the start of a new term and a new academic year, so there’s plenty to keep us all busy.

With all the fun and joy we’ve all had with syllabi, it’s somehow reassuring that we THINK we have another 300 or so still to collect for the fall term - and admittedly some of those are for one-hour workshops and the like.  Don’t get me wrong–we want them.  We’d like to have all the syllabi so students will have access to them online.  Of course, we also have another motive; it helps us to verify who’s actually teaching the classes in case the names don’t match the “official” record.  With the latest round of fun and games, Serenity, Ben and Simon have further tweaked the process to ensure that we avoid misclassifying syllabi as missing when they are, in fact, not missing but perhaps sitting in a queue awaiting action.  [We still look at each syllabus before we post one.  We're trying to make sure we can find the basics like student learning objectives, a grading matrix or scale, and ... well, you get the idea.]

On the WeaveOnline front, Robert has again done the big demo for another group, and this time he only scared one person into considering early retirement.  Okay, an exaggeration, of sorts.  It can be daunting to look at WeaveOnline and imagine yourself working with it until you actually get into it and start playing with it.  The learning curve just isn’t that steep.  Also, Tarleton State is now considering WeaveOnline - in case that matters.  The point:  We’re not the only people using this product.  We may not be using it for every single assessment we run, but we’re using it a lot.

JoyLynn, Simon, and Robert met with the Core Curriculum Committee this week.  It was a good meeting — Simon rolled out the new core curriculum assessment tool for the web - and it met with praise and, thus, success.  A couple of tweaks and it’s ready to roll…at least in beta version.  We still have some changes to make, especially in the reporting segment, but it’s basically ready to roll for the first phase of the assessments for fall core curriculum classes (and could I make this sentence any longer?).  Also, Dr. Buhrmester prepared a list of the actual analysis tasks that will need to occur once all the data files are collected.  Very cool.

On the “so much more” side, Pete is still working on reviewing academic program assessments.  We continue to find things that seem to pop up out of nowhere - things that make us go “hmmmm.”  For example, if a certificate program is semi-kinda-not-official (because it’s under some magic number of hours), do we need to track all the awarded certificates?  Uh, probably.  Uh, wouldn’t it be nice if that information was captured in the student system?  Uh, well, uh, ….

Beyond that, Abby, Diane, and Justin are still working on faculty credentialing and making headway.  One of the changes resulting from their labors has been a new process for credentialing new hires at the point of an accepted offer.  Robert reviews the available credentials and seeks additional documentation when necessary.

Yes, it’s been another busy week.  And this doesn’t even include any reference to the development of the style guide - except to say we’re almost done.  (Way to go, Serenity.)

Sometimes I find it difficult to conceptualize just how we’re supposed to “take a snapshot” of the university and its “effectiveness” when all the measures, objectives, and even players keep changing, sometimes as often as the wind changes direction.  Trying just to identify the programs to be assessed can prove daunting–we add a new PhD program, we drop a Master’s program, and we re-name another program altogether–”ya can’t tell the players without a score card” but the score card is written in pencil.

Yes, I know.  That’s university life.  So it is.  Things change, constantly.  Even so, it seems we need to build a better system for monitoring the “big” changes as well as the day-to-day stuff.  We all tend to know when someone leaves the university, but we don’t always know when some shifts responsibility within a department.  We generally know when a new program is approved (after all, it’s taken a remarkably long time to get the thing approved), but we don’t always know when someone decides to stop admitting students into a major.  If there are no new majors, is that a sign the program is being phased out or that someone is just rethinking the program’s goals and processes?

Perhaps not only the SACS Team needs a better communication strategy.  Maybe the university (as well as perhaps even the system) needs a better communication strategy.  Maybe that’s one of the goals we need to bring forward for the community’s response–to help people get the information they need in a more timely, reliable, and public manner.

Okay, that’s about two cents’ worth.

Despite a holiday on Monday, it seems the week was still an over-40-hour week with plenty going on. On the tech side, Simon and JoyLynn were finalizing the new core assessment tool for online entry; they meet with the Core Curriculm committee on 13 September to “finalize” the look and feel. Serenity continues to press forward on a style guide and will meet next week with Robert, Julie and Simon to put the finishing touches on a first draft. Metta continues to track the progress overall of each team’s efforts and identified some looming deadlines for three different “principles.” Pete continued to work on program assessment, working with a couple of schools in particular to keep moving forward. Diane reported on Wednesday that the faculty credentialing is wrapping up the spring 06 term. Ben continues to work on tagging dox and identifying concurrently offered courses that should be coded as cross-listed. Mike Coleman is working to coordinate an assessment response from the Undergraduate Education component, including intellectual competitions, undergraduate advising, and a host of other related pieces. And so on.

Perhaps a highlight of the week, at least for me, was John Sibert’s development of a more evolutionary view of the QEP and its emerging process. John has spent considerable time meeting with various constituent groups, and each group seems to bring forward its own unique take on what issues are of most concern regarding student achievement at UTD. As a result, John finds he has a notebook crammed with ideas that have begun to meld into sets or potential themes. Despite this development, he continues to seek more input and look for more avenues for people to contribute. He and Simon met on Friday to discuss a revamp of the QEP web site so that people can more easily offer their ideas about what is good and bad about UTD’s educational offerings and about any and all random issues people think should be addressed, reviewed, or at least considered.

What this week points out most clearly to me is that we still need to identify, quantify, build (pick a verb) a communication plan that is all encompassing–something that includes the SACS compliance component as well as the QEP, its roll-out, its purpose, its process, etc. So that now goes on my list as well, along with continuing to “clean up” course titles for another 1,500 or so courses, drafting more of the assessment pieces for the Provost’s Office, and working with Simon to finalize an online evaluation system for faculty evaluation of academic administrators (not entirely a SACs issues, but it has some potential overlap down the road).

All that on the table, there’s a small celebration this week-end as the executive team gathers to unwind and take a breath. It’s been a hectic year already, and we’re walking down the tunnel toward a bright light…no, that’s the near death thing. Okay, we’re looking forward in time and seeing a light at the end of a tunnel (okay, maybe that’s a bit better); the catch is, the light is still kinda small. It’ll be nice when it’s a bright and blinding light and it’s a signal that we’re reaching the end of the journey.

Uh, maybe that’s not quite right either. I don’t think this journey ever really ends. Only the first part of the journey will end; the processes that we are building will continue long past the “compliance audit.” In some ways, we really are building a new culture of assessment or a new culture of evidence on the campus. It just seems it’s a building that has a lot more foundation than walls so far.

In an academic setting, one comes to expect that certain things are “given.”  One such item, at least in my head, is that there is a standard style guide that sort of exists within everyone’s head - by default.  Academic writing, as dry as it can be at times, at least has certain common features:

1. It’s direct
2. It’s concise (not like my blog entries)
3. It’s relatively formal in tone (again, not like my blog .. you get the idea)
4. It’s generally active voice
5. It’s generally 3rd person

BUT…. Writing the standards for a compliance certification project presents its own set of challenges to those characteristics.  Okay, it should be direct, concise, yada yada yada.  But then the specific questions arise.

1. Can we use charts within the compliance certification report, or are charts restricted to supporting documents?
2. Okay, if we use charts, are they pie charts, bar charts, time-series charts…?
3. Will tables be in the basic compliance report?
4. If any of the above come into play, will the compliance certificate be written in html for the web and thus formatted by the web’s presence - or will the compliance cert be written in Word (or whatever) and then turned into an Adobe pdf and presented as a pdf?
5. Who will edit which parts of which documents?  Who edits for style and tone?  Who edits for content?  Who edits for …. ?

Maybe this isn’t such a big deal in the real world of hurricanes, terror threats, nuclear negotiations, and the like.  It is, however, enough to keep some of us wondering how best to resolve some of these questions so that we manage to present a unified and coherent report for our accreditation evaluators.

The good news is that we’ve delegated much of the decision making to Serenity and Simon.  They will propose a final “template” after the Labor Day Week-end, and a small group will review that proposal in light of the group members’ own experiences with previous SACS (and other) accreditation reviews.  Bottom Line: We should have a style guide in final form within a week for distribution to the committees.