A Tale of Two Assessments: Part One

In my last blog I compared assessment and grading in an attempt to demystify a process many of us feel is just too bureaucratic. The one response I received made a lovely analogy : "If grading is the weather, then assessment is the climate," which reiterates my point that the difference between grading and assessment is largely one of scale. This week, with apologies to Charles Dickens, I would like to present the first of two examples of assessment at Kenyon that grow out of a need or desire to know something. In keeping with my philosophy of reaccreditation, I believe that we must take ownership of assessment, asking ourselves, "what's in it for us?" What do we want to know about our students' performance? When I am teaching I always tell my students that the only questions worth answering are the ones that they have a pressing desire to answer. Their research should grow out of a need to find out the answer to some crucial question. What would happen if we thought of assessment as a form of research in which we pose questions to which we really (really) want answers?

To test out my question, I thought I would go to the sports analogy. I interviewed Kenyon swim coach Jim Steen. Steen is one of the most successful coaches in college athletics with some some 51 national titles to his name: his men's swim team has won 29 consecutive national titles and his women's program has claimed 22 of the past 25 titles. By any measure this is a successful program. So how do you get to be this successful? I talked to Coach Steen about the ways in which he uses data to improve his teams' performances.

I asked the following questions:

1. How do you use data in your coaching? How do you assess your swimmers’ performances to help them improve?
2. How do you use the information you collect to recruit new swimmers?
3. Why do you do it this way?
4. When did you start?
5 How did you learn to do it?

No outside agency has told the swim team that they must assess swimmers' performances. Coach Steen collects data because he passionate about human performance and curious about the intricacies of truly superior performances. He says: "Success in this program has come from measuring as precisely as we can in as many areas as we can how people are performing and comparing their performances." The data he collects (both from his own swimmers and from other world class swimmers) help him to maximize training, making the work the swimmers do both more efficient and more effective. Notice in the examples below that he is not evaluating his workouts (inputs); he is evaluating the components of his swimmers' performances (not just their final time) and then adapting his training methods to achieve the desired performance. To be fair, most of his measures are fairly objective and the final outcome is also. Those of us assessing student learning may not have the luxury of such a cut and dried outcome. But I still think we can learn from Coach Steen's philosophy of assessment.

To get a fuller understanding of the ways in which Coach Steen puts the information he collects to work, we looked at three examples of data that he and his coaching staff regularly use.

1) Every year, each swimmer gets a lactate profile. This measurement gives the coaches some idea of the swimmer’s anaerobic and aerobic capacities. These can be excellent predictors of a swimmer's likely success in sprint and distant events. In addition, they are used, for instance, in designing workouts.

2) Stroke tempo rates help the coaches and swimmers target training more precisely. It tells them if a particular swimmer need to improve by adding tempo (turning over more quickly) or distance per stroke. These changes would involve different kinds of training to achieve. For instance, a swimmer who needed to increase distance per stroke might focus more on strength training.

3) A comparison of splits (intermediate times in a race, say, the first 50 yds of a 100 yd race) and drop off rates (the time difference between the second half of the race and the first), as well as the ratio between the swimmer's best 50 and the first 50 yds of a 100 yd race (for example) are another set of benchmarks he uses. This is a good measure of how tired swimmers will get at the end of a race (or the lactic buildup in their muscles). Swimmers’ training can be adjusted so they can work on component parts of a race. Swimmers’ know their ideal zone for splits and can work toward that zone.

These measures enable both swimmers and coaches to do more than just hope that swimmers will swim faster. It enables them to understand which components (body type, metabolism, strength, pace) contribute what to the swimmer's performance and to work with them separately and together.

Coach Steen is able to use the data he collects to talk to recruits. He can tell them fairly precisely the impact the program will likely have on their performance over their four years. He has collected information over many years about where swimmers were out of high school and where they were after 4 years at Kenyon. He looks, he says,at the good, bad, and ugly, collecting 33 years of average improvements. What he offers recruits is a plan for their college swimming career, an idea of the possibilities.

When I asked him how he began doing this kind of assessment, he said he has always done it as long as he has been coaching (since he was 14 he told me). He learned it by beginning with simple rudimentary measurements and over the years has refined them by working with them. His research (and it is research) has led to more complex and more precise measurements. He admits: "Can you assess too much? Can you have too much data? Absolutely. But it’s better than knowing nothing."

"It raises their game," he says.

And for those of you following my exercise travails you will be happy to know that I have purchased a bookweight from Levingers so that I will be able to read novels rather than assessment manuals on the elliptical.

Stayed tuned for A Tale of Two Assessments: Part Deux.