Off the Mark (review)
by Jack Schneider and Ethan Hutt
In my last post, I wrote about a paper by Jack Schneider and Ethan Hutt. Since then, I’ve read their book Off the Mark, and I enjoyed it.1 According to them, reforming grades is tricky because people expect grades to serve three purposes:
Communication: In the short term, grades provide a succinct way for teachers to communicate to students (and potentially their parents) about their progress. In the long term, colleges, graduate schools, and employers rely on grades to make admissions and hiring decisions.
Motivation: Learning isn’t easy; as the book puts it, “Learning is an active process that involves invisible work and considerable effort.” Thus, grades “have been used as a way of getting students to take school seriously.”
Synchronization: As a whole, the education system has various moving parts: students change schools, transfer to other districts, apply to colleges around the world, etc. Grades help tie the system together by providing schools with a means of synchronization. For example, transfer students typically receive credit for a course if they took the equivalent at their previous school, even if the two offerings aren’t perfectly equivalent.
The book proposes some improvements to the standard grading practices that, it argues, simultaneously preserve the purposes listed above while addressing negative aspects of the status quo (e.g., cheating, teaching to the test, avoiding challenges). I’ll describe some of the proposals below along with my thoughts on how I’ve considered implementing them in an undergraduate CS course.
1: Provide grade-less feedback
Teachers should provide narratives, or “thick descriptions of student abilities,” in addition to regular grades. Unfortunately (like many alternative grading methods), this sounds nice but seems difficult to scale to hundreds of students, since giving good feedback takes time and experience. It’s also quite demoralizing when teachers realize that many students ignore it. Of course these days, there are AI tutors, but I’m not convinced that they can solve the core problems with grading. They’re still (depending on the context) unreliable, and personally, I’m less likely to take an AI tutor’s feedback seriously than an instructor’s. (Of course, AI has been developing quite rapidly, so it’s hard to rule out AI-based solutions completely.)
One thing I’ve done, which is relatively easy to implement, is assigning homework problems that are worth an extremely small (possibly zero) number of points. My hope is that, because these problems essentially don’t affect students’ final grades, students can use them as opportunities to test their true abilities and receive feedback. Usually, when every problem matters for their grade, students are more likely to rely on external assistance, which obfuscates the communication of their true abilities to their instructors.
2: Create digital portfolios
When someone sees a letter grade on a student’s transcript, it’s usually hard for them to tell what exactly that letter grade means. To remedy this, the book proposes that the work itself, not just a grade, should be visible to future audiences via a digital portfolio. The book argues that these “double-clickable” transcripts should include larger projects, skills, and competencies, which can be “interpreted through holistic human judgment” and minimize the negative parts of grading. Teaching to the test might be fine if the “test” is actually a series of interesting projects.
I don’t know what a digital portfolio looks like in an algorithms course, but I think it’d be motivating for students to leave the course with something substantial beyond a grade and a bunch of small, independent assignments. One idea I’ve considered is having students create a video in which they explain the solution to a problem. After all, teaching is a great way to learn, and creating an educational video could be useful career preparation. (According to a 2019 poll, “YouTuber” and “teacher” are the top two career aspirations for children in the US and UK!)
3: Foster intrinsic motivation
As I’ve written about before, external motivation (e.g., grades) can crowd out intrinsic motivation, which many argue is a negative effect of the education system. However, as the authors readily acknowledge, fostering intrinsic motivation is difficult because motivation involves many complex elements such as school culture and family support. Still, they believe schools should try to create assignments that are tied to authentic competencies and lead to longer-term goals. (The opposite would be pure busywork.) Unfortunately, it’s much easier to administer and consistently grade, say, multiple-choice quizzes than open-ended prompts. But I definitely see their point: if a student only engaged with a subject via multiple-choice quizzes, their interest in the subject would probably stagnate (or even decrease).
Instead of focusing on the exchange-value of education (i.e., do these tasks in exchange for an A), educators should focus on its use-value. By “use,” the authors don’t mean “useful for getting a job” but rather, they mean something like “useful for living a good life.” (I suspect that they think “useful for getting a job,” still sounds too transactional.) They use art as an example: for most people, drawing and painting aren’t useful for enhancing one’s résumé, but they can certainly be useful for living a good life.
Similarly, in my opinion, developing an appreciation for theoretical computer science is similar to developing an appreciation for art, jazz, physics, nature, etc. For most people, there’s no obvious quantifiable reason to appreciate any of these things. But, without getting too philosophical, I think that part of “living a good life” is cultivating a sense of curiosity and awe.
4: Use common performance tasks
Almost every school has a course called, say, “Algebra 1,” but these courses vary in content and difficulty. This especially becomes a problem when students change schools. To preserve synchronization across institutions, the authors “envision common performance tasks” that allow us to shift our focus “from the current formalisms to what students can actually do.” Even though some subjects (e.g., history?) seem harder to break into concrete skills than others (e.g., math), I think common performance tasks would be useful for every subject.
In computer science, a few major organizations (ACM/IEEE/AAAI) released curricular guidelines in 2023; here is the report for “Algorithmic Foundations.” In my opinion, their list of topics is quite long, and it’s not clear what the learning outcomes entail. For example, one of the outcomes is “Evaluate whether a greedy approach leads to an optimal solution.” This can be either extremely easy or very difficult, depending on the details of the situation! But still, for the sake of synchronization, I think it’s worth it for CS programs to at least be aware of the guidelines, if not try to follow them.
5: Make grades overwritable
The last proposal I’ll discuss sounds the most radical. The authors use the following analogy: if someone can ride a bike, it basically doesn’t matter how long it took them to learn. Similarly, they argue:
Instead of permanent marks, we can imagine assessment systems that update constantly to reflect what students know and can do today. Can the person in question ride a bike now? If so, previous struggles are suddenly no longer relevant information — they are archival noise that should be scrubbed from the data.
In computer science, there was a paper from SIGCSE 2023 that explores this very idea: “A’s for All (As Time and Interest Allow).” Both the book and the paper make a strong case for overwritable grades, but I worry about the negative consequences (many of which could be unforeseen). For example, the book notes that one potential downside is that students become even more obsessed with getting straight A’s, possibly to the detriment of their learning and mental health. Still, the idea is interesting, and maybe even useful at a smaller scale — I’ve written about one of my own experiments with alternative grading, which included reattempts without penalty.
Conclusion
Grades are deeply embedded in our education system, so if we want to improve grading practices, it’s important to know their history and the purposes they serve. Off the Mark provides a great overview in addition to multiple proposals for improvement. Some of them are pretty radical, but others are relatively easier to implement. I discussed their paper in my last post, and it ends rather pessimistically, so I’m glad that this book provides a nice sense of hope.
Disclaimer: Ethan Hutt and I both currently work at UNC-Chapel Hill.