My high school chemistry teacher was known for his catchphrases. The one that I remember most is (somewhat ironically) “Memorizers will fail!”
A paper from ICER 2020, “Syntax Exercises in CS1” by John Edwards, Joseph Ditton, Dragan Trninic, Hillary Swanson, Shelsey Sullivan, and Chad Mano, challenges this idea. Most instructors want students to learn how to use some tool to solve some problem. However, when students must simultaneously learn the syntax of that tool, this creates extraneous load on their working memory, which compromises their ability to learn how to solve the problem.
The authors suggest that one way to mitigate this problem is for students to first become fluent in syntax through exercises, then engage in problem solving:
The present intervention draws on this fundamental idea of reducing students’ unnecessary cognitive burden by automating their production of code syntax, thereby enhancing their learning. We accomplish this by providing students with exercises that develop their fluency with syntax, preparing them for future learning from instruction.
(This paper was published years before the proliferation of generative AI tools, which perhaps reduces the benefits of learning syntax. But I suspect that even if humans manually write a lot less code than they used to, learning syntax separately from problem solving could still be beneficial. If you want to become a great chef, it probably helps to chop a lot of vegetables.)
Their study took place during weeks 3-7 of a CS1 course (in Python) across two consecutive semesters. The control group refers to students in the first semester, while the test group refers to those in the second. In each of those five weeks, there were three lectures and a programming project (Projects 4-8) due each Friday. Before each lecture, students in the test group were required to complete 20-60 syntax exercises on an online platform. There was an auto-graded midterm exam at the end of week 5 (i.e., the third week of the study).1
The syntax exercises were designed to take 20-40 seconds each, and the set was designed to take at most 10 minutes. (On average, students spent 8 minutes on each set, for a total of 24 minutes per week.) The exercises were quite straightforward: they increased in complexity “but always in such a way that the student needn’t design code, nor even think very long about what they need to do.” The exercises did not teach semantics or applications; those discussions happened in the following lecture.
The authors report three outcomes: attrition, project and exam scores, and number of keystrokes on the projects.
1: Attrition
Of the students who attempted Project 4, 94.1% of control students attempted Project 8, while 97.2% of test students attempted Project 8, where “attempting” a project is defined as executing at least one event (e.g., keystroke, paste) tracked by the course’s IDE. (Exercises and projects were all submitted through the course’s IDE, but pasting was disabled for the exercises.)
2: Project and exam scores
Surprisingly, project scores were (slightly) lower in the test group than in the control group, but the average on Exam 1 was much higher (78.9 vs. 71.9). The authors acknowledge that this might be explained by the fact that “the second semester of the study used an identical exam to the first semester for comparison purposes, whereas the exam was changed each semester prior to the study.” However, on Exam 2, the difference was smaller (63.4 vs. 61.6), which somewhat undermines this explanation, as the authors note.
3: Number of keystrokes per project
Surprisingly, again, students in the test group used more keystrokes on average than students in the control group. In Project 8, the median number of keystrokes in the control group was 3896, while the corresponding number in the test group was 7207. However, “the control group had many more projects with large amounts of code pasted into the editor than the test group.” But even after excluding projects with large amounts of pasted code, the authors still found that students in the test group used more keystrokes.
One explanation the authors suggest is that the test group students are more likely to “tinker more” or “type everything when refactoring” (instead of cutting and pasting) because they’re more accustomed to typing (due to the syntax exercises). Another explanation is that the higher attrition rate in the test group might be attributed to students who normally would have dropped out, and “those are the very students that we would expect to require more time to complete their projects.”
The authors also analyzed typing speed (i.e., time elapsed between two keystrokes) and found that the test group spent less time on code typing.
Discussion
In Project 8, the percentage of control students with a “pasted” project (i.e., the number of pasted characters exceeded the total number of keystrokes by a factor of four) was 24%, and the corresponding percentage for test students was 11%.
The authors give two possible explanations. The first is that doing the syntax exercises made the test students feel more familiar with the course’s IDE, so they were less likely to work on their projects in a different IDE and paste into the course’s. The other explanation is a drop in plagiarism, which could explain the decrease in project scores and the increase in exam scores. In fact, the authors note that this could explain all of their results:
The hypothesis that plagiarism is reduced with syntax exercises is consistent with every expected and unexpected result from our study: syntax exercises resulting in lower project scores, higher test scores, lower digraph latencies, and more keystrokes on projects, on average.
Exam scores might have increased due to a decrease in cognitive load from writing syntax. But the authors provide another possible explanation: one of the authors, Dragan Trninic, wrote in another paper that “repetitive exercises, sometimes derided as drills, can nonetheless be a fountainhead of insight and discovery.” In other words, according to the authors of this paper, “the very act of repetitive practice, when exercises are well-designed, provides opportunities for reflection and deeper learning.”2
In conclusion, the students who did syntax exercises had higher exam scores and lower attrition rates. Even though their project scores were lower, and they used a higher number of keystrokes, those results can be explained by a reduction in plagiarism and improved outcomes for students who would have dropped the course. On average, the exercises only took 24 minutes per week and don’t require other changes to the course, so I think they’re worth considering. More generally, I suspect that many skills that look like one activity (e.g., programming) can be separated into various components, and it probably helps to sometimes consider them separately.
As we’ll see, the project and exam were identical for the control and test groups. Project graders were different across semesters, and five of the six sections (three per semester) were taught by the same instructor.
I’m no music expert, but this reminds me of scales and etudes. The physical mechanics are important, but I suspect that the mental processes that occur during automated practicing are just as important. It also reminds me of practicing arithmetic (e.g., long division/multiplication) without really understanding why the process works.