Stephen George-Williams

ACSME Panel: What is the future of assessment?

George Williams

Dr Stephen George-Williams

The University of Sydney

2020. Wow, what a year! Not that 2021 has exactly been a cakewalk, but the insane number of constant changes last year really did make for an intense experience for everyone, let alone for anyone in the teaching and learning space!

When the COVID-19 pandemic first hit, I was acting as the laboratory director for all of our first-year chemistry units at the University of Sydney. As such, I was given the ‘wonderful’ challenge of creating a range of online experiences that were meant to somehow replace actually being in a teaching laboratory. You may think that this is impossible … and, well, you’d be mostly right! While some skills, such as data analysis and experimental design, can be taught and even assessed, there is no true replacement for getting into a laboratory and working with instruments and scientific glassware.

It was highly fortuitous then that I had already organised, one week before our spending was frozen, to purchase access to range of technique-focused simulations for use in the teaching of practical skills. Coupled with a range of recorded videos of in-house experiments, we could have detailed conversations with students about technical skills, even when online. Upon returning to our actual teaching laboratories (albeit socially distanced) the students noted that the simulations made them fill more confident and less anxious. Assessment was only a range of quizzes at first, but competency based in-person observations came to the fore in our face-to-face activities in late 2020, which helped reinforce the technical skills the students were developing. But what about those students enrolled remotely?

To combat that thorny issue, we turned to student-recorded videos. The students were given an experimental layout and asked to describe how they would physically set-up a range of equipment and instrumentation in order to meet the research questions raised. Students then recorded their plans in a PowerPoint presentation, uploading the final video for assessment. The resulting conversations with students about why certain techniques were done were astounding, and clearly the outcome of a more meaningful assessment focused more directly on technical skills. We liked it so much, that it was implemented for all students in 2021 (although, now under the leadership of the new laboratory director, Dr Shane Wilkinson).

So, I hear probably no one asking, what was I was doing if not ruling over the teaching laboratories with an iron fist? Well, for my sins, I was now the Director (read: unit coordinator) of ALL first-year chemistry units. This meant that my focus very much shifted from laboratory assessment to theoretical assessment of content knowledge (which the previous assessments tended to focus on). Of all the assessments that existed previously, none has driven me more insane that the final theory exams!

In 2020, we did not utilise proctoring software. What happened? Academic misconduct cases rose astronomically (as did grades in some units!) and we struggled to make ‘google-proof’ exams with no upload options made available. In 2021, we chose (read: were forced) to use proctored exams. What happened? Academic misconduct cases dropped but cases of technological issues and internet connectivity woes became a much larger issue. We also still struggled to make non-googleable questions and were fast running out of ways to check on basic knowledge. How many different ways can I ask about something that is fact?!

At this stage, I’m not going to pretend I have the answer to the exam issue. Some of my colleagues would see exams taken out the back, shot, and perhaps set on fire. While I agree in theory, I do worry about how to ensure students engage with some level of content. Exams are in no way great, but what would the actual outcome be from their complete removal? If we do remove them, what do we replace them with? Authentic assessment is very much du jour (and I like it!) but the debate on how best to implement it is alive and well. But hey, that’s why we have multiple panels and three days to debate this at ACSME 😊