One of the most closely-watched conundrums of online education is how best to approximate traditional on-premise teaching methods. Grading multiple choice and short answer exam questions is straightforward, but how do you approach grading student submitted essays in a way to allow the online platform to scale to handle tens of thousands of students? Enter machine learning.
The edX MOOC platform currently uses machine learning for this purpose but is working on ways to improve its automatic essay scoring system. An interesting article appeared in the June 7, 2013 issue of The Tech, MIT’s campus newspaper – “EdX releases open source code, demographic data” that describes this effort. MIT is a founding educational member of edX.
Machine learning is used to look for specific features in essays, such as the use of transition words, that can be used to help determine a grade. But not everyone is happy with the current state-of-the-art for automatic grading. On Hacker News, a social news site popular within the technology community, LightSIDE Labs founder Eljiah Mayfield expressed disapproval upon seeing the code edX uses for automatic grading (the code for edX was released as open source). Like edX, LightSIDE Labs has developed a tool which uses machine learning to assess written text.
Looking through this codebase, it’s roughly the level of machine learning work that I’d expect from a first-year graduate student,” Mayfield posted. “It gets the job done but is really only tailored to the very narrow task of essay grading and doesn’t offer much extensibility. My guess is that for that narrow task, it does a pretty good job, but in my experience teachers are rarely satisfied with just getting a machine learning-generated score and nothing else.”
In contrast Anant Agarwal, President of edX, told The New York Times in April:
This is machine learning and there is a long way to go, but it’s good enough and the upside is huge. We found that the quality of the grading is similar to the variation you find from instructor to instructor.”
For the long-term viability of MOOC platforms, the grading problem must attract a robust solution and machine learning seems to be the most promising path.