A firestorm broke out when a study released by Hewlett Foundation suggested that automated scoring systems can produce scores similar (have a high correlation) with those scored by us human folk.
Based on the reactions posted on the #ncte and the #engchat feeds you would have thought armageddon was upon us and Pearson merged with the Cyberdyne Systems Corporation.
It is affirmative. We do not need to fear the robots. In fact they can be our friends (I will not go into the methodology and limitations of automated scoring systems...mainly because I cannot do a better job that Justin Reich did in his three part treatsie).
Basically the cries over the rise of robots was misguided. It seemed to fall in two strands. The first was they cannot recognize good literature. No one is asking the robots to do this. Basically they are being asked to identify textual elements n patterns that replicate what their human trainers would do.
The second big fear was that the scoring systems could be gamed. Students could use long sentences and big words but write gibberish. This does not concern me in the least. If you show me a student who is creative enough, and has the ability to say nothing while stringing together a massive vocabulary and complex sentences--well you are showing me a very talented writer.
Overall, automated systems will improve HST testings as it can include the assessment of more complex and open ended questions. However you feel about HST moving away from bubbles has to be a good thing? Right?
High stakes tests and accountability do not get at the practices used by good writers nor does it enage stduents in connected learning. I think the robots, however, can also help on this front.
Assessing the Stream
I recently had the pleasure of setting on in on #ConnectedLearning Google+hangout panel with Paul Oh, of the National Writing Project, Ellen Middaugh, Associate Director of the Civic Engagement Research Group at Mills College, and Howard Reinghold. We were commenting on the work of Anetero Garcia is doing wonderful things around agency and active involvement.
A question from the audience came up asking how do we bring in principles of connected and participatory learning in classrooms so focused on student achievement . While these two outcome do seem dichotomously opposed they do not have to be. And the robots can help.
The digitization of literacy creates a lot of data. Achievement folks love data. They salivate for it. Teachers can use this as a hook to demonstrate that participatory learning can lead to gains when you assess what Dan Hickey calls the residue of learning.
Basically, as Justing Reich pointed out in his third post, automated scoring systems can provide wonderful formative assessment data. This also involves assessing the growth over time and looking for gains more in the process and social practices of writing rather than a final product.
Imagine if an automated scoring system could look at drafts of an essay and analyze the amount of sourced material (already possible). You could take this further and what if blogs could be analyzed for their use of having a clear main idea, media, and supporting evidence. The analyzing the stream would allow you to look at discurse patterns in online discussion.
All of this can be used to inform your practice-the essence of formative assessment. The robots just make it quicker-the challenge of most formative assessments.
Replacing the Teacher
Does this mean the teacher isn't necessary? Of course not. No one said this. All the humans are not gone. You will still conference with writers and set individual goals. That is the heart of what it means to work with young writers. The robots, not even a T-800, would could possibly complete such a feat.
The robots, when trained, can just find elements in a text that we want students to use. I do not think this is a bad thing.