Essay Grading Software: Trivial, Reductive, Inaccurate

Go read this article:  Essay-Grading Software Offers Professors a Break  Go on - I'll wait for you.

Now let's talk.  What's wrong with this?  Everything!  I know from experience that grading essays is by far the most time-consuming and repetitive part of a teacher's job, and often not the most rewarding.  But to accept that a machine can replace the insights of an experienced professor, or can properly evaluate the written word, demeans teachers, students and the act of writing.

Another article has this to say:  "College students pulling all-nighters to carefully craft their essays may soon be denied the dignity of having a human being actually grade their work."*  It irks me to think of countless students working so hard, only to receive a computer-generated response for their efforts, while their professors punch a button and go back to bed.  I realize that many of today's college students are not hardworking and do not put a colossal effort, or any effort really, into their writing, and perhaps they deserve an automated response.  But what about those who do labor to create something worth reading?  When I was in college, I once received an essay exam with nothing but a big, red "B" on it.  I took it in to the professor to ask what I could have done better, (and why he hadn't bothered to give me more information in the first place.)  "Oh, no one ever reads comments," he said, "so I've stopped making them."  I asked how then, I was supposed to know what to improve.  "Go talk to your T/A," he said, returning to his work.  I was astounded.  After all, I had worked hard writing the paper; why couldn't he even add a few red scribbles?  Nowadays, as teacher, I have trouble not writing too much on my students' papers.

Don't students deserve better?  College tution costs are astonomical, and I'm afraid that the next debt crisis will be the default on millions of impossibly-high student loans.  To attend an in-state public college for the 2012-13 academic year, the average overall cost (or "sticker price") for students who don't receive any financial aid is a record $22,261, according to http://money.cnn.com/2012/10/24/pf/college/public-college-tuition/index.html.  A moderate budget at a private college averages $43,289, and the most expensive private colleges cost around $60,000 per year!  Would you want your child spending for four years what amounts to more than the average annual salary, in order to be taught by a computer?

The article on this software points out that it, "[enables] students to take tests and write essays over and over and improve the quality of their answers."  It sounds to me a bit like a dart game; if a student types in random words over and over, they'll eventually hit on the right combination.  Would you want a student with that kind of training working for you?  This reminds me of the "infinite monkey theorem", which states that a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type a given text, such as the complete works of William Shakespeare (Wikipedia.)  And here I've been working so hard to teach the nuances of polished, effective writing, when I should instead have just taught faster typing.

In what way will automatically-generated responses improve a student's thought process and ability to craft a well-worded argument?  The whole notion that a computer can replace the carefully-thought-out comments of a good teacher is ludicrous.  When I grade a student's paper, I consider a number of things: What does this student need to work on most?  Often, in order not to overwhem with too much detail, I don't tell a student everything s/he needs to change; I just focus on a few important points.  How thorough an explanation will s/he need to grasp the concept and put it into use?  Sometimes I write a word or two; sometimes a paragraph or two.  I focus on both the clarity of ideas the students are trying to express, and the careful choice of words they use in sharing them.  Can a computer really be that sophisticated?

The group Professionals Against Machine Scoring Of Student Essays doesn't think so.  Their website, http://humanreaders.org/petition/ has this to say:

Computers cannot “read.” They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others. Independent and industry studies show that by its nature computerized essay rating is:

trivial, rating essays only on surface features such as word size, topic vocabulary, and essay length
reductive, handling extended prose written only at a grade-school level
inaccurate, missing much error in student writing and finding much error where it does not exist
undiagnostic, correlating hardly at all with subsequent writing performance
unfair, discriminating against minority groups and second-language writers
secretive, with testing companies blocking independent research into their products

If a professor is seeking a mere rehearsal of facts, then software is obviously a very effective and appropriate tool for grading.  But if that's the case, why assign an essay?  Why not just rely on true/false or multiple choice questions?  A true/false test, after all, is a binary operation - perfect for a computer, and very little thought required from professor, student or machine.  However, if a professor is going to assign an essay, then surely something more is involved - the crafting of the writer's thoughts into something clear, logical, persuasive, and even elegant.  A good (or bad, for that matter) piece of writing is nothing less than a window into the mind of the writer.  Shouldn't it then be treated with the respect a human mind deserves?

To reduce the assessment of a piece of writing, even such nonsense as many college students no doubt produce, to a mere software formula, utterly debases the act of writing.  How can a machine evaluate logic, organization, fluidity, and clarity, let alone syntax, vocabulary, imagery and the grace of beautiful language? How can a computer consider the sheer magnificence of a piece of writing like Patrick Henry's "Give Me Liberty" speech, or Martin Luther King, Jr.'s "I Have a Dream"?  In the middle school English class I teach, we've just been reading famous speeches and discussing what makes them so beautifully persuasive.  Consider this from Patrick Henry:

Mr. President, it is natural to man to indulge in the illusions of hope. We are apt to shut our eyes against a painful truth, and listen to the song of that siren till she transforms us into beasts. Is this the part of wise men, engaged in a great and arduous struggle for liberty? Are we disposed to be of the numbers of those who, having eyes, see not, and, having ears, hear not, the things which so nearly concern their temporal salvation? For my part, whatever anguish of spirit it may cost, I am willing to know the whole truth, to know the worst, and to provide for it.

Here, Henry refers to both Greek mythology ("the song of the siren"), and the the Biblical words of Jesus ("Having eyes, see ye not?" Mark 8:18.)  Can a computer spot literary references?  Likely not.  Does it matter?  It should, to every literate person who hopes to take part in The Great Conversation.

Henry further asks a series of questions to get his listeners thinking:

Are fleets and armies necessary to a work of love and reconciliation? Have we shown ourselves so unwilling to be reconciled that force must be called in to win back our love? Let us not deceive ourselves, sir. These are the implements of war and subjugation; the last arguments to which kings resort. I ask gentlemen, sir, what means this martial array, if its purpose be not to force us to submission? Can gentlement assign any other possible motive for it? Has Great Britain any enemy, in this quarter of the world, to call for all this accumulation of navies and armies?

Good point, Sir.  The great orator also invokes some powerful imagery to drive home his argument:

They are sent over to bind and rivet upon us those chains which the British ministry have been so long forging...If we wish to be free--if we mean to preserve inviolate those inestimable privileges for which we have been so long contending--if we mean not basely to abandon the noble struggle in which we have been so long engaged, and which we have pledged ourselves never to abandon until the glorious object of our contest shall be obtained--we must fight!...There is no retreat but in submission and slavery! Our chains are forged! Their clanking may be heard on the plains of Boston!

He exhorts, he pleads, he contrasts a life of slavery with the boldness and courage required to prevent it, and he drives home his argument with some of the most compelling rhetoric ever written.

Martin Luther King, Jr. has a less forceful style, full of the imagery of hope and decency.  King uses repetition, um, repeatedly.  Would a computer rate that favorably?  It should, in this case, though perhaps not in many others.

Five score years ago, a great American, in whose symbolic shadow we stand signed the Emancipation Proclamation...But one hundred years later, we must face the tragic fact that the Negro is still not free. One hundred years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination. One hundred years later, the Negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity. One hundred years later, the Negro is still languishing in the corners of American society and finds himself an exile in his own land. So we have come here today to dramatize an appalling condition.

How's this for a persuasive analogy:

In a sense we have come to our nation's capital to cash a check. When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all men would be guaranteed the inalienable rights of life, liberty, and the pursuit of happiness.   It is obvious today that America has defaulted on this promissory note insofar as her citizens of color are concerned. Instead of honoring this sacred obligation, America has given the Negro people a bad check which has come back marked "insufficient funds."

And could even the most sophisticated piece of electronic equipment ever appreciate the sheer beauty of this imagery?:

This will be the day when all of God's children will be able to sing with a new meaning, "My country, 'tis of thee, sweet land of liberty, of thee I sing. Land where my fathers died, land of the pilgrim's pride, from every mountainside, let freedom ring."
And if America is to be a great nation this must become true. So let freedom ring from the prodigious hilltops of New Hampshire. Let freedom ring from the mighty mountains of New York. Let freedom ring from the heightening Alleghenies of Pennsylvania!
Let freedom ring from the snowcapped Rockies of Colorado!
Let freedom ring from the curvaceous slopes of California!
But not only that; let freedom ring from Stone Mountain of Georgia!
Let freedom ring from Lookout Mountain of Tennessee!
Let freedom ring from every hill and molehill of Mississippi. From every mountainside, let freedom ring.

I rest my case.

*Read more: http://www.dailymail.co.uk/news/article-2304317/Outrage-software-automatically-grades-college-essays-spare-professors-having-assess-students-work.html#ixzz2Pt2cbian

What's Up Doc?

Another movie night with the usual suspects.  This time over to Beth's for a delightful screwball comedy, "What's Up Doc?" starring Ryan O'Neal and Barbra Streisand, and a hilarious Madeline Kahn - a really funny move around a plot of switched luggage, with every imaginable chase scene cliche, all really well done.  The kids laughed through the whole movie, but we noticed that as soon as the main characters started kissing, the kids all suddenly started crackling their popcorn bags - then the parents started cracking up!