Grade the Bot! 

Dave Burgess (Twitter: @dbc_inc) and his team have a great idea for Educators to leverage ChatGPT in the classroom. Grade the bot! 

The Good

ChatGPT has successfully answered many of my semi-technical questions. YouTube has many videos of particularly challenging questions, and the ability to drill-down into ChatGPT answers. The grammar, syntax and readability of the responses have been very satisfactory (a little editing is to be expected). If this project gains access to larger, more current data, the results will be amazing. 

The Bad

The exiting (limited) dataset can be exposed by asking questions about current topics. Some topics associated with Social Security stumped ChatGPT. Responses to a tricky question "I'm 10 years older than my wife. What are suggestions for maximizing my social security check while protecting her income?" came back with a very generic answer and a suggestion to "check with a social security expert". When the question was reworded, the answers were very much the same. Similar "weak" responses for COVID related questions were noted. 

The Ugly 

The inability to understand Government programs like social security - at least for this version of ChatGPT, is an interesting blind spot. While many have pointed out that the data sources are unknown, the current narrative is that ChatGPT will skew liberal to reflect existing online content. 

The addition of moderators will introduce a new collection of bias. Moderators will have the challenge of being fair, without polarizing the question response sets. One person's conspiracy theory is another person's truth (and vice-versa). 

The Bottom Line

A fantastic classroom assignment would have the class challenge ChatGPT with a narrow set of questions about a specific topic, have the students collect the responses, then critique the responses, providing evidence for their positions. The range of observations may prove exciting!

If students select the topic, examine the results, and apply critical thinking to evaluate the response, the lesson will go a long way towards understanding the use, limits, and pitfalls of AI chat solutions. 

Tot1

(No ChatGPT content was used to generate this post). 

(Weird disclaimer, right?)