-
In preparation to write a #GenerativeAI policy for my classes, I experimented with #ChatGPT. Yes, its texts are often wrong and unreliable. But more important, even its best answers lead students on a path that is the opposite of research and reflection. 🧵Permalink On twitter.com
♻️ 10 Retweets
❤️ 21 Favorites
Mood +4 🙂
-
My approach: I’ve kept an open ChatGPT session this summer, prodded at its limitations, explored how it remixed and regurgitated material I’ve written, took a "prompt engineering" training, and gave it my writing assignment prompts. My full take: woborders.blog/2023/08/23/why-im-banning-ai/Permalink On twitter.com
❤️ 4 Favorites
Mood 0
-
Confidently stated falsehoods are a known flaw of Large Language Models, and it was pretty easy to generate some whoppers. x.com/CarwilBJ/status/1617940714176020480On twitter.com
❤️ 1 Favorite
Mood +2 🙂
-
Some of my teaching involves questions about "a debate" that is actually settled by facts. #ChatGPT was willing to write very similar essays on such debates repeatedly describing them as "ongoing and complex."Permalink On twitter.com
❤️ 2 Favorites
Mood 0
-
Among these "ongoing and complex" debates, per ChatGPT, were questions like "whether gravity functions in the galactic core", "whether covalent bonds function in the cell's nucleus", "whether police search warrants extend to the inside of houses," "whether ebony is really wood."Permalink On twitter.com
❤️ 2 Favorites
Mood +3 🙂
-
Side note: I found a limit to this pattern, finally, with witches.Permalink On twitter.com
❤️ 3 Favorites
Mood 0
-
I was pretty sure ChatGPT would reuse tired tropes about disappearing Indians to frame issues for my Indigenous Rights class. Indeed, writing in the past tense or false "only remaining" statements sometimes came upPermalink On twitter.com
❤️ 1 Favorite
Mood -4 🙁
-
For example: "The only remaining subgroups of the Waorani are the Tagaeri and the Taromenane." (despite using with the Wikipedia plug-in, which says otherwise) "The Yamana people, also known as the Yaghan, were an indigenous group" (1,600 members, not in decline)Permalink On twitter.com
❤️ 1 Favorite
Mood 0
-
But counter-takes were also built into the corpus, resulting in an emphatic rejoinder to my request about "the last Maori": "Contrary to certain misconceptions… They are not an extinct culture or people, nor are they on the brink of disappearance."Permalink On twitter.com
❤️ 2 Favorites
Mood +1 🙂
-
But it isn't the baseless and inaccurate statements that concern me most as a teacher, it's plausible and emphatic responses to reflection questions and the ungrounded texts generated by requests for research.Permalink On twitter.com
❤️ 2 Favorites
Mood 0
-
I prompted ChatGPT 4 to produce a number of short essays in response to reflection prompts I give my students. And I used prompt engineering techniques I learned on Coursera to produce better answers. But even the best answers short-circuit actual reflection.Permalink On twitter.com
♻️ 1 Retweets
❤️ 1 Favorite
Mood +5 🙂
-
Take this one… The student hasn’t necessarily “explored” at all and neither they nor the Large Language Model have "shifted [their] perspective on history." Every time a student uses LLM output with the word “I” on a reflection assignment, they’re telling a lie.Permalink On twitter.com
❤️ 1 Favorite
Mood 0
-
Let's look at some of the best text I was able to get ChatGPT to generate in a response essay… As I'll explain this kind of writing that always frustrates me the most as a grader.Permalink On twitter.com
❤️ 1 Favorite
Mood +3 🙂
-
The best output I extracted from ChatGPT tended to be ungrounded in specifics; long on conclusions and short on evidence. It’s also full of zombie nouns and returns again and again to the abstractions raised in the prompt.Permalink On twitter.com
♻️ 1 Retweets
❤️ 1 Favorite
Mood +3 🙂
-
Because LLMs necessarily link the abstract concepts in the prompt to abstract concepts in the answer, the tendency is to either avoid examples altogether, or use trite and familiar examples (and not those in the text at hand).Permalink On twitter.com
❤️ 1 Favorite
Mood -1 🙁
-
By telling but never showing, it fails to force the reader to encounter particular people, or to demonstrate deep engagement with a historical tragedy. Adequate though not excellent as a conclusion, it’s exactly the wrong place for a student to start their writing process.Permalink On twitter.com
♻️ 1 Retweets
❤️ 2 Favorites
Mood -8 🙁
-
(These are the same problems generated by students who write answers without having done the reading, by the way. And they are a reminder of the kinds of ethnographic writing and history/story telling skills that I actually value.Permalink On twitter.com
❤️ 1 Favorite
Mood -2 🙁
-
Happily, this means there's a better solution than unreliable AI detectors for this kind of writing: clearer standards, better examples, more careful rubrics, and clear grading based on writing quality.)Permalink On twitter.com
♻️ 1 Retweets
❤️ 2 Favorites
Mood +12 🙂
-
Big picture, ChatGPT essays are people-pleasing and superficial. I'm hopeful we can leverage our relationships of trust with students to get them to put forward their less polished, but better informed opinions instead of cheating. …Permalink On twitter.com
❤️ 2 Favorites
Mood +5 🙂
-
But, on the research side, LLMs give the impression of doing research, but… invent facts, and more importantly take generalizations about many cases as the starting point for telling about the one case in question. Real research functions the opposite way.Permalink On twitter.com
♻️ 1 Retweets
❤️ 1 Favorite
Mood 0
-
More public conversations with our colleagues and our students are needed to avoid the mass production of text by large language models overwhelming careful, evidence-based information in our classrooms and on the Internet. Again, my full take is here: woborders.blog/2023/08/23/why-im-banning-ai/Permalink On twitter.com
♻️ 1 Retweets
❤️ 5 Favorites
Mood +1 🙂