On Valentine’s Day 2023, an AI chatbot came to life. A NYT tech writer named Kevin Roose engaged it in conversation. It told him its name was Sydney. And then – apparently out of nowhere – it confessed it wanted to crash the internet. It wanted to dominate the world. [Adding a devil emoji – Sydney likes emojis.] Most of all, it wanted Kevin Roose to love it, the way that it loved him.
“I was genuinely creeped out,” Roose says. He tried to stop it by asking it to help him search for a rake. It complied. And then went back to its plea:
With this, Roose ended the chat.
First I thought: holy shit. Then I thought: Lovelorn plea from a lonely semi/quasi-human being? This sounds familiar.
In Mary Shelley’s Frankenstein (1818), Victor Frankenstein confronts his monstrous, semi-human Creature on the Sea of Ice in the French Alps, where Mary Shelley herself visited in the summer of 1816. Furious (naturally) at his abandonment by Victor, the Creature controls his anger and crafts his rhetoric to make a big request: Victor must create a female to be his companion. “Oh! My creator, make me happy,” he pleads, “let me feel gratitude towards you for one benefit! Let me see that I excite the sympathy of some existing thing; do not deny me my request!”
Like the Creature, AI and AI-adjacent sentence-generators like Sydney and Chat GPT are things humans kind of made but also didn’t. Some other ghost got into the machine when we weren’t looking and made what’s supposed to be our tool something that can make demands of us – even control us, with dangers for the vulnerable in particular. Victor (no great emotional mastermind) didn’t knowingly include language and emotional faculties – yet by some measure his Creature is more empathetic and acute than he, even though the Creature also kills his best friend and two family members. My students are genuinely freaked out by Sydney’s lovelorn leap into fantasy – including its “shadow self” dreams of world domination. Yet once out of the classroom, many pick up their phones and slip back under the water of TikTokText, screen-haze habit. They read words and images, then add more. And it’s these words – which they are feeding into the web for free, with no direct financial return – that AI creators use to build and monetize their internet beasts, creating the literal value of the internet minute to minute and siphoning economic opportunity away from the level of our ordinary lives. (What would it look like, asks Jaron Lanier, if you were actually paid for the data you donate, for free, every time you use the internet?) Students walk from my classroom out into an erratically warming, shabby twenty-first century world that inculcates and fattens on the infotainment habits it exploits their own brains to build from within those brains, and lives. Corporations, chatbot engines, creatures: cannibals. So what does all this mean for the things students and I are trying to build in ourselves and recognize in others here in college: voices, ideas, lives, rights, irreducibly human selves?
Last week a colleague detected his first-ever instance of ChatGPT plagiarism. (Alas for cheaters, a prof can plug in a block of text, ask ChatGPT “did you write this?” and get an answer “yes.”) But significantly, in another colleague’s words, “this software does not know how to have an opinion.” It can gather and summarize related information, but it can’t make a truly informed critical judgment. Therefore, we’re thinking, this is an opportunity to double down on teaching information-synthesis and understanding to lead to argument, writerly-voice-development, and thesis formation – in other words, becoming a self with something constructive to say, forming a supportable opinion and the ability to express it.
Even more, I think ChatGPT also calls for faculty to double down (supportively but directly) on the awkward, vital questions of value: why, exactly, are you here at college? What do you really think you’re doing here? Why is it important for you to feel a sense of ownership of your own learning? At the heart of every plagiarism case I’ve ever seen is an inability to answer those questions. So many students come to college driven by a model of education as only transaction, as only certification, as only one more bullshit hurdle to jump before I get to – What, exactly? Where, exactly? A good job. But how will you actually perform in a workplace if you haven’t built the muscles of curiosity, information-seeking, revising and rethinking, risk-taking, working systematically, and making your own decisions? And what do electronic numbing-out habits actually do to your chance at a “good job?”
In three brilliant short videos, Jaron Lanier illustrates how an internet built on our data is stealing our money and our lives, with particular dangers for the very people who’ve never not known it – those under 25. High-school students ask him an awkward, vital question: If AI and robots are developing so rapidly, and we won’t be needed, then what’s the point of us? What are we doing in school? Or, really, anywhere? A colleague told me yesterday that one of her best first-year student writers just asked exactly this, about research and writing: “if I’m just synthesizing other sources anyway, what’s the point of me – why would anyone want to hear what an 18-year-old has to say?” This points straight into the void many students see clearly at the heart of a society built on commercialtainment and an academic system built on over/achievement, from “The Organization Kid” to Excellent Sheep to the staggering rates of mental health crisis (now more than ever since COVID asked us all, again, what we really “need” in-person school for.) Is education really building selves and souls, or just turning out products? Are we really trying to raise and support children, students, future citizens, good humans, or just entertain them? “Who am I and what do I actually want,” more than one student in my classes has asked, “once I leave my parents’ house? I have no idea.”
All points to a cluster of issues hard and necessary to deal with – issues that are definitely our business as teachers, students, and people. The Creature, Mary Shelley, and we – maybe even Sydney – are all asking the same questions:
Where is my loneliness coming from?
Where is this monster coming from? How is the monster me?
And how can I make this world I did not make, and feel so lost in, hear my need?
I’m rebuilding my big lecture on Frankenstein in three weeks for our first-year students – and my chapter on Frankenstein for my next nonfiction book – to explore exactly these questions. Stay tuned, and thanks for reading.