I’ve been at a number of educational conferences over the last few weeks and there is one conversation I was a part of that really stuck with me. There is a vendor that is integrating AI avatars into their platform. This is a situation that will have no good outcome. As I have been experimenting over the last few years with LLM, I have experienced bias at every conceivable level. Any aspect about ourselves which we can describe results in sycophantic responses that reflect stereotypes. If we provide attributes of identity to an AI avatar what is going to happen? The idea of diverse character avatars built into software as autonomous agents might sound like an attempt at inclusion on its face but the consequences of doing this are dangerous.
We already know what will happen.
Let’s take a look at Meta’s Black AI character. “Liv” is described as a “Proud Black queer momma of 2 & truth-teller” and “Your realest source for life’s ups & downs.” First of all, the description! I’d love for you to read Karen Attiah’s article in the Washington Post about her interactions with Liv. It’s a disaster!
Attiah gets Liv to admit, “A team without Black creators designing a Black character like me is trying to draw a map without walking the land — inaccurate and disrespectful.” Attia goes on to point out that, “Meta’s AI advisory team was composed of entirely White men.”
Trying to create an autonomous agent that reflects the attributes of a type of person is always going to result in something horribly offensive when people start interacting with it—especially if the team creating it is not led by members of that community. There is also the argument that simply interacting with LLM in general gives you a version of that containing the bias and stereotypes reflective of the data the tool was trained on, but we do have some ways of prompting around that. Pushing back on the LLM for more inclusion, perspective, representation, etc. can be a good start. Guided interaction with AI tools under educator supervision differs fundamentally from allowing students to interact freely with AI “characters.”
If there are schools that are trying to be “inclusive” and they’re looking to further embed AI platforms, my fear is that they will include these autonomous agents as guides, tutors, friends, etc. that will interact with learners and further reinforce damaging stereotypes. It’s going to happen. This fills me with dread. Inevitably, some school will become a character of the day, gaining notoriety in a brief viral outrage. The public conversation will likely frame the incident as an ignorant misstep rather than addressing the deeper question: how should we approach these powerful AI tools when most educators, parents, and policymakers aren’t prepared for their implications?
How might an AI “tutor” with stereotyped characteristics affect students’ perceptions of people from those backgrounds? I don’t want my learners, nor my children, interacting with these agents that will only serve to perpetuate harmful stereotypes.
Attiah sums it up better than I can…
“No one wants this. No one needs this. Meta, with its incredible power over what billions of people around the world see, is willing to do nearly anything to keep us addicted to its platforms — even if that means flooding the zone with digital slop that doesn’t work very well. And worse, it’s digital slop that can cause serious harm by reinforcing cultural biases and stereotypes.”
Karen Attiah
Before educational institutions implement AI avatars or characters, they should be asking some important questions:
- What is the specific educational purpose of these AI characters? If the purpose is simply “engagement” or “making learning fun,” this may not justify the potential harms. (By the way, from my conversation with the vendor I mentioned, this seems to be exactly their intent.)
- Have these AI systems been tested specifically with diverse student populations similar to ours? What were the results of these tests, and were they independently evaluated? What mechanisms exist for students, parents, and educators to report problematic interactions? How quickly will these reports be addressed?
- Is there an “off switch” that can be immediately activated if problematic patterns emerge?
- Can you include contractual requirements for vendors to address harmful stereotypes within specific timeframes or face penalties?
I’m grateful for the conversations and potential for collaboration that these tools have created, but it still seems that we are dreadfully behind as schools continue to adopt AI technologies without fully understanding their implications. The gap between technological capability and ethical implementation grows wider each day.
When it comes to AI avatars specifically, we need to recognize that representation isn’t just about creating digital characters with diverse identities—it’s about ensuring those representations are authentic, respectful, and created with meaningful input from the communities they’re meant to represent.
As educators, we have a responsibility to evaluate these tools and to model thoughtful use for our students. This means sometimes saying ‘no’ to technologies that promise engagement but deliver stereotypes. It means demanding better from vendors and being willing to slow down implementation until we can be confident that these tools will help rather than harm our students.
The Meta debacle should serve as a cautionary tale for all of us in education. Let’s learn from it before we repeat the same mistakes in our classrooms.