Key points:
- The ethics of AI in education hinge on who decides what data is collected and how it influences learning outcomes
- DEI under siege: How AI can bridge the gap in higher education
- A new era of admissions: AI’s potential to transform equity and diversity
- For more news on AI in classrooms, visit eCN’s AI in Education hub
Imagine a classroom where artificial intelligence (AI) tailors lessons to each student’s unique learning style, providing instant feedback and helping both pupils and teachers refine their skills. This is the promise of AI in education–personalized learning at an unprecedented scale. However, alongside this potential comes a pressing need to address the ethical implications of AI’s growing influence. At the heart of this discussion lies a critical question: Who decides what information AI systems are built upon, and how does this impact the next generation of learners?
Addressing the politics of data collection, biases embedded in AI systems, and their societal implications is essential to ensure transparency, accountability, and inclusivity in how these technologies shape education for all communities.
The politics of data collection
AI systems rely on vast datasets, but selecting and curating this data is anything but neutral. Decisions about what data to include–and what to exclude–are made by people, often reflecting the perspectives and priorities of those in power. The way data is collected shapes the predictions AI makes. Without diverse voices guiding this process, biases are perpetuated.
Consider the case of proctoring software used in Dutch universities, which failed to recognize dark-skinned students accurately, leading to formal complaints about discrimination. Similarly, students from low-income families faced challenges when family members inadvertently passing behind their screens were flagged as “aberrant behavior.” Biased data collection practices can lead to the marginalization of already vulnerable groups.
Biases and their far-reaching implications
Biases embedded in AI tools extend beyond individual experiences to reinforce systemic inequalities. For instance, while AI-powered tutoring systems can adapt to a student’s pace, they cannot address root causes of educational disparity, such as poverty or systemic discrimination.
These biases often have the greatest impact on marginalized communities. AI systems that disallow essential accommodations, like breaks for students with disabilities, can exacerbate stress and exclusion. AI must support sound educational policies, not replace them. Relying on technology alone risks deepening the very divide we seek to bridge.
Unchecked biases also pose broader societal risks, from normalizing discriminatory practices to eroding trust in educational institutions.
The need for diverse perspectives in data governance
To create equitable AI systems, including diverse voices in data governance is vital. Currently, women and people from underrepresented backgrounds remain a minority in the AI industry. Globally, only 28 percent of STEM researchers are women, a disparity with significant consequences for technology design and implementation.
It’s not enough to have more women coders. We need diverse perspectives at every decision-making level, from governance to auditing, to ensure AI serves all communities fairly. Global conversations, including those in educational settings, are crucial to ensuring AI technologies do not perpetuate existing inequalities or create new ones.
Advocating for transparency and accountability
Transparency in AI begins with clear, accessible information about how data is collected, processed, and used. Organizations and governments must shoulder the responsibility for ensuring their AI systems meet ethical standards.
The idea that individuals alone are responsible for their online safety is unfair. We need strong regulations that hold companies accountable and promote AI literacy among users. UNESCO’s 2023 guidance on generative AI in education provides a framework, emphasizing cultural diversity, safeguarding human agency, and testing AI systems locally to ensure they meet community needs.
Recommendations for ethical and inclusive data practices
To address these challenges, educators, policymakers, and technologists must collaborate on solutions that prioritize human values:
- Establish ethical frameworks: Evaluate AI products from multiple perspectives, including privacy, functionality, and human rights impacts.
- Adopt inclusive policies: Implement regulations that promote equity and accountability at all levels of AI development.
- Empower through AI literacy: Encourage critical thinking in educational settings, equipping students with tools to question and analyze AI-generated content.
- Reframe AI as a human-centered discipline: Integrate ethics, social sciences, and humanities into AI education to foster a multidisciplinary understanding of its societal impact.
Shaping an ethical AI future
Integrating AI in education offers immense potential but presents significant ethical challenges. By addressing issues of bias, transparency, and inclusivity, we can ensure that AI serves as a tool for progress rather than perpetuating inequality.
We must move beyond embracing AI’s potential and actively shape its development to create a fairer, more just future for all. Through ongoing dialogue, collaboration, and a commitment to human-centered innovation, we can prepare the next generation to use AI and shape it responsibly.
Ivana Bartoletti is the Global Privacy Officer at Wipro and an internationally recognized thought leader in the field of responsible technology.
- Stop calling Gen Z lazy - March 14, 2025
- Tightening Title II: A blueprint for digital accessibility in higher ed - March 13, 2025
- The Trump administration’s DEI orders are dividing Colorado universities–and their students - March 7, 2025