Baldwin Upper School Students Author Cutting-Edge AI Security Paper

At the Baldwin School, Computer Science & Engineering Teacher Dr. Thomas Heverin’s Artificial (AI) Intelligence and Society class of 13 high school students is boldly advancing into the realm of AI research — territory often reserved for professors, PhD students and scientists from leading tech companies and universities. Their journey began with an introduction to prompt injection attacks, a type of cybersecurity threat where specially crafted prompts are used to trick AI systems into bypassing their built-in safeguards. For example, these prompts can be designed to coax a language model into generating misleading information it would normally refuse to produce. Inspired by this challenge, these 10th to 12th graders ethically devised inventive prompts that successfully bypassed the defenses of multiple large language models (LLMs), demonstrating the limitations of AI security. Then they reported their findings to an AI company to help improve its security. 

 

Guided by Dr. Heverin, the students then transformed their experimental findings into a structured, comprehensive study, culminating in a publication on arXiv (pronounced “archive), an open-access repository widely recognized in the science and technology research communities. Experts who also recently posted prompt-injection research on arXiv come from Meta’s Fundamental AI Research (FAIR) Team, the Centre for European Research in Trusted AI, Microsoft’s AI Red Team (an AI hacking team), IBM Research, UC Berkeley, the University of Wisconsin, Stanford University, Northwestern University and more.

 

Now, Baldwin School students can be counted among these AI pioneers! Their paper, titled Systematically Analyzing Prompt Injection Vulnerabilities in Diverse LLM Architectures, can be accessed here: http://arxiv.org/abs/2410.23308

 

Through this in-class project, Baldwin students are bringing fresh perspectives to an academic conversation traditionally dominated by seasoned professionals. With Dr. Heverin’s mentorship, these young researchers have not only developed insights into AI security but have also shown that high school students can be essential contributors to the advancing fields of AI and cybersecurity. 

 

Baldwin AI Researchers include Tori Benjamin ’25, Emily Braca ’27, Izzy Carter ’25, Hafsa Kanchwala ’25, Nava Khojasteh ’27, Charly Landow ’25, Leona Luo ’26, Caroline Ma ’27, Anna Magarelli ’25, Rae Mirin ’25, Avery Moyer ’27, Kai Simpson ’27, Amelia Skawinski ’25 and Dr. Thomas Heverin.