Earlier this year, South Australia’s Department for Education decided to bring generative AI into its classrooms. But before they opened the doors, one question loomed large: how to do it responsibly?
The core concern revolved around helping to protect students from potentially harmful or inappropriate content that may be reflected in the output of a large language model, given its training on vast and unfiltered expanses of the internet, said Simon Chapman, the department’s director of digital architecture.
“What’s available with the public versions of generative AI is extremely limited for education,” he said. “There are no guardrails around how students might interact with it.”
Fast forward to today, and the department is wrapping up a pilot of EdChat, an AI-powered chatbot. Nearly 1,500 students and 150 teachers across eight secondary schools tested the bot’s ability to help research everything from cellular division to the plot of John Steinbeck’s “Of Mice and Men.”
The department is still evaluating the results, Chapman said. But the Microsoft system it used as its guardrails received high marks: Azure AI Content Safety, an AI-powered platform to help organizations create safer online environments.
EdChat’s built-in safety features blocked inappropriate input queries and filtered harmful responses, allowing teachers to focus on the technology’s educational benefits rather than content oversight, he said.
“We wouldn’t have been able to proceed at this pace without having the content safety service in there from Day 1,” Chapman said. “It’s a must-have.”
Microsoft today announced the general availability of Azure AI Content Safety within the Azure AI platform, which uses advanced language and vision models to help detect hate, violence, sexual and self-harm content. When the models detect potentially harmful content, they mark it with an estimated severity score. That allows businesses and organizations to tailor the service to block or flag content based on their policies.
Initially introduced as a part of Azure OpenAI Service, Azure AI Content Safety is now a standalone system. That means customers can use it for AI-generated content from open-source models and other companies’ models as well as for user-generated content as part of their content systems, expanding its utility, Microsoft said.
As generative AI enters the mainstream, Microsoft aims to equip businesses with the tools needed to deploy it more safely, said Eric Boyd, corporate vice president of Microsoft AI Platform.
“We’re at a pretty amazing moment right now where companies are seeing the incredible power of generative AI,” Boyd said. “Releasing Azure AI Content Safety as a standalone product means we can serve a whole host of customers with a much wider array of business needs.”
It’s just one example of how Microsoft has been putting its principles for creating AI responsibly into practice. The company has nearly 350 people working on responsible AI. Over the last few years, Microsoft has advocated for responsible AI governance and supported U.S. White House voluntary AI commitments; pushed the boundaries of responsible AI research to inform product development; released a Responsible AI Standard for designing, building and testing AI systems; and put tools like Azure Machine Learning’s Responsible AI Dashboard into the hands of customers.
“This is not an afterthought but a key part of our AI innovation story. We’re bringing the best tools to figure out how to solve these responsible AI challenges,” said Sarah Bird, a partner group product manager at Microsoft who leads responsible AI for foundational technologies. “Azure AI Content Safety has been a foundational piece that allows us to ship our generative AI applications to more customers.”
Customizable for different use cases
Internally, Microsoft has relied on Azure AI Content Safety to help protect users of its own AI-powered products. The technology was essential to releasing chat-based innovations in products like Bing, GitHub Copilot, Microsoft 365 Copilot and Azure Machine Learning responsibly, Bird said.
In a world awash in content, responsible AI problems cut across businesses and industries. That’s why a key advantage of Azure AI Content Safety is its configurability, Bird said. Policies can be tailored to fit each customer’s specific environment and use cases. A platform focused on gaming, for example, will likely set different standards for things like violent language compared to one for classroom education.
For example, energy company Shell developed a generative AI platform, Shell E. The platform allows teams to build and deploy applications based on open source and other large language models, which enable employees to ask questions and retrieve information to complement their workflows.
Shell E is intended to help workers access and use company knowledge more efficiently, said Siva Chamarti, senior technical manager for AI Systems at Shell. Use cases cover all areas of Shell and include drafting, summarizing and translating content; code generation and testing; and knowledge management across all technical areas. The teams are also exploring more complex use cases such as molecular generation.
Azure AI Content Safety plays a pivotal role in Shell E platform governance by enabling text and image generation while restricting inappropriate or harmful responses, he said. The tool checks each input and output for policy compliance before allowing generated content to reach users.
“We’re seeing huge enthusiasm for generative AI,” Chamarti said. “It’s still early days, but colleagues are coming up with creative uses for their own work, and our backlog of use cases just gets longer.”
Building trust in the AI era
As generative AI continues to proliferate, so will potential threats to online spaces.
To keep ahead of those threats, Microsoft will continuously improve the technology through research and customer feedback, Bird said. For example, ongoing work on multimodal models will strengthen detection of potentially objectionable combinations of images and text – think memes – that can be missed by individual checks.
Boyd noted that Microsoft has dealt with content moderation for decades, creating tools that helped protect users everywhere from Xbox forums to Bing. Azure AI Content Safety builds on that experience — and on Azure Content Moderator, the previous service that enabled customers to tap into the power of content detection capabilities — and takes advantage of powerful new language and vision models.
By thinking holistically about safety, Microsoft hopes to enable broad adoption of AI that consumers can trust.
“Trust is something that has been a cornerstone of the Microsoft brand and of Azure really since the start,” Boyd said. “As we move into this AI era, we’re just continuing to expand on how we make sure that we protect our customers.”
Related resources:
- Learn more: Azure AI Content Safety
- Read: Reflecting on our responsible AI program: Three critical elements for progress
- Read: Announcing Microsoft’s AI Customer Commitments
- Read: Our commitments to advance safe, secure and trustworthy AI
- Read: Announcing General Availability of Azure AI Content Safety
Top image: A student and teacher from Mitcham Girls High School in Adelaide work with EdChat, an AI-powered chatbot that uses Azure AI Content Safety to help protect students from inappropriate or harmful content in output that may be returned by large language models. Photo courtesy of South Australia Department for Education.