Biwan, Haryana, INDIA – In this arid farming village about two hours by car south of New Delhi, there are many needs.
One farmer needed help applying for pensions for his aged parents. Another wanted to know why his government assistance payments mysteriously stopped – and how to restart them. A university student needed a scholarship to fund her studies.
They all turned to Jugalbandi, a new generative AI-driven chatbot on mobile devices for government assistance. It can understand questions in multiple languages, whether spoken or typed. It retrieves information on relevant programs – usually written in English – and relays it back in local language.
In India, the language of government, business and public life is English, but it is spoken by just 11% of the 1.4 billion population. Some government documents are also in Hindi, spoken by 57% of Indians. That leaves vast numbers of the population unable to access government programs because of language barriers.
While the Jugalbandi chatbot is still new, it could one day offer all Indians easy access to information in local language through a mobile phone, instead of having to head to the local community service center and stand in line just to get basic information.
“We saw this Jugalbandi as a kind of ‘chatbot plus plus’ because it’s like a personalized agent,” said Abhigyan Raman, a project officer at AI4Bharat, an open-source language AI center based at the Indian Institute of Technology Madras in Chennai that is a collaborator on the chatbot.
“It understands your exact problem in your language and then tries to deliver the right information reliably and cheaply, even if that exists in some other language in a database somewhere.”
One of those who tested the chatbot is Vandna, an 18-year-old from Biwan who is a freshman at the Government College in the nearby village of Ferozepur Jhirka.
Vandna’s mother is a community health worker. Her father is ill and unable to work. The teenager is teaching part-time at a local elementary school to help pay for college.
When community volunteers introduced Jugalbandi to people in her village, in early April, she remembers typing her question in Hindi: “What kind of scholarships are available for me?” She added her course of study: Political Science, Hindi and History.
The chatbot replied with a list of central and state government programs. She picked one and asked about eligibility criteria. Jugalbandi provided those and also told her what documents she needed to support the application.
A traditional web search would have given her a long list of links to plow through, she said.
“Jugalbandi gives me one answer at a time,” she said in Hindi. “It’s easier to understand.” She applied for a scholarship in mid-April and is waiting to hear back.
The name Jugalbandi refers to a duet between two musicians in Indian classical music where they riff off each other to create something new together. The Jugalbandi AI assistant is powered by language models from AI4Bharat, a government-backed initiative, and reasoning models from Microsoft Azure OpenAI Service. It is accessed through the mobile messaging system WhatsApp, which is widely used in India, and the duet in this case is the conversation between the user and the chatbot.
It is part of the fast-growing field of generative AI tools that can synthesize vast troves of data to generate text, images and more. In this case, the AI chatbot has fine-tuned these models using data from Indian government databases that are being uploaded one by one. Using Azure OpenAI Service also helps ensure data security and includes features such as responsible AI protections that allow entities to filter inappropriate content.
Since Jugalbandi was introduced to villagers in Biwan in early April, it has expanded to cover 10 of India’s 22 official languages and 171 of a total of approximately 20,000 government programs, said Smita Gupta, a lawyer who works for OpenNyAI, a collaborative whose mission is to bring greater access to law and justice through AI. It is one of several groups working on the chatbot.
This is just the beginning.
In the future, such chatbots could be used for any interaction between a person and an institution, whether a patient seeking medical information in Urdu or someone retrieving English-language court documents in Tamil. Once connected to computer interfaces in government departments, citizens could theoretically complete an application just by speaking or typing.
India’s complexity makes it a test bed for multilingual settings everywhere.
“If you can solve and build for India,” said Gupta, “you can solve and build for the world.”
India has myriad government schemes and welfare programs, each with its own criteria and requirements. Official websites can be hard to navigate – or impossible if you can’t read or write or don’t know English. Getting precise answers early means fewer trips to the government service centers in each village for help and fewer trips home to retrieve missing documents.
India also has a track record of building digital public goods and rolling them out at scale.
More than a decade ago, the government launched Aadhaar, a 12-digit biometric identity system that can be linked to services such as banking, mobile phones and government services. A few years ago, it built a Unified Payments Interface for digital payments, which now sees 8.9 billion transactions a month.
In July 2022, the Indian government launched Bhashini to provide language solutions as digital public goods. The goal is to enable Indians to access the internet and digital services in their own language by drawing on emerging technologies such as AI to develop products and services through an ecosystem of start-ups, industry, academia and government. Research groups like the Indian Institutes of Technology, the International Institute of Information Technology and the Center for Development of Advanced Computing are all experimenting to augment the Bhashini platform.
AI4Bharat is one such research group. It has received funding from Nilekani Philanthropies, started by the co-founder of software giant Infosys Nandan Nilekani and his wife Rohini, as well as from Microsoft.
Pratyush Kumar, co-principal investigator at AI4Bharat and a principal researcher at Microsoft Research India, said the team initially explored how to translate legal judgments, working with the Supreme Court of India. They also worked with schools and colleges to transcribe videos and add subtitles, which can help children learn more effectively.
At the same time, organizations like OpenNyAI were thinking of applications on the ground. “We brainstormed a bit,” said Kumar.
One of the results was Jugalbandi.
This is how it works: A villager sends a text or audio message to a WhatsApp number, which initiates the Jugalbandi bot. That is transcribed to text using the AI4Bharat speech recognition model. That in turn is translated to English by the Bhashini translation model trained by AI4Bharat. Based on the prompt, Azure OpenAI Service’s model retrieves information on the relevant government scheme. The answer is translated to Hindi. That is then synthesized with the AI4Bharat text-to-speech model and sent back to WhatsApp – and the villager’s ear.
Building a chatbot used to be a complex task, requiring a flow chart. “Whereas now, with a bit of language tech, the government can just upload documents and it works,” said Kumar. “This democratizes not just who consumes this but also who produces it.”
Jugalbandi uses GPT models via Azure OpenAI Service.
There remain rough edges. “Sometimes these models do make errors. They are probabilistic machines,” Kumar said. “We need people to try them out with different inputs to see different responses. And if there are errors, to flag them. People still play an important role to see what works and what doesn’t work.”
That’s where organizations like Gram Vaani come in. The Delhi-based social enterprise has been working with farmers for years, providing agricultural news and a tech platform for requests for help and grievances, all through basic cellphones.
Its Mobile Vaani interactive voice platform has three million users across northern and central India. Local volunteers work one-on-one with those who need help.
Jugalbandi “can really scale a lot of our work,” said Aaditeshwar Seth, co-founder of Gram Vaani.
Gram Vaani’s community volunteers are evaluating the chatbot for integration with Mobile Vaani. The villagers are also giving feedback on improvements they would like to see.
Abdullah Khan, a 26-year-old farmer in Biwan, has used Jugalbandi to help fellow farmers find out about government financial assistance programs.
He has a suggestion. Besides farming, many people in Biwan drive trucks for a living. There’s no union and no benefits.
“Something that supports drivers will be helpful,” said Khan. He suggested Jugalbandi help figure out requirements for applying for drivers’ licenses, and even facilitate the application itself so drivers don’t have to pay middlemen to do so.
As more people think of more uses for the chatbot, Raman of AI4Bharat predicts adoption will speed up. For example, OpenNyAI is working on how the chatbot can help domestic workers and garbage collectors in Bengaluru understand their legal rights.
“This is revolutionary for people who could not interact with tech because of language barriers,” said Raman.
Top image: Abdullah Khan, left, shows the Jugalbandi chatbot to fellow farmer Sapat Khan, right. Photo by Probohadh Singh for Microsoft.