ARGUS: A smart shopping assistant for blind and visually impaired people
Non-accessible supermarkets pose a number of challenges for blind and visually impaired people: First of all, they have to orientate themselves in the aisles. Navigation is made even more difficult by the frequent changes in the sales floor, such as temporarily placed special offers. Once the right shelf is found, product recognition is often difficult because packages are too similar in material and shape. The hurdles for informing oneself and comparing products are accordingly high, and this applies all the more to independence in everyday life and the fun of shopping.
In the ARGUS project, Hochschule Bonn-Rhein-Sieg is working on a smart digital agent to support blind people in finding desired products and obtaining information about them. Integrated into everyday devices such as smartphones, this assistant is to be controlled via speech and an app and provide consumers with individual support in the shop. Artificial intelligence plays a key role here: as a learning assistant, the digital companion is to recognise products on the shelf and get to know its human and his or her preferences better and better in order to discuss taste and quality with him or her.
For the project team led by Professor Alexander Boden and Margarita Esau-Held, it is important to involve future users in the development of the digital shopping companion from the very beginning. For example, they conducted interviews with the target group to learn about their everyday practices and requirements. In order to experience problems described in everyday life, they accompanied blind people while shopping. One study participant describes her shopping experiences like this:
"For me, shopping is one of the biggest stressful situations. I would perhaps much rather go shopping because of the assistant, because it somehow takes the stress away from me and I don't have to make an effort to find anything. I could also help myself in a certain way. That's also important to me, that I can do as much as I can myself.
During our inception study, we became really aware again of the abundance of visual information and similar packaging that consumers are exposed to," says Margarita Esau-Held, project manager at H-BRS. "Therefore, the key challenge is to translate information such as notices, explanations, product features, colours, materials, which are all visually available at the same time, auditorily. And in doing so, to design a temporal sequence of information that meets people's diverse information needs."
In developing the digital shopping assistant, H-BRS and its partners in the project are relying on the increasing popularity of voice assistants in everyday life. However, the new type of shopping assistant will be specially tailored to the needs and skills of non-sighted people and thus go far beyond known voice assistants.
H-BRS is working on the ARGUS project with the start-ups SonicView and VAGO solutions and Petanux. The project has a duration of three years and is funded by the Federal Ministry of Education and Research. Within the H-BRS, the Institute for Digital Consumption (IVI) is responsible for the project. Based at the Sankt Augustin Campus, the IVI has extensive research experience in the fields of digitalisation of consumer fields, human-computer interaction and artificial intelligence. It also has extensive expertise in data management along the food supply chain.
The findings from the discussions and observations will be incorporated into the smart assistant, which will take shape more and more in the coming months. The exchange with those affected has already produced one important insight: the one solution that works equally well for all blind and visually impaired people will probably not exist - the needs are too different for that. On the other hand, the new shopping companion has the potential to be adapted for user groups other than people with visual impairments and to make shopping an informative and interactive experience.