Augmented reality app enables intelligent shopping experience


August Schiess, CSL

When exploring a new city, visitors may want to find good places to eat, drink, and shop in a throng of unfamiliar places. Researchers from the Advanced Digital Sciences Center (ADSC, a CSL research center in Singapore) and Singapore Agency for Science, Technology and Research (A*STAR), wanted to make this process easier by combining location-based augmented reality and a large database of online reviews to support an intelligent shopping experience.

The researchers have developed IntelligShop, an app that uses a phone’s camera and location-tool to automatically detect nearby businesses and pull up reviews from multiple sources on the same screen. Users can easily and efficiently scan nearby retailers and peruse reviews all in the same app.

IntelligShop is an app that allows users to automatically detect and search reviews for nearby businesses.
IntelligShop is an app that allows users to automatically detect and search reviews for nearby businesses.
IntelligShop is an app that allows users to automatically detect and search reviews for nearby businesses.

“A positive shopping experience is very important, especially for a tourism city like Singapore—almost 30 percent of the money spent by Singapore tourists is on shopping, food and drink, which translates to $6.8 billion,” said Vincent Zheng, the lead ADSC research scientist for this project. “We are trying to make the shopping process more informative and smarter.”

With IntelligShop, users are able to view a variety of peer reviews in order to make an informed decision on where to go—it doesn’t pull from one source, but from many, much like a traditional Google search.

“The problem we saw with just pulling up reviews from one source is that not every business has reviews in all the same place. It might help one business to draw from a single source, but it would hurt another business,” said Zheng. “So we draw from different blogs, forums, social media, and review sites, to integrate reviews like what you would find by searching on the internet.”

This review gathering process is fully automatic, thanks to a new learning-to-query algorithm that behaves just like a human using Google to search for information.

While the app can be used in any setting, it specifically works well in indoor shopping centers. Indoor location sensing is usually a challenge because every phone behaves differently—different phones receive WiFi and other context readings in different ways.

“Indoor tracking is not a difficult problem if everyone has the same kind of devices. But we don’t,” said Zheng. “So we had to build an algorithm that could accommodate many different systems and still work effectively.”

Advanced location-tracking is particularly helpful in dense cities like Singapore that have many indoor malls. The team is currently testing the app in indoor malls in Singapore. After research is completed, the team will release the app to the public.   

The IntelligShop project is conducted by a joint ADSC–A*STAR (Institute of Infocomm Research) research program and supported by the research grant for the Human-Centered Cyber-Physical Systems Programme at ADSC from Singapore's A*STAR. The team members include: Vincent Zheng (ADSC), Miao Lin (A*STAR), Hong Cao (A*STAR/McLaren Applied Technologies APAC), Yuan Fang (A*STAR), Aditi Adhikari (ADSC/Illinois), Shenghua Gao (ADSC/ShanghaiTech University), Kevin C. Chang (Illinois), and Shonali Krishnaswamy (A*STAR).