Cashier Robot.
An emotive robot to make the grocery store experience a little better.
Roles: All designers. This was a group project with Isabel Freedman and Abigail Radey
Timeline: 3 weeks
Context: Coursework
Motivation: We designed a cashier robot to simplify and speed up the workflow of checking out at a grocery store. Our product takes up less space, works faster, senses the user's moods, answers questions, and provides valuable insights about the items the customer purchases.
We began by brainstorming the difficult parts of out daily lives. Then, we began planning the automation and analyzing the tasks.
Tools: Adobe XD, Adobe Photoshop, Adobe Illustrator, Excel
Background to issues.
We decided to create a cashier social robot because currently there are many pain points to the checkout process. We analyzed two modes of checkouts when deciding what issues to solve in the checkout process: self checkout and cashier checkout.
Self Checkout.
Self checkout is beneficial because it allows customers to check out their own items and potentially move faster, especially if they have fewer items. This is because they don’t have to go back and forth between themselves and the cashier. However, current pain points with process are when buyers have to check out an item without a barcode, for example, produce. This requires the user to have to look up the item, weigh it on the scale, then get the price. This process is long and usually irritating for the customer. Additionally, since it’s self checkout, the customer has to bag their items themselves which adds time to the process.
Cashier Checkout.
Cashier checkout is beneficial because the cashier knows how to scan many items quickly, usually knowing where their barcodes are and how to position the item to get scanned in one try. Additionally, for items without barcodes, they often have their codes memorized allowing for quick entry versus when the customer does it at self checkout. However, often there are two people needed to work the cashier and bag items, which prevents using employees in other parts of the store. Despite cashiers’ efforts to streamline the checkout process, they often run into human error with scanning items and the order in which they scan them in.
Users
User Physicality and Psychology.
Since we made a robot that to interact with people, their food, and their credit cards, we needed to ensure that our robot is trustworthy. To make sure that our users feel as though they can trust the robot, we made it behave similarly to a human. This is apparent through the interactions that people generally have at grocery stores with common questions regarding if the shopper was able to find everything they were looking for etc.
Additionally, when people pack grocery bags, they are methodical about which items they scan and pack first. Because of this, our robot is trained with machine learning to identify the aspects of products that determine the placement of that product in the bag.
Our system also has a camera to quickly scan the cart and to decide how the different products compare to each other. To enhance the ability of our robotic cashier, we included a conveyor belt to transfer the customer’s food outside so that the shopper can exit the store and get their car with free hands, and then pull around to the side of the grocery store to pick up the rest of their food.
Automation
Our Solution
The Cashier Bot sits behind a small checkout counter and offer the convenience and privacy of a self checkout, but the speed and reliability of a normal cashier.
What is the Cashier Bot?
We have designed a robot that will take the place of a cashier. This robot will interact with the customer, asking if they have found everything they need, etc. The robot will then empty the customer’s cart, and make sure all of the items are in good condition and are not about to expire. It will then scan all of the items and systematically pack the bags in an efficient manner.
What does the Cashier Bot do?
The cashier bot interacts with the customer, scans items with a camera to ensure they are all of good quality, reads the barcodes of each item to add them to the bill, and efficiently packs up grocery bags.
Why does the Cashier Bot change the game?
We decided to make a checkout robot for this setting mostly for quality assurance. It can be time consuming for customers and cashiers to check every expiration date, package items, verify the state and quality of produce, etc., so we made a robot that can quickly scan barcodes and food, to ensure that it will not rot or get moldy soon to help ensure that the customer can purchase items without fear of them going bad soon.
How does it work?
Our system uses several different sensors located and hidden in our cashier bot. Each sensor allows for different modes of interaction with the outside world to simulate a perfect cashier as possible. Our high-tech solution combines inputs from all the sensors in real time to accurately perceive the world around the cashier bot and process all the data in real-time to decide on modes of interaction and create a seamless checkout experience.
The primary actuators for the cashier robot are the arms. These arms use an accordion-style connection to create a full range of movement with internal gearing to maintain grip strength. These arms, together, mimic a human cashier but do not tire nor get repetitive stress injuries. The reason for two arms versus other shapes was to maintain familiarity for customers and avoid the uncanny valley.
Microphone.
The microphone is used to listen to the customer and respond in real time. There are several microphones positioned around the checkout counter and robot so that the system can differentiate background noise from the customer’s words.
Radar.
The radar system is used to help measure the shape, size, and position of the limbs of the robot to different items as well as help discern the general orientation of products. This system is overall faster and easier to process than some other data streams so it can help the robot work faster, which will help endear it to the customer.
Lidar.
Lidar is used much the same way as the radar system. This system is more accurate and can help the system build a good 3D model of the workspace and customer’s cart.
Force Sensor/Scale.
The force sensor and scale are used to measure how much produce the customer bought. Rather than positioning the produce on a scale that is built in, the system can pick up the produce, measure how much force was exerted on the arm, and then create an accurate measurement of the produce. This process is faster and requires fewer steps, which will help customers trust and enjoy the system more because it has outward speed benefits for them.
Air sensor.
The air sensor is used to determine if a food is rotten. This system can help the store avoid liability for accidentally selling spoiled foods and can also help the customer feel like the robot is on their team which will garner trust.
Barcode Scanner.
It all begins with an idea. Maybe you want to launch a business. Maybe you want to turn a hobby into something more. Or maybe you have a creative project to share with the world. Whatever it is, the way you tell your story online can make all the difference.
Camera.
The system uses cameras for many functions such as greeting the customer and maintaining eye contact and eye lines as well as identifying products for a faster and more accurate scanning experience without the need to look up items by name such as in the case of ‘dumb’ self checkouts. Furthermore, the system uses machine vision and AI based on the camera input to identify rotten and spoiled produce which will help customers trust the system more over time. Most importantly, the cameras will help measure the mood of the customer to facilitate good interactions between the customer and the robot.
AI.
Although an AI/machine learning system is not technically a sensor, this system is part of the larger functioning of the sensors and is critical to their seamless operation of the system as a whole. The AI is used to understand everything about the customer such as their mood, their gaze, their demeanor, and their tone. Similarly, the system uses AI to identify different items without the need to scan them as well as optimizing how to bag the groceries based on best practices and socially typical practices.
Big Data.
Similarly to the above, while big data is not a sensor, it is used to train the AI and continue its evolution and optimization. One large benefit of the cashier robot is that they can all communicate with each other and improve the algorithm across all bots in all stores. While each unit is shipped with the ability to recognize over 150,000 different products found in most grocery stores across the U.S., the cashier robot is constantly learning and adding new items to its catalog. Therefore, even more boutique stores with different items will find this system useful as it is constantly learning items and helping grow the capabilities of other units across the world. Likewise, as there are different social customs in different parts of the world, the system uses all the data it collects to learn about different customer types and expectations to improve the abilities of all the robots. Therefore, even though this system has originally been trained primarily in the U.S., it will use big data to adapt to other markets.
How does the Cashier Bot fit into the checkout experience?
1. Bring your groceries to the Cashier Bot.
2. Wait while the Cashier Bot automatically scans.
3. Load your items into the car.
Emotive Displays
Welcoming.
The robot displays this face when the customer approaches with their groceries, and again when the customer is leaving with their groceries. The robot does this to make the customer feel comfortable, signal that they can approach, and to mimic a human cashier.
Friendly.
The robot displays this face while greeting the customer, and again when the customer is leaving with their groceries. The robot uses this face to make the customer feel comfortable and mimic a person.
Curious.
The robot uses this display while asking the customer questions at the beginning and end about their experience, and while unloading the cart. The robot does this to mimic how a person would act and instill the sense of processing and thinking by engaging the customer.
Thinking.
The robot uses this display while assessing quality of groceries, scanning groceries, and packing grocery bags. The robot does this to mark a shift towards the primary function of food and performance analysis.
Checkout UI.
The cashier bot uses a checkout system and display similar to a typical grocery store so that the user can see the items that they are purchasing at a glance.
Cart Screen.
The user can watch on a screen what the robot scans to verify that the system is working as designed. This system helps the user gain trust because there is increased transparecy.
Payment Screen.
Once the system verifies that there are no remaining items in the cart, the system is ready for the user to select a payment method and check out. The user either selects or tells the cashier bot the method of payment then follows the robots prompts to finish paying.
Modes.
The system has two main modes: quiet and talkative.
Quiet.
When the cashier bot senses that the customer is not interested in talking, the system will only sent the necessary prompting to the user.
Talkative.
When the cashier bot senses that the user would like to talk, the system will engage in small talk with the user during the checkout process.
Future Steps
Going forward, we are interested in expanding our robot to also help with getting the customers groceries to their car. Another pain point of grocery shopping is having to carry a lot of heavy grocery bags to the car. Therefore, if we were to expand the capabilities of our cashier bot, we would design an efficient and trustworthy way to have the robot move the groceries out of the store and ensure that they make it into the right car.