I had the pleasure of visiting Amazon’s HQ in Seattle last week to give a talk on my research. I also got to experience Amazon Go – the new grocery store they have created which has transformed how you purchase goods. You simply sign up, walk in, pick up and walk out. The paying happens automatically in the background. It is perfect for those who are in a rush and need to grab some lunch or a snack. No need to wait again for someone taking for ever deciding what to buy at the deli or fumbling with their purse to pay. ‘Just walk out shopping’ is their slogan.

It is a big step up from the last supermarket innovation of the barking self-checkout machines. Instead of scanning each item, you download the Amazon Go app and then place the QR code that appears on your mobile phone at a turnstile reader that lets you into the store. You are then registered as a customer, enabling you to take anything off the shelves, place it in your pocket or bag and then simply walk out when you have got what you want. Then, seconds after exiting, you get a receipt that itemizes what you took and deducts the amount owing from your Amazon account. It is all done through AI tech.

I thought Amazon’s 1-click ordering for online shopping was very clever at reducing the burden of shopping. But this new-fangled idea of shopping without having to go to a checkout is even more innovative. Except you feel a tad guilty – at least the first time – walking out of the store without the physical action of paying – almost like shoplifting in broad daylight.

So how does the tech do it? Has it stopped shoplifting? Can you trick the system? The smarts are in how the computer vision, sensor fusion and deep learning are combined to track you and what you take from a shelf. Look up to the ceiling and you see an array of depth sensor cameras protruding that track you and your arm movements as you move around and lift your arm towards a shelf. The imaging is coupled with what is detected as being removed using weight sensors that are placed on the shelves underneath each item. All pretty clever and reliable. The only time it may get confused is if two people in close proximity put their arm out at the same time, one on top of the other and reach out to a shelf at the same time. It could mistake the arms so that each gets billed for the wrong item. But it is smart enough to know if someone picks something up and put it back down again – even in the wrong place. Unlike the first generation of smart hotel mini-bars that billed you for just opening the fridge and lifting something out to take a look.

There are 17 of these stores now in the US and more being built each month. What next one wonders – if this kind of automated convenience store takes off. Besides revolutionizing how we purchase goods it could be used in other places, such as bars and pubs, removing the need to have to wait in line to buy a drink. You could even pull your own pint – but someone will still have to pick up the dirty glasses afterwards.