To combat rising retail theft, Harris Teeter uses AI cameras to track self-checkouts

In retail, the unaccounted loss of inventory is known as shrink, and at Kroger, the largest U.S. grocery store by revenue, shrink has grown. Last month, Kroger Chief Financial Officer Gary Millerchip told investors shrink at his company increased this spring, with the hike “primarily due to rising theft and organized retail crime.”

Retail theft is not limited to Kroger, which operates 2,750 stores in 35 states — including 150 Harris Teeters in North Carolina (Kroger acquired Harris Teeter in 2014). Target, Foot Locker and Dollar General all identified stolen goods as significant issues this year. Walmart has estimated theft costs the retail giant annually around $3 billion.

According to a survey released last week by the National Retail Federation, retail shrink went from $94 billion in 2021 to $112 billion in 2022. The largest factor contributing to these unaccounted losses is external theft, which continued to be responsible for roughly 36% of overall shrink.

“We are implementing initiatives to mitigate the financial impact, including increased security and new technology solutions,” Millerchip said during Kroger’s Sept. 8 earnings call. “But we expect shrink trends will continue to be a challenge for the remainder of the year.”

One part of the grocer’s approach to security involves an emerging field of artificial intelligence called computer vision.

At more than 1,900 Kroger-affiliated stores nationwide, cameras hover directly above individual self-service checkout kiosks and capture customer behavior below, including items and arm movements. This footage is then fed to a platform called Everseen Visual AI, which analyzes the video to learn standard shopper behavior and pinpoint irregularities.

Everseen is an Irish company, but its software runs on Lenovo servers designed in Research Triangle Park. With around 5,000 local workers, the Hong Kong-based Lenovo is among the Triangle’s largest tech employers and has its North American headquarters in Morrisville.

“All our server design comes from the RTP location,” said Robert Daigle, Lenovo’s director of global AI business. “The engineers that are actually designing the systems are based here. Our AI team is headquartered out of RTP as well.”

How self-checkout AI works

Everseen tracks both customers’ arm movements and their purchases, said Reed Hayes, a criminologist at the University of Florida who researches the artificial intelligence program.

“When you’re scanning items, your arms might normally go left to right,” he said. “Now, if somebody is not scanning across where the object would be scannable — if they move it towards their body so it’s not over the scanner or they are using their hands to cover up the bar code — those are examples of (the AI) recognizing aberrant behavior.”

Some irregularities are innocent — shoppers who mistakenly miss a scan — while others are intentional, like customers who cover the barcodes of more expensive products with tags of less pricey items. When an error is spotted, the system sends a screen message to shoppers urging them to correct. If the issue goes unresolved, staff are then notified to intervene.

“Over 75% of the time, customers are able to resolve scanning errors themselves, with no intervention from our associates,” said Chris McCarrick, Kroger’s senior manager of asset protection solutions, in a case study Lenovo released in November.

Hayes, who also serves as the director of the Loss Prevention Research Council in Gainesville, said he has worked with several other retailers who have implemented Everseen technology, including the largest U.S. retailer, Walmart.

Walmart confirmed to the N&O it had previously used Everseen platforms but that it no longer does so.

In 2019, Kroger began to roll out computer vision, and by last year, Everseen was operating on Lenovo servers in more than two-thirds of its stores. Kroger does not have any locations in North Carolina; it left the Tar Heel State in 2018, but its Harris Teeter stores remain.

Daigle confirmed computer vision is used in North Carolina Harris Teeter stores, adding his company’s servers run Everseen AI platforms for other retailers around the world.

How successful the platform has been to mitigate theft is less clear. When asked about the performance of Everseen, a Harris Teeter spokesperson directed The News & Observer to send its questions to Kroger’s head of media relations who did not respond to multiple emails and phone calls.

How to stop theft outside the self-checkout

Estimating the amount retailers lose each year at self-checkouts is also imprecise.

Hayes says his organization’s research shows between “half a billion to $2 billion or $3 billion” worth of goods is misplaced each year due to purposeful or accidental self-checkout errors.

However, not all retail crime occurs at checkouts. Hayes said “an extreme problem” is when people simply push their shopping carts out of the store without paying. Stores are also addressing cart push-outs with technology; for example, a platform from the Canadian company Gatekeeper Systems recognizes carts that didn’t go through the checkout lanes and locks the wheels before they are taken outside.

“With any AI, it’s designed as a head’s up,” Hayes said. “’Hey, take a look at this.’”

AI in stores, fishing boats and NFL stadiums

While much of the current hype surrounding AI emphasizes software (think ChatGPT), companies like Lenovo provide the hardware that allows artificial intelligence to function.

In 2017, Lenovo launched a major AI initiative, partnering with the chipmaker Nvidia to add graphics processing unit (GPUs) to its existing servers. GPUs are a category of accelerators, which are essential to current artificial intelligence solutions that use deep learning, Daigle said.

In recent years, on-premise servers have become faster, smaller and, importantly, quieter.

“When you (used to) start those things up, it sounded like a 747 starting its jet engines,” Hayes said.

At Kroger, each Lenovo server can analyze real-time data from as many as 20 cameras. About the width and length of a personal desk, they are stored on site in backrooms and small enclosures. The servers’ proximity to the checkout permits faster performance.

Positioning the servers closer to the data points is a trend known as edge computing. Instead of routing the data to centralized data centers, companies can obtain real-time information by having the data travel a few feet rather than many miles.

This closeness comes with challenges. Being on site gives Lenovo less control over temperature and conditions. AI platforms demand more power from their servers, and they operate at higher temperatures. For example, while centralized data centers might be kept at 70 degrees Fahrenheit, some on-site AI-capable servers run above 120 degrees, Daigle said.

“I mean, it’s a hockey stick curve on the power increase for servers that run AI workloads,” he added.

Lenovo facilitates AI software beyond checkout aisles. The major Mexican tuna producer Grupo Pinsa uses Lenovo servers to track catches on fishing boats. In the United States, Lenovo hardware enables AI to assess customer movements at locations like the Mall of America and the Denver Broncos stadium.

By knowing where people congregate, organizations can better allocate their resources, like security and cleaning staff. They can also give this information back to consumers, like when digital displays project estimated wait times in lines.

As Hayes said, artificial intelligence gives companies “a head’s up.” What people do with that information is what makes a difference.

Open Source

Do you enjoy Triangle tech news? Subscribe to Open Source, The News & Observer's weekly technology newsletter and look for it in your inbox every Friday morning. Sign up here.