Wait, why is there a camera hole in that Coke vending machine?

Ben Hasty—MediaNews Group/Reading Eagle/Getty Images

If you’ve recently visited a fast-food restaurant like Wendy’s or Burger King or a movie theater like AMC, there’s a good chance you’ve grabbed a drink from a modern-looking soda fountain with a touch screen.

The machine in question, dubbed the Coca-Cola Freestyle, debuted in 2009 and lets customers choose from more than 100 drinks and flavors—from the traditional Coke or Sprite to fringe faves like flavored ginger ales. Though carbonation snobs have been known to complain that these soda fountains allow remnants from one drink to seep into the next pour, the Freestyle devices have been popular enough that more than 50,000 of them are in public use.

And yet, one mystery about the machines has popped up occasionally over time … If you are looking closely enough: Why do many of them have what appears to be a small camera lens embedded above the touch screen?

This question began with a tip passed along to this reporter: A salesman for a Big Tech company had boasted that Coca-Cola was capturing images of soda fountain customers through cameras, and then working with said cloud-computing giant to gain insights on customers’ facial reactions when they blend certain flavors together to create a custom drink. With that data, the thinking went, Coca-Cola might have new insights into new flavors it should consider producing.

A source who worked for Coca-Cola for many years told Fortune they had heard a similar rumor, but couldn’t confirm it for sure. A few Reddit posts also pondered the question of the camera hole with mostly unsatisfactory responses. And a Coca-Cola spec sheet found online for one of the Freestyle models lists an embedded camera as one of the soda fountain parts, along with “future capability for motion sense and facial recognition.” Hmm.

While a camera embedded in a soda machine might sound absurd, businesses and event venues are increasingly using new facial analysis and recognition technologies in public settings. MSG Entertainment, which owns the Madison Square Garden sports arena as well as other venues like Radio City Music Hall, has recently used facial recognition technology to bar lawyers working for firms engaged in legal battles with the company from attending any events at its New York venues. More airports are adding it at security checkpoints, too. And just this February, students at the University of Waterloo pressured its administration to remove vending machines secretly embedded with facial analysis technology.

So what does Coke have to say about all of this?

A spokesman for the company, Scott Leith, said in a statement: “Camera capabilities were included in earlier designs and tested in a laboratory setting in 2018 and 2019. The company has no plans to use cameras in the future.”

Leith did not elaborate on whether machines in the field still have cameras in them nor on the exact nature of the testing. But a patent application filed in the summer of 2018 by two former Coca-Cola employees may provide a snapshot of the intent, saying the system “may be able to determine the sentiment of the consumer.”

“For example, whether a consumer who appears angry seems to appear happier after dispensing a beverage and/or whether the consumer enjoyed the beverage,” it says. “Such data also may be used to identify new mixes that are preferred by consumers … Entirely new beverage combinations thus may be created and promoted.” This reasoning sounds a lot like the original tip this reporter received.

The application also cited the potential for a soda machine to identify demographic information about a customer “for marketing purposes as well as for an improved consumer experience.”

“The method may include the steps of sensing physical characteristics of the consumer, matching those physical characteristics of the consumer with demographic characteristics, promoting a beverage selection to the consumer based upon the matched demographic characteristics, and providing a beverage to the consumer,” the application read.

But there’s a bit more. As of last month, a case study on the website of an AI company called Quantiphi referenced working on a facial recognition project related to the Coca-Cola Freestyle machine.

“Each vending machine comes with a camera installed, in which an image is captured for every customer interacting with the machine,” the case study on Quantiphi’s website read. “Coca Cola’s marketing team aspired to use these images to generate insights on consumer preferences and usage patterns,” including to “analyze the types of ‘mocktails’ preferred by their consumers.”

The mocktails in question reference when a Freestyle soda fountain customer creates their own drink by mixing two or more flavors available from the machine.

The case study also said that Quantiphi built “a custom machine learning model trained on 8,000 to 10,000 images, capable of detecting customer demographics … using facial feature recognition. As a result, Coca-Cola’s marketing team is able to capture and evaluate demographic details of customers, which are then used to develop focused marketing strategies and launch potential individual products, consequently driving growth.”

A Quantiphi spokesperson, Hadley Mayes, said the partnership occurred in 2017 but that it was just an experiment. Quantiphi later removed the case study from its website after Fortune inquired about it, saying it did not reflect the company’s current capabilities. Mayes declined to provide any more details on the initiative, such as where it took place or whether it was in a public setting, citing a nondisclosure agreement.

Today, thousands of Coca-Cola Freestyle machines remain in public view with a clear slot for a camera and some questions about how the company has used the cameras to date. If you have any info, this reporter would very much love to learn more.

And with that, here’s what else is happening in tech news today.

Jason Del Rey

Want to send thoughts or suggestions to Data Sheet? Drop a line here.

The rest of today’s Data Sheet was written by Alexei Oreskovic.

This story was originally featured on Fortune.com

Advertisement