This billboard knows if you have a moustache or not.
Engineering senior and computer science major Cynthia Mai, and Engineering seniors and computer engineering majors Eric Berdinis and Jeff Kiske have designed a video billboard which tailors the advertisements it displays to the people walking by based on their age, gender, glasses and facial hair.
The team created this digital billboard screen, called “Advise,” for their senior design project — a capstone project for students who are pursuing a bachelor’s degree in Engineering. Students are supposed to apply what they have learned in class to create a device with real-world applications.
The team recognized from the start that billboard advertisement is ineffective if the demographic of people walking by would never buy the advertised product.
“Any time you are at a mall or an airport and you see these billboard ads, they’re not specific to who you are,” said Kiske. “Say … you’re a guy and you see an ad for a purse. It’s like, ‘why would I want to see that?’”
A Kinect — a motion-sensing device first made for Xbox gaming — on top of the billboard screen periodically takes photos of passerby. The billboard then crops out everything in the photo except for the people’s faces. Using a database called Betaface, it uses the proportions of people’s faces to determine their age and gender.
“Little kids tend to have larger eyes compared to the rest of their face,” Kiske said as an example of what conclusions the machine uses from Betaface.
Advise also determines if the people are wearing glasses or have beards — a demographic that a glasses store or razor brand might be interested in targeting.
“We actually distinguish between moustache and beard but for the demo, we’re only showing it as one metric,” Berdinis said.
The billboard can either target one person who is looking directly at the advertisement or find the average characteristics of a crowd and target an ad to them — changing what’s displayed onscreen.
For example, “If it’s a lot of 30-year-olds but its both male and female, we’d show an ad like Toyota, which is gender neutral,” Berdinis said.
In the future, people may be able to use a credit card to buy the product from the billboard, Berdinis predicts.
Though a few other engineers have developed similar systems, the team does not know of any device currently on the market that uses facial recognition to target billboard advertising.
“Online, there are systems like Google and Facebook which track based on … your age and gender and where you are from and whatever websites you have visited, but there is no real corollary to that in the real world,” Kiske explained.
According to Berdinis, online advertisers use a method called AB testing to test and improve their advertisement. Based on how many people click on various versions of the same advertisement, they edit the advertisement. Advise will do something similar in that it will compile metrics about which demographics look at an advertisement longest.
“If an advertiser thinks that [its advertisement] should be targeted at a 40-year-old but we find that 18-year-olds are looking at the ad longer, we can tell the advertiser … ‘you’re targeting the wrong people,’” Kiske said.
The owners of the billboard can also use the metrics to figure out who passes that billboard the most often.
“Say someone wants to put an ad up for an 18-year-old and it’s at a mall, and thousands of 18-year-olds pass through every day — they can charge more for it,” said Berdinis. “That way display owners can charge more appropriately for the display.”
The team admits that passersby might be worried about their privacy. For that reason the device destroys all of the periodically taken photos.
“Anything you are outwardly showing to the public, we think is ok to use to target an ad,” said Kiske. “Any deeper connections, like your Facebook profile, we’re staying away from. We can’t tie your face to your statistics that we got back. Nobody’s information is kept.”
Furthermore, after talking to people at the CIS Senior Design Poster Contest, the team wondered if the product might be more popular in Asia.
“Apparently they do something in Japan with a vending machine that guesses what kind of soda you might like. We think it might be more culturally accepted there,” said Kiske.
The team took third place in the Computer Science Senior Design Fair. They will be competing against teams from all engineering departments this Thursday, April 25, at the CIS Senior Design Poster Contest at the Wu and Chen Auditorium in Levine Hall.
Please note All comments are eligible for publication in The Daily Pennsylvanian.