Skip to Content

Facebook’s new creepy VR tech uses your ENTIRE face — You have to see this

In case you aren’t aware of the current innovations with VR or Virtual Reality technology, it’s changing, and fast. One-on-one conversations have been evolving with the advent of new technology from the days of typing, video and now with VR.

We’ve seen VR tech with Oculus Rift, video games, apps and even our television series such as Star Trek for years now. It’s a bit like talking to someone in a Star Trek holodeck, where participants could have the ability to hang out in simulated environments like they were actually there.

We’ve previously seen VR tech where you could control another person’s body, or swim in the ocean or even fly with cheap VR glasses.

Facebook’s engineers have recently released information and footage surrounding their latest creation of VR-based social connection capabilities. They believe that by having incredibly accurate virtual representations of ourselves – virtual avatars – will be a game changer. But it’s also a bit creepy, when you see full-face representations of people.

What’s the goal?

What if you wanted to talk with someone and actually see them as if they were right in front of you? That’s a question the Director of Research at Facebook Reality Labs in Pittsburgh Yaser Sheikh looks to answer.

“Most of us, myself included, don’t live in the places where we grew up.” He says, “I’ve spent my life moving from city to city, and each time, I’ve left relationships that are important to me.”

Facebook Reality Lab states that the goal is to overcome the challenges of the actual distance between people and systems. Facebook is utilizing new groundbreaking 3D capture technology and AI systems with their Codec Avatars to address that issue.

(Courtesy Facebook Engineering)

What is Facebook’s Codec Avatars project?

Codec Avatars is Facebook’s project for creating lifelike virtual avatars of the users in the digital meeting grounds. The plan is that it would help in social connections because the faces would be very natural and common to those in real life.

The technology combines cutting-edge graphics and advanced motion tracking along with natural interpretations with people in virtual reality as if someone is right in front of you. The challenge lies in creating authentic interactions in artificial environments.

How does it work?

Human expressions are measured at the Codec Avatars’ Pittsburgh lab using 180 cameras and 500 lights. Various encoders uses cameras and microphones on a headset to capture what the subject is doing and where.

The decoders translate those into audio and video signals to the recipient. The codec then creates a database of physical traits to create a way for future consumers to do all of this without a capture studio and extensive data.

The result is creepily accurate. Don’t believe me? Take a look below.

(Courtesy Facebook Engineering)

What to take away from this

Is this a little creepy seeing someone as if they are right in front of you? I would have to say so. While the hope is to give those using this new technology the ability to have a social presence using lifelike avatars and connecting with people anywhere, there are security concerns.

One particular concern in recent years is deepfakes, which makes it look like people – celebrities now, mostly – are doing things they never did. As technology gets better, we’ll see more of it, and it’ll be harder to tell it’s faked.

But do you trust Facebook?

OK this is all cool, if a bit creepy. But the creepiest part is that it’s Facebook that’s capturing information about, of all things, your face. We all know Facebook’s track record with keeping our information safe – it’s bad. Real bad.

Do you really want Facebook taking data of your face, at 180 gigs per second, to be stored God knows where?

When many of us signed up for Facebook, we gave them our phone number to prevent hackers from being able to access our account information. It’s called two-factor authentication and Facebook used our phone numbers to allow people to look us up and even sold that info to third parties. Kim looks at how this practice is still in use even thought Facebook said they stopped it quite some time ago.

Stop robocalls once and for all

Robocalls are not only annoying, but they scam Americans out of millions every year. Learn Kim's tricks for stopping them for good in this handy guide.

Get the eBook