Google has its eyes locked on a bunch of futuristicadvancements in AI, most of which were discussed atGoogle I/Oearlier this month. However, for exactly two years now, the company has been teasing a technology calledProject Starline, which could revolutionize video calling as we know it. It could make virtual interaction feel lifelike, even through a flat display, without the use of 3D glasses or VR headsets. Google recently allowed YouTuber Marques Brownlee to experience Starline firsthand.
Even today, conventional cameras struggle to recreate the perception of depth the naked eye can see. Combine that with the camera’s awkward placement on most phones, tablets, and laptops, and you may’t even maintain eye contact in video calls. People have attempted stereoscopic imaging using two or more camera lenses for decades, but none of them became mainstream.
Project Starline is Google’s attempt at redoing stereoscopic imaging for one-on-one video interactions, with the help of modern tech like computer vision, machine learning, body tracking, and spatial audio. The result is a convincing illusion that you’re sharing the same space with another person, albeit through a screen and over the internet, but with no glasses or headsets involved.
Tech YouTuber Marques Brownlee (of MKBHD fame) is one of the first few people outside of Google to film a hands-on experience with Project Starline. When Starline broke cover, it required a dedicated room full of computers, cameras, depth sensors, and a giant display to create the immersive effect. Googlerecently unveiled a compact unitcomprising a 65-inch display, which Marques and his team got to test out.
The new design removes depth sensors and uses the power of AI to create depth maps of the person from plain old camera data. The depth map combined with the head tracking creates a parallax effect of sorts, where you can look around the on-screen person by moving your head. What’s more impressive is the real-time plain white background rendered in, complete with your shadows on the virtual wall behind you. Spatial audio seals the deal, making sound seem lifelike.
However, there are a few limitations. Segments of POV footage in this MKBHD video were shot using a single camera, creating a flat image lacking depth, like we mentioned at the outset. Additionally, glitchy artifacts are seen outlining the subject throughout the video, much like virtual backgrounds on Google Meet calls.
It’s worth noting that Project Starline only works for one-on-one interaction at the moment. Hopefully, regular people get to try this tech out in person sometime soon, even though it may not be viable for daily life, at least in its current form.
