Beem, an app that lets you live-stream yourself in Augmented Reality.

What’s the next step beyond Zoom calls and FaceTime? How about beaming yourself from one device to another in real-time using augmented reality? That’s the premise behind a startup called Beem, which is today announcing its first consumer app, $4 million in seed funding, and its longer-term plan to become a communications technology for the AR glasses of the future.

Consumers today know AR technology thanks to interactions with Snapchat Lenses and TikTok Effects, through mobile games like Pokémon GO, and by visualizing products they’re considering buying — like furniture they place in their room, or makeup they virtually try on via an AR filter. But using AR for telepresence, as Beem intends, isn’t a common use case.

Surprisingly, the idea for the startup comes from a founder who grew up without much access to technology. Beem CEO Janosch Amstutz describes his parents as “hippies” who lived in a small Australian beachside town, Byron Bay, where they raised cows and chickens, used solar power, collected rainwater, and didn’t even have a telephone — just a two-way radio that the community would share.

“We put a very big emphasis on face-to-face communication,” he explains of his alternative childhood environment’s lack of modern technology.

Meanwhile, communications continued to evolve from home telephones to cell phones to Skype. “But then we kind of stopped,” says Amstutz. “The premise of Beem is that there’s going to be an inevitable next step in the way that humans communicate digitally, which is more credible and more immersive than doing a video call.”

Amstutz initially rebelled against his upbringing by working in physical commodities trading in the steel industry. But he didn’t find the work fulfilling. He eventually realized he wanted to do something else — and specifically, he wanted to tackle the problem of evolving modern communication.

The founder put together a team of computer scientists and researchers to work on the concept, which today includes Chief of Product Damian Hickey, previously product head at AR pioneer Blippar; and Denis Islamov, co-founder and CTO, whose background is in applied mathematics and physics.

After its official founding in 2017, Beem (previously HoloMe) did not fully target the consumer market. Instead, it developed AR projects and campaigns for brands, businesses, and other organizations, including Vogue, Carolina Herrera, Hermes, Louis Vuitton, H&M, Forever 21, Warner Music, LADBible (in partnership with KSI and Craig David), the British Army, TEDx, and several others.

During this time, Beem’s technology was used to stream an artist’s music in AR, deliver a live conference to users’ homes in AR, create “virtual catwalks” and create other sorts of AR experiences for its clients.

This experience allowed Beem to stay afloat while testing and furthering its technology, but the company never intended to be a development studio, says Amstutz — it wanted to deliver a consumer experience.

With the Beem mobile app, which soft-launched in its latest iteration last month, the goal is to put this sort of live AR — or 3D-like experience, if you prefer — into consumers’ hands.

To work, users download the Beem app for iOS or Android and position themselves in the camera’s viewfinder either by mounting the phone to capture their full-frame or by having a friend record them. Beem takes the video, segments the human in the video away from the background using its proprietary computer vision algorithms and cloud infrastructure, processes the asset in real-time, and packages it for the viewer.

The recipient receives the link, which they open in their mobile browser where they’re directed to a microsite that’s spun up for each different web AR-style interaction, then spun back down when the communication is finished. Here, Beem accesses the phone’s accelerometer, which allows the calculations on the server to skew and morph the video feed of the sender in your own space to give you the illusion of three-dimensionality.

The recipient holds their phone up and places the “live beamed” person in their own space by tapping a dot on the floor — much as you do with other AR tools.

The end result is what appears to be a sort of hologram of the person in your room, talking to you in real-time. You can pinch and zoom on the person and move them around, but you can’t view them from all sides, since that wasn’t captured in the original video. The quality is not as good as, say, that cute AR “green guy” now trending on TikTok or Google Search’s AR animals, but it’s clever because it’s live.

“We don’t need to be super technical…We need the mind of the viewer to believe what they’re seeing is a real human and that’s enough,” says Amstutz.

Beem users can use the app to host “conferences,” like AR Zoom calls, where they stream the holographic version of themselves up to 25 people with two-way audio. They can also launch one-way live streams to a larger audience, or send pre-recorded video messages.

Source: Techcrunch.com