Yooooooo, I’ve been waiting to write this post for over two years now.
I originally got into VR because of how immersive it is. And while it can transport you to an entirely different world, it’s still a very isolating medium. In most cases, everyone else in the room is completely clueless to what the person in VR is experiencing. I believe phone AR is an excellent way to change that.
Forever ago, you may have seen this tweet that I posted from the Normal twitter account:
In this video, I’m using a custom iPad app that can see into the VR space that Kirsty is in. This video was filmed in realtime on an iPad using ARKit and Normcore.
I love this concept because it’s an app that anyone can just pick up and use. Anyone in the room can grab their iOS or Android device, and use it as a magic window to see into the VR world.
You can see what it looks like behind the scenes here:
How it works
On paper, it’s actually pretty simple. We’re both running the same app. One on the VR headset, one on the iPad, and we’re using Normcore to synchronize the state of each copy of the app.
Unity makes it pretty easy to export one project that works across multiple platforms. In this case, once I had the app running on iOS, all I had to do was import Unity’s ARKit plugin, and use Normcore to synchronize the avatar and brush strokes.
That’s it :)
With everything set up, someone on a mobile device like an iPad can launch the app, connect to the same room as someone who’s running the app in VR and navigate around in the same virtual space. By far my favorite detail is that VR players see an avatar for people in AR. For our own apps, we’ve created a cute floating iPad with a Wanda inside. VR players can see where the AR players are in the virtual space and can even talk to them using voice chat as they show them around the virtual world :)
Want to try this in your own app? We’ve written up detailed instructions in our docs on how to set this up. Check it out here.
Max
