Frame Update: Apple Vision Pro Support, 3D Model Generation, and More

Last month I reflected a bit on my journey so far with Frame and some new directions we'll be taking the product. You can read about that here in case you missed it.

Frame is the easiest way to come together to collaborate and create with others in a customizable 3D space, right from the browser. Social, spatial computing on desktop, mobile, and immersive hardware. Frame is great for small groups but we can also handle events with thousands of simultaneoud users.

In this update, we launch basic support for the Apple Vision Pro, AI-powered 3D model generation, a gorgeous new environment, a simple interface for those on mobile, and some new camera features. Let's dive in!

Apple Vision Pro Support

In June of last year I wrote an article called "Apple and the Spatial Web: A Winding Path Towards a Bright Future". While I still maintain that the future looks bright, the path was windier than I anticipated. I expect more twists and turns.

Back then, I thought Frame would "just work" on Vision Pro when it launched. As it turned out, Safari on the Vision Pro had a few quirks (to put it politely) when it launched. Some of those have since been resolved, and you can now have a basic, immersive Frame experience in the Vision Pro!

To try it out, go into your Safari settings on AVP and find the WebXR feature flag. Make sure it's turned on. Then, navigate to a Frame link in Safari. Get yourself logged in, and make sure in your Profile in Frame that you have the experimental hand-tracking feature enabled.

You should see a little headset icon in the lower right corner. Click on that, and you'll enter immersive mode. Moving around is still a little bit strange. You have to look where you want to move and then "click" using your fingers and hold the click for a few seconds. You'll see the teleport beam extend and after a few seconds, you'll teleport where you're looking.

I wouldn't yet try to use the VR menu because we still have to fine-tune it to the general input system For simply getting into a meeting and talking in immersive mode, you're all set. If you want to live on the wild side you can try opening the watch menu, turning on edit mode, and dragging assets around or resizing them.

The Oculus Browser on the Meta Quest is currently lapping Safari without breaking a sweat, so here's hoping Apple continues to treat the immersive web seriously. Signs suggest they will!

PS: this support was made possible not just by Apple's work on Safari for the VIsion Pro, but also by a huge update to the framework we use to build Frame, Babylon.js.

If you'd like to hear more about Babylon and the Babylon 7 update, we're holding a Babylon Meetup in Frame on April 12. You can add it to your calendar here.

3D Model Generation and "Wild Card" Prompting

We think that AI has the power to supercharge creativity and productivity, and we think a 3D canvas is the best place for people to leverage AI together. In the post linked at the top of this article, you can read about how and why I'm making a dramatic pivot towards AI in Frame. This update has a few of the early building blocks towards the broader goal of making Frame the best place for human + AI teams to work together.

You can now generate 3D models using text prompts or image references. We're going to gently roll this out, so you can create five per day (for now). If you like one of the models it makes, you can "refine it" which will make a higher quality version of it. You get one refinement per day. We'll open this up more in the weeks ahead.

When you generate one of these models, we also add it to a shared library that all users can browse and import from.

You should also try out our "wild card" prompt input where you can prompt a wide variety of things into existence. Take a look at some of the possibilities:

Note: Wild Card prompting is highly experimental. We're working actively on it. If your mind isn't blown by it yet, we think it will be.

Neo Office

There's a spacious new environment called the Neo Office! Explore it - you can really get lost in there.

We built it for a particularly large team that uses Frame and simply needed more room than they could find in our existing office environments. If they outgrow this one, we'll need even more digital lumber. As always, our awesome artists continue to push the boundaries of what can be achieved right in the web browser. This environment loads just fine even from a mobile device.

Quick Mode

For those on mobile who just want to hear what's going on and talk, we've revamped our simple, 2D interface for Frame. Any Frame can be loaded this way - just put "quick." in front of your normal Frame url. So quick.framevr.io/gabe instead of framevr.io/gabe

Those who use this interface can still see streaming screens, look at any shared pdfs, and navigate between voice zones. Nice.

Camera Tweaks

We get it - sometimes you just want to sit in the back row of the Atlas Hall auditorium environment. There's a new "Zoom to Stage" button you can find in the toolbar that will zoom your camera to the stage to give you a great view no matter how far back you are.

Also, if you're walking around the Zen Office in third-person camera mode and you walk into someone's office, we switch your camera to first-person camera mode automatically to give you a better viewing angle.

Next Up

The next time we do a big Frame release, you'll get a glimpse at some of the new places we're taking the product. Stay tuned!