I hope that by going through this small project, you will learn:
You can also "Remix" any of the code samples you see throughout the project by using the "Remix to Edit" button you'll see next to them. This lets you make a copy of my code that you can edit, build upon, and make your own. You can also always copy and paste my code into your own project..
To save your Glitch project so you can keep working on it whenever you want, you have to sign in to Glitch with a free account. The sign in button is on the top right of Glitch.
Reminder: Have a question or idea? Check out the Frame Academy Community on Discord.
This project will build on the immersive gallery we worked on in Project #4, so the starter project might look familiar. We're going to add to it throughout this project by incorporating sounds and videos into the gallery.
There are lots of reasons to bring sound and video into immersive projects. Sounds can contribute a lot to the ambiance of a space, and video is a powerful medium for communicating all manner of things. If you're making a museum, you can use sounds to add audio captions to items, or if you're making a training experience, you can use audio or video to add helpful information for the user. Or, maybe you'd like to add some "click" noises to a user interface to help users know when they've clicked on something. Those are just a few possibilities.
In the Starter Project above, I've already imported a few files into the Glitch assets folder that you can find in the sidebar on the left. I dragged in an audio file (a .mp3) and two videos (.mp4 files). This way, the files are hosted online and can be referenced with a URL. You can click on a file that's been uploaded to the Glitch assets folder to find its URL.
I'm going to encourage you to bring your own files into the project in the challenge below, so feel free to get a head start by dragging your own files into the assets folder of your own project.
- For audio, use .mp3 or .wav files
- For video, use .mp4
- For 360 video, use equirectangular .mp4
Here's how to copy an asset's URL once it has been uploaded to the Glitch assets folder.
Now that you have URLs for your files, you can preload the files using the Asset Management System that is part of A-Frame. I went over the Asset Management system in this part of Project 3, and if you need a refresher on that you should review it. To quickly sum it up, though, inside the a-assets element in your HTML, you can preload assets so that they will load and cache in the browser as soon as your WebXR site loads, improving the performance of your site and preventing lag as the user experiences your site. In previous projects, we preloaded some images and 3D models, but now on lines 23-35 of the starter project you'll see how I preloaded the audio and video. I know the URLS that Glitch generates for the assets are ugly and long- don't be scared of them:
<audio id="song" src="https://cdn.glitch.com/96ab1f6e-16b9-43c1-9df3-fbe9b24a7490%2FLeft%20and%20Leaving.mp3?1549919550870" preload="auto"></audio>
<video id="videoclip" autoplay loop="true" src="https://cdn.glitch.com/96ab1f6e-16b9-43c1-9df3-fbe9b24a7490%2FTimelapse%20Video.mp4?1549921266764"></video>
<video id="spherevidclip" autoplay loop="true" src="https://cdn.glitch.com/96ab1f6e-16b9-43c1-9df3-fbe9b24a7490%2F360%20Timelapse.mp4?1549922819582"> </video>
You use an audio element for audio and a video element for video. Be sure to give these assets an id attribute so that you have a way to reference them throughout your HTML. For the src attribute, you put the link for the asset that you can find in the Glitch assets folder by clicking on your asset and clicking the "copy link" button.
Remember, preloading assets inside of a-assets doesn't actually make the assets play or show up in your scene. It simply preloads them so that they are readily available and stored in the browser. We'll see how to actually make use of these assets throughout the project.
We're going to start with sound. The .mp3 asset I imported and preloaded is an instrumental song I wrote, and in this example I'd like to make a music player that lets the user play, pause, or fully stop the song. By going through this example, you'll have the tools you need to use audio in your project.
First, I want to make a user interface for this so that the user has buttons in the gallery to press that play, pause, or stop my song. A user interface is the way that the user interacts with your site and uses its features. I've already built the visuals for this user interface in the starter project, and you can see how it looks by loading the project and looking at the wall on the left. Keep in mind that the buttons don't work just yet - I've only made the visuals.
You can see how I've set up the entities for this panel in the HTML file on lines 78-107. There is one parent entity with a lot of children entity nested inside of it that include the play, stop, and pause button and the text. Poke through it so you can see how I set it up.
Tip: User interface design is an art of its own, and I'll admit I'm not great at it! Look around at some sites or apps that you think are both beautiful and easy to navigate to get inspiration for making your own user interfaces. There are people who specialize in user interface design, and for good reason. It doesn't matter if your site has loads of sweet features if it's difficult to use them!
Each part of this project comes with a challenge. As you go through the challenges, you'll be building a WebXR site that will work on desktop, mobile, or VR!
For this challenge, start to make the starter project your own. Some ideas? Add your own images to the walls, swap mine out, bring a new photosphere in for the sky, bring in other 3D models, and arrange things however you want them. For a refresher on how to bring in those assets, check out Project 1. Start making a cool gallery about something you love, or an e-portfolio that shows off some of your work, or just an exploratory playground.