Tags: augmented reality, iOS, virtual reality
Mobile applications industry delivers good news: augmented reality (AR), voice control, virtual reality (VR) and other amazing technologies become more and more popular. More importantly, modern smartphones have enough power to process the graphics for AR and VR. You can simply insert your smartphone into an affordable VR headset in order to dive into virtual reality. So, the most part of smartphone users now can access this variety of modern breathtaking technologies. Moreover, these technologies are not expensive and they do not require extra power from a mobile device.
To see how the DB Best powered AR & VR application works, please, watch the following video.
We’re confident that you’re already wondering how you can use these incredible technologies. DB Best can work with your company to create a similar application for your business in a very short time frame. Contact us now to learn how we can bring your ideas to life and implement them in an AR & VR application for your business.
Read below for more details on how this application prototype works.
Background on Augmented Reality technology
Augmented Reality (AR) is all about adding virtual objects to the real-world environment. A typical example of implementation of the augmented reality is when a driver sees the projection of the current velocity on the windscreen of his car. Another cool AR example is when you see the translated text on the smartphone’s screen instead of the original text simply using the Google Translator application. Generally, the AR technology has a pretty wide scope of use: education, games, navigation, construction, interior design, etc.
What does augmented reality actually means and how it works in particular cases? The DB Best powered application integrates the selected 3D model of an object into the live picture from the camera. So, the user can see a combination of a real-world picture from camera and the virtual object on the smartphone’s screen.
The virtual objects behavior and the way they fit into to the real-world environment may depend on the actual picture from the camera (for example, the size of the real objects may be considered). We use the voice commands recognition and the color marker motion detection to control the position of the virtual objects on the screen. You can find more information about how we actually do that in the corresponding paragraph.
Background on Virtual Reality technology
Virtual Reality (VR) is a realistic and immersive simulation of a 3D environment. Usually, the developers use the 360-degree panoramic content to simulate the virtual environment.
Using a VR headset is the easiest way to experience the virtual reality in action. The virtual reality headsets can be divided into 2 types. Some of them have built-in screens and can work as an autonomous device (Oculus Rift, HTC Vive). And the VR headsets of the second type require connecting a smartphone because they use its screen to display the image in the side by side format (affordable and accessible Google Cardboard).
In addition to a compatible device, you need the appropriate software. DB Best developers are top-class experts in this segment. To prove this, let’s discover the capabilities of a DB Best powered application, which combines both augmented reality and virtual reality.
DB Best created this application for iOS devices. To dive into the virtual reality, you have to place the smartphone into a VR headset (we use a two-dollar Google Cardboard). Then the application transforms the picture on the iPhone’s screen into the stereoscopic format and splits it into 2 square images, placed side by side. The application renders AR image as a side by side stereo picture. In the above-mentioned video, you can see the split side by side picture at the top right corner of the video. We use the mirroring of the iPhone’s screen to demonstrate how the technology works.
The virtual reality technology can totally replace the real-world environment with a virtual one. In our case, we use the combination of AR and VR technologies to build an image based on the real-world environment. To do so, we duplicate the image from the smartphone’s camera, so that the user can see the real objects in virtual reality glasses. Some people call this approach Mediated Reality, but we prefer to define it as a symbiosis of two technologies (AR & VR).
Application features
So, the DB Best powered application uses the iPhone’s camera to capture the real-world environment and adds the 3D model of an object at the predetermined location. The application supports any 3D models in .obj or .dae format. We use the 3D models of a car, a digital camera or planets of the solar system. Then, the application renders the AR image into a stereoscopic format and displays it on the smartphone’s screen.
That all sounds pretty simple, but, in fact, some very complex algorithms stand behind this task, and our development team did a great job to achieve the desired refresh rate (which is 60 frames per second) regardless of the smartphone’s internal power. Of course, we had to compromise the image quality of older iPhones to achieve this result.
So, now we know all about how the application works to display an AR picture in the VR headset. Actually, this is only the first step and an easy task. On the next step, the application must be able to control the position of a 3D model. And this one seems to be a much harder task. Since the user cannot use the touchscreen of the smartphone, installed in the VR headset, we decided to provide 2 control options, which are actually used together. And we’re talking about:
- Voice recognition and
- Color human interface control recognition.
With such voice commands as resize, rotate and so on the user can select the working mode of the application. To control the actual 3D model’s position, the user can move the color marker in front of the smartphone’s camera. You can set the color of the marker directly in the application. Basically, we can detect the movement of the user’s hand to control the object, but the color marker looks much more elegant.
We use a specific AR marker to set the smartphone’s attitude. This marker can be printed or simply shown on the screen of a PC. Using a complex board marker provides much better user experience compared to using a single marker (for example, the stereoscopic image doesn’t tremble, which can be very noticeable in the VR glasses).
Application structure
Let’s discover the general application’s diagram.
The application constantly analyzes the data from smartphone’s camera, microphone, and other sensors (i.e. the gyroscope). Detecting the color marker and recognizing the voice commands, the application can determine the 3D model’s position. Then the application places the 3D object it into the predefined place in the live image from the camera. Later this AR image renders as a side by side stereo picture. And then the application displays it on the smartphone’s screen. Now the user can review the 3D virtual reality scene through the virtual reality glasses.
DB Best developers can customize this application prototype for any customer’s needs. If you’re considering the development of a cool mobile application, using the augmented reality or virtual reality technologies, be sure to contact DB Best. Our developers will implement your ideas into a jaw-dropping mobile application, and your clients will definitely be amazed and satisfied!
