Note: This post was originally published on July 10th, 2017.
Over the past few days, I have been building a custom virtual reality paint application using the Unity 3D game engine. Much like Google’s Tilt Brush, I created an experience where users can control a virtual ‘pen’ via virtual controllers to draw in virtual space. They can change the color of the lines that are drawn, increase the size of the lines, and erase their 3D canvas if they wish to clean their space. A special shoutout to FusedVR’s video tutorial for walking me through the steps needed to create my own custom VR painting app. I used much what he described in that video and my own unique additions to make the app more user-friendly. In this post, I will discuss how this app works, and how I coded my own custom additions.
How It Works
Although the app itself looks complex, it was deceptively simple to build and just required a basic understanding of 3D geometry and the C# language. Do you see the green-colored lines coming out of my virtual ‘pen’ in the GIF above? Those visuals are composed of pairs of triangles that are connected to one another, creating the illusion of a solid line. These triangles then hold a specific material that can be changed by a color that the user picks via the color-picker. With some knowledge of the SteamVR plugin and hand controller setup, users now have the ability to draw custom lines in VR!
Using Unity’s Renderer’s
As mentioned above, drawing lines in a virtual space starts with drawing small triangles and connecting them together. In FusedVR’s tutorial video, there were using two separate methods of completing them action. The first was Unity’s built-in LineRenderer engine, and the second was a custom Mesh-Renderer script that manually stated how and where the triangles would be created.
Using Unity’s built in LineRenderer created lines out of sectioned triangles, but it did something funky when the user created these lines in VR; each of the newly created lines would automatically rotate to face the user as the user moved around the virtual space. Although this method would work perfectly for non-VR applications, another method had to be used for line-drawing to work in VR. The solution was to use a custom Mesh-Renderer script that created separate meshes that had properties independent of the Unity 3D engine, which was in contrast to the LineRender objects which were controlled and generated by Unity’s own scripts.
In addition to creating virtual lines, FusedVR’s tutorial also covered how to change the color of these lines. This was done by importing an external Unity plugin called “Simple Color Picker” (downloaded via the Unity asset store) and then dragging several objects from this plugin into the game scene. After adjusting some code so that the color picker recognized the raycasts of the VR controllers, users could directly pick the color of the line they wanted to draw.
My Own Additions:
Aside from a color picker, I thought it might also be valuable for the user to erase all current lines in the game scene as a way of ‘wiping’ their 3D canvas. I did this by writing some code that gave all newly drawn lines the object name “line” (original, right?), then wrote code that told Unity to delete all the objects that had the name “line” when the ‘grip’ button of the right controller was pressed. This effectively wiped the canvas of all drawn objects.
I also wrote custom code that increased or decreased the width of the line that was drawn. Using the y-values on the Vive controller’s touchpad, I could increase the mesh width when the top of the touchpad is pressed, and decrease of the mesh with when the bottom of the touchpad is pressed. Simply touching the touchpad would not yield any results.
I am always willing to share my knowledge and help others, so you can find the complete Unity package of my project here. Keep in mind you’ll need an HTC Vive, a somewhat VR-ready computer, and SteamVR!
Let me know if you have any questions in the comments below, and see you in the next post!
