Reflecting on My First Ever Reality Virtually Hackathon
Note: This article was first published on March 17th, 2018
Earlier this month, I attended the Reality Virtually Hackathon, an event hosted by MIT’s Media Lab to foster creativity and innovation within the global VR/AR community. I was lucky enough to be one of 400 participants selected out of over 1,500 applicants, and this event marked my first ever hackathon. The event was spread over 4 separate days, where the first day allowed participants to attend workshops and form teams, the next two days allotted for “hacking”, while the final day was reserved for showcasing team projects as well as an award ceremony which rewarded those with the best ideas.
I chose to compete in the category “VR/AR for Good” as a way of getting out of my comfort zone and meeting people who didn’t think in the constructs of game mechanics. I ended up joining a team that had a goal of using augmented reality technology to tell the stories of domestic violence survivors. Below documents the journey of surviving my first ever hackathon:
45 Hours Left
The original idea by our team lead (the member who came up with the idea) was to tell the stories of survivors of domestic violence through the use of a MergeCube; a MergeCube is a small physical cube that fits in your hand and allows augmented reality experiences for anyone with a computer-powered camera. The MergeCube has specific patterns on its sides that allow cameras to detect the spatial placement and orientation of the cube in physical space. The proposed plan was to have users connect multiple MergeCubes together to piece the stories of these domestic violence survivors.
However, we soon found a few limitations with this approach: there can only be one instance of a MergeCube in view by a camera at a time, and the MergeCube API is currently tied to mobile platforms which could create potential performance issues. Although myself and the other programmer on the team pitched the idea of using holographic technology to project a cube into the hands of users, our team lead was adamant about having users hold a physical object. The team pitched other ideas but the fact we wanted users to touch and feel a physical object limited the number of options available. As a result, we spent about 8 hours discussing options, trying to force ourselves into using these cubes as a device to tell stories. Looking back at this moment, a better path may have been to search other approaches that accomplish the same goal of telling impactful stories and then choose the most viable/doable. Instead, we were caught up with the assumption that users had to hold something.
24 Hours Left
We had 24 hours left to complete our project and we still had not decided on an agreeable approach on which to build our experience. To make matters tenser, another very important question was brought up: who are we making this application for? We quickly realized that half of our team thought we were building an application for general audiences to bring awareness of domestic violence issues, while the other half thought this application was to be used as a mental health tool for survivors of domestic violence. This confusion led to even more discussion, and at times, very heated arguments about how we should build the application a certain way if we wanted any chance of winning. As a result, we were not functioning as a team and many interesting ideas for other approaches were shut down by the stubbornness of a few members who would not budge from their expectation of how the experience should be built.
After quite literally an entire day of discussion, we had finally decided that our project would be a virtual reality experience used by survivors of domestic violence to help in the healing process. We would have users pick up and interact with a broken vase and a phone as a mental exercise of healing with the aftermath of violence. We would also have voice-over throughout the experience to guide the user, and conclude the experience by transporting the user to a relaxing forest environment as a way of allowing them to meditate.
While everyone slept, I worked through the night to finish programming object interactions in VR, which would then leave myself open come morning to help build the environments for the experience. Although I had some trouble here and there with object snapping, object handling, and controlling object rigidbodies and trigger states, I managed to create a way for users to pick any two objects up and piece them together. However, I was a bit disappointed that I didn’t get a chance to learn something new. Sure programming these interactions were new to me, but I joined this hackathon with the purpose of learning or working on something that I was not comfortable with as a way of getting myself to think differently (i.e., AR holograms or using Vive Trackers). To play devil’s advocate, due to the whole team’s indecisiveness, we didn’t leave ourselves much time to learn new things, which is why I feel we had to focus on a technology our team was competent in.
10 Hours Left
We now had 10 hours left until our project was due and the only things built were the object interactions in VR. I and two other members rushed to create the environments for the experience while the other half of our team worked on the script for the voice-over that will be guiding the user through the experience. During this crunch time, I was heavily involved with the technical side of the project, but I had no idea what the members working on the script/voice over were thinking about or expecting the experience to be. This miscommunication reared its ugly head during a team meeting when we realized members of the team still had contrasting ideas about what the experience should be about, what the user will be doing, and how to properly guide the user to interact with specific objects. Reflecting upon this moment is quite painful because I should have taken it upon myself to connect with the whole team and get on the same page instead of working in a vacuum and later ending up doing more work.
The deadline to submit our project was getting closer, and the pieces of our project were slowly coming together. Our team was now on the same page about the beats our whole experience, and we were starting to work together as a team to create the best work possible. In fact, because everyone now understood the concepts that I programmed into the project, I personally found it easier to bounce off ideas with other members on how to polish up some interactions. I didn’t care if they were not programmers, but because they understood the current logic in how the experience was built, talking to other members brought about creative solutions to fix pesky bugs. It was in these moments when the whole team raced against the clock that we felt in tune with each other and being on the same page enabled us to think out loud and lean on each other for help.
2 Hours Left
The last hours of the hackathon were quite a blur. There so many unforeseen problems when testing out the experience, such as environment scale, lighting performance issues, and circumstances where objects fell out of the user’s play space. I fondly remember rushing to code the transitions of light and sound, as well as make sure the voice-over clips played correctly after specific object interactions. Dealing with triggers, timers, and a bit of headset hair I managed to get the experience up and running. I remember we had 30 minutes left to submit our project and I still could not get a specific scene transition to work that allowed the user to be transported to a calming forest environment. I spent 29 minutes trying to get SteamVR’s loading system to work, and at the last minute, I used Unity’s pre-packaged scene manager to change scenes. Without testing, I wrote the final line of code and submitted the project via DevPost. Phew!
Judgement Hour
With the project submitted, I was extremely relieved and proud to have survived my first hackathon, but I was still extremely worried that my last minute changes and lack of immediate testing to see if these changes worked would crash the experience and reduce all our hard work to an endless loading screen. After the project deadline we had about an hour before judges would grade our project, and in that time we ran into a number of issues with the hardware that powered the VR headset. Subsequently, we could not test the experience to make sure everything was running smoothly, and just as we got things to work, it was judging time.
The judges listened to our pitch and then tried our experience. The first judge was almost to the end of the experience and was about to be transported to the forest environment. It was the moment of truth. Then I heard them say “Wow”. They were successfully seeing the end environment and I could tell the look on their face was one of wonder. I jumped up in joy and looked at my teammates. We had pulled it off.
Demo Day & Conclusion
The next day we demoed our project to the public. Many were amazed at the atmosphere we created as well as the difficult issued we tackled. As with any demo, there were many technical issues that popped out of nowhere; interference with the Vive base stations, faulty connections between the VR headset and the PC hardware, and in-and-out tracking of the Vive controllers. But overall it was a great personal reward to show off something I was part of and put 110% towards.
Although we didn’t receive any awards for our work, participating in this hackathon taught me more about teamwork and team management than learning new programming skills. And that’s a good thing because it is something I didn’t expect to learn but was a by-product of being part of a tense, sticky situation. I am going to make more of an effort to join hackathon-like events, and I certainly look forward to next year’s Reality, Virtually Hackathon.
See you in the next post,
Tejas
