my role

Project Video

About Electric Graffiti

The team of us, from NUS and NTU, arrived at an opportunity to work with Disney+ on their Ms Marvel launch event at TenSquare thanks to Akbar Yunus and Gary Hong. The brief was to create and deploy an interactive installation to draw attendees into the event space for the Cosmic Block party.

I was part of the NUS team of people who mainly did the tech design and implementation, while the art and assets were custom-made by the team of NTU students. Together, the team built an interactive, responsive graffiti wall for an outdoor event that garnered over 400 attendees over the weekend.

event, space and concepts

To begin, we had a quick recce to where the installation would be deployed. The main things we were thinking of at this point were key considerations like weather (as it was outdoors) the reflections of the cars passing by and of course, the overall ambience of the piece. These LED panels were right across the road’s intersection and having too many bright flashing lights here would cause a significant safety hazard for the road. We also took key measurements of the space to understand the scale of the whole project.

As conversations took place, and the rest of the event’s plan formed, we could develop some fleshed-out concepts. The concepts ranged from electronic graffiti to a platformer where you play as Ms. Marvel. At this initial stage, things that mattered more were the concept and the relevance to the project – rather than how/what tech could be used. As you know, the final concept that was chosen was the electronic graffiti wall.

initial benchmarking

We started running and fumbling around trying to figure out ways to realize the concept. A big help to get us started was the Tel Imagarium in NUS. They had hacked an HTC Vive to make it into a graffiti simulator. An ingenious solution…or so we thought.

This implementation fell apart for our applications for the following reasons. Firstly, the spray cans themselves have embedded HTC Vive trackers. We came to realize that the system can only operate up to two such cans by default, and up to four with tinkering – it wasn’t designed to work with more than one person at hand. We needed to be able to at least accommodate 5 people or cans drawing at the same time with the estimate of over 200 people showing up to the event.

Secondly, by lugging this whole Vive setup outdoors, we realized one major issue. These trackers use infrared. Do you know what else throws infrared around outdoors? The Sun. To add the final nail in the coffin, these individual spray cans consist of expensive ($200 plus) Vive parts. Do we want our users to be handling those parts? What if they were to break on accident?

Not all hope was lost though, as a last-case back up, we still knew that using a webcam and PoseNet was a viable option for the project – but that wouldn’t be a perfect solution, so we kept researching.

test and prototypes

Over the next few weeks, we researched and tested various methods to get the position inputs. Using phones, or even microbits embedded in the spray cans was considered. At one point we were thinking of using QR codes or aruco markers on cans to track the movement of the cans.

Out of luck, however, we stumbled upon compact infrared lidars. Honestly, “it’s super overkill”, as the guys over from Untitled Singapore said when we shared the makings of this project with them – but hey, the reliability that came with the cost was worth it for this project. While thinking of ways to mount the sensors to the area, we realized that the roof was made out of tin… so we 3d printed a mount for the lidar and attached it to the roof painlessly using magnets!

software and deployment

On the software side, we went through over 10 revisions over a 2-week timespan, checking for usability and adding more features. The entire thing, being coded and run in a web browser using JS meant that we did not need such a powerful computer as compared to the initial Vive setup that ran in Unity. The final version of the web app which could handle interactions from up to 10 people concurrently (on a 7m x 2m screen), was all running on a laptop hooked behind the screen.

In the final weeks, we added features to select rainbow brushes by hovering your hands on the screen, as well as special icons/stencils that we related to Ms. Marvel. The stencils were a good addition for people who didn’t know what/how to draw, while the more ambitious artists just went ham with the spray brushes.

It was also at this point that we added a feature to make the paint fade away over time. Through testing it became very obvious that people did not want to draw over other people’s work…which soon resulted in a static wall filled with colour – that no one wanted to overwrite.

the finale

What resulted was an event filled with colour and interactivity thanks to Disney+, Island Boys, the other vendors, and of course, the performers. it was a project that was in no way easy to accomplish – even if the concept was approved a couple of months in advance, we had hit and broken through) many walls of challenges. It also required more work to go down and test everything at the location – as we realised that the oncoming lights from the traffic, as well as the ambient lighting, played a huge part in the experience of the place. This meant that we had to go down to the location and fine-tune the brightness and colours of the individual elements and find the sweet spot that worked both with the 6 pm sunlight, and the 9 pm moonlight.

All in all, it was heartwarming to see the months of work coming to fruition as the veil of the event lifted, and the crowds started rolling in. Definitely shagged out, but 10/10 will do it again!