I see Unicorns and Rainbows!

What an incredibly cute swag, right? But how did it turn into a winning augmented reality (AR) hack at the recent Wallifornia Music Festival Hackathon in Liège, Belgium? Attribute it to the uncomfortable silence in the room.

At that hackathon, the room was full and several participants pitched their project ideas to the crowd. When the pitching ended, the room fell silent. It was that slightly uncomfortable moment when a roomful of smart developers go dead silent, and everyone is wondering, “Is that all?” I’d been to many hackathons and knew only too well that when such silence descends, someone had better remind the participants that hackathons are about having fun.

At that moment, the room was just too serious for a music tech hackathon.

I grabbed Cloudinary’s übercool unicorn plush toy, stepped up to the stage, and offered a silly idea pitch: “I have an idea! Build a hack that uses this unicorn. Make it animated, do something interesting with AR and Cloudinary. Surprise us!”

The room filled with laughter.

That broke the ice, and several other participants made their way to the stage to pitch far more interesting ideas … or so I thought.

About an hour later, a young developer, Kiara Brückner, stepped up to our booth and asked if she could have a dozen plush unicorns.

Uniscore Team

She explained that she and her teammates — her father, Tom Brückner, along with Luis Feres Molina and Roald Baudoux — wanted to use the unicorn for their hack project and that they planned to hand out unicorns to the audience for the project’s interactive experience. We had plenty of swag and, frankly, I knew that our new evangelist, Tamas Piros, wouldn’t be thrilled with having to carry leftover swag on the plane home to Budapest. So, I was happy to oblige: “No problem, take as many as you’d like.”

A bit later, Tom showed us a demo of the first iteration of their project and concept, which they cleverly called Uniscore: Cheering for Introverts.

The main premise was to empower introverts to participate in a meaningful way during live music performances. As Tom described it: “It’s hard for live artists to get whoops from introverted people, who typically just stand around in the audience but don’t cheer like the extroverted ones. To give introverts a way to show their appreciation, a cool stuffed toy, along with an AR app and a stage display, gamely rises to the occasion.” Ingenious!

What Does Uniscore Do?

Uniscore is a project developed to enrich the experience of the audience at concerts, especially for introverted people.

The people can give feedback to the musicians using their smartphone and a unicorn plush with a specific tag. The smartphone’s camera is directed to the plush and when it is detected, a halo appears around it. People can show their appreciation by raising up the plush toy more or less in front of the camera. As they do that a rainbow grows in the bottom of the screen. When the rainbow is complete with all its colors, a running unicorn runs over the rainbow.

Simultaneously, some control messages are sent back to the musician to control an additional track (in this case it triggers the sound of a running unicorn). Some markers in the musician’s software also triggers the apparition of specific pictures related to the universe of magic and fairies over the unicorn.

The Uniscore team used Cloudiary’s plushy unicorn, Unity and Vuforia for the AR app, NodeJS for the server backend, and Reaper + Reactor Blocks for the live music. They had planned to use Philips Hue API to control special effects lighting and additional Cloudinary APIs for delivering the product merchandising images within the app. Finicky hardware and time constraints pushed those ideas off the table for making it into the teams minimum viable product (MVP) used to demonstrate the idea.

The team is really proud of making some nice music for the demo. They were able to develop a functional AR setup and submitted the app to the Google Play Store and launch before the end of the hackathon. They had the iOS app is ready, as well, and that’s pending Apple’s review process. Kiara created great drawings for the augmented reality experience.

It’s great to see a father-daughter team take such a silly concept and turn it into something fun and interesting. The judges and Cloudinary were impressed by the app experience for concert goers and the potential for marketability of cute concert swag and related products.

Cloudinary Unicorn

In fact, as AR concert experiences continue to emerge in the music industry, their idea to tie concert swag in with AR is as brilliant as those beautiful dancing unicorns and rainbows drawn by our young artist and developer.

So next time you go to a concert and see a AR experience, remember Wallifornia… and those unicorns and rainbows.

I see Unicorns and Rainbows! was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.