Development of our E-commerce Showcase

Article by Ronald Klarenbeek, Product Manager @ YOM

I’m delighted to see our most recent Ecommerce showcase experience (store.yom.ooo) has launched. As it displays several significant features, I’d like to spend a little time highlighting what’s going on behind the scenes. I will also share insights into our efforts getting to here and conclude with some feature perspectives.

Some weeks ago we decided we needed a good showcase environment where both the power of our pixel streaming platform as well as the extensive feature set of our YOM SDK would be put on display for anyone to experience first-hand, so that anyone can discover these new concepts that we are rapidly making available. The idea was to make this experience available on our website. This would help us during talks with potential partners but it would also be a tool for signed partners to help convince their customers (brands) of the possibilities of working on our platform.

We initially devised a “Holodeck” space for this — clean, sterile and with all our SDK features on display on a path to follow, kind of like how it was to walk around in the Portal game series. But after some days of working on this and getting our SDK features implemented we felt the case was simply not speaking enough and it would be more powerful to release something more close to an actual experience that could be launched by brands.

Holodeck 2 in development, early version

One of the nice things about working with Unreal Engine is that there is a large community and marketplace with existing content, plugins and other assets readily available for use. Using these sources, we had previously successfully released a simple tech showcase level on our homepage YOM SDK Portal Unreal Engine which was a jungle map including advanced lighting (dynamic shadows) and complex foliage. We decided to use the same approach for this showcase, starting off with a map readily available on the marketplace which had a style and aesthetic that we felt had a great fit with our desired experience.

Showcasing our platform capabilities in the YOM jungle

After downloading the level and installing our SDK into it, we deployed the level to our platform and were quite impressed with the result. We spent a little time enhancing the experience with some assets that we felt needed to showcase the capabilities of our platform, notably some NPC’s from various sources (a Metahuman, a VRChat-like avatar and a big robot). Then we noticed that in certain areas the performance was suboptimal so we did a deep dive of the performance. We found out that specifically the real time lighting and shadows were causing frame rate drops down to 20–30 FPS which is below our target of running 40 FPS minimal. So after disabling some of those dynamic shadows and using baked lighting instead, we noticed we could regain performance to our minimum acceptable level and carry on.

Playtesting the experience with 40 bots to check performance

We then added the ecommerce features: We added a pedestal with a 3d model of a black t-shirt which is something that we actually wear in the office and is up for physical sale. Interacting with the item will trigger a PayPal overlay which handles payment. Experiencing this first-hand helped me get even more convinced of the strong potential of shopping in this new channel.

Physical and virtual items for sale in our shop. Interacting will trigger PayPal or web3 wallet popups.

Of course we also had to add a web3-powered transaction so we added a cool hoverboard that, when interacted with, allows the visitor to mint the item straight into their web3 wallet as a virtual item NFT. All the virtual items in the visitor’s wallet are available for use in the experience — right now they can be dropped and have physics which can lead to hilarious results, depending on what is in your wallet.

Things get messy real fast when dropping items from certain web3 wallets in the experience

We then added another smaller shop with some plants (assets again from the Unreal marketplace) and made the plant in the center open a popup straight to Amazon. This showcases that using the capabilities of the web browser, it is also possible to refer to other websites or landing pages straight from inside an experience without breaking the “flow”.

Plant shop with direct deep link to existing ecommerce platforms

We then worked on showcasing gated access. We want to make it explicitly clear that you can define rooms or areas in an experience that can only be accessed by people that possess a certain item in their wallet. This could be a access card, a guest pass, a wristband or any other kind of item that provides access to gated experiences. This way it is possible to create exclusivity which brings value to those access items. We showcase this by having the user walk past a club blasting quite loud music (made by yours truly) but with doors closed. Next to the doors is a sign stating a specific item is required to enter. The item can be found elsewhere in the experience, which users will get airdropped to their wallet upon finding it. When in possession of said item, the doors will open and the user can walk in to the club only to find a little surprise there..

Club $YOM, only accessible with a certain item in your wallet. Try and find it!

This completed our initial set of goals to showcase. We then added the speaking robot (that is running a script generated with AI about the history of YOM and was synthesized to speech using Prime Voice AI). The end is the best so try listening to the whole thing — it is quite long! It showcases nicely how our platform does not have any limitation when it comes to file size of assets used. It could also have been a full movie on display, as we are employing pixel streaming there is no upfront download needed for it to work on the visitor’s machine.

Welcoming you to our experience, this robot has a lot to say but he won’t listen to you.

Finally we have integrated with emerging technology from convai that allows us to put an avatar in the experience that “listens” for voice input and actually responds speaking, using an AI model that we could “pre train” with some scripts as source of conversation. As our platform supports microphone input through the browser natively, this is a very direct way of getting these advanced AI-based avatars straight into people’s desktops. We first imagined setting up a separate space where a well-known philosopher like Plato would be available for deep AI-powered conversations but quickly abandoned the idea in favor of a virtual shop assistant. When playtesting this feature it took me about 5 seconds to forget about all the technology involved and end up simply having a fun conversation. I am very excited about the potential of this technology and I am happy to see it fits our platform so well.

Virtual shop assistant, listening for microphone input and talking back. Screenshots don’t do it justice.

So how much time did we spend on all this? Well, we are a startup. Our budgets and capacity are limited. It only took a small team with limited time to get to this point. We combined a lot of existing assets and technologies and made it fit to our own vision of what we wanted to achieve. I hope that this inspires others to also start experimenting with building their own experiences. I am very happy with the result we achieved, and I hope that lots of people will visit it and experience the amazing new opportunities that arise in this new channel.

Looking at today’s young generation and how they seamlessly switch from Roblox to Fortnite and understand the marketplaces and virtual items, realizing that these kids will grow older over time I strongly believe there will be a paradigm shift from current means of shopping on native phone apps like Zalando (basically swiping through the catalog) to more immersive ways of navigating a store. I can imagine going on a shopping spree in a virtual space together with a friend that could live hundreds of miles away. I also believe there are very strong use cases of this new technology for service and support as well as (mental) health care purposes. I hope with the launch of this experience, we have helped make a small step towards that vision.