Leading the Way with Redis

Real-Time Leaderboard Results, Powered by Redis

Image Credit

There’s nothing like a leaderboard to trigger feelings of sincere satisfaction or motivational malaise. Much like Pavlov’s dogs, we start salivating at the sight this engrossing stimulus. Whether we’re motivated by the recognition of our ambitious achievements or by a dreadful sense of judgment and self-reproach, leaderboards manage to hit nerves and rev engines.

What was once a staple of the gaming community is now a tool that drives engagement across a variety of platforms. GitHub, for instance, gamified the process of ‘committing’ to repositories by publicly displaying contribution stats on a calendar. As a developer, I can attest to the power of sprawling swaths of green.

Today we’re happy to release Contributions: a new addition to profile pagesthat lets you see what everyone has been up to on GitHub…The contributions calendar shows how frequently you’ve been contributing over the past year…You’re making contributions to projects all over GitHub and we want to show everyone what you’re doing… Show off the fancy repositories you’ve created.

— GitHub Blog, 2013

Image Credit

The Art of Crafting a Leaderboard

Crafting effective leaderboards has become something of an art as a wider range of disciplines explore and deploy various gamification mechanisms. Fields from education and health to managing and marketing are leveraging leaderboards to elicit positive behaviors and enhance initiative. In fact, building transparency has become such a popular tactic that countless resources are devoted to detailing how to do it right.

Broadly speaking, there are two types of leaderboards: Relative and Absolute. While absolute leaderboards display the top X number of people, relative leaderboards position users among others of a similar rank, mitigating feelings of discouragement and deflation. The spellbinding psychology of winning and losing offers insights into what makes a highly effective table that captures the nuances of motivation and averts disengagement.

The satisfaction of completing a challenge might evoke a ripple of euphoria. Similarly, the relief of not coming in last often conjures a wave of delight. These insights are invaluable in the context of leaderboard development, but other factors weigh just as heavily on successful implementation. It would be foolish to ignore the significance of speed and time in a society driven by instant gratification.

Image CreditFor the Love of Gaming, Update in Real-Time

Fending off frustration and assuaging impatience takes a special kind of database. Here’s the deal; Leaderboards are deceptively difficult to implement. When Space Invaders led the way in the late 1970s, leaderboards catered to a niche community and were capable only of featuring the top ten players and their baller scores. The badass boards of today, however, are responsible for an astounding amount of data and possess a wealth of capabilities that make the underlying technology particularly complex.

Nowadays, developers have to consider factors like cross-platform potential, social integration, and notifications. Users expect to see their latest leaderboard stats across multiple devices and appreciate being updated when their standings switch. Impressive leaderboards also grapple with multiple attributes that could require complicated calculations. A single application might need a global leaderboard, as well as boards distinguished by location, level, or time. A solution that optimizes for all these requirements needs to be fast and flexible.

Traditional, relational databases struggle to serve mammoth volumes of active users. The more users, the slower the queries. When an application uses an external data source, it’s at risk of performing sub-optimally due to data transmission and latency. “High load” environments demand more flexible infrastructure that can effectively process millions of events in a short span of time. A backend leaderboard solution must deal with continuously adding users, removing users, updating scores, finding rankings, etc. Luckily, Redis lends itself to the task!

Redis to the Rescue

Image by Redis

High volumes of read and write operations are likely to exceed the capabilities of traditional databases. But by keeping frequently needed data close to the application, Redis is capable of drastically accelerating response time. Redis works from an in-memory dataset, which makes it uniquely qualified to be leveraged for leaderboards. Rather than repeatedly hitting the database, developers can rely on Redis’ caching layer to generate real-time responses.

Caching is one of the coolest tools on the block when it comes to scalability, and is often the first step taken when performance needs to be improved. It’s advantageous in terms of speed because it translates to far fewer database accesses and much less traffic. For these reasons, developers have been turning to NoSQL data management solutions, like Memcached, for years. Memcached is a well-established and high-performance caching system, but there’s a reason why Redis has been coined, “Memcached on steroids.”

One way Redis is unique is that it offers fine control over data eviction. Developers can choose between various eviction policies to create more room for new information, like updated rankings on a leaderboard. Redis as a cache has tremendous potential, but it’s not the only thing that sets it apart from its alternatives. Unlike Memcached, which only supports the simple key-value structure, Redis supports strings, hashes, lists, sets and sorted sets.

Photo by RedisLabs

Redis is fantastically flexible regarding the objects that can be cached. With five primary data structures to pick from, it provides sophisticated ways to tackle complicated tasks. Sorted Sets, in particular, help Redis kick ass at adding, removing, and updating data at lightning speed. This data structure falls somewhere between a traditional set and a hash. It’s a non-repeating collection of strings, but the elements inside are mapped to a value, not ordered. Sorted sets are especially powerful because they are able to operate on ranges, making it incredibly easy to access any span of scores, ranks, etc.

In the case of leaderboards, each unique user is associated with a score (that can be repeated) and different sorted sets represent various types of rankings. Built-in Redis ZSET(sorted set) commands make it easy to sort and access the exact information, or range of information, that you need. Let’s wrap things up by walking through a simple implementation ofZSET!

Implementing ZSET

We’re going to record scores in a sorted set by setting a value for a for a given key. Scores can be modified with theZADD command.

redis> ZADD "scores" 100 "anna"(integer) 1redis> ZADD "scores" 95 "david"(integer) 1

Now let’s query the scores and retrieve them by rank. We’ll use ZRANGEBYSCORE to return all the elements in a sorted set at a specific key, with a score between a specified min and max.

redis> ZRANGE 'scores' 0 11) "david"2) "anna"

We can even return users alongside their scores by using WITHSCORES:

redis> ZRANGE 'scores' 0 1 WITHSCORES1) "david"2) "95"3) "anna"4) "100"

Common Commands Cheat Sheet

Refer to this list of common ZSETcommands for additional clarification.

Photo by Redis

References

• Redis Labs | Database for the Instant Experience
• Data types - Redis
• Command reference - Redis
• Introducing Contributions - The GitHub Blog
• [Making a Browser Game] Daily Leaderboard with Redis
• Leaderboards - The original and best social feature
• How to Build a Leaderboard with PHP and Redis
• How to take advantage of Redis just adding it to your stack

Leading the Way with Redis was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.