Making a web voting app handling 245 million votes with $0.12

The Inspiration

I always liked the emotion of 1x1 online voting duels like Big Brother Brasil and The Voice Battle. The way how it reaches millions of people is incredible. So, I decided to do a simple poll application able to deal with 1 day of voting, in a large country like Brazil and USA. With a challenge: I wanted do that, from scratch, in 2 hours, with some cents.

As goal, I imagined a country of 300 million people, where 70% of population will vote. That is 210 million votes in one day, 8.75 million in one hour and approximately 2500 req/sec.

The way

To do that, I chose the simple and powerful combination of Golang and Redis (with AOF persistence). Running in a simple Digital Ocean droplet with Ubuntu, without any SO fine tunning. I didn’t worry about auth, crawlers and rate limiting in this project.

For this goal, I created a project with 3 files: docker-compose.yml, Dockerfile and main.go. (https://github.com/danhenriquesc/go-poll)

The docker-compose.yml and Dockerfile files basically builds up a redis server and a Golang 1.11 API. Running on localhost:8000. I just need to run a simple command: docker-compose up

The main.go is a file of 96 lines that creates an API with 2 endpoints:

• POST /vote/{candidate:[1|2]} to vote in one of the candidates and return the current status of poll,
• GET /vote to get the current status of poll.

Two Golang packages were used in this API:

• github.com/gorilla/mux to facilitate HTTP router,
• github.com/go-redis/redis to facilitate Redis access.

The HTTP Benchmarking execution

After creating that project, I decided to check how many requests it could handle, with wrk tool. So I tested in a cheap and realistic way, with Digital Ocean droplets.

I choosed 3 Standard Droplets options to test:

• $5/month droplet: 1GB RAM, 1 vCPU 1.8GHz
• $15/month droplet: 1GB RAM, 3vCPU 1.8GHz
• $80/month droplet: 16GB RAM, 6vCPU 1.8GHz

The three running Ubuntu 16.04.4 x64, without any fine tuning.

You can see the benchmark sets below:

$5/month droplet

Execution 1

root@ubuntu-s-1vcpu-1gb-nyc1–01:~# wrk -t 1000 -c 10000 -d 1m -s ./post.lua http://68.183.137.138:8000/vote/2Running 1m test @ http://68.183.137.138:8000/vote/2 1000 threads and 10000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 607.99ms 157.48ms 1.99s 82.22% Req/Sec 24.63 22.40 292.00 75.02% 345301 requests in 1.00m, 52.72MB readRequests/sec: 5755.02Transfer/sec: 0.88MB

Execution 2

root@ubuntu-s-1vcpu-1gb-nyc1–01:~# wrk -t 1000 -c 20000 -d 1m -s ./post.lua http://68.183.137.138:8000/vote/2Running 1m test @ http://68.183.137.138:8000/vote/2 1000 threads and 20000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 860.65ms 296.29ms 2.00s 65.63% Req/Sec 39.72 45.10 686.00 83.99% 358947 requests in 1.00m, 54.94MB readRequests/sec: 5982.46Transfer/sec: 0.91MB

$15/month droplet

Execution 1

root@ubuntu-s-3vcpu-1gb-nyc1-01:~# wrk -t 7500 -c 10000 -d 1m -s ./post.lua http://204.48.20.138:8000/vote/2                                                                                                                                   Running 1m test @ http://204.48.20.138:8000/vote/2  7500 threads and 10000 connections  Thread Stats   Avg      Stdev     Max   +/- Stdev    Latency   631.39ms  107.29ms 798.00ms   91.50%    Req/Sec     1.25      2.28   272.00     97.79%  735058 requests in 1.01m, 112.16MB readRequests/sec:  12091.66Transfer/sec:      1.85MB

Execution 2

root@ubuntu-s-3vcpu-1gb-nyc1-01:~# wrk -t 1000 -c 13000 -d 1m -s ./post.lua http://204.48.20.138:8000/vote/1                                                                                                                                   Running 1m test @ http://204.48.20.138:8000/vote/1  1000 threads and 13000 connections  Thread Stats   Avg      Stdev     Max   +/- Stdev    Latency     1.08s    89.31ms   1.91s    80.95%    Req/Sec    22.67     29.20   650.00     87.31%  714698 requests in 1.00m, 113.13MB readRequests/sec:  11884.94Transfer/sec:      1.88MB

Execution 3

root@ubuntu-s-3vcpu-1gb-nyc1-01:~# wrk -t 1000 -c 15000 -d 1m -s ./post.lua http://204.48.20.138:8000/vote/1                                                                                                                                   Running 1m test @ http://204.48.20.138:8000/vote/1  1000 threads and 15000 connections  Thread Stats   Avg      Stdev     Max   +/- Stdev    Latency     1.32s   128.68ms   1.99s    77.60%    Req/Sec    24.45     33.76     1.09k    85.97%  671964 requests in 1.00m, 102.71MB readRequests/sec:  11199.40Transfer/sec:      1.71MB

$80/month droplet

Execution 1

root@ubuntu-s-6vcpu-16gb-nyc1-01:~# wrk -t 1000 -c 70000 -d 1m -s ./post.lua http://204.48.20.138:8000/vote/1                                                                                                                                   Running 1m test @ http://204.48.20.138:8000/vote/1  1000 threads and 70000 connections  Thread Stats   Avg      Stdev     Max   +/- Stdev    Latency   902.54ms  103.71ms   2.00s    78.21%    Req/Sec    79.04     71.04     3.75k    84.72%  4117831 requests in 1.01m, 637.73MB readRequests/sec:  68630.52Transfer/sec:     10.54MB

Execution 2

root@ubuntu-s-6vcpu-16gb-nyc1-01:~# wrk -t 1000 -c 30000 -d 1m -s ./post.lua http://204.48.20.138:8000/vote/1 Running 1m test @ http://204.48.20.138:8000/vote/1  1000 threads and 30000 connections  Thread Stats   Avg      Stdev     Max   +/- Stdev    Latency   942.14ms  127.26ms   2.00s    85.55%    Req/Sec    47.32     55.14     1.33k    88.25%  1830829 requests in 1.00m, 281.99MB readRequests/sec:  30513.82Transfer/sec:      4.68MB

Execution 3

root@ubuntu-s-6vcpu-16gb-nyc1-01:~# wrk -t 1000 -c 20000 -d 1m -s ./post.lua http://204.48.20.138:8000/vote/2Running 1m test @ http://204.48.20.138:8000/vote/2  1000 threads and 20000 connections  Thread Stats   Avg      Stdev     Max   +/- Stdev    Latency   656.35ms   89.49ms   1.61s    84.31%    Req/Sec    40.59     41.40     1.13k    87.53%  1828994 requests in 1.00m, 292.16MB readRequests/sec:  30483.21Transfer/sec:      4.85MB

The analysis of Benchmarking

$5/month droplet

Firstly, I tested with the cheapest droplet of Digital Ocean, that costs $5/month. As I want to deal with 1 day of voting, so, I had the 1-day cost of approximately $0.16.

In this droplet, we had something about 5800 req/sec, that can handle:

• 348,000 votes per minute
• 20,880,000 votes in an hour with $0.007
• 501,120,000 votes in a day with $0.16

In this droplet, I can easily achieve the initial goal of handle 210 million votes in one day, with $0.16.

$15/month droplet

So, with the mission completed, why not stress a more powerful droplet? So, I changed for the $15/month droplet, that costs $0.02/hr and $0.53/day.

In this droplet, we got approximately 12,000 req/sec, basically twice of $5/mo droplet. It wasn’t good because paying three times more didn’t give me three times the performance.

$80/month droplet

Jumping to a new level, I decided to try the $40/mo server. But, because of a missclick, I selected the $80/mo server, and there I went: A big surprise. This droplet costs of $0.12/hr and $2.85/day, and reached about 68,000 req/sec.

With this ratio, we can handle:

• 4,080,000 votes per minute
• 244,800,000 votes in an hour with $0.12
• 5,875,200,000 votes in a day with $2.85

The Conclusion

With this benchmark, we can get some interesting insights:

• We can handle 501,120,000 votes, scattered throughout the day, with $0.16. That is the almost the sum of population of USA and Indonesia, the 3rd and 4th biggest populations of the world.
• We can handle 244,800,000 votes, in a single hour, with $0.12. That is more than the population of Brazil, the 5th biggest population of the world.
• We can handle 5,875,200,000 votes, scattered throughout the day, with $2.856. That is more than 4x the population of China, the biggest population of the world.

With that number, we can realize that some applications appear to be harder and more expensive than in fact are. Even though some business rules were despised, the cost would remain low for any small business.

Besides that, other architectures like serverless functions in AWS or GCP can provide better performance and scalability with low cost. It gives a lot of opportunities for new business have high-scalable applications.

The possibilities are there, just try it!

Making a web voting app handling 245 million votes with $0.12 was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.