Kicking the tires of quantum processors

If you have n bits sitting in your computer, there are 2ⁿ possible bit strings that they could be storing. By charting a suitable course through this huge space of possibilities, your computer can solve virtually any problem.

If you have a quantum computer with quantum bits, there is also an exponentially large space of possibilities. But it one that allows more than just bit strings: quantum mechanical concepts such as superposition and non-locality can be used too. These allow new and more subtle ways to move around the space, giving us new and more efficient routes from input to output. Quantum computers will give us practical solutions to currently intractable problems in many fields.

Currently, no device has actually been built that can live up to this promise. But there are certainly prototypes, and they need testing out. So what shall we try?

One thing we could do is just generate a completely random quantum program. This will generate random bit strings as an output, but they won’t come out with equal probabilities. Instead, there will be small variations, providing a signature of their quantum nature.

In fact, for sufficiently large prototype quantum computers, this random generation of bit strings is something that no regular computer could reproduce. This means we could use this task as a proof that quantum computers can do things that normal ones can’t. Not a very useful task, but good enough for a proof-of-principle.

This is Google’s plan for achieving what has been called quantum computational supremacy. Though it may be a while before they manage to pull it off. For one thing, you need quite a few qubits in your device before regular computers are unable to simulate them

Google, Alibaba Spar Over Timeline for 'Quantum Supremacy'

Current prototype devices are typically small enough to simulate. But as well as not having enough qubits, they also bit too noisy for this proof-of-principle task. The details of small variations in probabilities depend on the random program that we generate. So if there are extra random effects, due to imperfections in the device, we won’t be able to verify that we have the output we need. Then we’d just have a boring, and pretty easy to replicate, generator of random bit strings.

So how can we benchmark how good current devices are at random circuits? How can we easily distinguish the quantum randomness that we mean to create from the noise that will destroy it?

One option is to compose a program full of random operations, but which doesn’t actually do anything. We can loop through a series of rounds in which random things are done, and then instantly undone. This tests out all aspects of the programs we want to run, but ensures that the output should be well known and easy to verify.

This is basically what I did in a paper I just put out. I tested out four current devices made by two different manufacturers: IBM Research and Rigetti.

Benchmarking of quantum processors with random circuits

Let’s take a quick look at what the results should look like.

Since this is a blog post rather than a paper, I won’t regale you with details on what MWPM is. But just know that its average correctness provides a good measure of how close the output is to something nice and sensible.

If we are actively trying to bring on the chaotic generation of random bit strings, we want to see it decay quickly to a value where it will converge. That’s exactly what we see with the orange curve above, which simulates a random program.

The blue curve is one where we undo almost all of the random operations in our programs. This means that we would hope to see a much slower descent into chaos. That happens in the graph above, because it is a simulation of a device with no noise. But what will we see for a real prototype quantum processor?

We’ll start with the smallest device that is currently on the cloud: IBM’s 5 qubit device known as Tenerife, or ibmqx4 to its friends.

Here the orange and blue are as they were before, though with programs built with the specific abilities of ibmqx4 in mind.

We ran jobs on ibmqx5 that should have an even slower descent into chaos than the blue line. So if it falls faster, we know that it is because of the damaging effects of noise, rather than the quantum effects we actually want to see.

The results we got are shown with the green line. As you can see, it falls pretty fast. Faster than for a completely random circuit, in fact. Noise is obviously quite powerful.

This is partly because we tend to expect computers to be perfect these days. We take the output we are given and trust that all the transistors that created it were doing what they should.

For quantum computers, we need to dial our attitude to computers back a few decades. We need to expect the unexpected, and figure out how to work around imperfections.

This is what the red curve shows. We did a bit of simple postprocessing on the output to clean things up. This slows the descent into chaos down massively, and brings the output far closer to what we expect it to be. So ibmqx5 certainly has potential to do good things, we just have to remember to fight for it.

Now let’s do the same for a bigger device: Rigetti’s 8Q-Agave. As you might guess from the name, it has 8 qubits.

We see similar quantitive features to before. Though the green curve should go down slower that the blue one, it turns out to descend into chaos even faster than the completely random program of the orange curve. Though our error mitigation again cleans things up, as shown by the red curve, it still gives results similar to the completely random program.

Now let’s double our qubit number and go for IBM’s Rueshlikon, also known as ibmqx5.

Hopefully you are starting to know what you are looking for in these graphs. The red curve is best attempt at making something similar to the blue curve. At the very least, it should decay much slower than the orange one. For this device, the blue curve does indeed stay significantly above the orange (which is good), but is still significantly below the orange (not so good).

Finally, we go up to 19 qubits with Rigetti’s 19Q-Acorn.

This has some similarities to its sibling: 8Q-Agave. But there’s one major difference between this device and all the others we’ve looked at so far. They all have the red curve start off at 1, showing that the devices are very good at avoiding noise when they run short programs. For 19Q-Acorn, the red curve starts off at 0.9. So there’s a degree of noise that’s difficult tackle, even from the beginning.

So now you have an idea of how current quantum processors work. If you’ve been told scare stories about how quantum computers are about to break all crypotography and create Skynet, hopefully you’ll now be able to sleep easily once more. It’s a very exciting time to be in the field of quantum computing, and a great time to get involved, but the next decade will involve a lot of figuring out where the errors are and what to do with them.

If you want to know more, but can’t be bothered to read my paper? Don’t worry, I have you covered. The random circuits used in the study above were designed to be the basis of a puzzle game. So you can literally play with the data used in my study. This will also help you learn about more about the devices than just their noise on the devices. Check out the links below.

• decodoku/A_Game_to_Benchmark_Quantum_Computers
• Using a simple puzzle game to benchmark quantum computers

You can also get to know the devices better by using them yourself. They are all on the cloud, and available for the public to run jobs (except 19Q-Acorn, which was recently retired). Start your journey into programming these devices with the links below.

• QISKit
• Rigetti

Find me on Twitter

Kicking the tires of quantum processors was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.