6 common mistakes when using spikes

During refinement a team member says:”I have no clue how to build this feature”. She warns we should not start working on the feature before doing a spike.

Mentioning spikes often creates confusion in a team. Every team member has their own opinion about what a spike is and the right time to use one. I was part of teams that wanted a spike for almost everything they worked on. I also worked with teams that rarely used spikes. Both approaches where suboptimal. When is using a spike appropriate?

Before we expand on when and how spikes should be used, let’s talk about the origin of the term ‘Spike’.

The term spike stems from Extreme Programming (XP) and has it’s roots in rock climbing.

When rock climbing, you sometimes cannot climb further because there is nothing to grab on to. To climb further, you need to drive a spike into the rock surface. The spike does not move you closer to the top, but forges a path for you to continue climbing on.

A spike in software development serves a similar purpose. Spikes help to move forward when knowledge is lacking to build something.

When should you use a spike?

Backlog refinement is all about removing uncertainty and creating common understanding. Lack of knowledge is the starting point in refinement. So, it is important to understand when using a spike is appropriate or overkill.

In product backlog refinement three types of uncertainty are discussed with the development team:

1. Purpose uncertainty. Why do we need to build this?
2. End uncertainty. What do we need to build?
3. Means uncertainty. How can we build it?

At the start of refinement, everything is unclear. After refining a feature, all three types of uncertainty should be sufficiently clear to be able to estimate the feature. If your team does not use estimates, you should be able to split the feature up in sprint-sized chunks.

Sometimes everything is clear, except how to build a feature. If the team has no idea how to build the feature, then the work cannot be accurately estimated. It cannot be split up to fit in a single sprint either. The risk of building a technical solution that is unfeasible increases.

Now is the right moment to use a spike. When the team has no idea how to build a feature, then planning a spike is appropriate.

A spike is an investigation the team has to perform before they can start working on the feature it relates to. The outcome of the spike needs to be explicitly defined up-front.

A spike is usually time-boxed. This means the investigation is over when the time-box expires, regardless whether the spike outcome is met or not. A time-box helps to manage time and prevent endless research without results. By definition, research is more open-ended and divergent than delivering something.

When a spike is completed (with or without the time-box expiring), there are two options:

• You have enough information on the technical solution to proceed. You can now estimate the feature. If your team does not use estimates, you have enough information to split the feature up in chunks that take less than a sprint.
• The investigation fails to provide enough information on the technical solution to proceed. Work cannot be started yet, unless the risk of being unable to finish the feature in a single sprint is accepted.

Sometimes people make the distinction between functional and technical spikes. The only difference is the type of research. A technical spike is planned when it is unclear how to build something. A functional spike is planned when more information is needed to decide what to build from functional perspective.

Now that you have basic understanding about what a spike is and when you should use one, let’s discuss 6 common mistakes teams make when using spikes.

1. Performing a spike when many other things are unclear

If the purpose of the backlog item or how it should work are unclear, then you are better off reducing these types of uncertainty first. Otherwise, how can you be sure that a spike is necessary?

With a better understanding of the problem, you give the team more freedom to come up with a suitable technical solution. Uncertainty in the ‘why’ and ‘what’ propagates to the technical solution. Maybe there is a different way of solving the problem your team does not consider, because they do not understand the problem well enough.

2. Using spikes for implementation details you can figure out as you go along

A spike should give your team confidence it can build a feature and split it up in chunks that fit in a sprint. It should not be used to figure out every little detail before you can work on a feature.

Some teams will use spikes to nail down all details. They believe that everything needs to be crystal clear before you can work on a feature. The team wastes time researching implementation details that could be figured out as you go along.

As a result, sprints alternate between research and delivery work. The time-to-market of features increases. Features requiring a spike will take at least two sprints to deliver, because you need to do the spike one sprint before. If the team is confident it can finish the spike and feature it belongs to in a single sprint, why do the spike at all?

Figuring out implementation details during the same sprint you perform the work is more effective. You minimize waste, hand-overs and doing up-front analysis. The feature will also be released sooner, allowing you to process valuable market feedback earlier.

3. Doing spikes before considering to reduce scope

When a team member says a spike is necessary, you should try to understand what technical part makes the solution difficult. If possible, just remove the difficult part from the functional scope. Just pick up what is easy first and worry about the hard part later.

This way the feature can be estimated and picked up without any preceding spike. Sometimes just by starting to work on a feature the team acquires enough knowledge to work on the difficult part. If this is not the case, then at least you will have more knowledge before doing the spike.

4. Not defining a clear spike outcome

Before working on a spike, the expected outcome of the investigation should be clear. By making the expected outcome clear, you prevent your team from being unable to move forward even though the spike has been completed.

What is the information you need to research in order to move forward? Should it be a working proof-of-concept? Or is a document that describes the technical solution good enough? What will give the team enough confidence to move forward?

5. Executing a spike long before you need it

Analysis work many sprints before you start working on a feature is waste you should try to prevent. When an spike has been completed, the outcome is fresh. As time passes on, spikes become stale just like food.

The system landscape may have changed so much, the spike is no longer relevant. It might also be that the feature is no longer important, because it no longer fits the direction of the product. As time passes on, the team loses knowledge it gained through the spike. The team will need more time to understand the outcome of the original investigation.

You should try to plan a spike as close to the moment you want to pick-up the backlog item to minimize all these different kinds of waste.

6. Planning spikes without digging deeper

When your team keeps asking to plan spikes, this could point to bigger problems the team is facing, such as:

• Knowledge the team misses in specific areas or a single team member having most of the knowledge.
• Legacy systems that are very difficult to maintain or adjust. Nobody feels comfortable estimating the work.

If this keeps on happening, it may pay off to get to the bottom of these problems . To see if there is something you can do to prevent these spikes from being necessary. Maybe it pays off to refactor some parts of the system instead of performing the spike. It could also be you just need to plan a knowledge sharing session, so the team gains knowledge about part(s) of the system or a specific framework.

Spikes help manage uncertainty, but should be used sparingly

Spikes should be the exception, not the rule. You should ask these questions before deciding if a spike is appropriate:

• Does the team understand the problem well enough to come up with the best solution? Get everything else that is uncertain, as clear as possible, before you decide a spike is necessary. Uncertainty in why we are building something and what we need to build propagates to the technical solution.
• Is the spike really necessary? Sometimes you can figure out the technical details as you go along. Some uncertainty is to be expected. Is the team suffering from a waterfall mindset or is the spike justified?
• Can the scope be reduced to move what is difficult out of scope? This way you can start working on the feature without a spike. After finishing the feature, sometimes you will have gained enough knowledge to work on the difficult part without a spike. If you still need one, then at least you will have more knowledge before starting the investigation.
• Did the team define a clear expected outcome in the spike? The expected outcome of the spike needs to be clear to minimize the chance of failure.
• Will the feature where the spike relates to be picked up soon? Spikes become stale. Changes in technical landscape or business priority may make spikes obsolete. So plan spikes as close to the actual work as possible to prevent waste.
• Is the excessive usage of spikes not symptomatic of bigger problems? Dig deeper to find out what problems your team is facing. Then make a decision if it is better to use a spike or spend time to address the root cause.

6 common mistakes when using spikes was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.