Simplifying Jenkinsfiles

Using Jenkins to unfuck Jenkins.

I’ve been going on an absolute mad one through the inner workings of Jenkins over the past few weeks. We’ve been scouring for a nice, convenient way of simplifying the Jenkinsfile at the root of our application repositories. We like Jenkins and after a bit of googling and some experimentation, we found something cool. BUT FIRST, the problem.

I’ve tried to eloquently describe how grim some of our Jenkinsfiles were, but a colleague of mine beat me to the punch:

That Jenkinsfile looks like a Groovy developer threw up into the cat command.

He wasn’t wrong. By Jesus, we had some pretty crazy shit going on in our files. All of it needed but none of it pretty. Check this bad boy out:

pipeline {    agent any

    parameters {        string (defaultValue: '', description: 'Imagine like 10 parameters here', name: 'buildNumber')

    tools {        // Yep, everyone was using the same maven and JDK version.

maven 'Maven 3.5.0'        jdk 'jdk8'    }    environment {        BUCKET='my-app-bucket'

        // ABOUT 15 IDENTICAL ENVIRONMENT VARIABLES    }    stages {        stage("Initialise"){

            steps{                echo "Config Aws - Yep, everyone does this too"                sh"""aws do-some-repetitive-shit                """            }        }        stage("Test"){            when {                expression {                    params.buildNumber == ''                }            }            steps{                echo 'Running Tests'                sh "Run the same maven command as every build"                }            }            post {                always {                    junit 'target/surefire-reports/**/*.xml'                }                success {sh "Tell Slack the build went okay"                }                failure {sh "Tell Slack the build is knackered"                }            }        }        stage('Cleanup') {            when {// Do the same conditional logic in every file.            }            steps {sh "Do the same clean up for every app"            }        }

        stage('Packaging') {            when {// Some more conditional logic? Fuck it, why not.            }            steps {                // You guessed it you animals, more of the same.                sh "Literally run mvn package"

            }            post {                failure {sh "A failure condition that has never fired"                }            }        }

        stage(’Push App Version’) {            when {// It’s not complicated enough. MORE CONDITIONALS.            }            steps {// You bet this is copied and pasted.             }            }

        stage(’Deploy’) {            when {// Ayyy. It’s not a party without conditional logic.            }            steps{                echo "Deploying cos to dev"sh "THE SAME COMMAND EVERY TIME SERIOUSLY OMG"            }        }    }}

HOLY SHIT CHRIS

I KNOW. And we’re running freaking microservices, this type of madness was sprayed all over the show. Dozens of this exact file (minus the swear words) AND it was ENTIRELY my fault. I remember designing this damn file in the first place. I recall lazily copying and pasting. I KNEW this was going to bite me in the arse and, like an S&M enthusiast, I didn’t care. It felt good to be bad.

Well, it stopped feeling good when we had to make changes to our CI/CD pipelines. I took off my gimp mask and did some googling. Then I struck gold.

Global Shared Libraries

Right, so the first thing to recognise is that your pipeline is just a set of groovy scripts. If it’s declarative or scripted, it’s just running a bunch of groovy commands. Groovy, like Java, supports the import statement at the top of the file. Once that was obvious, a few things occurred to us:

• All the builds were doing the same stuff in the same way
• Because of the copying and pasting, some builds were using older versions of maven or running different flavours of commands.
• All the applications were deploying into the same infrastructure.

So I turned to my colleague and I told that beautiful bastard that I’ve got an idea. He ignored me because I say that 100 times a day but THIS TIME I was onto something.

I learnt some things along the way and so I’ve put together a simple example repository with some of the caveats we found along the way. The Jenkins documentation is comprehensive, but I thought a simple code example can get you going. Well, the grand unveiling, what does our Jenkinsfile look like now?

import com.central.ci.*

APP_NAME = "my-app"

Pipelines pipelines = new Pipelines()

pipelines.javaPipeline(APP_NAME)

// They could also invoke pipelines.emptyJavaPipeline() and define their own, but without the need to set everything up.

And doing it was mega easy!

First, make a git repository. Jenkins supports some others but it isn’t 2006 any more. Remember, Jenkins needs the appropriate credentials to be able to access your repository if it’s private.

Second, tell Jenkins that this repository exists. Click on “Manage Jenkins” and scroll down until you see this:

You’ve struck oil you lucky bastard.

You need to specify the location of the repository including any credentials you need. From there, treat your Jenkinsfile like any other Groovy script and import your library. I’d recommend ticking the “Load Implicitly” box if you want to make sure all of the builds have this by default.

So yeah

Couldn’t believe how well it worked to be honest. We’ve not given it an exhaustive run out, but so far it has been well received and we’re in the process of rolling it out to our existing applications.

Simplifying Jenkinsfiles was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.