How the largest food company provisions Jenkins instances in less than 10 minutes

Valentin Delaye, DevOps Solution Architect at Nestlé

Valentin Delaye, DevOps Solution Architect, explains how to build a "Jenkins as a Service" using Jira, Apwide, Kubernetes, Rancher, Jenkins and Terraform.

Our leading Swiss consumer goods company builds and maintains hundreds of web and mobile applications used by our brands, in partnership with hundreds of software agencies. A few years ago, the majority of those agencies were still using their own toolchains with multiple instances of Atlassian suites, GitLab, GitHub, Azure DevOps, etc.

With that many suppliers, each using different toolchains, it was almost impossible to apply global compliance and security standards. People responsible for the delivery were dreaming about a day when every single supplier would use a unified toolchain managed centrally.

Challenge

Our company launched its first large-scale eCommerce platform back in 2003, and since then hundreds of eCommerce platforms have been launched. Some were custom-made, many use industry products like Magento or SAP Hybris - often requiring a lot of customizations to make them as unique as their products, and keep their competitive advantage.

In almost two decades, our company worked with hundreds of external agencies across the world, from small companies to big players. Although we owned the intellectual property on all the code, it was stored in various systems and locations, often managed by the external agencies.

I started to work here as DevOps Solution Architect at the end of 2017, just as an ambitious initiative was launched in order to improve compliance and security: moving forwards, external agencies would have to use our central toolchain in order to manage their work, store source code, and orchestrate deployments.

For suppliers to access the toolchain easily, we decided to move it to a cloud-based Environment.

We deployed new instances of Jira, Confluence, and Bitbucket on Microsoft Azure. The 2 toolchains (1 on-premise, 1 on Azure) would live in parallel for a while, in order to avoid painful data migrations. 

But all new projects would use the new toolchain.

For Jenkins, we needed 1 instance per team, this meant hundreds of instances!

Instances of tools like Jira, Confluence and Bitbucket could be shared by all delivery teams, but what about Jenkins? We didn’t want to end up with a huge Jenkinstein for several reasons:

  • Sharing Jenkins resources between all teams is dangerous
  • An outage of the shared instance would impact everyone
  • Different teams have different needs in terms of their Jenkins plugins
  • We wanted the infrastructure to be horizontally scalable

But having 1 instance per team and maintaining it manually would be error-prone and time-consuming. We had to think about something fully automated and fully implemented “as code”.

Solution

Shortly after moving from the HQ in Switzerland, to the newly created Global Digital Hub in Barcelona,  we worked with Apwide’s experts in order to build from scratch a provisioning self-service (Jenkins as a Service) for spinning up new Jenkins instances on demand. Now any internal or external delivery team could request a new Jenkins instance in a few clicks!

For teams asking for new Jenkins instances, it works like this:

New Jenkins instance provisioning: Jira Service Desk Request

  1. 1
    Open a "New Jenkins instance" request on our Jira Service Desk portal requiring only 3 fields: The team name, the instance name and the link to the Bitbucket repo.
  2. 2
    Wait for the notification that the request’s status has changed to “completed” (maximum 10 minutes).
  3. 3
    Start using the new Jenkins instance, already linked to the Bitbucket project

Yes, it’s like magic! But what kind of automation takes place in the background?

Pipeline for the Master Jenkins to provision new Jenkins instances

  1. 1
    The Jira Service Desk request is submitted by the user
  2. 2
    A webhook is executed on a Jenkins master. With the help of Terraform, Rancher and a bit of Ansible a new Jenkins instance is deployed on Kubernetes
  3. 3
    The master Jenkins configures the new Jenkins instance to map it with the Bitbucket project defined in the user request
  4. 4
    The new Jenkins registers itself in Jira (using the Apwide Golive Jira app) with its status (e.g. up, updates, warnings) and the Jenkins version deployed
  5. 5
    The master Jenkins changes the JSD request to "Completed" so that the user gets a notification that his Jenkins instance is ready to use
Your new Jenkins instance has been deployed

Message posted automatically in the Jira Service Desk ticket

After the creation:

  • A master Jenkins job checks the status and version deployed on all Jenkins environments on a regular basis. It updates Apwide Golive in real-time so that the Environment Repository is always up to date.
  • People interested in a specific Jenkins Environment can subscribe and receive notifications.

Apwide Golive Dashboard in Jira: Jenkins instances inventory

Results

This new self-service in Apwide Golive changed the game for our organization! We are now in the process of on-boarding all internal and external delivery teams to our new toolchain.

To get a new Jenkins instance used to be fully manual and the lead time was over a month. It now takes 10 minutes for a team to request and start using its new Jenkins instance!

It now takes 10 minutes for a team to request and start using its new Jenkins instance!

As it is fully automated, it can even be created outside business hours!

We are also in the process of leveraging Apwide Golive to schedule activities on our Environments. All internal and external Jira and Confluence users will not only know the current status of Jenkins instances, but also the planned outages.

I have now more free time to discover Barcelona (wonderful city!) and sleep well because our Jenkins instances are managed in a much better way. Kubernetes and Rancher give key metrics that can be visible in Jira using Apwide Golive, when needed.

Thanks for reading and contact me for any question!

Valentin Delaye


Apwide Golive is a Jira app that gives you visibility and control over your environments.

  • save time when looking for environment information: no need to login to several tools
  • avoid environment booking conflicts: no more config issue in the middle of a demo
  • stop being spammed: each user decides what notification he/she wants to receive and can unsubscribe in a click
  • plan environments and release activities faster: use drag-and-drop on a timeline
Southwest Airlines Company
Mercedes-Benz Company
Manulife Financial Corporation is a Canadian multinational insurance company and financial services provider.
Sky Television Company
Barclays Bank

Many leading companies have already chosen Apwide Golive for Test Environment Management as it integrates easily with many other DevOps tools.

Check our free trial on Jira Server or Jira Cloud!

More about Test Environment Management

Kill your Project Status Reports; Use Interactive Jira Dashboards

As an IT organization grows, so does the number of projects and initiatives launched in parallel. The need to aggregate reporting becomes more and more Read more

How Interactive Release Dashboards in Jira will empty your Mailbox

Mature software delivery organizations release often, with an automated or almost automated process. But come on, the majority of large organizations have not reached that Read more