How the largest food company provisions Jenkins instances in less than 10 minutes

by Valentin Delaye // Last updated on March 27, 2023  

10Min To Get A New Jenkins
Valentin Delaye, Devops Solution Architect At Nestlé

Valentin Delaye, DevOps Solution Architect, explains how to build a "Jenkins as a Service" using Jira, Apwide, Kubernetes, Rancher, Jenkins and Terraform.

Our leading Swiss consumer goods company builds and maintains hundreds of web and mobile applications used by our brands, in partnership with hundreds of software agencies. A few years ago, the majority of those agencies were still using their own toolchains with multiple instances of Atlassian suites, GitLab, GitHub, Azure DevOps, etc.

With that many suppliers, each using different toolchains, it was almost impossible to apply global compliance and security standards. People responsible for the delivery were dreaming about a day when every single supplier would use a unified toolchain managed centrally.

Manytools

Challenge

Our company launched its first large-scale eCommerce platform back in 2003, and since then hundreds of eCommerce platforms have been launched. Some were custom-made, many use industry products like Magento or SAP Hybris - often requiring a lot of customizations to make them as unique as their products, and keep their competitive advantage.

In almost two decades, our company worked with hundreds of external agencies across the world, from small companies to big players. Although we owned the intellectual property on all the code, it was stored in various systems and locations, often managed by the external agencies.

I started to work here as DevOps Solution Architect at the end of 2017, just as an ambitious initiative was launched in order to improve compliance and security: moving forwards, external agencies would have to use our central toolchain in order to manage their work, store source code, and orchestrate deployments.

For suppliers to access the toolchain easily, we decided to move it to a cloud-based Environment.

We deployed new instances of Jira, Confluence, and Bitbucket on Microsoft Azure. The 2 toolchains (1 on-premise, 1 on Azure) would live in parallel for a while, in order to avoid painful data migrations. 

But all new projects would use the new toolchain.

For Jenkins, we needed 1 instance per team, this meant hundreds of instances!

Instances of tools like Jira, Confluence and Bitbucket could be shared by all delivery teams, but what about Jenkins? We didn’t want to end up with a huge Jenkinstein for several reasons:

  • Sharing Jenkins resources between all teams is dangerous
  • An outage of the shared instance would impact everyone
  • Different teams have different needs in terms of their Jenkins plugins
  • We wanted the infrastructure to be horizontally scalable

But having 1 instance per team and maintaining it manually would be error-prone and time-consuming. We had to think about something fully automated and fully implemented “as code”.

Solution

Shortly after moving from the HQ in Switzerland, to the newly created Global Digital Hub in Barcelona,  we worked with Apwide’s experts in order to build from scratch a provisioning self-service (Jenkins as a Service) for spinning up new Jenkins instances on demand. Now any internal or external delivery team could request a new Jenkins instance in a few clicks!

For teams asking for new Jenkins instances, it works like this:

Nestle Jenkins Instance Creation Jsd Request

New Jenkins instance provisioning: Jira Service Desk Request

  1. 1
    Open a "New Jenkins instance" request on our Jira Service Desk portal requiring only 3 fields: The team name, the instance name and the link to the Bitbucket repo.
  2. 2
    Wait for the notification that the request’s status has changed to “completed” (maximum 10 minutes).
  3. 3
    Start using the new Jenkins instance, already linked to the Bitbucket project

Yes, it’s like magic! But what kind of automation takes place in the background?

Nestle Jenkins Provisioning Pipeline

Pipeline for the Master Jenkins to provision new Jenkins instances

  1. 1
    The Jira Service Desk request is submitted by the user
  2. 2
    A webhook is executed on a Jenkins master. With the help of Terraform, Rancher and a bit of Ansible a new Jenkins instance is deployed on Kubernetes
  3. 3
    The master Jenkins configures the new Jenkins instance to map it with the Bitbucket project defined in the user request
  4. 4
    The new Jenkins registers itself in Jira (using the Apwide Golive Jira app) with its status (e.g. up, updates, warnings) and the Jenkins version deployed
  5. 5
    The master Jenkins changes the JSD request to "Completed" so that the user gets a notification that his Jenkins instance is ready to use
Your New Jenkins Instance Has Been Deployed

Message posted automatically in the Jira Service Desk ticket

After the creation:

  • A master Jenkins job checks the status and version deployed on all Jenkins environments on a regular basis. It updates Apwide Golive in real-time so that the Environment Repository is always up to date.
  • People interested in a specific Jenkins Environment can subscribe and receive notifications.
Nestle Jenkins Provisioning Apwide Golive Inventory

Apwide Golive Dashboard in Jira: Jenkins instances inventory

Results

This new self-service in Apwide Golive changed the game for our organization! We are now in the process of on-boarding all internal and external delivery teams to our new toolchain.

To get a new Jenkins instance used to be fully manual and the lead time was over a month. It now takes 10 minutes for a team to request and start using its new Jenkins instance!

It now takes 10 minutes for a team to request and start using its new Jenkins instance!

As it is fully automated, it can even be created outside business hours!

We are also in the process of leveraging Apwide Golive to schedule activities on our Environments. All internal and external Jira and Confluence users will not only know the current status of Jenkins instances, but also the planned outages.

I have now more free time to discover Barcelona (wonderful city!) and sleep well because our Jenkins instances are managed in a much better way. Kubernetes and Rancher give key metrics that can be visible in Jira using Apwide Golive, when needed.

Thanks for reading and contact me for any question!

Valentin Delaye


Apwide Golive Logo

Transform your Test Environment Management with Apwide Golive:

  • Never hunt for environment info again,
    it's all in Jira where your team already is!
  • Say goodbye to environment booking conflicts,
    and hello to seamless test campaigns and demos
  • Keep your inbox organized,
    by choosing the environment notifications you need via email, MS Teams or Slack
  • Streamline your environment planning,
    with easy drag-and-drop on an intuitive timeline

Leading companies have already Golive as part of their DevOps toolchain:

Southwest Airlines Company
Mercedes-Benz Company
Manulife Financial Corporation Is A Canadian Multinational Insurance Company And Financial Services Provider.
Sky Television Company
Barclays Bank

Free trial / Free forever up to 10 Jira Cloud users!

More Insightful Articles

Swiss Re Halves Release Management Team’s Support Time

Swiss Re used Excel spreadsheets for their Release Management. Discover why they decided to move to Confluence Team Calendars, and finally to Golive for Jira.

Swiss Re Halves Release Management Team’s Support Time