Configure Seed Jobs and Pipelines

This document explains how to configure Jobs and Pipelines in Carthago Operator for Jenkins.

To preserve your Jenkins pipelines and automate their recreation in case of instance failures, we strongly recommend using configuration as code files to set up pipelines.

Created jobs

Carthago Operator for Jenkins uses job-dsl and kubernetes-credentials-provider plugins for configuring jobs and deploy keys.

We have to start by preparing pipeline and job definitions in your project’s GitHub repository main folder using the following structure:

cicd/
├── jobs
│   └── build.jenkins
├── pipelines
│   └── build.jenkins

Jenkins will always check the configurations directly from those files in your repository, so you don’t have to update the configuration every time you change the pipeline code itself.

Seed Job Definition

A seed job represents Jenkins job that creates one or more pipelines in Jenkins. It uses pipeline configuration files from your GitHub cicd folder. Let’s create a job configuration file and store it at https://github.com/your-project-repo/cicd/jobs/build.jenkins:

#!/usr/bin/env groovy

pipelineJob('example-job') {
    displayName('Example Job')

    definition {
        cpsScm {
            scm {
                git {
                    remote {
                        url('https://github.com/your-account/your-repo.git')
                    }
                    branches('*/master')
                }
            }
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

pipelineJob – name of pipeline resource that we will create in a while
displayName – name of seed job that will be displayed in Jenkins UI
remote – here you specify the url of GitHub repository of your project and username you are using in Carthago Operator for Jenkins, in order to access this repository
branches – branches from which you want to access the repo
scriptPath – the path in the above repo in which you store pipeline files from which you want to create Jenkins pipelines during this job run

Pipeline Definition

Now we can create the pipeline configuration file and store it at https://github.com/your-project-repo/cicd/pipelines/build.jenkins. This file will create pod with containers to run commands on each step (stage) in your pipeline.

#!/usr/bin/env groovy

def label = "example-pipeline-${UUID.randomUUID().toString()}"

podTemplate(label: label,
        containers: [
                containerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent:latest'),
        ],
        ) {
    node(label) {
        stage('Init') {
            timeout(time: 3, unit: 'MINUTES') {
                checkout scm
            }
        }
        stage('Dep') {
            echo 'Hello from Dep stage'
        }
        stage('Test') {
            echo 'Hello from Test stage'
        }
        stage('Build') {
            echo 'Hello from Build stage'
        }
    }
}

label – here specify the name of the pipeline (pipelineJob value from Seed Job)
podTemplate – a pod that will be created during this pipeline run, to execute necessary steps
containers – containers in a pod that will be used to run necessary steps. One jnlp container is always needed. All the others need to use images of programmes needed for the pipeline’s stages. Full list of possible functionalities can be found here. If you need to run kubectl commands you need to use container with an image that incorporates kubectl, because it is not provided by default.
stage – at each stage you specify stage name, scripts, commands and container which needs to run them

Update JenkinsSeedJob

After you create the repository containing seed job files, you need to specify the repository URL and the location of seed job files in the repository in JenkinsSeedJob Custom Resource.

Carthago Operator will create a default JenkinsKubernetesAgent seed-job-agent and seed job will be processed by default agent.

Jenkins will then create Jenkins jobs from seed job files from your git repository and if you run them in the Jenkins UI, they will create the necessary pipelines.

apiVersion: carthago.cloud/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    carthago.cloud/jenkins: example
spec:
  repository:
    url: https://github.com/jenkinsci/kubernetes-operator.git
    branch: master
    targets: "cicd/jobs/*.jenkins"

You have to specify the Jenkins Custom Resource name in the label carthago.cloud/jenkins to connect the Seed Job with relevant Jenkins. It’s especially important in case of running multiple Jenkins instances.

Operator will then automatically discover and configure all the seed jobs.

Using your own Kubernetes Agent to run Seed Job

If you have your own Kubernetes Agent, you can use it to run your seed jobs. To do that, reference the Agent in spec.agentRef field of JenkinsSeedJobs:

apiVersion: carthago.cloud/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    carthago.cloud/jenkins: example
spec:
  repository:
    url: https://github.com/jenkinsci/kubernetes-operator.git
    branch: master
    targets: "cicd/jobs/*.jenkins"
  agentRef:
    name: agent-name
    namespace: default

To find out how to create and configure Kubernetes Agents using JenkinsKubernetesAgent CR, visit this page.

Authentication

If your GitHub repository is private, or you need to authenticate to any other applications, you have to configure SSH or username/password authentication.

Using SSH Keys is a more secure option, while username/password method is good enough with solutions incorporating central location redirecting users for authentication or with multistep authentication.

Basic SSH authentication

Generate SSH Keys

To generate a private/public pair of keys we need to type the following command into our repo

$ ssh-keygen -t rsa -b 2048 -C "[email protected]"
Generating public/private rsa key pair.

Next, you will be asked for a file name. Use the default path:

Enter file in which to save the key (/Users/johndoe/.ssh/id_rsa):

Now you can optionally set a password. In that case leave the password empty.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/johndoe/.ssh/id_rsa.
Your public key has been saved in /Users/johndoe/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:M0HppoJgPAhw2NYS0SVuYpyllpA7MbFfu+U0F0y8EDA [email protected]
The key's randomart image is:
+---[RSA 2048]----+
|      o. .   o   |
|     o .o + +    |
|   ...+o . + .   |
|  E .ooo .. .    |
|   o  + S . .o   |
|  .  o   . o. +  |
| .    .   .o.=   |
|  . .... [email protected]  |
| .oo.o .o+=@*=   |
+----[SHA256]-----+

We need PEM format:

$ ssh-keygen -p -f /Users/johndoe/.ssh/id_rsa -m pem

Key has comment '[email protected]'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

Now we have to copy the content of the public key file:

$ pbcopy < /Users/johndoe/.ssh/id_rsa.pub

and add it to our GitHub repository.

In your project repository enter Settings > Deploy keys and click “Add deploy key”. Give it some title and paste our copied key.

Key deploymnet

We can see the key was added.

Deployed key

Configure SSH authentication

First, create a Secret file with your GitHub username and generated SSH private key.

Copy contents of the id_rsa file (not id_rsa.pub):

macOS

$ pbcopy < /Users/johndoe/.ssh/id_rsa

Linux

$ xsel -b -i < /Users/johndoe/.ssh/id_rsa

And paste it into privateKey field like this:

apiVersion: v1
kind: Secret
metadata:
  name: k8s-ssh
  labels:
    "carthago.cloud/jenkinsseedjob": "example-jenkins-seedjob"
    "jenkins.io/credentials-type": "basicSSHUserPrivateKey"
stringData:
  privateKey: |
    -----BEGIN PRIVATE KEY-----
    MIIJKAIBAAKCAgEAxxDpleJjMCN5nusfW/AtBAZhx8UVVlhhhIKXvQ+dFODQIdzO
    oDXybs1zVHWOj31zqbbJnsfsVZ9Uf3p9k6xpJ3WFY9b85WasqTDN1xmSd6swD4N8
    ...    
  username: github_user_name

Note: you have to specify the name of the JenkinsSeedJob Custom Resource in the labels to connect the secret with respective Seed Job.

Second, create a Kubernetes Secret resource from this Secret config file.

$ kubectl -n jenkins apply -f mySecret.yaml

You can verify that Operator recognizes the secret in the Credentials tab.

SSH Secret in Credentials

In the seedJob you added to your JenkinsSeedJob Custom Resource file you can specify basicSSHUserPrivateKey as credentialType and add the name of the Secret, with your GitHub username and SSH key, in credentialID field’s value.

apiVersion: carthago.cloud/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    carthago.cloud/jenkins: example
spec:
  repository:
    url: [email protected]:your-account/your-repository.git
    branch: master
    targets: "cicd/jobs/*.jenkins"
    credentialType: basicSSHUserPrivateKey
    credentialID: k8s-ssh

Now we need to specify newly created credentials in your Seed Job definition file. They will be used to connect to specified GitHub repository. Don’t forget to also change the url for SSH:

#!/usr/bin/env groovy

pipelineJob('build-carthago-operator-for-jenkins') {
    displayName('Build Carthago Operator for Jenkins')

    definition {
        cpsScm {
            scm {
                git {
                    remote {
                        url('[email protected]:your-account/your-repo.git')
                        credentials('k8s-sh')
                    }
                    branches('*/master')
                }
            }
            scriptPath('cicd/pipelines/build.jenkins')
        }
    }
}

Username & password authentication

First, create a Secret file with your GitHub credentials.

apiVersion: v1
kind: Secret
metadata:
  name: k8s-user-pass
  labels:
    "carthago.cloud/jenkinsseedjob": "example-jenkins-seedjob"
stringData:
  username: github_user_name
  password: password_or_token

Second, create a Kubernetes Secret resource from this file.

$ kubectl -n jenkins apply -f mySecret.yaml

In the seedJob you added to your Jenkins Custom Resource file you can specify usernamePassword as credentialType and add the name of the Secret, with your GitHub credentials, in the credentialID field’s value.

apiVersion: carthago.cloud/v1beta1
kind: JenkinsSeedJob
metadata:
  name: example-jenkins-seedjob
  namespace: default
  labels:
    carthago.cloud/jenkins: example
spec:
  repository:
    url: https://github.com/your-github-account/your-github-repository.git
    branch: master
    targets: "cicd/jobs/*.jenkins"
    credentialType: usernamePassword
    credentialID: k8s-user-pass