Using Jenkins with PKS

Posted by

I’m going to show how to create a Jenkins pipeline to deploy an application to Pivotal Container Service (PKS). You can also check out Route to Cloud’s article on how to use Jenkins X and PKS. In order to keep the length down, I’m going to cut some corners such as installing most things on the Jenkins master and not using any slaves so please don’t consider this production grade.

Versions Used

  • PKS 1.1.4
  • Harbor 1.5.2
  • Jenkins 2.121.2 on CentOS 7.5

GitHub Overview

The GitHub repo can be found here. If you’d like to follow along, you can fork the repo and make the necessary changes for your environment. I’m go breakdown each of these files below:

2018-08-08_21-41-46.gif

main.go

The application that will be run in PKS.

main_test.go

The application’s test suite.

Dockerfile

A simple Dockerfile that builds the application and exposes it on port 8181.

kubernetes.yaml

A YAML file that creates a service and deployment within PKS. The deployment’s image is set to $DOCKER_IMAGE_NAME:$BUILD_NUMBER, which will be provided by the Jenkins pipeline.

Jenkinsfile

This file is scanned each time the Jenkins job runs and defines the Jenkins pipeline. It specifies the following stages:

  1. Compiling the source code
  2. Creating the docker imagge
  3. Pushing the docker image to Harbor
  4. Deploy the YAML file to PKS
  5. Getting the application’s IP address

In step 4 I’m deleting the previous instance of the Kuberntes service and deployment and deploying a new one. I’m only doing this because I don’t know how to use the Jenkins Kubernetes plugin to udpate the deployment’s image yet. If I can’t figure that out, I may update the pipeline to update the deployment’s image using the kubectl set image command.

Jenkins OS Configuration

As explained previously, I’m only going to be using a Jenkins master server running on CentOS 7.5. Here is how I prepared the OS.

  1. Install Jenkins with the recommended plugins. Additionally install install Kubernetes Continuous Deploy Plugin
  2. yum -y install docker sshpass git
  3. Install Go
  4. useradd jenkins
  5. passwd jenkins
  6. groupadd docker
  7. usermod -aG docker jenkins
  8. systemctl enable docker
  9. systemctl start docker

Jenkins Setup

Add Environmental Variables

  1. Manage Jenkins
  2. Configure System
  3. Check Environment variables
  4. Add

Create a new environment variable for my VM that runs has all of my BOSH / PKS tools. We will SSH into this machine to kubectl commands:

2018-08-08_21-21-20.gif

Select Save

Add Credentials

Now we need to add various credentials that we will use throughout our pipeline. These credentials are referenced in the Jenkins file so if you decide to change any of the IDs, you’ll need to make the corresponding change in the Jenkins file.

Add Kubernetes Credentials

  1. Credentials
  2. Select global
  3. Add Credentials
  4. Kind: Kubernetes configuration (kubeconfig). You can get this by running pks get-credentials <cluster name> and copying the context of ~/.kube/config
  5. Enter a name and ID
  6. For Kubeconfig, select Enter directly and paste in your Kubernetes config file

It should look like this when your finished:

2018-08-11_16-07-48.gif

Press OK to save.

Add credentials for Harbor or Docker Hub 

  1. Credentials
  2. Select global
  3. Add Credentials
  4. Kind: Username with password
  5. Enter the Username, Password, ID and Description

Create a credential for your PKS client machine

This credential is for logging into your PKS client machine and is where you would run the pks utility. In my lab it’s named pks-client.

2018-08-11_16-04-39.gif

My credentials look like this:

2018-08-11_16-08-38.gif

Create Jenkins Job

Prior to creating the Jenkins job you will need a Personal Access Token from GitHub. You can do this by going to github.com and selecting

  1. Your icon in the upper-right
  2. Settings
  3. Developer Settings
  4. Personal Access Tokens
  5. Generate new token
  6. Give it the admin:repo_hook permission
  7. Generate token

Now that we have our token, we can go into Jenkins and create our Jenkins job. Select

  1. Select New Item
  2. Name it (I named mine go-cicd-kubernetes)
  3. Select Multibranch Pipeline and press OK.
  4. Under Branch Sources select Add source > GitHub
  5. Next to Credentials select Add > Jenkins
  6. Enter your GitHub username and paste in your personal access token into the password field.
  7. Enter github for the ID and Description

Once you’re done it should look like this:

2018-08-08_21-33-10.gif

Press Add to create the credentials. Now that we have created our credentials we still need to select it like so:

2018-08-08_21-34-26.gif

Enter your GitHub name in the Owner field and after a brief pause the Repository field should show all of your repositories. Select the go-cicd-kubernetes repository:

2018-08-08_21-36-55.gif

Select Save.

Running the job

After you press Save in the previous step a build will automatically start:

2018-08-12_18-10-19.gif

Here we can see each of the steps defined in our Jenkinsfile:

2018-08-12_18-11-21.gif

You can look at the job’s console output for all of the details or highlight a step and select logs. Let’s look at the logs for the last step so we can get the Service’s IP:

2018-08-12_18-13-26.gif

Now we can click the link and be taking to the application. The IP above is a VIP on an NSX-T Load Balancer. Let’s pretend this is the first commit and we are building our a new application to monitor PKS. At this point the only thing we have is a navigation bar that says PKS Monitor:

2018-08-12_18-15-53.gif

For the next step we will add Clusters to the navigation bar. I updated the code, pushed it to Github and will now run the Jenkins job again by selecting Build Now. Below we can see that the build was successful:

2018-08-12_18-21-58.gif

Now when we access the site we can see the new Clusters entry:

2018-08-12_18-24-09.gif

In Harbor we can see the images that were built:

2018-08-12_18-38-40.gif

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s