Using Jenkins with PKSPosted: August 13, 2018
I’m going to show how to create a Jenkins pipeline to deploy an application to Pivotal Container Service (PKS). You can also check out Route to Cloud’s article on how to use Jenkins X and PKS. In order to keep the length down, I’m going to cut some corners such as installing most things on the Jenkins master and not using any slaves so please don’t consider this production grade.
- PKS 1.1.4
- Harbor 1.5.2
- Jenkins 2.121.2 on CentOS 7.5
The GitHub repo can be found here. If you’d like to follow along, you can fork the repo and make the necessary changes for your environment. I’m go breakdown each of these files below:
The application that will be run in PKS.
The application’s test suite.
A simple Dockerfile that builds the application and exposes it on port 8181.
A YAML file that creates a service and deployment within PKS. The deployment’s image is set to $DOCKER_IMAGE_NAME:$BUILD_NUMBER, which will be provided by the Jenkins pipeline.
This file is scanned each time the Jenkins job runs and defines the Jenkins pipeline. It specifies the following stages:
- Compiling the source code
- Creating the docker imagge
- Pushing the docker image to Harbor
- Deploy the YAML file to PKS
- Getting the application’s IP address
In step 4 I’m deleting the previous instance of the Kuberntes service and deployment and deploying a new one. I’m only doing this because I don’t know how to use the Jenkins Kubernetes plugin to udpate the deployment’s image yet. If I can’t figure that out, I may update the pipeline to update the deployment’s image using the kubectl set image command.
Jenkins OS Configuration
As explained previously, I’m only going to be using a Jenkins master server running on CentOS 7.5. Here is how I prepared the OS.
- Install Jenkins with the recommended plugins. Additionally install install Kubernetes Continuous Deploy Plugin
- yum -y install docker sshpass git
- Install Go
- useradd jenkins
- passwd jenkins
- groupadd docker
- usermod -aG docker jenkins
- systemctl enable docker
- systemctl start docker
Add Environmental Variables
- Manage Jenkins
- Configure System
- Check Environment variables
Create a new environment variable for my VM that runs has all of my BOSH / PKS tools. We will SSH into this machine to kubectl commands:
Now we need to add various credentials that we will use throughout our pipeline. These credentials are referenced in the Jenkins file so if you decide to change any of the IDs, you’ll need to make the corresponding change in the Jenkins file.
Add Kubernetes Credentials
- Select global
- Add Credentials
- Kind: Kubernetes configuration (kubeconfig). You can get this by running pks get-credentials <cluster name> and copying the context of ~/.kube/config
- Enter a name and ID
- For Kubeconfig, select Enter directly and paste in your Kubernetes config file
It should look like this when your finished:
Press OK to save.
Add credentials for Harbor or Docker Hub
- Select global
- Add Credentials
- Kind: Username with password
- Enter the Username, Password, ID and Description
Create a credential for your PKS client machine
This credential is for logging into your PKS client machine and is where you would run the pks utility. In my lab it’s named pks-client.
My credentials look like this:
Create Jenkins Job
Prior to creating the Jenkins job you will need a Personal Access Token from GitHub. You can do this by going to github.com and selecting
- Your icon in the upper-right
- Developer Settings
- Personal Access Tokens
- Generate new token
- Give it the admin:repo_hook permission
- Generate token
Now that we have our token, we can go into Jenkins and create our Jenkins job. Select
- Select New Item
- Name it (I named mine go-cicd-kubernetes)
- Select Multibranch Pipeline and press OK.
- Under Branch Sources select Add source > GitHub
- Next to Credentials select Add > Jenkins
- Enter your GitHub username and paste in your personal access token into the password field.
- Enter github for the ID and Description
Once you’re done it should look like this:
Press Add to create the credentials. Now that we have created our credentials we still need to select it like so:
Enter your GitHub name in the Owner field and after a brief pause the Repository field should show all of your repositories. Select the go-cicd-kubernetes repository:
Running the job
After you press Save in the previous step a build will automatically start:
Here we can see each of the steps defined in our Jenkinsfile:
You can look at the job’s console output for all of the details or highlight a step and select logs. Let’s look at the logs for the last step so we can get the Service’s IP:
Now we can click the link and be taking to the application. The IP above is a VIP on an NSX-T Load Balancer. Let’s pretend this is the first commit and we are building our a new application to monitor PKS. At this point the only thing we have is a navigation bar that says PKS Monitor:
For the next step we will add Clusters to the navigation bar. I updated the code, pushed it to Github and will now run the Jenkins job again by selecting Build Now. Below we can see that the build was successful:
Now when we access the site we can see the new Clusters entry:
In Harbor we can see the images that were built: