As of version 1.1.4, PKS uses version 2.0 of the NCP pod. This pod is responsible for performing managing resources in NSX-T, which includes creating routers, switches, allocating IPs, etc. Unfortunately, in version 2.0 of the NCP pod there is an issue where each time the pod is restarted, it will create duplicate virtual servers in NSX-T. This is especially problematic because PKS uses a small load balancer which can only have 10 virtual servers. Let’s see this in action.
First I’m going to explain the issue, but if you’d like to go directly to the fix, see the section Preventing the duplicate virtual servers below.
I’m going to show how to create a Jenkins pipeline to deploy an application to Pivotal Container Service (PKS). You can also check out Route to Cloud’s article on how to use Jenkins X and PKS. In order to keep the length down, I’m going to cut some corners such as installing most things on the Jenkins master and not using any slaves so please don’t consider this production grade.
There will eventually be a KB for this, but in the meantime I wanted to show how you can clean up PKS created NSX-T resources in NSX-T manager. You may need to do this when a PKS cluster fails to create or delete properly, and doing it manually is tedious and error prone.
Please make sure you have valid backups of the NSX-T manager before proceeding.
- curl -LO https://storage.googleapis.com/pks-releases/pks_cleanup_linux
- chmod +x pks_cleanup_linux
- sudo mv pks_cleanup_linux /usr/local/bin/pks_cleanup