There will eventually be a KB for this, but in the meantime I wanted to show how you can clean up PKS created NSX-T resources in NSX-T manager. You may need to do this when a PKS cluster fails to create or delete properly, and doing it manually is tedious and error prone.

Please make sure you have valid backups of the NSX-T manager before proceeding.

Installation

1. curl -LO https://storage.googleapis.com/pks-releases/pks_cleanup_linux
2. chmod +x pks_cleanup_linux
3. sudo mv pks_cleanup_linux /usr/local/bin/pks_cleanup

Execution

You can use the utility’s help system to see all of the options but the following shows an example run and has the required options:

pks_cleanup –mgr-ip=192.168.111.46 \
–username=admin \
–password=VMware1! \
–cluster=pks-18ef47d8-d4ac-4d6c-9d77-301860c3a98f \
–read-only=false \
–pks \
–floating-ip-pool-id=5a35b05c-70d4-4337-9f8e-b8b8533476c7 \
–ip-block-id=d5aab712-4b83-4690-a16f-f6a3583c9056

• mgr-ip is your NSX-T manager
• username and password for NSX-T manager
• cluster is pks followed by the PKS cluster UUID you’re cleaning up
• Setting read-only to true will show you what will be deleted but won’t actually delete anything.
• pks specifies that you want to delete PKS created resources
• floating-ip-pool-id is defined in NSX-T manager > Inventory > Groups > IP Pools
• ip-block-id is the master/worker node IP pool defined in NSX-T manager > DDI > IPAM

You can ignore the following messages:

ResourceDeleteFunc(): unrecognized resource type: TIER0
ResourceCollectFunc(): unrecognized resource type: NatRule
ResourceDeleteFunc(): unrecognized resource type: NatRule

Script

Since a lot of the parameters will usually be the same, I created this script so that you can just specify the PKS cluster UUID and the read-only mode.

Create file named pks-cleanup.sh and paste in the contents below. You’ll need to adjust the NSX constants at the beginning to match your environment:

#!/usr/bin/env bash
NSX_MANAGER_USERNAME=admin
NSX_MANAGER_PASSWORD=VMware1!
NSX_MANAGER_IP=192.168.100.110
PKS_CLUSTER_UUID=$1
READ_ONLY=$2

# FLOATING_IP_POOL_ID is LB pool defined in NSX-T manager > Inventory > Groups > IP Pools
FLOATING_IP_POOL_ID=725ed0d6-c197-4b2b-ac5e-8c4981caa5fb

# IP_BLOCK_ID is the node ip pool defined in NSX-T manager > DDI > IPAM
IP_BLOCK_ID=ad51f33b-e7ae-45f5-81dd-fd481177f1dc

# Usage:
# Read-only mode: ./pks-cleanup.sh <PKS_CLUSTER_UUID> true
# Delete mode: ./pks-cleanup.sh <PKS_CLUSTER_UUID> false

pks_cleanup --mgr-ip ${NSX_MANAGER_IP} \
            --username ${NSX_MANAGER_USERNAME} \
            --password ${NSX_MANAGER_PASSWORD} \
            --cluster "pks-${PKS_CLUSTER_UUID}" \
            --read-only=${READ_ONLY} \
            --pks \
            --floating-ip-pool-id ${FLOATING_IP_POOL_ID} \
            --ip-block-id ${IP_BLOCK_ID}

Make the script executable

chmod +x pks-cleanup.sh

Run in read-only mode to see what would be deleted in NSX-T

./pks-cleanup.sh <PKS_CLUSTER_UUID> true

Run in write mode to delete items in NSX-T

./pks-cleanup.sh <PKS_CLUSTER_UUID> false