I’ve been using pfSense in my home lab for about 10 years now and it has worked great with every product from vCD 1.0 to vSphere with Kubernetes. However, when I tried to install vSphere with Tanzu using HAProxy instead of NSX-T I kept see connection timeout errors in the logs. The enabling of the supervisor cluster was successful, but I was unable to create any Tanzu Kubernetes Clusters.
All of my pfSense firewall rules are set to allow all and I’ll create a deny rule if I want to see what happens when connectivity isn’t present. I was told that others were seeing this behavior and that turning off the pfSense firewall resolves the issue. I couldn’t turn off the firewall since it also disables NAT, which I need. I did some experimenting and found out I needed to create two additional firewall rules. The first rule was a floating rule that looks likes this:
You’ll notice that in the Currently viewing drop-down box that Floating is selected and that there is a little gear next to the green check mark. This indicates that an advanced option was changed from its default. Edit the firewall rule and you’ll see that the following advanced options have been modified:
- TCP Flags is set to Any flags
- State type is set to Sloppy
Then I had to create an identical rule for the workload network.
I was able to deploy Tanzu Kubernetes Clusters after these changes were made. I had no issues with pfSense for so long I forgot that if you go to Firewall > Rules, you’ll see a button to view the firewall logs. You can also get to there by going to http://pfsense-url/status_logs_filter.php
It’s here where I saw that traffic was being blocked, which was surprising to me since I knew that I had created allow all rules for each interface.
Since each person’s lab is most likely to be different I can’t guarantee that this will work for you, but I hope it helps.