Kubernetes 1.3 HA Walkthrough – controllers

Posted by

Table of Contents

You can find all of the config files on the GitHub page.

Overview

This post will cover installing the Kubernetes API server, manager, scheduler and kubectl.

Install the Kubernetes API server, manager, scheduler and kubectl

Perform the following steps on each controller node.

mkdir -p /var/lib/kubernetes
mv /root/ca.pem /root/kubernetes.pem /root/kubernetes-key.pem /var/lib/kubernetes/

curl -O https://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/linux/amd64/kube-apiserver

curl -O https://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/linux/amd64/kube-controller-manager

curl -O https://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/linux/amd64/kube-scheduler

curl -O https://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/linux/amd64/kubectl

chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl

mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/bin/

Set up our authorization files

curl -O https://raw.githubusercontent.com/kelseyhightower/kubernetes-the-hard-way/master/token.csv

Change the password if you want. I’m changing it to VMware1!

sed -i ‘s/chAng3m3/VMware1!/g’ token.csv mv token.csv /var/lib/kubernetes/

curl -O https://raw.githubusercontent.com/kelseyhightower/kubernetes-the-hard-way/master/authorization-policy.jsonl mv authorization-policy.jsonl /var/lib/kubernetes/

Configure the API Server

These steps need to be performed individually per controller node.

kube-controller0

/etc/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/usr/bin/kube-apiserver \
 --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
 --advertise-address=192.168.3.176 \
 --allow-privileged=true \
 --apiserver-count=3 \
 --authorization-mode=ABAC \
 --authorization-policy-file=/var/lib/kubernetes/authorization-policy.jsonl \
 --bind-address=0.0.0.0 \
 --enable-swagger-ui=true \
 --etcd-cafile=/var/lib/kubernetes/ca.pem \
 --insecure-bind-address=0.0.0.0 \
 --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \
 --etcd-servers=https://kube-etcd0.vmware.local:2379,https://kube-etcd1.vmware.local:2379,https://kube-etcd2.vmware.local:2379 \
 --service-account-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --service-cluster-ip-range=172.16.0.0/24 \
 --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \
 --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --token-auth-file=/var/lib/kubernetes/token.csv \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

kube-controller1

/etc/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/usr/bin/kube-apiserver \
 --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
 --advertise-address=192.168.3.177 \
 --allow-privileged=true \
 --apiserver-count=3 \
 --authorization-mode=ABAC \
 --authorization-policy-file=/var/lib/kubernetes/authorization-policy.jsonl \
 --bind-address=0.0.0.0 \
 --enable-swagger-ui=true \
 --etcd-cafile=/var/lib/kubernetes/ca.pem \
 --insecure-bind-address=0.0.0.0 \
 --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \
 --etcd-servers=https://kube-etcd0.vmware.local:2379,https://kube-etcd1.vmware.local:2379,https://kube-etcd2.vmware.local:2379 \
 --service-account-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --service-cluster-ip-range=172.16.0.0/16 \
 --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \
 --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --token-auth-file=/var/lib/kubernetes/token.csv \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

kube-controller2

/etc/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/usr/bin/kube-apiserver \
 --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
 --advertise-address=192.168.3.178 \
 --allow-privileged=true \
 --apiserver-count=3 \
 --authorization-mode=ABAC \
 --authorization-policy-file=/var/lib/kubernetes/authorization-policy.jsonl \
 --bind-address=0.0.0.0 \
 --enable-swagger-ui=true \
 --etcd-cafile=/var/lib/kubernetes/ca.pem \
 --insecure-bind-address=0.0.0.0 \
 --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \
 --etcd-servers=https://kube-etcd0.vmware.local:2379,https://kube-etcd1.vmware.local:2379,https://kube-etcd2.vmware.local:2379 \
 --service-account-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --service-cluster-ip-range=172.16.0.0/16 \
 --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \
 --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --token-auth-file=/var/lib/kubernetes/token.csv \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Enable and start the service on each controller node

systemctl daemon-reload
systemctl enable kube-apiserver
systemctl restart kube-apiserver

Verify that the service is running

systemctl status kube-apiserver –no-pager

Configure the Manager service

Perform the following steps on each controller node.

kube-controller0

/etc/systemd/system/kube-controller-manager.service

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service

[Service]
ExecStart=/usr/bin/kube-controller-manager \
 --allocate-node-cidrs=true \
 --cluster-cidr=172.16.0.0/16 \
 --cluster-name=kubernetes \
 --leader-elect=true \
 --master=http://kube-controller1.vmware.local:8080 \
 --root-ca-file=/var/lib/kubernetes/ca.pem \
 --service-account-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --service-cluster-ip-range=172.16.0.0/16 \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

kube-controller1

/etc/systemd/system/kube-controller-manager.service

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service

[Service]
ExecStart=/usr/bin/kube-controller-manager \
 --allocate-node-cidrs=true \
 --cluster-cidr=172.16.0.0/16 \
 --cluster-name=kubernetes \
 --leader-elect=true \
 --master=http://kube-controller1.vmware.local:8080 \
 --root-ca-file=/var/lib/kubernetes/ca.pem \
 --service-account-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --service-cluster-ip-range=172.16.0.0/16 \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

kube-controller2

/etc/systemd/system/kube-controller-manager.service

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service

[Service]
ExecStart=/usr/bin/kube-controller-manager \
 --allocate-node-cidrs=true \
 --cluster-cidr=172.16.0.0/16 \
 --cluster-name=kubernetes \
 --leader-elect=true \
 --master=http://kube-controller2.vmware.local:8080 \
 --root-ca-file=/var/lib/kubernetes/ca.pem \
 --service-account-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
 --service-cluster-ip-range=172.16.0.0/16 \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Enable and start the service on each controller node

systemctl daemon-reload
systemctl enable kube-controller-manager
systemctl restart kube-controller-manager

Verify that the service is running

systemctl status kube-controller-manager –no-pager

Configure the Scheduler

Perform the following steps on each controller node.

kube-controller0

/etc/systemd/system/kube-scheduler.service

[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service

[Service]
ExecStart=/usr/bin/kube-scheduler \
 --leader-elect=true \
 --master=http://kube-controller0.vmware.local:8080 \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

kube-controller1

/etc/systemd/system/kube-scheduler.service

[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service

[Service]
ExecStart=/usr/bin/kube-scheduler \
 --leader-elect=true \
 --master=http://kube-controller1.vmware.local:8080 \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

kube-controller2

/etc/systemd/system/kube-scheduler.service

[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service

[Service]
ExecStart=/usr/bin/kube-scheduler \
 --leader-elect=true \
 --master=http://kube-controller2.vmware.local:8080 \
 --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Enable and start the service on each controller node

systemctl daemon-reload
systemctl enable kube-scheduler
systemctl restart kube-scheduler

Verify that the service is running

systemctl status kube-scheduler –no-pager

Verify Functionality

curl http://kube-controller0.vmware.local:8080/healthz

ok

curl http://kube-controller1.vmware.local:8080/healthz

ok

curl http://kube-controller2.vmware.local:8080/healthz

ok

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s