Kubernetes 1.3 HA Walkthrough – etcd

Table of Contents

• Overview
• Certificates
• etcd
• Controllers
• Workers
• HAProxy
• kubectl
• SkyDNS
• NGINX

You can find all of the config files on the GitHub page.

Install etcd

For info on etcd see the GitHub page.

Perform the following on each of the etcd nodes.

mkdir -p /etc/etcd/
curl -L https://github.com/coreos/etcd/releases/download/v3.0.1/etcd-v3.0.1-linux-amd64.tar.gz -o etcd-v3.0.1-linux-amd64.tar.gz
tar -xvf etcd-v3.0.1-linux-amd64.tar.gz
cp etcd-v3.0.1-linux-amd64/etcd* /usr/bin/
mkdir -p /var/lib/etcd

Configure etcd

Perform the following on each of the etcd nodes.

kube-etcd0

/etc/systemd/system/etcd.service

[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
ExecStart=/usr/bin/etcd --name kube-etcd0 \
 --cert-file=/etc/etcd/kubernetes.pem \
 --key-file=/etc/etcd/kubernetes-key.pem \
 --peer-cert-file=/etc/etcd/kubernetes.pem \
 --peer-key-file=/etc/etcd/kubernetes-key.pem \
 --trusted-ca-file=/etc/etcd/ca.pem \
 --peer-trusted-ca-file=/etc/etcd/ca.pem \
 --initial-advertise-peer-urls https://kube-etcd0.vmware.local:2380 \
 --listen-peer-urls https://kube-etcd0.vmware.local:2380 \
 --listen-client-urls https://kube-etcd0.vmware.local:2379 \
 --advertise-client-urls https://kube-etcd0.vmware.local:2379 \
 --initial-cluster-token etcd-cluster-0 \
 --initial-cluster kube-etcd0=https://kube-etcd0.vmware.local:2380,kube-etcd1=https://kube-etcd1.vmware.local:2380,kube-etcd2=https://kube-etcd2.vmware.local:2380 \
 --initial-cluster-state new \
 --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

kube-etcd1

/etc/systemd/system/etcd.service

[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
ExecStart=/usr/bin/etcd --name kube-etcd1 \
 --cert-file=/etc/etcd/kubernetes.pem \
 --key-file=/etc/etcd/kubernetes-key.pem \
 --peer-cert-file=/etc/etcd/kubernetes.pem \
 --peer-key-file=/etc/etcd/kubernetes-key.pem \
 --trusted-ca-file=/etc/etcd/ca.pem \
 --peer-trusted-ca-file=/etc/etcd/ca.pem \
 --initial-advertise-peer-urls https://kube-etcd1.vmware.local:2380 \
 --listen-peer-urls https://kube-etcd1.vmware.local:2380 \
 --listen-client-urls https://kube-etcd1.vmware.local:2379 \
 --advertise-client-urls https://kube-etcd1.vmware.local:2379 \
 --initial-cluster-token etcd-cluster-0 \
 --initial-cluster kube-etcd0=https://kube-etcd0.vmware.local:2380,kube-etcd1=https://kube-etcd1.vmware.local:2380,kube-etcd2=https://kube-etcd2.vmware.local:2380 \
 --initial-cluster-state new \
 --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

kube-etcd2

/etc/systemd/system/etcd.service

[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
ExecStart=/usr/bin/etcd --name kube-etcd2 \
 --cert-file=/etc/etcd/kubernetes.pem \
 --key-file=/etc/etcd/kubernetes-key.pem \
 --peer-cert-file=/etc/etcd/kubernetes.pem \
 --peer-key-file=/etc/etcd/kubernetes-key.pem \
 --trusted-ca-file=/etc/etcd/ca.pem \
 --peer-trusted-ca-file=/etc/etcd/ca.pem \
 --initial-advertise-peer-urls https://kube-etcd2.vmware.local:2380 \
 --listen-peer-urls https://kube-etcd2.vmware.local:2380 \
 --listen-client-urls https://kube-etcd2.vmware.local:2379 \
 --advertise-client-urls https://kube-etcd2.vmware.local:2379 \
 --initial-cluster-token etcd-cluster-0 \
 --initial-cluster kube-etcd0=https://kube-etcd0.vmware.local:2380,kube-etcd1=https://kube-etcd1.vmware.local:2380,kube-etcd2=https://kube-etcd2.vmware.local:2380 \
 --initial-cluster-state existing \
 --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

Enable and start etcd on each etcd node

systemctl daemon-reload
systemctl enable etcd
systemctl restart etcd

Verify that the service started correctly

systemctl status etcd –no-pager

Verify cluster state

etcdctl –ca-file=/etc/etcd/ca.pem –endpoint https://kube-etcd0.vmware.local:2379 cluster-health

member 2673865ccad1f13b is healthy: got healthy result from https://192.168.3.180:2379
member 3a8b584f84134f4f is healthy: got healthy result from https://192.168.3.181:2379
member 8c75d01c2de98466 is healthy: got healthy result from https://192.168.3.179:2379
cluster is healthy

Create flannel network

When each of our Kubernetes worker nodes come online, they will be given a network from etcd. Kubernetes pods will then be given IPs from these networks. We will see more of this in a later post, but for now let’s just create the networks in etcd.

Create flanneld etcd config

etcdctl –ca-file=/etc/etcd/ca.pem –endpoint https://kube-etcd0.vmware.local:2379 set /coreos.com/network/config ‘{“Network”: “172.16.0.0/16”}’

Get flanneld etcd config

etcdctl –ca-file=/etc/etcd/linuxca-inter.pem –endpoint https://kube-etcd0.vmware.local:2379 get /coreos.com/network/config