Kubernetes 1.3 HA Walkthrough – etcd

Table of Contents

You can find all of the config files on the GitHub page.

Install etcd

For info on etcd see the GitHub page.

Perform the following on each of the etcd nodes.

mkdir -p /etc/etcd/
curl -L https://github.com/coreos/etcd/releases/download/v3.0.1/etcd-v3.0.1-linux-amd64.tar.gz -o etcd-v3.0.1-linux-amd64.tar.gz
tar -xvf etcd-v3.0.1-linux-amd64.tar.gz
cp etcd-v3.0.1-linux-amd64/etcd* /usr/bin/
mkdir -p /var/lib/etcd

Configure etcd

Perform the following on each of the etcd nodes.

kube-etcd0

/etc/systemd/system/etcd.service

[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
ExecStart=/usr/bin/etcd --name kube-etcd0 \
 --cert-file=/etc/etcd/kubernetes.pem \
 --key-file=/etc/etcd/kubernetes-key.pem \
 --peer-cert-file=/etc/etcd/kubernetes.pem \
 --peer-key-file=/etc/etcd/kubernetes-key.pem \
 --trusted-ca-file=/etc/etcd/ca.pem \
 --peer-trusted-ca-file=/etc/etcd/ca.pem \
 --initial-advertise-peer-urls https://kube-etcd0.vmware.local:2380 \
 --listen-peer-urls https://kube-etcd0.vmware.local:2380 \
 --listen-client-urls https://kube-etcd0.vmware.local:2379 \
 --advertise-client-urls https://kube-etcd0.vmware.local:2379 \
 --initial-cluster-token etcd-cluster-0 \
 --initial-cluster kube-etcd0=https://kube-etcd0.vmware.local:2380,kube-etcd1=https://kube-etcd1.vmware.local:2380,kube-etcd2=https://kube-etcd2.vmware.local:2380 \
 --initial-cluster-state new \
 --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

kube-etcd1

/etc/systemd/system/etcd.service

[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
ExecStart=/usr/bin/etcd --name kube-etcd1 \
 --cert-file=/etc/etcd/kubernetes.pem \
 --key-file=/etc/etcd/kubernetes-key.pem \
 --peer-cert-file=/etc/etcd/kubernetes.pem \
 --peer-key-file=/etc/etcd/kubernetes-key.pem \
 --trusted-ca-file=/etc/etcd/ca.pem \
 --peer-trusted-ca-file=/etc/etcd/ca.pem \
 --initial-advertise-peer-urls https://kube-etcd1.vmware.local:2380 \
 --listen-peer-urls https://kube-etcd1.vmware.local:2380 \
 --listen-client-urls https://kube-etcd1.vmware.local:2379 \
 --advertise-client-urls https://kube-etcd1.vmware.local:2379 \
 --initial-cluster-token etcd-cluster-0 \
 --initial-cluster kube-etcd0=https://kube-etcd0.vmware.local:2380,kube-etcd1=https://kube-etcd1.vmware.local:2380,kube-etcd2=https://kube-etcd2.vmware.local:2380 \
 --initial-cluster-state new \
 --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

kube-etcd2

/etc/systemd/system/etcd.service


[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
ExecStart=/usr/bin/etcd --name kube-etcd2 \
 --cert-file=/etc/etcd/kubernetes.pem \
 --key-file=/etc/etcd/kubernetes-key.pem \
 --peer-cert-file=/etc/etcd/kubernetes.pem \
 --peer-key-file=/etc/etcd/kubernetes-key.pem \
 --trusted-ca-file=/etc/etcd/ca.pem \
 --peer-trusted-ca-file=/etc/etcd/ca.pem \
 --initial-advertise-peer-urls https://kube-etcd2.vmware.local:2380 \
 --listen-peer-urls https://kube-etcd2.vmware.local:2380 \
 --listen-client-urls https://kube-etcd2.vmware.local:2379 \
 --advertise-client-urls https://kube-etcd2.vmware.local:2379 \
 --initial-cluster-token etcd-cluster-0 \
 --initial-cluster kube-etcd0=https://kube-etcd0.vmware.local:2380,kube-etcd1=https://kube-etcd1.vmware.local:2380,kube-etcd2=https://kube-etcd2.vmware.local:2380 \
 --initial-cluster-state existing \
 --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

Enable and start etcd on each etcd node

systemctl daemon-reload
systemctl enable etcd
systemctl restart etcd

Verify that the service started correctly

systemctl status etcd –no-pager

Verify cluster state

etcdctl –ca-file=/etc/etcd/ca.pem –endpoint https://kube-etcd0.vmware.local:2379 cluster-health

member 2673865ccad1f13b is healthy: got healthy result from https://192.168.3.180:2379
member 3a8b584f84134f4f is healthy: got healthy result from https://192.168.3.181:2379
member 8c75d01c2de98466 is healthy: got healthy result from https://192.168.3.179:2379
cluster is healthy

Create flannel network

When each of our Kubernetes worker nodes come online, they will be given a network from etcd. Kubernetes pods will then be given IPs from these networks. We will see more of this in a later post, but for now let’s just create the networks in etcd.

Create flanneld etcd config

etcdctl –ca-file=/etc/etcd/ca.pem –endpoint https://kube-etcd0.vmware.local:2379 set /coreos.com/network/config ‘{“Network”: “172.16.0.0/16”}’

Get flanneld etcd config

etcdctl –ca-file=/etc/etcd/linuxca-inter.pem –endpoint https://kube-etcd0.vmware.local:2379 get /coreos.com/network/config

Advertisements


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s