[CKA] Cluster Architecture, Installation & Configuration (1)

• CKA 시험 비중 (25%)
• ETCD

* 쿠버네티스 클러스터의 정보를 저장하는 곳 (메모리)
* key: value 형태의 데이터 스토리지
* 모든 ETCD 데이터는 ETCD 데이터베이스 파일에 보관 -> /var/lib/etcd
* ETCD 관리 명령: etcdctl
* 마스터 3대, ETCD 3대 -> 고가용성(HA)

 

• 실습1

What is the version of ETCD running on the cluster?

$ kubectl describe pod etcd-controlplane -n kube-system

 

Image: registry.k8s.io/etcd:3.5.6-0

 

• 실습2

At what address can you reach the ETCD cluster from the controlplane node?

$ kubectl describe pod etcd-controlplane -n kube-system​

 

--listen-client-urls=https://127.0.0.1:2379,https://192.10.151.3:2379

 

• 실습3

Where is the ETCD server certificate file located?

$ kubectl describe pod etcd-controlplane -n kube-system​

 

--cert-file=/etc/kubernetes/pki/etcd/server.crt

 

• 실습4

Where is the ETCD CA Certificate file located?

$ kubectl describe pod etcd-controlplane -n kube-system​

 

--trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

 

• 실습5

The master node in our cluster is planned for a regular maintenance reboot tonight. While we do not anticipate anything to go wrong, we are required to take the necessary backups. Take a snapshot of the ETCD database using the built-in snapshot functionality. Store the backup file at location /opt/snapshot-pre-boot.db

$ export ETCDCTL_API=3
$ etcdctl snapshot
$ cat /etc/kubernetes/manifests/etcd.yaml
$ etcdctl snapshot save --endpoints=127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key /opt/snapshot-pre-boot.db​

--endpoints: - --listen-client-urls=https://127.0.0.1:2379,https://192.10.151.3:2379
--cacert: - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
--cert: - --cert-file=/etc/kubernetes/pki/etcd/server.crt
--key: - --key-file=/etc/kubernetes/pki/etcd/server.key

 

• 실습6

Restore the original state of the cluster using the backup file.

$ etcdctl snapshot restore --data-dir /var/lib/etcd-from-backup /opt/snapshot-pre-boot.db
2023-05-13 05:28:17.739536 I | mvcc: restore compact to 1674
2023-05-13 05:28:17.747075 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32

$ ls /var/lib/etcd-from-backup/
member

$ vi /etc/kubernetes/manifests/etcd.yaml
volumes:
- hostPath:
       path: /etc/kubernetes/pki/etcd
       type: DirectoryOrCreate
   name: etcd-certs
- hostPath:
       path: /var/lib/etcd-from-backup
       type: DirectoryOrCreate
   name: etcd-data

$ kubectl get pods -n kube-system (새 정보를 기반으로 생성하므로 시간이 좀 걸린다.)

$ kubectl delete pod etcd-controlplane -n kube-system (클러스터가 재구성되었지만 etcd-controlplane이 재생성되진 않았다. 그래서 파드 삭제 후 재생성)​

 

• Kubernetes OS Upgrade

• 실습1

We need to take node01 out for maintenance. Empty the node of all applications and mark it unschedulable.

$ kubectl drain node01 --ignore-daemonsets​

 

• 실습2

The maintenance tasks have been completed. Configure the node node01 to be schedulable again.

$ kubectl uncordon node01

 

• 실습3

hr-app is a critical app and we do not want it to be removed and we do not want to schedule any more pods on node01.Mark node01 as unschedulable so that no new pods are scheduled on this node.
Make sure that hr-app is not affected.

$ kubectl cordon node01

 

• Kubernetes Cluster Upgrade

* 여긴 외울 수 없으니 docs를 적극 참고하자.
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/

 

• 실습1

This lab tests your skills on upgrading a kubernetes cluster. We have a production cluster with applications running on it. Let us explore the setup first.
What is the current version of the cluster?

$ kubectl get nodes

 

• 실습2

What is the latest stable version of Kubernetes as of today?
Look at the remote version in the output of the kubeadm upgrade plan command.

$ kubeadm upgrade plan

 

• 실습3

We will be upgrading the controlplane node first. Drain the controlplane node of workloads and mark it 
UnSchedulable.

$ kubectl drain controlplane --ignore-daemonsets

 

• 실습4

Upgrade the controlplane components to exact version v1.26.0
Upgrade the kubeadm tool (if not already), then the controlplane components, and finally the kubelet. Practice referring to the Kubernetes documentation page.Note: While upgrading kubelet, if you hit dependency issues while running the apt-get upgrade kubelet command, use the apt install kubelet=1.26.0-00 command instead.

* 컨트롤 플레인 노드 업그레이드

$ cat /etc/*release*
$ apt update
$ apt-cache madison kubeadm
# 목록에서 최신 버전(1.26)을 찾는다
# 1.26.x-00과 같아야 한다. 여기서 x는 최신 패치이다.

$ apt-mark unhold kubeadm && apt-get update && apt-get install -y kubeadm=1.26.0-00 && apt-mark hold kubeadm

$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.0", GitCommit:"b46a3f887ca979b1a5d14fd39cb1af43e7e5d12d", GitTreeState:"clean", BuildDate:"2022-12-08T19:57:06Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}

$ kubeadm upgrade plan

$ sudo kubeadm upgrade apply v1.26.0

-- kubelet과 kubectl 업그레이드

# replace x in 1.26.x-00의 x를 최신 패치 버전으로 바꾼다
$ apt-mark unhold kubelet kubectl && apt-get update && apt-get install -y kubelet=1.26.0-00 kubectl=1.26.0-00 && apt-mark hold kubelet kubectl

# kubelet을 다시 시작한다.
$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet​

 

• 실습5

Mark the controlplane node as "Schedulable" again.

$ kubectl uncordon controlplane

 

• 실습6

Next is the worker node. Drain the worker node of the workloads and mark it UnSchedulable.

$ kubectl drain node01 --ignore-daemonsets
$ kubectl get nodes
$ kubectl get pods -o wide​

 

• 실습7

Upgrade the worker node to the exact version v1.26.0

* 워커 노드 업그레이드
* kubeadm 업그레이드

$ ssh node01
$ apt-mark unhold kubeadm && apt-get update && apt-get install -y kubeadm=1.26.0-00 && apt-mark hold kubeadm
$ sudo kubeadm upgrade node​

* kubelet과 kubectl 업그레이드

$ apt-mark unhold kubelet kubectl && apt-get update && apt-get install -y kubelet=1.26.0-00 kubectl=1.26.0-00 && apt-mark hold kubelet kubectl
$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet
$ exit
$ kubectl get nodes​

 

• 실습8

Remove the restriction and mark the worker node as schedulable again.

$ kubectl uncordon node01