Jusene's Blog

kubeadm 构建k8s集群

Courtesy costs nothing.

kubernetes

Docker的出现几乎改变了整个传统的架构,微服务化,CI/CD系统,DEVOPS的概念正在有落地的可能,而kubernetes的出现,也几乎正在颠覆整个IT系统部署与流程的改进,2018年是kubernetes的元年,几乎击败了任何市面上的容器编排。而相对的kubernetes的构建的复杂度也令许多想要接触kubernetes的人望而却步,当然这不是google的初衷,为了简化kubernetes的部署,官方推出了kubeadm来简化完成kubernetes的部署:

首先了解下kubernetes的集群架构:

再者需要了解kubernetes的网络结构:

实践kubeadm部署

  • 环境:

master, etcd: 10.211.55.6
node1: 10.211.55.16
node2: 10.211.55.17

  • 前提:
  1. 基于主机名通信: /etc/hosts
  2. 时间同步
  3. 关闭firewalld和iptables
  • 安装配置步骤:
  1. etcd cluster, 仅master节点
  2. flannel,集群的所有节点
  3. 配置k8s的master节点: kube-apiserver, kube-scheduler, kube-controller-manager
  4. 配置k8s的各node节点: 先设定启动docker服务 kube-proxy,kubelet

master:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
# 配置yum源
~]# cat kubernetes.repo
[kubernetes]
name=kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=0
enable=1
~]# wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# 安装kubeadm,docker-ce,kubelet,kubectl
~]# yum install -y docker-ce kubelet kubeadm kubectl
# 国内网络无法访问gcr镜像库,所以我们需要在docker中加入代理
~]# vim /usr/lib/systemd/system/docker.service
...
[Service]
Environment="HTTPS_PROXY=http://www.ik8s.io:10080"
...
~]# systemctl daemon-reload && systemctl start docker
# 查看是否生效
~]# docker info
...
HTTPS Proxy: http://www.ik8s.io:10080
...
# kubernetes集群不允许开启swap,所以我们需要忽略这个错误
~]# vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false"
~]# systemctl enable docker
~]# systemctl enabel kubelet
~]# kubeadm init --kubernetes-version=v1.11.2 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 --ignore-preflight-errors=Swap # 需要下载镜像,时间有点长
# 配置kubectl与apiserver的认证
~]# mkdir -p $HOME/.kube
~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 检查健康状态
~]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health": "true"}
# 查看节点状态
~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
init NotReady master 4m v1.11.2
# 我们没有配置flannel网络,pod间网络不通,所以一直农田ready
~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml # 镜像也需要下载
~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
init Ready master 10m v1.11.2
~]# ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/kube-proxy-amd64 v1.11.1 d5c25579d0ff 7 weeks ago 97.8MB
k8s.gcr.io/kube-controller-manager-amd64 v1.11.1 52096ee87d0e 7 weeks ago 155MB
k8s.gcr.io/kube-scheduler-amd64 v1.11.1 272b3a60cd68 7 weeks ago 56.8MB
k8s.gcr.io/kube-apiserver-amd64 v1.11.1 816332bd9d11 7 weeks ago 187MB
k8s.gcr.io/coredns 1.1.3 b3b94275d97c 3 months ago 45.6MB
k8s.gcr.io/etcd-amd64 3.2.18 b8df3b177be2 5 months ago 219MB
quay.io/coreos/flannel v0.10.0-amd64 f0fad859c909 7 months ago 44.6MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 8 months ago 742kB
~]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE
coredns-78fcdf6894-n5kzm 1/1 Running 0 23h 10.244.0.3 init
coredns-78fcdf6894-rqfjq 1/1 Running 0 23h 10.244.0.2 init
etcd-init 1/1 Running 0 23h 10.211.55.6 init
kube-apiserver-init 1/1 Running 0 23h 10.211.55.6 init
kube-controller-manager-init 1/1 Running 0 23h 10.211.55.6 init
kube-flannel-ds-amd64-wrx27 1/1 Running 0 23h 10.211.55.6 init
kube-proxy-65h9b 1/1 Running 0 23h 10.211.55.6 init
kube-scheduler-init 1/1 Running 0 23h 10.211.55.6 init
~]# kubectl get deployment -n kube-system
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
coredns 2 2 2 2 23h
~}# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 23h
~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h

node:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# 配置yum源
~]# cat kubernetes.repo
[kubernetes]
name=kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=0
enable=1
~]# wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# 安装kubeadm,docker-ce,kubelet
~]# yum install -y docker-ce kubelet kubeadm
# 国内网络无法访问gcr镜像库,所以我们需要在docker中加入代理
~]# vim /usr/lib/systemd/system/docker.service
...
[Service]
Environment="HTTPS_PROXY=http://www.ik8s.io:10080"
...
~]# systemctl daemon-reload && systemctl start docker
# 查看是否生效
~]# docker info
...
HTTPS Proxy: http://www.ik8s.io:10080
...
# kubernetes集群不允许开启swap,所以我们需要忽略这个错误
~]# vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false"
~]# systemctl enable docker
~]# systemctl enabel kubelet
# 加入kubernetes集群
~]# kubeadm join 10.211.55.6:6443 --token a8uq9g.fwx2hubk66x68a5g --discovery-token-ca-cert-hash sha256:52385b782fcff923f8d794ff07f44f00b0a095c03a8361875ad33806df61f34d --ignore-preflight-errors=Swap # token与hash在kubeadm的完成后有提示,记得记录
master查看:
~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
init Ready master 23h v1.11.2
node1 Ready <none> 23h v1.11.2
node2 Ready <none> 23h v1.11.2

kubernetes简单应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
# 起一个pod
~]# kubectl run nginx-deploy --image=nginx:1.14-alpine --replicas=1 --port=80
~]# kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nginx-deploy 1 1 1 1 4h
~]# kubectl get pod
nginx-deploy-5b595999-mks9p 1/1 Running 0 4h
~]# kubectl get pods -o wide
kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-deploy-5b595999-mks9p 1/1 Running 0 4h 10.244.2.2 node2
~]# curl 10.244.2.2 # pod网络,所以集群内的pod可以通信
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
~]# kubectl expose deployment nginx-deploy --name=nginx --port=80 --target-port=80 --protocol=TCP # 部署个service,为nginx-deploy的pod提供统一的访问节点
~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.105.30.179 <none> 80/TCP 4h
~]# curl 10.105.30.179
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
# 开启一个pod终端
~]# kubectl run client --image=busybox -it --restart=Never
If you don't see a command prompt, try pressing enter.
/ # cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local localdomain
options ndots:5
/ # wget -O - http://nginx/ # pod内部可以根据service name进行通信
Connecting to nginx (10.105.30.179:80)
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
~]# dig -t A nginx.default.svc.cluster.local @10.96.0.10 # 集群内部也可以根据coredns来解析
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> -t A nginx.default.svc.cluster.local @10.96.0.10
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7916
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;nginx.default.svc.cluster.local. IN A
;; ANSWER SECTION:
nginx.default.svc.cluster.local. 5 IN A 10.105.30.179
;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Mon Sep 10 09:33:28 EDT 2018
;; MSG SIZE rcvd: 107
# 扩容
~]# kubectl run myapp --image=ikubernetes/myapp:v1 --replicas=2
~]# kubectl expose deployment myapp --name=myapp --port=80
~]# kubectl scale --replicas=5 deployment myapp
# 缩容
~]# kubectl scale --replicas=3 deployment myapp
# 滚动发布
~]# kubectl set image deployment myapp myapp=ikubernetes/myapp:v2
# 发布回滚
~]# kubectl rollout undo deployment myapp

本文标题:kubeadm 构建k8s集群

文章作者:Jusene

发布时间:2018年09月10日 - 20:09

最后更新:2018年09月10日 - 21:09

原始链接:http://jusene.me/2018/09/10/k8s-1/

许可协议: 署名-非商业性使用-禁止演绎 4.0 国际 转载请保留原文链接及作者。