Ansible部署Kubernetes集群

Kubernetes官网提供了多种场景的部署方案,例如kubeadm,参照:Creating a Cluster。但还是觉得ansible的部署最方便,k8s的contrib中提供了ansible部署脚本,使用方法参照:Kubernetes Ansible


1. 服务器构成

No Type IP Hostname OS
1 master 192.168.34.52 cts1 Centos7
2 etcd 192.168.34.52 cts1 Centos7
3 minion 192.168.34.180 cts2 Centos7

2. 安装ansible

在master上安装ansible:

1
[root@cts1 ~]# yum -y install ansible

安装后确认:
1
2
3
4
[root@cts1 ~]# ansible --version
ansible 2.2.0.0
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides


3. 设置ssh

该部署方案需要使用ssh免密登录,在所有机器上生成ssh key:

1
2
3
[root@cts1 ~]# ssh-keygen

[root@cts2 ~]# ssh-keygen

可以通过hostname [newname]修改hostname,设定两台机器的/etc/hosts :
1
2
3
4
5
6
7
[root@cts1 ~]# grep cts /etc/hosts
192.168.34.52 cts1
192.168.34.180 cts2

[root@cts2 ~]# grep cts /etc/hosts
192.168.34.52 cts1
192.168.34.180 cts2

在2台机器上都作如下设定,保证ssh可免密登录:
1
2
3
4
5
[root@cts1 ~]# ssh-copy-id -i cts1
[root@cts1 ~]# ssh-copy-id -i cts2

[root@cts2 ~]# ssh-copy-id -i cts1
[root@cts2 ~]# ssh-copy-id -i cts2

在ansible所安装的机器上,追加机器信息到/etc/ansible/hosts中:
1
2
3
[root@cts1 ~]# grep cts /etc/ansible/hosts
cts1
cts2

确认ansible正常动作:
1
2
3
4
5
6
7
8
9
10
[root@cts1 ~]# ansible cts1 -m ping
cts1 | SUCCESS => {
"changed": false,
"ping": "pong"
}
[root@cts1 ~]# ansible cts2 -m ping
cts2 | SUCCESS => {
"changed": false,
"ping": "pong"
}


4. 下载Contrib

1
[root@cts1 ~]# git clone https://github.com/kubernetes/contrib.git

5. 创建inventory文件

按照contrib/ansible说明,在inventory目录下创建inventory文件:

1
2
3
4
5
6
7
8
9
10
11
12
[root@cts1 inventory]# pwd
/data/contrib/ansible/inventory
[root@cts1 inventory]# vim inventory

[masters]
cts1

[etcd:children]
masters

[nodes]
cts2

开始误把etcd:children的值也配成cts1了,setup时报出Syntax Error while loading YAML的错误。


6. 安装所需package

1
[root@cts1 inventory]# yum -y install python-netaddr

7. 执行安装文件

执行cd scripts/ && ./deploy-cluster.sh时总是报出如下错误:

1
2
TASK [kubernetes : Read back the CA certificate] 
fatal: [cts2 -> cts1]: FAILED! => {"changed": false, "failed": true, "msg": "file not found: /etc/kubernetes/certs/ca.crt"}

于是按照Targeted runs逐个执行:
Etcd:
1
2
3
4
[root@cts1 scripts]# ./deploy-cluster.sh --tags=etcd

cts1 : ok=28 changed=2 unreachable=0 failed=0
cts2 : ok=6 changed=0 unreachable=0 failed=0

确认etcd:
1
2
3
4
5
6
7
8
[root@cts1 scripts]# etcd --version
etcd Version: 3.0.15
Git SHA: fc00305
Go Version: go1.6.3
Go OS/Arch: linux/amd64
[root@cts1 scripts]# etcdctl --version
etcdctl version: 3.0.15
API version: 2

Kubernetes master:
1
2
3
4
[root@cts1 scripts]# ./deploy-cluster.sh --tags=masters

cts1 : ok=67 changed=4 unreachable=0 failed=0
cts2 : ok=6 changed=0 unreachable=0 failed=0

Kubernetes nodes:
目前contrib版本中Install fluentd时因fluentd-es.yaml地址已不存在,会出错,解决办法参照:Support Fluentd migration to DaemonSet,将如下文件进行修改:

  • ansible/roles/kubernetes-addons/files/common/kube-addon-update.sh
  • ansible/roles/kubernetes-addons/tasks/cluster-logging.yml
  • ansible/roles/kubernetes-addons/templates/cluster-logging/fluentd-es-ds.yaml.j2
  • ansible/roles/node/tasks/fluentd-install.yml
  • ansible/roles/node/tasks/main.yml

然后执行安装命令:

1
2
3
4
[root@cts1 scripts]# ./deploy-cluster.sh --tags=nodes

cts1 : ok=8 changed=0 unreachable=0 failed=0
cts2 : ok=57 changed=3 unreachable=0 failed=0

Addons:
1
2
3
4
[root@cts1 scripts]# ./deploy-cluster.sh --tags=addons

cts1 : ok=58 changed=1 unreachable=0 failed=0
cts2 : ok=6 changed=0 unreachable=0 failed=0


8. 确认Kubernetes状况

1
2
3
4
5
6
7
8
[root@cts1 scripts]# kubectl get nodes
NAME STATUS AGE
cts2 Ready 2h
[root@cts1 scripts]# kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 2h
[root@cts1 scripts]# kubectl --version
Kubernetes v1.4.0

参照资料:
Kubernetes 1.3 从入门到进阶 安装篇(2)
Kubernetes Ansible
Support Fluentd migration to DaemonSet #2174