k8s【安装篇】05:【离线部署】X86 KylinV10 SP3【k8s】【v1.23.4】集群
·
一、环境准备
1、环境信息
ip a
10.229.42.222 master
10.229.42.223 node01
10.229.42.224 node02

2、设置主机名
(1)222节点执行
hostnamectl set-hostname master
hostname master
su -

(1)223节点执行
hostnamectl set-hostname node01
hostname node01
su -

(1)224节点执行
hostnamectl set-hostname node02
hostname node02
su -

3、修改hosts文件
3个节点都执行
cat >> /etc/hosts << EOF
10.229.42.222 master
10.229.42.223 node01
10.229.42.224 node02
EOF

cat /etc/hosts

4、配置本地源
(1)查看是否挂载
lsblk
如下,说明已经挂载,如果没有挂载,自己挂载一下。
(2)创建挂载目录
mkdir -p /media/kylin/

(3)进行挂载
mount /dev/cdrom /media/kylin/

(4)创建本地源文件
cd /etc/yum.repos.d
vim local.repo
添加如下内容:
[kylin]
name=kylin
baseurl=file:///media/kylin/
gpgcheck=0
enabled=1

删除其他的.repo文件,我这里只有一个kylin_x86_64.repo
cd /etc/yum.repos.d
rm -rf kylin_x86_64.repo
ll

(5)清除缓存生成新的缓存
yum clean all
yum makecache

(6)将挂载配置到开启启动中
vi /etc/rc.d/rc.local
添加如下内容:
mount /dev/cdrom /media/kylin/

5、关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl status firewalld.service

6、关闭selinux
getenforce
setenforce 0
sed -i 's/^ *SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
cat /etc/selinux/config

设置完重启
shutdown -r now

7、禁用swap
swapoff -a
sed -i.bak '/swap/s/^/#/' /etc/fstab
cat /etc/fstab

8、安装依赖
yum install -y yum-utils device-mapper-persistent-data lvm2

二、安装Docker环境
1、下载并上传docker安装包
我这里的离线安装包是20.10.7版本:docker-20.10.7
mkdir -p /data/soft/
ll

2、解压安装包
cd /data/soft/
unzip docker-20.10.7.zip
ll

cd /data/soft/docker-20.10.7/
tar -zxvf docker-20.10.7.tgz
chmod +x ./docker/*
cp docker/* /usr/bin/

3、编辑docker的系统服务文件
vi /usr/lib/systemd/system/docker.service
添加如下内容:
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target

添加可执行权限
chmod +x /usr/lib/systemd/system/docker.service

4、编辑daemon.json
mkdir -p /etc/docker
vi /etc/docker/daemon.json
添加如下内容:
{
"registry-mirrors": ["https://registry.docker-cn.com"],
"exec-opts": ["native.cgroupdriver=cgroupfs"]
}

5、启动docker
systemctl daemon-reload
systemctl restart docker
systemctl enable docker
systemctl status docker

6、安装命令补全
yum -y install bash-completion
source /etc/profile.d/bash_completion.sh

7、验证
docker info
docker -v

三、安装k8s
1、内核参数修改
sysctl net.bridge.bridge-nf-call-iptables=1
sysctl net.bridge.bridge-nf-call-ip6tables=1

cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl -p /etc/sysctl.d/k8s.conf

2、下载并上传k8s安装包
cd /data/soft/
ll

3、解压k8s安装包
cd /data/soft/
unzip k8s-rpm-x86.zip
cd /data/soft/k8s-rpm-x86
ll

4、直接rpm安装
cd /data/soft/k8s-rpm-x86
rpm -ivh *.rpm --nodeps --force

5、配置kubelet的cgroup
vim /etc/sysconfig/kubelet
添加如下内容:
KUBELET_EXTRA_ARGS="--cgroup-driver=cgroupfs"
KUBE_PROXY_MODE="ipvs"

6、设置kubelet开机启动
systemctl enable kubelet

7、kubelet命令补全
echo "source <(kubectl completion bash)" >> ~/.bash_profile
source ~/.bash_profile

8、下载并导入镜像
(1)下载上传镜像
(2)解压导入镜像
cd /data/soft/
unzip k8s-image-x86.zip
cd /data/soft/k8s-image-x86
docker load -i k8s_1.23.4.tar.gz

(3)查看镜像
docker images

注意,这里少了一个calico/kube-controllers的镜像,这里再导入这个镜像
calico-kube-controllers镜像
所有节点都要执行
cd /data/soft/
docker load < calico-kube-controllers.tar

修改镜像tag,这里为了保持一致。
docker tag calico/kube-controllers:v3.22.0 calico/kube-controllers:v3.18.0
docker images

四、初始化节点
1、初始化master节点
在master节点进行操作!!
(1)初始化节点操作
kubeadm init \
--apiserver-advertise-address=10.229.42.222 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.23.4 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--ignore-preflight-errors=all

记录kubeadmjoin的输出,后面要加入node01和node02节点中
kubeadm join 10.229.42.222:6443 --token n2npu0.62nqy57b53kzg8wr \
--discovery-token-ca-cert-hash sha256:afbc4700e2b3ca52f81de5e2cd2f8d708cc4064bfb2f2902d1dd43e0c670e25c
(2)加载环境变量
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile

若不是root用户,执行如下操作
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

2、初始化node01节点
kubeadm join 10.229.42.222:6443 --token n2npu0.62nqy57b53kzg8wr \
--discovery-token-ca-cert-hash sha256:afbc4700e2b3ca52f81de5e2cd2f8d708cc4064bfb2f2902d1dd43e0c670e25c

3、初始化node02节点
kubeadm join 10.229.42.222:6443 --token n2npu0.62nqy57b53kzg8wr \
--discovery-token-ca-cert-hash sha256:afbc4700e2b3ca52f81de5e2cd2f8d708cc4064bfb2f2902d1dd43e0c670e25c

4、查看集群节点状况
在主节点执行
kubectl get nodes

5、节点打标签
可以对节点进行打标签,例如将node01、node02打标签为work
kubectl label nodes node01 node-role.kubernetes.io/work=work
kubectl label nodes node02 node-role.kubernetes.io/work=work
kubectl get nodes

五、部署容器网络
在master节点进行操作!
1、下载并上传calico.yaml
cd /data/soft
ll

2、进行部署calico
cd /data/soft/
kubectl apply -f calico.yaml

3、查看状态
kubectl get pods -n kube-system
kubectl get pod --all-namespaces -o wide

六、安装界面化管理工具
1、上传脚本
cd /data/soft/
vim kuboard.sh
内容如下:
#!/bin/bash
ipaddr=$(ip addr | awk '/^[0-9]+: / {}; /inet.*global/ {print gensub(/(.*)\/(.*)/, "\\1", "g", $2)}' | awk 'NR==1{print}')
echo "current ip address is ${ipaddr}"
echo "create file /root/kuboard-sa.yaml"
echo
cat > /root/kuboard-sa.yaml << EOF
---
kind: Namespace
apiVersion: v1
metadata:
name: kuboard
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: kuboard-admin
namespace: kuboard
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kuboard-admin-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kuboard-admin
namespace: kuboard
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: kuboard-viewer
namespace: kuboard
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kuboard-viewer-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: kuboard-viewer
namespace: kuboard
EOF
echo "kubectl apply -f /root/kuboard-sa.yaml"
kubectl apply -f /root/kuboard-sa.yaml
echo
echo "create file /etc/kubernetes/manifests/kuboard.yaml"
echo
cat > /etc/kubernetes/manifests/kuboard.yaml << EOF
---
apiVersion: v1
kind: Pod
metadata:
annotations: {}
labels:
k8s.kuboard.cn/name: kuboard-v3
name: kuboard-v3
namespace: kuboard
spec:
containers:
- env:
- name: KUBOARD_ENDPOINT
value: "http://${ipaddr}:80"
- name: KUBOARD_AGENT_SERVER_TCP_PORT
value: "10081"
image: 'eipwork/kuboard:v3'
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /kuboard-resources/version.json
port: 80
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: kuboard
ports:
- containerPort: 80
hostPort: 80
name: web
protocol: TCP
- containerPort: 10081
name: peer
protocol: TCP
hostPort: 10081
- containerPort: 10081
name: peer-u
protocol: UDP
hostPort: 10081
readinessProbe:
failureThreshold: 3
httpGet:
path: /kuboard-resources/version.json
port: 80
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
volumeMounts:
- mountPath: /data
name: data
- mountPath: /init-etcd-scripts/import-cluster-once.yaml
name: import-cluster-yaml
volumes:
- hostPath:
path: "/usr/share/kuboard"
name: data
- hostPath:
path: "/usr/share/kuboard/import-cluster-once.yaml"
name: import-cluster-yaml
dnsPolicy: ClusterFirst
restartPolicy: Always
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
EOF
echo "restart kubelet"
systemctl restart kubelet
host_name=$(hostname)
echo
echo "\033[34m检查状态\033[0m 待 kuboard-v3-${host_name} 的容器组变为 Running 状态后,则安装成功,可以通过 http://${ipaddr} 访问 kuboard 界面"
echo
kubectl get pods -n kuboard -o wide
2、执行脚本
cd /data/soft/
sh kuboard.sh

kubectl get pods -n kuboard -o wide

3、进行访问
访问http://10.229.42.222:80
用户名:admin
密码:Kuboard123



七、扩容
例如,需要添加第二个控制节点,这里需要进行安装dokcer,k8s,这里就不多赘述,我这里扩容的是10.229.42.230
ip a

1、修改主机名为master02
hostnamectl set-hostname master02
hostname master02
su -

2、编辑/etc/hosts
四个节点都进行编辑添加
3、配置本地源
参考上面
4、关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl status firewalld.service
5、关闭selinux
getenforce
setenforce 0
sed -i 's/^ *SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
cat /etc/selinux/config
6、禁用swap
swapoff -a
sed -i.bak '/swap/s/^/#/' /etc/fstab
cat /etc/fstab
7、安装依赖
yum install -y yum-utils device-mapper-persistent-data lvm2
8、安装docker环境
参考上面
9、安装k8s
参考上面
10、导入镜像
参考上面
11、master02创建证书目录
cd /root && mkdir -p /etc/kubernetes/pki/etcd &&mkdir -p ~/.kube/
12、拷贝master01的证书到master02上
这里可以搞一个互信,不然,输密码太麻烦了
scp /etc/kubernetes/pki/ca.crt master02:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/ca.key master02:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/sa.key master02:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/sa.pub master02:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/front-proxy-ca.crt master02:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/front-proxy-ca.key master02:/etc/kubernetes/pki/
scp /etc/kubernetes/pki/etcd/ca.crt master02:/etc/kubernetes/pki/etcd/
scp /etc/kubernetes/pki/etcd/ca.key master02:/etc/kubernetes/pki/etcd/

13、检查 kubeadm-config ConfigMap 是否正确配置了 controlPlaneEndpoint
可以使用 kubectl 命令获取 kubeadm-config ConfigMap 的信息,在master01上执行:
kubectl -n kube-system edit cm kubeadm-config -o yaml
添加如下字段:
controlPlaneEndpoint: "10.229.42.222:6443"

14、重启kubelet
master01节点重启kubelet
systemctl restart kubelet
15、在master01查看加入集群的命令
kubeadm token create --print-join-command

16、master02加入集群
master节点执行
kubeadm join 10.229.42.222:6443 --token 8ewhq6.9hhvyz5zgbj7vmnd --discovery-token-ca-cert-hash sha256:afbc4700e2b3ca52f81de5e2cd2f8d708cc4064bfb2f2902d1dd43e0c670e25c

17、创建配置
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
18、master01查看集群状况
kubectl get nodes

八、其他注意项
1、tocken有效期为24小时
默认token有效期为24小时,当过期之后,该token就不可用了。这时就需要重新创建token,可以直接使用命令快捷生成:
kubeadm token create --print-join-command
kubeadm join 192.168.184.171:6443 --token n00ho8.gqolqq9opbtlbif3 --discovery-token-ca-cert-hash sha256:c2b52d1d45276d1c8a74fd341afaa6cef37f75b44e7088de01390ebd38a24abc

更多推荐



所有评论(0)