Centos搭建k8s环境并实战本地仓库和configmap使用

枫红愁绪

385人浏览 · 2025-09-15 10:13:09

枫红愁绪 · 2025-09-15 10:13:09 发布

1、集群规划

操作系统：CentOS 7.6（CentOS-7-x86_64-DVD-1810）
Docker: 19.03.15
Kubernetes：1.20.0

节点配置：
节点名称 ip
master 192.168.50.10
node01 192.168.50.20
node02 192.168.50.30

2、虚机配置

1. 关闭selinux
[root]# setenforce 0
[root]# vi /etc/selinux/config
SELINUX=disabled
2. 关闭firewalld
[root]# systemctl stop firewalld
[root]# systemctl disable firewall
3. 关闭swap
[root]# swapoff -a
[root]# vi /etc/fstab#/dev/mapper/centos-swap swap                    swap    defaults        0 0
4. 修改grub
[root]# vi /etc/default/grub
添加如下部分：
GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"
5. 配置hosts文件
[root]# vi /etc/hosts
192.168.50.10 master
192.168.50.20 node01
192.168.50.30 node02

3、安装k8s

3.1安装docker（三台）

三个节点均安装Docker
[root]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root]# yum install docker-ce-19.03.15 -y
cat > /etc/docker/daemon.json << EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
[root]# systemctl enable docker
[root]# systemctl start docker

3.2 安装kubelet、kubeadm、kubectl （三台）

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

[root]# yum install -y kubelet-1.20.0 kubeadm-1.20.0 kubectl-1.20.0
[root]# systemctl enable kubelet

3.3 master端初始化（master执行）

内含警告处理方式

[root]# kubeadm init --apiserver-advertise-address=192.168.50.10 --pod-network-cidr=192.168.0.0/16 --image-repository registry.aliyuncs.com/google_containers
I0517 10:42:25.377859   10391 version.go:251] remote version is much newer: v1.24.0; falling back to: stable-1.20
[init] Using Kubernetes version: v1.20.15
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.96.0.1 192.168.1.73]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [192.168.1.73 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [192.168.1.73 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 14.016126 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"
[mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: pkhisz.whbtwtmxw05led93
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf$HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
  export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.1.73:6443 --token pkhisz.whbtwtmxw05led93 
--discovery-token-ca-cert-hash sha256:e6e087b6ea5fe65c70b99f5ad0050642b3c7072224081fc7978ce026520907da 
提示：
  上面输出日志中，有提示：如何配置环境变量、如何配置网络以及如何将NODE节点加入到群集中。
 
警告1（可忽略）
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
警告1解决办法
[root]# cat > /etc/docker/daemon.json << EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
[root]# systemctl restart docker
重置kubeadm后重新初始化
[root]# kubeadm reset
[root]# kubeadm init --apiserver-advertise-address=192.168.1.73 --pod-network-cidr=192.168.0.0/16 --image-repository registry.aliyuncs.com/google_containers
 
警告2（可忽略）
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
警告2解决办法：
[root]# systemctl enable kubelet
重置kubeadm后重新初始化
[root]# kubeadm reset
[root]# kubeadm init --apiserver-advertise-address=192.168.1.73 --pod-network-cidr=192.168.0.0/16 --image-repository registry.aliyuncs.com/google_containers

3.4 master端配置环境变量（master）

[root]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /root/.bashrc
[root]# source /root/.bashrc
 
配置非root用户环境变量
[k8s]# mkdir -p $HOME/.kube
[k8s]# sudo cp -i /etc/kubernetes/admin.conf$HOME/.kube/config
[k8s]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
 
如果环境变量没有配置好，下面Master端配置网络会有如下报错：
[root]# kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
The connection to the server localhost:8080 was refused - did you specify the right host or port?

3.5 master端配置网络（master）

[root@master ~]# kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
configmap/calico-config created
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created

3.6 master配置kubectl命令自动补齐

查看completion帮助，根据提示配置kubectl命令配置自动补齐功能
[root]# kubectl completion -h
Output shell completion code for the specified shell (bash or zsh). The shell code must be evaluated to provide
interactive completion of kubectl commands.  This can be done by sourcing it from the .bash_profile.
 Detailed instructions on how to do this are available here:
https://kubernetes.io/docs/tasks/tools/install-kubectl/#enabling-shell-autocompletion
 Note for zsh users: [1] zsh completions are only supported in versions of zsh >= 5.2
Examples:
Installing bash completion on macOS using homebrew
If running Bash 3.2 included with macOS
  brew install bash-completion
or, if running Bash 4.1+
  brew install bash-completion@2
If kubectl is installed via homebrew, this should start working immediately.
If you've installed via other means, you may need add the completion to your completion directory
  kubectl completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl

Installing bash completion on Linux
If bash-completion is not installed on Linux, please install the 'bash-completion' package
via your distribution's package manager.
Load the kubectl completion code for bash into the current shell
  source <(kubectl completion bash)
Write bash completion code to a file and source if from .bash_profile
  kubectl completion bash > ~/.kube/completion.bash.inc
  printf "
Kubectl shell completion
  source '$HOME/.kube/completion.bash.inc'
  " >>$HOME/.bash_profile
  source $HOME/.bash_profile
Load the kubectl completion code for zsh[1] into the current shell
  source <(kubectl completion zsh)
Set the kubectl completion code for zsh[1] to autoload on startup
  kubectl completion zsh > "${fpath[1]}/_kubectl"
Usage:
  kubectl completion SHELL [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).
 
配置kubectl命令自动补齐功能
[root]# echo "source <(kubectl completion bash)" >> /root/.bashrc
[root]# source /root/.bashrc

3.7 master查看群集状态

[root]# kubectl get componentstatuses
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS      MESSAGE                                                                                       ERROR
controller-manager   Unhealthy   Get "http://127.0.0.1:10252/healthz": dial tcp 127.0.0.1:10252: connect: connection refused
scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused
etcd-0               Healthy     {"health":"true"}
注释：
controller-manager状态异常：Unhealthy;
scheduler状态异常：Unhealthy。
 
解决办法：注释或删除kube-scheduler.yaml、kube-controller-manager.yaml文件中“- --port=0”部分
[root]# vi /etc/kubernetes/manifests/kube-scheduler.yaml
[root]# vi /etc/kubernetes/manifests/kube-controller-manager.yaml
 
再次查看群集组件状态
[root]# kubectl get componentstatuses 
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health":"true"}
查看群集信息
[root]# kubectl cluster-info 
Kubernetes control plane is running at https://192.168.1.73:6443
KubeDNS is running at https://192.168.1.73:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
 
查看核心组件运行状态
[root]# kubectl -n kube-system get pod
NAME                                      READY   STATUS    RESTARTS   AGE
calico-kube-controllers-bcc6f659f-rwbnl   1/1     Running   0          174m
calico-node-9cqwz                         1/1     Running   0          174m
coredns-7f89b7bc75-5948b                  1/1     Running   0          3h27m
coredns-7f89b7bc75-ss8r7                  1/1     Running   0          3h27m
etcd-master                               1/1     Running   0          3h27m
kube-apiserver-master                     1/1     Running   0          3h27m
kube-controller-manager-master            1/1     Running   0          2m47s
kube-proxy-ccmz5                          1/1     Running   0          3h27m
kube-scheduler-master                     1/1     Running   0          7m4s

3.8 将node节点加入到集群（node节点）


master端创建永久token，缺省情况下创建的token有效期是24h
[root]# # kubeadm token create --ttl 0
6m4lq3.wspl9bkqpijog3ne
[root]# kubeadm token list
TOKEN                     TTL         EXPIRES   USAGES                   DESCRIPTION                                                EXTRA GROUPS
6m4lq3.wspl9bkqpijog3ne         authentication,signing                                                        system:bootstrappers:kubeadm:default-node-token   从master端获取discovery-token-ca-cert-hash值 [root]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' df98f4545e43a430a1bd2903d770172880f7f0d5090625654ef0653a7e3025bd   node1、node2节点执行加入到群集的操作(注意格式) [root]# kubeadm join 192.168.1.73:6443 --token 6m4lq3.wspl9bkqpijog3ne --discovery-token-ca-cert-hash sha256:df98f4545e43a430a1bd2903d770172880f7f0d5090625654ef0653a7e3025bd
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
● Certificate signing request was sent to apiserver and a response was received.
● The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
 
查看群集节点信息
[root]# kubectl get nodes -o wide


NAME     STATUS   ROLES                  AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
master   Ready    control-plane,master   3h56m   v1.20.0   192.168.1.73           CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://19.3.15 node1    Ready                     88s     v1.20.0   192.168.1.74           CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://19.3.15 node2    Ready                     8m48s   v1.20.0   192.168.1.75           CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://19.3.15 至此，kubernetes群集部署完成。


如果node状态不正常、检查kube-system下的pod是否都runnning  遇到过拉不到镜像的问题、可以手动上传镜像

4.安装dashboard

kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3.yaml
docker pull registry.openanolis.cn/openanolis/nginx:1.14.1-8.6

5.安装nginx实战-并使用 configmap

5.1创建一个名空间并 pull镜像

#三台都拉一下nginx镜像
docker pull registry.openanolis.cn/openanolis/nginx:1.14.1-8.6
在master创建一个名空间：

5.2创建一个 configmap 用于挂载nginx的index主页

kubectl create configmap index-cm --from-file=index.conf -n web

index.conf 内容：

<h1>Hello from ConfigMap</h1>

写一个yaml用来起deployment

创建一个configmap 用来替换nginx主页
如果使用--from-file方式创建configmap 那文件的key就是文件名、引用configmap时 key要注意

nginx.yaml: ⬇️ ！！！

---
apiVersion: v1
kind: Namespace
metadata:
  name: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: web
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeSelector:
        nginx: "yes"
      containers:
      - name: nginx
        image: testyuming:V1.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 200m
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 128Mi
        volumeMounts:
        - name: html
          mountPath: /usr/share/nginx/html/index.html
          subPath: index.html
          readOnly: true
      volumes:
      - name: html
        configMap:
          name: index-cm
          items:
          - key: index.conf
            path: index.html
          defaultMode: 0644
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  namespace: web
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30080
  selector:
    app: nginx

5.3启动 nginx的deploy

kubectl apply -f Nginx.yaml

6.搭建本地registry仓库

#
sudo mkdir -p /data/registry
sudo chmod 777 /data/registry  # 简化权限设置，生产环境需更严格


启动registry容器：
docker run -d \
  --restart=always \
  --name registry \
  -p 5000:5000 \
  -v /data/registry:/var/lib/registry \
  registry:laster

# 查看容器状态
docker ps | grep registry

# 查看仓库信息
curl http://localhost:5000/v2/_catalog
正常会返回空仓库信息：{"repositories":[]}

配置 Docker 信任非 HTTPS 仓库

vi /etc/docker/daemon.json    （三台）

{
  "insecure-registries": ["registry.testyuming:5000"]
}

#例如如下  我这里同时有镜像加速
[root@master ~]# cat /etc/docker/daemon.json  
{
  "registry-mirrors": [
    "http://hub-mirror.c.163.com",
    "https://docker.1panel.live"
  ],
  "insecure-registries": ["registry.testyuming:5000"]
}
[root@master ~]#

测试仓库推送和拉取

# 打标签（格式：仓库IP:端口/镜像名:标签）
docker  tag cdd9bdf89346 registry.testyuming/nginx:1.14.1-8.6 

# 推送到本地仓库
docker push registry.testyuming:5000/nginx:v1.0

# 查看仓库中的镜像
curl http://localhost:5000/v2/_catalog
# 应该返回：{"repositories":["nginx"]}

# 测试拉取（先删除本地镜像）
docker rmi  registry.openanolis.cn/openanolis/nginx:1.14.1-8.6
docker rmi  registry.testyuming/nginx:1.14.1-8.6
docker rmi  registry.testyuming:5000/nginx:v1.0

docker pull registry.testyuming:5000/nginx:v1.0

测试 node 使用 master 镜像仓库拉取镜像

修改yaml的image ：改为域名+版本

/etc/docker/daemon.json                  #此文件记得同步到node节点
192.168.100.100 registry.testyumin       #hosts也要同步到node节点

删除deploy重新apply

码道开发者社区

码道开发者社区，聚焦华为云码道 CodeArts 代码智能体，沉淀 Agent、Skill、鸿蒙开发实战内容，供开发者查阅资料、交流技术、分享工程实践

更多推荐

AI 已经能直接改设计稿了:Windows部署Penpot，接入Codex MCP实测

码道开发者社区

【AI 业务流架构师】AI 业务流架构师方法论：Agentic AI 范式转移与业务编排思维

2026 年，Anthropic CEO Dario Amodei 在 Code with Claude 大会上的一个判断广为流传：一个人运营一家十亿美元级公司的概率是 70%–80%。这句话听起来像科幻小说的设定，但它背后是一个正在发生的结构性变化——AI 把"执行"的成本压到了历史低点，同时把"编排"的价值推到了前台。这意味着我们需要一套新的思考框架，来回答两个问题：一个人如何拥有一个公司的产