安装K8s

1. 下载

   访问https://github.com/k3s-io/k3s/releases/download/v1.24.8%2Bk3s1/k3s 下载k3s二进制文件

2. 赋权并复制到bin目录

   chmod u+x k3s
   cp k3s /usr/local/bin/
   

3. k3s管理节点安装

   使用命令：

   curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_MIRROR=cn INSTALL_K3S_EXEC="--docker --node-name k3s-master --data-dir /data/k3s" sh -
   

–doceker指定docker作为k3s的容器服务
–node-name指定节点的名称
–data-dir指定k3s存储目录
如需防止容器ip网段和宿主机网段冲突，添加--cluster-cidr 172.24.0.0/16 --service-cidr 172.25.0.0/16参数到INSTALL_K3S_EXEC中。
4. 工作节点安装

在管理节点查看token

cat /data/k3s/server/token

使用如下命令加入节点

curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC="agent --docker --node-name k3s-worker01 --server https://管理节点ip:6443 --token 管理节点查看到的token --data-dir /data/k3s" sh -

dashboard安装

1. 下载

   访问https://github.com/cnrancher/kube-explorer/releases/download/v0.2.13/kube-explorer-linux-amd64 ,下载kube-explorer二进制文件

2. 启动

   使用以下命令后台运行kube-explorer

   nohup kube-explorer --kubeconfig=/etc/rancher/k3s/k3s.yaml --http-listen-port=9000 --https-listen-port=0 >/var/log/kube.log 2>&1 &
   

   –kubeconfig 指定k3s的认证config
   –http-listen-port指定访问端口

3. 访问
   使用http://ip:9000 访问kube-explorer，进入k3s的监控页面。

安装nvidia-device-plugin

编辑nvidia-device-plugin.yml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nvidia-device-plugin-daemonset
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: nvidia-device-plugin-ds
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        name: nvidia-device-plugin-ds
    spec:
      tolerations:
      - key: nvidia.com/gpu
        operator: Exists
        effect: NoSchedule
      # Mark this pod as a critical add-on; when enabled, the critical add-on
      # scheduler reserves resources for critical add-on pods so that they can
      # be rescheduled after a failure.
      # See https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/
      priorityClassName: "system-node-critical"
      containers:
      - image: nvcr.io/nvidia/k8s-device-plugin:v0.12.3
        name: nvidia-device-plugin-ctr
        env:
          - name: FAIL_ON_INIT_ERROR
            value: "false"
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
        volumeMounts:
          - name: device-plugin
            mountPath: /var/lib/kubelet/device-plugins
      volumes:
        - name: device-plugin
          hostPath:
            path: /var/lib/kubelet/device-plugins

安装

kubectl create -f nvidia-device-plugin.yml

简单示例

编辑pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: gpu-pod
  labels:
    app: gpu-pod
spec:
  restartPolicy: Never
  containers:
    - name: cuda-container
      image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda10.2
      resources:
        limits:
          nvidia.com/gpu: 1
  tolerations:
  - key: nvidia.com/gpu
    operator: Exists
    effect: NoSchedule

启动，查看pod日志是否成功

kubectl create -f pod.yaml