K8S Cluster Create (On Rancher)¶

1. Create K8s Cluster¶

1.1 Login rancher, click ☰ -> Cluster Managerment -> Create

1.2 Enable RKE2/K3s, and click Custom

1.3 Input the Cluster Name, and select Kubernetes Version

1.4 Switch to Networking and assign Cluster CIDR and Service CIDR IP range, then click Create

2. Setup registries¶

2.1 Click ┇ -> Edit YAML

2.2 Find machineSelectorConfig: & registries: {}, and replace with the follow content, then click Save

    machineSelectorConfig:
      - config:
          kubelet-arg:
            - max-pods=250
            - kube-api-burst=100
            - kube-api-qps=50
          protect-kernel-defaults: false
    registries:
      configs:
        wzs-sat-qas-harbor.wistron.com:
          authConfigSecretName: null
          caBundle: ''
          insecureSkipVerify: true
          tlsSecretName: null
      mirrors:
        docker.io:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.docker.io/$1
            ^[^/]+$: wzs-rke2/proxy.docker.io/library/$1
        gcr.io:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.gcr.io/$1
        k8s.gcr.io:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.k8s.gcr.io/$1
        ghcr.io:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.ghcr.io/$1
        quay.io:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.quay.io/$1
        mcr.microsoft.com:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.microsoft.com/$1
        registry.k8s.io:
          endpoint:
            - https://wzs-sat-qas-harbor.wistron.com
          rewrite:
            ^(.*)$: wzs-rke2/proxy.registry.k8s.io/$1

3. Add master to cluster¶

3.1 Click cluster name wzs-sat-poc-01

3.2 Unchecked Worker, and checked Insecure: Select this to skip TLS verification if your server has a self-signed certificate, then click the script to copied.

3.3 Login master 01, and paste script into terminal to run

curl --insecure -fL https://rancher-wzsqas.wistron.com/system-agent-install.sh | sudo  sh -s - --server https://rancher-wzsqas.wistron.com --label 'cattle.io/os=linux' --token rtmbdlncnlmgkk7xrtkm28f2xnr5zn4sxhcphp4f7tdlzlbdhr44ws --ca-checksum 738dc7ec672f776fd642ece280f4357f822ac47494974b6702651e8c357fc700 --etcd --controlplane

3.4 Login master 02, and paste script into terminal to run

curl --insecure -fL https://rancher-wzsqas.wistron.com/system-agent-install.sh | sudo  sh -s - --server https://rancher-wzsqas.wistron.com --label 'cattle.io/os=linux' --token rtmbdlncnlmgkk7xrtkm28f2xnr5zn4sxhcphp4f7tdlzlbdhr44ws --ca-checksum 738dc7ec672f776fd642ece280f4357f822ac47494974b6702651e8c357fc700 --etcd --controlplane

3.5 Login master 03, and paste script into terminal to run

curl --insecure -fL https://rancher-wzsqas.wistron.com/system-agent-install.sh | sudo  sh -s - --server https://rancher-wzsqas.wistron.com --label 'cattle.io/os=linux' --token rtmbdlncnlmgkk7xrtkm28f2xnr5zn4sxhcphp4f7tdlzlbdhr44ws --ca-checksum 738dc7ec672f776fd642ece280f4357f822ac47494974b6702651e8c357fc700 --etcd --controlplane

4. Add worker to cluster¶

4.1 Switch to Registration, and Unchecked etcd & Control Plane, and checked Insecure: Select this to skip TLS verification if your server has a self-signed certificate, then click the script to copied.

4.2 Login worker 01, and paste script into terminal to run

curl --insecure -fL https://rancher-wzsqas.wistron.com/system-agent-install.sh | sudo  sh -s - --server https://rancher-wzsqas.wistron.com --label 'cattle.io/os=linux' --token rtmbdlncnlmgkk7xrtkm28f2xnr5zn4sxhcphp4f7tdlzlbdhr44ws --ca-checksum 738dc7ec672f776fd642ece280f4357f822ac47494974b6702651e8c357fc700 --worker

4.3 Login worker 02, and paste script into terminal to run

curl --insecure -fL https://rancher-wzsqas.wistron.com/system-agent-install.sh | sudo  sh -s - --server https://rancher-wzsqas.wistron.com --label 'cattle.io/os=linux' --token rtmbdlncnlmgkk7xrtkm28f2xnr5zn4sxhcphp4f7tdlzlbdhr44ws --ca-checksum 738dc7ec672f776fd642ece280f4357f822ac47494974b6702651e8c357fc700 --worker

4.4 Login worker 03, and paste script into terminal to run

curl --insecure -fL https://rancher-wzsqas.wistron.com/system-agent-install.sh | sudo  sh -s - --server https://rancher-wzsqas.wistron.com --label 'cattle.io/os=linux' --token rtmbdlncnlmgkk7xrtkm28f2xnr5zn4sxhcphp4f7tdlzlbdhr44ws --ca-checksum 738dc7ec672f776fd642ece280f4357f822ac47494974b6702651e8c357fc700 --worker

4.5 Check cluster status

waiting for all node added to cluster, and status is running, and cluster status Active

4.6 Config kubectl environment variables for all master node

Config to each master node

cat >> /etc/profile <<EOF
export PATH=/var/lib/rancher/rke2/bin:$PATH
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml 
EOF
source /etc/profile

cat >> ~/.bashrc <<EOF
export PATH="$PATH:/usr/local/bin:/var/lib/rancher/rke2/bin"
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
EOF

mkdir -p ~/.kube && cd ~/.kube
ln -s /etc/rancher/rke2/rke2.yaml ~/.kube/config
ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml
chmod 644 ~/.kube/config
cd

5. Longhorn install¶

5.1 Click cluster wzs-sat-poc-01, and click Apps -> Chart, then search longhorn,and click Longhorn apps.

5.2 Click Install

5.3 Select install location to System, and click Next

5.4 Keep default, and click Install, then waiting longhorn finish install

5.5 Check longhorn

Click Longhorn -> Longhorn icon

Check Dashboard, Node, Volume