Architecture¶
Hardware¶
AP servers¶
- 3x master, 3x worker (tolerate 1 node down)
- Physical or VM
- Model: HP DL380 G10 or later, Dell Rack/HCI
- Worker: start from 8 core, 32GB (x2/x4 for medium/large), OS disk 900GB SSD RAID each
- Master: 4 core, 8GB, 100GB SSD
- Rancher: 4 core, 8GB, 100GB SSD
- Harbor: 8 core, 16GB, 500GB SSD
Storage (for app data)¶
- Opt 1. Longhorn (0.5-1TB x3)
- Opt 2. NetApp AFF C190 (960GB SSD x8 = 13TB, shared)
Networking¶
- 10G switches
- L4 switch / NLB
master(NAT)/worker -> Rancher:443
add rancher domain name of the router to the SFCS DNS server
the domain name of worker A10 is resolved to SFCS DNS server
port 80/443 from master/worker (NAT) to Harbor proxy cache
site k8s node to corp harbor/gitlab/vault access
Software¶
WiMES-managed/used services¶
- Elasticsearch
- MinIO
- MQTT
- Job Scheduler???
Cluster-managed serivces¶
- Longhorn
- MetalLB
- K8s ingress
- External-DNS??? (coredns-based solution)
- CoreDNS
- Prometheus
- Istio??? (under evaluation)
Site hosted¶
- Rancher
- Harbor
Regional hosted (WHQ, WKS, WZS)¶
- Keycloak
- Matomo
Global hosted¶
- GitLab
- OWASP Zap
- SonarQube
- Sonatype Nexus
Server spec¶
QAS
Common service
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| CNZHSRKE2Q1 | 10.41.21.175 | Rancher node | 4C8G 100GB | Ubuntu 22.04 | 10.41.9.20 | rancher-wzsqas.wistron.com |
| CNZHSRKE2Q2 | 10.41.21.176 | Rancher node | 4C8G 100GB | Ubuntu 22.04 | 10.41.9.20 | rancher-wzsqas.wistron.com |
| CNZHSRKE2Q3 | 10.41.21.177 | Rancher node | 4C8G 100GB | Ubuntu 22.04 | 10.41.9.20 | rancher-wzsqas.wistron.com |
| ZSGITLABD | 10.41.21.20 | Gitlab DEV(PoC) | 4C8G 60GB | Ubuntu 22.04 | wzs-sat-poc-gitlab.wistron.com | |
| ZSGITLABQ | 10.41.21.21 | Gitlab QAS | 4C8G 60GB | Ubuntu 22.04 | wzs-sat-qas-gitlab.wistron.com | |
| ZSHARBORD | 10.41.21.30 | Harbor DEV(PoC) | 4C6G 80GB | Ubuntu 22.04 | wzs-sat-poc-harbor.wistron.com | |
| ZSHARBORQ | 10.41.21.31 | Harbor QAS | 4C6G 80GB | Ubuntu 22.04 | wzs-sat-qas-harbor.wistron.com |
sat-qas-01
p1/p3/p6 共用QAS Cluster
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| zswimesqm1 | 10.41.243.31 | Master node | 10C16G 100GB | Ubuntu 22.04 | ||
| zswimesqm2 | 10.41.243.32 | Master node | 10C16G 100GB | Ubuntu 22.04 | ||
| zswimesqm3 | 10.41.243.33 | Master node | 10C16G 100GB | Ubuntu 22.04 | ||
| zswimesqw1 | 10.41.243.41 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
| zswimesqw2 | 10.41.243.42 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
| zswimesqw3 | 10.41.243.43 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
| zswimesqw4 | 10.41.243.44 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
| zswimesqw5 | 10.41.243.45 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
| zswimesqw6 | 10.41.243.46 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
| zswimesqw7 | 10.41.243.47 | Worker node | 24C40G 300GB | Ubuntu 22.04 | 10.41.244.10 | *.wzs-sat-qas-01.k8s.wistron.com |
sat-poc-01
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| zsmespocm1 | 10.41.243.35 | Master node | 6C8G 80GB | Ubuntu 22.04 | ||
| zsmespocm2 | 10.41.243.36 | Master node | 6C8G 80GB | Ubuntu 22.04 | ||
| zsmespocm3 | 10.41.243.37 | Master node | 6C8G 80GB | Ubuntu 22.04 | ||
| zsmespocw1 | 10.41.243.51 | Worker node | 10C16G 100GB | Ubuntu 22.04 | 10.41.244.11 | *.wzs-sat-poc-01.k8s.wistron.com |
| zsmespocw1 | 10.41.243.52 | Worker node | 10C16G 100GB | Ubuntu 22.04 | 10.41.244.11 | *.wzs-sat-poc-01.k8s.wistron.com |
| zsmespocw1 | 10.41.243.53 | Worker node | 10C16G 100GB | Ubuntu 22.04 | 10.41.244.11 | *.wzs-sat-poc-01.k8s.wistron.com |
PRD
Common service
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| cnzhsranchp1 | 10.41.23.175 | Rancher node | 6C10G 100GB | Ubuntu 22.04 | 10.41.9.23 | rancher-wzsprd.wistron.com |
| cnzhsranchp2 | 10.41.23.176 | Rancher node | 6C10G 100GB | Ubuntu 22.04 | 10.41.9.23 | rancher-wzsprd.wistron.com |
| cnzhsranchp3 | 10.41.23.177 | Rancher node | 6C10G 100GB | Ubuntu 22.04 | 10.41.9.23 | rancher-wzsprd.wistron.com |
| ZSGITLABP | 10.41.241.108 | Site Gitlab | 8C16G 100GB | Ubuntu 22.04 | wzs-sat-prd-gitlab.wistron.com | |
| ZSHARBORP | 172.30.50.56 | Site Harbor | 8C8G 200GB | Ubuntu 22.04 | 10.41.16.56 | wzs-sat-prd-harbor.wistron.com |
sat-prd-p1
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| ZSMESP1PM1 | 172.30.81.21 | Master node | 10C16G 100GB | Ubuntu 22.04 | 10.41.16.151 | |
| ZSMESP1PM2 | 172.30.81.22 | Master node | 10C16G 100GB | Ubuntu 22.04 | 10.41.16.152 | |
| ZSMESP1PM3 | 172.30.81.24 | Master node | 10C16G 100GB | Ubuntu 22.04 | 10.41.16.153 | |
| ZSMESP1PW1 | 10.41.245.31 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.41.244.14 | *.wzs-sat-prd-p1.k8s.wistron.com |
| ZSMESP1PW2 | 10.41.245.32 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.41.244.14 | *.wzs-sat-prd-p1.k8s.wistron.com |
| ZSMESP1PW3 | 10.41.245.33 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.41.244.14 | *.wzs-sat-prd-p1.k8s.wistron.com |
| ZSMESP1PW4 | 10.41.245.34 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.41.244.14 | *.wzs-sat-prd-p1.k8s.wistron.com |
sat-prd-p3
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| ZSMESP3PM1 | 172.30.51.24 | Master node | 16C32G 100GB | Ubuntu 20.04 | 10.41.16.74 | |
| ZSMESP3PM2 | 172.30.51.25 | Master node | 16C32G 100GB | Ubuntu 20.04 | 10.41.16.75 | |
| ZSMESP3PM3 | 172.30.51.26 | Master node | 16C32G 100GB | Ubuntu 20.04 | 10.41.16.77 | |
| ZSMESP3PW1 | 10.41.246.34 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 20.04 | 10.41.244.13 | *.wzs-sat-prd-p3.k8s.wistron.com |
| ZSMESP3PW2 | 10.41.246.35 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 20.04 | 10.41.244.13 | *.wzs-sat-prd-p3.k8s.wistron.com |
| ZSMESP3PW3 | 10.41.246.36 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 20.04 | 10.41.244.13 | *.wzs-sat-prd-p3.k8s.wistron.com |
| ZSMESP3PW4 | 10.41.246.33 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 20.04 | 10.41.244.13 | *.wzs-sat-prd-p3.k8s.wistron.com |
sat-prd-p6
| Hostname | IP Address | Function | Spec | OS | L4 VIP IP | FQDN |
|---|---|---|---|---|---|---|
| ZSMESP6PM1 | 10.55.241.21 | Master node | 12C24G 100GB | Ubuntu 22.04 | ||
| ZSMESP6PM2 | 10.55.241.22 | Master node | 12C24G 100GB | Ubuntu 22.04 | ||
| ZSMESP6PM3 | 10.55.241.23 | Master node | 12C24G 100GB | Ubuntu 22.04 | ||
| ZSMESP6PW1 | 10.55.240.31 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.55.240.30 | *.wzs-sat-prd-p6.k8s.wistron.com |
| ZSMESP6PW2 | 10.55.240.32 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.55.240.30 | *.wzs-sat-prd-p6.k8s.wistron.com |
| ZSMESP6PW3 | 10.55.240.33 | Worker node | 2P40C256G 1.92TB*5 | Ubuntu 22.04 | 10.55.240.30 | *.wzs-sat-prd-p6.k8s.wistron.com |