网站建设与制作视频教学,wordpress用什么数据库连接,网站开发接单群,网络推广合作资源平台Kubernetes概述 使用kubeadm快速部署一个k8s集群 Kubernetes高可用集群二进制部署#xff08;一#xff09;主机准备和负载均衡器安装 Kubernetes高可用集群二进制部署#xff08;二#xff09;ETCD集群部署 Kubernetes高可用集群二进制部署#xff08;三#xff09;部署…Kubernetes概述 使用kubeadm快速部署一个k8s集群 Kubernetes高可用集群二进制部署一主机准备和负载均衡器安装 Kubernetes高可用集群二进制部署二ETCD集群部署 Kubernetes高可用集群二进制部署三部署api-server Kubernetes高可用集群二进制部署四部署kubectl和kube-controller-manager、kube-scheduler Kubernetes高可用集群二进制部署五kubelet、kube-proxy、Calico、CoreDNS Kubernetes高可用集群二进制部署六Kubernetes集群节点添加
1. 部署kubectl
1.1 创建kubectl证书请求文件
在master1执行
[rootk8s-master1 /]#cd /data/k8s-work
cat admin-csr.json EOF
{CN: admin,hosts: [], #通过conf文件对集群进行访问所以这里不指定主机key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:masters, OU: system}]
}
EOF说明后续 kube-apiserver 使用 RBAC 对客户端(如 kubelet、kube-proxy、Pod)请求进行授权
kube-apiserver 预定义了一些 RBAC 使用的 RoleBindings如 cluster-admin 将 Group system:masters 与 Role cluster-admin 绑定该 Role 授予了调用kube-apiserver 的所有 API的权限
O指定该证书的 Group 为 system:masterskubelet 使用该证书访问 kube-apiserver 时 由于证书被 CA 签名所以认证通过同时由于证书用户组为经过预授权的 system:masters所以被授予访问所有 API 的权限
注
这个admin 证书是将来生成管理员用的kubeconfig 配置文件用的现在我们一般建议使用RBAC 来对kubernetes 进行角色权限控制 kubernetes 将证书中的CN 字段 作为User O 字段作为 Group
O: system:masters, 必须是system:masters否则后面kubectl create clusterrolebinding报错。1.2 生成证书文件
cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes admin-csr.json | cfssljson -bare admin1.3 复制文件到指定目录
cp admin*.pem /etc/kubernetes/ssl/1.4 生成kubeconfig配置文件
kube.config 为 kubectl 的配置文件包含访问 apiserver 的所有信息如 apiserver 地址、CA 证书和自身使用的证书
kubectl config set-cluster kubernetes --certificate-authorityca.pem --embed-certstrue --serverhttps://192.168.10.100:6443 --kubeconfigkube.config# cat kube.config#设置管理员证书
kubectl config set-credentials admin --client-certificateadmin.pem --client-keyadmin-key.pem --embed-certstrue --kubeconfigkube.config# cat kube.config#设置安全上下文
kubectl config set-context kubernetes --clusterkubernetes --useradmin --kubeconfigkube.config# cat kube.configkubectl config use-context kubernetes --kubeconfigkube.config1.5 准备kubectl配置文件并进行角色绑定
mkdir ~/.kube
cp kube.config ~/.kube/config
kubectl create clusterrolebinding kube-apiserver:kubelet-apis --clusterrolesystem:kubelet-api-admin --user kubernetes --kubeconfig/root/.kube/config1.6 查看集群状态
export KUBECONFIG$HOME/.kube/configkubectl cluster-info
kubectl get componentstatuses
kubectl get all --all-namespaces1.7 同步kubectl配置文件到集群其它master节点
k8s-master2:
mkdir /root/.kubek8s-master3:
mkdir /root/.kubescp /root/.kube/config k8s-master2:/root/.kube/config
scp /root/.kube/config k8s-master3:/root/.kube/config1.8 配置kubectl命令补全(可选)
yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source (kubectl completion bash)
kubectl completion bash ~/.kube/completion.bash.inc
source /root/.kube/completion.bash.inc
source $HOME/.bash_profile2. 部署kube-controller-manager
2.1 创建kube-controller-manager证书请求文件
在master1执行
[rootk8s-master1 k8s-work]#cat kube-controller-manager-csr.json EOF
{CN: system:kube-controller-manager,key: {algo: rsa,size: 2048},hosts: [127.0.0.1,192.168.10.103, #三台master的ip192.168.10.104,192.168.10.105],names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-controller-manager,OU: system}]
}
EOF说明hosts 列表包含所有 kube-controller-manager 节点 IP
CN 为 system:kube-controller-manager、O 为 system:kube-controller-managerkubernetes 内置的 ClusterRoleBindings system:kube-controller-manager 赋予 kube-controller-manager 工作所需的权限2.2 创建kube-controller-manager证书文件
cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager# lskube-controller-manager.csr
kube-controller-manager-csr.json
kube-controller-manager-key.pem
kube-controller-manager.pem2.3 创建kube-controller-manager的kube-controller-manager.kubeconfig
kubectl config set-cluster kubernetes --certificate-authorityca.pem --embed-certstrue --serverhttps://192.168.10.100:6443 --kubeconfigkube-controller-manager.kubeconfigkubectl config set-credentials system:kube-controller-manager --client-certificatekube-controller-manager.pem --client-keykube-controller-manager-key.pem --embed-certstrue --kubeconfigkube-controller-manager.kubeconfigkubectl config set-context system:kube-controller-manager --clusterkubernetes --usersystem:kube-controller-manager --kubeconfigkube-controller-manager.kubeconfigkubectl config use-context system:kube-controller-manager --kubeconfigkube-controller-manager.kubeconfig2.4 创建kube-controller-manager配置文件
cat kube-controller-manager.conf EOF
KUBE_CONTROLLER_MANAGER_OPTS--port10252 \--secure-port10257 \--bind-address127.0.0.1 \--kubeconfig/etc/kubernetes/kube-controller-manager.kubeconfig \--service-cluster-ip-range10.96.0.0/16 \--cluster-namekubernetes \--cluster-signing-cert-file/etc/kubernetes/ssl/ca.pem \--cluster-signing-key-file/etc/kubernetes/ssl/ca-key.pem \--allocate-node-cidrstrue \--cluster-cidr10.244.0.0/16 \--experimental-cluster-signing-duration87600h \--root-ca-file/etc/kubernetes/ssl/ca.pem \--service-account-private-key-file/etc/kubernetes/ssl/ca-key.pem \--leader-electtrue \--feature-gatesRotateKubeletServerCertificatetrue \--controllers*,bootstrapsigner,tokencleaner \--horizontal-pod-autoscaler-use-rest-clientstrue \--horizontal-pod-autoscaler-sync-period10s \--tls-cert-file/etc/kubernetes/ssl/kube-controller-manager.pem \--tls-private-key-file/etc/kubernetes/ssl/kube-controller-manager-key.pem \--use-service-account-credentialstrue \--alsologtostderrtrue \--logtostderrfalse \--log-dir/var/log/kubernetes \--v2
EOF2.5 创建服务启动文件
[rootk8s-master1 k8s-work]# cat kube-controller-manager.service EOF
[Unit]
DescriptionKubernetes Controller Manager
Documentationhttps://github.com/kubernetes/kubernetes[Service]
EnvironmentFile-/etc/kubernetes/kube-controller-manager.conf
ExecStart/usr/local/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_OPTS
Restarton-failure
RestartSec5[Install]
WantedBymulti-user.target
EOF2.6 同步文件到集群master节点
cp kube-controller-manager*.pem /etc/kubernetes/ssl/
cp kube-controller-manager.kubeconfig /etc/kubernetes/
cp kube-controller-manager.conf /etc/kubernetes/
cp kube-controller-manager.service /usr/lib/systemd/system/scp kube-controller-manager*.pem k8s-master2:/etc/kubernetes/ssl/
scp kube-controller-manager*.pem k8s-master3:/etc/kubernetes/ssl/
scp kube-controller-manager.kubeconfig kube-controller-manager.conf k8s-master2:/etc/kubernetes/
scp kube-controller-manager.kubeconfig kube-controller-manager.conf k8s-master3:/etc/kubernetes/
scp kube-controller-manager.service k8s-master2:/usr/lib/systemd/system/
scp kube-controller-manager.service k8s-master3:/usr/lib/systemd/system/#查看证书
openssl x509 -in /etc/kubernetes/ssl/kube-controller-manager.pem -noout -text2.7 启动服务
systemctl daemon-reload
systemctl enable --now kube-controller-manager
systemctl status kube-controller-managerkubectl get componentstatuses3. 部署kube-scheduler
3.1 创建kube-scheduler证书请求文件
[rootk8s-master1 k8s-work]# cat kube-scheduler-csr.json EOF
{CN: system:kube-scheduler,hosts: [127.0.0.1,192.168.10.103,192.168.10.104,192.168.10.105],key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-scheduler,OU: system}]
}
EOF3.2 生成kube-scheduler证书
cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler# ls
kube-scheduler.csr
kube-scheduler-csr.json
kube-scheduler-key.pem
kube-scheduler.pem3.3 创建kube-scheduler的kubeconfig
kubectl config set-cluster kubernetes --certificate-authorityca.pem --embed-certstrue --serverhttps://192.168.10.100:6443 --kubeconfigkube-scheduler.kubeconfigkubectl config set-credentials system:kube-scheduler --client-certificatekube-scheduler.pem --client-keykube-scheduler-key.pem --embed-certstrue --kubeconfigkube-scheduler.kubeconfigkubectl config set-context system:kube-scheduler --clusterkubernetes --usersystem:kube-scheduler --kubeconfigkube-scheduler.kubeconfigkubectl config use-context system:kube-scheduler --kubeconfigkube-scheduler.kubeconfig3.4 创建服务配置文件
cat kube-scheduler.conf EOF
KUBE_SCHEDULER_OPTS--address127.0.0.1 \
--kubeconfig/etc/kubernetes/kube-scheduler.kubeconfig \
--leader-electtrue \
--alsologtostderrtrue \
--logtostderrfalse \
--log-dir/var/log/kubernetes \
--v2
EOF3.5创建服务启动配置文件
cat kube-scheduler.service EOF
[Unit]
DescriptionKubernetes Scheduler
Documentationhttps://github.com/kubernetes/kubernetes[Service]
EnvironmentFile-/etc/kubernetes/kube-scheduler.conf
ExecStart/usr/local/bin/kube-scheduler $KUBE_SCHEDULER_OPTS
Restarton-failure
RestartSec5[Install]
WantedBymulti-user.target
EOF3.6 同步文件至集群master节点
cp kube-scheduler*.pem /etc/kubernetes/ssl/
cp kube-scheduler.kubeconfig /etc/kubernetes/
cp kube-scheduler.conf /etc/kubernetes/
cp kube-scheduler.service /usr/lib/systemd/system/scp kube-scheduler*.pem k8s-master2:/etc/kubernetes/ssl/
scp kube-scheduler*.pem k8s-master3:/etc/kubernetes/ssl/
scp kube-scheduler.kubeconfig kube-scheduler.conf k8s-master2:/etc/kubernetes/
scp kube-scheduler.kubeconfig kube-scheduler.conf k8s-master3:/etc/kubernetes/
scp kube-scheduler.service k8s-master2:/usr/lib/systemd/system/
scp kube-scheduler.service k8s-master3:/usr/lib/systemd/system/3.7 启动服务
systemctl daemon-reload
systemctl enable --now kube-scheduler
systemctl status kube-schedulerkubectl get componentstatuses[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-KAnxrdBq-1691074083716)(img/2023-08-02-22-15-45.png)]## 1. 部署kubectl
1.1 创建kubectl证书请求文件
在master1执行
[rootk8s-master1 /]#cd /data/k8s-work
cat admin-csr.json EOF
{CN: admin,hosts: [], #通过conf文件对集群进行访问所以这里不指定主机key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:masters, OU: system}]
}
EOF说明后续 kube-apiserver 使用 RBAC 对客户端(如 kubelet、kube-proxy、Pod)请求进行授权
kube-apiserver 预定义了一些 RBAC 使用的 RoleBindings如 cluster-admin 将 Group system:masters 与 Role cluster-admin 绑定该 Role 授予了调用kube-apiserver 的所有 API的权限
O指定该证书的 Group 为 system:masterskubelet 使用该证书访问 kube-apiserver 时 由于证书被 CA 签名所以认证通过同时由于证书用户组为经过预授权的 system:masters所以被授予访问所有 API 的权限
注
这个admin 证书是将来生成管理员用的kubeconfig 配置文件用的现在我们一般建议使用RBAC 来对kubernetes 进行角色权限控制 kubernetes 将证书中的CN 字段 作为User O 字段作为 Group
O: system:masters, 必须是system:masters否则后面kubectl create clusterrolebinding报错。1.2 生成证书文件
cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes admin-csr.json | cfssljson -bare admin1.3 复制文件到指定目录
cp admin*.pem /etc/kubernetes/ssl/1.4 生成kubeconfig配置文件
kube.config 为 kubectl 的配置文件包含访问 apiserver 的所有信息如 apiserver 地址、CA 证书和自身使用的证书
kubectl config set-cluster kubernetes --certificate-authorityca.pem --embed-certstrue --serverhttps://192.168.10.100:6443 --kubeconfigkube.config# cat kube.config#设置管理员证书
kubectl config set-credentials admin --client-certificateadmin.pem --client-keyadmin-key.pem --embed-certstrue --kubeconfigkube.config# cat kube.config#设置安全上下文
kubectl config set-context kubernetes --clusterkubernetes --useradmin --kubeconfigkube.config# cat kube.configkubectl config use-context kubernetes --kubeconfigkube.config1.5 准备kubectl配置文件并进行角色绑定
mkdir ~/.kube
cp kube.config ~/.kube/config
kubectl create clusterrolebinding kube-apiserver:kubelet-apis --clusterrolesystem:kubelet-api-admin --user kubernetes --kubeconfig/root/.kube/config1.6 查看集群状态
export KUBECONFIG$HOME/.kube/configkubectl cluster-info
kubectl get componentstatuses
kubectl get all --all-namespaces1.7 同步kubectl配置文件到集群其它master节点
k8s-master2:
mkdir /root/.kubek8s-master3:
mkdir /root/.kubescp /root/.kube/config k8s-master2:/root/.kube/config
scp /root/.kube/config k8s-master3:/root/.kube/config1.8 配置kubectl命令补全(可选)
yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source (kubectl completion bash)
kubectl completion bash ~/.kube/completion.bash.inc
source /root/.kube/completion.bash.inc
source $HOME/.bash_profile2. 部署kube-controller-manager
2.1 创建kube-controller-manager证书请求文件
在master1执行
[rootk8s-master1 k8s-work]#cat kube-controller-manager-csr.json EOF
{CN: system:kube-controller-manager,key: {algo: rsa,size: 2048},hosts: [127.0.0.1,192.168.10.103, #三台master的ip192.168.10.104,192.168.10.105],names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-controller-manager,OU: system}]
}
EOF说明hosts 列表包含所有 kube-controller-manager 节点 IP
CN 为 system:kube-controller-manager、O 为 system:kube-controller-managerkubernetes 内置的 ClusterRoleBindings system:kube-controller-manager 赋予 kube-controller-manager 工作所需的权限2.2 创建kube-controller-manager证书文件
cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager# lskube-controller-manager.csr
kube-controller-manager-csr.json
kube-controller-manager-key.pem
kube-controller-manager.pem2.3 创建kube-controller-manager的kube-controller-manager.kubeconfig
kubectl config set-cluster kubernetes --certificate-authorityca.pem --embed-certstrue --serverhttps://192.168.10.100:6443 --kubeconfigkube-controller-manager.kubeconfigkubectl config set-credentials system:kube-controller-manager --client-certificatekube-controller-manager.pem --client-keykube-controller-manager-key.pem --embed-certstrue --kubeconfigkube-controller-manager.kubeconfigkubectl config set-context system:kube-controller-manager --clusterkubernetes --usersystem:kube-controller-manager --kubeconfigkube-controller-manager.kubeconfigkubectl config use-context system:kube-controller-manager --kubeconfigkube-controller-manager.kubeconfig2.4 创建kube-controller-manager配置文件
cat kube-controller-manager.conf EOF
KUBE_CONTROLLER_MANAGER_OPTS--port10252 \--secure-port10257 \--bind-address127.0.0.1 \--kubeconfig/etc/kubernetes/kube-controller-manager.kubeconfig \--service-cluster-ip-range10.96.0.0/16 \--cluster-namekubernetes \--cluster-signing-cert-file/etc/kubernetes/ssl/ca.pem \--cluster-signing-key-file/etc/kubernetes/ssl/ca-key.pem \--allocate-node-cidrstrue \--cluster-cidr10.244.0.0/16 \--experimental-cluster-signing-duration87600h \--root-ca-file/etc/kubernetes/ssl/ca.pem \--service-account-private-key-file/etc/kubernetes/ssl/ca-key.pem \--leader-electtrue \--feature-gatesRotateKubeletServerCertificatetrue \--controllers*,bootstrapsigner,tokencleaner \--horizontal-pod-autoscaler-use-rest-clientstrue \--horizontal-pod-autoscaler-sync-period10s \--tls-cert-file/etc/kubernetes/ssl/kube-controller-manager.pem \--tls-private-key-file/etc/kubernetes/ssl/kube-controller-manager-key.pem \--use-service-account-credentialstrue \--alsologtostderrtrue \--logtostderrfalse \--log-dir/var/log/kubernetes \--v2
EOF2.5 创建服务启动文件
[rootk8s-master1 k8s-work]# cat kube-controller-manager.service EOF
[Unit]
DescriptionKubernetes Controller Manager
Documentationhttps://github.com/kubernetes/kubernetes[Service]
EnvironmentFile-/etc/kubernetes/kube-controller-manager.conf
ExecStart/usr/local/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_OPTS
Restarton-failure
RestartSec5[Install]
WantedBymulti-user.target
EOF2.6 同步文件到集群master节点
cp kube-controller-manager*.pem /etc/kubernetes/ssl/
cp kube-controller-manager.kubeconfig /etc/kubernetes/
cp kube-controller-manager.conf /etc/kubernetes/
cp kube-controller-manager.service /usr/lib/systemd/system/scp kube-controller-manager*.pem k8s-master2:/etc/kubernetes/ssl/
scp kube-controller-manager*.pem k8s-master3:/etc/kubernetes/ssl/
scp kube-controller-manager.kubeconfig kube-controller-manager.conf k8s-master2:/etc/kubernetes/
scp kube-controller-manager.kubeconfig kube-controller-manager.conf k8s-master3:/etc/kubernetes/
scp kube-controller-manager.service k8s-master2:/usr/lib/systemd/system/
scp kube-controller-manager.service k8s-master3:/usr/lib/systemd/system/#查看证书
openssl x509 -in /etc/kubernetes/ssl/kube-controller-manager.pem -noout -text2.7 启动服务
systemctl daemon-reload
systemctl enable --now kube-controller-manager
systemctl status kube-controller-managerkubectl get componentstatuses3. 部署kube-scheduler
3.1 创建kube-scheduler证书请求文件
[rootk8s-master1 k8s-work]# cat kube-scheduler-csr.json EOF
{CN: system:kube-scheduler,hosts: [127.0.0.1,192.168.10.103,192.168.10.104,192.168.10.105],key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-scheduler,OU: system}]
}
EOF3.2 生成kube-scheduler证书
cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler# ls
kube-scheduler.csr
kube-scheduler-csr.json
kube-scheduler-key.pem
kube-scheduler.pem3.3 创建kube-scheduler的kubeconfig
kubectl config set-cluster kubernetes --certificate-authorityca.pem --embed-certstrue --serverhttps://192.168.10.100:6443 --kubeconfigkube-scheduler.kubeconfigkubectl config set-credentials system:kube-scheduler --client-certificatekube-scheduler.pem --client-keykube-scheduler-key.pem --embed-certstrue --kubeconfigkube-scheduler.kubeconfigkubectl config set-context system:kube-scheduler --clusterkubernetes --usersystem:kube-scheduler --kubeconfigkube-scheduler.kubeconfigkubectl config use-context system:kube-scheduler --kubeconfigkube-scheduler.kubeconfig3.4 创建服务配置文件
cat kube-scheduler.conf EOF
KUBE_SCHEDULER_OPTS--address127.0.0.1 \
--kubeconfig/etc/kubernetes/kube-scheduler.kubeconfig \
--leader-electtrue \
--alsologtostderrtrue \
--logtostderrfalse \
--log-dir/var/log/kubernetes \
--v2
EOF3.5创建服务启动配置文件
cat kube-scheduler.service EOF
[Unit]
DescriptionKubernetes Scheduler
Documentationhttps://github.com/kubernetes/kubernetes[Service]
EnvironmentFile-/etc/kubernetes/kube-scheduler.conf
ExecStart/usr/local/bin/kube-scheduler $KUBE_SCHEDULER_OPTS
Restarton-failure
RestartSec5[Install]
WantedBymulti-user.target
EOF3.6 同步文件至集群master节点
cp kube-scheduler*.pem /etc/kubernetes/ssl/
cp kube-scheduler.kubeconfig /etc/kubernetes/
cp kube-scheduler.conf /etc/kubernetes/
cp kube-scheduler.service /usr/lib/systemd/system/scp kube-scheduler*.pem k8s-master2:/etc/kubernetes/ssl/
scp kube-scheduler*.pem k8s-master3:/etc/kubernetes/ssl/
scp kube-scheduler.kubeconfig kube-scheduler.conf k8s-master2:/etc/kubernetes/
scp kube-scheduler.kubeconfig kube-scheduler.conf k8s-master3:/etc/kubernetes/
scp kube-scheduler.service k8s-master2:/usr/lib/systemd/system/
scp kube-scheduler.service k8s-master3:/usr/lib/systemd/system/3.7 启动服务
systemctl daemon-reload
systemctl enable --now kube-scheduler
systemctl status kube-schedulerkubectl get componentstatuses
