kubernetes基础知识（kubernetes炼气期之k8s平台快速搭建）

原创 Marionxue 云原生生态圈

来自专辑

kubernetes实践案例

环境说明

功能名称IP配置k8s-master192.168.10.2314c8gk8s-node1192.168.10.2328c16gK8s-node2192.168.10.2338c16gk8s-node3192.168.10.2348c16gk8s-node4192.168.10.2358c16g

环境初始化

1. 更新环境

yumupdate-y yuminstall-ywgetvimnet-toolsepel-release

2. 关闭filewalld

systemctldisablefirewalld systemctlstopfirewalld

3. 关闭selinux

sed-i"s/SELINUX=enforcing/SELINUX=disabled/g"/etc/selinux/config sed-i"s/SELINUX=enforcing/SELINUX=disabled/g"/etc/sysconfig/selinux if[`getenforce`=="Enforcing"];then setenforce0 else echo"currentselinuxstatus..."`getenforce` fi

4. 关闭swap

swapoff-a sed-i's/.*swap.*/#&/'/etc/fstab

5. 增加主机名解析

cat<<EOF/etc/host 192.168.10.231dev-k8s-01.example.com 192.168.10.232dev-k8s-02.example.com 192.168.10.233dev-k8s-03.example.com 192.168.10.234dev-k8s-04.example.com 192.168.10.235dev-k8s-05.example.com EOF 6.优化内核参数 ```bash cat<<EOF>>/etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables=1 net.bridge.bridge-nf-call-iptables=1 EOF sysctl--system

7. 更新Yum源配置

mv/etc/yum.repos.d/CentOS-Base.repo/etc/yum.repos.d/CentOS-Base.repo.`date %F`.backup wget-O/etc/yum.repos.d/CentOS-Base.repohttp://mirrors.aliyun.com/repo/Centos-7.repo yummakecachefast cat<<EOF/etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 EOF yumcleanall yummakecachefast yum-yupdate

8. 安装docker

yum-yinstallyum-utilsdevice-mapper-persistent-datalvm2 yum-config-manager--add-repohttp://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo yuminstall-ydocker-ce-18.09.9-3.el7 mkdir/etc/docker-pv cat>/etc/docker/daemon.json<<EOF { "registry-mirrors":["https://********.mirror.aliyuncs.com"], "exec-opts":["native.cgroupdriver=systemd"], "log-driver":"json-file", "log-opts":{ "max-size":"100m" }, "storage-driver":"overlay2", "storage-opts":[ "overlay2.override_kernel_check=true" ] } EOF systemctlenable--nowdocker.service

9. 安装初始化工具

yuminstall-ykubeadmkubelet

10. 获取基础镜像

KUBE_VERSION=v1.16.0 KUBE_PAUSE_VERSION=3.1 ETCD_VERSION=3.3.15-0 CORE_DNS_VERSION=1.6.2 GCR_URL=k8s.gcr.io ALIYUN_URL=registry.cn-hangzhou.aliyuncs.com/google_containers images=(kube-proxy:${KUBE_VERSION} kube-scheduler:${KUBE_VERSION} kube-controller-manager:${KUBE_VERSION} kube-apiserver:${KUBE_VERSION} pause:${KUBE_PAUSE_VERSION} etcd:${ETCD_VERSION} coredns:${CORE_DNS_VERSION}) forimageNamein${images[@]};do dockerpull$ALIYUN_URL/$imageName dockertag$ALIYUN_URL/$imageName$GCR_URL/$imageName dockerrmi$ALIYUN_URL/$imageName done

以上10部建议在所有的节点上安装,在node节点上可以不用安装kubeadm

部署集群

kubeadm初始化集群

[root@dev-k8s-01~]#sudokubeadminit\ >--apiserver-advertise-address192.168.10.231\ >--kubernetes-version=v1.16.0\ >--pod-network-cidr=10.244.0.0/16 [init]UsingKubernetesversion:v1.16.0 [preflight]Runningpre-flightchecks [WARNINGService-Kubelet]:kubeletserviceisnotenabled,pleaserun'systemctlenablekubelet.service' [preflight]PullingimagesrequiredforsettingupaKubernetescluster [preflight]Thismighttakeaminuteortwo,dependingonthespeedofyourinternetconnection [preflight]Youcanalsoperformthisactioninbeforehandusing'kubeadmconfigimagespull' [kubelet-start]Writingkubeletenvironmentfilewithflagstofile"/var/lib/kubelet/kubeadm-flags.env" [kubelet-start]Writingkubeletconfigurationtofile"/var/lib/kubelet/config.yaml" [kubelet-start]Activatingthekubeletservice [certs]UsingcertificateDirfolder"/etc/kubernetes/pki" [certs]Generating"ca"certificateandkey [certs]Generating"apiserver"certificateandkey [certs]apiserverservingcertissignedforDNSnames[dev-k8s-01.example.comkuberneteskubernetes.defaultkubernetes.default.svckubernetes.default.svc.cluster.local]andIPs[10.96.0.1192.168.10.231] [certs]Generating"apiserver-kubelet-client"certificateandkey [certs]Generating"front-proxy-ca"certificateandkey [certs]Generating"front-proxy-client"certificateandkey [certs]Generating"etcd/ca"certificateandkey [certs]Generating"etcd/server"certificateandkey [certs]etcd/serverservingcertissignedforDNSnames[dev-k8s-01.example.comlocalhost]andIPs[192.168.10.231127.0.0.1::1] [certs]Generating"etcd/peer"certificateandkey [certs]etcd/peerservingcertissignedforDNSnames[dev-k8s-01.example.comlocalhost]andIPs[192.168.10.231127.0.0.1::1] [certs]Generating"etcd/healthcheck-client"certificateandkey [certs]Generating"apiserver-etcd-client"certificateandkey [certs]Generating"sa"keyandpublickey [kubeconfig]Usingkubeconfigfolder"/etc/kubernetes" [kubeconfig]Writing"admin.conf"kubeconfigfile [kubeconfig]Writing"kubelet.conf"kubeconfigfile [kubeconfig]Writing"controller-manager.conf"kubeconfigfile [kubeconfig]Writing"scheduler.conf"kubeconfigfile [control-plane]Usingmanifestfolder"/etc/kubernetes/manifests" [control-plane]CreatingstaticPodmanifestfor"kube-apiserver" [control-plane]CreatingstaticPodmanifestfor"kube-controller-manager" [control-plane]CreatingstaticPodmanifestfor"kube-scheduler" [etcd]CreatingstaticPodmanifestforlocaletcdin"/etc/kubernetes/manifests" [wait-control-plane]WaitingforthekubelettobootupthecontrolplaneasstaticPodsfromdirectory"/etc/kubernetes/manifests".Thiscantakeupto4m0s [apiclient]Allcontrolplanecomponentsarehealthyafter39.003840seconds [upload-config]StoringtheconfigurationusedinConfigMap"kubeadm-config"inthe"kube-system"Namespace [kubelet]CreatingaConfigMap"kubelet-config-1.16"innamespacekube-systemwiththeconfigurationforthekubeletsinthecluster [upload-certs]Skippingphase.Pleasesee--upload-certs [mark-control-plane]Markingthenodedev-k8s-01.example.comascontrol-planebyaddingthelabel"node-role.kubernetes.io/master=''" [mark-control-plane]Markingthenodedev-k8s-01.example.comascontrol-planebyaddingthetaints[node-role.kubernetes.io/master:NoSchedule] [kubelet-check]Initialtimeoutof40spassed. [bootstrap-token]Usingtoken:9nwjok.ykyphybsveka8gev [bootstrap-token]Configuringbootstraptokens,cluster-infoConfigMap,RBACRoles [bootstrap-token]configuredRBACrulestoallowNodeBootstraptokenstopostCSRsinorderfornodestogetlongtermcertificatecredentials [bootstrap-token]configuredRBACrulestoallowthecsrapprovercontrollerautomaticallyapproveCSRsfromaNodeBootstrapToken [bootstrap-token]configuredRBACrulestoallowcertificaterotationforallnodeclientcertificatesinthecluster [bootstrap-token]Creatingthe"cluster-info"ConfigMapinthe"kube-public"namespace [addons]Appliedessentialaddon:CoreDNS [addons]Appliedessentialaddon:kube-proxy YourKubernetescontrol-planehasinitializedsuccessfully! Tostartusingyourcluster,youneedtorunthefollowingasaregularuser: mkdir-p$HOME/.kube sudocp-i/etc/kubernetes/admin.conf$HOME/.kube/config sudochown$(id-u):$(id-g)$HOME/.kube/config Youshouldnowdeployapodnetworktothecluster. Run"kubectlapply-f[podnetwork].yaml"withoneoftheoptionslistedat: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Thenyoucanjoinanynumberofworkernodesbyrunningthefollowingoneachasroot: kubeadmjoin192.168.10.231:6443--token9nwjok.ykyphybsveka8gev\ --discovery-token-ca-cert-hashsha256:b92d7553a1da683a315ad2f4f5fcc855e2d630da0c7553467cdf2db3bd25a3ff

初始化kubectl配置文件

[root@dev-k8s-01~]#mkdir-p$HOME/.kube [root@dev-k8s-01~]#sudocp-i/etc/kubernetes/admin.conf$HOME/.kube/config [root@dev-k8s-01~]#sudochown$(id-u):$(id-g)$HOME/.kube/config

添加节点

• 添加192.168.10.232

[root@dev-k8s-05~]#kubeadmjoin192.168.10.231:6443--token9pr3rj.0u8m510fai0op75h\ --discovery-token-ca-cert-hashsha256:b86bdaaa0bed56e846adb0abc625cf29902dec9e3130d0ff7dae42ffb2e13349 [preflight]Runningpre-flightchecks [WARNINGService-Kubelet]:kubeletserviceisnotenabled,pleaserun'systemctlenablekubelet.service' [preflight]Readingconfigurationfromthecluster... [preflight]FYI:Youcanlookatthisconfigfilewith'kubectl-nkube-systemgetcmkubeadm-config-oyaml' [kubelet-start]Downloadingconfigurationforthekubeletfromthe"kubelet-config-1.16"ConfigMapinthekube-systemnamespace [kubelet-start]Writingkubeletconfigurationtofile"/var/lib/kubelet/config.yaml" [kubelet-start]Writingkubeletenvironmentfilewithflagstofile"/var/lib/kubelet/kubeadm-flags.env" [kubelet-start]Activatingthekubeletservice [kubelet-start]WaitingforthekubelettoperformtheTLSBootstrap... Thisnodehasjoinedthecluster: *Certificatesigningrequestwassenttoapiserverandaresponsewasreceived. *TheKubeletwasinformedofthenewsecureconnectiondetails. Run'kubectlgetnodes'onthecontrol-planetoseethisnodejointhecluster. [root@dev-k8s-05~]#systemctlenablekubelet.service Createdsymlinkfrom/etc/systemd/system/multi-user.target.wants/kubelet.serviceto/usr/lib/systemd/system/kubelet.service. [root@dev-k8s-05~]#

如上所示依旧添加192.168.10.233节点

验证集群状态

[root@dev-k8s-01~]#kubectlcluster-info Kubernetesmasterisrunningathttps://192.168.10.231:6443 KubeDNSisrunningathttps://192.168.10.231:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy Tofurtherdebuganddiagnoseclusterproblems,use'kubectlcluster-infodump'. ➜~(☸kubernetes-admin@kubernetes:default)kubectlgetnodes-owide NAMESTATUSROLESAGEVERSIONINTERNAL-IPEXTERNAL-IPOS-IMAGEKERNEL-VERSIONCONTAINER-RUNTIME dev-k8s-01.example.comReadymaster14hv1.16.3192.168.10.231<none>CentOSLinux7(Core)3.10.0-957.21.3.el7.x86_64docker://18.9.9 dev-k8s-02.example.comReady<none>14hv1.16.3192.168.10.232<none>CentOSLinux7(Core)3.10.0-957.21.3.el7.x86_64docker://18.9.9 dev-k8s-03.example.comReady<none>14hv1.16.3192.168.10.233<none>CentOSLinux7(Core)3.10.0-957.21.3.el7.x86_64docker://18.9.9 dev-k8s-04.example.comReady<none>14hv1.16.3192.168.10.234<none>CentOSLinux7(Core)3.10.0-1062.4.1.el7.x86_64docker://18.9.9 dev-k8s-05.example.comReady<none>13hv1.16.3192.168.10.235<none>CentOSLinux7(Core)3.10.0-957.el7.x86_64docker://18.9.9 ➜~(☸kubernetes-admin@kubernetes:default)kubectlgetpods--all-namespaces-owide NAMESPACENAMEREADYSTATUSRESTARTSAGEIPNODENOMINATEDNODEREADINESSGATES kube-systemcoredns-5644d7b6d9-96xm61/1Running014h10.244.3.2dev-k8s-04.example.com<none><none> kube-systemcoredns-5644d7b6d9-nkb9f1/1Running014h10.244.1.2dev-k8s-02.example.com<none><none> kube-systemetcd-dev-k8s-01.example.com1/1Running014h192.168.10.231dev-k8s-01.example.com<none><none> kube-systemkube-apiserver-dev-k8s-01.example.com1/1Running014h192.168.10.231dev-k8s-01.example.com<none><none> kube-systemkube-controller-manager-dev-k8s-01.example.com1/1Running014h192.168.10.231dev-k8s-01.example.com<none><none> kube-systemkube-proxy-bhtjc1/1Running014h192.168.10.232dev-k8s-02.example.com<none><none> kube-systemkube-proxy-h2ltx1/1Running014h192.168.10.231dev-k8s-01.example.com<none><none> kube-systemkube-proxy-kh9k91/1Running014h192.168.10.234dev-k8s-04.example.com<none><none> kube-systemkube-proxy-lfh461/1Running014h192.168.10.233dev-k8s-03.example.com<none><none> kube-systemkube-proxy-pcm5d1/1Running013h192.168.10.235dev-k8s-05.example.com<none><none> kube-systemkube-scheduler-dev-k8s-01.example.com1/1Running014h192.168.10.231dev-k8s-01.example.com<none><none>

安装插件

安装flannel网络插件

wget-O/opt/k8sworkspces/kube-flannel.ymlhttps://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml FLANNEL_VERSION=v0.11.0 QUAY_URL=quay.io/coreos QINIU_URL=quay-mirror.qiniu.com/coreos images=(flannel:${FLANNEL_VERSION}-amd64 flannel:${FLANNEL_VERSION}-arm64 flannel:${FLANNEL_VERSION}-arm flannel:${FLANNEL_VERSION}-ppc64le flannel:${FLANNEL_VERSION}-s390x) forimageNamein${images[@]};do dockerpull$QINIU_URL/$imageName dockertag$QINIU_URL/$imageName$QUAY_URL/$imageName dockerrmi$QINIU_URL/$imageName done#也可以只拉去你机器适配的架构版本`rpm-qcentos-release` kubectlapply-f/opt/k8sworkspces/kube-flannel.yml#安装flannel ➜~(☸kubernetes-admin@kubernetes:default)kubectlgetpods--all-namespaces-owide|grepflannel kube-systemkube-flannel-ds-amd64-9tnc71/1Running014h192.168.10.234dev-k8s-04.example.com<none><none> kube-systemkube-flannel-ds-amd64-cjh4s1/1Running014h192.168.10.231dev-k8s-01.example.com<none><none> kube-systemkube-flannel-ds-amd64-fhlk41/1Running013h192.168.10.235dev-k8s-05.example.com<none><none> kube-systemkube-flannel-ds-amd64-fnfpj1/1Running014h192.168.10.233dev-k8s-03.example.com<none><none> kube-systemkube-flannel-ds-amd64-v5qtj1/1Running014h192.168.10.232dev-k8s-02.example.com<none><none>

krew

krew 能够很方便的管理kubectl的插件包,包括安装卸载，查询升级

安装

( set-x;cd/opt/k8sworkspces/krew&& curl-fsSLO"https://github.com/kubernetes-sigs/krew/releases/download/v0.3.2/krew.{tar.gz,yaml}"&& tarzxvfkrew.tar.gz&& ./krew-"$(uname|tr'[:upper:]''[:lower:]')_amd64"install\ --manifest=krew.yaml--archive=krew.tar.gz ) exportPATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH" [root@dev-k8s-01krew]#kubectlkrewinstallca-cert#安装一个ca-caert的插件 [root@dev-k8s-01krew]#kubectlca-cert -----BEGINCERTIFICATE----- MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl cm5ldGVzMB4XDTE5MTExNTA0MjEzOVoXDTI5MTExMjA0MjEzOVowFTETMBEGA1UE AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANEi tPdWINQZfZqM4c/uaOzsBBByn0CLLLmMdiKF4Gpk9proDoR9eOMhQiiVLZ4tFFsb POTwq MvHe4kEsunl/hBwNbXvGfbvnr vX9ZsDfU5FT5O55Zryq5jgANDKFChKx9 R91QsbCeQKIWlc9AFdot8ig9LhYTfHJRfMeUBYl5Xzoof8YRMsJ0jOKLWca oCfd doLKda9VpahU2AEmEFHuD6ctwBGFObadSktoOvr0Gfzo4cXRkjGXp4G1U8O1LLsU HiypNN4m7Romy4tIjPAxDAoDDyjA8OrbPlvJt8Oo0CHcAxFZDJCsKAG1s0nS7PJj vR2ULtIrHAm5QZa8BmMCAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB /wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAKAi1Fg/2MlFxPbq9yaNkBhAV2ou /VbbuEJF1c92Tk24cuJV3vuYoTmWIGp1LYTLTW/xcfwFoanLRPBlBONoJRzXLIZD /mmuYMrTaKMwbCz2t4awqQyDb8A3RcgTrSfCWMs0uyvjPVgiJDfMlg0WDJ4kPb3Y SQv7UaaNa57gkEHB1PJy10n1E3gAcb6NVxvly7cHVaJlenZY6mkT40K8zVOXuM/G ausCNXEfEUXED2C8Ippj/sr1TgRlD8Gfi Xp7XzHTeu5A ac4YPmnoW8jurzo5z5 Q5TDBFRaOTyRgUxYt PKv01S9tTiHgkxHoBzPQF7Z2TuRNKXoVQeXiUzW/s= -----ENDCERTIFICATE----- [root@dev-k8s-01krew]#kubectlkrew--help#查看krew的支持选项

,