kubelet 주요 이슈 확인

kubelet이 기동할때 호출되는 구성요소

$ service kubelet status

● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Thu 2021-04-08 07:05:20 UTC; 3min 13s ago
     Docs: https://kubernetes.io/docs/home/
 Main PID: 30579 (kubelet)
    Tasks: 14 (limit: 4678)
   CGroup: /system.slice/kubelet.service
           └─30579 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --network-plugin=cni --pod-infra-container-image=k8s.gcr

-> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

kubeadm이 kubelet 구성시 위 구성 요소를 보고 kubelet을 구성함

 

/etc/systemd/system/kubelet.service.d/10-kubeadm.conf 내용을 확인하면

$ cat /etc/systemd/system/kubelet.service.d//10-kubeadm.conf

# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

주요 구성 파일로는 KUBELET_KUBECONFIG ARG 구성 파일인 

--kubeconfig=/etc/kubernetes/kubelet.conf

와 KUBELET CONFIG ARG 구성 파일인 

--config=/var/lib/kubelet/config.yaml

이 있다.

 

/etc/kubernetes/kubelet.conf안에 내용을 확인해 보면

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EUXdPREEyTURBME9Wb1hEVE14TURRd05qQTJNREEwT1Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTFBVClg4VzM5WWV5elVZc0xuTGRIUjdYZng4R2hqanQxaWNObVpHaU10R2hiTmhlNGQ1VElCYUxPTmZMTFVkVjQ0d1IKV2hYUThjQVJlUUNhNnQrRnYwL3E4dFN5T0YvS0t5UnViRWQ4OFkxTDF5NUJyYmZSTTJiUjVCeTFZdkJ2S0FpVQpMcWpwYnVzd0dNM0NHa0JlbWxLNExYSGR4MitqUVllZlFKQzg0ZER6WFc0YjZHRmVhaG41eGxib0Y2ZGZsbUQ0CkJwYkpnNDBhbkRld0pjMHI4OHJNeFczQ1VSMFUzdUUxVVEzY201dSt4YzVZb1RxM0ZkSnA0blhKT29kaTI3eGUKVmxZT1Q1ciszdnQ3VlN4SVNRcFFPdUZPdEtxSE5nYnpnVC91UktveDhHVVE5OVk1cFBzRWlXZklIS1pxUDhGYQpJSFRsOWU0bDhoS1dEM2xMRGJjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZHdDEzdHI4cU56R2grTWZKUk5DcERRQVlFVC9NQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFDazNIVWRpSnJnUDVyRGFYNkYzdkpXL2RLTE1jRkt1MjgvTVFCWW9KZXByUmtvMHZ2NgpENFc5Tnl5c1lrYXcyRGtMMjNhT1MxUkVjQkkrbjliT1UxYm1OY2JhMkprYzIxRmdTYVFvT1Rsa3U2dVlvMFhZCnhoVFBiczlqTmN0ckY2UTZ4Z1YrYXduYzhlWlAxWHptUjAzRldOS1N6aTFjVUZTU2c0Y21kY2dXM0ZPY2xkZ1cKaDh4cVZCaWp1bUZJWnVGWlVVRjhyVUpLWTlOTUtaMldrMUxzVXl5WVFtVDQzaFJjVCtvZFA4Rkc0V1VpQjZOeQpteHFXWUprOVI3b2hBL2ErVU5hVmtiUzJHcFgyMlpsQlFoekxYMXpoaDZnUzMxWXpjQU5ZUXhMRzRRZ2xZTVBJCldadXFoQ0pKLzBMUkRtelRFbzZrN3V4TEVxNzZjTytlbUNmLwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://172.17.0.11:6443
  name: default-cluster
contexts:
- context:
    cluster: default-cluster
    namespace: default
    user: default-auth
  name: default-context
current-context: default-context
kind: Config
preferences: {}
users:
- name: default-auth
  user:
    client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem
    client-key: /var/lib/kubelet/pki/kubelet-client-current.pem

이 파일로 kubernetes cluster에 연결할 서버 정보와 인증서 및 계정 정보를 갖게 된다.  따라서 kubernetes api 서버와 연결이 안되는 경우 이 kubelete.conf 파일을 확인한다.

 

 

/var/lib/kubelet/config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 0s
    cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
cpuManagerReconcilePeriod: 0s
evictionPressureTransitionPeriod: 0s
fileCheckFrequency: 0s
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
logging: {}
nodeStatusReportFrequency: 0s
nodeStatusUpdateFrequency: 0s
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 0s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
volumeStatsAggPeriod: 0s

이 파일을 통해 kubelet이 구성되는 주요 파일을 설정한다. 실제 정상적으로 kubelet이 기동이 안되는 경우 여기의 DNS와 CA 구성이 잘못되어 그럴수 있다.

또한 staticPod 생성위치를 기록하고 있어 staticPod가 생성이 안되는경우 이 파일을 확인한다.