| View previous topic :: View next topic |
| Author |
Message |
leonchik1976
Guru
Joined: 24 Jan 2010
Posts: 325
|
 Posted: Sun Jul 09, 2023 6:21 am Post subject: kubeadm issue |
 |
|
hi, trying to bootstrap kubernetes cluster with kubeadm and got following errors, though kubelet seems running and no crashed components found. Anyone faced such issue?
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher |
|
| Back to top |
|
 |
alamahant
Advocate
Joined: 23 Mar 2019
Posts: 3879
|
| Posted: Sun Jul 09, 2023 10:16 am Post subject: |
|
|
How many nodes are you using?
Have you installed docker kubeadm kubelet and kubectl in all nodes?
Is kubelet running and enabled in all nodes?
Did you initialize the control plane in master node?
Something like
| Code: |
kubeadm init --apiserver-advertise-address=<ip-of-master-node>
|
?
_________________
Last edited by alamahant on Sun Jul 09, 2023 10:24 am; edited 1 time in total |
|
| Back to top |
|
|
leonchik1976
Guru
Joined: 24 Jan 2010
Posts: 325
|
| Posted: Sun Jul 09, 2023 10:17 am Post subject: |
|
|
| alamahant wrote: | How many nodes are you using?
Have you installed docker kubeadm kubelet and kubectl in all nodes?
Did you initialize the control plane in master node?
Something like
| Code: |
kubeadm init --apiserver-advertise-address=<ip-of-master-node>
|
? |
all prerequisites installed (docker, kubeadm, kubelet....)
the error is on kubeadm init command |
|
| Back to top |
|
|
alamahant
Advocate
Joined: 23 Mar 2019
Posts: 3879
|
| Posted: Sun Jul 09, 2023 10:25 am Post subject: |
|
|
Is kubelet running and enabled in all nodes?
Do you have a properly configured kernel for networking?
What does
| Code: |
journalctl -xeu kubelet
|
say?
_________________
|
|
| Back to top |
|
|
leonchik1976
Guru
Joined: 24 Jan 2010
Posts: 325
|
| Posted: Sun Jul 09, 2023 10:27 am Post subject: |
|
|
| alamahant wrote: | Is kubelet running and enabled in all nodes?
Do you have a properly configured kernel for networking? |
kubelet running and enabled in all nodes - what do you mean? control plane still not exists |
|
| Back to top |
|
|
alamahant
Advocate
Joined: 23 Mar 2019
Posts: 3879
|
| Posted: Sun Jul 09, 2023 10:37 am Post subject: |
|
|
Plz check relevant logs for kubelet.
Also check cgroups.
Swap between unified and hybrid and see what works.
A reboot is necessary.
Also plz check if using the binary kernel helps.In which case your kernel may lack config.
Also plz run "kubeadm init" as root.
Additionally turn swap off
| Code: |
swapoff /dev/<my-swap>
|
as in my case is seen in
| Code: |
cat /var/log/kubelet/kubelet.log
I0709 13:44:18.121792 11095 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
I0709 13:44:18.121865 11095 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0709 13:44:18.122099 11095 server.go:578] "Standalone mode, no API client"
I0709 13:44:18.126540 11095 server.go:466] "No api server defined - no events will be sent to API server"
I0709 13:44:18.126561 11095 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
E0709 13:44:18.126765 11095 run.go:74] "command failed" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority /dev/dm-1 partition\t16777212\t0\t\t-2]"
I0709 13:46:22.480473 4171 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
I0709 13:46:22.480712 4171 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0709 13:46:22.481421 4171 server.go:578] "Standalone mode, no API client"
I0709 13:46:22.487890 4171 server.go:466] "No api server defined - no events will be sent to API server"
I0709 13:46:22.487910 4171 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
E0709 13:46:22.488737 4171 run.go:74] "command failed" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority /dev/dm-1 partition\t16777212\t0\t\t-2]"
|
NOTE:Ok when swapping off kubelet starts just fine.
_________________
|
|
| Back to top |
|
|
leonchik1976
Guru
Joined: 24 Jan 2010
Posts: 325
|
| Posted: Mon Jul 10, 2023 5:34 pm Post subject: |
|
|
| alamahant wrote: | Plz check relevant logs for kubelet.
Also check cgroups.
Swap between unified and hybrid and see what works.
A reboot is necessary.
Also plz check if using the binary kernel helps.In which case your kernel may lack config.
Also plz run "kubeadm init" as root.
Additionally turn swap off
| Code: |
swapoff /dev/<my-swap>
|
as in my case is seen in
| Code: |
cat /var/log/kubelet/kubelet.log
I0709 13:44:18.121792 11095 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
I0709 13:44:18.121865 11095 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0709 13:44:18.122099 11095 server.go:578] "Standalone mode, no API client"
I0709 13:44:18.126540 11095 server.go:466] "No api server defined - no events will be sent to API server"
I0709 13:44:18.126561 11095 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
E0709 13:44:18.126765 11095 run.go:74] "command failed" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority /dev/dm-1 partition\t16777212\t0\t\t-2]"
I0709 13:46:22.480473 4171 server.go:415] "Kubelet version" kubeletVersion="v1.27.3"
I0709 13:46:22.480712 4171 server.go:417] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0709 13:46:22.481421 4171 server.go:578] "Standalone mode, no API client"
I0709 13:46:22.487890 4171 server.go:466] "No api server defined - no events will be sent to API server"
I0709 13:46:22.487910 4171 server.go:662] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /"
E0709 13:46:22.488737 4171 run.go:74] "command failed" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority /dev/dm-1 partition\t16777212\t0\t\t-2]"
|
NOTE:Ok when swapping off kubelet starts just fine. |
swap not exists at all, and kubeadm init of course i run as root. only binary kernel still didn't tried |
|
| Back to top |
|
|
leonchik1976
Guru
Joined: 24 Jan 2010
Posts: 325
|
| Posted: Mon Jul 10, 2023 7:49 pm Post subject: |
|
|
| i couldn't make dist kernel to boot in any way, but all pre-flight checks passes. anyway, i think i will give it up, because on another distribution the same works without any issues |
|
| Back to top |
|
|
leonchik1976
Guru
Joined: 24 Jan 2010
Posts: 325
|
|
| Back to top |
|
|
|
Installing Gentoo
