Allow scheduling of pods on Kubernetes master?

Question

I set up Kubernetes on CoreOS on bare metal using the generic install scripts. It's running the current stable release, 1298.6.0, with Kubernetes version 1.5.4.

We'd like to have a highly available master setup, but we don't have enough hardware at this time to dedicate three servers to serving only as Kubernetes masters, so I would like to be able to allow user pods to be scheduled on the Kubernetes master. I set --register-schedulable=true in /etc/systemd/system/kubelet.service but it still showed up as SchedulingDisabled.

I tried to add settings for including the node as a worker, including adding worker TLS certs to /etc/kubernetes/ssl, adding those settings to kubelet.service, adding an /etc/kubernetes/worker-kubeconfig.yaml that pointed to those certs, and added that information to the /etc/kubernetes/manifests/kube-proxy.yaml. I used my existing nodes as a template for what to add. This registered another node under the master's hostname and then both it and the original master node showed up as NotReady,SchedulingDisabled.

This question indicates that scheduling pods on the master node should be possible, but there is barely anything else that I can find on the subject.

erikbstack · Accepted Answer · 2018-06-07 08:20:37Z

61

If you are using Kubernetes 1.7 and above:

kubectl taint node mymasternode node-role.kubernetes.io/master:NoSchedule-

edited Jun 7, 2018 at 8:20

erikbstack

13.3k24 gold badges87 silver badges115 bronze badges

answered Oct 21, 2017 at 18:40

Magnus Runesson

7735 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Victor G Over a year ago

I believe this command should be with a minus at the end to actually remove this taint from the master. Right?

Aryak Sengupta Over a year ago

Should be the accepted answer. @VictorG - Yes, It should be kubectl taint node dashboard2.pvi.com node-role.kubernetes.io/master:NoSchedule-

Pascal Over a year ago

Starting with 1.20 the command should be: kubectl taint node mymasternode node-role.kubernetes.io/control-plane:NoSchedule- See more: kubernetes.io/docs/reference/labels-annotations-taints/…

akahunahi Over a year ago

"Suggested edit queue is full" when trying to edit answer. @Pascal is correct and should be added to this answer. Version <1.20 vs >1.20. Console output to add: taint "node-role.kubernetes.io/master:NoSchedule" not found

TheDHM · Accepted Answer · 2024-04-26 08:22:08Z

16

node-role.kubernetes.io/master

is deprecated in favor of:

node-role.kubernetes.io/control-plane

Official kubernetes documentation: node-role-kubernetes-io-master

So for versions +v1.20 the solution is :

kubectl taint node <master-node> node-role.kubernetes.io/control-plane:NoSchedule-
kubectl taint node <master-node> node-role.kubernetes.io/master:NoSchedule-

edited Apr 26, 2024 at 8:22

answered May 16, 2022 at 18:44

TheDHM

1611 silver badge5 bronze badges

3 Comments

Andy Brown Over a year ago

Upvoted but just to point out on my v1.24.0 installation I had both the master and control-plane taints and both had the NoSchedule effect and needed to be removed.

TheDHM Over a year ago

Thank you, yes I forgot to mention that both taints should be removed.

kayochin Jun 25 at 9:50

Tested on v1.33.2. Only the first line is needed. kubectl taint node <master-node> node-role.kubernetes.io/control-plane:NoSchedule-

P Ekambaram · Accepted Answer · 2019-03-27 11:00:05Z

13

Use the below command to untaint all masters

kubectl taint nodes --all node-role.kubernetes.io/master-

answered Mar 27, 2019 at 11:00

P Ekambaram

17.9k9 gold badges46 silver badges69 bronze badges

Comments

yasin lachini · Accepted Answer · 2019-04-26 16:13:23Z

11

First, get the name of the master

kubectl get nodes

NAME     STATUS   ROLES    AGE   VERSION
yasin   Ready    master   11d   v1.13.4

as we can see there is one node with the name of yasin and the role is master. If we want to use it as worker we should run

kubectl taint nodes yasin node-role.kubernetes.io/master-

edited Apr 26, 2019 at 16:13

answered Mar 27, 2019 at 12:40

yasin lachini

6,1649 gold badges37 silver badges59 bronze badges

Comments

illagrenan · Accepted Answer · 2018-10-12 08:21:04Z

7

For anyone using kops on AWS. I wanted to enable scheduling of Pods on master.

$ kubectl get nodes -owide was giving me this output:

NAME                                          STATUS
...
...
ip-1**-**-**-***.********.compute.internal    Ready                      node
ip-1**-**-**-***.********.master.internal     Ready,SchedulingDisabled   master
                                                    ^^^^^^^^^^^^^^^^^^
ip-1**-**-**-***.********.compute.internal    Ready                      node
...
...

And $ kubectl describe nodes ip-1**-**-**-***.********.master.internal:

...
...
Taints:             <none>
Unschedulable:      true
...                 ^^^^
...

Patching the master with this command:

$ kubectl patch node MASTER_NAME -p "{\"spec\":{\"unschedulable\":false}}"

worked for me and scheduling of Pods is now enabled.

Ref: https://github.com/kubernetes/kops/issues/639#issuecomment-287015882

answered Oct 12, 2018 at 8:21

illagrenan

6,6352 gold badges59 silver badges67 bronze badges

2 Comments

jhaavist Over a year ago

This does work on the OP's bare-metal, e.g., "the-hard-way" installations as well. This patch also persists over kubelet upgrades, so extra points for that! Thanks.

saumilsdk Over a year ago

Beautifully worked!

akaihola · Accepted Answer · 2018-08-18 09:44:40Z

4

I don't know why the master node shows up as NotReady; it shouldn't. Try executing kubectl describe node mymasternode to find out.

The SchedulingDisabled is because the master node is tainted with dedicated=master:NoSchedule

Execute this command against all your masters to remove the taint:

kubectl taint nodes mymasternode dedicated-

To understand why that works read up on taints and tolerations.

edited Aug 18, 2018 at 9:44

akaihola

26.9k7 gold badges64 silver badges76 bronze badges

answered Mar 31, 2017 at 20:52

Janos Lenart

27.4k5 gold badges80 silver badges77 bronze badges

Comments

johntellsall · Accepted Answer · 2020-11-06 18:57:07Z

2

Allow scheduling of pods on the master

kubectl taint node --all node-role.kubernetes.io/master:NoSchedule-

Verify the master isn't tainted

kubectl describe node | egrep -i taint

Taints: <none>

Schedule and run test pod in master

kubectl run -it  busybox-$RANDOM --image=busybox --restart=Never -- date

This answer is a combination of other SO answers, from Victor G, Aryak Sengupta, and others.

edited Nov 6, 2020 at 18:57

answered Nov 6, 2020 at 18:31

johntellsall

15.2k4 gold badges48 silver badges40 bronze badges

Comments

NicoKowe · Accepted Answer · 2019-11-22 12:30:11Z

0

Another way to list all taints in nodes and untaint the tainted one.

root@lab-a:~# kubectl get nodes -o json | jq ".items[]|{name:.metadata.name, taints:.spec.taints}"
{
  "name": "lab-a",
  "taints": null
}
{
  "name": "lab-b",
  "taints": [
    {
      "effect": "NoSchedule",
      "key": "node-role.kubernetes.io/master"
    }
  ]
}

lab-a does not have any taint. so we untaint lab-b:

root@lab-a:~# k taint node lab-b node-role.kubernetes.io/master:NoSchedule-
node/lab-b untainted

Install jq in ubuntu by: apt-get install jq

edited Nov 22, 2019 at 12:30

answered Nov 22, 2019 at 12:01

NicoKowe

3,5352 gold badges24 silver badges27 bronze badges

Comments

Baptiste Mille-Mathias · Accepted Answer · 2020-08-02 15:53:34Z

0

Since Openshift 4.x CoreOs is directly integrated on Kubernetes configuration (you can make all masters schedulable this way

# edit the field spec.mastersSchedulable to set a value true
$ oc patch schedulers.config.openshift.io cluster --type json \
     -p '[{"op": "add", "path": "/spec/mastersSchedulable", "value": true}]'

or using

oc edit schedulers.config.openshift.io cluster

and edit the field

spec:
    mastersSchedulable: true

edited Aug 2, 2020 at 15:53

answered Apr 22, 2020 at 16:47

Baptiste Mille-Mathias

2,1874 gold badges31 silver badges39 bronze badges

Comments

alay deliwala · Accepted Answer · 2021-05-27 14:49:37Z

0

The answer is

kubectl taint nodes --all node-role.kubernetes.io/master-

according to: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#control-plane-node-isolation

answered May 27, 2021 at 14:49

alay deliwala

1

Comments

Evan Q · Accepted Answer · 2024-01-16 07:34:40Z

0

kubectl taint nodes --all node-role.kubernetes.io/control-plane-
kubectl taint nodes --all node-role.kubernetes.io/master-

answered Jan 16, 2024 at 7:34

Evan Q

1481 silver badge6 bronze badges

1 Comment

Jeremy Caney Over a year ago

Thank you for your interest in contributing to the Stack Overflow community. This question already has quite a few answers—including one that has been extensively validated by the community. Are you certain your approach hasn’t been given previously? If so, it would be useful to explain how your approach is different, under what circumstances your approach might be preferred, and/or why you think the previous answers aren’t sufficient. Can you kindly edit your answer to offer an explanation?

Collectives™ on Stack Overflow

Allow scheduling of pods on Kubernetes master?

11 Answers 11

4 Comments

node-role.kubernetes.io/master

node-role.kubernetes.io/control-plane

3 Comments

Comments

Comments

2 Comments

Comments

Allow scheduling of pods on the master

Verify the master isn't tainted

Schedule and run test pod in master

Comments

Comments

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

11 Answers 11

4 Comments

node-role.kubernetes.io/master

node-role.kubernetes.io/control-plane

3 Comments

Comments

Comments

2 Comments

Comments

Allow scheduling of pods on the master

Verify the master isn't tainted

Schedule and run test pod in master

Comments

Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related