Kubernetes nslookup on AWS

Hi

I had the following cluster of kubernetes 1.16.
3 Nodes, 1 master and 2 other workers.
Tried to do nslookup from the master to the 'kubernetes'

using the following : 

'kubectl exec -it busybox -- nslookup kubernetes'
(ip doesn't matter)

got this
Server:    10.96.0.10
Address 1: 10.96.0.10

nslookup: can't resolve 'kubernetes'
command terminated with exit code 1

What was it ? 

i was missing UDP rule in the security group of the instances. Allow traffic. Solved it. 

Regards,

Dor

Kubernetes finding the leader

Hi

Following is a good way of finding the leader, usually also the master node.

kubectl get endpoints kube-scheduler -n kube-system -o yaml

Here is my example, taken from a k8s cluster i have created on AWS with 3 nodes, 1 master and 2 workers.

apiVersion: v1

kind: Endpoints

metadata:

  annotations:

    control-plane.alpha.kubernetes.io/leader: '{"holderIdentity":"ip-172-31-15-105_805a03a9-0065-11ea-9ec9-060d741267c8","leaseDurationSeconds":15,"acquireTime":"2019-11-06T07:18:04Z","renewTime":"2019-11-06T07:38:49Z","leaderTransitions":2}'

  creationTimestamp: 2019-11-04T14:36:03Z

  name: kube-scheduler

  namespace: kube-system

  resourceVersion: "70343"

  selfLink: /api/v1/namespaces/kube-system/endpoints/kube-scheduler

  uid: 6db973fb-ff10-11e9-ab52-060d741267c8

Regards,

Dor

CassandraRoleManager skipped default role setup: some nodes were not ready

Hi

I had this issue with my Cassandra 3.11.3 cluster.
'CassandraRoleManager skipped default role setup: some nodes were not ready'

Actually i didn't raised one of my nodes. The status was:

Datacenter: datacenter1

=======================

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

--  Address        Load       Tokens       Owns (effective)  Host ID                               Rack

UN  X.X.15.141  279.12 KiB  256          49.1%             65b80558-219d-478b-ad24-083a79d78348  rack1

UN  X.X.15.140  235.39 KiB  256          49.2%             0efb432f-0789-455f-op24-29c6e2cce9ac  rack1

UN  X.X.15.142  287.49 KiB  256          49.6%             3558e3f0-717d-783UI-a22c-6891a3bd427f  rack1

DN  X.X.15.144  ?          256          52.2%             8e8cdbac-9088-3421-963d-4857a78087d3  rack1

To solve it, i used the nodetool with 'removenode' method. Reference


Removing this node solved the problem for me.

Regards,
Dor