I will be taking multiple k8s tutorials and the gist of it I will summarize and write here.
"Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach." - CNCF
Cloud-native technologies are open-source projects designed to deploy and scale applications automatically by cloud computing services.
source: ByteByteGo.com
source: ByteByteGo.com
There are two main components of k8s:
- Control Plane or Master node is responsible for managing the state of the cluster.
- There are four core components of the control plane:
- API Server
- It exposes a RESTful API to communicate between the workers and Admin.
- it also acts as a gatekeeper for authentication.
- etcd
- It is a distributed key-value store. It stores a cluster's persistence state.
- Scheduler
- It schedules the pods.
- however, the actual work is done by the kubelet inside every node.
- Control Manager
- It is responsible for running controllers to manage the state of the cluster.
- A few examples like ReplicationController, DeploymentController.
- It detects the state changes of a pod.
- It makes changes to the scheduler.
- API Server
- There are four core components of the control plane:
- Worker nodes run the containerized application workloads.
- Pods are the smallest deployable units in the k8s.
- Pods can run 1 or more containers and provides shared storage and networking for them.
- Pods are manged by the control plane.
source: ByteByteGo.com
Further detailed components of the Worker nodes:
- kubelet: communicates betweent the control pane and the pod. How many container to run in a pod etc.
- container runtime: Pull the container images from registry, managing the containers
- kube-proxy: It makes sure that the traffic is distributed evenly across the pods.
Pros
- Highly scalable.
- Self Healing
- Automatic Rollbacks
- Horizontal Scaling
- It can run on private, public or hybrid cloud environment.
Cons
- Complex to setup and operate
- Cost
- YAGNI for smaller organizations.
# Kubernetes Tutorial for Beginners
by Nana
!300
- Each pod gets its own IP address. However, pods are ephermmal like containers.
- Service is a static IP address which can be assigned to any pod. Service is also a load balancer.
- Service remains avaialble even if a pod dies.
- There are two kinds of services called External facing services and Internal facing services.
- To make the external facing services from IP address PORT to a good looking domain name we need Ingress.
- There is a configMap for the configurations needed in between the pods.
- Secret: Stores the secrets in base64 encoded format.
- Deployment stores the blueprint of every application.
- Statefulset is maintains the state between two pods which uses the same shared resources like an external DB.
Minikube
!500
- Minikube is a one node virtualbox. It is a cluster which runs both master and worker processes.
- Deployment --> replicaSet --> Pod --> Container
- All the CRUD operations gets executed at the deployment level. Rest all are handled by k8s.
Commands
Description | Command |
---|---|
Create deployment | kubectl create deployment [name] |
Edit deployment | kubectl edit deployment [name] |
Delete deployment | kubectl delete deployment [name] |
Status of different K8s components | kubectl get nodes I pods I services I replicasets | deployments |
Log to console | kubectl logs [pod name] |
Get Interactive Terminal | kubectl exec -it [pod name] -- bin/bash |
Apply a configuration file | kubectl apply -f [file name] |
Delete with configuration file | kubectl delete -f [file name] |
Get info about pod | kubectl describe pod [pod namel |
YAML Configuration file
Example of a deployment config:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.16
ports:
- containerPort: 80
Example of a service config:
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 8080
targetPort: 8080
Each config file has three parts:
- Metadata
- Specification
- Inside the specification there will be a template, where it will have its own metadata and specification. This if for the pod.
- Status (Auto generated and managed by k8s)
- k8s gets status data from etcd
Demo
Mongo Express Service --> Mongo Express Pod --> MongoDB service --> MongoDB pod
k8s Namespaces
- Organise resources in namespaces.
- It's like a virtual cluster inside a cluster.
- There are four namespaces by default.
- kube-system -> Do not change anythig here. This is only for system use.
- kube-public
- publicely accessible data
- A configmap, which contains cluster information
- kube-node-lease
- heartbeats of nodes
- each node has associated lease object in namespace
- determines the availability of a node
- default
- configMaps can not be shared from another namespace, it needs to be created in each namespace. Same for secrets too.
- However, services of one NS can be shared with another NS.
- Volume and Nodes can not be isolated. It needs to be shared across the NS in a cluster.
Resources which can be shared across namespaces in a cluster
> kubectl api-resources --namespaced=false
NAME SHORTNAMES APIVERSION NAMESPACED KIND
componentstatuses cs v1 false ComponentStatus
namespaces ns v1 false Namespace
nodes no v1 false Node
persistentvolumes pv v1 false PersistentVolume
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
apiservices apiregistration.k8s.io/v1 false APIService
selfsubjectreviews authentication.k8s.io/v1 false SelfSubjectReview
tokenreviews authentication.k8s.io/v1 false TokenReview
selfsubjectaccessreviews authorization.k8s.io/v1 false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io/v1 false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io/v1 false SubjectAccessReview
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
flowschemas flowcontrol.apiserver.k8s.io/v1 false FlowSchema
prioritylevelconfigurations flowcontrol.apiserver.k8s.io/v1 false PriorityLevelConfiguration
etcdsnapshotfiles k3s.cattle.io/v1 false ETCDSnapshotFile
ingressclasses networking.k8s.io/v1 false IngressClass
runtimeclasses node.k8s.io/v1 false RuntimeClass
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
priorityclasses pc scheduling.k8s.io/v1 false PriorityClass
csidrivers storage.k8s.io/v1 false CSIDriver
csinodes storage.k8s.io/v1 false CSINode
storageclasses sc storage.k8s.io/v1 false StorageClass
volumeattachments storage.k8s.io/v1 false VolumeAttachment
Resources which can NOT be shared across namespaces in a cluster
> kubectl api-resources --namespaced=true
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
persistentvolumeclaims pvc v1 true PersistentVolumeClaim
pods po v1 true Pod
podtemplates v1 true PodTemplate
replicationcontrollers rc v1 true ReplicationController
resourcequotas quota v1 true ResourceQuota
secrets v1 true Secret
serviceaccounts sa v1 true ServiceAccount
services svc v1 true Service
controllerrevisions apps/v1 true ControllerRevision
daemonsets ds apps/v1 true DaemonSet
deployments deploy apps/v1 true Deployment
replicasets rs apps/v1 true ReplicaSet
statefulsets sts apps/v1 true StatefulSet
localsubjectaccessreviews authorization.k8s.io/v1 true LocalSubjectAccessReview
horizontalpodautoscalers hpa autoscaling/v2 true HorizontalPodAutoscaler
cronjobs cj batch/v1 true CronJob
jobs batch/v1 true Job
leases coordination.k8s.io/v1 true Lease
endpointslices discovery.k8s.io/v1 true EndpointSlice
events ev events.k8s.io/v1 true Event
helmchartconfigs helm.cattle.io/v1 true HelmChartConfig
helmcharts helm.cattle.io/v1 true HelmChart
addons k3s.cattle.io/v1 true Addon
ingresses ing networking.k8s.io/v1 true Ingress
networkpolicies netpol networking.k8s.io/v1 true NetworkPolicy
poddisruptionbudgets pdb policy/v1 true PodDisruptionBudget
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
roles rbac.authorization.k8s.io/v1 true Role
csistoragecapacities storage.k8s.io/v1 true CSIStorageCapacity
Best Practices
- Whenever you start you should group your applications through namespaces. For e.g. all montoroing depoyments/nodes in monitoring namespace, similarly dedicated namespaces for elastic stack, Database, Ingress etc.
- If you have multiple teams, better to use dedicated namespaces.
- Staging and development environment can share the same cluster. Namespace can be used to differentiate between the two environments.
- For B-G deployment.
- Authorization, Isolation across teams.
- Resources limitation across teams using resources in the same cluster.
- In summary use namespaces to:
- Structure your components
- Avoid conflicts between teams
- Share services between different environments
- Access and Resource Limits on Namespaces Level
- Provide the namespace details in the yaml files under the
metadata
section.
Ingress
!600
apiVersion: networking.k8s.io/vlbetal
kind: Ingress
metadata:
name: myapp-ingress
spec:
rules:
- host: myapp.com
http:
paths:
- backend:
serviceName: myapp-internal-service
servicePort: 8080
- Ingress Controller
- evaluates all the rules
- manages redirections
- entrypoint to cluster
- There are many 3rd-party implementations like k8s nginx ingress controller
- multiple path or sub-domains to different sevices can be rerouted in the Ingress level.
TLS certificate
Helm Charts
- It is a package manager like homebrew of k8s world.
- Helm charts are a bundle of YAML files with pre-defined templated configurations for general application stack like Elastic.
- Also helm charts can be used to create template files which can be used as a blueprint to create similar types of microservices.
- This helps to replace a yaml file for each pod with a single yaml file.
- This is useful in CI/CD.
- Directory structure:
- Top level mychart folder @ name of chart
- Chart. yaml @ meta info about chart
- values.yaml @ values for the template files
- charts folder @ chart dependencies
- templates folder @ the actual template files
- Helm chart also does release management.
- In Helm version 2 there were two components:
- Client (helm CLI)
- Server (Tiller) -> It has power to do CRUD operations inside k8s cluster. This caused security issue. Therefore, in Helm3 it is removed.
k8s Volumes
There are three components:
- Persistent Volume (PV)
- Persistent Volume Claim (PVC)
- Storage Class (SC)
Storage requirements:
- Storage that doesn't depend on the pod lifecycle.
- Storage must be available on all nodes.
- Storage needs to survive even if cluster crashes.
Persistent Volume
- a cluster resource
- created via YAML file
- kind: PersistentVolume
- spec: e.g. how much storage?
- Needs actual physical storage, like local disk or NFS server
- PVs are accessible to all the namespaces in the clusters.
- There are two kinds of volumes:
- Local
- Local volume types violate 2. and 3. requirement for data persistence i.e.
- ❌ Being tied to 1 specific node
- ❌ surviving cluster crashes
- Therefore for DB persistence almost always use remote storage.
- Local volume types violate 2. and 3. requirement for data persistence i.e.
- Remote
- Local
PVC
- To access the volumes in the apps developers need to claim the PVs via PVCs.
- These are also created with yaml config. Example:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-name
spec:
storageClassName: manual
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
Now this PVC needs to be added in the pods configuration
apiVersion: vl
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: pvc-name
- Volume is directory with some data.
- These volumes are accisible in containers in a pod.
- How made available, backed by which storage medium
- defined by specific volume types
Storage Class
- For admins to allocate PVs to address the PVCs is time consuming and complex, to address this issue we got SC. Example:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: storage-class-name
provisioner: kubernetes.io/aws-ebs
parameters:
type: iol
iopsPerGB: "10"
fsType: ext4
- It creates PVs dynamically in the background via "provisioner" attribute.
- each storage backend has own provisioner
- internal provisioner - "kubernetes.io"
- external provisioner
- configure parameters for storage we want to request for PV
- abstracts underlying storage provider and parameters for that storage
- This SC needs to be mentioned in the PVC to get PV.
Roles
- There are two kinds of roles:
- k8s admins (SysAdmins or DevOps Engineers)
- K8s Admin sets up and maintains the clusters.
- They make sure that there are enough resouces in the clusters.
- k8s users (Developers of DevOps Teams)
- They deploy applications in the clusters either directly or via CI pipeline.
- k8s admins (SysAdmins or DevOps Engineers)
Statefulset
Stateful | Stateless |
---|---|
Stores state/data | Do not store state |
Databases or any app those store data |
Frontend Application |
Each request is different | Each request is the same |
deployed using statefulset | deployed using deployments |
created from same specification, but not interchangeable! |
identical and interchangable |
can't be created/deleted at same time | created in random order with random hashes |
can't be randomly addressed replica Pods are not identical - Pod Identity |
one Service that load balances to any Pod |
The names are predictable and persisted | random hashes get attached to the pod names |
- k8s is not easy to manage the stateful app as we need to manage the synchronization and all.
k8s Services
- What is a Kubernetes Service and when we need it?
- The pods are ephimoral i.e. the IP address of a pod may change.
- The service is used as to interact between external and internal services.
- Different Service types explained
- ClusterIP Services (default)
- Headless Services
- NodePort Services
- LoadBalancer Services
- Differences between them and when to use which one
Source
- https://www.linkedin.com/learning/learning-kubernetes-16086900
- Kubernetes Explained in 6 Minutes | k8s Architecture - YouTube
- Kubernetes Tutorial for Beginners [FULL COURSE in 4 Hours] - YouTube