👨‍💻 Kubernetes Advanced: Production-Ready Deployments

Kubernetes Advanced: Production-Ready Deployments

Welcome to the final part of our Kubernetes tutorial series! This advanced guide dives deep into production-grade features and enterprise-level best practices. We’ll explore how to build robust, scalable, and maintainable Kubernetes deployments that can handle real-world workloads.

What We’ll Cover

1. StatefulSets and Persistent Storage - Managing stateful applications and data persistence
2. Service Mesh with Istio - Advanced networking and service management
3. Advanced Monitoring and Logging - Complete observability stack
4. High Availability Patterns - Ensuring 24/7 uptime
5. Production Tools and Utilities - Essential tools for Kubernetes management

Prerequisites

• Completed our Intermediate Kubernetes tutorial
• Familiarity with Kubernetes resources
• Understanding of microservices architecture
• Basic knowledge of storage concepts

StatefulSets and Persistent Storage

Understanding Storage in Kubernetes

Storage management in Kubernetes involves several key concepts:

1. Volumes: Temporary or persistent storage attached to pods
2. PersistentVolumes (PV): Cluster-wide storage resources
3. PersistentVolumeClaims (PVC): Storage requests by applications
4. StorageClasses: Dynamic provisioning of storage

Storage Classes

StorageClasses define different types of storage with varying performance characteristics:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: fast-ssd provisioner: kubernetes.io/aws-ebs # Cloud provider specific parameters: type: gp3 iopsPerGB: "10" encrypted: "true" reclaimPolicy: Retain volumeBindingMode: WaitForFirstConsumer

Persistent Volumes

Define available storage resources:

apiVersion: v1 kind: PersistentVolume metadata: name: pv-storage spec: capacity: storage: 10Gi volumeMode: Filesystem accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: fast-ssd hostPath: # Example for local testing path: /data/storage

Persistent Volume Claims

Request storage for applications:

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: database-storage spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi storageClassName: fast-ssd

StatefulSets with Persistent Storage

StatefulSets are perfect for applications that need:

• Stable, unique network identifiers
• Stable, persistent storage
• Ordered deployment and scaling
• Ordered automated rolling updates

Here’s a comprehensive example of a StatefulSet with persistent storage:

apiVersion: v1 kind: Service metadata: name: postgresql-headless spec: clusterIP: None selector: app: postgresql ports: - port: 5432 --- apiVersion: apps/v1 kind: StatefulSet metadata: name: postgresql spec: serviceName: postgresql-headless replicas: 3 selector: matchLabels: app: postgresql template: metadata: labels: app: postgresql spec: containers: - name: postgresql image: postgres:14 env: - name: POSTGRES_PASSWORD valueFrom: secretKeyRef: name: postgres-secrets key: password ports: - containerPort: 5432 name: postgresql volumeMounts: - name: data mountPath: /var/lib/postgresql/data - name: config mountPath: /etc/postgresql/conf.d readinessProbe: exec: command: - pg_isready initialDelaySeconds: 5 periodSeconds: 10 resources: requests: memory: "1Gi" cpu: "500m" limits: memory: "2Gi" cpu: "1000m" volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: fast-ssd resources: requests: storage: 10Gi

Best Practices for Storage Management

1. Capacity Planning:
   • Monitor storage usage trends
   • Set up alerts for capacity thresholds
   • Use volume expansion features when available
2. Backup and Recovery:
   apiVersion: velero.io/v1 kind: Backup metadata: name: daily-backup spec: includedNamespaces: - "*" storageLocation: default volumeSnapshotLocations: - default schedule: "0 1 * * *" retention: keepDaily: 7 keepWeekly: 4
3. Storage Performance:
   • Use the right storage class for your workload
   • Monitor I/O metrics
   • Consider using local volumes for high-performance needs
4. Security: ```yaml apiVersion: v1 kind: PersistentVolume metadata: name: secure-storage spec: capacity: storage: 5Gi accessModes:
   • ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: encrypted-storage csi: driver: ebs.csi.aws.com volumeHandle: vol-xyz volumeAttributes: encrypted: “true” ```

Service Mesh with Istio

Installing Istio

istioctl install --set profile=demo -y

Enabling Istio Injection

kubectl label namespace default istio-injection=enabled

Virtual Service Configuration

apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews-route spec: hosts: - reviews http: - match: - headers: end-user: exact: jason route: - destination: host: reviews subset: v2 - route: - destination: host: reviews subset: v1

Advanced Monitoring

Prometheus Setup

apiVersion: monitoring.coreos.com/v1 kind: Prometheus metadata: name: prometheus spec: serviceAccountName: prometheus serviceMonitorSelector: matchLabels: team: frontend resources: requests: memory: 400Mi enableAdminAPI: false

Grafana Dashboard Configuration

apiVersion: integreatly.org/v1alpha1 kind: GrafanaDashboard metadata: name: golang-dashboard spec: json: > { "dashboard": { "id": null, "title": "Golang Dashboard", ... } }

High Availability Patterns

Pod Disruption Budget

apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: app-pdb spec: minAvailable: 2 selector: matchLabels: app: critical-app

Anti-Affinity Rules

apiVersion: apps/v1 kind: Deployment metadata: name: high-availability-app spec: replicas: 3 template: spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - high-availability-app topologyKey: "kubernetes.io/hostname"

Essential Kubernetes Tools and Utilities

The Kubernetes ecosystem is rich with tools that can help you manage, monitor, and optimize your clusters. Here’s a curated list of essential tools for different aspects of Kubernetes management, inspired by the comprehensive collection at KubeTools:

1. Development Tools

IDE Plugins

• Kubernetes for VS Code: Syntax highlighting and cluster management
• Lens: The Kubernetes IDE
• K9s: Terminal-based UI for managing clusters

Local Development

# Using Telepresence for local development telepresence connect telepresence intercept my-service --port 8080:80

2. Cluster Management

Cluster Creation and Management

• kind: Local clusters using Docker
• k3s: Lightweight production-grade K8s
• kubeadm: Official cluster bootstrapping
• kops: Production grade K8s installation on AWS

Policy Management

# Example OPA/Gatekeeper policy apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sRequiredLabels metadata: name: ns-require-labels spec: match: kinds: - apiGroups: [""] kinds: ["Namespace"] parameters: labels: ["owner", "environment"]

3. Security Tools

Scanner and Analyzers

• Trivy: Container vulnerability scanner
• Falco: Runtime security monitoring
• Snyk: Dependency and container security

Security Management

# Example NetworkPolicy apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: strict-policy spec: podSelector: matchLabels: app: secure-app policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: role: frontend ports: - protocol: TCP port: 80

4. Monitoring and Debugging

Monitoring Stacks

• Prometheus + Grafana: Metrics and visualization
• Elastic Stack: Logging and analysis
• Jaeger: Distributed tracing

Debugging Tools

# Using kubectl-debug kubectl debug node/my-node -it --image=ubuntu # Using stern for log tailing stern my-app --tail 50 # Using ktunnel for local debugging ktunnel expose deployment my-deployment 8080:80

5. CI/CD Tools

Pipeline Tools

• ArgoCD: GitOps continuous delivery
• Flux: GitOps for cluster management
• Jenkins X: Cloud native CI/CD

Deployment Tools

• Helm: Package manager for Kubernetes
• Kustomize: Template-free configuration
• Skaffold: Local development workflow

6. Storage and Backup

Storage Management

• Rook: Cloud native storage orchestrator
• Velero: Backup and migrate resources
• OpenEBS: Container attached storage

Backup Solutions

# Example Velero Schedule apiVersion: velero.io/v1 kind: Schedule metadata: name: daily-backup spec: schedule: "@daily" template: includedNamespaces: - "*" includedResources: - "*" storageLocation: default ttl: 720h0m0s

7. Network Tools

Service Mesh

• Istio: Comprehensive service mesh
• Linkerd: Lightweight service mesh
• Consul: Service networking platform

Ingress Controllers

• Nginx Ingress: Popular ingress controller
• Traefik: Cloud native edge router
• Contour: High-performance ingress

8. Development Workflows

Local Development

# Using Tilt for local development tilt up # Using Skaffold for continuous development skaffold dev

Testing Tools

• kube-monkey: Chaos testing
• k6: Load testing
• Testkube: Testing framework

Getting Started with Tools

1. Tool Selection:
   • Start with essential tools
   • Add tools as needs grow
   • Consider team expertise
2. Installation Methods: ```bash

   Using kubectl krew

   kubectl krew install neat kubectl krew install ctx kubectl krew install ns

Using Helm

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install prometheus prometheus-community/prometheus

Using operators

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.0/cert-manager.yaml ```

1. Integration Tips:
   • Use GitOps workflows
   • Automate tool installation
   • Maintain documentation

Visit KubeTools for a complete, up-to-date list of Kubernetes tools and utilities.

Production Checklist

1. Security:
   • Network Policies
   • RBAC configuration
   • Pod Security Policies
2. Monitoring:
   • Prometheus metrics
   • Grafana dashboards
   • Alert management
3. Backup:
   • etcd backup
   • PV snapshots
   • Disaster recovery plan
4. Scaling:
   • HPA configuration
   • VPA setup
   • Cluster autoscaling

What’s Next?

Consider exploring:

• GitOps workflows with Flux/ArgoCD
• Custom Resource Definitions (CRDs)
• Operator pattern implementation
• Cloud-native security practices

Additional Resources

• Istio Documentation
• Prometheus Operator
• Kubernetes Production Best Practices