KinD - Kubernetes in Docker

Introduction

KinD (Kubernetes in Docker) is a tool for running local Kubernetes clusters using Docker containers as nodes. It's designed primarily for testing Kubernetes itself, but is perfect for local development of Kubernetes applications and operators.

Prerequisites

• Docker installed and running
• kubectl installed (for interacting with the cluster)
• A terminal or command prompt

Installation

Linux

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

macOS

Using Homebrew:

brew install kind

Or manually:

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-darwin-amd64
chmod +x ./kind
mv ./kind /usr/local/bin/kind

Windows

Using Chocolatey:

choco install kind

Using PowerShell:

curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.20.0/kind-windows-amd64
Move-Item .\kind-windows-amd64.exe c:\some-dir-in-your-PATH\kind.exe

Basic Usage

Creating a Cluster

Create a default cluster:

kind create cluster

Create a named cluster:

kind create cluster --name dev-cluster

Interacting with Clusters

List all KinD clusters:

kind get clusters

kubectl cluster-info --context kind-kind

Set kubectl context to KinD cluster:

kubectl cluster-info --context kind-dev-cluster

Deleting a Cluster

kind delete cluster --name dev-cluster

Advanced Configuration

Multi-Node Clusters

Create a config file named multi-node.yaml:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
  - role: worker
  - role: worker

Create the cluster using the config:

kind create cluster --config multi-node.yaml --name multi-node

Production-Like Cluster Configuration

For a more comprehensive development setup that mimics a production environment, you can use a configuration with:

• A single control-plane (master) node
• Multiple worker nodes
• Registry integration
• External port mappings

Create a file named kind-cluster-config.yaml:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: dev-cluster

# Configure registry access
containerdConfigPatches:
  - |-
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
      endpoint = ["http://local-registry:5000"]

# Node configurations
nodes:
  # Control plane (master) node with external port mappings
  - role: control-plane
    kubeadmConfigPatches:
      - |
        kind: InitConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "ingress-ready=true"
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
      - containerPort: 443
        hostPort: 443
        protocol: TCP

  # Worker nodes
  - role: worker
  - role: worker

Create the cluster using:

kind create cluster --config kind-cluster-config.yaml

Or use the rebuild script:

./rebuild-kind.sh dev-cluster kind-cluster-config.yaml

This configuration:

1. Creates a single control-plane node and two worker nodes
2. Configures the control-plane node for ingress controllers
3. Maps ports 80 and 443 from the host to the control-plane node
4. Sets up container registry integration

Before creating this cluster, start a local registry:

docker run -d --name local-registry -p 5000:5000 registry:2
docker network connect kind local-registry

Container Registry Integration

For local development, you'll often need to push images to a registry accessible by the KinD cluster:

1. Create a local registry:

docker run -d --name local-registry -p 5000:5000 registry:2

2. Create a cluster with registry access:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
  - |-
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
      endpoint = ["http://local-registry:5000"]
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
      - containerPort: 443
        hostPort: 443
        protocol: TCP

3. Connect registry to KinD network:

docker network connect kind local-registry

Loading Images into Cluster

Build and load image directly:

docker build -t my-app:latest .
kind load docker-image my-app:latest --name dev-cluster

Troubleshooting

Common Issues

1. Insufficient Resources: KinD requires adequate CPU and memory resources. Adjust Docker's resource limits if needed.
2. Port Conflicts: Ensure the ports mapped are not already in use.
3. Failed Node Creation: Check Docker logs with docker logs <node-container-id>.

Logs and Debugging

Get cluster logs:

kind export logs --name dev-cluster

Best Practices for Development

1. Resource Limitations: Set resource limits in your cluster configuration to avoid overwhelming your host.
2. Persistent Volumes: Use the hostPath provisioner for development storage.
3. Port Forwarding: Use kubectl port-forward to access services inside the cluster.
4. Namespace Organization: Use namespaces to organize your development components.
5. Script Common Tasks: Create shell scripts for common workflows like cluster creation, image loading, etc.

Example Development Workflow

1. Create development cluster with appropriate configuration
2. Build your application container
3. Load container into KinD cluster
4. Deploy application with kubectl
5. Test and debug
6. Repeat steps 2-5 as needed

Cleaning Up and Rebuilding Clusters

Sometimes you need to start fresh with a clean cluster. Here's how to properly clean up and rebuild a KinD cluster:

Basic Cleanup

1. Delete the KinD cluster:

kind delete cluster --name dev-cluster

2. Verify it's gone:

kind get clusters

Thorough Cleanup

If you want to ensure all resources are removed:

1. Remove all KinD clusters:

kind get clusters | xargs -n1 kind delete cluster --name

2. Remove orphaned Docker containers (if any):

docker ps -a | grep 'kind-' | awk '{print $1}' | xargs -r docker rm -f

3. Remove orphaned Docker networks (optional):

docker network ls | grep 'kind' | awk '{print $1}' | xargs -r docker network rm

4. Remove Docker volumes (optional, BE CAREFUL):

docker volume ls | grep 'kind' | awk '{print $2}' | xargs -r docker volume rm

Rebuilding with the Same Configuration

To rebuild a cluster with the same configuration:

1. Save your configuration to a file (if not already done):

# Example config file: kind-config.yaml
cat > kind-config.yaml << EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
EOF

2. Delete the existing cluster:

kind delete cluster --name dev-cluster

3. Recreate with the same config:

kind create cluster --name dev-cluster --config kind-config.yaml

Automated Rebuild Script

Create a rebuild script for convenience that leverages the cleanup script:

# Show help with examples
./rebuild-kind.sh --help

# Rebuild default cluster with default config
./rebuild-kind.sh

# Rebuild specific cluster with specific config
./rebuild-kind.sh prod-cluster prod-config.yaml

# Perform thorough cleanup (containers, networks, etc.) before rebuilding
./rebuild-kind.sh dev-cluster config.yaml --all

The script is available at /home/kelly/ws/guide/kubernetes/kind-scripts/rebuild-kind.sh and automatically uses the cleanup script before creating a new cluster.

Advanced Cleanup Script

For more sophisticated cleanup operations, you can use a dedicated cleanup script that handles:

• Selective cleanup of clusters, containers, networks, and volumes
• Safety confirmations for destructive operations
• Detailed information about what resources will be removed

The script is available at /home/kelly/ws/guide/kubernetes/kind-scripts/kind-cleanup.sh and can be used as follows:

# Show help
./kind-cleanup.sh --help

# Delete all KinD clusters (default behavior)
./kind-cleanup.sh

# Delete a specific KinD cluster
./kind-cleanup.sh --name my-cluster

# Delete everything related to KinD (clusters, containers, networks, volumes)
./kind-cleanup.sh --all

# Delete only orphaned KinD containers
./kind-cleanup.sh --containers

# Force deletion without confirmation prompts
./kind-cleanup.sh --all --force

Adding SSL to Local KinD Clusters

SSL certificates enable secure HTTPS connections to your applications running in KinD. Here's how to set up SSL for local development.

Installing cert-manager

First, install cert-manager to automate certificate management:

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io
helm repo update

# Install cert-manager with CRDs
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.crds.yaml
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.13.1





## Conclusion

Happy KinD development!