Upgrading and Rebuilding KinD with Istio and Cert-Manager on Ubuntu Server

This guide provides step-by-step instructions for upgrading an existing Kubernetes in Docker (KinD) cluster on Ubuntu Server, including reinstallation of Istio and cert-manager.

Table of Contents

• Prerequisites
• Backing Up Existing Resources
• Removing the Old KinD Cluster
• Upgrading KinD
• Creating a New KinD Cluster
• Installing Istio
• Installing Cert-Manager
• Restoring Workloads
• Verification and Testing
• Troubleshooting

Prerequisites

• Ubuntu Server (18.04+)
• Docker installed and configured
• kubectl CLI tool
• Administrator access (sudo privileges)

Backing Up Existing Resources

Before making any changes, back up your existing cluster resources:

# Create a directory for backups
mkdir -p ~/kind-backup/$(date +%F)
cd ~/kind-backup/$(date +%F)

# List all namespaces and save them
kubectl get namespaces -o yaml > namespaces.yaml

# Export all resources per namespace
for ns in $(kubectl get ns -o jsonpath='{.items[*].metadata.name}'); do
  mkdir -p $ns
  for resource in $(kubectl api-resources --namespaced=true --verbs=list -o name); do
    kubectl get -n $ns $resource -o yaml > $ns/$resource.yaml 2>/dev/null
  done
done

# Backup any persistent volumes if applicable
kubectl get pv -o yaml > persistent-volumes.yaml

# Backup KinD configuration if available
cp ~/.kube/kind-config-* ./

Removing the Old KinD Cluster

# List existing KinD clusters
kind get clusters

# Delete the existing cluster (replace 'kind' with your cluster name if different)
kind delete cluster --name kind

# Verify removal
kind get clusters

Upgrading KinD

# Check current KinD version (if installed)
kind --version

# Install or upgrade KinD
curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/

# Verify the new version
kind --version

Creating a New KinD Cluster

Create a configuration file for your new KinD cluster:

# Create a configuration file
cat > kind-config.yaml << EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
- role: worker
- role: worker
EOF

# Create a new cluster with the configuration
kind create cluster --config kind-config.yaml --name kind

Installing Istio

# Download Istio
curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.19.0 sh -

# Move to the Istio directory
cd istio-1.19.0

# Add istioctl to PATH
export PATH=$PWD/bin:$PATH

# Install Istio with demo profile
istioctl install --set profile=demo -y

# Enable automatic sidecar injection for default namespace
kubectl label namespace default istio-injection=enabled

# Install Istio addons (Kiali, Prometheus, Grafana, etc.)
kubectl apply -f samples/addons

Installing Cert-Manager

# Install cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml

# Wait for cert-manager to be ready
kubectl wait --for=condition=Ready pods -l app.kubernetes.io/instance=cert-manager -n cert-manager --timeout=180s

# Create a ClusterIssuer for Let's Encrypt (optional)
cat > letsencrypt-issuer.yaml << EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: your-email@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: istio
EOF

kubectl apply -f letsencrypt-issuer.yaml

Restoring Workloads

Restore your applications and configurations from the backup:

# Create namespaces first (excluding system namespaces)
cd ~/kind-backup/$(ls -1tr ~/kind-backup | tail -1)
grep -v "kubernetes\|kube-system\|kube-public\|kube-node-lease" namespaces.yaml | kubectl apply -f -

# Restore non-system resources per namespace
for ns in $(ls -d */ | cut -f1 -d'/'); do
  # Skip system namespaces
  if [[ "$ns" != "kube-"* && "$ns" != "kubernetes"* ]]; then
    echo "Restoring namespace: $ns"
    kubectl create ns $ns 2>/dev/null || true

    # Apply configurations but filter out status fields
    for config in $ns/*.yaml; do
      if [ -f "$config" ]; then
        cat $config | grep -v -e "resourceVersion" -e "uid" -e "status:" | kubectl apply -f - || true
      fi
    done
  fi
done

Verification and Testing

# Verify all components are running
kubectl get pods --all-namespaces

# Check Istio installation
istioctl verify-install

# Test Istio ingress gateway
kubectl get svc istio-ingressgateway -n istio-system

# Verify cert-manager installation
kubectl get pods -n cert-manager

Troubleshooting

Common Issues and Solutions

1. Pod Startup Issues

# Check pod status
kubectl describe pod <pod-name> -n <namespace>

# Check logs
kubectl logs <pod-name> -n <namespace>

2. Istio Connectivity Problems

# Check Envoy proxy configuration
istioctl proxy-config all <pod-name>.<namespace>

# Analyze Istio configuration
istioctl analyze

3. Cert-Manager Certificate Issues

# Check certificate status
kubectl get certificates -A

# Check certificate request details
kubectl describe certificaterequest -A

4. Node Resource Constraints

# Check node resource usage
kubectl describe nodes

This guide should help you successfully upgrade and rebuild your KinD cluster with Istio and cert-manager on Ubuntu Server. Adapt the specific versions and configurations as needed for your environment.