Guide to DevSecOps

Appearance

Setting Up a RabbitMQ Cluster with Docker ​

This guide demonstrates how to set up a RabbitMQ cluster using Docker with three individual RabbitMQ containers.

Prerequisites ​

  • Docker installed on your system
  • Docker Compose (optional but recommended)
  • Basic understanding of RabbitMQ concepts
  • Basic knowledge of Docker networking

Overview ​

We'll create a three-node RabbitMQ cluster with the following characteristics:

  • Each node will run in its own container
  • The nodes will communicate with each other over a shared network
  • The cluster will use Erlang Cookie authentication for node communication
  • Management UI will be available for monitoring the cluster

Setup ​

Step 1: Create a Docker Network ​

First, create a dedicated Docker network for your RabbitMQ cluster:

bash
docker network create rabbitmq-cluster

Step 2: Set Up Environment Variables ​

Create a .env file to store shared configuration:

bash
# RabbitMQ version
RABBITMQ_VERSION=3.12-management

# Common Erlang cookie for cluster authentication
RABBITMQ_ERLANG_COOKIE=DSHEVCXBBETJJVJWTOWT

# Default user credentials
RABBITMQ_DEFAULT_USER=admin
RABBITMQ_DEFAULT_PASS=admin

Step 3: Create Docker Compose File ​

Create a docker-compose.yml file with the following content:

yaml
version: "3.8"

services:
  rabbitmq1:
    image: rabbitmq:${RABBITMQ_VERSION}
    hostname: rabbitmq1
    container_name: rabbitmq1
    environment:
      - RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
      - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
    ports:
      - "5672:5672" # AMQP port
      - "15672:15672" # Management UI port
    volumes:
      - rabbitmq1_data:/var/lib/rabbitmq
    networks:
      - rabbitmq-cluster

  rabbitmq2:
    image: rabbitmq:${RABBITMQ_VERSION}
    hostname: rabbitmq2
    container_name: rabbitmq2
    environment:
      - RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
    ports:
      - "5673:5672" # AMQP port
      - "15673:15672" # Management UI port
    volumes:
      - rabbitmq2_data:/var/lib/rabbitmq
    networks:
      - rabbitmq-cluster
    depends_on:
      - rabbitmq1

  rabbitmq3:
    image: rabbitmq:${RABBITMQ_VERSION}
    hostname: rabbitmq3
    container_name: rabbitmq3
    environment:
      - RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
    ports:
      - "5674:5672" # AMQP port
      - "15674:15672" # Management UI port
    volumes:
      - rabbitmq3_data:/var/lib/rabbitmq
    networks:
      - rabbitmq-cluster
    depends_on:
      - rabbitmq1

networks:
  rabbitmq-cluster:
    external: true

volumes:
  rabbitmq1_data:
  rabbitmq2_data:
  rabbitmq3_data:

Step 4: Start the Containers ​

Start all containers:

bash
docker-compose up -d

Step 5: Form the Cluster ​

After the containers are up and running, join the second and third nodes to the first node to form a cluster:

  1. Wait for the containers to fully start (about 30-60 seconds)

  2. Join rabbitmq2 to rabbitmq1:

bash
docker exec -it rabbitmq2 bash -c "rabbitmqctl stop_app && rabbitmqctl reset && rabbitmqctl join_cluster rabbit@rabbitmq1 && rabbitmqctl start_app"
  1. Join rabbitmq3 to rabbitmq1:
bash
docker exec -it rabbitmq3 bash -c "rabbitmqctl stop_app && rabbitmqctl reset && rabbitmqctl join_cluster rabbit@rabbitmq1 && rabbitmqctl start_app"

Step 6: Verify the Cluster Status ​

Check the cluster status:

bash
docker exec -it rabbitmq1 rabbitmqctl cluster_status

You should see all three nodes listed in the cluster.

Alternative Setup: Using Standalone Docker Commands ​

If you prefer not to use Docker Compose, you can set up the cluster using standalone Docker commands:

Step 1: Create a Docker Network ​

bash
docker network create rabbitmq-cluster

Step 2: Start the First Node (rabbitmq1) ​

bash
docker run -d --name rabbitmq1 \
  --hostname rabbitmq1 \
  --network rabbitmq-cluster \
  -p 5672:5672 -p 15672:15672 \
  -e RABBITMQ_ERLANG_COOKIE=DSHEVCXBBETJJVJWTOWT \
  -e RABBITMQ_DEFAULT_USER=admin \
  -e RABBITMQ_DEFAULT_PASS=admin \
  rabbitmq:3.12-management

Step 3: Start the Second Node (rabbitmq2) ​

bash
docker run -d --name rabbitmq2 \
  --hostname rabbitmq2 \
  --network rabbitmq-cluster \
  -p 5673:5672 -p 15673:15672 \
  -e RABBITMQ_ERLANG_COOKIE=DSHEVCXBBETJJVJWTOWT \
  rabbitmq:3.12-management

Step 4: Start the Third Node (rabbitmq3) ​

bash
docker run -d --name rabbitmq3 \
  --hostname rabbitmq3 \
  --network rabbitmq-cluster \
  -p 5674:5672 -p 15674:15672 \
  -e RABBITMQ_ERLANG_COOKIE=DSHEVCXBBETJJVJWTOWT \
  rabbitmq:3.12-management

Step 5: Form the Cluster ​

Join the second and third nodes to the first node:

bash
docker exec -it rabbitmq2 bash -c "rabbitmqctl stop_app && rabbitmqctl reset && rabbitmqctl join_cluster rabbit@rabbitmq1 && rabbitmqctl start_app"

docker exec -it rabbitmq3 bash -c "rabbitmqctl stop_app && rabbitmqctl reset && rabbitmqctl join_cluster rabbit@rabbitmq1 && rabbitmqctl start_app"

Accessing the Management UI ​

You can access the RabbitMQ Management UI for each node at:

  • Node 1: http://localhost:15672
  • Node 2: http://localhost:15673
  • Node 3: http://localhost:15674

Use the credentials:

  • Username: admin
  • Password: admin

Cluster Management ​

Check Cluster Status ​

bash
docker exec -it rabbitmq1 rabbitmqctl cluster_status

Add a Queue with Mirroring ​

To create a queue with mirroring across all nodes (high availability):

bash
docker exec -it rabbitmq1 rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'

Manually Restart a Node ​

bash
docker restart rabbitmq2

Remove a Node from the Cluster ​

If you need to remove the third node from the cluster:

bash
# On the node being removed
docker exec -it rabbitmq3 bash -c "rabbitmqctl stop_app"

# On any remaining node
docker exec -it rabbitmq1 bash -c "rabbitmqctl forget_cluster_node rabbit@rabbitmq3"

Testing the Cluster ​

You can test the cluster by creating a queue on one node and consuming from another node:

  1. Create a test message publisher:
bash
# Create a simple test queue and publish a message from rabbitmq1
docker exec -it rabbitmq1 bash -c "rabbitmqadmin declare queue name=test_queue durable=true && rabbitmqadmin publish exchange=amq.default routing_key=test_queue payload='test message'"
  1. Consume the message from another node:
bash
# Consume the message from rabbitmq2
docker exec -it rabbitmq2 bash -c "rabbitmqadmin get queue=test_queue"

Troubleshooting ​

Nodes Won't Join the Cluster ​

  • Ensure all containers are using the same Erlang cookie
  • Check network connectivity between containers
  • Verify hostnames are correctly set
  • Ensure the first node is fully started before joining others

If you encounter cookie mismatch errors, you may need to manually set the cookie:

bash
docker exec -it rabbitmq1 bash -c "echo 'DSHEVCXBBETJJVJWTOWT' > /var/lib/rabbitmq/.erlang.cookie && chmod 400 /var/lib/rabbitmq/.erlang.cookie"

Repeat for each node and restart the containers.

Node Communication Issues ​

Ensure that RabbitMQ nodes can resolve each other's hostnames:

bash
docker exec -it rabbitmq1 bash -c "ping rabbitmq2"
docker exec -it rabbitmq1 bash -c "ping rabbitmq3"

Cleanup ​

To stop and remove all containers, networks, and volumes:

bash
docker-compose down -v

Or if using standalone commands:

bash
docker stop rabbitmq1 rabbitmq2 rabbitmq3
docker rm rabbitmq1 rabbitmq2 rabbitmq3
docker network rm rabbitmq-cluster

Additional Configuration ​

For more advanced configuration, you can create custom configuration files and mount them into the containers:

yaml
volumes:
  - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
  - ./enabled_plugins:/etc/rabbitmq/enabled_plugins

Example enabled_plugins file to ensure management plugin is enabled:

[rabbitmq_management,rabbitmq_prometheus,rabbitmq_shovel,rabbitmq_shovel_management].

Resources ​