Changing Container's runtime from Docker to Podman in CEPH

Managing high-performance storage requires staying ahead of the stack. We’ve successfully transitioned our Ceph cluster runtimes to Podman, leaning into a daemonless, more secure container architecture.

Check out the migration breakdown and the scripts we used to ensure zero downtime.

# ==================================================
# STEP 1: PREPARATION (Run on all nodes)
# ==================================================
# Update repositories and install Podman
sudo apt update && sudo apt install podman -y

# ==================================================
# STEP 2: MAINTENANCE (Run on the Admin Node)
# ==================================================
# Replace <hostname> with the node you are currently migrating
ceph orch host maintenance enter <hostname>

# ===================================================
# STEP 3: SERVICE HALT (Run on the Target Node)
# ===================================================
# Stop Docker to prevent runtime conflicts
sudo systemctl stop docker docker.socket

# ===================================================
# STEP 4: MIGRATION SCRIPT (Run on the Target Node)
# ===================================================
# Create the fix script
cat << 'EOF' > ceph_runtime_fix.sh
#!/bin/bash
# Emergency fix for Ceph Docker -> Podman transition
# Target Cluster ID: 1760c18a-b5ee-11f0-bfab-3fc18abc7370

CEPH_PATH="/var/lib/ceph/1760c18a-b5ee-11f0-bfab-3fc18abc7370"
SYSTEMD_UNIT="/etc/systemd/system/ceph-1760c18a-b5ee-11f0-bfab-3fc18abc7370@.service"

echo "[*] 1. Updating unit.run files to use Podman..."
find "$CEPH_PATH" -name "unit.run" -not -path "*/removed/*" -exec sed -i 's|/usr/bin/docker|/usr/bin/podman|g' {} +
find "$CEPH_PATH" -name "unit.run" -not -path "*/removed/*" -exec sed -i 's|docker |podman |g' {} +

echo "[*] 2. Updating Systemd unit dependencies..."
if [ -f "$SYSTEMD_UNIT" ]; then
    # Disable Docker dependencies
    sed -i 's/^After=docker.service/#After=docker.service/' "$SYSTEMD_UNIT"
    sed -i 's/^Wants=docker.service/#Wants=docker.service/' "$SYSTEMD_UNIT"
        # Add Podman dependency if missing
    if ! grep -q "After=podman.service" "$SYSTEMD_UNIT"; then
        sed -i '/\[Unit\]/a After=podman.service' "$SYSTEMD_UNIT"
    fi
fi

echo "[*] 3. Reloading Systemd and restarting services..."
systemctl daemon-reload

# Restart Core services (MON/MGR) first
systemctl restart "ceph-1760c18a-b5ee-11f0-bfab-3fc18abc7370@mon.openceph-20-02.service"
systemctl restart "ceph-1760c18a-b5ee-11f0-bfab-3fc18abc7370@mgr.openceph-20-02.jwfdls.service"

sleep 3

# Restart everything else (OSDs, RGW, etc.)
systemctl restart "ceph-1760c18a-b5ee-11f0-bfab-3fc18abc7370.target"

echo "[*] 4. Verification..."
podman ps
echo "==== ✅ Migration Complete for this node ===="
EOF

# Make executable and run
chmod +x ceph_runtime_fix.sh
sudo ./ceph_runtime_fix.sh

# =================================================================
# STEP 5: REJOIN & VERIFY (Run on the Admin Node)
# =================================================================
# Exit maintenance mode
ceph orch host maintenance exit <hostname>

# Check overall cluster health
ceph -s