Docker Reference
Guide

Types of Virtualization

Evolution Timeline

Docker Architecture

Docker Architecture: Shows the three-layer stack — runtime (runc + containerd), daemon (dockerd), and the orchestrator (Swarm). The daemon exposes a REST API consumed by the client CLI.

Open Container Initiative (OCI)

A governance council responsible for standardizing low-level container infrastructure. Two key specs:

• image-spec — defines what a container image is
• runtime-spec — defines how to run a container

Install on Ubuntu (Linux)

# 1. Remove old versions
  $ sudo apt-get remove docker docker-engine docker.io containerd runc

  # 2. Install prerequisites
  $ sudo apt-get update
  $ sudo apt-get install ca-certificates curl gnupg lsb-release

  # 3. Add Docker GPG key
  $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
    sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

  # 4. Add Docker repository
  $ echo "deb [arch=$(dpkg --print-architecture) \
    signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
    https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable" | \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

  # 5. Install Docker Engine
  $ sudo apt-get update
  $ sudo apt-get install docker-ce docker-ce-cli containerd.io

  # 6. Verify
  $ sudo docker --version
  $ sudo docker info

Post-install: Avoid sudo

# Check current groups
  $ sudo getent group
  $ groups

  # Add user to docker group
  $ sudo usermod -a -G docker <username>
  $ groups   # verify

Ops Quick Commands

# Confirm installation
  $ docker version

  # Pull image
  $ docker image pull ubuntu:latest

  # Launch interactive container
  $ docker container run -it ubuntu:latest /bin/bash

  # Exit without terminating
  Ctrl-PQ

  # Attach to running container
  $ docker container exec -it <name> bash

  # Stop / Start / Delete
  $ docker container stop <name>
  $ docker container start <name>
  $ docker container rm <name>

Dev Quick Commands

# Clone repo
  $ git clone https://github.com/nigelpoulton/psweb.git

  # Inspect Dockerfile
  $ cat Dockerfile

  # Build image
  $ docker image build -t test:latest .

  # Run containerized app with port mapping
  $ docker container run -d \
    --name web1 \
    --publish 8080:8080 \
    test:latest

Docker Engine Overview: The top-level architecture showing dockerd, containerd, and runc in layers. Each handles progressively lower-level container operations.

Docker Engine Deep Dive: Detailed view of how the client CLI communicates with dockerd over a UNIX socket, dockerd calls containerd, and containerd spawns runc for each container creation event.

runc — The OCI Layer

• CLI wrapper, standalone container runtime tool
• Only creates containers — no long-running daemon
• Reference implementation of the OCI runtime-spec
• Also called "the OCI layer"

containerd

• Background daemon process: ps -elf | grep containerd
• Manages full container lifecycle: start | stop | pause | rm
• Handles image pulls, volumes, networks
• Initially by Docker, Inc. → donated to CNCF

Container Creation Flow

$ docker container run --name ctr1 -it alpine:latest sh

Container Creation Flow: The request travels from the CLI → dockerd → containerd → runc → OS kernel. After the container starts, runc exits and a shim process remains as the parent.

The Shim

A reduced version of containerd that stays running after runc exits:

• Keeps STDIN/STDOUT streams open even after daemon restart
• Reports container exit status back to the daemon
• Enables "daemonless containers" — you can upgrade dockerd without killing running containers

Process Summary

$ ps -elf | grep container
  # You'll see: dockerd, docker-containerd, docker-containerd-shim, docker-runc

Client ↔ Daemon Communication

TLS-Secured Client-Daemon Communication: When deploying Docker remotely, mutual TLS (mTLS) must be configured. Both the daemon and client present certificates signed by the same CA. Port 2376 is the standard TLS port.

Images vs Containers: An image is a static blueprint stored in a registry. When you run an image, Docker adds a thin writable layer on top to create a container — a live, running instance of that image.

Image Layers

A Docker image is a stack of loosely-connected read-only layers. Each layer is one or more files. Docker uses a storage driver to stack and merge layers into a single unified filesystem.

Image Layers: Each Dockerfile instruction that modifies the filesystem creates a new layer. Layers are identified by cryptographic hashes of their content. Unchanged layers are cached and reused.

Building Layers: Example of a Python app image built on Ubuntu 20.04. Layer 1 = Ubuntu base, Layer 2 = Python runtime added via RUN, Layer 3 = App source code via COPY. Each instruction = one layer.

Image Registry & Naming

Registry → Repository → Tag Hierarchy: Docker Hub is the default registry. Each registry contains repositories. Each repository stores one or more tagged versions of an image. Format: registry/repo:tag

Image Commands

# Pull image
  $ docker image pull alpine:latest
  $ docker image pull mongo:4.2.6
  $ docker image pull -a nigelpoulton/tu-demo  # all tags

  # List images
  $ docker image ls
  $ docker image ls --digests alpine

  # Inspect layers and metadata
  $ docker image inspect ubuntu:latest
  $ docker image history web:latest

  # Search Docker Hub
  $ docker search alpine --filter "is-official=true"

  # Delete images
  $ docker image rm <name>
  $ docker image rm $(docker image ls -q) -f  # delete all
  $ docker image prune       # remove unreferenced images
  $ docker image prune -a    # remove all dangling images

Image Digests (Content Addressable Storage)

Docker 1.10+ uses cryptographic content hashes (digests) to uniquely identify images. Use a digest to pull the exact image you expect — immune to tag mutations.

$ docker image pull alpine
  $ docker image ls --digests alpine
  # Output shows sha256:... digest per image

Multi-Architecture Images

Multi-Architecture Manifests: A single image tag (e.g., alpine:latest) can support multiple CPU architectures and OS types. When you pull, Docker inspects your platform and fetches the correct variant from the manifest list automatically. All official images have manifest lists.

# Inspect manifest list
  $ docker manifest inspect golang

  # Build for a specific platform
  $ docker buildx build --platform linux/arm/v7 -t myimage:arm-v7 .

  # Create custom manifest list
  $ docker manifest create

Containers vs VMs

VM Architecture: Each VM contains a full OS guest on top of a hypervisor. The hypervisor performs hardware virtualization — allocating virtual CPU, RAM, and disk to each VM. This causes significant overhead: memory, CPU cycles, licenses.

Container Architecture: Containers run directly on the host OS kernel, isolated via Linux namespaces and cgroups. No guest OS = near-zero overhead. All containers share one kernel, dramatically reducing resource usage.

Container Lifecycle

# Start interactive container with name
  $ docker container run --name myapp -it ubuntu:latest /bin/bash

  # Flags
  -i    → Keep STDIN open (interactive)
  -t    → Allocate pseudo-TTY
  -d    → Detached mode (background)
  --name → Set container name

  # Exit without killing (detach)
  Ctrl-PQ

  # Attach to running container
  $ docker container exec -it <name> bash

  # Stop (sends SIGTERM, then SIGKILL after 10s)
  $ docker container stop <name>

  # Start stopped container
  $ docker container start <name>

  # Delete (must be stopped)
  $ docker container rm <name>

  # Force delete running container
  $ docker container rm -f <name>

  # Delete ALL containers
  $ docker container rm $(docker container ls -aq) -f

Restart Policies

$ docker container run --name <name> --restart always <image> <cmd>

Real-World Examples

# Web server (nginx on port 80)
  $ docker container run -d --name webserver -p 80:80 nginx:latest

  # SQL Server (MSSQL 2019)
  $ docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=P@ssw0rd" \
    -e "MSSQL_PID=Express" \
    -p 1433:1433 \
    -d mcr.microsoft.com/mssql/server:2019-latest

Inspect a Container

$ docker container inspect <name>
  # Returns JSON with: Cmd, Entrypoint, Mounts, NetworkSettings, RestartCount, State...

Container Commands Reference

Containerizing = taking an application and configuring it to run as a container.

Containerization Workflow: The process flows from source code → Dockerfile → docker image build → image in local cache → optional push to registry → container run. Each step builds on the previous.

Containerization Steps

Dockerfile Instructions Reference

Example Dockerfile (Node.js App)

# Base layer — Alpine Linux
  FROM alpine

  # Metadata
  LABEL maintainer="nigelpoulton@hotmail.com"

  # Install runtime deps (creates new layer)
  RUN apk add --update nodejs nodejs-npm

  # Copy app source into image (creates new layer)
  COPY . /src

  # Set working directory
  WORKDIR /src

  # Install npm dependencies (creates new layer)
  RUN npm install

  # Document port
  EXPOSE 8080

  # Default app to run
  ENTRYPOINT ["node", "./app.js"]

Build, Tag, and Push

# Build image
  $ docker image build -t web:latest .

  # Verify
  $ docker image ls
  $ docker image inspect web:latest
  $ docker image history web:latest

  # Login to Docker Hub
  $ docker login

  # Tag (must include your account name)
  $ docker image tag web:latest yourusername/web:latest

  # Push
  $ docker image push yourusername/web:latest

  # Run from registry
  $ docker container run -d --name c1 -p 80:8080 yourusername/web:latest

Pushing Layers to Docker Hub: Only layers that differ from what's already in the registry are uploaded. Shared base layers (e.g., Alpine) are skipped if already present. This delta-push mechanism saves bandwidth.

Multi-Stage Builds

One Dockerfile with multiple FROM instructions. Each is a build stage. Final stage only copies what's needed — keeping production image small.

FROM node:latest AS storefront
  WORKDIR /usr/src/atsea/app/react-app
  COPY react-app .
  RUN npm install
  RUN npm run build

  FROM maven:latest AS appserver
  WORKDIR /usr/src/atsea
  COPY pom.xml .
  RUN mvn -B -f pom.xml -s /usr/share/maven/ref/settings-docker.xml dependency:resolve
  COPY . .
  RUN mvn -B -s /usr/share/maven/ref/settings-docker.xml package -DskipTests

  FROM java:8-jdk-alpine AS production
  RUN adduser -Dh /home/gordon gordon
  WORKDIR /static
  # Copy ONLY the built artifacts from previous stages
  COPY --from=storefront /usr/src/atsea/app/react-app/build/ .
  WORKDIR /app
  COPY --from=appserver /usr/src/atsea/target/AtSea-0.0.1-SNAPSHOT.jar .
  ENTRYPOINT ["java", "-jar", "/app/AtSea-0.0.1-SNAPSHOT.jar"]
  CMD ["--spring.profiles.active=postgres"]

Build Best Practices

Squashed Image: Left = standard layered image (each RUN creates a layer). Right = squashed image (all layers merged into one). Useful when intermediate layers contain secrets or sensitive build state that shouldn't be accessible.

Example Microservices App

docker-compose.yml Example

version: "3.8"
  services:
    web-fe:
      build: .                   # Build from Dockerfile in current dir
      command: python app.py
      ports:
        - target: 5000
          published: 5000
      networks:
        - counter-net
      volumes:
        - type: volume
          source: counter-vol
          target: /code

    redis:
      image: "redis:alpine"       # Pull from Docker Hub
      networks:
        counter-net:

  networks:
    counter-net:

  volumes:
    counter-vol:

Installing Compose on Linux

# Download binary
  $ sudo curl -L \
    https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 \
    -o /usr/local/bin/docker-compose

  # Make executable
  $ sudo chmod +x /usr/local/bin/docker-compose

  # Verify
  $ docker-compose --version

Deploying & Managing a Compose App

# Deploy (build images + start containers)
  $ docker-compose up

  # Deploy in background
  $ docker-compose up -d

  # Use custom compose file name
  $ docker-compose -f prod-config.yml up

  # List containers in app
  $ docker-compose ps

  # List processes in each service
  $ docker-compose top

  # Stop without deleting
  $ docker-compose stop

  # Start stopped app
  $ docker-compose restart

  # Delete stopped app (keeps volumes!)
  $ docker-compose rm

  # Stop AND delete (preserves volumes + images)
  $ docker-compose down

Compose Commands Summary

Docker Networking Architecture: Built on the Container Network Model (CNM). libnetwork is Docker's Go implementation of CNM, providing the control plane. Network drivers implement the data plane for each network type (bridge, overlay, macvlan, etc.).

Container Network Model (CNM) Building Blocks

CNM Building Blocks: Sandboxes contain the network stack of a container. Endpoints are virtual NICs that attach a sandbox to a network. Multiple sandboxes on the same network can communicate via their endpoints.

Network Drivers

Single-Host Bridge Networks

Single-Host Bridge Network: Docker creates a virtual bridge (docker0 on Linux) that connects containers on the same host. All containers on the bridge can communicate. The bridge maps to the external network via NAT for outbound traffic.

# List all networks
  $ docker network ls

  # Inspect a network
  $ docker network inspect bridge

  # Check underlying Linux bridge
  $ ip link show   # Look for docker0

  # Create custom bridge network
  $ docker network create -d bridge localnet

  # Connect container to custom network
  $ docker container run -d --name c1 --network localnet alpine sleep 1d

  # Create c2 on same network — can ping c1 by name
  $ docker container run -it --name c2 --network localnet alpine sh
  # ping c1   ← works on user-defined networks (built-in DNS)

Port Mapping

# Map host port 5000 → container port 80
  $ docker container run -d --name web \
    --network localnet \
    --publish 5000:80 \
    nginx

  # Verify port mapping
  $ docker port web

Port Mapping: Bridge networks are isolated from the outside world by default. Port mapping (-p host_port:container_port) creates a NAT rule so external traffic hitting the host port gets forwarded to the container port.

MACVLAN — Connecting to Existing Networks

The macvlan driver gives each container its own MAC address on the physical network — making them appear as physical devices. Requires NIC in promiscuous mode.

MACVLAN Network: Containers get their own MAC and IP on the existing VLAN. Useful for integrating with legacy applications that expect network-level identities. Containers appear as physical hosts to the rest of the network.

# Create MACVLAN network
  $ docker network create -d macvlan \
    --subnet=10.0.0.0/24 \
    --ip-range=10.0.0.0/25 \
    --gateway=10.0.0.1 \
    -o parent=eth0.100 \
    macvlan100

  # Run container on MACVLAN
  $ docker container run -d --name mactainer1 \
    --network macvlan100 \
    alpine sleep 1d

Service Discovery

Service Discovery: Docker's built-in DNS resolves container names to IPs within the same network. Each container has a local DNS resolver (127.0.0.11) that forwards queries to Docker's internal DNS server. Scope is network-level — containers on different networks cannot resolve each other by name.

# Custom DNS and search domain
  $ docker container run -it --name c1 \
    --dns=8.8.8.8 \
    --dns-search=example.com \
    alpine sh

Networking Commands

Non-Persistent Storage: Every container gets a thin read-write layer on top of its image. This is the "ephemeral" storage. It's created with the container and destroyed with it. Managed by the storage driver (e.g., overlay2).

Persistent Volume Storage: A volume is a separate object stored outside the container's writable layer. It's mounted into the container filesystem at a specified path. Deleting the container doesn't touch the volume or its data.

Creating and Managing Volumes

# Create a volume (using local driver by default)
  $ docker volume create myvol

  # List volumes
  $ docker volume ls

  # Inspect volume
  $ docker volume inspect myvol
  # Key fields: Mountpoint, Driver, Scope

  # Delete specific volume
  $ docker volume rm myvol

  # Delete ALL unused volumes (use with caution!)
  $ docker volume prune

Volume Inspect Output

[
      {
          "CreatedAt": "2020-05-02T17:44:34Z",
          "Driver": "local",
          "Labels": {},
          "Mountpoint": "/var/lib/docker/volumes/myvol/_data",  ← host path
          "Name": "myvol",
          "Options": {},
          "Scope": "local"   ← only this host can access it
      }
  ]

Using Volumes with Containers

# Mount volume into container
  $ docker container run -dit \
    --name voltainer \
    --mount source=bizvol,target=/vol \
    alpine

  # Write data to the volume
  $ docker container exec -it voltainer sh
    # echo "persistent data" > /vol/file1
    # cat /vol/file1
    # exit

  # Delete the container
  $ docker container rm voltainer -f

  # Volume and data still exists!
  $ docker volume ls
  $ ls -l /var/lib/docker/volumes/bizvol/_data/

  # Attach same volume to new container
  $ docker run --name hellcat \
    --mount source=bizvol,target=/vol \
    alpine sleep 1d

Volumes in Dockerfiles

# Declare a volume mount point in Dockerfile
  VOLUME /data
  # Note: host path cannot be specified in Dockerfile — only container mount point

Sharing Volumes Across Cluster Nodes

Shared Storage Across Nodes: The local driver stores data on the Docker host. For multi-host sharing (e.g., in Swarm), third-party volume plugins (available on Docker Hub) integrate with cloud storage, SAN, or NAS — enabling containers on different hosts to share the same volume data.

Volume Commands Reference

Images

Containers

Compose

Networking

Volumes

System