diff --git a/docker/README.md b/docker/README.md index dfbcdf0..972c7e1 100644 --- a/docker/README.md +++ b/docker/README.md @@ -2,18 +2,18 @@ gitea: none include_toc: true --- + # Docker -Installation: See separate page [Docker Installation](docker-install.md) - - + - Installation: \ + See separate page [Docker Installation](docker-install.md) + - Docker Compose: \ + See separate page [Docker Compose](docker-compose.md) ## Summary Docker simplifies the development and deployment of applications by providing a lightweight, portable, and consistent containerized environment. It bridges the gap between development and production, enabling developers to focus on building applications without worrying about environment-specific issues. The applications run consistently across different computing environments, whether on a developer's laptop, a test server, or in production. - - ## Advantages of Docker - __Portability:__ Containers ensure applications behave the same regardless of the environment (development, testing, production). @@ -21,8 +21,6 @@ Docker simplifies the development and deployment of applications by providing a - __Scalability:__ Docker enables rapid scaling of applications by spinning up multiple container instances as needed. - __Isolation:__ Each container runs independently, preventing conflicts between applications. - - ## Key Concepts of Docker ### Containers: @@ -77,6 +75,7 @@ Tools like Docker Compose and Kubernetes are used to manage and scale multiple c 3. Run the image as a __container__ using __docker run__. 4. Use __Docker Compose__ to manage __multiple containers__ for a complete application (e.g., web server + database). +--- # Docker Image @@ -121,7 +120,7 @@ Um dem Image einen Namen und einen Tag zu geben (1.0 ist im folgenden Beispiel d docker build -t node-app:1.0 . ``` -#### Layers +#### Layers The commands `RUN`, `COPY` and `ADD` may possibly generate a new layer, if the files have changed. Otherwise, the __cached__ layers will be used. @@ -198,21 +197,6 @@ RUN npm install --production CMD [ "node", "app.js" ] ``` -## Run docker image - -Image, which was either pulled from a registry or was built on the system, can be run with: - -``` -docker run -d -p 3000:80 -e FOO='bar' -FOO2='bar2' --name myApp node-app:1.0 -``` - - - `-d` Optional. For detached mode (run in background) - - `-p 3000:80` Optional. Traffic on port 3000, and port 80 inside the container. - - `-e` Optional. Environment Parameter, that will be passed on to the container - - `--name` Optional. Name for container. - - `--restart` Optional. Restart policy. `no`, `on-failure:5`, `always` - - `node-app:1.0` image/tag to be started. - ## Show all docker images ``` @@ -261,156 +245,210 @@ docker push # Docker Container -Ein Container ist ein Image, welches gerade ausgeführt wird. Wenn ein Image mit `docker run nginx` ausgeführt wird, spricht man von einem Container. Es ist vergleichbar mit einem Prozess. Container wird auf Basis eines Ausgangs-Images gestartet. +A Docker container is a Docker image that is currently being executed. If an image is executed with `docker run`, it is referred to as a container. It is comparable to a process. +## Run docker image + +A Docker image, which was __either pulled from a registry or was built on the system__, can be run with: + +``` +docker run --init -d -p 3000:80 -v /home/roger/meine-dateien:/home/daten -e FOO='bar' -FOO2='bar2' --network testnetwork --network-alias myapp --name myApp node-app:1.0 +``` + + - `--init` Optional, but highly recommended. Tells Docker to run a tiny init process (based on Tini) as PID 1 inside the container before your command. + - `-d` Optional. For detached mode (run in background) + - `-p 3000:80` Optional. Traffic on port 3000, and port 80 inside the container + - `-v` Optional. Bind mount + - `-e` Optional. Environment Parameter, that will be passed on to the container + - `--network testnetwork` Optional. Use own network instead of bridge. + - `--network-alias myapp` Optional. Name of the network instance. Used if you want to access a database in container from the app container, instead of using the IP of the container. + - `--name` Optional. Name for container. + - `--restart` Optional. Restart policy. `no`, `on-failure:5`, `always` + - `node-app:1.0` image/tag to be started. ## Share data -### 1. Möglichkeit: Verzeichnisse mappen (Host-Verzeichnis im Container mappen) +which are stored and handled internally by docker (c.f. docker config to choose where they are actually stored). + +### Bind mounts + +Directories on the host system are mapped into the container with `-v`. ``` docker run -v : + +# Example: +docker run -it -v ~/Desktop/LokaleDaten:/home/daten --name Datentest ubuntu:20.04 /bin/bash ``` -Konkret: +### Named volumes + +A volume is a folder somewhere on the host (depending on the OS, can be configured in docker config). It doesn't really matter where it is. Create it: ``` -$ docker run -it -v ~/Desktop/LokaleDaten:/home/daten --name Datentest ubuntu:20.04 /bin/bash +# create volume "myfiles" +docker volume create myfiles + +# check if it exists: +docker volume ls ``` -### 2. Möglichkeit: Daten-Container verwenden (zwischen Container und Container) +You can then connect it to the container with: -Datencontainer mit Volume `/data/db` erstellen +``` +docker run -v : + +# Example: +docker run -v myfiles:/home/roger/daten --name Datentest ubuntu:20.04 /bin/bash +``` + +Delete volume: + +``` +docker volume rm myfiles + +# delete all unused volumes, that are not being used by a running container +docker volume prune +``` + +### Data container + +You can create a container that only serves as a data box. The following creates such a container with the volume `/data/db`. It must not run, therefore we use just `docker create`. ``` docker create -v /data/db --name datenbox busybox true ``` -Neuen Container für Anwendung erstellen +__BusyBox__ is a small, monolithic program that bundles a variety of standard Unix tools in a single executable file. It is often referred to as the "Swiss army knife" for embedded or lightweight Linux systems. BusyBox also contains the command true, which always returns with exit code 0 and is used in scripts or as a placeholder command (as with your docker create ... busybox true). + +Now you can use the data container in any other container with `--volumes-from`, like this: ``` -$ docker run -it --volumes-from datenbox ubuntu:20.04 /bin/bash -$ cd /data/db -$ touch datei.txt -$ exit +docker run -it --volumes-from datenbox alpine:latest /bin/sh +cd /data/db +touch hello.txt # the file is now in the data container +exit ``` -Die Datei ist jetzt im Datencontainer unter `/data/db`. Der Datencontainer muss gar nicht gestartet werden um ihn zu verwenden. +Check if the file is there: - - -## Container verlinken - -### Ports verbinden - -Beispiel Image martinblaha/testapp starten, localhost Port 4000, docker Port 1337 +1. With a "helper" container ``` -docker run -it -d --name myApp -p 4000:1337 martinblaha/testapp sh -c 'cd /var/www/test/myapp && sails lift' +docker run --rm --volumes-from datenbox busybox ls /data/db ``` -### Environment Variablen - -Variante 1 +2. Copy the file to the local host ``` -$ docker run -it --name TestEnv -e MY_VAR="hello world" ubunut:20.04 /bin/bash -$ env +docker cp datenbox:/data/db/hello.txt ./hello.txt ``` -Variante 2 +--- -So wird der Wert von MY_VAR vom hostsystem automatisch mitgegeben: +## Environment Variables + +### Set env variable and pass it to the container ``` -$ docker run -it --name TestEnv -e MY_VAR ubunut:20.04 /bin/bash -$ env +docker run -it --name TestEnv -e MY_VAR="hello world" ubunut:20.04 /bin/bash +env ``` -Variante 3 - -Env Variable Liste übergeben +### Use existing env variable from the host and pass it to the container ``` -$ docker run -it --name TestEnv -env-file ./env.list ubunut:20.04 /bin/bash -$ env +docker run -it --name TestEnv -e MY_VAR ubunut:20.04 /bin/bash +env ``` -### Link zwischen 2 Container - -Sicherer Tunnel zwischen beiden Container. Ports müssen nicht geöffnet werden. Dabei geschieht folgendes: - - - Umgebungsvariablen des Quellcontainers werden im Zielcontainer veröffentlicht. - - Einträge in ''/etc/hosts'' des Zielcontainers gesetzt, die zum Quellcontainer führen. - -Beispiel: - -Ubuntu 20.04 Image erstellen und mit mongodb:mongo Container verlinken. +### Pass a list of env variables to the container ``` -mongodb: NAMES, mongo: IMAGE (sieht man, wenn man docker ps macht) -$ docker run -it -P --link mongodb:mongo ubuntu:20.04 /bin/bash +docker run -it --name TestEnv -env-file ./env.list ubunut:20.04 /bin/bash +env ``` +## Docker Networking +The mostly used type of docker network is __bridge__, and has also the name "bridge". Every running container uses the bridge network by default. This also means that all container share the same network, if not otherwise defined. -## Container Commands +Show current networks: -### Anzeigen aller Container +``` +docker network ls +NETWORK ID NAME DRIVER SCOPE +0dd07c40b577 bridge bridge local +e33da267b703 host host local +54c49b8d1c44 none null local +``` + +### Create individual bridge network + +``` +docker network create testnetwork +``` + +### Start a container using this network + +All ports of every container in the network are accessible, so `-p` does not have any effect. + +``` +docker run --network testnetwork ... +``` + +### Attach a running container to a network + +``` +docker network connect testnetwork +``` + +### Detach + +``` +docker network disconnect testnetwork +``` + +## Container commands + +### List all containers ``` docker ps -a docker container ls -a # alternativ -``` -Nur laufende: - -``` +# only running containers: docker ps -docker container ls # alternativ +docker container ls # alternative ``` -### Ausgabe eines Containers anzeigen +### Show output of a container ``` docker logs ``` -Docker Logs generell: - -``` -journalctl -xu docker.service -``` - -### Container starten - -``` -docker run -d -p 3000:80 -e --name -``` - -### Container stoppen / neu starten +### Stop container / restart ``` docker stop docker restart ``` -### Container (schneller) stoppen und löschen +### Stop container (faster) / delete container ``` docker kill docker rm ``` -### Befehl in Docker Container ausführen - -Z.B. MySQL, wobei `wordpress-baumfreunde_db_1` der Container-Name ist, den man mit `docker ps` herausfindet. +### Run command inside container ``` -sudo docker exec -it wordpress-baumfreunde_db_1 mysql -uroot -p +sudo docker exec -it container-name mysql -uroot -p ``` -### Import DB in docker container +### Import DB in container ``` sudo docker exec -i wp_db mysql -h 172.17.0.1 -P 3306 --protocol=tcp -uroot -p wp_baum < /var/www/wordpress-from-docker/wp_baum.sql @@ -425,12 +463,9 @@ docker exec -it wordpress-baumfreunde_db_1 mysqldump --add-drop-table -uroot -pX ### Bash in container ``` -sudo docker exec –it /bin/bash +sudo docker exec –it /bin/bash # or /bin/sh for Alpine -# Alpine -sudo docker exec –it /bin/sh - -# als root +# as root docker exec -u root -it /bin/bash ``` @@ -452,185 +487,20 @@ sudo docker cp "c30c199ec89c:/home/actions/conf /home/rogrut docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name ``` -### Remove docker container +--- -`docker stop Test_run` und `docker rm Test_run`. +# General Docker commands +## Cleanup - -# Docker compose - - - __Purpose:__ Defines and manages multi-container Docker applications. - - __Usage:__ Orchestrates multiple services (containers), networks, and volumes for an application. - - __Key Features:__ - - Describes how to run one or more containers together. - - Can manage container networking and persistent storage. - - Useful for applications with multiple services (e.g., a web app + database). - - __Output:__ A running application consisting of multiple containers. - -`docker-compose.yml` is the file which includes all nescessary information. It can include multiple services like web (built from a Dockerfile) and db (pulled from Docker Hub). - -## docker-compose vs docker compose - - - If you're using a modern Docker installation (20.10+), use `docker compose`. - - If you're working on a system with an older Docker version or have compatibility concerns, use `docker-compose`. - -| Feature | docker compose | docker-compose | -| --- | --- | --- | -| Integration | Integrated into Docker CLI | Standalone binary | -| Compose Specification | Supports Compose V2 | Supports older Compose V1 | -| Performance | Faster and more efficient | Slower compared to Compose V2 | -| Availability | Docker 20.10+ and Docker Desktop | Any Docker version, standalone | -| Future-proof | Actively developed and maintained | Legacy, less maintained | - -check your docker compose version (examples): - -``` -docker compose version - Docker Compose version v2.32.1 - # is installed together with docker v27.4.1 -docker-compose --version - docker-compose version 1.25.0, build unknown - # separate installation for old version. -``` - -## Image Location - -Example: - -``` -services: - nproxy-app: - image: 'jc21/nginx-proxy-manager:latest' - build: . -``` - -### Registry - - -In docker compose, use `image` to use an existing image from the registry. - -``` -services: - redis: - image: redis:latest -``` - -### Local Image - -Don't use `image`, but `build`. - -``` -services: - my-local-app: - build: . -``` - -`build: .` tells Docker Compose to look for a `Dockerfile` in the same directory as the `docker-compose.yml`. - -If the Dockerfile is in a subdirectory, specify the context and Dockerfile path: - -``` -services: - my-local-app: - build: - context: ./my-app - dockerfile: Dockerfile.dev -``` - - - `context:` Specifies the directory containing the application code. - - `dockerfile:` Specifies the name of the Dockerfile if it’s not the default Dockerfile. - -You can mix local builds and images from a registry in the same docker-compose.yml file: - -``` -services: - my-local-app: - build: . - ports: - - "8080:8080" - - redis: - image: redis:latest - - webapp: - build: . - image: registry.cs.zi.uzh.ch/zi-adb-dba/export-digicert-report:latest -``` - -## Docker Volumes - -There are three volume types: - -1. Docker volumes which are stored and handled internally by docker (c.f. docker config to choose where they are actually stored). - -``` -version: '3.9' -services: - caddy: - image: caddy:2.6.2 - volumes: - - caddy_data:/data -volumes: - caddy_data -``` - -2. Bind mounts which are direct access to the host file system from a container - -``` -version: '3.9' -services: - caddy: - image: caddy:2.6.2 - volumes: - - /opt/docuteam/ssl/certifcate.pem:/cert.pem:ro - -3. Bind mounts of remote share which are defined through docker volumes -version: '3.9' -services: - fedora: - image: docker.cloudsmith.io/docuteam/docker/fcrepo:6.2.0 - volumes: - - fedora_data:/fcrepo_home -volumes: - fedora_data: - driver_opts: - type: cifs - device: //remote-hostname.com/path/to/share/fedora - o: addr=remote-hostname.com,username=user,password=mysuperpassword,nodev,noexec,nosuid,vers=2.1,uid=1000,gid=1000 -``` - -### Show docker volumes - -``` -docker volume ls -``` - -## Commands - -### Start - -Ins Verzeichnis gehen wo docker-compose.yml liegt, und dann `docker-compose start -d`. Mit `-d` wird es im Hintergrund ausgeführt. - -### Stop - - - The `docker-compose stop` command will stop your containers, but it won’t remove them. - - The `docker-compose down` command will stop your containers, but it also removes the stopped containers as well as any networks that were created. - - You can take down 1 step further and add the `-v` flag to remove all volumes too. This is great for doing a full blown reset on your environment by running `docker-compose down -v`. - -### Events - -``` -docker compose events -``` - - -# Allgemeine Commands - -## Aufräumen - -Löscht alle Images, Volumes, Netzwerke, die nicht mit mind. 1 Container verbunden sind, sowie den Build Cache. +Deletes all images, volumes, networks, that are not linked to at least 1 running container, as well as the build cache. ``` docker system prune --all --volumes -``` \ No newline at end of file +``` + +## Docker logs + +``` +journalctl -xu docker.service +``` diff --git a/docker/docker-compose.md b/docker/docker-compose.md new file mode 100644 index 0000000..a5fdfca --- /dev/null +++ b/docker/docker-compose.md @@ -0,0 +1,375 @@ +--- +gitea: none +include_toc: true +--- +# Docker compose + + - __Purpose:__ Defines and manages multi-container Docker applications. + - __Usage:__ Orchestrates multiple services (containers), networks, and volumes for an application. + - __Key Features:__ + - Describes how to run one or more containers together. + - Can manage container networking and persistent storage. + - Useful for applications with multiple services (e.g., a web app + database). + - __Output:__ A running application consisting of multiple containers. + +`docker-compose.yml` is the file which includes all nescessary information. It can include multiple services like web (built from a Dockerfile) and db (pulled from Docker Hub). + +## docker-compose vs. docker compose + + - If you're using a modern Docker installation (20.10+), use `docker compose`. + - If you're working on a system with an older Docker version or have compatibility concerns, use `docker-compose`. + +| Feature | docker compose | docker-compose | +| --- | --- | --- | +| Integration | Integrated into Docker CLI | Standalone binary | +| Compose Specification | Supports Compose V2 | Supports older Compose V1 | +| Performance | Faster and more efficient | Slower compared to Compose V2 | +| Availability | Docker 20.10+ and Docker Desktop | Any Docker version, standalone | +| Future-proof | Actively developed and maintained | Legacy, less maintained | + +check your docker compose version (examples): + +``` +docker compose version + Docker Compose version v2.32.1 + # is installed together with docker v27.4.1 +docker-compose --version + docker-compose version 1.25.0, build unknown + # separate installation for old version. +``` + +## docker-compose.yaml version + +Use `version: "3.9"` which is the latest version. Or, with Docker Compose v2 (the newer, built-in CLI plugin), you no longer need a version: line. If you remove it, Compose will simply default to the current spec. + +## Image location + +### Registry + +In docker compose, use `image` to use an existing image from the registry. + +``` +services: + redis: + init: true + image: redis:latest +``` + +### Local Image + +If the image needs to be built, use `build`. + +``` +services: + my-local-app: + init: true + build: . +``` + +`build: .` tells Docker Compose to look for a `Dockerfile` in the same directory as the `docker-compose.yml`. + +If the Dockerfile is in a subdirectory, specify the context and Dockerfile path: + +``` +services: + my-local-app: + init: true + build: + context: ./my-app + dockerfile: Dockerfile.dev +``` + + - `context:` Specifies the directory containing the application code. + - `dockerfile:` Specifies the name of the Dockerfile if it’s not the default Dockerfile. + +You can mix local builds and images from a registry in the same docker-compose.yml file: + +``` +services: + myapp: + init: true + build: ./my-app + ports: + - 8080:8080 + + redis: + init: true + image: redis:latest + + webapp: + init: true + build: . + image: registry.cs.zi.uzh.ch/zi-adb-dba/export-digicert-report:latest +``` + +## Docker Volumes + +Bind mounts: + +``` +version: '3.9' +services: + caddy: + image: caddy:2.6.2 + volumes: + - /opt/roger/ssl/certifcate.pem:/cert.pem:ro + +# OR: Bind mounts of remote share which are defined through docker volumes +version: '3.9' +services: + fedora: + init: true + image: docker.cloudsmith.io/docuteam/docker/fcrepo:6.2.0 + volumes: + - fedora_data:/fcrepo_home +volumes: + fedora_data: + driver_opts: + type: cifs + device: //remote-hostname.com/path/to/share/fedora + o: addr=remote-hostname.com,username=user,password=mysuperpassword,nodev,noexec,nosuid,vers=2.1,uid=1000,gid=1000 +``` + +Docker volumes: + +``` +version: '3.9' +services: + caddy: + init: true + image: caddy:2.6.2 + volumes: + - caddy_data:/data +volumes: + caddy_data +``` + +### Read-Only + +Add `:ro` at the end of the line. + +``` + volumes: + - /opt/roger/ssl/certifcate.pem:/cert.pem:ro +``` + +## Dependencies + +If there is a special order of starting the containers, you can use `depends_on`, like this: + +``` +services: + myapp: + init: true + build: ./my-app + ports: + - 8080:8080 + + redis: + init: true + image: redis:latest + depends_on: + - myapp +``` + +You can also add `condition`s, like `service_started`, `service_healthy` and `service_completed_successfully`: + +``` +services: + myapp: + init: true + build: ./my-app + ports: + - 8080:8080 + + redis: + init: true + image: redis:latest + depends_on: + myapp: + condition: service_started +``` + +In the following example, Compose will first start the database service, execute the pg_isready command specified in the healthcheck.test property, start the migration service if the value returned by the test command is 0, and finally start the api service if the exit status of the migration service's container is also 0. + +``` +services: + api: + init: true + build: . + depends_on: + database: + condition: service_healthy + migration: + condition: service_completed_successfully + database: + init: true + image: postgres + healthcheck: + test: ["CMD-SHELL", "pg_isready"] + migration: + init: true + image: migration-tool +``` + +## Variables + +### Variables for docker-compose.yaml + +Use `${VARIABLE_NAME}`, like this: + +``` +services: + ui: + init: true + image: nginx:${NGINX_VERSION}-alpine +``` + +You can pass the variable when starting docker compose like this: + +``` +NINGX_VERSION=1.21.5 docker compose up -d +``` + +You can also use an environment file. If no variable is set on calling `docker compose up`, then it will use the value in the `.env` file. + +``` +touch .env + +# then add the variables: +NINGX_VERSION=1.21.4 +``` + +If the file has a different name or is located in a different path, you can specify the location when starting docker-compose. + +``` +docker compose --env-file /path/to/environmentfile up -d +``` + +### Variables for containers + +If a variable is already set in the shell, like `PORT` in this example, it will be automatically passed to docker compose, like this: + +``` +services: + ui: + init: true + image: nginx:1.21.5-alpine + environment: + - PORT +``` + +To specify the variable in docker compose: + +``` +services: + ui: + init: true + image: nginx:1.21.5-alpine + environment: + - PORT=81 +``` + +You can also specify a file: + +``` +services: + ui: + init: true + image: nginx:1.21.5-alpine + environment: + - PORT=81 + env_file: + - ./path/to/envfile +``` + +## Profiles + +Start only containers within a certain profile. Note that services WITHOUT a profile will always start! + +``` +services: + myapp: + init: true + build: ./my-app + ports: + - 8080:8080 + profiles: + - blabla + + ui: + init: true + image: nginx:1.21.5-alpine + profiles: + - profilename +``` + +start like this, and it will start only the "ui" service. + +``` +docker compose --profile profilename up -d +``` + +## Networks + +All containers in a docker compose setup can communicate with eachother. The services, like `myapp` or `ui` can be used as hostnames. This will make linking the containers easier. + +If the network must be specialized, e.g. not all containers have connection to other containers, you need to build your own networks. + +``` +services: + myapp: + init: true + build: ./my-app + networks: + - test + + ui: + init: true + image: nginx:1.21.5-alpine + profiles: + - profilename + networks: + - test + +networks: + test: + driver: bridge + attachable: true # default; set to false, so that other containers can not be added to the network in the future. +``` + +## Restart policy + +``` + ui: + init: true + image: nginx:1.21.5-alpine + restart: always # or other settings... +``` + +## Commands + +### Config + +The following command shows you the docker-compose.yaml as it is interpreted by docker. + +``` +docker compose config +``` + +### Start + +Ins Verzeichnis gehen wo `docker-compose.yml` liegt, und dann `docker-compose start -d`. Mit `-d` wird es im Hintergrund ausgeführt. + +### Stop + + - The `docker compose stop` command will stop your containers, but it won’t remove them. + - The `docker compose kill` command will stop your containers faster, but it won’t remove them. + - The `docker compose down` command will stop your containers, but it also removes the stopped containers as well as any networks that were created. + - You can take down 1 step further and add the `-v` flag to remove all volumes too. This is great for doing a full blown reset on your environment by running `docker compose down -v`. + +### Events + +``` +docker compose events +```