Persistent volumes

Outline

In this chapter you will learn:

• Why perisistent volumes are important.
• How to create a virtual storage volume for storing persistent data generated by containers.
• How to attach a volume to a container.

Walkthrough

By default all files created inside a container are stored on an ephemeral filesystem layer. This layer is tightly coupled with the container lifecycle. After the container is removed its associated writable layer is also removed. Consequently, all data populated by the container is lost.

Task 1: Why do we need persistent storage volumes

The objective of this task is to ilustrate the problem of persistent data storage in containers.

Start with running a Redis container in the background:

$ docker run --name novoltest -d redis

Create a file in the running container:

$ docker exec novoltest touch testfile.txt

Ensure that the file is present in the container:

$ docker exec novoltest ls -l testfile.txt
-rw-r--r-- 1 root root 0 May  4 21:47 testfile.txt

Restart the container:

$ docker stop novoltest
novoltest
$ docker start novoltest
novoltest

Check if the file is present in the container:

$ docker exec novoltest ls -l testfile.txt
-rw-r--r-- 1 root root 0 May  4 21:47 testfile.txt

The file is still present in the container.

Now, let's remove the container and start a new one from the same image:

$ docker stop novoltest && docker rm novoltest
$ docker run --name novoltest -d redis

Check if the file is present in the container:

$ docker exec novoltest ls -l testfile.txt
ls: cannot access 'testfile.txt': No such file or directory

testfile.txt is no longer present, because it was stored only in the ephemeral filesystem layer created for the previous container. That layer has been removed with the old container. The new container has its own writable layer created with an empty slate.

Task 2: Creating a volume

Volumes are the preferred mechanism for persisting data generated by containers. They are directories created and managed by Docker engine in the host filesystem. Volumes are not dependent on container's lifecycle and as such are not removed with the containers.

Let's create a sample volume:

$ docker volume create mydata

Confirm that the volume has been successfully created by listing it:

$ docker volume list
DRIVER              VOLUME NAME
local               2b3b42e40634ef8a95884eb6c7813402caa65988e90f699c8803480fc393e25a
...
local               mydata

Display volume details:

$ docker volume inspect mydata
[
    {
        "CreatedAt": "2019-05-05T00:03:32+02:00",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/mydata/_data",
        "Name": "mydata",
        "Options": {},
        "Scope": "local"
    }
]

Analyze the command output.

Note location of the created volume: /var/lib/docker/volumes/mydata/_data. /var/lib/docker/volumes is a Docker-managed directory in which all volumes are stored. Nested mydata/_data directory is a concrete instance of our mydata volume object.

The volume directory can be accessed from the Docker host:

$ sudo ls -l /var/lib/docker/volumes/mydata/_data

Task 3: Attaching volume to a container

After creating a volume we can attach it to a new container using --mount option in docker run command. It has the following syntax:

--mount type=volume,source=VOLUME_NAME,destination=CONTAINER_PATH[,readonly]

VOLUME_NAME is a name of an existing volume object. CONTAINER_PATH is a path inside a container. Docker will mount the volume in the provided path and the path will be accessible by processes running inside the container.

Let's create a Redis container with persistent volume attached:

$ docker run \
  -d \
  --name voltest1 \
  --mount type=volume,source=mydata,destination=/data \
  redis redis-server --appendonly yes

The above command runs a container from Redis image (redis) in the background (-d), names it voltest1 (--name), mounts volume mydata under /data path in the container (--mount), and executes redis-server --appendonly yes command inside the container.

By providing option --appendonly yes we instruct Redis to store data in /data directory under which we mount the volume. Our goal is to ensure persistent data store for our Redis service.

Add some sample data into Redis using redis-cli utility:

$ docker exec --interactive --tty voltest1 bash
$ root@b527993a6929:/data# redis-cli
$ 127.0.0.1:6379> set foo bar
OK
$ 127.0.0.1:6379> set bar baz
OK
$ 127.0.0.1:6379> exit
$ root@b527993a6929:/data# exit
exit

Now, remove the Redis container and start a new one with the same storage volume attached:

$ docker stop voltest1 && docker rm voltest1
$ docker run \
  -d \
  --name voltest1 \
  --mount type=volume,source=mydata,destination=/data \
  redis redis-server --appendonly yes

Ensure that Redis data is restored after removal of the previous container:

$ docker exec --interactive --tty voltest1 bash
$ root@b527993a6929:/data# redis-cli
$ 127.0.0.1:6379> get foo
"bar"
$ 127.0.0.1:6379> get bar
"baz
$ 127.0.0.1:6379> exit
$ root@b527993a6929:/data# exit
exit

Analyze the command output.

Keys foo and bar created in the previous container are still present in the new container even though the previous container has been removed.

Exercises

1. Run another Redis container with mydata volume attached. Ensure that data is shared between containers via the storage volume, e.g., set different keys on each Redis instance using redis-cli utility and verify that both instances have the same sets of keys.