3. Daggerize an App

Daggerize an App

The Challenge

After we have learned the basic Dagger Functions, we want to apply our new knowledge to solve a real life problem:

We would like to conduct a survey regarding the popularity of the different Dagger SDKs!

The Candidate

Fortunately, there is a free open-source quiz app called ClassQuiz .
It allows the creation of shareable, fully customizable quizzes and surveys.
The app is split in a frontend and an api part:

The frontend is written in TypeScript and uses a Redis memcache.
The backend is mostly written in python, uses a PostgreSQL database and Meilisearch.

Caddy is used as a reverse proxy to keep the parts together.

The Journey

Prerequisites

Create a fork of the ClassQuiz Repo on Github , this will come in handy later.

Then, check out your fork of ClassQuiz:

git clone https://github.com/_your-Github-user_/ClassQuiz.git

Get familiar with the source - take a closer look at the docker-compse.yaml , which is particularly interesting for our purpose.

The app binds the privileged port 80, which would be an obstacle.
So let’s replace all occurrences of :80 in Caddyfile-docker with :8081.
Additionally the missing protocol has to be added to the last reverse_proxy line. Add http:// in front of api:80.

Do it by hand or use the following sed commands:

sed -i 's# api:80# http://api:80#g' Caddyfile-docker
sed -i 's#api:80#api:8081#g' Caddyfile-docker

If patching does not work, overwrite the file Caddyfile-docker with the content from the following Caddyfile-docker file.

show final Caddyfile-docker file

# SPDX-FileCopyrightText: 2023 Marlon W (Mawoka)
#
# SPDX-License-Identifier: MPL-2.0

:8080 {
	reverse_proxy * http://frontend:3000
	reverse_proxy /api/* http://api:8081
	reverse_proxy /openapi.json http://api:8081 # Only use if you need to serve the OpenAPI spec
	reverse_proxy /socket.io/* http://api:8081

}

Start using Dagger

As we learnt in the first labs, Dagger functions are needed to encapsulate our pipeline functionality.

In Dagger, everything is a Module, therefore the first step is to initialize a Dagger Module.

A new Dagger module in Go, Python or TypeScript can be initialized by running dagger init inside the app’s root directory, using the --source flag to specify a directory for the module’s source code.

We will use the Python SDK for this example:

dagger init --sdk=python --source=./ci

This leaves us with a generated dagger.json module metadata file, an initial ci/src/class_quiz/main.py source code template, ci/pyproject.toml and other needed files, as well as a generated ci/sdk folder for local development.
The configuration file sets the name of the module to the name of the current directory, unless an alternative is specified with the --name argument.

To check if the module works and what example functions are created, run the functions command.

$ dagger functions
✔ connect 0.3s
✔ load module 1.4s

Name             Description
container-echo   Returns a container that echoes whatever string argument is provided
grep-dir         Returns lines that match a pattern in the files of the provided Directory

Run the App locally

The generated ci/src/class_quiz/main.py is the starting point, which needs to be extended.
It has already some example functions that are ready to use or extend.

The ClassQuiz repository has two Dockerfile. One to build the frontend and one to build the backend.
A starting point is to use the Dockerfile for a Docker build.
The resulting Docker image can be used to run the app inside a container.

As a first step, we could implement a simple build function:

function name: build
argument: context - the folder containing the Docker build context, including the Dockerfile
return: a Dagger Container

Note

The Dagger Engine has no access to your host computer. Therefore you have to explicitly provide folders as arguments.

    @function
    def build(self, context: dagger.Directory) -> dagger.Container:
        """Returns a container built with the given context."""
        return (
            dag.container()
            .build(context)
        )

The entrypoint to accessing the Dagger API from your own module’s code is dag, the Dagger client, which is pre-initialized.
It contains all the core types (like Container, Directory, etc.), as well as bindings to any dependencies your module has declared.

The Python SDK Reference documents all Dagger API types and functions.
Our function starts by creating a container (dag.container()). Here is the reference to the Python documentation.

The build executes the Docker build with the given files.

This function allows us to build the frontend as Container.
With function chaining we expose the container as a Service to the localhost on port 3000:

dagger call --mod ./ci/ build --context=./frontend/ with-exposed-port --port=3000 as-service up

Here we do the previous explained Function Chaining .

Our build method returns a Dagger container.
with-exposed-port --port=3000 opens the port to the Container (expose)
as-service Turn the container into a Service that runs the app.
up Opens the connection to the app Service. (Creates a tunnel that forwards traffic from the caller’s network to this service.)

Note

As we are in the root directory of the Dagger module, we do not need to provide the module (--mod) option.
This is the simplified command:

dagger call build --context=./frontend/ with-exposed-port --port=3000 as-service up

Use Ctrl +c to stop the container.

And the backend as well with its context folder:

dagger call build --context=. with-exposed-port --port=8000 as-service up

Warning

Unfortunately the Dagger call stops after a while. We have to analyze this!

If we do not see the relevant logs of the app in the output of the Dagger call, we should change the verbosity.
Try to make the output more verbose. This is implemented with the -v option.

Run the call again with the verbosity option:

dagger call -v build --context=. with-exposed-port --port=8000 as-service up

Note

If the output does still not contain the needed information of the problem, increase the verbosity of the Dagger call even more to get to the goal.
E.g. by adding two more levels at once (-v -> -vvv)

If we have a closer look to the console output, we will discover some error messages due to missing configurations.

As we have seen before, the two parts of the app depend on several components:

Redis
PostgreSQL
Meilisearch
Caddy

We have to implement each component as a Service , which then can be used. For Redis this could look like this:

    @function
    def redis(self) -> dagger.Service:
        """Returns a redis service from a container built with the given params."""
        return (
            dag.container()
            .from_("redis:alpine")
            .with_exposed_port(6379)
            .as_service()
        )

Note

This Container build does not use a Dockerfile. The Container is defined using the Dagger API.
The exposing to a Service, that we did with Bash before, is done in the function.

Add the redis function to your module.

Task 3.1: Implement Services

Add the remaining Services as well. Consult docker-compse.yaml for the required ports and params.

Note

A simple check for your function code is to run dagger functions. This will build/compile your code including all Dagger dependencies.
When you see your functions listed, then the syntax is right.

While the implementations of PostgreSQL and Meilisearch are very similar and quite simple:

show solution

    @function
    def postgres(self) -> dagger.Service:
        """Returns a postgres database service from a container built with the given params."""
        return (
            dag.container()
            .from_("postgres:14-alpine")
            .with_env_variable("POSTGRES_PASSWORD", "classquiz")
            .with_env_variable("POSTGRES_DB", "classquiz")
            .with_env_variable("POSTGRES_USER", "postgres")
            .with_exposed_port(5432)
            .as_service()
        )

    @function
    def meilisearch(self) -> dagger.Service:
        """Returns a meilisearch service from a container built with the given params."""
        return (
            dag.container()
            .from_("getmeili/meilisearch:v0.28.0")
            .with_exposed_port(7700)
            .as_service()
        )

The implementation of Caddy is a bit more sophisticated, as the proxy is our new entry point, which “glues” all the pieces together.

Official documentation about how to Bind services in functions .

Note

Important detail from the docs: The name used for the service binding defines the host name to be used by the function!

show solution

    @function
    def proxy(self, context: dagger.Directory, proxy_config: dagger.File) -> dagger.Service:
        """Returns a caddy proxy service encapsulating the front and backend services. This service must be bound to port 8000 in order to match some hard coded configuration: --ports 8000:8080"""
        return (
            dag.container()
            .from_("caddy:alpine")
            .with_service_binding("frontend", self.build(context.directory("frontend")).as_service())
            .with_service_binding("api", self.build(context).as_service())
            .with_file("/etc/caddy/Caddyfile", proxy_config)
            .with_exposed_port(8080)
            .as_service()
        )

You could try to run the ClassQuiz app now. But it will not work because of some missing configuration.

show solution

dagger call proxy --context=. --proxy-config=Caddyfile-docker up --ports=8000:8080

Task 3.2: Create separate Front- and Backend functions

Our initial build function can be used to create both, front- and backend containers.
But in fact, the two app parts require different config params and dependencies:
The frontend only communicates with the api of the backend, which is encapsulated by the Caddy reverse proxy, while the backend relies on the services we created earlier.

Hints:

Start with the backend and pass the required environment variables found in the docker-compose.yml.
For PORT use the port that you set inside the Caddyfile-docker earlier in this lab.

show required environment variables

$$MAX_WORKERS PORT REDIS SKIP_EMAIL_VERIFICATION DB_URL MAIL_ADDRESS MAIL_PASSWORD MAIL_USERNAME MAIL_SERVER MAIL_PORT SECRET_KEY MEILISEARCH_URL STORAGE_BACKEND STORAGE_PATH$$

show solution

    @function
    def backend(self, context: dagger.Directory) -> dagger.Container:
        """Returns a backend container built with the given context, params and service bindings."""
        return (
            dag.container()
            .with_env_variable("MAX_WORKERS", "1")
            .with_env_variable("PORT", "8081")
            .with_env_variable("REDIS", "redis://redisd:6379/0?decode_responses=True")
            .with_env_variable("SKIP_EMAIL_VERIFICATION", "True")
            .with_env_variable("DB_URL", "postgresql://postgres:classquiz@postgresd:5432/classquiz")
            .with_env_variable("MAIL_ADDRESS", "some@example.org")
            .with_env_variable("MAIL_PASSWORD", "some@example.org")
            .with_env_variable("MAIL_USERNAME", "some@example.org")
            .with_env_variable("MAIL_SERVER", "some.example.org")
            .with_env_variable("MAIL_PORT", "525")
            .with_env_variable("SECRET_KEY", "secret")
            .with_env_variable("MEILISEARCH_URL", "http://meilisearchd:7700")
            .with_env_variable("STORAGE_BACKEND", "local")
            .with_env_variable("STORAGE_PATH", "/app/data")
            .with_service_binding("postgresd", self.postgres())
            .with_service_binding("meilisearchd", self.meilisearch())
            .with_service_binding("redisd", self.redis())
            .build(context)
        )

And the frontend:

show solution

    @function
    def frontend(self, context: dagger.Directory) -> dagger.Container:
        """Returns a frontend container built with the given context and params."""
        return (
            dag.container()
            .with_env_variable("API_URL", "http://api:8081")
            .with_env_variable("REDIS_URL", "redis://redisd:6379/0?decode_responses=True")
            .build(context)
        )

Now the two service bindings in the proxy function needs to be changed accordingly.

Before:

            .with_service_binding("frontend", self.build(context.directory("frontend")).as_service())
            .with_service_binding("api", self.build(context).as_service())

After:

            .with_service_binding("frontend", self.frontend(context.directory("frontend")).as_service())
            .with_service_binding("api", self.backend(context).as_service())

Now we can run ClassQuiz locally:

dagger call proxy --context=. --proxy-config=Caddyfile-docker up --ports=8000:8080

And then visit localhost:8000 - where, after registering ourselves, we can log in and create our survey!

Note

Sometimes old cookies or session storage corrupts the app, especially when applying changes.
To fix this, delete all cookies and session data or open it in an incognito tab.

Complete Solution

ci/src/class_quiz/main.py:

import dagger
from dagger import dag, function, object_type

@object_type
class ClassQuiz:

    @function
    def frontend(self, context: dagger.Directory) -> dagger.Container:
        """Returns a frontend container built with the given context and params."""
        return (
            dag.container()
            .with_env_variable("API_URL", "http://api:8081")
            .with_env_variable("REDIS_URL", "redis://redisd:6379/0?decode_responses=True")
            .build(context)
        )

    @function
    def backend(self, context: dagger.Directory) -> dagger.Container:
        """Returns a backend container built with the given context, params and service bindings."""
        return (
            dag.container()
            .with_env_variable("MAX_WORKERS", "1")
            .with_env_variable("PORT", "8081")
            .with_env_variable("REDIS", "redis://redisd:6379/0?decode_responses=True")
            .with_env_variable("SKIP_EMAIL_VERIFICATION", "True")
            .with_env_variable("DB_URL", "postgresql://postgres:classquiz@postgresd:5432/classquiz")
            .with_env_variable("MAIL_ADDRESS", "some@example.org")
            .with_env_variable("MAIL_PASSWORD", "some@example.org")
            .with_env_variable("MAIL_USERNAME", "some@example.org")
            .with_env_variable("MAIL_SERVER", "some.example.org")
            .with_env_variable("MAIL_PORT", "525")
            .with_env_variable("SECRET_KEY", "secret")
            .with_env_variable("MEILISEARCH_URL", "http://meilisearchd:7700")
            .with_env_variable("STORAGE_BACKEND", "local")
            .with_env_variable("STORAGE_PATH", "/app/data")
            .with_service_binding("postgresd", self.postgres())
            .with_service_binding("meilisearchd", self.meilisearch())
            .with_service_binding("redisd", self.redis())
            .build(context)
        )

    @function
    def redis(self) -> dagger.Service:
        """Returns a redis service from a container built with the given params."""
        return (
            dag.container()
            .from_("redis:alpine")
            .with_exposed_port(6379)
            .as_service()
        )

    @function
    def postgres(self) -> dagger.Service:
        """Returns a postgres database service from a container built with the given params."""
        return (
            dag.container()
            .from_("postgres:14-alpine")
            .with_env_variable("POSTGRES_PASSWORD", "classquiz")
            .with_env_variable("POSTGRES_DB", "classquiz")
            .with_env_variable("POSTGRES_USER", "postgres")
            .with_exposed_port(5432)
            .as_service()
        )

    @function
    def meilisearch(self) -> dagger.Service:
        """Returns a meilisearch service from a container built with the given params."""
        return (
            dag.container()
            .from_("getmeili/meilisearch:v0.28.0")
            .with_exposed_port(7700)
            .as_service()
        )

    @function
    def proxy(self, context: dagger.Directory, proxy_config: dagger.File) -> dagger.Service:
        """Returns a caddy proxy service encapsulating the front and backend services. This service must be bound to port 8000 in order to match some hard coded configuration: --ports 8000:8080"""
        return (
            dag.container()
            .from_("caddy:alpine")
            .with_service_binding("frontend", self.frontend(context.directory("frontend")).as_service())
            .with_service_binding("api", self.backend(context).as_service())
            .with_file("/etc/caddy/Caddyfile", proxy_config)
            .with_exposed_port(8080)
            .as_service()
        )

Cadyyfile-docker:

# SPDX-FileCopyrightText: 2023 Marlon W (Mawoka)
#
# SPDX-License-Identifier: MPL-2.0

:8080 {
	reverse_proxy * http://frontend:3000
	reverse_proxy /api/* http://api:8081
	reverse_proxy /openapi.json http://api:8081 # Only use if you need to serve the OpenAPI spec
	reverse_proxy /socket.io/* http://api:8081

}

Last modified May 16, 2025: Merge pull request #108 from puzzle/config (04f5c81)