Linux Server

This guide explains how to deploy a Brezel instance to a (virtual) server. For this guide, we assume the base domain is called example.io and Brezel will be accessible under brezel.example.io. The example IP of the server is given as 5.35.243.342.

Make sure to replace the path to your instance and domain in all examples below

Otherwise, your brezel will not work correctly!

Prerequisites

• A Brezel based on a recent version of brezel/api. i.e. brezel/api@1.0.0 or higher.
• GitLab repository for that Brezel instance
• An Ubuntu or Debian based server satisfying the requirements
• sudo access on that machine

Basic server setup

On a brand new, empty server

If this is a new server with nothing else on it, you should create a new system user with sudo privileges.

This user will not run Brezel, but will be used to set up the server and any administrative tasks. (Using root for this is not recommended.)

Change the name of that user to your liking, we will use “kibro” here.

sudo adduser kibro
sudo usermod -aG sudo kibro
sudo usermod -aG systemd-journal kibro

Harden the server

1. Add a public key for SSH access to the new user.

   First, create a new key pair on your local machine:

   ssh-keygen -C "kibro"

   Make sure to put those keys (public and private) in a safe and secure place. Possibly share them with the team using a password manager (e.g. Passbolt).

   Now, still on your local machine run this with the correct values to copy the public key to the server so you can log in with the private key:

   ssh-copy-id -i /path/to/the/public/key/to/use.pub kibro@5.35.243.342

   Note, if you are on windows and ssh-copy-id is not found, you can use

   ssh kibro@5.35.243.342 "mkdir .ssh; touch .ssh/authorized_keys"
   
   type /path/to/the/public/key/to/use.pub | ssh kibro@5.35.243.342 "cat >> .ssh/authorized_keys"

   Finally, validate that you can log in with the new user:

   ssh kibro@5.35.243.342 -i /path/to/the/private/key/to/use

   Additionally, you might want to setup your .ssh/config in a way that allows you to easily connect to the host without specifying the key each time.

   Host your-prefered-name
      HostName 5.35.243.342
      User kibro
      IdentitiesOnly yes
      IdentityFile /path/to/the/private/key/to/use (e.g. ~/.ssh/kibro)

2. Now disable password SSH based authentication in /etc/ssh/sshd_config:

   Do the next steps as the kibro user on your server!

   This will lock out anyone not using a certificate based SSH access

   That includes you if you did not test the certificate based access before!

   It will also lock out any other user that does not have a certificate based access set up. You should only do this if you are sure that noone else needs to access the server via SSH and a password.

   PasswordAuthentication no

   Make sure no file in /etc/ssh/sshd_config.d/ overrides this setting.

   e.g. on Hetzner servers, you need to delete the file /etc/ssh/sshd_config.d/50-cloud-init.conf

     sudo systemctl restart sshd

Note, on some Ubuntu images the service is called ssh not sshd. Use sudo systemctl restart ssh if it complains that the service sshd is not found, try this.

3. Set up a ufw Firewall:

   This will block unwanted traffic to your server.

   We allow SSH, HTTP and HTTPS traffic here. HTTP is only needed for the certificate auto (re)-generation.

   sudo ufw allow ssh
   sudo ufw allow http
   sudo ufw allow https
   sudo ufw disable
   sudo ufw enable

   The disable / enable dance at the end is needed to restart the firewall with the new rules.

4. Install and setup CrowdSec or fail2ban:

   Tools like this will block spammy requests to your server and protect it from brute force attacks.

   We will use CrowdSec here, but fail2ban is a good alternative.

   Install it using the latest instructions from the CrowdSec documentation. You most probably need to register the repository of them, see here: Add CrowdSec repository.

   Do the following:

   1. Install the CrowdSec repository
   2. Install the CrowdSec engine
   3. Install the correct firewall bouncer for your system

   Make sure you install the correct one! E.g. check with iptables -V if you need iptables or nftables support.

   Make sure to check the official documentation for the most up-to-date installation instructions.

   Now you can validate that the bouncer and the engine are running:

   sudo cscli bouncers list

   To see more detailed infos about what CroudSec is doing, you can check the metrics:

   sudo cscli metrics

You are now considered being on “an already setup server”

Continue with the steps for that situation.

(i.e. don’t skip the next subchapter :D)

On an already setup server

You fall into this category if you already have a user with sudo access on the server.

For improved security, we recommend always following the step “using certificate based SSH access” from the “hardening” chapter above!

This makes interaction with the server way more secure and is an easy prevention against simple SSH brute force attacks.

Although, if you cannot disable password based SSH access, this step mostly brings convenience and not security.

Now create a new user for your brezel instance:

sudo adduser brezel

Add your brezel user to the www-data group to ensure the correct permissions in combination with the Nginx web server we’ll set up and use later on.

sudo usermod -aG www-data brezel

This will be the user that runs and operates the Brezel instance. Most of the actually Brezel related setup in this guide will use this user, only some commands will require sudo and thus the other user.

Rule of thumb: If sudo is NOT used in a command, try it first with brezel, especially for any php or supervisor commands. (If additional instructions don’t say otherwise.)

Install needed dependencies

Use our sudo user for this (in our case kibro).

0. (Optional but highly recommended) Useful utilities to work and debug:

   sudo apt-get update && sudo apt-get upgrade
   sudo apt-get install fzf htop ncdu mycli python3-pygments

   Activate fzf for your user by adding the following to your ~/.bashrc:

   # Set up fzf key bindings and fuzzy completion
   eval "$(fzf --bash)"

   What to do if fzf warns about this command not available on this version

   Older Ubuntu versions might not have an up to date fzf installed that have this easy install availabe. In that case, add this to your ~/.bashrc instead:

   # Enable fzf key bindings and fuzzy auto-completion (Debian/Ubuntu)
   if [ -f /usr/share/doc/fzf/examples/key-bindings.bash ]; then
     source /usr/share/doc/fzf/examples/key-bindings.bash
   fi
   
   if [ -f /usr/share/bash-completion/completions/fzf ]; then
     source /usr/share/bash-completion/completions/fzf
   fi

   After that you have a nice searchable history available to you via ctrl+r.

   Use mycli -u brezel to interact with the DB in a cleaner way

1. Nginx, PHP, PHP extensions, and other necessary dependencies:

   sudo apt-get update && sudo apt-get upgrade
   sudo add-apt-repository ppa:ondrej/php
   sudo apt install nginx
   sudo apt-get install php8.4-cli php8.4-fpm php8.4-curl php8.4-xml php8.4-opcache php8.4-mysql php8.4-zip php8.4-exif php8.4-bcmath php8.4-sockets php8.4-gmp php8.4-imap php8.4-intl php8.4-gd php8.4-mbstring
   sudo apt-get install unzip ghostscript qpdf

2. Imagick:

   This assumes that the ondrej/php ppa is available. It should be if you followed the steps above.

   ImageMagick is needed to generate thumbnails. Due to weird issues with weird aspect ratio PDFs when generating high DPI (“original”) quality thumbnails, it is highly recommended to use ImageMagick 7.x! Unfortunately, the default repositories and the default php8.4-imagick package use ImageMagick 6.x, so we need to build it from source. It is important that we have the development libraries for various formats (especially the ghostscript libs for PDF support) installed before building ImageMagick from source.

   Make sure you uninstall any existing imagick version first to avoid conflicts. Just run the following commands to be sure (some might error out, but that is fine):

   sudo pecl uninstall imagick
   sudo apt remove php8.4-imagick
   sudo apt remove imagemagick imagemagick-6-common libmagickwand-6.q16-dev libmagickcore-6.q16-dev
   sudo apt autoremove

   Now we can go ahead and install the necessary dependencies, build ImageMagick 7 from source and install the imagick php extension:

   You might see some warnings or errors during make install, as long as the rest seems to have worked fine, you are ok.

   # Install required libraries and build tools for compiling ImageMagick 7
   sudo apt-get install -y libgs-dev libjpeg-dev libpng-dev \
   libtiff-dev libwebp-dev build-essential php8.4-dev curl
   
   # Compile ImageMagick 7 from source using the official GitHub repository
   cd /usr/src/
   sudo wget https://github.com/ImageMagick/ImageMagick/archive/refs/heads/main.tar.gz -O ImageMagick.tar.gz
   sudo tar xvf ImageMagick.tar.gz
   cd ImageMagick-main
   sudo ./configure
   sudo make -j$(nproc)
   sudo make install
   sudo ldconfig
   
   # Download and install the latest PIE (PHP Installer for Extensions)
   sudo curl -fL --output /usr/local/bin/pie https://github.com/php/pie/releases/latest/download/pie.phar
   sudo chmod +x /usr/local/bin/pie
   
   # Install the Imagick PHP extension using pkg-config to locate the custom ImageMagick 7 build
   sudo env PKG_CONFIG_PATH=/usr/local/lib/pkgconfig pie install imagick/imagick
   
   # Ensure the extension is enabled in the PHP configuration
   if [ ! -f /etc/php/8.4/mods-available/imagick.ini ]; then
   echo "extension=imagick.so" | sudo tee /etc/php/8.4/mods-available/imagick.ini
   sudo phpenmod -v 8.4 imagick
   fi
   
   # Restart PHP-FPM to apply changes
   sudo systemctl restart php8.4-fpm.service

   If you indeed got some errors, or just want to verify that it worked, run these commands:

   php -m | grep imagick # Should output "imagick"
   php --ri imagick # Should show version info and you should find "PDF" in the supported formats

3. Setup for SSL certificates: There are two variants to choose from. The one we’ve always used is “Certbot”, see option 1. However, especially when you are dealing with an installation in a closed network that is only accessible via a VPN, you might want to use the “acme.sh” variant instead.

   1. Certbot See certbot.eff.org for up-to-date instructions.

      sudo snap install --classic certbot
      sudo ln -s /snap/bin/certbot /usr/bin/certbot

   If sudo snap install --classic certbot does not work because you are behind some kind of firewall or don’t have access to snap, you can also install certbot via sudo apt install certbot python3-certbot-nginx

   2. acme.sh See acme.sh for up-to-date instructions.

   Current example:

   curl https://get.acme.sh | sh

   You can now issue certs using the DNS challenge for your nginx domains like this:

   sudo acme.sh --issue --dns --nginx -d example.com

   Remember you need to register the certs manually in NGINX!

4. Install MariaDB using a guide like this.

   Or follow these rough steps:

   sudo apt update
   sudo apt install mariadb-server
   sudo mysql_secure_installation

   In the installation process, read closely what it asks you and answer accordingly. You want to remove the test database and disallow remote root login. And you should set a secure password for the root user.

   At this point you can already create a “brezel” database user with the necessary permissions while you are at it.

   mariadb -u root -p

   CREATE USER 'brezel'@'localhost' IDENTIFIED BY '<your secure password>';
   GRANT ALL ON *.* TO 'brezel'@'localhost' WITH GRANT OPTION;
   FLUSH PRIVILEGES;

   Make sure to use a password that fits into the limits of MariaDB

   By default, recent MariaDB with mysql_native_password plugin only allows passwords up to 80 characters.

   Especially passphrases generated by password managers can be longer than that.

   This also applies to your root user of the db, not just the brezel user.

Server Performance Tweaks

These are recommended performance tweaks for your server. You can skip them, but your Brezel might not perform as well as it could.

Nginx configuration

Ensure these values in your top level/etc/nginx/nginx.conf file:

worker_processes auto;

events {
    worker_connections 2048;
}

If they have already been set, be sure to change the values and not just add a new line with the same directive.

Restart Nginx:

sudo systemctl restart nginx

Configure the Database

Adjust your database configuration in /etc/mysql/my.cnf by adding this block to the very bottom:

[mysqld]
max_connections = 500
innodb_buffer_pool_size = 4G
innodb_log_file_size = 512M
innodb_log_buffer_size = 16M
max_allowed_packet = 64M

If you are running MariaDB add the following options too:

thread_pool_size = 100
query_cache_size = 64M

Now restart the database (Use mysql if you are running MySQL):

sudo systemctl restart mariadb

PHP-FPM configuration

Configure your php-fpm settings in /etc/php/8.4/fpm/pool.d/www.conf:

pm = dynamic
pm.max_children = 300 ; Handle at max X requests at once
pm.max_requests = 1500 ; Restart worker after X requests to prevent memory leaks
pm.start_servers = 10
pm.min_spare_servers = 10
pm.max_spare_servers = 50
pm.process_idle_timeout = 15s

You can either just paste these values at the end of the file or search for the values and change them.

Now ensure that file permissions are properly kept. For that edit the php-fpm systemd service file using sudo systemctl edit php8.4-fpm.service and add the following lines IN BETWEEN THE COMMENTS, as instructed:

[Service]
UMask=0002

Finally, restart PHP-FPM:

sudo systemctl daemon-reload
sudo systemctl restart php8.4-fpm

PHP configuration

Finally, add the following values to your /etc/php/8.4/fpm/php.ini file:

; Enable opcache and configure for laravel
opcache.enable_cli=1
opcache.enable = 1
opcache.memory_consumption = 192
opcache.interned_strings_buffer = 16
opcache.max_accelerated_files = 10000
opcache.revalidate_freq = 0
opcache.validate_timestamps = 1
opcache.max_wasted_percentage = 10
opcache.fast_shutdown = 1

; Security tweaks
expose_php = Off ; Remove X-Powered-By header
disable_functions = shell_exec,system ; we need exec for file merging. passthru is also used for composer installation

; File size settings
upload_max_filesize = 25M
post_max_size = 25M

; Memory limit
memory_limit = 1G

; Execution time
max_execution_time = 180

; This is needed because we hackily backported hyn to php 8. We should remove this once we migrate tenancy framework
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off

Add DNS records

Login to your external DNS provider and add the following DNS records (replace the bold values with your use case):

Name	Type	Value
brezel.example.io	A	5.35.243.342
api.brezel.example.io	A	5.35.243.342
ws.brezel.example.io	A	5.35.243.342

Add the SPA domain

In /etc/nginx/sites-available, create a new file called spa:

server {
    listen 80;
    listen [::]:80;
    server_name brezel.example.io;

    location / {
        root /var/www/vhosts/api.brezel.example.io/dist;
        index  index.html index.htm;
        try_files $uri $uri/ /index.html;
    }

    # redirect server error pages to the static page /50x.html
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root /var/www/vhosts/api.brezel.example.io/dist;
    }
}

Save the file, then create a symlink:

sudo ln -s /etc/nginx/sites-available/spa /etc/nginx/sites-enabled/spa

Restart NGINX:

sudo systemctl restart nginx

Finally, secure the domain with an SSL certificate from certbot:

sudo certbot --nginx -d brezel.example.io

Add and configure the API domain

We start off with a few performance tweaks for the server.

Subdomain

Now we create a nginx php-fpm config on the subdomain for the API.

In /etc/nginx/sites-available, create a new file called api:

server {
    listen 80;
    listen [::]:80;
    server_name api.brezel.example.io;

    root /var/www/vhosts/api.brezel.example.io/public;

    # Increase max upload size
    client_max_body_size 25M;

    # Allow bigger queries e.g. for "id in" resource table filters with many ids
    large_client_header_buffers 8 64k;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";

    # Add index.php to the list if you are using PHP
    index index.php index.html index.htm index.nginx-debian.html;

    charset utf-8;

    location / {
            try_files $uri $uri/ /index.php?$query_string;
            gzip_static on;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.4-fpm.sock;
        fastcgi_read_timeout 180s;
        include snippets/fastcgi-php.conf;
    }
}

Save the file, then create a symlink:

sudo ln -s /etc/nginx/sites-available/api /etc/nginx/sites-enabled/api

Restart Nginx:

sudo systemctl restart nginx

Finally, secure the domain with an SSL certificate from certbot:

sudo certbot --nginx -d api.brezel.example.io

Get Composer ready

Switch to the brezel (su - brezel) user and install composer via the official composer docs.

Once you have the compoer.phar, leave it where it is in your brezel users home folder!

Then switch back to the kibro user (exit in the current su session) and use this command to make it available globally:

sudo mv /home/brezel/composer.phar /usr/local/bin/composer

The Brotcast domain

Brotcast is a websocket server that is used for real-time communication between the Brezel instance and the frontend.

In /etc/nginx/sites-available, create a new file called brotcast:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 80;
    listen [::]:80;

    server_name ws.brezel.example.io;

    location / {
        proxy_http_version 1.1;

        # Dynamic Connection header based on upgrade request
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        # Additional headers for better proxy handling
        proxy_set_header Host $http_host;
        proxy_set_header Scheme $scheme;
        proxy_set_header SERVER_PORT $server_port;
        proxy_set_header REMOTE_ADDR $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_pass http://127.0.0.1:8086/; # When hosted via docker compose or laravel reverb
    }
}

Save the file, then create a symlink:

sudo ln -s /etc/nginx/sites-available/brotcast /etc/nginx/sites-enabled/brotcast

Restart Nginx:

sudo systemctl restart nginx

Finally, secure the domain with an SSL certificate from certbot:

sudo certbot --nginx -d ws.brezel.example.io;

Prepare for an incoming Brezel

1. Ensure that you have a user on your server called brezel wich is in the www-data group.

2. Create the web root directory where your Brezel instance will live

   sudo mkdir -p /var/www/vhosts/your-brezel

3. Set the correct initial permissions for the new directory:

   sudo chown -R brezel:www-data /var/www/vhosts/your-brezel
   sudo chmod -R 2775 /var/www/vhosts/your-brezel

4. For convenience, create a symlink in your brezel users home directory to the web root directory:

   sudo ln -s /var/www/vhosts/your-brezel /home/brezel/brezel

   Now you can easily access the web root directory by just running cd ~/brezel after you login via ssh and change to the brezel user.

Get your Brezel onto the server

Connect to GitLab

This will create SSH keys for the brezel user and add them to GitLab as “deploy keys”. That way, the server can pull your Brezel from the remote repository.

1. Switch to the brezel user.

   su - brezel

2. Generate a new key pair:

   ssh-keygen -b 4096

   You will be asked for a path. The default is probably fine. Just ensure it is in the home directory of the brezel user (~/.ssh/id_<something>).

   Hit enter.

   You will then be asked for a passphrase. We want no passphrase. Leave the field empty and hit enter.

   Hit enter again to confirm the empty password. It will then output something like this:

   Your identification has been saved in /home/brezel/.ssh/id_<something>.
   Your public key has been saved in /home/brezel/.ssh/id_<something>.pub.
   The key fingerprint is:
   SHA256:hcsvjIXxM8ZS16X3IHIoGFwPSHr5Cjw35HOIZMEIW1c brezel@server.de
   The key's randomart image is:
   +---[RSA 4096]----+
   |...oo+Eoo     .  |
   | o...oo+ + o o   |
   |.   + * + * = o  |
   |   + = X = o o o |
   |    = O S       .|
   |     + @ +       |
   |      o o .      |
   |         .       |
   |                 |
   +----[SHA256]-----+

3. Now, go to GitLab, to the repository of your Brezel instance and go to Settings > Repository > Deploy Keys. We want our server to be able to read from the GitLab repository. Paste the contents of ~/.ssh/id_<something>.pub to the Key field. Give it a meaningful title like Production [brezel.example.io].

Clone the repository

1. Move to your new user’s web root directory where we will clone the application repository to.

   cd /var/www/vhosts/api.brezel.example.io

2. Clone the instance repository (Get the SSH URL from the GitLab repository):

Install Git if it’s not already installed.

Switch back to kibro and run:

sudo apt-get install git

Remember to go back to brezel after this!

git clone --branch main git@gitlab.kiwis-and-brownies.de:kibro/basedonbrezel/example.git .

Configure file system permissions

Make sure that the instance folder is owned by brezel and www-data with both having permissions to read, write and create files.

We also need to make sure that this ownership is propagated to all subdirectories and files, including newly created files. To do that, run the following commands:

Make sure to replace the path to your instance

Otherwise, your brezel will not work correctly!

Make sure umask is properly set

Ensure that php-fpm has UMask=0002 set in its systemd service file as described above! Your async queues running via supervisor also need to have this configured as above!

Otherwise, newly created files and folders via SPA will not have the correct permissions and your brezel will break in mysterious ways.

E.g. async jobs not having access to some files or folders.

Switch back to kibro and run:

# Set your project's root directory
APP_PATH="/var/www/vhosts/api.brezel.example.io"
# Set ownership to your user and the www-data group
sudo chown -R brezel:www-data ${APP_PATH}
# Set permissions for all directories to 2775 (rwxrwx-r-x) including the setgid bit
sudo find ${APP_PATH} -type d -exec chmod 2775 {} \;
# Set permissions for all files to 664 (rw-rw-r--)
sudo find ${APP_PATH} -type f -exec chmod 664 {} \;
# Apply the 'setgid' bit to all directories, just in case
sudo find ${APP_PATH} -type d -exec chmod g+s {} \;
# Ensure laravel specific directories are writable as per docs (technically already done, but better be safe than sorry)
sudo chmod -R 2775 ${APP_PATH}/storage
# Make sure acl is installed
sudo apt-get install acl
# Make sure existing folders that need it follow the rules that both brezel and www-data can read/write them
sudo setfacl -R -m u:brezel:rwX,u:www-data:rwX ${APP_PATH}/storage
# Now do the same for all future files and folders created in those folders
sudo setfacl -R -d -m u:brezel:rwX,u:www-data:rwX ${APP_PATH}/storage

Use brezel/api 3.2.3 or higher to have better handling of folder permissions

This version introduced a workaround for a stupid laravel behaviour (https://github.com/laravel/framework/issues/30367), so folders created via SPA will have the correct permissions so that your brezel user can access them from queue-jobs to read/write files in them and vice versa.

Set up Brezel

In general the Brezel instance will be deployed to the API subdomain. The SPA domain will receive the contents of the minified frontend found in the dist/ folder.

Configure the environment

Ensure you are the brezel user and in the correct directory.

su - brezel
cd /var/www/vhosts/api.brezel.example.io

Copy the .env.example file to .env and adjust the values to your needs.

cp .env.example .env

Follow the comments and instructions in the .env file to set up the environment. Here are some key tips:

• APP_URL should be set to your API URL, e.g. https://api.brezel.example.io
• Make sure to set APP_ENV to production and APP_DEBUG to false

Configure Brezel to use the database

Variant 1: Use root credentials

TENANCY_DATABASE="brezel_meta"
TENANCY_USERNAME="root"
TENANCY_PASSWORD="<password>"

Variant 2: Use a privileged database user

Note

This is the one to use when you followed this guide.

Or, if you have root privileges, you can create a brezel MySQL/MariaDB user with ALL privileges:

GRANT ALL ON *.* TO 'brezel'@'localhost' identified by '<password>' WITH GRANT OPTION;
FLUSH PRIVILEGES;

You don’t need to do that if you did it while installing MariaDB above!

There we already created a brezel user with the necessary permissions.

And set the following credentials:

TENANCY_DATABASE="brezel_meta"
TENANCY_USERNAME="brezel"
TENANCY_PASSWORD="<password>"

Install Brezel

Automatically via a Pipeline (recommended)

Follow the Pipeline guide for SSH based deployment. You can skip step 1 in there as you already generated the keys.

You will need to allow SSH for the brezel user

This is needed for the pipeline to work.

Change to the brezel user if you are not already and run cat ~/.ssh/id_<something>.pub >> ~/.ssh/authorized_keys. Use the same value you put into GitLab as the deploy key.

Now you can trigger an initial deployment by pushing to the main branch

This should trigger the pipeline and deploy the instance.

Create your system

Now all dependencies should be installed, the needed files for SPA and API should be in place and the environment should be set up.

Note that the pipeline should have NOT failed, but your system will not be ready yet! For that, the next steps are here.

After this block, the system will be up and running ans subsequent pipelines will just update as usual.

Make sure to be the brezel user when running ANY bakery command

Failure to do this NOW OR LATER will break your instance as it will create files with the wrong permissions.

You can now create your system by its name by running the following command:

php bakery system create kab

Finally, run the setup commands:

php bakery init --force
php bakery migrate --force
php bakery apply
php bakery load --force
php bakery make:supervisor

These are the same commands the pipeline would run

You could also just re-trigger the pipeline after creating the system

We now only need to do one more file-permission fix before everything is ready

Switch back to kibro and run:

sudo chmod -R 2775 /var/www/vhosts/brezel.example.io/storage

This will update the permissions of the newly created auth keys to that both www-data and the brezel user can read and write them which is needed for login to work!

After this, a basic Brezel should be up and running. For the full functionality, you need to set up additional services.

Set up additional services

Setting up the brotcast environment

Installation

Available since: brezel/api@1.0.0 Since 1.0.0, Brezel ships with a Laravel Reverb based Brotcast server integrated! Once you setup Supervisor, the server will be started automatically.

If you are not using the Laravel Reverb based Brotcast server, setup brotcast via Docker

This feature requires a container runtime like docker

Make sure you have docker installed.

Then create a docker-compose.yml file with the following content:

services:
  brotcast:
    image: registry.kiwis-and-brownies.de/kibro/brezel/brotcast-server:latest
    ports:
      - "8086:8086"
    environment:
      - SOKETI_PORT=8086
      - SOKETI_DEFAULT_APP_ID=brotcast
      - SOKETI_DEFAULT_APP_KEY=brotcast-pusher
      - SOKETI_DEFAULT_APP_SECRET=<a alphanumerical secret>

Run docker compose up in the same directory.

Performance tweaks for a Reverb based Brotcast server

You might want to take a look at https://laravel.com/docs/11.x/reverb#production for some possible performance considerations.

Especially increasing the available ports / file descriptors as well as switching to a different runtime might be beneficial.

Configuration

Configure the brotcast environment in the .env file:

# Broadcasting settings
BROADCAST_DRIVER=reverb# either `pusher` or `reverb` Use `reverb` if you are using the Laravel Reverb based Brotcast server (Recommended)
# Where to run the Brotcast server itself if using reverb
BREZEL_BROTCAST_SERVER_HOST=127.0.0.1
BREZEL_BROTCAST_SERVER_PORT=8086
# How to connect to the Brotcast server. This url should point to a pusher-compatible websocket server.
# If you are using the `pusher` driver, this should point to a brezel/brotcast-server instance
# If using the `reverb` driver, this should point to the Brotcast server endpoint defined above
BREZEL_BROTCAST_HOST=ws.brezel.example.io
BREZEL_BROTCAST_PORT=443
# Common Brotcast settings
# When using the `pusher` driver, these should be your pusher credentials and match what is configured in the Brotcast server
BREZEL_BROTCAST_APP_ID=brezel
BREZEL_BROTCAST_KEY=brotcast-pusher
BREZEL_BROTCAST_SECRET=an-alphanumerical-secret
BREZEL_BROTCAST_SCHEME=https
# Only relevant when using the `pusher` driver
BREZEL_BROTCAST_APP_CLUSTER=mt1

Set up crons

If you want to use event/cron, you need to set up a cronjob. Very basically: this should trigger php bakery schedule in the directory of your instance every minute as the user that normally runs your brezel (when using nginx and following this setup it will be www-data). How you achieve this depends on you, especially on your local setup, but on a server it makes sense to use cron.

Make sure you are the brezel user and run

crontab -e

Now add the following line:

Make sure to replace the path

Otherwise, crons will not run and just fail silently.

* * * * * cd /var/www/vhosts/api.brezel.example.io && php bakery schedule >> /dev/null 2>&1

Finally, verify that this was saved correctly by running crontab -l and checking if the line is there.

Set up the export service

This feature requires a container runtime like docker

Install it if possible! See the docker docs.

In permission constrained environments udocker may be used as an alternative. You will however have more friction setting up the service and possibly some compatibility issues.

If your brezel does stuff like creating pdfs or filling .docx templates, you need to set up the export service.

This is a container based environment that provides services like WKHtmlToPdf, Pandoc and more.

If you choose to use the Docker variant, make sure you add the kibro user to the docker group and continue this installation as the kibro user. Adding brezel to the docker group is not recommended as it gives root-like permissions to the brezel user, which is not desired.

sudo usermod -aG docker kibro

Installation instructions and a handy CLI can be found here: kibro/brezel/export/export-installer.

The easiest way is to install bun and then follow the guide in the linked Repo. Hint: do this in /home/kibro/ in a new directory called export.

If you use a 1.x or lower version of brezel/api you need to set up the export service manually

In the installer repository metioned above, there is a old_export-docker_compose.yml file that you can use to set up the export service manually. Remove the services you don’t need (you should always at least keep OCR).

Then run sudo docker compose up -d in the same directory as the file.

Now you need to configure the export service in your .env file:

BREZEL_EXPORT_URL=http://127.0.0.1:5580

Install and configure supervisor

Supervisor handles the starting and stopping all the php-based services Brezel needs and ships with brezel/api. That includes brotcast (if you use Reverb), the queues and so on.

Installing supervisor

Switch to a user with sudo privileges and install supervisor:

sudo apt-get update
sudo apt-get install supervisor

This is the last time you do anything supervisor related as a user with sudo privileges

EVERYTHING else will be done as the brezel user.

Otherwise, you will run into permission troubles.

Generate the supervisor configuration

Switch back to the brezel user and go into the instance directory.

If you have a brezel/api version higher than 3.0, you need to adjust permissions

Make sure that this is in your systems .env file or else the supervisor jobs will try to run as www-data which will NOT work:

BREZEL_JOBS_SUPERVISOR_USER=brezel

Now copy supervisord.conf.example to supervisord.conf and adjust the socket names if desired.

Now run the following command to create the supervisor configuration:

php bakery make:supervisor

Now we register our supervisor programms with the supervisor daemon

We want our supervisor programs to start on system boot.

This part needs to be done as a user with sudo privileges

As it interacts with systemd.

For that, create the following systemd service file (/etc/systemd/system/supervisord-brezel.service) to start and manage our supervisor programms:

Make sure to replace the path to your instance

Otherwise, your brezel will not work correctly!

/etc/systemd/system/supervisord-brezel.service

[Unit]
Description=Run supervisord with the programs needed for the brezel instance on this server
Documentation=https://docs.brezel.io
After=network.target

[Service]
WorkingDirectory=/var/www/vhosts/api.brezel.example.io
ExecStart=/usr/bin/supervisord -c supervisord.conf -n
ExecStartPost=/bin/echo "[systemd] supervisord started at $(date)" >> storage/logs/supervisor.log
ExecReload=/usr/bin/supervisorctl -c supervisord.conf reload
ExecStop=/usr/bin/supervisorctl -c supervisord.conf stop all
ExecStopPost=/bin/echo "[systemd] supervisord stopped at $(date)" >> storage/logs/supervisor.log
KillMode=mixed
Restart=on-failure
RestartSec=42s
User=brezel
UMask=0002

[Install]
WantedBy=multi-user.target

Now run the following commands to enable and start the supervisord service:

sudo systemctl daemon-reload
sudo systemctl enable supervisord-brezel
sudo systemctl start supervisord-brezel

To check if all services are up and running, switch back to the brezel user, go into the instance directory and run:

supervisorctl status

Interacting with the supervisor service

Do not interact with supervisorctl directly more that necessary

Restarting using the supervisorctl restart command or other reloading actions are not recommended as it can mess up permissions if done via the wrong user.

If you want to interact with it, make sure you use the brezel user, but even then, no guarantees.

If you want to restart the supervisor service, do so using the following command:

sudo systemctl restart supervisord-brezel

Stopping the service is done with:

sudo systemctl stop supervisord-brezel

Seeing the status of the services can be done with:

systemctl status supervisord-brezel

Note

This is the only valid usecase to use supervisorctl directly. Run supervisorctl status in the instance directory to see the status of the services.

This can also be done with the brezel user.

And starting as defined above:

sudo systemctl start supervisord-brezel