Set up a development environment
Ce contenu n’est pas encore disponible dans votre langue.
This tutorial runs madbot from source on your machine: the API with its services (Vault, Redis, Celery), then the web client. At the end, you can sign in to your local madbot at http://localhost:3000.
Before you start
Section titled “Before you start”Install:
- Git;
- pixi, which installs Python and every dependency of the API;
- Node.js 22.23.4, the version required by the client, with Corepack, which provides the right version of pnpm.
# macOS and Linuxcurl -fsSL https://pixi.sh/install.sh | sh
# Windowspowershell -ExecutionPolicy ByPass -c "irm -useb https://pixi.sh/install.ps1 | iex"Docker is only needed for the optional parts: debugging emails with Mailhog and the Docker Compose stack.
Get the code
Section titled “Get the code”git clone https://gitlab.com/ifb-elixirfr/madbot/madbot.gitcd madbotThe repository holds the API (api/), the web client (client/), the plugins_api package (plugin_api/) and this documentation (documentation/).
Set up the API
Section titled “Set up the API”Install the dependencies
Section titled “Install the dependencies”cd apipixi installpixi install -e devThe default environment holds what madbot needs to run; the dev environment adds the test, lint and security tools. This also installs plugins-api from the package registry: you only need the plugin_api/ folder to change plugins_api itself.
Run commands with pixi
Section titled “Run commands with pixi”Run a madbot command with pixi run madbot <command>, from api/:
pixi run madbot migrateOr open a shell in the environment and call madbot directly:
pixi shell -e devmadbot migrateThe project also defines shortcuts such as pixi run test or pixi run lint: see Tests and quality checks.
Configure the API
Section titled “Configure the API”Copy the example configuration:
cp .env.example .env.env.example lists every setting with an example value and explains the syntax the file must follow. For a first run, fill in:
-
SECRET_KEY: required, and the same for the API and the Celery workers. madbot refuses to start with an empty or placeholder value. Generate one:Terminal window python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())" -
MADBOT_OIDC_RSA_PRIVATE_KEY: the private key with which madbot signs its OpenID Connect tokens, on a single line. Generate it and copy it:Terminal window openssl genrsa -out oidc.key 4096# macOSawk -v ORS='\\n' '1' oidc.key | pbcopy# Linuxawk -v ORS='\\n' '1' oidc.key | xclip -selection clipboard -
PUBLIC_BACKEND_DOMAIN: the address of the API, ashost:portwithout the scheme (defaultlocalhost:8000).
Start Vault
Section titled “Start Vault”madbot keeps the credentials of the connections in Vault. In a new terminal, from api/:
pixi run vaultThe script starts a local Vault and prints the settings madbot needs:
MADBOT_VAULT_URL=http://localhost:8200MADBOT_VAULT_MOUNT_POINT=madbotMADBOT_VAULT_ROLEID=<role id>MADBOT_VAULT_SECRETID=<secret id>Copy them into api/.env. Keep the terminal open.
Start Redis
Section titled “Start Redis”Redis carries the Celery tasks and the WebSocket messages. In a new terminal, from api/:
pixi run redispixi run redis uses REDIS_PORT from api/.env (default 6379).
Start Meilisearch (optional)
Section titled “Start Meilisearch (optional)”The global workspace search is backed by Meilisearch. It is off by default (SEARCH_ENABLED=false): no indexing task is dispatched, /api/search answers 503 and the client hides its entry point. Skip this section unless you work on the search itself.
Meilisearch ships as a single binary (the meilisearch package on conda-forge is stale, so it is not a pixi dependency). In a new terminal, from api/, install it with the official script and start it:
curl -L https://install.meilisearch.com | sh./meilisearch --master-key="<a-dev-master-key>"Running from api/ keeps the binary and the data.ms/ data directory where api/.gitignore already ignores them.
By default it listens on http://localhost:7700. The master key is a secret you choose (Meilisearch recommends at least 16 bytes). Locally, reuse that same value as MEILISEARCH_API_KEY.
In api/.env:
SEARCH_ENABLED=trueMEILISEARCH_HOST=http://localhost:7700MEILISEARCH_API_KEY=<the master key used above>Enabling the search also registers a Meilisearch check on /health-check as a non-critical service: an unreachable Meilisearch shows as degraded there without failing the endpoint.
You build the index once the database is created (see below). Day-to-day updates then go through Celery, so keep a worker running (pixi run madbot celery). Whenever you turn SEARCH_ENABLED back on later, run pixi run madbot rebuild_search_index again: nothing written while the search was off is indexed automatically.
To enable the search on a deployed instance, see Enable the global search.
Create the database
Section titled “Create the database”Back in the first terminal, from api/:
pixi run madbot migratepixi run madbot collectstaticpixi run madbot initialise_madbot_metadata_fieldspixi run madbot create_periodic_taskspixi run madbot reload_pluginsIf you enabled the search, build its index:
pixi run madbot rebuild_search_indexThe management commands reference explains each madbot command.
Create an administrator account. Adapt the username, email, password and names; the ROR identifier of the institution is optional here:
echo "from django.contrib.auth import get_user_model; User = get_user_model(); User.objects.create_superuser('admin', 'admin@example.org', '<password>', id='9da4931b-e89e-48f1-988d-0aeb6a2ea1bd', first_name='Madbot', last_name='Admin', ror='https://ror.org/<ror id>', is_approved=True)" | pixi run madbot shellRegister the web client as an OpenID Connect application of the API:
pixi run madbot createapplication --client-id=madbot --client-secret=madbot --name="madbot" --algorithm=RS256 --redirect-uris=http://localhost:3000/api/auth/callback/madbot --user="9da4931b-e89e-48f1-988d-0aeb6a2ea1bd" public authorization-codeIf the client does not run on port 3000, adapt --redirect-uris, and CORS_ALLOWED_ORIGINS and CSRF_TRUSTED_ORIGINS in api/.env.
Start the Celery workers
Section titled “Start the Celery workers”Celery runs the long tasks: submissions, copies, imports. In a new terminal, from api/:
pixi run madbot celeryWhen DEBUG is on, the worker restarts whenever the code changes. In another terminal, start the scheduler of the periodic tasks:
pixi run madbot beatTo follow the tasks in a web interface, start Flower and open http://localhost:5555:
pixi run celery --broker=redis://localhost flower --port=5555 --basic_auth=$CELERY_FLOWER_OPTIONSStart the API
Section titled “Start the API”pixi run runserverThe API answers at the address of PUBLIC_BACKEND_DOMAIN, http://localhost:8000 by default. Its interactive reference is at http://localhost:8000/api/schema/swagger-ui/.
Install plugins
Section titled “Install plugins”madbot alone has no source to link data from: plugins bring them. List the plugins to use in api/.env:
MADBOT_INSTALLED_PLUGINS=madbot_sshfs,madbot_zenodoThen install them, from api/:
pixi run install-pluginsThe script has two modes:
- editable (default, or
-e): it clones each plugin intoMADBOT_PLUGINS_DIR(defaultmadbot-plugins, a path relative to the folder that holdsmadbot/) and installs it withpip install -e. Your changes to the plugin code apply without reinstalling. When a plugin is already cloned, the script updates it withgit pullif it is onmainordevwithout local changes, and leaves it as it is otherwise;--no-update-checkskips this. - package (
-p, orpixi run install-plugins-package): it installs each plugin from its Git repository, as a regular package.
Add -y to skip the confirmation. Then reload the plugins and check what is installed:
pixi run madbot reload_pluginspixi run list-pluginsThe plugins reference lists the available plugins.
To install a plugin by hand, install it in both environments:
pixi run -e default pip install -e /path/to/pluginpixi run -e dev pip install -e /path/to/pluginSet up the client
Section titled “Set up the client”In a new terminal, from the root of the repository:
cd clientnode -vThe client requires Node.js 22.23.4. Then install the dependencies, with the version of pnpm pinned by the project:
corepack enablepnpm installIf the dependencies look corrupted, pnpm clean removes node_modules; run pnpm install again.
Configure the client
Section titled “Configure the client”cp .env.example .envGenerate NUXT_SECRET:
openssl rand -base64 48 | tr -dc 'A-Za-z0-9' | head -c 32; echoAUTH_ORIGIN and NEXTAUTH_URL are the full address of the authentication routes of the client, for example http://localhost:3000/api/auth. Keep them in line with NITRO_PORT: without NEXTAUTH_URL, the sign-in returns to port 3000.
Start the client
Section titled “Start the client”pnpm devOpen http://localhost:3000 (or the port of NITRO_PORT) and sign in with the administrator account.
When something does not start
Section titled “When something does not start”| Symptom | Check |
|---|---|
The API refuses to start and mentions SECRET_KEY | SECRET_KEY is set in api/.env, and is not a placeholder. |
| Errors about Vault | The terminal of pixi run vault is still open, and its settings are in api/.env. |
| Errors about Redis, tasks that never run | Redis and the Celery worker are running. |
| The client stays on “Loading madbot…” | The API answers, and its migrations are applied: pixi run madbot migrate. |
| A plugin is missing | It is listed in MADBOT_INSTALLED_PLUGINS, installed, and reload_plugins ran after the installation. |

