Architecture
madbot is a web client that talks to a REST API. The API keeps its data in a database, delegates the long operations to Celery workers, pushes live events over WebSockets, and reaches the outside world through plugins.
Browser │ ┌────────▼─────────┐ │ Web client │ Nuxt, Vue, Vuetify, Pinia └────────┬─────────┘ │ REST (/api/), WebSocket (/ws/) ┌────────▼─────────┐ ┌───────────────┐ │ API │◀──────▶│ Redis │◀──┐ │ Django, DRF, │ └───────────────┘ │ tasks │ Channels │ │ └──┬─────┬─────┬───┘ ┌───────────────┐ │ │ │ │ │ Celery worker │───┘ │ │ │ │ and beat │ │ │ │ └───────┬───────┘ │ │ └──── plugins ───────┤ │ │ (sources and repositories) │ └── Vault (connection credentials) └──── Database (SQLite in development, PostgreSQL in production)The repository
Section titled “The repository”The repository holds three packages, released separately, and this documentation:
| Folder | What it holds |
|---|---|
api/ | The API, the Django project madbot_api. |
client/ | The web client, a Nuxt application. |
plugin_api/ | The plugins_api package: the contract that plugins implement, installed by the API as plugins-api. |
documentation/ | This site. |
The API
Section titled “The API”The API is a Django application built on Django REST Framework. Its domain logic lives in the madbot_api.core app; analytics have their own app, madbot_api.analytics.
From the request to the database
Section titled “From the request to the database”| Layer | Where | Role |
|---|---|---|
| Routing | madbot_api/urls.py, core/url_api.py | Mounts the REST API under api/, OAuth under o/, the health checks; nested routers follow the resources (workspaces → members, nodes → samples, submissions → metadata objects…). |
| Views | core/views/ | ViewSets and API views. Most derive from MadbotViewSet (core/views/utils/drf.py), which brings authentication, approval of the user, scopes, pagination and error format. |
| Access rules | core/views/utils/mixin.py | Resolve the workspace of the request and apply the workspace, node and submission memberships. |
| Models | core/models.py | The entities: see the data models. |
| Tasks | core/tasks.py | The Celery tasks. |
The client names the workspace of a request in the X-Workspace header.
Authentication
Section titled “Authentication”| Mechanism | Used by |
|---|---|
OAuth 2 with OpenID Connect (django-oauth-toolkit, PKCE) | The web client. madbot signs its tokens with MADBOT_OIDC_RSA_PRIVATE_KEY. |
| Personal access tokens | Scripts and tools that call the API. |
| Sessions | The Django administration. |
The OAuth scopes are read, write, openid, email and profile. An account must be approved before it can use the API.
Long operations and live events
Section titled “Long operations and live events”The API hands the long operations to Celery workers, through Redis: creating the metadata objects of a submission, submitting, checking the connections and the access to data, copying a node and its subtree. Celery Beat runs the periodic tasks (see create_periodic_tasks).
While they run, the API pushes events to the client over WebSockets (Django Channels, served by Daphne):
| Path | Events |
|---|---|
ws/notifications/ | Notifications of the user. |
ws/workspaces/<workspace id>/ | Events of a workspace: connections, data. |
ws/submissions/<submission id>/ | Progress of a submission. |
Plugins
Section titled “Plugins”The API loads the plugins listed in MADBOT_INSTALLED_PLUGINS and adds their Django apps to INSTALLED_APPS. plugins_api defines the base classes they implement: data plugins, submission plugins and metadata field definition plugins. The connection and submission views call the plugin of the connection to browse a source, check credentials or send a submission. The credentials of the connections are kept in Vault, not in the database.
See the plugin development guide.
The web client
Section titled “The web client”The client is a Nuxt application, rendered in the browser only (ssr: false), written with Vue’s Composition API, Vuetify components and Pinia stores.
| Folder | Role |
|---|---|
pages/ | Routes, by file. The workspace pages are under pages/[w]/[wslug]/. |
components/ | Components, by domain: Workspace/, Node/, Connection/, DataObject/, Sample/, Submission/, Metadata/… |
stores/ | One Pinia store per domain: workspaces, nodes, connections, samples, metadata, submissions, notifications, WebSockets. |
composables/ | Shared logic, including the calls to the API and the WebSockets. |
server/ | Server routes, for the authentication. |
locales/ | The interface texts, in English and French. |
- Calls to the API go through
useAPIFetch, which adds the access token of the session and sends the user to a waiting page when the API does not answer. - Authentication uses
@sidebase/nuxt-auth: the client signs in with the OpenID Connect provider of the API. - Metadata forms are generated from the JSON schemas of the fields with JSON Forms, and madbot’s own renderers in
components/JsonFormRenderers/. - WebSockets are opened by
MadbotWebSocket, managed per workspace byuseWebsocketStore.
Where to make a change
Section titled “Where to make a change”| Change | Where |
|---|---|
| A model | api/madbot_api/core/models.py, then pixi run madbot makemigrations core. Use a UUID primary key, the audit fields and, where history matters, HistoricalRecords. |
| An endpoint | A view in core/views/, its route in core/url_api.py, following the existing filtering, pagination and permissions. |
| A long operation | A task in core/tasks.py. |
| A screen | A page in client/pages/, components in client/components/, the state in a store. |
| An interface text | client/locales/en.json and fr.json. |
| A connection to another system | A plugin, in its own repository. |
Write the tests with the change: see Tests and quality checks.

