Skip to content

Tests and quality checks

The CI runs these checks on every merge request that touches the code they cover. Run them on your machine first: a failure is quicker to fix before the push. The CI reference lists the jobs.

The repository defines pre-commit hooks that run most of the checks below on the files of each commit: migrations, ruff, ESLint, the icon map, pixi.lock, safety and bandit. With pre-commit installed, set up the hooks once, from the root of the repository:

Terminal window
pre-commit install

From api/:

Terminal window
# The whole suite, in parallel and in a random order
pixi run test
# One module
pixi run test madbot_api.core.tests.functional.workspaces
# In series, when a parallel failure is hard to read
pixi run test-serial

The tests run in the dev environment, on an in-memory SQLite database. A production instance runs PostgreSQL: check raw SQL on PostgreSQL by hand.

Terminal window
# Run the suite under coverage, then compare with the threshold
pixi run test-cov && pixi run cov-report
# Line-by-line report in htmlcov/
pixi run test-cov && pixi run cov-html

The CI fails when the coverage drops below the threshold held in api/pyproject.toml ([tool.coverage.report] fail_under). The threshold only goes up. api/docs/test-coverage-plan.md describes the measured perimeter, the exclusions and the state of the effort.

Terminal window
pixi run lint
pixi run format
pixi run format-check

format rewrites the files; format-check only reports, as the CI does. Run both lint and format-check: they are different checks, and the CI runs both.

Terminal window
# Known vulnerabilities in the dependencies
pixi run safety
# Insecure patterns in the code
pixi run bandit

pixi run quality-check runs lint, format-check, bandit and safety in one go.

plugin_api/ has its own pixi project and the same tasks. From plugin_api/:

Terminal window
pixi run lint
pixi run format-check
pixi run safety
pixi run bandit

From client/:

Terminal window
# Unit tests (Jest)
pnpm test
# End-to-end tests (Playwright), with or without the interface
pnpm e2e
pnpm e2e:ui
# Lint, with or without the automatic fixes
pnpm lint
pnpm lint:fix
# Check that the icon map lists every icon in use
pnpm icons:check
# Known vulnerabilities in the dependencies
pnpm audit --audit-level=high

The client shows Material Design icons from a generated list, client/utils/mdiIcons.ts. An mdi-* icon missing from it is shown as a question mark: after adding an icon, run pnpm icons and commit the regenerated file.

pnpm-workspace.yaml (in client/, at the root and in documentation/) sets pnpm’s supply-chain protections. Keep the copies in line.

  • Release quarantine (minimumReleaseAge): a version published less than a day ago is not installed. For an urgent security fix, add a commented minimumReleaseAgeExclude entry.
  • Trust policy (trustPolicy: no-downgrade): an install fails when the trust evidence of a package weakens compared with its earlier releases.
  • Build scripts allow-list (allowBuilds, strictDepBuilds): the install scripts of dependencies only run for the packages listed. When you add a dependency that needs one, approve it with pnpm approve-builds and commit the configuration.
  • No exotic sub-dependencies (blockExoticSubdeps): transitive dependencies come from the registry, not from Git or tarball URLs.

Never use dangerouslyAllowAllBuilds.

From documentation/:

Terminal window
pnpm build

The build fails on a broken internal link. See Write documentation.