# plugins_api.data_plugins

*module*

Data plugins.

This module contains plugin classes for data management operations.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L1-1)

## plugins_api.data_plugins.DEFAULT_BATCH_READ_SIZE

*attribute* · *module attribute*

```python
DEFAULT_BATCH_READ_SIZE = 100
```

Default maximum number of ids madbot sends per batch-read call.

Plugins can override ``DataPlugin.get_batch_read_size`` when their source has
a different limit, or return ``None`` to disable chunking.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L18-18)

## plugins_api.data_plugins.DataPlugin

*class*

```python
class DataPlugin(ConnectionPlugin)
```

Bases: `plugins_api.base.ConnectionPlugin`

Abstract class defining the minimum API for a data plugin.

Data plugins connect to external data sources and provide access to files and directories.
They are used to browse, list, and download data from external systems.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L32-383)

### plugins_api.data_plugins.DataPlugin.browse_data_objects

*method* · *abstract*

```python
def browse_data_objects(parent_id: str | None) -> list[PluginDataObject]
```

Return the direct children of a parent data object.

**Parameters**

- `parent_id` (`str or None`) — ID of the parent object. When ``None``, browse from the root of the source (same as passing ``get_root_id()``).

**Returns**

- (`List[PluginDataObject]`) — Direct children of ``parent_id``, or of the root when ``parent_id`` is ``None``.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L129-147)

### plugins_api.data_plugins.DataPlugin.download_data

*method* · *abstract*

```python
def download_data(external_dataobject_id: str, destination_path: str) -> str
```

Download a data object to local storage and returns an download report.

It is **strongly recommended** to implement data integrity verification in this method.
This may include, but is not limited to:
- Checksum validation (e.g., MD5, SHA256) if available.
- File size comparison with the source.
- Any other consistency checks relevant to the data source.

Implementing integrity checks helps prevent data corruption or incomplete downloads,
ensuring reliable processing of the imported data.

**Parameters**

- `external_dataobject_id` (`str`) — Unique identifier of the data object in the external system.
- `destination_path` (`str`) — Local path where the file should be saved.

**Returns**

- (`str`) — Path to the local file.

**Raises**

- `NotImplementedError` — This method must be implemented in a subclass.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L351-383)

### plugins_api.data_plugins.DataPlugin.get_batch_read_size

*method* · *classmethod*

```python
def get_batch_read_size() -> int | None
```

Return how many ids madbot should send per batch-read call.

Madbot uses this when calling ``get_data_objects_by_ids``,
``get_metadata_for_data_objects`` and
``get_data_associations_for_data_objects`` for a long list of ids
on one connection.

**Returns**

- (`int or None`) — A positive integer to split the id list into chunks of that size, or ``None`` to pass the full list in a single call. The default is :data:`DEFAULT_BATCH_READ_SIZE` (100).

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L110-127)

### plugins_api.data_plugins.DataPlugin.get_data_associations_for_data_objects

*method*

```python
def get_data_associations_for_data_objects(object_ids: list[str]) -> dict[str, list[PluginDataAssociation] | PluginError]
```

Return data associations for each requested data object.

Each association represents a group of data objects that should be
associated together. A data object can participate in multiple
associations, each returned as a separate PluginDataAssociation.

Associations should be returned bidirectionally - if data objects A,
B, and C are associated together, then when querying A, B, or C, they
should all return the same association group containing all three.

**Parameters**

- `object_ids` (`list[str]`) — IDs of the data objects.

**Returns**

- (`dict[str, list[PluginDataAssociation] | PluginError]`) — Mapping from object id to its association list, or to a per-id ``PluginError``. Returns ``{}`` when ``object_ids`` is empty. The default implementation returns an empty list for each distinct id, making this method optional for plugins that have no associations. Only requested ids may appear in the result.

> **Notes**
>
> Per-id errors follow the same rules as
> ``get_data_objects_by_ids``. Source-level errors must be raised.

**Examples**

If data object A is associated with B and C in one group:

```pycon
>>> {"A": [PluginDataAssociation(associated_external_ids=["A", "B", "C"])]}
```

If data object A has two separate associations (one with B, one with C):

```pycon
>>> {
...     "A": [
...         PluginDataAssociation(associated_external_ids=["A", "B"]),
...         PluginDataAssociation(associated_external_ids=["A", "C"]),
...     ]
... }
```

With metadata:

```pycon
>>> {
...     "A": [
...         PluginDataAssociation(
...             associated_external_ids=["A", "B", "C"],
...             metadata=[
...                 PluginMetadata(
...                     field=MetadataFieldID(
...                         slug="association_type",
...                         source="referential",
...                         name="Association Type",
...                     ),
...                     value=MetadataValue(value="paired"),
...                 )
...             ],
...         )
...     ]
... }
```

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L236-302)

### plugins_api.data_plugins.DataPlugin.get_data_objects_by_ids

*method* · *abstract*

```python
def get_data_objects_by_ids(object_ids: list[str]) -> dict[str, PluginDataObject | PluginError]
```

Return detailed information for each requested data object.

Every distinct id from ``object_ids`` must appear in the result, either
as a ``PluginDataObject`` or as the ``PluginError`` raised while reading
that id (for example ``DataNotFoundError`` or ``PluginPermissionError``).
The result must contain **only** those requested ids. Extras should be
ignored if the remote API returns more.

Errors that affect the whole source (see ``SOURCE_ERRORS``) must be
raised, not stored in the result.

Plugins whose source has no batch API can implement this in one line
with ``_read_each``:

```python
def get_data_objects_by_ids(self, object_ids):
    return self._read_each(object_ids, self._read_data_object)
```

When the source can answer several ids in one request, no need to call
``_read_each``. Instead, call the remote API directly:

```python
def get_data_objects_by_ids(self, object_ids):
    unique_ids = list(dict.fromkeys(object_ids))
    try:
        remote = self.client.fetch_many(unique_ids)
    except ConnectionError as exc:
        raise UnreachableError(self.host) from exc

    result = {}
    for object_id in unique_ids:
        item = remote.get(object_id)
        if item is None:
            result[object_id] = DataNotFoundError(object_id)
        else:
            result[object_id] = self._to_plugin_data_object(item)
    return result
```

**Parameters**

- `object_ids` (`list[str]`) — IDs of the data objects to read. May be empty or contain duplicates.

**Returns**

- (`dict[str, PluginDataObject | PluginError]`) — Mapping from object id to the object or a per-id plugin error. Returns ``{}`` when ``object_ids`` is empty.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L149-206)

### plugins_api.data_plugins.DataPlugin.get_metadata_for_data_objects

*method*

```python
def get_metadata_for_data_objects(object_ids: list[str]) -> dict[str, list[PluginMetadata] | PluginError]
```

Return the metadata for each requested data object.

**Parameters**

- `object_ids` (`list[str]`) — IDs of the data objects.

**Returns**

- (`dict[str, list[PluginMetadata] | PluginError]`) — Mapping from object id to its metadata list, or to a per-id ``PluginError``. Returns ``{}`` when ``object_ids`` is empty. The default implementation returns an empty list for each distinct id, making this method optional for plugins that have no metadata. Only requested ids may appear in the result.

> **Notes**
>
> Per-id errors follow the same rules as
> ``get_data_objects_by_ids``. Source-level errors must be raised.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L208-234)

### plugins_api.data_plugins.DataPlugin.get_root_id

*method* · *abstract* · *staticmethod*

```python
def get_root_id() -> str
```

Return the unique identifier of the root data object.

**Returns**

- (`str`) — Unique identifier of the root data object.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L96-108)

### plugins_api.data_plugins.DataPlugin.get_root_name

*method* · *abstract* · *staticmethod*

```python
def get_root_name() -> str
```

Return the name of the root data object.

**Returns**

- (`str`) — Name of the root data object.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/data_plugins.py#L82-94)

### plugins_api.data_plugins.DataPlugin.close

*method*

```python
def close() -> None
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Release connection resources (optional).

Plugins that hold long-lived connections (e.g. SSH, persistent HTTP
sessions) may override this method to close the underlying connection
and release file descriptors. Callers (e.g. core) should call
``close()`` when done with the plugin instance.

The base implementation does nothing. Plugins that do not need to
release resources need not override it.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L237-249)

### plugins_api.data_plugins.DataPlugin.get_connection_fields

*method* · *abstract* · *classmethod*

```python
def get_connection_fields() -> tuple[list[ConnectionField], list[ConnectionField]]
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Retrieve the connection fields for the connection plugin.

**Returns**

- (`tuple[list[ConnectionField], list[ConnectionField]]`) — ``(shared_fields, private_fields)``. Shared fields are instance-level (URL, host). Private fields are per-user secrets (token, password).

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L206-222)

### plugins_api.data_plugins.DataPlugin.get_instance_name

*method* · *abstract*

```python
def get_instance_name() -> str
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Return the instance name of the connection plugin.

**Returns**

- (`str`) — Instance name.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L193-204)

### plugins_api.data_plugins.DataPlugin.get_shared_parameters

*method* · *abstract*

```python
def get_shared_parameters() -> dict[str, Any]
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Return all shared parameters in a cleaned form.

**Returns**

- (`dict`) — Shared parameters cleaned.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L224-235)

### plugins_api.data_plugins.DataPlugin.is_sandbox

*method* · *classmethod*

```python
def is_sandbox(shared_parameters: dict[str, Any]) -> bool
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Return whether the shared parameters are for a sandbox connection or not.

**Parameters**

- `shared_parameters` (`dict`) — The shared parameters to check.

**Returns**

- (`bool`) — Plugin is sandbox.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L251-268)

### plugins_api.data_plugins.DataPlugin.on_before_store_credentials

*method* · *classmethod*

```python
def on_before_store_credentials(shared_params: dict, private_params: dict) -> dict
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Transform the private parameters just before they are written to the Vault.

Allows the plugin to transform or enrich the private_params.
By default, returns the params unchanged.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L270-282)

### plugins_api.data_plugins.DataPlugin.on_delete_credentials

*method* · *classmethod*

```python
def on_delete_credentials(shared_params: dict, vault_secret: dict) -> None
```

Inherited from: `plugins_api.base.ConnectionPlugin`

Clean up just before the credentials are deleted from the Vault.

Allows the plugin to perform cleanup on the remote server side.
By default, does nothing.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L284-296)

### plugins_api.data_plugins.DataPlugin.get_author

*method* · *abstract* · *classmethod*

```python
def get_author() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return plugin author.

**Returns**

- (`str`) — Plugin author.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L82-94)

### plugins_api.data_plugins.DataPlugin.get_color

*method* · *abstract* · *classmethod*

```python
def get_color() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the color of the plugin.

Typically a hex value such as ``"#3B82F6"``.

**Returns**

- (`str`) — Plugin color.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L113-127)

### plugins_api.data_plugins.DataPlugin.get_description

*method* · *abstract* · *classmethod*

```python
def get_description() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return plugin description.

**Returns**

- (`str`) — Plugin description.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L68-80)

### plugins_api.data_plugins.DataPlugin.get_id

*method* · *classmethod*

```python
def get_id() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the unique identifier of the plugin.

**Returns**

- (`str`) — Unique identifier in the format 'module.class'.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L24-35)

### plugins_api.data_plugins.DataPlugin.get_logo

*method* · *abstract* · *classmethod*

```python
def get_logo() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the static path of the plugin logo.

Path is relative to the plugin package static files.
Example: ``"images/ena.png"``.

**Returns**

- (`str`) — Plugin logo static path.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L96-111)

### plugins_api.data_plugins.DataPlugin.get_name

*method* · *abstract* · *classmethod*

```python
def get_name() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the plugin name.

**Returns**

- (`str`) — Plugin name.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L37-49)

### plugins_api.data_plugins.DataPlugin.get_slug

*method* · *abstract* · *classmethod*

```python
def get_slug() -> str
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the plugin slug.

Lowercase, no spaces (e.g. ``"ena"``, ``"omero"``).
Used as ``MetadataFieldID.source`` and as the schema source at reload.

**Returns**

- (`str`) — Plugin slug.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L51-66)

### plugins_api.data_plugins.DataPlugin.get_status

*method* · *abstract* · *classmethod*

```python
def get_status() -> dict[str, PluginStatus]
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the maturity status of the plugin, keyed by capability type.

A plugin implementing several capability types at once (e.g. a class
inheriting from both `SubmissionPlugin` and
`MetadataFieldDefinitionPlugin`) must return one entry per type it
implements, using the same type strings as `get_types()`.

**Returns**

- (`dict[str, PluginStatus]`) — Mapping of plugin type (as returned by `get_types()`) to its maturity status.

**Examples**

A plugin implementing a single type:

```pycon
>>> print(self.get_status())
{"data_plugin": PluginStatus.PREVIEW}
```

A plugin implementing several types at once:

```pycon
>>> print(self.get_status())
{
    "submission_plugin": PluginStatus.STABLE,
    "metadata_field_definition_plugin": PluginStatus.STABLE,
}
```

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L129-162)

### plugins_api.data_plugins.DataPlugin.get_types

*method* · *classmethod*

```python
def get_types() -> list[str]
```

Inherited from: `plugins_api.base.MadbotPlugin`

Return the list of the plugin types in snake_case.

**Returns**

- (`list`) — List of plugin types.

[View source](https://gitlab.com/ifb-elixirfr/madbot/madbot/-/blob/3aa4c278bf4620ccd23a16eb233b035646016bd2/plugin_api/plugins_api/base.py#L164-181)
