Lineage Tracker
This is the developers' documentation of the Lineage Tracker component available in PISTIS factories.
Introduction
The Lineage Tracker monitors the evolution of datasets within the PISTIS ecosystem. It records every dataset create/update/delete operation, allowing users to track the lineage and family tree of any dataset. It captures details such as who performed an operation, what changed, and when that change occurred.
The API endpoints can be utilized to document operations, retrieve lineage information, and verify data integrity via blockchain anchoring. Most endpoints require authorization using a Bearer token from the PISTIS Keycloak.
Access
The API of the Lineage Tracker is available through: https://{FACTORYNAME}.pistis-market.eu/srv/lineage-tracker/
Usage
Operation Documentation
Usage for documenting dataset lifecycle events.
| Method | Endpoint | Description |
|---|---|---|
| POST | /create_dataset | Document the creation of a new dataset. |
| POST | /update_dataset | Document an update to an existing dataset. |
| POST | /delete_dataset | Document the deletion of a dataset. |
| POST | /delete_family_tree | Document the deletion of an entire dataset family tree. |
Information Retrieval
Usage for retrieving lineage, history, and statistics.
| Method | Endpoint | Description |
|---|---|---|
| GET | /get_dataset_family_tree | Retrieve the complete family tree of a dataset. |
| GET | /get_dataset_lineage | Retrieve the lineage path from root to a specific dataset. |
| GET | /get_dataset_history | Retrieve the complete operation history of a dataset. |
| GET | /get_dataset_status | Retrieve the last operation performed on a dataset. |
| GET | /get_user_history | Retrieve the history of operations performed by a specific user. |
| GET | /get_node_stats | Retrieve statistics (depth, children count) for a dataset family tree. |
| GET | /get_dataset_num_operations | Get the count of each operation type for a dataset. |
| GET | /get_datasets_diff | Retrieve differences between two datasets. |
| GET | /get_datasets_diff_limit | Retrieve a limited sample of differences between two datasets. |
Lineage Transfer
Usage for moving lineage information between environments (e.g., Factory and Cloud).
| Method | Endpoint | Description |
|---|---|---|
| GET | /read_lineage | content for exporting lineage from root to a specific dataset. |
| POST | /write_lineage | Import/paste dataset lineage into the store. |
Blockchain Integration
Usage for data integrity verification.
| Method | Endpoint | Description |
|---|---|---|
| GET | /blockchain-hash | Retrieve the blockchain hash for a dataset family tree. |
Admin Operations
Administrative endpoints for maintenance.
| Method | Endpoint | Description |
|---|---|---|
| POST | /admin/regenerate_family_tree_descriptions | Regenerate update descriptions for a family tree. |
| POST | /admin/regenerate_blockchain_hash | Regenerate and store the blockchain hash. |
| POST | /admin/purge_dataset_family_tree | Permanently delete a family tree from the triple store. |