Data Enrichment

This is the developers' documentation about the data enrichment component in PISTIS factories.

Introduction

Data Enrichment service is responsible to map data schema of raw datasets into PISTIS Data Model.

This service can be authenticated using the PISTIS Credentials.

Access

Backend API: https://{FACTORYNAME}.pistis-market.eu/srv/data-enrichment-backend/ Frontend: Frontend can be triggered from the distributions in the Factory Catalogue UI

Backend endpoints

The backend endpoints of this service is built for serving requests that is required by the GUI. It has endpoints to display a dataset, perform a livesearch of the data model properties and to save the enriched dataset to the Factory Data Storage along with creating a new distribution in the Factory Data Catalogue. The newly created distribution will have all the basic metadata as well as the the data model that was selected by the user.

Automated execution

Once a user has completed the enrichment process by selecting the required properties from the PISTIS data model and saving it as a new distribution in the Factory Data Catalogue, it is possible to automate this process for new datasets that can also have the exact same properties. This is done by fetching the data model properties from the already saved distribution metadata from the Factory Data Catalogue. The API description to perform this operation is,