Skip to content

Automated Ingestion

Automated Ingestion of test files can be setup for your PHC project.

Automated Ingestion Process

The automating ingestion process consists of uploading sequencing result files to a PHC Project via SFTP, File Service, cli or Jupyter Notebooks. The uploaded file triggers a pre-configured File Action that will run the vendor conversion CWL workflow to create the Omics files. One of these output files is a test manifest file which will start the ingestion of the Omics data.

Steps

Access Control

You must have Administer Rules access for the PHC Project that will be receiving the files to perform these steps.

  1. Generate a CWL workflow and any required steps in the workflow to process the test data. The workflow steps should consist of Docker Containers.
  2. Create a File Action that triggers the workflow you had added to the PHC Tool Registry Service. This is done by uploading a file or set of files by following the directions found in the File Action article.
  3. The configuration can be verified by simply uploading a vendor file to the specified directory that matches the file pattern used in the File Action creation. The upload will trigger the workflow which can be monitored from here Automation View. Once the workflow completes, the test indexing progress can be monitored from the Omics Dashboard.

Last update: 2021-10-04