Automated Ingestion of test files can be setup for your PHC project.
Automated Ingestion Process¶
The automating ingestion process consists of uploading sequencing result files to a PHC Project via SFTP, File Service, cli or Jupyter Notebooks. The uploaded file triggers a pre-configured File Action that will run the vendor conversion CWL workflow to create the Omics files. One of these output files is a test manifest file which will start the ingestion of the Omics data.
You must have
Administer Rules access for the PHC Project that will be receiving the files to perform these steps.
- Generate a CWL workflow and any required steps in the workflow to process the test data. The workflow steps should consist of Docker Containers.
- The completed workflow should be uploaded to the PHC Tool Registry Service.
- The individual Docker Containers (steps) may be uploaded to the PHC Tool Registry Service or any publicly available Container repository, i.e. DockerHub.
- PHC currently supports (has the workflows and steps already generated) the following vendors Ashion, Caris, Foundation Medicine and NantOmics.
- Create a File Action that triggers the workflow you had added to the PHC Tool Registry Service. This is done by uploading a file or set of files by following the directions found in the File Action article.
- The configuration can be verified by simply uploading a vendor file to the specified directory that matches the file pattern used in the File Action creation. The upload will trigger the workflow which can be monitored from here Automation View. Once the workflow completes, the test indexing progress can be monitored from the Omics Dashboard.