Basic Workflow and App Inputs¶
This tutorial extends the previously created “Hello World” workflow and app to process a single input file. The “One-Step Workflow: Hello World” tutorial must be completed, and its resulting “Hello World!” workflow should be committed to a Git repo prior to beginning this tutorial.
Modify the “Hello World!” App¶
Clone the previous “Hello World!” app from the Git repo (be sure to replace Git repo URL with the appropriate URL for your app):
cd ~/geneflow_work
git clone https://github.com/[USER]/hello-world-gf2.git
Note: if the hello-world-gf2 directory still exists with the contents of the previous app tutorial, do a git pull instead:
cd ~/geneflow_work/hello-world-gf2
git pull
Define an App Input¶
Update the app metadata to reflect the new functionality:
vi ./app.yaml
Change the description field to indicate a change in the app; add a git field (with [USER] replaced with your username, and github.com replaced with the appropriate Git service); update the version field to 0.2:
# Basic app information
# name: standard GeneFlow app name
name: hello-world-gf
# description: short description for the app
description: Updated hello world GeneFlow app
# git: link to the app's git repo
git: https://github.com/[USER]/hello-world-gf2.git
# version: must be incremented every time this file, or any file in the app
# project is modified
version: '0.2'
Next, update the file input in the inputs section of the app.yaml file. Change the label and description of the file input to Input text file. Change the required field to true and add a test_value field with value ${SCRIPT_DIR}/data/file.txt. The test_value field is only used when running the app’s test script. We will define the /data/file.txt file in the next section. ${SCRIPT_DIR} in the context of the test script is the directory in which the test script is located. The updated inputs section of the app.yaml file should look like:
inputs:
file:
label: Input Text File
description: Input text file
type: File
required: true
test_value: ${SCRIPT_DIR}/data/file.txt
Finally, change the execution commands of the app in the exec_methods section.
exec_methods:
- name: environment
if:
- in_path: 'cat'
exec:
- run: cat ${FILE_FULL}
stdout: ${OUTPUT_FULL}
The modified execution block first checks if the cat command is available (it should be available in all standard Linux systems). It then runs the cat command to print the contents of the file passed as the file input. ${FILE_FULL} is a Bash variable that is automatically defined for the file input, and represents the full path of the file input.
Save and close the app.yaml file.
Add a Test File to the App¶
Previously, we added a file called file.txt to the test folder. Modify this file so that it contains the string Hello World File Contents!:
cd ~/geneflow_work/hello-world-gf2/test/data
vi ./file.txt
Make sure the file has the contents:
Hello World File Contents!
Save and close the file.txt file.
Re-“Make” the App¶
Now that the app.yaml file has been updated and the test file defined. Rebuild the app with the following commands:
cd ~/geneflow_work/hello-world-gf2
geneflow make-app .
This should re-generate the agave-app-def.json.j2, hello-world-gf2.sh, and test.sh scripts.
Test the App¶
Run the test.sh script to test the app:
cd ~/geneflow_work/hello-world-gf2/test
./test.sh
You should see output similar to the following:
CMD=/home/[USER]/geneflow_work/hello-world-gf/test/../assets/hello-world-gf.sh --file="/home/[USER]/geneflow_work/hello-world-gf/test/data/file.txt" --output="output.txt" --exec_method="auto"
File: /home/[USER]/geneflow_work/hello-world-gf/test/data/file.txt
Output: output.txt
Execution Method: auto
Detected Execution Method: environment
CMD=cat /home/[USER]/geneflow_work/hello-world-gf/test/data/file.txt >"/home/[USER]/geneflow_work/hello-world-gf/test/output.txt"
Exit code: 0
Exit code: 0
The output.txt file should have been created in the test directory with the text Hello World File Contents!. View it with:
cat ./output.txt
And you should see:
Hello World File Contents!
Update the App README¶
Update the app README.rst file to reflect changes to the app:
cd ~/geneflow_work/hello-world-gf2
vi ./README.rst
Modify the file so it looks like the following:
Hello World! Updated GeneFlow App
=================================
Version: 0.2
This is a basic GeneFlow app with an input.
Inputs
------
1. file: Input text file.
Parameters
----------
1. output: Output text file where "Hello World File Contents!" will be printed.
Commit and Tag the New App¶
Finally, commit the updated app to the Git repo and update its tag to reflect the new version number:
cd ~/geneflow_work/hello-world-gf2
git add -u
git commit -m "update app input"
git tag 0.2
git push --tags origin master
Modify the “Hello World!” Workflow¶
Now that the app has been updated, we can update the “Hello World!” workflow to use the new app.
Checkout the previous “Hello World!” workflow:
cd ~/geneflow_work
git clone https://github.com/[USER]/hello-world-workflow-gf2.git
Note: if the hello-world-workflow-gf2 directory still exists with the contents of the previous workflow tutorial, do a git pull instead:
cd ~/geneflow_work/hello-world-workflow-gf2
git pull
Update Workflow Metadata¶
Update the workflow metadata by incrementing the version number to 0.2:
vi ./workflow.yaml
The metadata section should have the line:
version: '0.2'
Update Apps¶
Update the “Apps” section by incrementing the version:
apps:
hello-world:
git: https://github.com/[USERS]/hello-world-gf2.git
version: '0.2'
Be sure to replace the “git” field with the correct Git repo to which you committed the “Hello World!” app.
Define a Workflow Input¶
Update the inputs section of workflow.yaml so that the file input label and description are correct:
inputs:
file:
label: Input Text File
description: Input text file
type: File
enable: true
visible: true
Pass the Workflow Input to the App¶
The steps section of workflow.yaml should already be defined to take the correct workflow input:
steps:
hello:
app: hello-world
depend: []
template:
file: ${workflow->file}
output: output.txt
Save and close the workflow.yaml file.
Update the Workflow README¶
Update the README.rst file to reflect the new workflow input:
cd ~/geneflow_work/hello-world-workflow-gf2
vi ./README.rst
Modify the file so it looks like the following:
Hello World! One-Step GeneFlow Workflow
=======================================
Version: 0.2
This is a basic one-step GeneFlow workflow that prints the contexts of an input text file to an output text file.
Inputs
------
1. file: Input text file.
Parameters
----------
None
Commit and Tag the New Workflow¶
Commit the new version of the workflow to the Git repo:
cd ~/geneflow_work/hello-world-workflow-gf2
git add -u
git commit -m "updated hello world workflow"
git tag 0.2
git push --tags origin master
Install and Test the Workflow¶
Now that the workflow has been updated and committed to a Git repo, we can install and test it:
cd ~/geneflow_work
geneflow install-workflow -g https://github.com/[USER]/hello-world-workflow-gf2.git -c --make_apps ./test-workflow-0.2
This command installs the updated “Hello World!” one-step workflow, and its updated “Hello World!” app into the directory test-workflow-0.2. Remember to replace the Git URL with the URL to which you committed the workflow.
Test the workflow to validate its functionality:
geneflow run ./test-workflow-2.0 -o output --in.file=./test-workflow-0.2/data/test.txt
This command runs the workflow in the test-workflow-0.2 directory using the test data and copies the output to the output directory.
Once complete, you should see a file called output.txt with the same text as the input test.txt file, Test Hello World!:
cat ./output/geneflow-job-[JOB ID]/hello/output.txt
Be sure to replace [JOB ID] with the ID of the GeneFlow job. The job ID is a randomly generated string and ensures that workflow jobs do not overwrite existing job output. You should see the following text in the output.txt file:
Test Hello World!
Summary¶
Congratulations! You updated the one-step GeneFlow workflow so that it can handle a basic input file. The next tutorial will expand on this workflow by using an app with piped execution.