Workflows

New in Quilt 3.3
quilt3 provides an API to ensure the quality of package metadata. To get started, create a configuration file in your registry under .quilt/workflows/config.yml. For example:
version: "1"
workflows:
  alpha:
    name: Search for aliens
    is_message_required: true
  beta:
    name: Studying superpowers
    metadata_schema: superheroes
  gamma:
    name: Nothing special
    description: TOP SECRET
    is_message_required: true
    metadata_schema: top-secret
schemas:
 superheroes:
   url: s3://quilt-sergey-dev-metadata/schemas/superheroes.schema.json
 top-secret:
   url: s3://quilt-sergey-dev-metadata/schemas/top-secret.schema.json
With the above configuration you must specify a workflow before you can push:
>>> import quilt3
>>> quilt3.Package().push('test/package', registry='s3://quilt-sergey-dev-metadata')
​
QuiltException: Workflow required, but none specified.
Let's try with with workflow parameter set:
>>> quilt3.Package().push('test/package', registry='s3://quilt-sergey-dev-metadata', workflow='alpha')
​
QuiltException: Commit message is required by workflow, but none was provided.
This behavior is caused by is_message_required: true option. It requires you specify a message parameter:
>>> quilt3.Package().push(
        'test/package',
        registry='s3://quilt-sergey-dev-metadata',
        message='added info about UFO',
        workflow='alpha')
​
Package test/package@bc9a838 pushed to s3://quilt-sergey-dev-metadata
Now let's push with workflow='beta':
>>> quilt3.Package().push('test/package', registry='s3://quilt-sergey-dev-metadata', workflow='beta')
​
QuiltException: Metadata failed validation: 'superhero' is a required property.
It fails because we have metadata_schema: superheroes specified for beta workflow. It makes package metadata to be validated against JSON Schema located at s3://quilt-sergey-dev-metadata/schemas/superheroes.schema.json.
Here's it that schema:
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://example.com/superheroes.schema.json",
  "properties": {
    "superhero": {
      "enum": [
        "Spider-Man",
        "Superman",
        "Batman"
      ]
    }
  },
  "required": [
    "superhero"
  ]
}
It requires superhero property to be set:
>>> quilt3.Package().set_meta({'superhero': 'Batman'}).push(
        'test/package',
        registry='s3://quilt-sergey-dev-metadata',
        workflow='beta')
​
Package test/package@c4691d8 pushed to s3://quilt-sergey-dev-metadata
Currently there are these limitations is JSON Schema support:
only Draft 7 Json Schemas are supported
schemas with $ref are not supported
schemas must be on S3
For gamma workflow both is_message_required: true and metadata_schema are set, so both message and package metadata are validated:
>>> quilt3.Package().push('test/package', registry='s3://quilt-sergey-dev-metadata', workflow='gamma')
​
QuiltException: Metadata failed validation: 'answer' is a required property.
​
>>> quilt3.Package().set_meta({'answer': 42}).push(
        'test/package',
        registry='s3://quilt-sergey-dev-metadata',
        workflow='gamma')
​
QuiltException: Commit message is required by workflow, but none was provided.
​
>>> quilt3.Package().set_meta({'answer': 42}).push(
        'test/package',
        registry='s3://quilt-sergey-dev-metadata',
        message='at last all is set up',
        workflow='gamma')
​
Package test/package@6331508 pushed to s3://quilt-sergey-dev-metadata
You can make workflow validation optional, you need to set is_workflow_required: false in config and specify workflow=None:
>>> quilt3.Package().push('test/package', registry='s3://quilt-sergey-dev-metadata', workflow=None)
​
Package test/package@06b2815 pushed to s3://quilt-sergey-dev-metadata
Also default_workflow can be set in config to specify which workflow will be used if workflow parameter is not provided.
You could also check JSON Schema that is used for validation of workflow file.
Advanced Usage - Previous
S3 Select
Next - API Reference
quilt3
Last updated 1 week ago