version-3.6.x
Walkthrough
Advanced Usage
Installing a Package
Searching for packages
As explained in "Uploading a Package", packages are managed using registries. There is a one local registry on your machine, and potentially many remote registries elsewhere "in the world". Use
list_packages to see the packages available on a registry:1
import quilt3
2
​
3
# list local packages
4
list(quilt3.list_packages())
Copied!
1
['aneesh/cli-push',
2
'examples/hurdat',
3
'aleksey/hurdat']
Copied!
1
# list remote packages
2
list(quilt3.list_packages("s3://quilt-example"))
Copied!
1
['aleksey/hurdat',
2
'examples/hurdat',
3
'quilt/altair',
4
'quilt/hurdat',
5
'quilt/open_fruit',
6
'quilt/open_images']
Copied!
Installing a package
To make a remote package and all of its data available locally,
install it.1
quilt3.Package.install(
2
"examples/hurdat",
3
"s3://quilt-example",
4
)
Copied!
1
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 7049.25entries/s]
2
​
3
Successfully installed package 'examples/hurdat', tophash=f8d1478 from s3://quilt-example
4
​
Copied!
Note that unless this registry is public, you will need to be logged into a user who has read access to this registry in order to install from it:
1
# only need to run this once
2
# ie quilt3.config('https://your-catalog-homepage/')
3
quilt3.config('https://open.quiltdata.com/')
4
​
5
# follow the instructions to finish login
6
quilt3.login()
Copied!
Data files that you download are written to a folder in your local registry by default. You can specify an alternative destination using dest:
1
quilt3.Package.install(
2
"examples/hurdat",
3
"s3://quilt-example",
4
dest="./"
5
)
Copied!
1
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 9027.77entries/s]
2
Copying objects: 100%|██████████| 3.62M/3.62M [00:00<00:00, 303MB/s]
3
​
4
Successfully installed package 'examples/hurdat', tophash=f8d1478 from s3://quilt-example
5
​
Copied!
Finally, you can install a specific version of a package by specifying the corresponding top hash:
1
quilt3.Package.install(
2
"examples/hurdat",
3
"s3://quilt-example",
4
top_hash="058e62c"
5
)
Copied!
1
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 11491.24entries/s]
2
Copying objects: 100%|██████████| 35.4k/35.4k [00:02<00:00, 14.3kB/s]
3
​
4
Successfully installed package 'examples/hurdat', tophash=058e62c from s3://quilt-example
5
​
Copied!
Browsing a package manifest
An alternative to
install is browse. browse downloads a package manifest without also downloading the data in the package.1
# load a package manifest from a remote registry
2
p = quilt3.Package.browse("examples/hurdat", "s3://quilt-example")
3
​
4
# load a package manifest from the default remote registry
5
quilt3.config(default_remote_registry="s3://quilt-example")
6
p = quilt3.Package.browse("examples/hurdat")
Copied!
1
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 7541.00entries/s]
2
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 10710.68entries/s]
Copied!
browse is advantageous when you don't want to download everything in a package at once. For example if you just want to look at a package's metadata.Importing a package
You can import a local package from within Python:
1
from quilt3.data.examples import hurdat
Copied!
1
Loading manifest: 100%|██████████| 5/5 [00:00<00:00, 9637.65entries/s]
Copied!
This allows you to manage your data and code dependencies all in one place in your Python scripts or Jupyter notebooks.
Last modified 1mo ago