quilt
dev
homecode
Search…
⌃K
Links
dev
Introduction
Installation
Quick start
Mental model
Architecture
Walkthrough
Editing a Package
Uploading a Package
Installing a Package
Getting Data from a Package
Working with the Catalog
Working with a Bucket
Working with ElasticSearch
API Reference
quilt3
quilt3.Package
quilt3.Bucket
CLI, environment
Known limitations
Catalog
Admin UI
Configuration
Embed
Metadata for teams
Preview
Search & query
Visualization & dashboards
Local Development Mode
Examples
Git-like operations for datasets and Jupyter notebooks
Nextflow
Advanced
Filtering a Package
.quiltignore
Materialization
Working with Manifests
S3 Select
Workflows
Enterprise install
S3 Events, EventBridge
Cross-account access
Querying Metadata with Athena
S3 Bucket Organization
GxP & Quilt
Private Endpoint Access
Restricting access to specific prefixes in a S3 bucket
More
Frequently Asked Questions
Troubleshooting
Contributing
Changelog
Powered By GitBook

Architecture

Quilt packages unify data and metadata

Quilt data packages are reproducible, discoverable, and trusted data sets that live in blob storage.
A package manifest includes both metadata and physical keys to primary data.
All package metadata and data are stored in your S3 buckets. A slice of the package-level metadata, as well as S3 object contents, are sent to an ElasticSearch cluster managed by Quilt. All Quilt package manifests are accessible via SQL using AWS Athena.
Previous
Mental model
Next - Walkthrough
Editing a Package
Last modified 5mo ago
Copy link