quilt3.Bucket
Bucket interface for Quilt.
__init__
Creates a Bucket object.
Arguments
- bucket_uri(str): URI of bucket to target. Must start with 's3://'
Returns
A new Bucket
Execute a search against the configured search endpoint.
Arguments
- query (str): query string to search
- limit (number): maximum number of results to return. Defaults to 10
Query Syntax: By default, a normal plaintext search will be executed over the query string. You can use field-match syntax to filter on exact matches for fields in your metadata. The syntax for field match is
user_meta.$field_name:"exact_match"
.Returns
a list of objects with the following structure:
[{
`"key"`: <key of the object>,
`"version_id"`: <version_id of object version>,
`"operation"`: <"Create" or "Delete">,
`"meta"`: <metadata attached to object>,
`"size"`: <size of object in bytes>,
`"text"`: <indexed text of object>,
`"source"`: <source document for object (what is actually stored in ElasticSeach)>,
`"time"`: <timestamp for operation>,
}...]
Stores file at path to key in bucket.
Arguments
- key(str): key in bucket to store file at
- path(str): string representing local path to file
Returns
None
Raises
- if no file exists at path
- if copy fails
Stores all files in the
directory
under the prefix key
.Arguments
- key(str): prefix to store files under in bucket
- directory(str): path to directory to grab files from
Returns
None
Raises
- if writing to bucket fails
Lists all keys in the bucket.
Returns
List of strings
Deletes a key from the bucket.
Arguments
- key(str): key to delete
Returns
None
Raises
- if delete fails
Delete a directory and all of its contents from the bucket.
Arguments
- path (str): path to the directory to delete
List data from the specified path.
Arguments
- path (str): bucket path to list
- recursive (bool): show subdirectories and their contents as well
Returns
list
: Return value structure has not yet been permanently decided Currently, it's a tuple
of list
objects, containing the following
: (directory info, file/object info, delete markers).Fetches file (or files) at
key
to path
.If
key
ends in '/', then all files with the prefix key
will match and will be stored in a directory at path
.Otherwise, only one file will be fetched and it will be stored at
path
.Arguments
- key(str): key in bucket to fetch
- path(str): path in local filesystem to store file or files fetched
Returns
None
Raises
- if path doesn't exist
- if download fails
Selects data from an S3 object.
Arguments
- key(str): key to query in bucket
- query(str): query to execute (SQL by default)
- query_type(str): other query type accepted by S3 service
- raw(bool): return the raw (but parsed) response
Returns
pandas.DataFrame
: results of queryLast modified 1yr ago