Easy-to-use interface

Iterator

class oss2.BucketIterator(service, prefix='', marker='', max_keys=100, max_retries=None)[source]

Iterator for bucket.

Returns a SimplifiedBucketInfo instance for each iteration (via next()).

Parameters:
  • service – Service instance .
  • prefix – Bucket name prefix. Only buckets with the prefix specified are listed.
  • marker – Paging marker. Only lists buckets whose name is after the marker in the lexicographic order.
  • max_keys – The maximum number of keys to return for list_objects . Note that the total entries the iterator returns can be more than that.
class oss2.ObjectIterator(bucket, prefix='', delimiter='', marker='', max_keys=100, max_retries=None)[source]

Iterator for files in a bucket.

Returns a SimplifiedObjectInfo instance for each iteration (via next()). When SimplifiedObjectInfo.is_prefix() is True, the object is a common prefix (directory, not a file); Otherwise, it is a file.

Parameters:
  • bucket – Bucket instance.
  • prefix – The file name prefix.
  • delimiter – Delimiter for the directory.
  • marker – Paging marker.
  • max_keys – The maximum number of keys to return for each list_objects call. Note that the total entries that the iterator returns can be more than this value.
class oss2.MultipartUploadIterator(bucket, prefix='', delimiter='', key_marker='', upload_id_marker='', max_uploads=1000, max_retries=None)[source]

Iterator of ongoing parts in multiparts upload.

Returns a MultipartUploadInfo instance for each iteration. When MultipartUploadInfo.is_prefix() is True, the instance is a folder. Otherwise it is a file.

Parameters:
  • bucket – Bucket instance.
  • prefix – The file key prefix. Only parts of those files will be listed.
  • delimiter – The directory delimeter.
  • key_marker – Paging marker.
  • upload_id_marker – Paging upload ID marker.
  • max_uploads – Maximum entries for each list_multipart_uploads call. Note that the total entries that the iterator returns can be more than this value.
class oss2.ObjectUploadIterator(bucket, key, max_uploads=1000, max_retries=None)[source]

Iterator of ongoing multiparts uploads.

It returns a MultipartUploadInfo instance for each iteration. When MultipartUploadInfo.is_prefix() is true, it means the common prefix (folder).

Parameters:
  • bucket – Bucket instance.
  • key – The object key.
  • max_uploads – Max entries for each list_multipart_uploads call. Note that the total entries the iterator returns can be more than this value.
class oss2.PartIterator(bucket, key, upload_id, marker='0', max_parts=1000, max_retries=None)[source]

Iterator of uploaded parts of a specific multipart upload.

It returns a PartInfo instance for each iteration.

Parameters:
  • bucket – Bucket instance.
  • key – Object key.
  • upload_id – Upload ID.
  • marker – Paging marker.
  • max_parts – The max parts for each list_parts call. Note that the total entries the iterator returns can be more than this value.

Multipart (upload/download)

oss2.resumable_upload(bucket, key, filename, store=None, headers=None, multipart_threshold=None, part_size=None, progress_callback=None, num_threads=None)[source]

Uses multipart upload to upload a local file.

The oss2.defaults.multipart_num_threads is used as the default parallel thread count. The upload progress is saved to the local disk. If the upload is interrupted, and a new upload is started with the same local file and destination, the upload will resume from where the last upload stopped based on the checkpoint file. This function saves the upload progress to the HOME folder on the local disk by default.

Note

  1. If CryptoBucket is used, the function degrades to a normal upload operation.
Parameters:
  • bucket – Bucket instance.
  • key – The OSS object key.
  • filename – The name of the local file to be uploaded.
  • store – Upload progress information storage. ResumableStore is used if not specified. See ResumableStore for more information.
  • headers – HTTP headers for put_object or init_multipart_upload.
  • multipart_threshold – Files exceeding this size will be uploaded with multipart upload.
  • part_size – Part size. The value is calculated automatically if not specified.
  • progress_callback – The progress callback. See ref:progress_callback for more information.
  • num_threads – Upload parallel thread count. oss2.defaults.multipart_num_threads will be used if not specified.
oss2.resumable_download(bucket, key, filename, multiget_threshold=None, part_size=None, progress_callback=None, num_threads=None, store=None)[source]

Resumable download.

The implementation :
  1. Create a temporary file with the same original file name plus a random suffix.
  2. Download the OSS file with specified Range into the temporary file.
  3. Once the download is finished, rename the temp file with the target file name.

During a download, the checkpoint information (finished range) is stored in the disk as a checkpoint file.

If the download is interrupted, the download can resume from the checkpoint file if the source and target files match. Only the missing parts will be downloaded.

By default, the checkpoint file is in the Home subfolder. The subfolder for storing the checkpoint file can be specified through the store parameter.

Note

  1. For the same source and target file, at any given time, there should be only one running instance of this API. Otherwise multiple calls could lead to checkpoint files overwriting each other.
  2. Don’t use a small part size. The suggested size is no less than oss2.defaults.multiget_part_size.
  3. The API will overwrite the target file if it exists already.
  4. If CryptoBucket is used, the function will become a normal download.
Parameters:
  • bucket – Bucket instance.
  • key (str) – OSS key object.
  • filename (str) – Local file name.
  • multiget_threshold (int) – The threshold of the file size to use multiget download.
  • part_size (int) – The preferred part size. The actual part size might be slightly different according to determine_part_size().
  • progress_callback – Progress callback. See Upload or Download Progress for more information.
  • num_threads – Parallel thread number. The default value is oss2.defaults.multiget_num_threads.
  • store (ResumableDownloadStore) – Specifies the persistent storage for checkpoint information. For example, the folder of the checkpoint file.
Raises:

If the source OSS file does not exist NotFound is thrown . Other exceptions may be thrown due to other issues.

FileObject adapter

class oss2.SizedFileAdapter(file_object, size)[source]

This adapter guarantees reading of data up to only the specified size, even if the original file_object size is larger.