google.resumable_media.requests.upload module¶
Support for resumable uploads.
Also supported here are simple (media) uploads and multipart uploads that contain both metadata and a small file as payload.
-
class
google.resumable_media.requests.upload.
MultipartUpload
(upload_url, headers=None)¶ Bases:
google.resumable_media.requests._helpers.RequestsMixin
,google.resumable_media._upload.MultipartUpload
Upload a resource with metadata to a Google API.
A multipart upload sends both metadata and the resource in a single (multipart) request.
Parameters: -
upload_url
¶ str – The URL where the content will be uploaded.
-
finished
¶ bool – Flag indicating if the upload has completed.
-
transmit
(transport, data, metadata, content_type)¶ Transmit the resource to be uploaded.
Parameters: - transport (Session) – A
requests
object which can make authenticated requests. - data (bytes) – The resource content to be uploaded.
- metadata (
Mapping
[str
,str
]) – The resource metadata, such as an ACL list. - content_type (str) – The content type of the resource, e.g. a JPEG
image has content type
image/jpeg
.
Returns: The HTTP response returned by
transport
.Return type: - transport (Session) – A
-
-
class
google.resumable_media.requests.upload.
ResumableUpload
(upload_url, chunk_size, headers=None)¶ Bases:
google.resumable_media.requests._helpers.RequestsMixin
,google.resumable_media._upload.ResumableUpload
Initiate and fulfill a resumable upload to a Google API.
A resumable upload sends an initial request with the resource metadata and then gets assigned an upload ID / upload URL to send bytes to. Using the upload URL, the upload is then done in chunks (determined by the user) until all bytes have been uploaded.
When constructing a resumable upload, only the resumable upload URL and the chunk size are required:
>>> from google.resumable_media.requests import ResumableUpload >>> >>> url_template = ( ... u'https://www.googleapis.com/upload/storage/v1/b/{bucket}/o?' ... u'uploadType=resumable') >>> upload_url = url_template.format(bucket=bucket) >>> >>> chunk_size = 3 * 1024 * 1024 # 3MB >>> upload = ResumableUpload(upload_url, chunk_size)
When initiating an upload (via
initiate()
), the caller is expected to pass the resource being uploaded as a file-likestream
. If the size of the resource is explicitly known, it can be passed in directly:>>> import os >>> >>> upload.total_bytes is None True >>> >>> stream = open(filename, u'rb') >>> total_bytes = os.path.getsize(filename) >>> metadata = {u'name': filename} >>> response = upload.initiate( ... transport, stream, metadata, u'text/plain', ... total_bytes=total_bytes) >>> response <Response [200]> >>> >>> upload.total_bytes == total_bytes True
If the stream is in a “final” state (i.e. it won’t have any more bytes written to it), the total number of bytes can be determined implicitly from the
stream
itself:>>> stream = io.BytesIO(data) >>> response = upload.initiate( ... transport, stream, metadata, content_type) >>> >>> upload.total_bytes == len(data) True
If the size of the resource is unknown when the upload is initiated, the
stream_final
argument can be used. This might occur if the resource is being dynamically created on the client (e.g. application logs). To use this argument:>>> response = upload.initiate( ... transport, stream, metadata, content_type, ... stream_final=False) >>> >>> upload.total_bytes is None True
Parameters: - upload_url (str) – The URL where the resumable upload will be initiated.
- chunk_size (int) – The size of each chunk used to upload the resource.
- headers (
Optional
[Mapping
[str
,str
] ]) – Extra headers that should be sent with theinitiate()
request, e.g. headers for encrypted data. These will not be sent withtransmit_next_chunk()
orrecover()
requests.
-
upload_url
¶ str – The URL where the content will be uploaded.
Raises: ValueError
– Ifchunk_size
is not a multiple ofUPLOAD_CHUNK_SIZE
.-
bytes_uploaded
¶ int – Number of bytes that have been uploaded.
-
chunk_size
¶ int – The size of each chunk used to upload the resource.
-
finished
¶ bool – Flag indicating if the upload has completed.
-
initiate
(transport, stream, metadata, content_type, total_bytes=None, stream_final=True)¶ Initiate a resumable upload.
By default, this method assumes your
stream
is in a “final” state ready to transmit. However,stream_final=False
can be used to indicate that the size of the resource is not known. This can happen if bytes are being dynamically fed intostream
, e.g. if the stream is attached to application logs.If
stream_final=False
is used,chunk_size
bytes will be read from the stream every timetransmit_next_chunk()
is called. If one of those reads produces strictly fewer bites than the chunk size, the upload will be concluded.Parameters: - transport (Session) – A
requests
object which can make authenticated requests. - stream (IO[bytes]) – The stream (i.e. file-like object) that will
be uploaded. The stream must be at the beginning (i.e.
stream.tell() == 0
). - metadata (
Mapping
[str
,str
]) – The resource metadata, such as an ACL list. - content_type (str) – The content type of the resource, e.g. a JPEG
image has content type
image/jpeg
. - total_bytes (
Optional
[int
]) – The total number of bytes to be uploaded. If specified, the upload size will not be determined from the stream (even ifstream_final=True
). - stream_final (
Optional
[bool
]) – Indicates if thestream
is “final” (i.e. no more bytes will be added to it). In this case we determine the upload size from the size of the stream. Iftotal_bytes
is passed, this argument will be ignored.
Returns: The HTTP response returned by
transport
.Return type: - transport (Session) – A
-
invalid
¶ bool – Indicates if the upload is in an invalid state.
This will occur if a call to
transmit_next_chunk()
fails. To recover from such a failure, callrecover()
.
-
recover
(transport)¶ Recover from a failure.
This method should be used when a
ResumableUpload
is in aninvalid
state due to a request failure.This will verify the progress with the server and make sure the current upload is in a valid state before
transmit_next_chunk()
can be used again.Parameters: transport (Session) – A requests
object which can make authenticated requests.Returns: The HTTP response returned by transport
.Return type: Response
-
total_bytes
¶ Optional
[int
] – The total number of bytes to be uploaded.If this upload is initiated (via
initiate()
) withstream_final=True
, this value will be populated based on the size of thestream
being uploaded. (By defaultstream_final=True
.)If this upload is initiated with
stream_final=False
,total_bytes
will beNone
since it cannot be determined from the stream.
-
transmit_next_chunk
(transport)¶ Transmit the next chunk of the resource to be uploaded.
If the current upload was initiated with
stream_final=False
, this method will dynamically determine if the upload has completed. The upload will be considered complete if the stream produces fewer thanchunk_size
bytes when a chunk is read from it.In the case of failure, an exception is thrown that preserves the failed response:
>>> error = None >>> try: ... upload.transmit_next_chunk(transport) ... except resumable_media.InvalidResponse as caught_exc: ... error = caught_exc ... >>> error InvalidResponse('Request failed with status code', 400, 'Expected one of', <HTTPStatus.OK: 200>, 308) >>> error.response <Response [400]>
Parameters: transport (Session) – A requests
object which can make authenticated requests.Returns: The HTTP response returned by transport
.Return type: Response Raises: InvalidResponse
– If the status code is not 200 or 308.
-
class
google.resumable_media.requests.upload.
SimpleUpload
(upload_url, headers=None)¶ Bases:
google.resumable_media.requests._helpers.RequestsMixin
,google.resumable_media._upload.SimpleUpload
Upload a resource to a Google API.
A simple media upload sends no metadata and completes the upload in a single request.
Parameters: -
upload_url
¶ str – The URL where the content will be uploaded.
-
finished
¶ bool – Flag indicating if the upload has completed.
-
transmit
(transport, data, content_type)¶ Transmit the resource to be uploaded.
Parameters: Returns: The HTTP response returned by
transport
.Return type:
-