Skip to content

Commit 4a2465f

Browse files
Marek Kaletatipabu
Marek Kaleta
authored and
tipabu
committed
Add copy object method
Implement copy object method in swiftclient Connection, Service and CLI. Although COPY functionality can be accomplished with 'X-Copy-From' header in PUT request, using copy is more convenient especially when using copy for updating object metadata non-destructively. Closes-Bug: 1474939 Change-Id: I1338ac411f418f4adb3d06753d044a484a7f32a4
1 parent e05464f commit 4a2465f

11 files changed

Lines changed: 914 additions & 6 deletions

File tree

‎doc/manpages/swift.1‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,19 @@ For more details and options see swift post \-\-help.
7979
\fBExample\fR: post \-m Color:Blue \-m Size:Large
8080
.RE
8181

82+
\fBcopy\fR [\fIcommand-options\fR] \fIcontainer\fR \fIobject\fR
83+
.RS 4
84+
Copies an object to a new destination or adds user metadata to the object (current
85+
user metadata will be preserved, in contrast with the post command) depending
86+
on the args given. The \-\-destination option sets the destination in the form
87+
/container/object. If not set, the object will be copied onto itself which is useful
88+
for adding metadata. The \-M or \-\-fresh\-metadata option copies the object without
89+
the existing user metadata. The \-m or \-\-meta option is always allowed and is used
90+
to define the user metadata items to set in the form Name:Value (this option
91+
can be repeated).
92+
For more details and options see swift copy \-\-help.
93+
.RE
94+
8295
\fBdownload\fR [\fIcommand-options\fR] [\fIcontainer\fR] [\fIobject\fR] [\fIobject\fR] [...]
8396
.RS 4
8497
Downloads everything in the account (with \-\-all), or everything in a

‎doc/source/cli.rst‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,20 @@ Delete
199199
of manifest objects will be deleted as well, unless you specify the
200200
``--leave-segments`` option.
201201

202+
Copy
203+
----
204+
205+
``copy [command-options] container object``
206+
207+
Copies an object to a new destination or adds user metadata to an object. Depending
208+
on the options supplied, you can preserve existing metadata in contrast to the post
209+
command. The ``--destination`` option sets the copy target destination in the form
210+
``/container/object``. If not set, the object will be copied onto itself which is useful
211+
for adding metadata. You can use the ``-M`` or ``--fresh-metadata`` option to copy
212+
an object without existing user meta data, and the ``-m`` or ``--meta`` option
213+
to define user meta data items to set in the form ``Name:Value``. You can repeat
214+
this option. For example: ``copy -m Color:Blue -m Size:Large``.
215+
202216
Capabilities
203217
------------
204218

‎doc/source/service-api.rst‎

Lines changed: 78 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,17 @@ Options
217217
are downloaded in lexically-sorted order. Setting this option to ``True``
218218
gives the same shuffling behaviour as the CLI.
219219

220+
``destination``: ``None``
221+
When copying objects, this specifies the destination where the object
222+
will be copied to. The default of None means copy will be the same as
223+
source.
224+
225+
``fresh_metadata``: ``None``
226+
When copying objects, this specifies that the object metadata on the
227+
source will *not* be applied to the destination object - the
228+
destination object will have a new fresh set of metadata that includes
229+
*only* the metadata specified in the meta option if any at all.
230+
220231
Other available options can be found in ``swiftclient/service.py`` in the
221232
source code for ``python-swiftclient``. Each ``SwiftService`` method also allows
222233
for an optional dictionary to override those specified at init time, and the
@@ -735,13 +746,76 @@ Example
735746

736747
The code below demonstrates the use of ``delete`` to remove a given list of
737748
objects from a specified container. As the objects are deleted the transaction
738-
id of the relevant request is printed along with the object name and number
739-
of attempts required. By printing the transaction id, the printed operations
749+
ID of the relevant request is printed along with the object name and number
750+
of attempts required. By printing the transaction ID, the printed operations
740751
can be easily linked to events in the swift server logs:
741752

742753
.. literalinclude:: ../../examples/delete.py
743754
:language: python
744755

756+
Copy
757+
~~~~
758+
759+
Copy can be called to copy an object or update the metadata on the given items.
760+
761+
Each element of the object list may be a plain string of the object name, or a
762+
``SwiftCopyObject`` that allows finer control over the options applied to each
763+
of the individual copy operations (destination, fresh_metadata, options).
764+
765+
Destination should be in format /container/object; if not set, the object will be
766+
copied onto itself. Fresh_metadata sets mode of operation on metadata. If not set,
767+
current object user metadata will be copied/preserved; if set, all current user
768+
metadata will be removed.
769+
770+
Returns an iterator over the results generated for each object copy (and may
771+
also include the results of creating destination containers).
772+
773+
When a string is given for the object name, destination and fresh metadata will
774+
default to None and None, which result in adding metadata to existing objects.
775+
776+
Successful copy results are dictionaries as described below:
777+
778+
.. code-block:: python
779+
780+
{
781+
'action': 'copy_object',
782+
'success': True,
783+
'container': <container>,
784+
'object': <object>,
785+
'destination': <destination>,
786+
'headers': {},
787+
'fresh_metadata': <boolean>,
788+
'response_dict': <HTTP response details>
789+
}
790+
791+
Any failure in a copy operation will return a failure dictionary as described
792+
below:
793+
794+
.. code-block:: python
795+
796+
{
797+
'action': 'copy_object',
798+
'success': False,
799+
'container': <container>,
800+
'object': <object>,
801+
'destination': <destination>,
802+
'headers': {},
803+
'fresh_metadata': <boolean>,
804+
'response_dict': <HTTP response details>,
805+
'error': <error>,
806+
'traceback': <traceback>,
807+
'error_timestamp': <timestamp>
808+
}
809+
810+
Example
811+
-------
812+
813+
The code below demonstrates the use of ``copy`` to add new user metadata for
814+
objects a and b, and to copy object c to d (with added metadata).
815+
816+
.. literalinclude:: ../../examples/copy.py
817+
:language: python
818+
745819
Capabilities
746820
~~~~~~~~~~~~
747821

@@ -764,7 +838,7 @@ returned with the contents described below:
764838
}
765839

766840
The contents of the capabilities dictionary contain the core swift capabilities
767-
under the key ``swift``, all other keys show the configuration options for
841+
under the key ``swift``; all other keys show the configuration options for
768842
additional middlewares deployed in the proxy pipeline. An example capabilities
769843
dictionary is given below:
770844

@@ -819,7 +893,7 @@ Example
819893
^^^^^^^
820894

821895
The code below demonstrates the use of ``capabilities`` to determine if the
822-
Swift cluster supports static large objects, and if so, the maximum number of
896+
Swift cluster supports static large objects, and if so, the maximum number of
823897
segments that can be described in a single manifest file, along with the
824898
size restrictions on those objects:
825899

‎examples/copy.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import logging
2+
3+
from swiftclient.service import SwiftService, SwiftCopyObject, SwiftError
4+
5+
logging.basicConfig(level=logging.ERROR)
6+
logging.getLogger("requests").setLevel(logging.CRITICAL)
7+
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
8+
logger = logging.getLogger(__name__)
9+
10+
with SwiftService() as swift:
11+
try:
12+
obj = SwiftCopyObject("c", {"Destination": "/cont/d"})
13+
for i in swift.copy(
14+
"cont", ["a", "b", obj],
15+
{"meta": ["foo:bar"], "Destination": "/cc"}):
16+
if i["success"]:
17+
if i["action"] == "copy_object":
18+
print(
19+
"object %s copied from /%s/%s" %
20+
(i["destination"], i["container"], i["object"])
21+
)
22+
if i["action"] == "create_container":
23+
print(
24+
"container %s created" % i["container"]
25+
)
26+
else:
27+
if "error" in i and isinstance(i["error"], Exception):
28+
raise i["error"]
29+
except SwiftError as e:
30+
logger.error(e.value)

‎swiftclient/client.py‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1319,6 +1319,70 @@ def post_object(url, token, container, name, headers, http_conn=None,
13191319
raise ClientException.from_response(resp, 'Object POST failed', body)
13201320

13211321

1322+
def copy_object(url, token, container, name, destination=None,
1323+
headers=None, fresh_metadata=None, http_conn=None,
1324+
response_dict=None, service_token=None):
1325+
"""
1326+
Copy object
1327+
1328+
:param url: storage URL
1329+
:param token: auth token; if None, no token will be sent
1330+
:param container: container name that the source object is in
1331+
:param name: source object name
1332+
:param destination: The container and object name of the destination object
1333+
in the form of /container/object; if None, the copy
1334+
will use the source as the destination.
1335+
:param headers: additional headers to include in the request
1336+
:param fresh_metadata: Enables object creation that omits existing user
1337+
metadata, default None
1338+
:param http_conn: HTTP connection object (If None, it will create the
1339+
conn object)
1340+
:param response_dict: an optional dictionary into which to place
1341+
the response - status, reason and headers
1342+
:param service_token: service auth token
1343+
:raises ClientException: HTTP COPY request failed
1344+
"""
1345+
if http_conn:
1346+
parsed, conn = http_conn
1347+
else:
1348+
parsed, conn = http_connection(url)
1349+
1350+
path = parsed.path
1351+
container = quote(container)
1352+
name = quote(name)
1353+
path = '%s/%s/%s' % (path.rstrip('/'), container, name)
1354+
1355+
headers = dict(headers) if headers else {}
1356+
1357+
if destination is not None:
1358+
headers['Destination'] = quote(destination)
1359+
elif container and name:
1360+
headers['Destination'] = '/%s/%s' % (container, name)
1361+
1362+
if token is not None:
1363+
headers['X-Auth-Token'] = token
1364+
if service_token is not None:
1365+
headers['X-Service-Token'] = service_token
1366+
1367+
if fresh_metadata is not None:
1368+
# remove potential fresh metadata headers
1369+
for fresh_hdr in [hdr for hdr in headers.keys()
1370+
if hdr.lower() == 'x-fresh-metadata']:
1371+
headers.pop(fresh_hdr)
1372+
headers['X-Fresh-Metadata'] = 'true' if fresh_metadata else 'false'
1373+
1374+
conn.request('COPY', path, '', headers)
1375+
resp = conn.getresponse()
1376+
body = resp.read()
1377+
http_log(('%s%s' % (url.replace(parsed.path, ''), path), 'COPY',),
1378+
{'headers': headers}, resp, body)
1379+
1380+
store_response(resp, response_dict)
1381+
1382+
if resp.status < 200 or resp.status >= 300:
1383+
raise ClientException.from_response(resp, 'Object COPY failed', body)
1384+
1385+
13221386
def delete_object(url, token=None, container=None, name=None, http_conn=None,
13231387
headers=None, proxy=None, query_string=None,
13241388
response_dict=None, service_token=None):
@@ -1711,6 +1775,13 @@ def post_object(self, container, obj, headers, response_dict=None):
17111775
return self._retry(None, post_object, container, obj, headers,
17121776
response_dict=response_dict)
17131777

1778+
def copy_object(self, container, obj, destination=None, headers=None,
1779+
fresh_metadata=None, response_dict=None):
1780+
"""Wrapper for :func:`copy_object`"""
1781+
return self._retry(None, copy_object, container, obj, destination,
1782+
headers, fresh_metadata,
1783+
response_dict=response_dict)
1784+
17141785
def delete_object(self, container, obj, query_string=None,
17151786
response_dict=None):
17161787
"""Wrapper for :func:`delete_object`"""

0 commit comments

Comments
 (0)