Repository navigation
Manifest file has its original size #208
Description
I am trying to upload a large zip file to object storage by using Static Large Objects feature upload.
Creating segments and upload to object storage was fine. However, when I creating a manifest file which has all segments information(path, etag, and size_bytes) and upload it to object storage, the manifest file has its original size. I think it suppose to be pretty small because it only has segments information. It happens when I give an option "queryParam("multipart-manifest", "put")" to ObjectPutOptions.create().
Expected Behavior
(I expect)
to uploading a 5.92mb zip file
- segment1(2mb),
- segment2(2mb)
- segment3(1.92mb)
- test.zip (400.00kb)
(what I got)
- segment1(2mb),
- segment2(2mb)
- segment3(1.92mb)
- test.zip (5.92mb)
java 1.8, eclipse-jee-2023-03-R-x64.exe, openstack4j-3.2.0
Impact
Steps to Reproduce
OSClientV3 client = OSFactory.clientFromToken(token);
ObjectStorageObjectService objectService = client.objectStorage().objects();
String objectName = "test.zip";
String contentType = "application/zip";
String segmentContainerName = "202304/13";
String segmentPrefix = "segment";
List<Map<String, Object>> segmentInfoList = new ArrayList<>();
File file = new File("D:/test.zip");
FileInputStream fs = new FileInputStream(file);
int bufferSize = 1024 * 1024 * 2; // 2MB
byte[] buffer = new byte[bufferSize];
int segmentNo = 1;
int bytesRead;
while ((bytesRead = fs.read(buffer)) != -1) {
Map<String, Object> segmentInfo = new HashMap<>();
InputStreamPayload payload = new InputStreamPayload(new ByteArrayInputStream(buffer, 0, bytesRead));
String etag = objectService.put(segmentContainerName, objectName + segmentPrefix + segmentNo, payload);
segmentInfo.put("path", segmentContainerName + "/" + objectName + segmentPrefix + segmentNo);
segmentInfo.put("etag", etag);
segmentInfo.put("size_bytes", bytesRead);
segmentInfoList.add(segmentInfo);
System.out.println(segmentNo + "complete");
segmentNo++;
}
fs.close();
String manifest = new ObjectMapper().writeValueAsString(segmentInfoList);
String etag = objectService.put(segmentContainerName, objectName,
Payloads.create(new ByteArrayInputStream(manifest.getBytes())),
ObjectPutOptions.create().queryParam("multipart-manifest", "put")
.contentType(contentType));
System.out.println(etag);
System.out.println("zip file created");
Activity
[-]Manifest file dose not created as I expected.[/-][+]Manifest file has its original size[/+]