S3 API Compatibility
This article explains how Orchesto delivers compatibility with, and extends, the AWS S3 API as well as known limitations.
Orchesto supports all of the core object storage functions in the Amazon S3 API (that is, create, read and delete operations needed to manage buckets and objects). The following list gives a basic outline:
|Bucket||HEAD, PUT, DELETE, GET, location||✔|
|Bucket||policy, acl, versioning||✘|
|Object||HEAD, PUT¹, GET, DELETE²||✔|
|Object||policy, acl, versioning, torrent||✘|
¹ Also supporting multi-part upload and copy.
² Delete either one object or multiple objects.
Note that Amazon S3 API features not outlined in the above table are not supported by Orchesto.
To simplify integration into existing architectures, Orchesto is designed to provide a high degree of transparency against targeted storage environments. In particular, it can preserve region, bucket and object names as well as security credentials and custom HTTP header fields.
Deploying Orchesto as a forward proxy against a storage provider with maximum transparency for a given client requires:
configuration of Orchesto with the storage backend with all regions, security credentials and buckets as-is;
configuration of the client to change its AWS S3 API endpoint URL to reference Orchesto instead
For example, let's assume you have buckets and objects in Amazon AWS S3, and you already use the AWS CLI tool to work with your storage.
$ aws s3 ls 2018-05-17 10:14:32 concretely 2018-05-17 10:14:31 cylindrocellular 2018-05-17 10:14:28 distinctiveness 2018-05-17 10:14:31 inexactitude 2018-05-17 10:14:31 olefinic 2018-05-17 10:14:27 poligraphical 2018-05-17 10:14:31 potful 2018-05-17 10:14:27 sportsmanlike 2018-05-17 10:14:32 unsymbolically
Then, by adding a corresponding Amazon storage backend, security credential, and buckets to Orchesto, the only configuration change required in clients is the endpoint URL.
$ aws s3 ls --endpoint-url http://127.0.0.1:9090 2018-05-23 15:17:22 concretely 2018-05-23 15:17:22 cylindrocellular 2018-05-23 15:17:22 distinctiveness 2018-05-23 15:17:22 inexactitude 2018-05-23 15:17:22 olefinic 2018-05-23 15:17:22 poligraphical 2018-05-23 15:17:22 potful 2018-05-23 15:17:22 sportsmanlike 2018-05-23 15:17:22 unsymbolically
When preparing Orchesto to enable access to already existing buckets, the corresponding virtual buckets that you create will have a distinct creation time.
To work as a multi-point gateway, Orchesto abstracts discrepancies in the object storage model across storage providers. This includes idiosyncracies in S3 API implementations and aligment with protocol translation.
As all storage resources are private, only authenticated requests can access them. Anonymous requests are relayed to remote storage providers, delegating access control to enable existing security policies to govern anonymous access.