BS Amazon S3 Connector

 

The S3 Connector is a BS CDN synchronization plugin, which mirrors a DAM folder to an AWS S3 (Simple Storage Server) bucket.

Amazon Simple Storage Service (Amazon S3) is an object storage service offering great scalability, data availability, security, and performance.

Using the Amazon S3 Connector each file or folder created, modified, or deleted will be replicated to a configurable Amazon S3 Bucket, to make use of further features around S3 file storage like:

  • Amazon Cloud Front

  • Amazon AWS processing

  • hundreds and thousands of other integrations compatible with Amazon S3.

Setting up the license

As usual first activate the BS license for the BS Amazon S3 Connector:

Setting up the AWS Account options

Once the license is activated you can provide your Amazon AWS credentials. S3 stores files in so called buckets, that can be created in your Amazon Console administration:

In this example, we have created a bucket called “bslive” in the eu-central-1 AWS region. To give access to such a bucket you also need to create an API user through the Identity and Access Management (IAM) of AWS:

We created one user here and added the “AmazonS3FullAccess” policy:

In the tab “Security credentials” create a new access key, which creates the access key ID and Secret access key:

With these settings you can now give Contentserv Access to your S3 buckets:

Connecting a DAM volume with an S3 bucket

The connector itself is implemented as an Event Listener plugin for the BS Event Queue:

You can find further details about the BS Event Queue here.

Add the BS Amazon S3 Synchronisation Event Listener:

For this listener you can define the following parameters:

  • Bucket: drop-down with all buckets found in your S3 account

  • Remove Path from Files: parts of the DAM paths that should be cut from the left when synchronizing to the S3 bucket

  • Prepend Path to Files: add this path to the left of the DAM asset paths

  • Object Key Field: select an attribute on the DAM assets that should retrieve the S3 Object Key after successful synchronization. The Object Key will be used for further updates or deletes to identify the asset on S3.

  • Object URL Field: select an attribute on the DAM assets that should retrieve the S3 Object URL after successful synchronization. The Object URL can be used for example for webshops to retrieve the asset on S3.

  • Preset for Images: select a DAM rendering preset that should be applied to image files before transmitting. It is possible to select also the pure “Master Preview” file or “Original File”. For all non-image files always the original file will be transmitted.

  • Process Mode: In the “Sync” mode, each file added, updated, or deleted in the DAM will be added, updated, or deleted on S3. In the “Delete” Mode each file that is added or updated and found by the event filters will be deleted from S3. This mode allows removing files based on the workflow state.

Creating attributes for the Amazon S3 Key and Url

When the connector creates or modifies assets on S3 it retrieves a URL and a Key from S3. Both identifiers are important and should be stored back to the asset in the DAM.

  • The Amazon S3 Key is the unique identifier within the S3 bucket. Only with this information, the connector knows, that this file was already created and where on the S3 side in case of an update or delete. When the connector creates the asset in S3 for the first time it will take the relative DAM path after prepending and removing the configured prefixes. This could be something like “Some/Folder/Sample Image.png”.

  • The Amazon S3 Url is the internet address to lookup the asset on the S3. This could be some URL like https://bslive.s3.eu-central-1.amazonaws.com/Some/Folder/Sample%20Image.png
    where “bslive” is the bucket name, “eu-central-1” is the AWS region and “/Some/Folder/Sample%20Image.png” is the url-encoded relative key.

Using the S3 Object URL in the DAM

Once your assets are synchronized to S3 they will have a public (if configured) URL that can be used as CDN URLs when publishing your DAM assets.

If the Event Listener configuration has provided an attribute, that should retrieve the S3 Object URL, you will find this URL in the DAM editor.

In the screenshot we have configured the Amazon S3 Url attribute for example:

To verify the content of the URL you can configure a second helper attribute of type “BS Amazon S3 Preview”:

This helper attribute has the following parameters to cooperate with the Object URL field:

Limitations and Warnings

Due to the post-event batch processing concept and the limitations of the S3 API there can occur situations that can not be synchronized to S3 for principal reasons.

Some examples are:

  • renaming or moving of folders with files and subfolders is not supported at the moment

  • changes on the S3 side are not mirrored to the CS DAM

  • combinations of multiple moving or changes followed by deleting an asset or similar chains of modifications will most likely lead to inconsistencies

  • changes that are followed by metadata or filter modifications which prevent the new file from being picked up at all in the next run

  • etc.

In general, a 100% mirroring precision can not be guaranteed and the concept should only be applied to quite stable and final folder structures.

Safe Filename Characters

The following table shows special characters in filenames, where + stands for supported and - for not supported:

Character

Support Contentserv DAM

Support in Amazon S3

Character

Support Contentserv DAM

Support in Amazon S3

Whitespace

+

+

_

+

+

-

+

+

,

+

+

;

+

+

:

-

-

!

+

+

?

-

-

.

+

+

'

+

+

-

-

( )

+

+

[ ]

+

+

{ }

+

+

@

+

+

*

-

-

/

-

-

\

-

-

|

-

-

&

+

+

#

+

+

%

+

+

´

+

+

`

+

+

^

+

+

©

+

+

∂ƒ

+

+

+

+

+

< >

-

-

=

+

+

|

-

-

~

+

+

$

+

+

+

+