BS Google Vision Connector

Google has a large offering of very powerful cloud services around AI and Machine Learning. One of these services is the Google Cloud Vision API, which is able to derive insights from images. The BS Google Vision Connector is Business Solution using this API and enriches Contentserv DAM metadata either on-demand or fully automated by workflow actions or background event management.

The information derived from images covers the following areas:

  • Detect objects automatically

  • Detect and understand text

  • Detect product logos

  • Derive and propose keyword

  • Find faces and moods

  • Analyze colors

Metadata can be assigned to configurable attributes at the DAM asset and also create special crop hint notes and focus points, which again can be used to automatically create best-fitting crop scenarios.

Setting up the Google Vision API

To use the Google Vision API customers require a valid Google Vision account including API credentials. The procedure is quite similar to the steps laid out in our article about the BS Youtube Connector. Read the section about Setting up the Youtube API Credentials first to get familiar with the Google Cloud APIs.

Start with creating a new project for your Google Vision API requests:

Within your new project click “Enable APIs and Services”:

Search for the Cloud Vision API:

Select the API and click “Enable”:

Once the API is created click on “Credentials”:

The BS Youtube Connector requires API Keys credentials. Click “Create Credentials” and choose “API Key”:

Copy the API Key to your clipboard.

It's a good idea to restrict the key to exactly those APIs you are using. Click “Restrict Key” and select at least the “Cloud Vision API”.

Meta Data from this API is always delivered in English. But it is possible to directly translate the data to other languages using Google Translate API. To activate this in the API go again to Libraries, search for “translate” and activate the Cloud Translation API:

This will require a billing account though. Proceed to register the billing credentials. The first 300$ are given as credit. This should be enough to test the API:

Setting up the BS Google Vision Connector

Once you have created your Google Vision API credentials you can provide them to the global settings of your Contentserv project:

 

Review the following options:

Parameter

Description

Parameter

Description

Google API Key

API Key as described above with at least the Vision AI and optionally the Translation API activated.

Activate Image Recognition

Must be checked to enable the connector and to avoid unwanted transaction costs. When the connector is activated for example the DAM file editor shows a magic wand button, that applies Google Vision insights to the current file.

Translate to Languages

Target languages can be selected here for automated translation of recognized meta data. The translation will automatically be applied to the following fields as soon as they are language specific.

Google API Feature Types

Select one or more Google Cloud Vision API functions to extract.

Description Field

Field for recognized description and OCR text

Object Detection

Object Name Field

Field for recognized Objects

Web Detection

Title Field

Field for the recognized title of the asset

Rename to Title

When activated the recognized image title will be used as new name for the file (plus the file extensions of course)

Label Detection

Keyword Field

Field for recognized keywords

Computed Image Properties

Color Field

Field for the recognized main color as HEX value

Color Name Field

Field for the recognized name of the main color

Crop Hints

Crop Field

Field to store recognized crop hints

Crop Ratios

Field to derive the requested crop ratios

Using the DAM Editor Toolbar Button

Once the connector is activated, every image DAM editor automatically offers a new additional button with the magic wand icon:

This button will synchronize the current image with the Google Vision API and show the result log in a popup window:

Depending on the global options selected the following updates are applied to the image:

Crop Hints, Focus Point and Face Detection

Keywords, Primary Color and Color Name and Description

Using the DAM Workflow Action Plugin

Another integration scenario that could be helpful is a special workflow action that comes with the BS Google Vision connector:

The action component has only one option:

If activated the workflow action will show the process log in a popup window.

Using the BS Google Vision Active Script

All previous integration scenarios actually trigger a background Active Script called the “BS Google Vision Script”. If it does not yet exist, it will be created automatically with the first request. For that reason, it is also possible to run this script directly.

It has the “Context” as the only parameter, where folders and or multiple files can be selected. All images selected will retrieve the Google Vision insights during the script run:

When checkbox “Context Sensitive” is activated and the script is made available in the “Automation” tab, the script will also be available in the DAM tree and list context menu:

Using the BS Google Vision BS Event Listener

Last but not least there is a way to automatically update images whenever they are uploaded or replaced by newer versions.

The concept was previously implemented as BS Media Event, which is now (April 2021) obsolete and has been replaced by the more flexible and generic BS Event engine.

To set up the event mechanism create a new Active Script and use the Script “BS Event Queue”.

Select the BS Google Vision Connector as an event listener.

In the Event Filter section, you can narrow down the DAM folders or files to listen to:

The BS Event Queue updates automatically the “Events from” timestamp whenever it runs without errors. When such an Active Script is scheduled for example every five minutes, it will process only changed files within these five minutes.

The event listener listens only to onAfterFileContentUpdate events, which means a simple check-in does not trigger a revalidation, as long as the physical file modification date remains unchanged.