Google has a large offering of very powerful cloud services around AI and Machine Learning. One of these services is the Google Cloud Vision API, which is able to derive insights from images. The BS Google Vision Connector is Business Solution using this API and enriches Contentserv DAM metadata either on-demand or fully automated by workflow actions or background event management.
The information derived from images covers the following areas:
Detect objects automatically
Detect and understand text
Detect product logos
Derive and propose keyword
Find faces and moods
Metadata can be assigned to configurable attributes at the DAM asset and also create special crop hint notes and focus points, which again can be used to automatically create best-fitting crop scenarios.
To use the Google Vision API customers require a valid Google Vision account including API credentials. The procedure is quite similar to the steps laid out in our article about the BS Youtube Connector. Read the section about Setting up the Youtube API Credentials first to get familiar with the Google Cloud APIs.
Start with creating a new project for your Google Vision API requests:
Within your new project click “Enable APIs and Services”:
Search for the Cloud Vision API:
Select the API and click “Enable”:
Once the API is created click on “Credentials”:
The BS Youtube Connector requires API Keys credentials. Click “Create Credentials” and choose “API Key”:
Copy the API Key to your clipboard.
It's a good idea to restrict the key to exactly those APIs you are using. Click “Restrict Key” and select at least the “Cloud Vision API”.
Meta Data from this API is always delivered in English. But it is possible to directly translate the data to other languages using Google Translate API. To activate this in the API go again to Libraries, search for “translate” and activate the Cloud Translation API:
This will require a billing account though. Proceed to register the billing credentials. The first 300$ are given as credit. This should be enough to test the API:
Setting up the BS Google Vision Connector
Once you have created your Google Vision API credentials you can provide them to the global settings of your Contentserv project:
Review the following options:
Google API Key
API Key as described above with at least the Vision AI and optionally the Translation API activated.
Activate Image Recognition
Must be checked to enable the connector and to avoid unwanted transaction costs. When the connector is activated for example the DAM file editor shows a magic wand button, that applies Google Vision insights to the current file.
Translate to Languages
Target languages can be selected here for automated translation of recognized meta data. The translation will automatically be applied to the following fields as soon as they are language specific.
Google API Feature Types
Select one or more Google Cloud Vision API functions to extract.
Field for recognized description and OCR text
Suggestions with a lower confidence score will be ignored
Object Name Field
Field for recognized Objects
Field for the recognized title of the asset
Rename to Title
When activated the recognized image title will be used as new name for the file (plus the file extensions of course)
Field for recognized keywords. If the field type is value range, then the keywords are automatically added to the value range (if they do not exist already) and assigned to the asset.
If the value range is multi lingual also the translations are stored. The English value is always the External Key.
Computed Image Properties
Field for the recognized main color as HEX value
Color Name Field
Field for the recognized name of the main color
Field to store recognized crop hints
Field to derive the requested crop ratios
Using the DAM Editor Toolbar Button
Once the connector is activated, every image DAM editor automatically offers a new additional button with the magic wand icon:
This button will synchronize the current image with the Google Vision API and show the result log in a popup window:
Depending on the global options selected the following updates are applied to the image:
Crop Hints, Focus Point and Face Detection
Keywords, Primary Color and Color Name and Description
Using the DAM Workflow Action Plugin
Another integration scenario that could be helpful is a special workflow action that comes with the BS Google Vision connector:
The action component has only one option:
If activated the workflow action will show the process log in a popup window.
Using the BS Google Vision Active Script
All previous integration scenarios actually trigger a background Active Script called the “BS Google Vision Script”. If it does not yet exist, it will be created automatically with the first request. For that reason, it is also possible to run this script directly.
It has the “Context” as the only parameter, where folders and or multiple files can be selected. All images selected will retrieve the Google Vision insights during the script run:
When checkbox “Context Sensitive” is activated and the script is made available in the “Automation” tab, the script will also be available in the DAM tree and list context menu:
Using the BS Google Vision BS Event Listener
Last but not least there is a way to automatically update images whenever they are uploaded or replaced by newer versions.
The concept was previously implemented as BS Media Event, which is now (April 2021) obsolete and has been replaced by the more flexible and generic BS Event engine.
To set up the event mechanism create a new Active Script and use the Script “BS Event Queue”.
Select the BS Google Vision Connector as an event listener.
In the Event Filter section, you can narrow down the DAM folders or files to listen to:
The BS Event Queue updates automatically the “Events from” timestamp whenever it runs without errors. When such an Active Script is scheduled for example every five minutes, it will process only changed files within these five minutes.
The event listener listens only to onAfterFileContentUpdate events, which means a simple check-in does not trigger a revalidation, as long as the physical file modification date remains unchanged.