search by color

Search has always been a vital component of the Nuxeo Platform. We are committed to meeting the highest standards and delivering the best enterprise search experience possible. This includes our integration with Elasticsearch as our standard search engine. Recently we received some requests from our customers to add a color search feature in the Digital Asset Management system module of the Nuxeo Platform. For example, easily query pictures that contains blue, green or any other color. This was a great opportunity for us to further improve the Nuxeo search capabilities.

From the end user perspective, searching assets by color is a simple search feature but the real deal is actually the automated indexing of colors when adding or modifying a picture. Let me walk you through how we implemented it and what the search interface looked like.

Extracting Colors from a Picture

Here we kept things simple and used the tools already available in the Nuxeo Platform. First we extract the color histogram from pictures using ImageMagick. This is a simple command line that we call from the Java stack and which returns a text file containing the pixel count for each individual color present in a picture.

Image for Extracting the color histogram with ImagemagickImage for extracting the color histogram

Extracting the color histogram with ImagemagickExtracting the color histogram with ImageMagick

Grouping Colors

If you think of the color Orange, for instance, there are actually thousands of variations of Orange: dark, light, vivid, fade, etc.

Some variations of Orange

In the Nuxeo Platform, when anyone searches for pictures with the color Orange, we wanted to return pictures that contain any shade of orange. This is where using the HSL color space proves very practical because all those variations of a single color share a common parameter which is the HUE.

So, we implemented a module that parses color histograms generated by ImageMagick and groups individual colors by hue. Finally, we filtered groups with a very low pixel count (less than 1% of the image) and stored the significant colors as a document property.

Search Interface

Since the colors have been extracted and stored in a document property, it is now a straightforward process to configure the search view using Nuxeo Studio. We also created a color selector widget to improve the user experience.

Search for pictures which contain BlueSearch for pictures which contain Blue

Search for pictures which contain GreenSearch for pictures which contain Green

The source is available on our github repository. All you need to do is build the package and install it on your Nuxeo instance…and voila! Try it out and don’t forget to tell us what you think about it!