[Monday Dev Heaven] How to manage external assets in Nuxeo, first try with oEmbed

Youtube Integration

Last week my colleague Brendan asked me if I knew a way to manage existing video assets in Nuxeo while keeping the video data on a remote platform like Youtube, Wistia, Kaltura or even Brightcove. I started working on this and came with different ideas. First I thought about doing this for more than only video. There are many things you might want to retrieve from a web page: OpenGraph tags, images, videos, text etc… And as they are different type of datas to extract, there are different ways to do it. So I needed to come up with something generic enough. Developers should be able to plug in any external assets provider and this would be a good way to experience the platform extensibility. Among the different existing protocol to retrieve such metadata, I stumbled upon embed.ly which I might use in the future. It looks really nice and supporting it would bring many differents providers at once. But for now I’ve chosen oEmbed as I’m already familiar with it:

oEmbed is a format for allowing an embedded representation of a URL on third party sites. The simple API allows a website to display embedded content (such as photos or videos) when a user posts a link to that resource, without having to parse the resource directly.

What I am going to do with this is a new document type for external resources. It will have only one field in the edit and create form, where the user will simply enter the link of the resource. Once we have a link we can do an AJAX request to Nuxeo that will try to extract properties from it. Those properties will be passed to the Ajax callback so that we can give feedback to the user. For now I will extract only two properties: the title and the HTML oEmbed code. The given link we’ll go through a list of contributed ExternalResourceProvider. Their role will be to extract properties from the given url.

Content Model

Let’s start by contributing a simple schema with the following properties:

<?xml version="1.0" encoding="UTF-8"?> <xs:schema xmlns:nxs="http://www.nuxeo.org/ecm/schemas/externalvideo" xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.nuxeo.org/ecm/schemas/externalvideo"> <xs:element name="link" type="xs:string"/> <xs:element name="html" type="xs:string"/> <xs:element name="provider" type="xs:string"/> <xs:element name="providerIcon" type="xs:string"/> </xs:schema> 

This schema will be binded to the ExternalResource Facet so that any document type can handle external link. For the purpose of this blog I will also add a new document type ExternalResource.

<component name="org.nuxeo.externalresource.core.type.contrib">

<require>org.nuxeo.ecm.core.CoreExtensions</require>

<extension target="org.nuxeo.ecm.core.schema.TypeService" point="schema"> <schema name="external_resource" src="schemas/external_resource.xsd" prefix="exr" /> </extension>

<extension target="org.nuxeo.ecm.core.schema.TypeService" point="doctype"> <facet name="ExternalResource"> <schema name="external_resource" /> </facet>

<doctype name="ExternalResource" extends="Document"> <schema name="common" /> <schema name="dublincore" /> <schema name="uid" /> <facet name="Commentable" /> <facet name="ExternalResource" /> </doctype> </extension> </component> 

And this is really all I need right now for my content model. Setting and displaying properties will be handle by the different widgets and layouts associated to my doc type.

External Resource Provider

So about those providers, what do they have to do? The ExternalResourceProvider interface should answer to that question:

package org.nuxeo.externalresource.provider;

import java.io.Serializable; import java.util.Map;

public interface ExternalResourceProvider {

/** * @return the name of the provider, also stored in the externalResource * schema. */ String getName();

/** * * @return the path to the provider icon defined in it's contribution. */ String getIcon();

/** * * @param url * @return must return true if the provider is able to handle the given URL. */ boolean match(String url);

/** * * @param url * @return the different properties extracted from the given url by the * provider. It must return at least a title and an HTML preview of * the content. */ Map<String, Serializable> getProperties(String url); }

And now we need something to register those providers. Let’s create a service and an extension point for that. The service also have to return the list of providers or a named provider. To define it, as usual, we create an XML component (that can be generated by the Nuxeo Component wizard):

<?xml version="1.0"?> <component name="org.nuxeo.externalresource.provider.ExternalResourceProviderService" version="1.0">

<implementation class="org.nuxeo.externalresource.provider.ExternalResourceServiceImpl" />

<documentation> The ExternalResourceProviderService register ExternalResourceProvider using the provider extension point. It can return all of them as a list or a specific one by name. @author Laurent Doguin ([email protected]) </documentation>

<service> <provide interface="org.nuxeo.externalresource.provider.ExternalResourceProviderService" /> </service>

<extension-point name="provider"> <documentation> The provider extension point register provider using a name and a class implementing the ExternalResourceProvider interface. <code> <provider name="youtube" class="org.nuxeo.externalresource.provider.YoutubeProvider" /> </code> @author Laurent Doguin ([email protected]) </documentation> <object class="org.nuxeo.externalresource.provider.ExternalResourceProviderDescriptor" /> </extension-point> </component> 

You can see here that we declare the service interface, its implementation and the associated extension point. The object tag class attribute points to the descriptor class converting the XML contribution into Java Object given to the service registerContribution method. It uses XMap annotation to do so:

package org.nuxeo.externalresource.provider;

import org.nuxeo.common.xmap.annotation.XNode; import org.nuxeo.common.xmap.annotation.XObject;

@XObject("provider") public class ExternalResourceProviderDescriptor {

@XNode("@name") private String name;

@XNode("@enabled") private boolean enabled = true;

@XNode("@class") private Class<ExternalResourceProvider> className;

public Class<ExternalResourceProvider> getClassName() { return className; }

public boolean isEnabled() { return enabled; }

public String getName() { return name; }

} 

Now let’s jump to the service implementation (partly generated by Nuxeo IDE Component wizard):

import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map;

import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.nuxeo.runtime.model.ComponentContext; import org.nuxeo.runtime.model.ComponentInstance; import org.nuxeo.runtime.model.DefaultComponent; import org.osgi.framework.Bundle;

/** * @author ldoguin */ public class ExternalResourceProviderServiceImpl extends DefaultComponent implements ExternalResourceProviderService {

private static final String PROVIDER_DECLARATION_EP = "provider";

public static final String NAME = "org.nuxeo.externalresource.provider.ExternalResourceService";;

private static final Log log = LogFactory.getLog(ExternalResourceProviderServiceImpl.class);

protected Map<String, ExternalResourceProvider> providerInstances;

protected Bundle bundle;

public Bundle getBundle() { return bundle; }

/** * Component activated notification. Called when the component is activated. * All component dependencies are resolved at that moment. Use this method * to initialize the component. * <p> * The default implementation of this method is storing the Bundle owning * that component in a class field. You can use the bundle object to lookup * for bundle resources: * <code>URL url = bundle.getEntry("META-INF/some.resource");</code>, load * classes or to interact with OSGi framework. * <p> * Note that you must always use the Bundle to lookup for resources in the * bundle. Do not use the classloader for this. * * @param context the component context. Use it to get the current bundle * context */ @Override public void activate(ComponentContext context) { this.bundle = context.getRuntimeContext().getBundle(); this.providerInstances = new HashMap<String, ExternalResourceProvider>(); }

/** * Component deactivated notification. Called before a component is * unregistered. Use this method to do cleanup if any and free any resources * held by the component. * * @param context the component context. Use it to get the current bundle * context */ @Override public void deactivate(ComponentContext context) { this.bundle = null; this.providerInstances = null; }

@Override public void registerContribution(Object contribution, String extensionPoint, ComponentInstance contributor) { if (extensionPoint.equals(PROVIDER_DECLARATION_EP)) { if (contribution instanceof ExternalResourceProviderDescriptor) { ExternalResourceProviderDescriptor provider = (ExternalResourceProviderDescriptor) contribution; try { String providerName = provider.getName(); if (provider.isEnabled()) { ExternalResourceProvider providerInstance = provider.getClassName().newInstance(); providerInstances.put(providerName, providerInstance); } else { if (providerInstances.containsKey(providerName)) { providerInstances.remove(providerName); } } } catch (InstantiationException e) { log.error("Error while creating instance of provider " + provider.getName() + " :" + e.getMessage()); } catch (IllegalAccessException e) { log.error("Error while creating instance of provider " + provider.getName() + " :" + e.getMessage()); }

} } }

@Override public void unregisterContribution(Object contribution, String extensionPoint, ComponentInstance contributor) throws Exception { if (extensionPoint.equals(PROVIDER_DECLARATION_EP)) { if (contribution instanceof ExternalResourceProviderDescriptor) { ExternalResourceProviderDescriptor provider = (ExternalResourceProviderDescriptor) contribution; String providerName = provider.getName(); if (providerInstances.containsKey(providerName)) { providerInstances.remove(providerName); } } } }

@Override public List<ExternalResourceProvider> getProviders() { return new ArrayList<ExternalResourceProvider>( providerInstances.values()); }

@Override public ExternalResourceProvider getProvider(String providerName) { return providerInstances.get(providerName); } } 

I’ve just implemented registerContribution, unregisterContribution, getProviders and getProvider. As you can see this is not doing much things. It only store the registered providers in a map using their name as key, override them if they are already in the map and expose them through two methods getProviders and getProvider(providerName). And about those providers, here’s the YouTube example:

package org.nuxeo.externalresource.provider.instance.oembed;

import java.io.Serializable; import java.util.Map; import java.util.regex.Matcher; import java.util.regex.Pattern;

public class YoutubeProvider extends AbstractOEmbedProvider {

protected Pattern youtubePattern = Pattern.compile("(.*youtube.*)", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);

@Override public boolean match(String url) { Matcher m = youtubePattern.matcher(url); if (m.matches()) { return true; } return false; }

@Override public String getName() { return "youtube"; }

@Override public String getIcon() { return "/img/youtube.gif"; }

@Override public Map<String, Serializable> getProperties(String url) { return getPropertiesWithAutoDiscovery(url); }

}

The match method assures that the given URL can be handled by the provider. The getProperties method should return at least a title and some HTML code to preview the page content. This provider is using java-oembed from Michael Simons to extract metadata from the page. As you can see getProperties return a map of Serializable objects so it should be flexible enough to use something else than oEmebed. I’m currently mostly thinking about OpenGraph.

That’s all for the core part. The next question is how to display/use this in our application.

Displaying the provider content with Widgets and Layouts

You can read this documentation page on widget and layouts if you don’t know what it is. Here’s what we need to display our provider informations, starting with the main widget:

<f:subview xmlns:f="http://java.sun.com/jsf/core" xmlns:h="http://java.sun.com/jsf/html" xmlns:a4j="https://ajax4jsf.dev.java.net/ajax" xmlns:nxu="http://nuxeo.org/nxweb/util" xmlns:nxdir="http://nuxeo.org/nxdirectory" xmlns:c="http://java.sun.com/jstl/core" xmlns:nxp="http://nuxeo.org/nxweb/pdf" id="#{widget.id}">

<c:if test="#{widget.mode != 'create' and widget.mode != 'edit'}"> <h3 class="summaryTitle">#{messages['label.externalResource.content']}</h3> <div id="oEmbedContent"> <h:outputText id="htmlContent" value="#{field_2}" escape="false" /> </div> <table class="dataInput"> <tbody> <tr> <td class="labelColumn"><h:outputText styleClass="labelColumn" value="#{messages['label.widget.externalResource.externalResource']}" /> </td> <td class="fieldColumn"><h:outputText id="link" value="#{field_0}" /> </td> </tr> </tbody> </table> </c:if>

<c:if test="#{widget.mode == 'create' or widget.mode == 'edit'}">

<a4j:region id="#{widget.id}_region"> <h:inputText id="oEmbedInput" value="#{field_0}"> <a4j:support event="onkeyup" reRender="oEmbedObjectContainer" requestDelay="1000" actionListener="#{externalResourceManager.inputChange}" ignoreDupResponses="true" eventsQueue="oEmbedInputQueue"> </a4j:support> </h:inputText> <span id="hidden_fields"> <h:inputHidden value="#{field_2}" /> <h:inputHidden value="#{field_3}" /> <h:inputHidden value="#{field_1}" /> <h:inputHidden value="#{field_4}" /> </span>

<a4j:status> <f:facet name="start"> <h:graphicImage value="/img/standart_waiter.gif" /> </f:facet> </a4j:status>

<a4j:outputPanel id="oEmbedObjectContainer"> <div> <h:outputText value="#{messages['label.externalResource.supportedSites']}" /> <br /> <div> <c:forEach var="provider" items="#{externalResourceManager.getProviderOptions()}"> <nxu:graphicImage alt="#{provider.getName()}" title="#{provider.getName()}" value="#{provider.getIcon()}" styleClass="#{nxu:test(provider.getName() == externalResourceManager.providerName, 'bigIcon itemSelected', 'bigIcon')}" /> </c:forEach> </div> </div>

<script type="text/javascript"> document.getElementById('hidden_fields').childNodes[0].value = '#{externalResourceManager.html}'; document.getElementById('hidden_fields').childNodes[1].value = '#{externalResourceManager.providerIcon}'; document.getElementById('hidden_fields').childNodes[2].value = '#{externalResourceManager.providerName}'; document.getElementById('hidden_fields').childNodes[3].value = '#{externalResourceManager.title}'; </script> <div> <h:outputText value="#{messages['label.externalResource.content']}" /> </div> <div> <h:outputText id="htmlContent" value="#{externalResourceManager.html}" escape="false" /> </div> </a4j:outputPanel> </a4j:region>

</c:if> </f:subview> 

Every time the user hits a key in the oEmbedInput text field, we do an ajax query to the server. The input is given to a method that goes through all providers match method. The first matching provider is used to extract the title and HTML code from the URL which are then assigned to externalResourceManager variables. I use the reRender attribute of my a4j support tag to simulate the callback. It reRenders some JavaScript that assigns the new values to some hidden input field. Here’s the Seam bean to use with the widget (again generated by a wizard):

package org.nuxeo.externalresource.provider;

import java.io.Serializable; import java.util.List; import java.util.Map;

import javax.faces.component.UIComponent; import javax.faces.component.ValueHolder; import javax.faces.event.AbortProcessingException; import javax.faces.event.ActionEvent;

import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.jboss.seam.ScopeType; import org.jboss.seam.annotations.In; import org.jboss.seam.annotations.Name; import org.jboss.seam.annotations.Scope; import org.jboss.seam.faces.FacesMessages; import org.nuxeo.ecm.core.api.ClientException; import org.nuxeo.ecm.core.api.CoreSession; import org.nuxeo.ecm.core.api.model.PropertyException; import org.nuxeo.ecm.platform.ui.web.api.NavigationContext; import org.nuxeo.ecm.webapp.helpers.ResourcesAccessor; import org.nuxeo.externalresource.ExternalResourceConstants;

@Name("externalResourceManager") @Scope(ScopeType.EVENT) public class externalResourceManagerBean implements Serializable {

private static final long serialVersionUID = 1L;

private static final Log log = LogFactory.getLog(externalResourceManagerBean.class);

@In(create = true, required = false) protected transient CoreSession documentManager;

@In(create = true) protected NavigationContext navigationContext;

@In(create = true, required = false) protected transient FacesMessages facesMessages;

@In(create = true) protected transient ResourcesAccessor resourcesAccessor;

@In(create = true) protected ExternalResourceProviderService externalResourceService;

protected List<ExternalResourceProvider> providerInstanceList;

protected String link;

private String title;

protected String html;

private String providerName;

private String providerIcon;;

public List<ExternalResourceProvider> getProviderOptions() { if (providerInstanceList == null) { providerInstanceList = externalResourceService.getProviders(); } return providerInstanceList; }

public void renderUrl() throws PropertyException, ClientException { if (link == null) { return; } for (ExternalResourceProvider provider : getProviderOptions()) { if (provider.match(link)) { Map<String, Serializable> properties = provider.getProperties(link); if (properties != null) { setHtml((String) properties.get(ExternalResourceConstants.EXTERNAL_RESOURCE_HTML_KEY)); setTitle((String) properties.get(ExternalResourceConstants.EXTERNAL_RESOURCE_TITLE_KEY)); setProviderIcon(provider.getIcon()); setProviderName(provider.getName()); } return; } } }

public void inputChange(ActionEvent event) throws PropertyException, ClientException { UIComponent input = event.getComponent().getParent(); if (input instanceof ValueHolder) { link = (String) ((ValueHolder) input).getValue(); link = link.trim(); renderUrl(); } else { log.error("Bad component returned " + input); throw new AbortProcessingException("Bad component returned " + input); } }

public void setLink(String link) { this.link = link; }

public String getLink() { return link; }

public void setHtml(String html) { this.html = html; }

public String getHtml() { return html; }

public void setProviderIcon(String providerIcon) { this.providerIcon = providerIcon; }

public String getProviderIcon() { return providerIcon; }

public void setProviderName(String providerName) { this.providerName = providerName; }

public String getProviderName() { return providerName; }

public void setTitle(String title) { this.title = title; }

public String getTitle() { return title; }

} 

The next thing to do is to register the widget and associate it to a Layout as usual through an extension point:

<?xml version="1.0" encoding="UTF-8"?> <component name="org.nuxeo.externalresource.layouts">

<extension target="org.nuxeo.ecm.platform.forms.layout.WebLayoutManager" point="widgets"> <widget name="externalResource" type="template"> <fields> <field>exr:link</field> <field>exr:provider</field> <field>exr:html</field> <field>exr:providerIcon</field> <field>dc:title</field> </fields> <properties mode="any"> <property name="template">/widgets/external_resource_widget.xhtml</property> </properties> </widget> <widget name="summary_current_document_externalResource_provider" type="template"> <fields> <field>exr:provider</field> <field>exr:providerIcon</field> </fields> <properties mode="any"> <property name="template">/widgets/external_resource_widget.xhtml</property> </properties> </widget> </extension>

<extension target="org.nuxeo.ecm.platform.forms.layout.WebLayoutManager" point="layouts"> <layout name="externalResource"> <templates> <template mode="any">/layouts/layout_default_template.xhtml</template> </templates> <rows> <row> <widget>externalResource</widget> </row> </rows> </layout>

<layout name="externalResource_summary_layout"> <templates> <template mode="any">/layouts/layout_summary_template.xhtml</template> </templates> <rows> <row> <widget>externalResource</widget> <widget>summary_current_document_dublincore</widget> <widget>summary_current_document_comments</widget> </row> <row> <widget>summary_current_document_actions</widget> <widget>summary_current_document_tagging</widget> <widget>summary_current_document_relations</widget> <widget>summary_current_document_externalResource_provider</widget> </row> <row> <widget>summary_current_document_single_tasks</widget> </row> </rows> </layout>

</extension>

</component> 

In the end I have a simple layout called externalResource that displays the externalResource widget and another layout called _externalResource_summary_layout_ used for the summary tab. Those layouts have to be associated to the ExternalResource document type through the following contribution:

<?xml version="1.0" encoding="UTF-8"?> <component name="org.nuxeo.externalresource.jsf.types">

<require>org.nuxeo.ecm.platform.types</require>

<extension target="org.nuxeo.ecm.platform.types.TypeService" point="types"> <type id="ExternalResource"> <label>ExternalResource</label> <default-view>view_documents</default-view> <icon>/icons/video.png</icon> <bigIcon>/icons/video_big.png</bigIcon> <category>SimpleDocument</category> <description>Video.description</description> <layouts mode="any"> <layout>heading</layout> <layout>externalResource</layout> </layouts> <layouts mode="edit"> <layout>externalResource</layout> </layouts> <layouts mode="create"> <layout>externalResource</layout> </layouts> <layouts mode="summary"> <layout>externalResource_summary_layout</layout> </layouts> </type>

<!-- This contributrion also defines where we can create an ExternalResource document. --> <type id="Workspace"> <subtypes> <type>ExternalResource</type> </subtypes> </type>

<type id="Folder"> <subtypes> <type>ExternalResource</type> </subtypes> </type>

</extension>

</component> 

And there we are, now Brendan can share YouTube videos or Flickr images in our intranet. But obviously we won’t stop here. There are many other things you might want to do with external links. We could add an import method that would store the content directly in Nuxeo and then lets us use the available converters. And then maybe add some code to edit metadatas in Flickr or YouTube from Nuxeo. We could also look for other suff than oEmbed object like OpenGraph tags :)

As usual it’s on GitHub, feel free to fork it and to send us pull request to add new providers :) See Ya’Friday.