Import Documents in the Nuxeo Platform Using Node.js
The main goal is to be able to import a hierarchy of documents in a Nuxeo Platform instance without having a direct access to the server by using the Automation and REST API exposed by the Nuxeo Platform.
Overview of the Importers
The first one, nuxeo-node-importer, imports a local folder to a Nuxeo Platform instance using the Automation and REST API.
Some behaviors can be configured while running the import such as:
- the document type to use when creating a Folder on Nuxeo
- the Automation chain to use when importing files
- the maximum number of concurrent requests
The second one, nuxeo-node-custom-importer, is a fork of the first one that shows how to customize it to import a custom hierarchy with custom rules.
You can take a look at the README file of nuxeo-node-importer which explains the logic we follow when creating documents on the remote instance.
Note that each call to the Nuxeo Platform instance is transactional but the whole import of a document is not. This means that if an error occurred in functions called after the document creation, the document won't be rollbacked / deleted.
How It Works
The main execution flow, for both importers, is:
- Asynchronously recursively walk the folder given as parameter
- For each file or folder, add a new task to the queue
- Each task processes the file or folder and calls a set of functions in a series (to avoid overloading the server)
We have a
queuewhere we put a task per file or folder. We define the
workerwhen creating it, and it will process each task we put in the
queueallows us to limit the number of concurrent calls on the Nuxeo Platform instance.
var queue = async.queue(worker, concurrency);
In each task in the
queue, we use
async.waterfallto execute a set of functions in series. For each task that runs, only one call at a time will be made on the server.
On nuxeo-node-importer, the set of functions is fairly simple:
On nuxeo-node-custom-importer, which is a sample of what we can do, the set of functions is more advanced (for creating a file):
Extend the Importers
The easiest way to extend the importers is to clone / fork one of the importer as a base, and modify the set of functions executed when a file or folder is imported.
To implement custom behaviour when importing documents, you need to add new functions or modify the existing ones and add them to the
funcs array (see the
Category: Product & Development