As you may have noted recently, we’ve shared several articles on the blog over the last month or so sharing some of the amazing work that our Product and Engineering teams have been doing to innovate and drive the Nuxeo Platform forward. We’ve covered a range of important topics and really big deliverables, like:
- our new, “point and click” interface for Nuxeo Insight
- SEC 17a certification for our Nuxeo Cloud on AWS
- a refreshed Nuxeo Salesforce connector
- and even integration for Project Management solutions.
Today, I want to focus on a topic that may be a bit smaller, but which is no less important to our customers and is a great example of how sometimes, innovation is all about the details.
For those of you who aren’t familiar with it, Nuxeo Drive is our desktop synchronization tool and it provides a vital set of collaborative capabilities for offline users who want access to critical information when they are not in the office or on a corporate VPN. With Nuxeo Drive, you can designate specific files or folders that you want to synchronize and then - as files are created, updated or deleted - any changes will automatically be synchronized across any number of devices. Similarly, users can make changes to these documents while disconnected, and then these changes will be synchronized with the Nuxeo Platform once the user reconnects. In short, Nuxeo Drive is a simple, but powerful tool for disconnected users and an important complement to the Nuxeo Platform.
Nuxeo Drive has been an available add-on for several years now, and today’s blog is all about innovation. How do those two things go together, you might ask? Well, through the ongoing innovation we are making with Nuxeo Drive, we unlocked an entirely new use case, and it’s a real game-changer for many of our customers.
Working with Large-Scale Objects
One of the critical challenges that many of our customers face is how to enable users to work with very large objects, e.g. high-resolution video, or with very large sets of objects. Let’s be honest, browser-based user interfaces aren’t always the best tools for moving large-scale objects around. Yet, many of our customers were using this same UI to upload entire file directories, compound document types, or complex, nested folder structures. All of these present some unique challenges for browser-based applications, particularly in terms of performance, resiliency, and ease of use, and the user experience is typically not very good. All of which got us to thinking that there must be a better way, which is a bit of a corporate mantra at Nuxeo.
So, we took advantage of our existing desktop client architecture. Nuxeo Drive was designed to efficiently move files around, in the background, with a simple, easy-to-use interface. Rather than only using it for synchronization, we expanded its functionality to provide an optimal user experience for users that were working with large object types or large sets of objects.
Let’s talk about just some of the little details in Nuxeo Drive that add up to impressive innovation for large-scale object transfer:
- Direct Transfer to Amazon S3 - Rather than using a Nuxeo API endpoint for ingesting a document or asset, Nuxeo Drive enables direct upload of the binary object to Amazon Simple Storage Service (Amazon S3), delivering between 1x and 4x faster ingestion while taking advantage of Amazon S3’s transfer acceleration. In keeping with Nuxeo best practices, the binary object is separated from its corresponding metadata and directly ingested into the object store. Nuxeo Drive also creates a corresponding document in the Nuxeo Platform which references the binary storage location.
- Multi-Threaded Upload - Multi-threading brings two key advantages. First, from a performance perspective, large object types or large sets of objects can be broken into smaller “chunks” and uploaded in parallel, resulting in faster file transfers. And in the event of a connection error or a failed file transfer, the service is more resilient, since it can simply resume from the last successful “chunk,” rather than restarting the entire upload. This is particularly useful for users who have low bandwidth or experience frequent interruptions in service.
- Asynchronous Transfer - Lack of asynchronous transfers is a critical shortcoming of browser-based interfaces, because they cannot do two things at once. With Nuxeo Drive, Nuxeo users can continue to do critical work in WebUI while allowing Nuxeo Drive to handle the file transfer in the background. And with multi-gigabyte objects or thousands of documents being uploaded, this is a critical feature for user productivity.
- Folder Hierarchies - When users upload large sets of documents, it is not uncommon for these objects to be stored in complex folder structures. Nuxeo Drive supports the selection and upload of multiple folders and folder hierarchies, enabling users to select a folder or multiple folders with any number of nested folders inside while preserving this structure within the Nuxeo Platform.
- Progress Tracking - With Nuxeo Drive is now able to track the upload progress, providing against the number, indicating - in real time - the number of objects that remain for upload.
- Pause and Resume - Nuxeo Drive now gives users the ability to manually pause an in-progress upload - perhaps as they are shutting down at the end of the workday - and then resume the upload the next time they are connected to the Nuxeo Platform. Combined with multi-threaded upload capability, Nuxeo Drive provides resiliency for in-progress uploads in the event of a number of different common interruptions, including temporary loss of connection, exit from Nuxeo Drive, proper computer shutdown, low battery shutdown, and even shut down in the event of a computer crash. It seems like a simple feature, but the result is that, once started, you don’t have to worry about the upload completing.
These are just a few of the key features of our latest version of Nuxeo Drive. I hope you will agree that innovation sometimes comes in small packages and that Nuxeo Drive provides a much more robust user experience for working with large object types and large sets of objects. I also hope you will take the time to learn more about Nuxeo Drive and the compelling advantages it can bring to your organization.
And, finally, since the proof (as they say) is in the pudding, let me tell you about our testing thus far. With Nuxeo Drive, we have already successfully loaded 50GB objects directly to Amazon S3 and, we have also successfully tested with 100,000-object file sets. Despite these benchmarks, we fully expect that Nuxeo Drive can support much higher loads that these. Together, Nuxeo Drive is not only providing a proper enterprise user experience; it is also providing a robust and proven enterprise service.