Migrating the Nuxeo 'audit' Index in Elasticsearch

For those of you who have previously enabled the storage of the Nuxeo Platform’s audit log in Elasticsearch, you may want to consider migrating your ‘audit’ index when you move to Nuxeo LTS 2015. Starting with Nuxeo LTS 2015 the ‘audit’ index has been renamed ‘nuxeo-audit’ (and, in fact, the name is configurable; see the upgrade notes).

There is no “out-of-the-box” way to copy or rename an index in Elasticsearch. In this example, we will use the Elasticsearch plugin ‘knapsack‘ to perform the migration.

Please note: the existing ‘audit’ index is not deleted after updating to Nuxeo LTS 2015, so any existing audit data will not be lost. The purpose of this migration is to ensure that the new index contains the existing audit data.

Install Knapsack

Go to https://github.com/jprante/elasticsearch-knapsack.

Locate the branch for your version of Elasticsearch. To get the Elasticsearch version, run curl http://localhost:9200/, adapting the URL for your server of course. Here is some example output:

   {
      "status" : 200,
      "name" : "Book",
      "cluster_name" : "elasticsearch",
      "version" : {
        "number" : "1.5.2",
        "build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",
        "build_timestamp" : "2015-04-27T09:21:06Z",
        "build_snapshot" : false,
       "lucene_version" : "4.10.4"
      },
      "tagline" : "You Know, for Search"
    }

Take note of the value in ‘version.number’.

Once the branch is located, check the readme for the installation command line. Copy the command line to install.

Note: you will need to locate the folder containing the Elasticsearch ‘bin’ folder, for example ‘/usr/share/elasticsearch’.

For example, to install knapsack version 1.5.2.2 for Elasticsearch 1.5.x.

    cd /usr/share/elasticsearch
    ./bin/plugin -install knapsack -url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-knapsack/1.5.2.2/elasticsearch-knapsack-1.5.2.2-plugin.zip

Once the plugin is installed, restart Elasticsearch. E.g.:

 /etc/init.d/elasticsearch restart

Prepare Nuxeo Upgrade

Prepare the Nuxeo application for the upgrade: create a backup, ensure you have the latest plugins, migrate the Studio project, etc. Be sure to stop the Nuxeo application during the index migration.

I recommend not upgrading Nuxeo until the index migration is complete because, for example, the update script automatically starts the server if using ‘apt’.

Export Existing ‘audit’ Index

Export the existing ‘audit’ index:

 curl -XPOST 'localhost:9200/audit/_export?pretty&map=\{"audit":"nuxeo-audit"\}&path=/var/lib/elasticsearch/elasticsearch/nodes/0/nuxeo-audit.zip'

The ‘map’ parameter causes knapsack to export the index as if it were ‘nuxeo-audit’.

Note: ‘/var/lib/elasticsearch’ is the default location of ‘data’ for Elasticsearch, but you may need to adjust this path. Check the Elasticsearch Directory Layout documentation if you are not sure where to look. The point is to choose a location that the Elasticsearch server can write to, otherwise the export will fail.

You can check the status of the export with:

 curl -XPOST 'localhost:9200/_export/state?pretty'

Import Data to New Index

Import the data to the ‘nuxeo-audit’ index:

 curl -XPOST 'localhost:9200/nuxeo-audit/_import?pretty&path=/var/lib/elasticsearch/elasticsearch/nodes/0/nuxeo-audit.zip'

To be clear this will create the ‘nuxeo-audit’ index, with the same settings as ‘audit’, and import the data.

You can check the status of the import with:

 curl -XPOST 'localhost:9200/_import/state?pretty'

Verify Results

Check to see if the index was created:

 curl 'http://localhost:9200/_cat/indices?v'

If something looks wrong, delete the new index and try again:

 curl -XDELETE 'http://localhost:9200/nuxeo-audit/?pretty'

If all goes well, optionally delete the old ‘audit’ index.

 curl -XDELETE 'http://localhost:9200/audit/?pretty'

Complete Nuxeo Upgrade

Complete the Nuxeo upgrade. When the server starts up the new ‘nuxeo-audit’ index will be used.