[Monday Dev Heaven] Nuxeo-lang-ext-assistant, a WebEngine site here to help you translate Nuxeo


Tue 24 July 2012 By Laurent Doguin

As you may know, we're in the process of a releasing Nuxeo 5.6. Feedbacks are welcome :-)

The good thing about the code freeze is that the different labels we have are very unlikely to change. Hence it's the perfect time to start translating Nuxeo in another language or update existing translations.

So I started a small WebEngine project to ease the process. It has really basic features right now. You can select any languages available in nuxeo-platform-lang-ext. You get to see every keys, English labels and labels of the selected language in a table. Those columns have a text filter input to ease navigation through the 2000+ labels. There's also a checkbox that display only empty labels for the selected languages. Fields you have modified are saved to the server automatically so that we can add them to the current messages file. This way you can hot reload them and see the result directly without restarting the server. This will allow translators to work faster. They can also upload an existing message file that will be merged with the existing one. Note that to benefit from HotReload, you have to run Nuxeo in Dev mode. You can download the whole messages file or the diff from nuxeo-platform-lang-ext at any time. There's also a link that removes duplicated keys. Only the last one remains, just like when it's loaded to a properties object in Java.




nuxeo-lang-ext-assistant homepage nuxeo-lang-ext-assistant homepage

Labels list Labels list

You can download and install the marketplace package right here. The sources are on GitHub.

And now I'm gonna show you how it's written so that you can add cool features and send me pull requests later :)

Code diving


It's a basic WebEngine application, I've used the Nuxeo IDE wizard to generate the skeleton. There are mainly three files. A Java WebEngine module and two Freemarker templates. Let's start with the WebEngine home page. Simple web page really, It displays the list of available languages and some explanation on this module using index.ftl. The available languages are fetched by looking directly into the nuxeo.war/WEB-INF/classes of the running server.

<@extends src="base.ftl">
<@block name="header">You signed in as ${Context.principal}<[email protected]>

<@block name="content">

<div style="margin: 10px 10px 10px 10px">

<table>
<tr>
<td width="30%">
<p>Installed Message bundles:</p>
<ul>
<#list availableLanguages as lang>
<li><a href="${This.path}/lang/${lang}">${lang.displayName}</a> (<a href="${This.path}/lang/${lang}/file">File</a>, <a href="${This.path}/lang/${lang}/diff">Diff</a>)</li>
</#list>
</ul>
</td>
<td>
<div style="margin:10px;">
<p>Hello and welcome to Nuxeo's translation assistant. This is currently a Beta version. It lacks certain features, you might find bugs, etc... So don't forget to save your work often by downloading the associated file. You'll be able to reload it using the file input below.</p>
<p>The list of available languages is displayed on the left. Click on any of those to display a table containing the keys, original english labels and selected language labels.</p>
<p>If you're looking for more details on the subject, try <a href="http://dev.blogs.nuxeo.com/2012/06/qa-friday-translating-nuxeo.html" >this blog post</a>.</p>
<form action="${This.path}/upload" method="post" enctype="multipart/form-data">
<p>Upload a message file : <input type="file" name="uploadedFile" size="50" /> <input type="submit" value="Upload It" />
<#if error_message>
<div><span class="errorMessage">${error_message}</span></div>
</#if>
</p>
<p>The selected file must be a valid messages_Lang.properties file. If the locale does not exist, it will be created. Newly create locale are not installed at the moment. This means you won't be able to use hot reload, download the file or the diff for them. To install a new locale, you need to modify the deployment-fragment.xml file in <a href="https://github.com/nuxeo/nuxeo-platform-lang-ext">nuxeo-platform-lang-ext</a>. Insert &lt;supported-locale>LANG&lt;/supported-locale&gt; like the others:</p>
<p>
<pre>
&lt;extension target="faces-config#APPLICATION_LOCALE"&gt;
&ltlocale-config&gt;
&lt;supported-locale&gt;ar&lt;/supported-locale&gt;
&lt;supported-locale&gt;ca&lt;/supported-locale&gt;
&lt;supported-locale&gt;cn&lt;/supported-locale&gt;
&lt;supported-locale&gt;de&lt;/supported-locale&gt;
&lt;supported-locale&gt;el_GR&lt;/supported-locale&gt;
&lt;supported-locale&gt;CUSTOM_LANG&lt;/supported-locale&gt;
&lt;supported-locale&gt;eu&lt;/supported-locale&gt;
&lt;supported-locale&gt;gl&lt;/supported-locale&gt;
&lt;supported-locale&gt;it&lt;/supported-locale&gt;
&lt;supported-locale&gt;sr&lt;/supported-locale&gt;
&lt;supported-locale&gt;vn&lt;/supported-locale&gt;
&lt;/locale-config&gt;
&lt;/extension&gt;
</pre>
</p>
</div>
</td>
</tr>
</table>
</div>

<[email protected]>
<[email protected]>

The main page where translators will spend most of their time is translationForm.ftl. It starts with a bunch of links to get the file or the diff, remove the duplicated keys, get back to the list and a checkbox displaying only empty labels or not. Then comes the big part. The first row of the table display the filters input each binded to a JQuery function that hide the rows which don't correspond to the filter. Then each row displays the key, English label and selected language label. Each time the user has finished typing in the label field, a JSon String is sent to the server to save the new label.

<@extends src="base.ftl">
<@block name="header">You signed in as ${Context.principal}<[email protected]>

<@block name="content">

<div style="margin: 10px 10px 10px 10px">

<form method="post">
<div id="dlLinks">
<ul>
<li><input id="emptyLabelFilter" type="checkbox" name="filter" value="emptyLabel" /> Show Empty Labels Only</li>
<li><a href="${This.path}/lang/${languageKey}/file">Download File</a></li>
<li><a href="${This.path}/lang/${languageKey}/diff">Download Git Diff</a></li>
<li><a href="${This.path}/lang/${languageKey}/removeDuplicatedKeys">Remove Duplicates Keys</a></li>
<li><a href="${This.path}">Back To List</a></li>
</ul>
</div>
<#if languageProperties>
<table style="width=100%" id="formTable">
<tr class="rowStyle">
<td style="padding-left:5px;padding-top:5px;"><h3>Key: <input id="keyFilterInput" type="text" name="keyFilter"/></h3></td><td style="padding-left:5px;padding-top:5px;" ><h3>Label en: <input id="labelFilterInput" type="text" name="labelFilter"/> </h3></td><td style="padding-left:5px;padding-top:5px;" ><h3>Label ${languageKey}: <input id="langFilterInput" type="text" name="langFilter"/> </h3></td>
</tr>
<#list sortedKeys as prop>
<tr class="keyLabel rowStyle" id="${prop}">
<td class="key keyStyle" ><div style="padding-left:10px;width:30em;word-wrap: break-word;">${prop}</div></td>
<td class="label labelStyle" style="padding-left:10px;">${defaultProperties[prop]?html}</td>
<td class="lang langStyle"><textarea class="target" id="input_${prop}" name="${prop}">${languageProperties[prop]?html}</textarea></td>
</tr>
</#list>
</table>
</#if>
</form>
</div>

<script src="http://api.jquery.com/scripts/events.js"></script>
<script>
function intersect(arr1, arr2) {
var r = [], o = {}, l = arr2.length, i, v;
for (i = 0; i < l; i++) {
o[arr2[i]] = true;
}
l = arr1.length;
for (i = 0; i < l; i++) {
v = arr1[i];
if (v in o) {
r.push(v);
}
}
return r;
}

var delay = (function(){
var timer = 0;
return function(callback, ms){
clearTimeout (timer);
timer = setTimeout(callback, ms);
};
})();

var showEmptyLabelsOnly = false;
var hideValidateLabels = false;
var ev;
$(".target").keypress(function(event) {
if ( event.which == 13 ) {
event.preventDefault();
}
save(event.currentTarget.name);
});

function save(changedId) {
var jsonObj = new Array();
var field, fieldValue;
field = changedId.replace(/(:|.)/g,'$1');
fieldValue = $('#input_'+field).val();
jsonObj.push({id: changedId, value: fieldValue});
data = JSON.stringify(jsonObj);
jQuery.ajax({
type: 'PUT',
contentType: 'application/json',
url: '${This.path}/lang/${languageKey}/update',
dataType: "json",
data: data,
success: function(data, textStatus, jqXHR){
modifiedFields = new Array();
},
error: function(jqXHR, textStatus, errorThrown){
alert('Update error: ' + textStatus);
}
});
}

var keyIdList;
function filterKeyId(event) {
delay(function(){
if ( event.which == 13 ) {
event.preventDefault();
}
keyIdList = new Array();
key = $.trim($('#keyFilterInput').val());
if (key != '') {
parentTR = $("#formTable tr[id*='"+key+"']");
for (var i = 0; i < parentTR.length; i++) {
keyIdList.push(parentTR[i].id);
}
}
filter();
}, 1000 );
}

var labelIdList;
function filterLabelId(event) {
delay(function(){
if ( event.which == 13 ) {
event.preventDefault();
}
labelIdList = new Array();
label = $.trim($('#labelFilterInput').val());
if (label != '') {
parentTR = $("#formTable td.label:contains('"+label+"')").parent();
for (var i = 0; i < parentTR.length; i++) {
labelIdList.push(parentTR[i].id);
}
}
filter();
}, 1000 );
}

var langIdList;
function filterLangId(event) {
delay(function(){
if ( event.which == 13 ) {
event.preventDefault();
}
langIdList = new Array();
lang = $.trim($('#langFilterInput').val());
if (lang != '') {
parentTR = $("#formTable textarea:contains('"+lang+"')").parent().parent();
for (var i = 0; i < parentTR.length; i++) {
langIdList.push(parentTR[i].id);
}
}
filter();
}, 1000 );
}

var emptyLangIdList;
function filterEmptyLangId() {
emptyLangIdList = new Array();
parentTR = $("textarea:empty").parent().parent();
for (var i = 0; i < parentTR.length; i++) {
emptyLangIdList.push(parentTR[i].id);
}
filter();
}

function filter() {
var commonId = new Array();
if (keyIdList && keyIdList.length > 0) {
commonId = keyIdList;
if (langIdList && langIdList.length > 0) {
commonId = intersect(commonId, langIdList);
}
if (labelIdList && labelIdList.length > 0) {
commonId = intersect(commonId, labelIdList);
}
} else if (labelIdList && labelIdList.length > 0) {
commonId = labelIdList;
if (langIdList && langIdList.length > 0) {
commonId = intersect(commonId, langIdList);
}
} else if (langIdList && langIdList.length > 0) {
commonId = langIdList;
}

if (showEmptyLabelsOnly) {
commonId = intersect(commonId,emptyLangIdList)
}

if (commonId && commonId.length > 0) {
$("#formTable tr.keyLabel").hide();
for (var i = 0; i < commonId.length; i++) {
id = commonId[i];
id = id.replace(/./g, '.')
$("#"+ id).show();
}
} else {
if (showEmptyLabelsOnly) {
parentTR = $("textarea:empty").parent().parent().show();
parentTR = $("textarea:parent").parent().parent().hide();
} else {
$("#formTable tr.keyLabel").show();
}
}
}

$('#keyFilterInput').keyup(filterKeyId);
$('#labelFilterInput').keyup(filterLabelId);
$('#langFilterInput').keyup(filterLangId);
$('#emptyLabelFilter').change(function () {
showEmptyLabelsOnly = $(this).is(':checked');
if (showEmptyLabelsOnly) {
filterEmptyLangId();
} else {
emptyLangIdList = new Array();
filterEmptyLangId();
}
});
</script>

<[email protected]>
<[email protected]>

Ant the last but definitely not least important is the Java WebEngine module. Take a close look at the initialize method as it does a lot of work. It starts by looking at your server's data folder. If it contains a nuxeo-platform-lang-ext folder, than it initialize it as a local Git repository using the JGit library (By the way did you know that all our studio project versioning are handled a local git repository and JGit?). If the folder does not exist, it will clone it. Then it takes the messages.properties and messages_en.properties in nuxeo.war/WEB-INF/classes and merge them. This contains all the label keys used in Nuxeo and will be used as base for all the other translations. Then the other languages are loaded in the locales map. Now the module is ready to display the list of installed languages on the home page, get the file and the git diff of any installed language. For Debugging purposes, there's a reset method that clears everything that has been done during initialization as well as all changes made after that. Don't go there unless you know what you're doing. The doGet method displays the index.ftl page while getLang displays translationForm.ftl according to the selected language. The upload method is called when the user uploads a file on the home page. It does a bit of validation like assuring the user uploads a text file, that the filename is valid and that the locale is supported by the server. If the file is valid, it loads its content and merge it with the labels from the properties in the locales map. Then there's the update method that is called using JQuery each time the user enters a new label or modify an existing one. It's simple JAX-RS and JSon stuff. The next method is getLangFile. It's called when you want to download a messages file. It will give you the original file from nuxeo-platform-lang-ext with the current diff applied. Then you have the getDiff method called when the user downloads the diff. It uses JGit to create the diff from the local repository. Finally you have the removeDuplicate method that takes the selected language file from the locale repository and strip the duplicated keys, leaving only the last one in the file. Other methods are here to do some text parsing, writing, messages loading etc... One worth the look would be reloadMessages is used to reload the messages.

/*
* Copyright (c) 2006-2012 Nuxeo SA (http://nuxeo.com/) and others.
*
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v1.0
* which accompanies this distribution, and is available at
* http://www.eclipse.org/legal/epl-v10.html
*
* Contributors:
* ldoguin
*
*/
package org.nuxeo.lang.ext;

import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Enumeration;
import java.util.Formatter;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Properties;
import java.util.Scanner;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.ResponseBuilder;
import javax.ws.rs.core.Response.Status;

import org.apache.commons.lang.LocaleUtils;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.CheckoutConflictException;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.InvalidRefNameException;
import org.eclipse.jgit.api.errors.RefAlreadyExistsException;
import org.eclipse.jgit.api.errors.RefNotFoundException;
import org.eclipse.jgit.diff.DiffFormatter;
import org.eclipse.jgit.dircache.DirCacheIterator;
import org.eclipse.jgit.errors.CorruptObjectException;
import org.eclipse.jgit.errors.NoWorkTreeException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.treewalk.FileTreeIterator;
import org.eclipse.jgit.treewalk.filter.PathFilterGroup;
import org.jboss.seam.log.LogProvider;
import org.jboss.seam.log.Logging;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import org.nuxeo.common.Environment;
import org.nuxeo.common.utils.FileUtils;
import org.nuxeo.ecm.core.api.Blob;
import org.nuxeo.ecm.webengine.forms.FormData;
import org.nuxeo.ecm.webengine.model.Template;
import org.nuxeo.ecm.webengine.model.WebObject;
import org.nuxeo.ecm.webengine.model.impl.ModuleRoot;
import org.nuxeo.runtime.api.Framework;
import org.nuxeo.runtime.reload.ReloadService;

/**
* The root entry for the WebEngine module.
*
* @author ldoguin
*/
@Path("/langExtAssistantRoot")
@Produces("text/html;charset=UTF-8")
@WebObject(type = "LangExtAssistantRoot")
public class LangExtAssistantRoot extends ModuleRoot {

private static final LogProvider log = Logging.getLogProvider(LangExtAssistantRoot.class);

private static final String ORIGINAL_FILE_EXTENSION = ".original";

private static final String MESSAGES_FILENAME = "messages%s.properties";

private static final List<String> nuxeoManagedLanguages = Arrays.asList(
"en", "default", "fr");

private static final Pattern messagesPattern = Pattern.compile(
"(messages_)(.*)(.properties)", Pattern.CASE_INSENSITIVE
| Pattern.DOTALL);

private static final Pattern localPattern = Pattern.compile(
"[a-zA-Z]{2}|[a-zA-Z]{2}_[a-zA-Z]{2}", Pattern.CASE_INSENSITIVE
| Pattern.DOTALL);

private static final String CLASSES_FOLDER_PATH = "src/main/resources/web/nuxeo.war/WEB-INF/classes/";

private static final String LOCAL_PATH = Environment.getDefault().getData().getAbsolutePath()
+ "/nuxeo-platform-lang-ext/";

private static final String ABSOLUTE_GIT_PATH_PATTERN = LOCAL_PATH
+ CLASSES_FOLDER_PATH + MESSAGES_FILENAME;

private static final String GIT_REPO_REMOTE_PATH = "git://github.com/nuxeo/nuxeo-platform-lang-ext.git";

private static final Properties mergedProp = new Properties();

private static final List<String> sortedKeys = new ArrayList<String>();

private static final Map<String, Properties> locales = new HashMap<String, Properties>();

private static Repository localRepo;

private static Git git;

@Override
protected void initialize(Object... args) {
super.initialize(args);
if (mergedProp.isEmpty()) {
try {
File localGitRepo = new File(LOCAL_PATH);
if (!localGitRepo.exists()) {
Git.cloneRepository().setURI(GIT_REPO_REMOTE_PATH).setDirectory(
localGitRepo).call();
}
git = Git.open(localGitRepo);
localRepo = git.getRepository();
Properties enProp = loadProperties("en",
String.format(MESSAGES_FILENAME, "_en"));
Properties defaultProp = loadProperties("default",
String.format(MESSAGES_FILENAME, ""));
mergedProp.putAll(defaultProp);
mergedProp.putAll(enProp);
Enumeration<Object> keys = mergedProp.keys();
while (keys.hasMoreElements()) {
sortedKeys.add((String) keys.nextElement());
}
Collections.sort(sortedKeys);
URL classesUrl = getClass().getResource("/");
File f = new File(classesUrl.toURI());
String[] messagesFiles = f.list();
for (String messagesFile : messagesFiles) {
Matcher m = messagesPattern.matcher(messagesFile);
if (m.matches()) {
String languageKey = m.group(2);
loadProperties(languageKey, messagesFile);
}
}
} catch (Exception e) {
throw new RuntimeException(
"could not initialize webengine module " + this, e);
}
}
}

@GET
@Path("reset")
public Object reset() throws RefAlreadyExistsException,
RefNotFoundException, InvalidRefNameException,
CheckoutConflictException, GitAPIException {
mergedProp.clear();
sortedKeys.clear();
locales.clear();
if (git != null) {
git.checkout().setAllPaths(true).call();
git = null;
localRepo = null;
}
initialize();
return Response.ok().build();
}

@GET
public Object doGet() {
Set<String> localesKeySet = locales.keySet();
// remove language manage by nuxeo
localesKeySet.removeAll(nuxeoManagedLanguages);
List<Locale> localeList = new ArrayList<Locale>(localesKeySet.size());
try {
for (String key : localesKeySet) {
Locale locale = LocaleUtils.toLocale(key);
localeList.add(locale);
}
return getView("index").arg("availableLanguages", localeList);
} catch (Exception e) {
return Response.serverError().build();
}
}

@GET
@Path("lang/{languageKey}")
public Object getLang(@PathParam("languageKey") String languageKey) {
if (!isKeyValid(languageKey)) {
return Response.status(404).build();
} else {
return getView("translationForm").arg("sortedKeys", sortedKeys).arg(
"defaultProperties", mergedProp).arg("languageProperties",
locales.get(languageKey)).arg("languageKey", languageKey);
}
}

@POST
@Path("upload")
@Consumes({ MediaType.MULTIPART_FORM_DATA })
public Object uploadMessageFile() throws IOException, JSONException {
FormData form = ctx.getForm();
Blob messageFile = form.getBlob("uploadedFile");
String fileName = messageFile.getFilename();
Matcher m = messagesPattern.matcher(fileName);
if (m.matches()) {
String languageKey = m.group(2);
if (nuxeoManagedLanguages.contains(languageKey)) {
Template template = (Template) doGet();
return template.arg("error_message",
"You cannot update file for default locale like en or fr.");
}
// Verify if the locale is supported by the server
try {
Locale locale = LocaleUtils.toLocale(languageKey);
} catch (Exception e) {
Template template = (Template) doGet();
return template.arg("error_message",
"Could not identify the locale.");
}
Properties properties = new Properties();
properties.load(messageFile.getStream());
Properties existingProperties = locales.get(languageKey);
if (existingProperties != null) {
// merge uploaded properties with the old one
Properties finalProperties = new Properties();
finalProperties.putAll(existingProperties);
finalProperties.putAll(properties);
locales.put(languageKey, finalProperties);
} else {
locales.put(languageKey, properties);
}
return doGet();
} else {
Template template = (Template) doGet();
return template.arg("error_message",
"Given file name was not like messages_LANG.properties");
}
}

@PUT
@Path("lang/{languageKey}/update")
@Consumes({ MediaType.APPLICATION_JSON })
public Object update(@PathParam("languageKey") String languageKey)
throws IOException, JSONException {
if (!isKeyValid(languageKey)
|| nuxeoManagedLanguages.contains(languageKey)) {
return Response.status(404).build();
} else {
Properties props = locales.get(languageKey);
if (props == null) {
return Response.serverError().build();
}
String content = new java.util.Scanner(
this.request.getInputStream()).useDelimiter("A").next();
JSONArray modifiedFields = new JSONArray(content);
for (int i = 0; i < modifiedFields.length(); i++) {
JSONObject field = (JSONObject) modifiedFields.get(i);
props.put(field.getString("id"), field.getString("value"));
}
locales.put(languageKey, props);
persistMessages(languageKey);
reloadMessages();
return "";
}
}

@GET
@Path("lang/{languageKey}/file")
@Produces(MediaType.TEXT_PLAIN)
public Response getLangFile(@PathParam("languageKey") String languageKey)
throws FileNotFoundException, IOException {
if (!isKeyValid(languageKey)) {
return Response.status(404).build();
} else {
if (locales.get(languageKey) != null) {

String filePath = getGitRepoAbsolutePath("_" + languageKey);
File f = new File(filePath);
ResponseBuilder response = Response.ok(f);
response.type("text/plain");
response.header("Content-Disposition",
"attachment; filename="" + f.getName() + """);
return response.build();
}
return Response.status(Status.NOT_FOUND).build();
}
}

@GET
@Path("lang/{languageKey}/diff")
@Produces(MediaType.TEXT_PLAIN)
public Response getDiff(@PathParam("languageKey") String languageKey)
throws FileNotFoundException, IOException {
if (!isKeyValid(languageKey)) {
return Response.status(404).build();
} else {
if (locales.get(languageKey) != null) {
addToRepo(languageKey);
String diff = getRepoDiff(languageKey);
String fileName = "messages_" + languageKey
+ ".properties.diff";
ResponseBuilder response = Response.ok(diff);
response.type("text/plain");
response.header("Content-Disposition",
"attachment; filename="" + fileName + """);
return response.build();
}
return Response.status(Status.NOT_FOUND).build();
}
}

@GET
@Path("lang/{languageKey}/removeDuplicatedKeys")
public Object removeDuplicate(@PathParam("languageKey") String languageKey)
throws FileNotFoundException, IOException {
if (!isKeyValid(languageKey)) {
return Response.status(404).build();
} else {
if (locales.get(languageKey) != null) {
removeDuplicateKeys(languageKey);
return getLang(languageKey);
}
return Response.status(Status.NOT_FOUND).build();
}
}

private String getRepoDiff(String languageKey) throws NoWorkTreeException,
CorruptObjectException, IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
String filePath = CLASSES_FOLDER_PATH + "messages_" + languageKey
+ ".properties";

DiffFormatter df = new DiffFormatter(out);
df.setRepository(localRepo);
df.setPathFilter(PathFilterGroup.createFromStrings(filePath));
DirCacheIterator oldTree = new DirCacheIterator(
localRepo.readDirCache());

FileTreeIterator newTree = new FileTreeIterator(localRepo);

df.format(oldTree, newTree);
df.flush();
df.release();
String diff = out.toString("utf-8");
return diff;
}

public void addToRepo(String languageKey) throws IOException {
String commitFile = getGitRepoAbsolutePath("_" + languageKey);
File originalFile = new File(commitFile);
File diffFile = File.createTempFile("messages", "properties");
Properties props = locales.get(languageKey);
if (!originalFile.exists()) {
originalFile.createNewFile();
}
Scanner freader = new Scanner(originalFile);
BufferedWriter writer = new BufferedWriter(new FileWriter(diffFile,
false));

String line = null;
String key = null;
String label = null;
String propertiesLabel = null;
String[] keyLabelPair = null;
StringBuffer sb = null;
String encodedPropertiesLabel;
while (freader.hasNextLine()) {
line = freader.nextLine();
if (line.startsWith("#")) {
// ignore comments
writer.write(line);
writer.newLine();
} else {
if (line.indexOf("=") == line.length() - 1 && line.length() > 2) {
key = line.substring(0, line.length() - 1);
if (props.get(key) != null && !"".equals(props.get(key))) {
propertiesLabel = (String) props.get(key);
} else {
propertiesLabel = null;
}
if (propertiesLabel != null) {
encodedPropertiesLabel = escapeUnicode(propertiesLabel);
sb = new StringBuffer();
sb.append(key);
sb.append("=");
sb.append(encodedPropertiesLabel);
writer.write(sb.toString());
writer.newLine();
} else {
// no difference
writer.write(line);
writer.newLine();
}

} else {
keyLabelPair = line.split("=", 2);
if (keyLabelPair.length != 2) {
// Something's wrong
writer.write(line);
writer.newLine();
} else {
key = keyLabelPair[0];
label = keyLabelPair[1];
if (props.get(key) != null) {
propertiesLabel = (String) props.get(key);
} else {
propertiesLabel = null;
}
if (propertiesLabel != null) {
encodedPropertiesLabel = escapeUnicode(propertiesLabel);
if (!encodedPropertiesLabel.equalsIgnoreCase(label)) {
sb = new StringBuffer();
sb.append(key);
sb.append("=");
sb.append(encodedPropertiesLabel);
writer.write(sb.toString());
writer.newLine();
} else {
// no difference
writer.write(line);
writer.newLine();
}
} else {
// no difference
writer.write(line);
writer.newLine();
}
}
}
}
}

freader.close();
writer.close();
FileUtils.copy(diffFile, originalFile);
}

private void removeDuplicateKeys(String languageKey) throws IOException {
String commitFile = getGitRepoAbsolutePath("_" + languageKey);
File originalFile = new File(commitFile);
File diffFile = File.createTempFile("messages", "properties");
if (!originalFile.exists()) {
originalFile.createNewFile();
}
Scanner freader = new Scanner(originalFile);

Map<String, List<Integer>> lines = new HashMap<String, List<Integer>>();
Integer lineIdx = 0;
String line = null;
String key = null;
String[] keyLabelPair = null;
while (freader.hasNextLine()) {
lineIdx++;
line = freader.nextLine();
if (!line.startsWith("#")) {
if (line.indexOf("=") == line.length() - 1 && line.length() > 2) {
key = line.substring(0, line.length() - 1);
} else {
keyLabelPair = line.split("=", 2);
if (keyLabelPair.length == 2) {
key = keyLabelPair[0];
}
}
}
if (key != null) {
List<Integer> keyOccurences = lines.get(key);
if (keyOccurences == null) {
keyOccurences = new ArrayList<Integer>();
}
keyOccurences.add(lineIdx);
lines.put(key, keyOccurences);
}
key = null;
}
freader.close();

freader = new Scanner(originalFile);
BufferedWriter writer = new BufferedWriter(new FileWriter(diffFile,
false));
lineIdx = 0;
while (freader.hasNextLine()) {
lineIdx++;
line = freader.nextLine();
if (!line.startsWith("#")) {
if (line.indexOf("=") == line.length() - 1 && line.length() > 2) {
key = line.substring(0, line.length() - 1);
} else {
keyLabelPair = line.split("=", 2);
if (keyLabelPair.length == 2) {
key = keyLabelPair[0];
}
}
}
if (key != null) {
List<Integer> keyOccurences = lines.get(key);
if (keyOccurences.size() > 1) {
keyOccurences.remove(lineIdx);
lines.put(key, keyOccurences);
} else {
// no difference
writer.write(line);
writer.newLine();
}
} else {
writer.write(line);
writer.newLine();
}
key = null;
}
freader.close();
writer.close();
FileUtils.copy(diffFile, originalFile);
}

public String escapeUnicode(String input) {
StringBuilder b = new StringBuilder(input.length());
Formatter f = new Formatter(b);
for (char c : input.toCharArray()) {
if (c < 128) {
b.append(c);
} else {
f.format("u%04x", (int) c);
}
}
return b.toString();
}

private Properties loadProperties(String key, String filePath) {
if (locales.get(key) == null) {
Properties properties = new Properties();
InputStream is = getClass().getResourceAsStream("/" + filePath);
if (null != is) {
try {
properties.load(is);
} catch (IOException ioe) {
throw new RuntimeException(ioe);
}
}
locales.put(key, properties);
}
return locales.get(key);
}

private void persistMessages(String languageKey)
throws FileNotFoundException, IOException {
Properties messages = locales.get(languageKey);
if (messages != null) {
String fileName = String.format(MESSAGES_FILENAME, "_"
+ languageKey);
URL url = getClass().getResource("/" + fileName);
File f = new File(url.getFile());
if (f.exists()) {
backupOriginalFile(f);
// Overwrite existing file
Properties props = locales.get(languageKey);
props.store(new FileOutputStream(f), null);
}
}
}

private String getGitRepoAbsolutePath(String languageKey) {
return String.format(ABSOLUTE_GIT_PATH_PATTERN, languageKey);

}

private void backupOriginalFile(File f) throws IOException {
String filePath = f.getAbsolutePath();
String originalFilePath = filePath.concat(ORIGINAL_FILE_EXTENSION);
File originalFile = new File(originalFilePath);
if (!originalFile.exists()) {
// no exisiting backup
FileUtils.copy(f, originalFile);
}
}

private void reloadMessages() {
if (Framework.isDevModeSet()) {
ReloadService srv = Framework.getLocalService(ReloadService.class);
try {
srv.flush();
} catch (Exception e) {
log.error("Error while flushing the application in dev mode", e);
}
}
}

private boolean isKeyValid(String key) {
Matcher m = localPattern.matcher(key);
if (m.matches()) {
return true;

}
return false;
}
}

In the future I'd like to add other functionalities:


  • Add a checkbox to say if a field has been validated by the translator or not

  • Add the corresponding filter

  • Send the diff by email

  • Or add a Github pull request button :)


If you have any other ideas, feel free to open a Jira or comment this blog :-)
That's it for today! This might be the last blog (at least written by me) on the developer blog until september. I will work on some other cool stuff for our developer community and take some vacations. See you in September!


Category: Product & Development
Tagged: Java, Monday Dev Heaven, Nuxeo Community