Documentation

This repository includes several documents describing how to perform internal tasks within the l10n-drivers team at Mozilla.

Topics covered:

How to read these documents

You will need a Unix-like environment (Linux, macOS) to run most of the tools. Across the entire documentation, commands to run in a terminal are described like this:

$ composer update

$ is not part of the command, it just indicates the terminal prompt. The command to type or copy and paste is simply composer update.

In several occasions documents use aliases instead of the complete path to commands: for example lang_update instead of path_to_langchecker_clone/app/scripts/lang_update. Check Setting up a Linux Virtual Machine for Webdashboards for instructions on how to setup a virtual machine running on Linux Ubuntu with all the necessary packages and aliases.

Updating the documentation

Please see the README file in our GitHub repository.

L10n and android-l10n

Some documentation specific for common tasks regarding android-l10n and l10n:

Adding new locales to specific projects

These are instructions in order to add new locales to Android mobile products living within the android-l10n project.

Edit the l10n.toml file(s)

All android-l10n products live within the android-l10n project. Depending on the product that requires a new locale to be added, the corresponding l10n.toml file will have to be edited by adding the new locale code in it.

The l10n.toml files live in their corresponding project folder, located here for:

Each l10n.toml file typically looks like this:

basepath = "."

locales = [
    "ab-CD",
    "an",
    "ar",
    "as",
    "ast",
    "az",
...
]

# Expose the following branches to localization
# Changes to this list should be announced to the l10n team ahead of time.
branches = [
    "master",
]

[env]

[[paths]]
  reference = "app/src/main/res/values/strings.xml"
  l10n = "app/src/main/res/values-{android_locale}/strings.xml"

Identify the locales section, add the desired new locale code in this list. With Atom and the Sort Lines package installed, you can press F5 to make sure that the list is in alphabetical order.

Save your edited file, then commit and push from your branch to the android-l10n repository.

Note that all products except Firefox for Fire TV share strings with Android Components, so you will also have to edit the Android Components l10n.toml file and add the new locale there.

Add new locales to Pontoon

Once the patch has landed, the new locale has to be added in Pontoon as well. The steps to follow can be found in the existing Pontoon documentation here.

Working with App Stores

Some documentation specific for common tasks on App Stores

For documentation about Webdashboard and Langchecker, see this page.

Firefox for Android - Manage a new release

When creating a new release, normally the What’s new section needs to be moved from Beta to Release, and there are new strings to expose for the Beta channel.

In this example:

  • What’s new strings for the current Release is stored in fx_android/whatsnew/android_46.lang.
  • What’s new strings for the current Beta is stored in fx_android/whatsnew/android_47.lang.

The actions to perform are:

  • Create a new file fx_android/whatsnew/android_48.lang for the upcoming Beta version.
  • Update comment references in fx_android/whatsnew/android_47.lang, moving from Beta to Release.
  • Track the files in both Langchecker and Stores apps.

This document assumes that you have set up the system as explained in this document, so aliases like lang_update are available, repositories are already cloned in ~/mozilla/repositories, Atom is available and includes the syntax highlighter for LANG files.

IMPORTANT: Remember to run gitup before you do anything. If you run the command in the middle of the work, don’t forget to go back to the branch (it will checkout master for all repositories).

Create the new .lang files for Beta and Release

For all changes you need to create branches and then open pull requests.

In this case you’re creating a branch beta48 in the appstores repository, and editing in Atom the reference en-US file.

$ cd ~/mozilla/repositories/appstores/
$ git branch beta48
$ git checkout beta48
$ atom en-US/fx_android/whatsnew/android_48.lang

This is the content of the new file (strings are communicated by Release Driver, about a week before the release date).

## NOTE: These strings are displayed on Google Play in the What’s new section for Firefox 48 beta
## NOTE: See https://l10n.mozilla-community.org/stores_l10n/locale/fx_android/beta/


;Content Notifications for infrequent websites
Content Notifications for infrequent websites


;Suggest “Add to home screen” for frequently used websites
Suggest “Add to home screen” for frequently used websites


;Migrate reading list to bookmarks
Migrate reading list to bookmarks

Now update the initial content of en-US/fx_android/whatsnew/android_47.lang, replacing references to beta with release. For example, from:

## NOTE: These strings are displayed on Google Play in the What’s new section for Firefox 47 beta
## NOTE: See https://l10n.mozilla-community.org/stores_l10n/locale/fx_android/beta/

to:

## NOTE: These strings are displayed on Google Play in the What’s new section for Firefox 47
## NOTE: See https://l10n.mozilla-community.org/stores_l10n/locale/fx_android/release/

These files will be committed later to the repository, since they’re still not tracked in Langchecker.

Track the files in dashboards updating Langchecker

You then need to start tracking these files in Langchecker. The process is described in detail in this document.

Again, you need to create a branch in the langchecker repository.

$ cd ~/mozilla/git/langchecker/
$ git branch beta48
$ git checkout beta48
$ atom app/config/sources.inc.php

At this point you’re ready to add the new files to $appstores_lang. The easiest way is to start by identifying the files used for the previous cycle. For example, if you’re adding 48 Beta and promoting 47 to release, identify files for 47 Beta and 46 Release:

$appstores_lang = [
    ...
    'fx_android/whatsnew/android_46.lang' => [
        'deadline'          => '2016-04-26',
        'supported_locales' => $fx_android_store,
    ],
    'fx_android/whatsnew/android_47.lang' => [
        'supported_locales' => $fx_android_store,
    ],

What you’ll need to do is:

  • Add the new file for 48 Beta (fx_android/whatsnew/android_48.lang).
  • Remove the deadline from the previous release and add it to the new file.
$appstores_lang = [
    ...
    'fx_android/whatsnew/android_46.lang' => [
        'supported_locales' => $fx_android_store,
    ],
    'fx_android/whatsnew/android_47.lang' => [
        'deadline'          => '2016-06-07',
        'supported_locales' => $fx_android_store,
    ],
    'fx_android/whatsnew/android_48.lang' => [
        'supported_locales' => $fx_android_store,
    ],

The previous release (46) is not obsolete yet, you can mark it as such the next time you update the file. The result would be:

$appstores_lang = [
    ...
    'fx_android/whatsnew/android_46.lang' => [
        'flags' => [
            'obsolete' => ['all'],
        ],
        'supported_locales' => $fx_android_store,
    ],

As a general rule, there should be only two non obsolete release files near the end of the cycle.

Now you can commit your changes to Langchecker. Always check with git status to confirm that you’re only including changes to sources.inc.php.

$ cd ~/mozilla/git/langchecker/
$ git add app/config/sources.inc.php
$ git commit -m "Track AppStores files for 47 and 48 beta"
$ git push origin beta48

Commit .lang files to the appstores repository

Now you’re ready to copy the new files to all locales, and propagate all comment changes to fx_android/whatsnew/android_47.lang:

$ lang_update all 12 all

Check your local installation of langchecker for errors by visiting http://localhost/langchecker/?action=errors

If there are no errors, check the status of this repository with git status, and the content of the new files for at least one locale, to confirm that strings were imported correctly.

$ cd ~/mozilla/repositories/appstores/
$ git add .
$ git commit -a -m "Add new Google Play files for Firefox 47 and 48 Beta"
$ git push origin beta48

Note that you need to explicitly add the files with git add, since most of them are not tracked yet.

Updating the list of supported locales

Before you start working on stores_l10n, remember to run the app/scripts/update_shipping_locales.py script in the ~/mozilla/git/stores_l10n/ directory, as this project is used to keep track of which locales ship in each version of supported apps (Firefox for Android/iOS, Focus for Android/iOS). You can use the same branch for running the script, then adding strings to templates (explained in the next section).

In fact, as a general rule, you should periodically run this script - especially before starting to work on stores_l10n. If the script changes the list of locales:

  • Run lang_update all 12 all to add missing files in the appstores repository.
  • Check in Pontoon if the new locales already have the Appstores project enabled. If not, you will have to add the locale in the Admin interface, but only after files have been added to the repository, i.e. all branches have been merged.

Add strings to templates in stores_l10n

Now you need to use the new files and strings in stores_l10n. Again, you’re going to work on a branch.

$ cd ~/mozilla/git/stores_l10n/
$ git branch beta48
$ git checkout beta48
$ atom app/classes/Stores/Project.php

The variable to update is $template. From:

public $templates = [
    'fx_android' => [
        'release' => [
            'template' => 'fx_android/release/listing_apr_2016.php',
            'langfile' => 'fx_android/description_release.lang',
            'whatsnew' => 'fx_android/whatsnew/android_46.lang',
            ],
        'beta' => [
            'template' => 'fx_android/beta/listing_may_2015.php',
            'langfile' => 'fx_android/description_beta.lang',
            'whatsnew' => 'fx_android/whatsnew/android_47.lang',
            ],
    ],

To:

public $templates = [
    'fx_android' => [
        'release' => [
            'template' => 'fx_android/release/listing_apr_2016.php',
            'langfile' => 'fx_android/description_release.lang',
            'whatsnew' => 'fx_android/whatsnew/android_47.lang',
            ],
        'beta' => [
            'template' => 'fx_android/beta/listing_may_2015.php',
            'langfile' => 'fx_android/description_beta.lang',
            'whatsnew' => 'fx_android/whatsnew/android_48.lang',
            ],
    ],

Then you need to update app/templates/fx_android/release/listing_apr_2016.php with the strings for Release, and app/templates/fx_android/beta/listing_may_2015.php with the strings for Beta.

At this point open the local installation of stores_l10n available at http://localhost/stores_l10n/ and check both Release and Beta channel. If everything looks good, you can commit and push.

$ git commit -a -m "Add new Google Play files for Firefox 47 and 48 Beta"
$ git push origin beta48

Now you’re ready to open pull requests for each of the three involved repositories.

If you’re using the l10n-drivers VM, both langchecker and stores_l10n are forks, so you’ll find them in your user account, e.g. https://github.com/flodolo/langchecker/. appstores, on the other hand, is a direct clone of the mozilla-l10n repository.

Firefox for iOS - Manage a new release

When creating a new release, normally there’s a new What’s new section for the upcoming version.

Assuming the new version is 6.0, the actions to perform are:

  • Create a new file fx_ios/whatsnew/ios_6_0.lang.
  • Track the file in both Langchecker and Stores apps.

This document assumes that you have set up the system as explained in this document, so aliases like lang_update are available, repositories are already cloned in ~/mozilla/repositories, Atom is available and includes the syntax highlighter for LANG files.

Create the .lang file for the new release

For all changes you need to create branches and then open pull requests.

In this case you’re creating a branch ios6_0 in the appstores repository, and editing in Atom the reference en-US file.

$ cd ~/mozilla/repositories/appstores/
$ git branch ios6_0
$ git checkout ios6_0
$ atom en-US/fx_ios/whatsnew/ios_6_0.lang

This is the content of the new file (strings are communicated by releng).

## NOTE: These strings are displayed on App Store in the What’s new section for Firefox for iOS 6.0
## NOTE: See https://l10n.mozilla-community.org/stores_l10n/locale/fx_ios/release/

;Automatically open web links in Firefox from mail apps such as Outlook, Airmail, Mail.Ru, myMail and Spark. Also, make one of these mail services your default mail app when sending emails from Firefox.
Automatically open web links in Firefox from mail apps such as Outlook, Airmail, Mail.Ru, myMail and Spark. Also, make one of these mail services your default mail app when sending emails from Firefox.

;Fixed other bugs
Fixed other bugs


;App optimization improvements
App optimization improvements

This file will be committed later to the repository, since it’s still not tracked in langchecker.

Track the new file in dashboards updating Langchecker

You then need to start tracking this file in Langchecker. The process is described in detail in this document.

Again, you need to create a branch in the langchecker repository.

$ cd ~/mozilla/git/langchecker/
$ git branch ios6_0
$ git checkout ios6_0
$ atom app/config/sources.inc.php

At this point you’re ready to add the new file to $appstores_lang. The easiest way is to start by identifying the file used for the previous cycle. For example, if you’re adding Firefox for iOS 6.0, identify the file used for 5.0:

$appstores_lang = [
    ...
    'fx_ios/whatsnew/ios_5_0.lang' => [
        'deadline'          => '2016-07-27',
        'supported_locales' => $fx_ios_store,
    ],

What you’ll need to do is:

  • Add the new file for 6.0 (fx_ios/whatsnew/ios_6_0.lang) by copying the one used in the previous cycle and updating the version number.
  • Mark the previous file as obsolete by adding to its definition as indicated in the following example. As a general rule, there should be only one obsolete file in total, and the previous obsolete file can be deleted.
'flags' => [
    'obsolete' => ['all'],
],
  • Remove the deadline from the previous file and add it to the new one.
$appstores_lang = [
    ...
    'fx_ios/whatsnew/ios_5_0.lang' => [
        'flags' => [
            'obsolete' => ['all'],
        ],
        'supported_locales' => $fx_ios_store,
    ],
    'fx_ios/whatsnew/ios_6_0.lang' => [
        'deadline'          => '2017-12-20',
        'supported_locales' => $fx_ios_store,
    ],

Now you can commit your changes to Langchecker. Always check with git status to confirm that you’re only including changes to sources.inc.php.

$ cd ~/mozilla/git/langchecker/
$ git add app/config/sources.inc.php
$ git commit -m "Track What's new page for iOS 6.0"
$ git push origin ios6_0

Commit .lang files to the appstores repository

At this point you’re ready to copy the new file to all locales.

$ lang_update all 12 all

Check your local installation of langchecker for errors by visiting http://localhost/langchecker/?action=errors

If there are no errors, check the status of this repository with git status, and the content of the new file for at least one locale.

$ cd ~/mozilla/repositories/appstores/
$ git add .
$ git commit -a -m "Add What's new file for iOS 6.0"
$ git push origin ios6_0

Note that you need to explicitly add the files with git add, since most of them are not tracked yet.

Updating the list of supported locales

Before you start working on stores_l10n, remember to run the app/scripts/update_shipping_locales.py script in the ~/mozilla/git/stores_l10n/ directory, as this project is used to keep track of which locales ship in each version of supported apps (Firefox for Android/iOS, Focus for Android/iOS). You can use the same branch for running the script, then adding strings to templates (explained in the next section).

In fact, as a general rule, you should periodically run this script - especially before starting to work on stores_l10n. If the script changes the list of locales:

  • Run lang_update all 12 all to add missing files in the appstores repository.
  • Check in Pontoon if the new locales already have the Appstores project enabled. If not, you will have to add the locale in the Admin interface, but only after files have been added to the repository, i.e. all branches have been merged.

Add strings to templates in stores_l10n

Now you need to use the new file and strings in stores_l10n. Again, you’re going to work on a branch.

$ cd ~/mozilla/git/stores_l10n/
$ git branch ios6_0
$ git checkout ios6_0
$ atom app/classes/Stores/Project.php

The variable to update is $template. From:

public $templates = [
   ...
   'fx_ios' => [
       'release' => [
           'template' => 'fx_ios/release/listing_sept_2015.php',
           'langfile' => 'fx_ios/description_release.lang',
           'whatsnew' => 'fx_ios/whatsnew/ios_5_0.lang',
       ],
   ],

To:

public $templates = [
   ...
   'fx_ios' => [
       'release' => [
           'template' => 'fx_ios/release/listing_sept_2015.php',
           'langfile' => 'fx_ios/description_release.lang',
           'whatsnew' => 'fx_ios/whatsnew/ios_6_0.lang',
       ],
   ],

Then you need to update app/templates/fx_ios/release/listing_sept_2015.php. Search for the $whatsnew variable, and replace the old strings between straight quotes, removing or adding new lines as necessary.

$whatsnew = function ($translations) use ($_) {
    return <<<OUT
• {$_('Automatically open web links in Firefox from mail apps such as Outlook, Airmail, Mail.Ru, myMail and Spark. Also, make one of these mail services your default mail app when sending emails from Firefox.')}
• {$_('Fixed other bugs')}
• {$_('App optimization improvements ')}
OUT;
};

At this point open the local installation of stores_l10n available at http://localhost/stores_l10n/ and check the Apple Appstore Release tab. If everything looks good, you can commit and push.

$ git commit -a -m "Add What's new file for iOS 6.0"
$ git push origin ios6_0

Now you’re ready to open pull requests for each of the three involved repositories.

If you’re using the l10n-drivers VM, both langchecker and stores_l10n are forks, so you’ll find them in your user account, e.g. https://github.com/flodolo/langchecker/. appstores, on the other hand, is a direct clone of the mozilla-l10n repository.

L10n and Firefox for Android

Some documentation specific for common tasks regarding Firefox for Android and l10n:

Adding a new locale to the single-locale build

Android has two different types of builds: a multi-locale build that ships in Google Play Store, and a single-locale build (containing only one locale) that is mostly used for testing.

The list of locales for which single-locale builds are created is stored inside a file named all-locales within Mozilla’s code repositories. Since the goal is to create Nightly builds, the file will be in mozilla-central: https://hg.mozilla.org/mozilla-central/file/default/browser/locales/all-locales

File a bug to add the new locale

You need to file a bug in Firefox for Android::General requesting the new locale (see for example this bug for Guaraní). You can use this bug template to make things faster (update with the appropriate locale code and language name).

You also want to file a bug to use as tracker (example for gn), adding an alias like fm-l10n-gn for gn and blocking fm-l10n-tracker.

Land initial content in l10n repository

With cross-channel, all builds of Firefox and Firefox Android are generated using strings from the l10n-central repository. If a project started localizing using a BitBucket repository, it’s fundamental to populate l10n-central with that content when requesting the official repository creation on https://hg.mozilla.org/l10n-central/LOCALE_CODE.

After the first content lands in l10n-central, it’s a good idea to perform some basic checks before enabling the build:

  • Check toolkit/global/intl.properties (en-US version) for evident mistakes.
  • Check if there’s a region.properties file in mobile/chrome/region.properties. If needed replace it with the stock version.

Set up searchplugins

Check the Set up searchplugins document for detailed instructions on how to set up searchplugins for new locales.

Creating a patch for build configuration

First of all make sure that your environment is correctly set up, and update your local mozilla-unified clone:

$ cd ~/mozilla/mercurial/mozilla-unified
$ hg pull -r default -u
$ hg update central

The file to modify is in mobile/android/locales/all-locales, open it with your text editor of choice.

$ atom mobile/android/locales/all-locales

And add the new locale to the list. With Atom and the Sort Lines package installed, you can press F5 to make sure that the list is in alphabetical order. Let’s say for example that you need to add ab-CD to the list of supported locales.

The second file to modify is mobile/android/locales/l10n.toml. This is the beginning of the file:

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

basepath = "../../.."

locales = [
    "an",
    "ar",
    "as",
    "ast",
    "az",
...

Identify the locales section, and add the new locale code between double quotes, followed by a comma. As before, you can use Atom to make sure the list is in alphabetical order (make sure to select only the lines with actual locale codes before pressing F5).

Once this is done, identify the exclude-multi-locale section. Since this locale is not shipping on multi-locale builds yet, you should add it there too.

After you’ve finished editing the files, check the status of the repository, and the diff.

$ hg status
M mobile/android/locales/all-locales

$ hg diff
--- a/mobile/android/locales/all-locales
+++ b/mobile/android/locales/all-locales
@@ -1,8 +1,9 @@
+ab-CD
 ar
 be
 ca
 cs
 da
 de
 es-AR
 es-ES

 diff --git a/mobile/android/locales/l10n.toml b/mobile/android/locales/l10n.toml
 --- a/mobile/android/locales/l10n.toml
 +++ b/mobile/android/locales/l10n.toml
 @@ -9,8 +9,9 @@ locales = [
 +    "ab-CD",
      "ar",
      "be",
      "ca",
      "cs",
      "da",
      "es-AR",
@@ -9,8 +9,9 @@ exclude-multi-locale = [
  +    "ab-CD",
       "ia",
       "oc",
...

M in hg status indicates that the file has been modified, + in hg diff that the line has been added. Follow the instructions available in this document to create a patch, submit it for review, and land it.

After the patch has landed

A couple of days after the patch has landed check on FTP if builds are being generated for this new locale: https://ftp.mozilla.org/pub/mobile/nightly/latest-mozilla-central-android-api-16-l10n/

If there are no builds present, it’s usually due to errors in the locale, for example straight single or double quotes in Android DTDs. Check the team’s dashboard for these kind of errors.

Adding a new locale to the multi-locales build

Android has two different types of builds: a multi-locale build that ships in Google Play Store, and a single-locale build (containing only one locale) that is mostly used for testing.

The list of locales included in the multi-locales build is stored inside a file named maemo-locales within Mozilla’s code repositories. Since the goal is to create Nightly builds, the file will be in mozilla-central: https://hg.mozilla.org/mozilla-central/file/default/mobile/android/locales/maemo-locales

File a bug to add the new locale

You need to file a bug in Firefox for Android::General requesting the new locale (see for example this bug for Serbian). You can use this bug template to make things faster:

  • Update it with the appropriate locale code and language name.
  • Update the target version for Firefox in comment 0.
  • Set the relnote-firefox flag, e.g.
Release Note Request (optional, but appreciated)
[Why is this notable]: New locale
[Suggested wording]: "Locale added: LANGUAGE (ab-CD)"
[Links (documentation, blog post, etc)]:

Creating a patch for build configuration

First of all make sure that your environment is correctly set up, and update your local mozilla-unified clone:

$ cd ~/mozilla/mercurial/mozilla-unified
$ hg pull -r default -u
$ hg update central

The first file to modify is in mobile/android/locales/maemo-locales, open it with your text editor of choice.

$ atom mobile/android/locales/maemo-locales

And add the new locale to the list. With Atom and the Sort Lines package installed, you can press F5 to make sure that the list is in alphabetical order. Let’s say for example that you need to add ab-CD to the list of supported locales.

The second file to modify is mobile/android/locales/l10n.toml. This is the beginning of the file:

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

basepath = "../../.."

locales = [
    "ab-CD",
    "an",
    "ar",
    "as",
    "ast",
    "az",
...

Identify the locales section, and make sure the locale is already listed under the locales section (it should have been added when enabling single locale builds). In the example above, the locale is represented by ab-CD.

Then locate the exclude-multi-locale section, and remove the locale from this list.

After you’ve finished editing the files, check the status of the repository, and the diff.

$ hg status
M mobile/android/locales/maemo-locales

$ hg diff
--- a/mobile/android/locales/maemo-locales
+++ b/mobile/android/locales/maemo-locales
@@ -1,8 +1,9 @@
+ab-CD
 ar
 be
 ca
 cs
 da
 de
 es-AR
 es-ES

 diff --git a/mobile/android/locales/l10n.toml b/mobile/android/locales/l10n.toml
 --- a/mobile/android/locales/l10n.toml
 +++ b/mobile/android/locales/l10n.toml
 @@ -9,8 +9,9 @@ locales = [
      "ab-CD",
      "ar",
      "be",
      "ca",
      "cs",
      "da",
      "es-AR",
@@ -9,8 +9,9 @@ exclude-multi-locale = [
  -    "ab-CD",
       "ia",
       "oc",
...

M in hg status indicates that the file has been modified, + in hg diff that the line has been added, and - means it was removed. Follow the instructions available in this document to create a patch, submit it for review, and land it.

IMPORTANT: an error in a single locale is going to break the multi-locales build for all locales, English included.

L10n and Firefox desktop

Some documentation specific for common tasks regarding Firefox desktop and l10n:

Adding a new locale to Nightly builds

There are a few steps to complete in order to have Nightly builds for a new locale:

  • File a tracking bug and dependencies.
  • Land initial content in l10n repositories.
  • Set up searchplugins.
  • Add new locale to build configuration.
  • Add new locale to product-details.

File a tracking bug and dependencies to add the new locale

The first step is to create all the bugs you need for this new locale. Let’s consider Lao (lo) as an example:

  • Locale: lo.
  • Language: Lao.
  • Component: leave Other if there’s no component on Bugzilla yet, otherwise type in the component name in the Mozilla Localizations product (in this case lo / Lao).
  • Name: first name of the locale leader. It will be used as part of the bug messages.
  • Bugzilla ID: Bugzilla ID (email address) of the locale leader. It will be used to CC them to all these bugs.
  • Application: leave Firefox.
  • Version: leave the default.

Click the Create Buglinks button, and a set of links will appear at the bottom of the page. Not all of them might be needed, for example LDAP is required only for Tableau access, repositories might already exist, etc. Make sure to always start with the shipping link, since that’s going to be the tracker bug, with an alias used in other bugs to set dependencies (fx-l10n-LOCALECODE, fx-l10n-lo in this example).

Always double check the content of the bugs for errors or outdated content. If the information is obsolete, you can update the templates in this Wiki page (it will require a new deployment of Elmo to production).

Land initial content in l10n repository

With cross-channel, all builds of Firefox are generated using strings from the l10n-central repository. If a project started localizing using a BitBucket repository, it’s fundamental to populate l10n-central with that content when requesting the official repository creation on https://hg.mozilla.org/l10n-central/LOCALE_CODE.

After the first content lands in l10n-central, it’s a good idea to perform some basic checks before enabling the build:

  • Check toolkit/global/intl.properties (en-US version) for evident mistakes.
  • Check if there’s a region.properties file in browser/chrome/browser-region/region.properties, if needed replace it with the stock version.

Set up searchplugins

Check the Set up searchplugins document for detailed instructions on how to set up searchplugins for new locales.

Add new locale to build configuration

First of all make sure that your environment is correctly set up, update your local mozilla-unified clone and make sure it’s on the central bookmark:

$ cd ~/mozilla/mercurial/mozilla-unified
$ hg pull -u
$ hg up central

The first file to modify is browser/locales/all-locales, open it with your text editor of choice.

$ atom browser/locales/all-locales

And add the new locale to the list. With Atom and the Sort Lines package installed, you can press F5 to make sure that the list is in alphabetical order.

The second file to modify is browser/locales/l10n.toml. This is the beginning of the file:

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

basepath = "../.."

locales = [
    "ach",
    "af",
    "an",
    "ar",
    "as",
...

Identify the locales section, and add the new locale code between double quotes, followed by a comma. As before, you can use Atom to make sure the list is in alphabetical order (make sure to select only the lines with actual locale codes before pressing F5).

After you’ve finished editing the files, check the status of the repository, and the diff.

$ hg status
M browser/locales/all-locales
M browser/locales/l10n.toml

$ hg diff
diff --git a/browser/locales/all-locales b/browser/locales/all-locales
--- a/browser/locales/all-locales
+++ b/browser/locales/all-locales
@@ -51,17 +51,16 @@ ja
 kn
 ko
 lij
+lo
 lt
 ltg
 lv

 diff --git a/browser/locales/l10n.toml b/browser/locales/l10n.toml
 --- a/browser/locales/l10n.toml
 +++ b/browser/locales/l10n.toml
 @@ -59,17 +59,16 @@ locales = [
      "kn",
      "ko",
      "lij",
 +    "lo",
      "lt",
      "ltg",
      "lv",

M in hg status indicates that the file has been modified, + in hg diff that the line has been added. Follow the instructions available in this document to create a patch, submit it for review, and land it.

Add new locale to product-details

Product-details, used to publish the build on mozilla.org, is generated by Ship It. Instructions are available in this README. Note that, once reviewed and merged, the change won’t be available until the new version of Ship It is deployed to production.

Adding a new locale to Beta and Release builds

The majority of the work has already been completed when setting up Nightly builds. There are a few prerequisites before adding a new locale to Beta and Release:

  • Localization should have reached a good level of completion, especially for user facing parts of the UI.
  • Localization team has demonstrated a consistent effort, being able to keep up for at least a couple of cycles with the incoming flow of new strings.
  • L10n-drivers have performed testing of the builds on different operative systems without identifying critical issues.
  • There is a relevant number of users on the Nightly channel. This number varies based on the language, for minority languages 10-20 users is a good result. Also, a trend showing growth is important as well.

To track this work, you need to file a bug in Firefox::Build Config (like this example), blocking the original tracking bug for the locale (fx-l10n-LOCALE).

Add locale to build configuration

First of all make sure that your environment is correctly set up, update your local mozilla-unified clone and make sure it’s on the central bookmark:

$ cd ~/mozilla/mercurial/mozilla-unified
$ hg pull -u
$ hg up central

The file to modify is in browser/locales/shipped-locales, open it with your text editor of choice.

$ atom browser/locales/shipped-locales

And add the new locale to the list. With Atom and the Sort Lines package installed, you can press F5 to make sure that the list is in alphabetical order.

After you’ve finished editing the file, check the status of the repository, and the diff. Let’s consider for example the task of adding Urdu (ur).

$ hg status
M browser/locales/shipped-locales

$ hg diff
--- a/browser/locales/shipped-locales
+++ b/browser/locales/shipped-locales
@@ -83,14 +84,13 @@ son
 te
 th
 tr
 uk
+ur
 uz
 vi
 xh
 zh-CN

M in hg status indicates that the file has been modified, + in hg diff that the line has been added. Follow the instructions available in this document to create a patch, submit it for review, and land it.

You also need to set the relnote-firefox flag to ? in the bug, the suggested comment is Suggested for release notes: "Locale added: LANGUAGE_NAME (LOCALE_CODE)".

In order to have builds when the Beta cycle starts, make sure to sign-off the locale on the dashboard (elmo) for central before next merge day, and verify that a sign-off is correctly available on beta before beta1.

Update language pack settings on AMO

The language pack for the new locale will be uploaded automatically by release automation on addons.mozilla.org (starting with the Beta version). A few follow-up manual steps are needed:

  • The Mozilla account needs to be added as author for the language pack.
  • The target_locale field needs to be set for the language pack (via Django admin UI).
  • The locale needs to be defined in this JavaScript file.

For the first two steps, an email needs to be sent to AMO administrators (amo-admins(at)mozilla.org).

These steps will make the language pack available in both the Language Tools page and the API used by Firefox to add new languages from preferences.

Removing a locale from Firefox desktop

Removing a locale from Firefox builds should be the last step in a long series of passages. A locale is removed when:

  • There’s no visible activity in Pontoon for a year.
  • The translation level of Firefox is dropping, causing a good portion of the user interface to appear in English.
  • Outreach to existing community and locale leaders doesn’t produce results.
  • Attempts to find new localizers, via social networks and other tools (e.g. snippets), doesn’t find any potential contributors.

Removing a locale from Nightly

If a locale is only shipping in Nightly, it’s enough to remove the locale from build configuration.

The process is identical to the one use to add a new locale, just removing the locale code from browser/locales/all-locales.

Locale also needs to be removed from Ship It.

Removing a locale from Beta and Release

Removing a locale already shipping in official builds (Beta, Release) is a much more complex process. These are the steps:

Mercurial repositories and Bugzilla components are not removed when dropping a locale.

Given the number of teams and processes involved, a tracking bug should be filed (example).

Check usage statistics

Existing users for this locale need to be moved to another language, otherwise they would remain forever on an obsolete build. The first step is to determine the current usage statistics:

If the locale has a significant userbase (in the hundreds of users), you need to determine the best locale to fall back to. Possible sources of information are:

  • Wikipedia page for the language to determine where it’s spoken, and if there are other languages commonly spoken in the area.
  • Tableau for the language distribution in these countries.

You also need to verify with Business Development if there are contractual obligations regarding search engines.

For example, Belarusian was recently removed from builds:

  • Belarusian is the official language of Belarus. The Wikipedia page also has information about the number of people speaking both Russian and Belarusian in the region.
  • Russian is an official language of Belarus.
  • Russia has a low percentage of population speaking English.

For these reasons, Russian was selected as the best fall back language for Belarusian. Unfortunately, sources might not always point into the same direction, in some cases en-US might be the only viable option.

Redirect existing users to another locale

Once identified the language to fall back to, a bug needs to be filed in Release Engineering::Releases to redirect users via update snippets. An example of such bug is available here.

Remove locale from Beta builds

The process is identical to the one use to add a new locale, just removing the locale code from browser/locales/shipped-locales in Beta. The change will ride the train to Release at the end of the cycle.

Review process

Sign-offs for Firefox desktop and Firefox for Android

Sign-offs are a way for l10n-drivers to indicate that a specific changeset is technically sound and ready to ship in Firefox desktop or Firefox for Android. Currently, thanks to cross-channel, we ship all versions of Firefox from a single localization repository, but sign-offs only happen for the Beta channel:

  • Nightly builds always use the tip of the repository, i.e. the latest changes available.
  • Beta builds only use the signed off version.
  • Release builds uses the latest signed off version from Beta, and can’t normally be updated to use a most recent version.

Sign-offs are tied to a specific app version. For example, Firefox 57 starts in Nightly and doesn’t get any sign-off; it then moves to Beta and get multiple sign-offs as the localization progresses. It then moves to release, and any dot release (e.g. 57.0.1) will use the latest sign-off available for the 57 version, done when it was still in Beta.

Note that sign-offs on Beta are not possible through the entire cycle: sign-offs deadline is normally on Wednesday, about 2 weeks before the release, to allow time for at least one more Beta build before the end of the cycle.

Beta sign-offs are performed on l10n.mozilla.org (elmo) in this page.

How to perform sign-offs

This page only includes sign-offs for Firefox desktop (fx_beta) and Firefox for Android (fennec_beta) beta.

Sign-offs table repository

A green checkmark is displayed in the Status column if the locale has a sign-off, while it’s empty if there are no sign-offs for this locale. This typically happens for brand new locales that are only shipping in Nightly.

The Action column can display several icons:

  • A prohibited icon indicates that there is no activity after the last sign-off (belonging to a previous version), and therefore it’s not possible to do an updated sign-off.
  • A sparkline icon indicates activity that should be checked:
    • Green indicates that the translation is complete (no error, no missing strings).
    • Orange means that the product is still incomplete.
    • Red means that there are errors in the localization. The number of errors and warnings is also reported within the table.

To perform the sign-off, click on the sparkline icon. It will open a new page, showing information about the last sign-off, and the changesets that follows it.

List of changesets

For each changeset, it’s possible to check compare-locale’s results by clicking on the link in the second column from the left. Errors and warnings would be reported in the link too.

Some changesets won’t have a link to compare-locales’s results (e.g. the second row in the image above). That indicates a changeset that doesn’t affect the product you’re currently signing off (e.g. a Thunderbird change in Firefox), and it’s not a good candidate for sign-offs.

By clicking in one of the cells in the rightmost column, by default you can display a diff between the latest changeset and the most recent sign-off. Note that both circles can be dragged, to display the diff between specific changesets. It’s also possible to load more changeset at the bottom of the table using the Load more button.

Diff view

In this view, green means that content has been added, red it’s been removed, while orange indicates a change. Each level of the diff can be collapsed simply by clicking on the same line as the file or folder name.

Once you have reviewed the diff, you can return to the previous page, and sign off the changeset by clicking the sign off… button. To accept the sign-off, simply click the Sign off button:

Requesting and accepting a sign-off

To explicitly reject a sign-off, unselect the Sign off and accept checkbox. Then click on Review… and reject the sign-off with a reason.

Rejecting a sign-off

Things to check when doing a sign-off

The first thing to do is to look at errors and warnings. For warnings, always double check if the number increased from the last sign-off, identify which changeset introduced new warnings, and if they’re acceptable.

Here are a few more things to look out for when doing sign-offs:

  • Strings, or portion of text, remaining in English or becoming empty.
  • Translated keyboard shortcuts (command keys). Unlike access keys, they should remain identical to English.
  • Changes to files that require review, like region.properties.
  • Changes to branding, i.e. translated or transliterated brand names (Mozilla, Firefox).
  • Changes to extensions/spellcheck (dictionaries). We can only ship dictionaries if they have a compatible license. If a dictionary was already present, it’s likely to be OK. A brand new dictionary should land with a bug associated.

Reviewing strings in Firefox desktop and Firefox for Android

Review landed strings

Starting from Firefox 57, all versions of Firefox desktop and Firefox for Android ship by localizing a single repository containining the reference English strings, called gecko-strings. It is generated from strings landing in the code repository for each branch (e.g. mozilla-central and comm-central for Nightly, mozilla-beta and comm-beta for Beta, etc.), and it’s exposed to localization tools like Pontoon.

There is a second repository, gecko-strings-quarantine, used as a buffer to avoid exposing poor strings to the larger audience of localizers.

The localizations for all channels can be found in l10n-central, with a single repository for each locale.

The review process consists of three parts:

  • Review strings landing in mozilla-central. Currently comm-central doesn’t undergo a similar review process.
  • Review strings landing in gecko-strings-quarantine. Currently, the quarantine repository is updated manually every few days.
  • Push reviewed strings to gecko-strings, and start the localization process.

Review strings landing in mozilla-central

You can get the list of changesets touching localized strings in the last 2 days from mozilla-central. Adjust the pushdate part if you want to see more or less days.

There are some irrelevant changesets, like en-US dictionary updates, but the majority of landings are relevant and need to be checked for localization issues.

You need to open each changeset, and identify changed files that are relevant for localization (.properties, .dtd, .ini).

Things to look out for:

  • Unclear strings and missing localization comments: the best way to identify them is to translate the strings, only having the string and comment as context (not the entire file, or the bug). For example: is the word used both a noun and a verb in English? Is the ID clear enough to give context (e.g. buttonLabel)?
  • String changes without new IDs.
  • Duplicated strings.
  • Localization issues, like misused plural forms, unclear comments, etc.

In case of issues, you have two options:

  • Ask sheriffs (via bug or IRC in #sheriffs) to back out the patch.
  • Ask clarifications in the bug, and decide if it’s worth to stop exposing new strings until the issue is fixed.

Review strings landing in gecko-strings-quarantine

The next step is to spot check changes landed in gecko-strings-quarantine. Here are some things to look out for:

  • Check if a changeset is removing strings. This should happen only when a string landed in Nightly and was removed during the same cycle, or at the beginning of a release cycle when a group of strings becomes unused in all shipping versions.
  • Compare the changeset with the original landing in mozilla-central. Each changeset’s header contains a set of references (consider this example), the most important one is X-Channel-Converted-Revision, which links to the original landing in the code repository.

Run compare-locales against gecko-strings

A good next step to check for issues is to run compare-locales against the gecko-strings repository.

First of all make sure that your environment is correctly set up, and update your local mozilla-unified clone.

compare-locales needs to be installed on your system. You can either install a specific release, or clone the hg repository and install it via pip install -e .. You can check that compare-locales is running correctly by checking its version:

$ compare-locales --version
compare-locales 2.1

Let’s assume that:

  • gecko-strings-quarantine is cloned in ~/l10n/gecko-strings-quarantine.
  • mozilla-unified is cloned in ~/src/mozilla-unified, and you checked out the version corresponding to the converted changeset.

Then run

$ compare-locales --unified --full ~/src/mozilla-unified/browser/locales/l10n.toml ~/src/mozilla-unified/mobile/android/locales/l10n.toml ~/l10n gecko-strings-quarantine

If you also have comm-central checked out, you can check Thunderbird and allies with:

$ compare-locales --unified --full -Dmozilla=~/src/mozilla-unified/ ~/src/comm-central/mail/locales/l10n.toml ~/src/comm-central/calendar/locales/l10n.toml ~/src/comm-central/suite/locales/l10n.toml ~/l10n gecko-strings-quarantine

When running these, you should see no errors or warnings. When running them against the central revisions, you should see no missing or changed strings, while having obsolete strings is expected. When running against beta or release revisions, expect to have changed strings, but again, no missing strings.

Note: when running compare-locales against a non-existing locale code, use the --full commandline argument to get all strings in submodules. In particular for gecko-strings, you need that, otherwise you only get the strings in the browser and mobile/android directories.

Run compare-locales against a localization repository

A good next step to check for issues is to run compare-locales against a localization repository frequently updated (Italian and French are good examples).

To run compare-locales against mozilla-unified and Italian, for both desktop and Android, you can run:

$ compare-locales --unified ~/src/mozilla-unified/browser/locales/l10n.toml ~/src/mozilla-unified/mobile/android/locales/l10n.toml ~/l10n it

To run compare-locales against gecko-strings-quarantine and Italian, for both desktop and Android, you can run:

$ compare-locales --unified ~/l10n/gecko-strings-quarantine/_configs/browser.toml ~/l10n/gecko-strings-quarantine/_configs/mobile-android.toml ~/l10n it

Both are really long commands, so it’s convenient to create Bash aliases in ~/.bash_profile for them, e.g.

cmp_moz="compare-locales --unified ~/src/mozilla-unified/browser/locales/l10n.toml ~/src/mozilla-unified/mobile/android/locales/l10n.toml ~/l10n it"
cmp_mozx="compare-locales --unified ~/l10n/gecko-strings-quarantine/_configs/browser.toml ~/l10n/gecko-strings-quarantine/_configs/mobile-android.toml ~/l10n it"

Let’s start with the output of compare-locales against gecko-strings-quarantine: most of the time, it should only report missing strings. There will be obsolete strings only if a string was removed, which is a rare event in cross-channel.

For example, this is the output for a fully localized locale.

$ compare-locales --unified ~/l10n/gecko-strings-quarantine/_configs/browser.toml ~/l10n/gecko-strings-quarantine/_configs/mobile-android.toml ~/l10n it
it:
changed: 9914
changed_w: 52351
keys: 1383
unchanged: 883
unchanged_w: 1085
91% of entries changed

Check the results for duplicated strings and errors. For example, if a new error shows up for a missing variable, it’s likely that a string changed without a new ID and introduced new variables.

The output of compare-locales against mozilla-unified is going to contain a lot of noise, since it includes all strings that are obsolete for mozilla-central, but are still needed for other branches. If you’re interested in only seeing missing strings, i.e. strings that need to be added to the l10n repository, you can grep the results by piping the output to egrep '^\s*\+'.

$ compare-locales --unified ~/src/mozilla-unified/browser/locales/l10n.toml ~/src/mozilla-unified/mobile/android/locales/l10n.toml ~/l10n it | egrep '^\s*\+'

Push reviewed strings to gecko-strings

If there are no issues in gecko-strings-quarantine, the next step is to push changes to gecko-strings and expose content to tools.

One time setup: after you cloned gecko-strings-quarantine on your system, you need to edit its .hg/hgrc file, and add gecko-strings as path. While you only need https for pulling the quarantine repository, you need ssh in order to push updates to gecko-strings.

The content of ~/l10n/gecko-strings-quarantine/.hg/hgrc should be similar to this:

[paths]
default = https://hg.mozilla.org/users/axel_mozilla.com/gecko-strings-quarantine
gecko-strings = ssh://hg.mozilla.org/l10n/gecko-strings

To push the current default to gecko-strings, from the gecko-strings-quarantine folder simply run:

$ hg push -r default gecko-strings

Instead of default, you can also push a specific changeset, e.g.

$ hg push -r 4c05bc050007 gecko-strings

L10n and Firefox for iOS

Some documentation specific for common tasks regarding Firefox for iOS and l10n:

Reviewing strings for a new release of Firefox for iOS

Usually, a new release of Firefox for iOS means an update to strings. If this is the case, a pull request will be made by the iOS team, in order to land these new strings in the mozilla-l10n firefoxios-l10n repository.

At this point, the PR is reviewed by an l10n-driver - most often by the PM in charge of the project.

Let’s go over some of the steps needed over time in order to review correctly strings for a new release.

Reviewing the PR

When you’re getting close to the end of the Firefox for iOS l10n cycle, the iOS team will make a pull request in order to land new strings for the upcoming release.

Let’s consider a past PR, which was done for v6.0: https://github.com/mozilla-l10n/firefoxios-l10n/pull/19

The first thing you’ll want to do is to check the changes to the /en-US folder: https://github.com/mozilla-l10n/firefoxios-l10n/pull/19/commits/edcee6500a4cec7cd40251e0dd2c047a5ffbe3ae

Acceptable changes

Changes to attributes like tool-version and build-num are expected, they happen every time you change the version of Xcode.

Same for the utf-8 -> UTF-8 at the beginning, and version number bumping to 6.0. Note that these should now be automatically translated (so will not appear as new strings for localizers and they will not need to touch them).

Changes to comments () are irrelevant in terms of string updates, so they’re also OK.

Remember also that the script converts Unicode literals into UTF-8 characters. Example: https://github.com/mozilla-l10n/firefoxios-l10n/pull/19/commits/713def82602224556d797b46e3f6d611297ee3e3#diff-42d28e544c7a30099950b7a09a69f2caL2289

As you can see, the &#173; was not lost - it was converted to the invisible UTF-8 character.

Potentially problematic changes (removals)

Then you start scrolling down, checking if the new strings make sense: the first one you find is NSCameraUsageDescription.

You mostly need to pay attention to the strings removed. Double-check each time that iOS team clearly realizes that they won’t be able to release updates to v5 (or any v5.x release) once strings are exported.

For example: Menu.NightModeAction.Title is removed. If it’s used in v5, and the iOS team plans to release an update to v5 in the future, localizations will miss that string.

Quickly review other locales

Once you’ve checked templates, you can pick at least one locale and see what changes: https://github.com/mozilla-l10n/firefoxios-l10n/pull/19/commits/a1c041a012e6996bd8bdaec88f21f3c34fe383b0

Reviewing strings during the cycle

It’s good to keep an eye on the strings landing during the Firefox for iOS cycle. In fact, if you wait to review strings when the PR is already opened, it’s going to be complicated to get a fix in (until Bug 1270855 is resolved).

You should try to periodically check out the strings landing, and try to localize them in your head: how would you translate them? Would you be able to do it without the app? Is the localization comment clear enough?

You might also need to identify the bug that added that string, see if there are screenshots, or ask if the iOS team can provide one. The point below explains how to find that bug.

How to find the bug that introduced a string

Let’s consider an example of new strings for Firefox for iOS, with this past PR.

The ID is NSCameraUsageDescription, the label This lets you take and upload photos It doesn’t have a localization comment, which is bad.

In fact currently, the only strings that can not have localization comments are strings that are located in InfoPlist.strings and Info.plist - see Bug 1277515 for more details. Otherwise, strings should always have localization comments.

So you open the main repository page: https://github.com/mozilla-mobile/firefox-ios

And use the search box at the top, searching for the string ID (or the string): https://github.com/mozilla-mobile/firefox-ios/search?utf8=%E2%9C%93&q=NSCameraUsageDescription

In this case you find two occurrences. The first one is the most interesting, so you open the file: https://github.com/mozilla-mobile/firefox-ios/blob/978bf46bb680291c61c5d21b6dc26472388a374f/Client/en.lproj/InfoPlist.strings

Then use the Blame link on top: https://github.com/mozilla-mobile/firefox-ios/blame/978bf46bb680291c61c5d21b6dc26472388a374f/Client/en.lproj/InfoPlist.strings

On the left, you should (almost always) find the bug reference: https://bugzilla.mozilla.org/show_bug.cgi?id=1312842

Wrapping up your work

So you’ve reviewed the strings during the cycle, and the PR looks good? Wait! Don’t merge it quite yet.

You will need to announce this on dev.l10n mailing list as soon as you merge the PR. Prepare your email in advance. Here’s an example of what it should contain.

Pontoon should get the new strings automatically, but it’s always a good idea to double check just in case.

Working on mozilla.org

Some documentation specific for common tasks on mozilla.org

For documentation about Webdashboard and Langchecker, see this page.

Process overview to expose mozilla.org content for localization

This document describes the tools and processes involved in converting a mozilla.org template in English to a .lang file containing localizable content and relevant information to help with localization.

This is a brief overview of the process

  • In Bedrock, when a page is in the final coding review phase in GitHub and content is finalized, an L10n label is added to the PR, signaling that it is ready for string extraction.
  • Strings are extracted from the identified templates, generating a temporary .lang file. The new strings are manually added to an existing en-US .lang file, or a brand new file added to the mozilla.org l10n repository. A pull request is opened to review the en-US initial content for issues or errors.
  • Another PR is created in the Langchecker repository. Langchecker is the tool used to keep track of all localizable files in the l10n repositories, and contains a set of scripts used to manage the repository.
  • After both PRs are reviewed and approved, a script will be run locally to propagate the en-US content into files in other locales. That means both adding a new file, or adding new strings to existing files.
  • Once merged to the master branch, files will be available on www.mozilla.org repository for localization, and the merge commit will be cherry-picked to bedrock-l10n, the localization production repository. Within 20 minutes, Pontoon will display the status of the new or updated files for all locales.
  • In some cases, an email through a mailing list will be sent out to communicate the localization request. At this point, the L10n tag is also removed from Bedrock’s PR.

Environment

In order to run Langchecker locally, you need a recent version of PHP (ideally 7 and above):

  • Langchecker repository has instructions on how to install the tool. The Wiki has detailed informations on all the available scripts.
  • While Apache is not needed (you can use PHP internal web server), it’s highly recommended.
  • L10n-drivers use a VMWare virtual machine based on Ubuntu LTS. This VM includes a set of Bash aliases and symbolic links designed to simplify the workflow. You can find the script used to set up the VM here, and the list of aliases here.

Step by step details for the entire process

Extract strings from a template in Bedrock

The process is detailed in this document.

Note: you only use this generated .lang file as a reference. You still need to add comments and reorder strings before exposing them for localization.

Prepare the en-US .lang file

It is very important to have a clean, error free .lang file in en-US. Once the file is approved and content is pushed into other locales, if an error is found then, it will take much more effort to correct. If localizers have already started their work, the cleanup is even messier. At that point, it is a fix in one file versus fixes in 100+ files.

Initial review should include checking the file against the page on the testing server and checking against the template. This is to ensure that there are no hard coded strings.

Note: if there is an update to an existing page, it is advised to still generate the .lang file, and copy the new string there instead of copying it from the .html file. This is to make sure, for example, that the syntax used for variable replacements is correct, and remove eventual extra white spaces between words that are not carried over to the .lang file.

Add notes and comments to the extracted file:

Add important information at the top of the file:

Proceed to the content of the file. Add a comment to the following type of strings to provide more context:

  • Page title and description. Typically, a page title string should be translated in a short or catchy way.
  • If the string contains a link, to which URL it points to. This gives context to localizers.
  • If a string is behind a tag, make sure to add the tag bindings.
  • A new string behind a tag makes another string obsolete. Make sure to add an obsolete comment to help localizers prioritize their work, and that old and new strings are next to each other. This will also make removal of old strings easier in further updates.
  • A comment for an alternative to an expression that may not be culturally relevant, or to a pun that doesn’t translate well.
  • Check English content for capitalization, hyphenated word to make sure they are consistent with the Firefox style guide (macOS vs. MacOS, sign in to vs. sign into).

When all is done, create a PR in the www.mozilla.org repo for review. Describe what the PR is for, add references to the Bedrock’s pull requests to add context for the review.

Check for possible errors such as duplicates or unmatched strings before requesting for review. In order to do this, you will need to run Langchecker locally, and make sure that both the l10n repository and Langchecker’s are using the branches you’re working on.

Work with the scripts in Langchecker

Below are some of the common commands and purposes of using Lanchecker.

You use Langchecker for the following purposes:

The following commands are used for some common tasks:

  • lang_update is to populate new and revised content in the en-US into localized files.
  • mark_active will add an ## active ## tag at the beginning of the file after checking that the entire file is translated and no errors are detected. The associate template will be live on production.
  • add_tags tracks the completion of the tagged strings, and add the tag on top of the file when all are translated.

For more details, check here.

This document details the process to update production repo.

Pontoon and Communication

Within 20 minutes, the content change will show up on Pontoon project dashboard. L10n-driver will update project deadline (high level against all other projects) and page level deadline (if applicable) when viewing the entire mozilla.org page status in a given locale.

An email communication about this localization request will be sent out soon after.

Working with Bedrock

This document covers a few tasks and tips regarding Bedrock.

Running Bedrock

You can install Bedrock by following the instructions available in this page.

If you’re using the virtual machine described in this document, Bedrock is already installed in ~/mozilla/git/bedrock.

Move into the main folder, activate the Virtual environment, and run the server

$ cd ~/mozilla/git/bedrock/
$ source venv/bin/activate
$ ./manage.py runserver

In a different terminal window run

$ gulp

At this point Bedrock will be available at http://localhost:8000

In the virtual machine the locale folder is a symbolic link to the mozilla.org trunk l10n repository.

Don’t forget to exit the virtual environment by running deactivate when you’re finished.

Extracting strings from a template

The first part is to identify the template (or templates) you need to extract strings from. For example, assuming that you want to extract strings from bedrock/mozorg/templates/mozorg/contribute/signup.html, you would run from Bedrock’s main folder:

$ cd ~/mozilla/git/bedrock/
$ source venv/bin/activate
(venv) $ ./manage.py l10n_extract bedrock/mozorg/templates/mozorg/contribute/signup.html

If the last command returns any error, try reinstalling the requirements with

$ pip install -r requirements/dev.txt

At this point a .lang file will be created inside /locale/templates with all the strings extracted. The name of the file depends on several factors: for example the template could include another template that specify a specific .lang file so, instead of having locale/templates/mozorg/contribute/signup.lang, you have locale/templates/mozorg/contribute/index.lang.

The suggestion is to always remove the templates folder before doing an extraction, to make sure you start from a clean situation.

Note that you can also use wildcards in the command, e.g. ./manage.py l10n_extract bedrock/mozorg/templates/mozorg/contribute/*.html, but you should try to extract strings only from the template you need.

One final note: you only use this generated .lang file as a reference, you still need to add comments and reorder strings before exposing them for localization.

Notes about templates and l10n

Wrapping strings

<span class="outline-button more">{{ _('Start using the Mozilla Stumbler app') }}</span>

Localizable strings are wrapped like _('This'). That means you can simply copy and paste the string This instead of performing a full extraction.

More complex strings can be wrapped in trans blocks, e.g.

<p class="license">
  {% trans url=url('foundation.licensing.website-content') %}
  Portions of this content are ©1998–{{ current_year }} by individual
  mozilla.org contributors. Content available under
  a <a href="{{ url }}">Creative Commons license</a>.
  {% endtrans %}
</p>

Shared .lang files

There are some shared .lang files that are available across all templates:

  • main.lang
  • download_button.lang

If a string is available in one of these files, it can be safely used by any of the other templates. This is useful, but these files shouldn’t become bigger than necessary.

Loading other .lang files

The add_lang_files directive is used in templates to specify a different .lang file, or include strings from another .lang file. Example above: normally a template called bedrock/mozorg/templates/mozorg/contribute/signup.html should generate a file called mozorg/contribute/signup.lang, but instead it generates a .lang file called mozorg/contribute/index. Why? Because the template has this directive:

{% add_lang_files "mozorg/contribute/index" %}

This is particularly important to understand .lang file activation.

Activating a template

A .lang file is active when it starts with the line ## ACTIVE ##. A file is activated by l10n-drivers when all strings inside this file are translated, and the page can be displayed in production. Example: https://www.mozilla.org/firefox/new uses firefox/new.lang for strings:

  • https://www.mozilla.org/en-US/firefox/new will always work, since it’s en-US.
  • https://www.mozilla.org/fr/firefox/new will work if fr/firefox/new.lang is activated. If it’s not, it will redirect to en-US (or any other active locale before en-US in the accept_languages header).
  • https://www-dev.allizom.org/fr/firefox/new will always work, because files are always active on the dev server.

If you’re running Bedrock locally, you need to change bedrock/settings/local.py to modify the behaviour:

  • DEV = True means that the server is set as DEV, and all locales will be enabled.
  • DEV = False means that the server is set as PROD, and specific pages will work only if a locale has the file activated.

One important fact to keep in mind: if you add a new .lang file with the add_lang_files directive, the activation status will be inherited. Always double check with the production settings.

It’s possible to activate l10n files by using the mark_active script available in Langchecker. It will activate only locales that are completely localized and don’t have any errors.

L10n tags

Bedrock supports l10n tags through the l10n_has_tag function. For example:

{% if l10n_has_tag('sync_device_note') %}
  {{ _('Learn more about bookmarks') }}
{% else %}
  {{ _('Learn more') }}
{% endif %}

l10n tags allow to add a new string in the template and to push it to production without waiting for localization to happen: if the l10n .lang file has this tag, the template will use the new string Learn more about bookmarks, otherwise if will fallback to the previous version Learn more.

The tag appears as sync_device_note in the code, but it will appear as ## sync_device_note ## at the beginning of the .lang file. Using underscores is preferable to dashes for tag names.

It’s possible to add tags to l10n files by using the add tags script available in Langchecker. It will add tags only if the file translated all strings associated to it.

All tags are enabled by default on a DEV server.

For more information about .lang files, for example how to bind a string to a tag, see the wiki page.

Try a patch locally

Let’s assume you want to test a pull request locally, for example to extract strings. The URL for this pull request is https://github.com/mozilla/bedrock/pull/4100. By adding .patch to the URL you get a patch for this PR: https://github.com/mozilla/bedrock/pull/4100.patch.

You can download this patch and apply it locally.

$ cd ~/mozilla/git/bedrock/
$ source venv/bin/activate
(venv) $ wget https://github.com/mozilla/bedrock/pull/4101.patch
(venv) $ git apply 4101.patch
(venv) $ gulp

Remember that Bedrock on the VM-l10n uses a link to the actual trunk repository, so make sure to delete any test you make. The templates folder will be automatically ignored.

Identifying l10n issues in a patch

The most common issue is that strings are not wrapped, so they’re not exposed by the extraction process or translated, even if the string is available in the .lang file. To identify them you need to run the patch locally with a modified .lang file, for example adding asterisks or other characters to the English string. There might be need for a tool to automatically create such a file.

How to read a patch and identify the string? One possible approach is to load the patch (add .patch to the PR URL) in a text editor, and use this regular expression to search for new/changed lines with wrapped strings: ^\+.*_\(.*\). The regular expression can be read as: search for a line starting with + and including a _(…) structure. Note that a change might be generated by a change in padding, so the block moves but the string is always the same.

Normally PRs are tagged as L10N to indicate that the PR has some impact on l10n (new strings), either by devs or l10n-drivers.

Importing translations from existing files

Sometimes it’s useful to import strings from existing files. A typical example is when an existing page is replaced by a new template, with some strings in common.

First of all, always make sure that your repositories are up to date. If you’re using the l10n-drivers VM, simply run gitup.

Let’s consider a simple example: the string Internet Health is currently available in main.lang. A new page is created (mozorg/internet-health.lang), and it includes that string in the template. Since main.lang is a shared file it would be possible to avoid adding the string to the new template, but in this case you want to make sure that localizers can use an alternative translation.

To start check that all locales have the untranslated string available in mozorg/internet-health.lang

$ lang_update mozorg/internet-health.lang 0 all

At this point you need to work on langchecker, editing your local copy of app/scripts/lang_update.

$ atom ~/mozilla/git/app/scripts/lang_update

Search for the variable $import_files and add the file from which you want to import strings (main.lang in this case). Make also sure that $import_website is set to 0 (identifier for mozilla.org).

$import_files = ['main.lang'];
$import_website = 0;

This will make the next run of lang_update import existing strings from main.lang. For this reason, always run lang_update only against the file you wish to update.

Now you can run the update, propagating updates to all locales:

$ lang_update mozorg/internet-health.lang 0 all

Note that $import_files is an array, which means you could have more source files, e.g.

$import_files = ['main.lang', 'mozorg/home/index.lang'];

The order is relevant: files are checked in the order they’re available in the array, the first matching translation is kept.

Important: make sure to not commit this change to langchecker. You can either ignore it and it will be removed the next time you run gitup, or reset the repository with git reset --hard.

$ cd ~/mozilla/git/langchecker
$ git reset --hard

Check your local installation of langchecker for errors by visiting http://localhost/langchecker/?action=errors

If there are no errors, check the status of this repository with git status, and the content of the updated files for at least one locale (with git diff LOCALE), to confirm that strings were imported correctly. Then push to your branch.

It’s a good idea to send localizers an email to explain why the string will be already translated when they start working on the file.

Updating mozilla.org production

For mozilla.org there are two separate repositories:

  • www.mozilla.org is used for trunk localization, it’s accessible to localizers and tools and is used to localize the DEV server.
  • bedrock-l10n is used for production localization of www.mozilla.org

l10n-drivers are in charge of manually moving files from trunk to prod after doing a technical review. On Linux meld is a great tool for this task: you can open both repositories, move files between them and visualize the changes between the two versions.

$ meld path/to/trunk_repository path/to/production_repository

If you’re using the virtual machine described in this document, there are a few shortcuts:

  • trunkst will move into the trunk folder, and check the status (git status).
  • prodst will do the same for production, but will also fetch updates from trunk in case you want to cherry-pick changesets (see next section).
  • mozmeld will open meld with trunk on the left, and production on the right. Use alt+up/down to move to the next change, use alt+right to move from trunk to production. Don’t move the README file, since they’re different between the two repositories.
  • gitup will update all Git repositories, discard pending changing, and checkout master branch.

Cherry-picking changes

Sometimes moving files manually between the two repositories in not viable, for example when updating a file in all locales. In this case it’s possible to cherry-pick a commit from trunk into production.

When you commit a change to the trunk repository you will get a SHA for the changes

$ trunkst
$ git commit -a -m "Update translation"
[master 31c3920] Update translation
 1 file changed, 1 insertion(+)
$ git push

In this case the changeset is 31c3920. From the production repository:

$ prodst
$ git cherry-pick 31c3920
[master 380ebdc] Update translation
 Date: Mon May 2 17:31:06 2016 +0200
 1 file changed, 1 insertion(+)
$ git push

Doing a manual push to production

If you want to do this all manually using Git, you can follow these steps:

  1. Make sure the master branches for your clones of both the trunk and production repositories are up to date.
  2. In your local clone of the production repo, add a remote that points to the trunk repo using git remote add trunk https://github.com/%GITUHB-USERNAME%/www.mozilla.org.git.
  3. Run git checkout master.
  4. Run git fetch trunk.
  5. Now git cherry-pick the commit from the trunk repo, referencing the Git hash of the commit you wish to include (e.g. git cherry-pick 31c3920).
  6. Push the changes to master using git push upstream master (assuming upstream is your remote that points to https://github.com/mozilla-l10n/bedrock-l10n.git).

Guidelines

There are some principles to keep in mind when updating the production repository:

  1. Before moving any file, check the Errors view, possibly on your local installation.
  2. Check the diff for each file before moving it to production, to see if there are any errors.
  3. In case of any doubts always check the page on your local installation of Bedrock or the DEV server, especially for new locales.

Enable a new locale on production for mozilla.org

All locales available in the www.mozilla.org GitHub repository are automatically enabled on the dev server. In order to enable a language for stage and production, it needs to be added to Bedrock’s settings.

Before doing that, make sure that the locale has reached a good level of completion, and test the main pages on the dev server.

First, file a bug to track this change. Bug number will be needed for the commit message later. Consider for example this bug, to add Nepali to production.

Create a new branch

A clone of your fork of Bedrock is already available in the virtual machine, so run gitup to make sure everything is up to date. Then move to Bedrock’s folder, and create a new branch, for example using the bug number as reference.

$ cd ~/mozilla/git/bedrock
$ git branch bug1338759
$ git checkout bug1338759

If you see a message referencing legal-docs like this one, it means that there’s an unwanted change in the legal-docs repository:

$ git checkout bug1338759
M	vendor-local/src/legal-docs
Switched to branch 'bug1338759'

You can verify it by running git status

$ git status
On branch bug1338759
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   vendor-local/src/legal-docs (new commits)

no changes added to commit (use "git add" and/or "git commit -a")

IMPORTANT: you need to get rid of this change before doing any work. If you don’t, your pull request will be rejected.

In order to remove this change, run the following commands

$ cd ~/mozilla/git/bedrock/vendor-local/src/legal-docs/
$ git checkout master
$ git pull
$ cd ~/mozilla/git/bedrock

Make sure you’re back to the bedrock folder in your virtual machine, and check that there are no pending changes with git status.

Edit Bedrock’s settings

The file to modify is bedrock/base/settings.py.

$ cd ~/mozilla/git/bedrock
$ atom bedrock/settings/base.py

Search for PROD_LANGUAGES.

PROD_LANGUAGES = ('ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'bg',
                  'bn-BD', 'bn-IN', 'br', 'bs', 'ca', 'cak', 'cs',
                  'cy', 'da', 'de', 'dsb', 'el', 'en-GB', 'en-US',
                  'en-ZA', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et',
                  'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd',
                  'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb',
                  'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ja-JP-mac',
                  'ka', 'kab', 'kk', 'km', 'kn', 'ko', 'lij', 'lt',
                  'ltg', 'lv', 'mai', 'mk', 'ml', 'mr', 'ms', 'my', 'nb-NO',
                  'nl', 'nn-NO', 'oc', 'or', 'pa-IN', 'pl', 'pt-BR', 'pt-PT',
                  'rm', 'ro', 'ru', 'si', 'sk', 'sl', 'son', 'sq',
                  'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'uk', 'ur',
                  'uz', 'vi', 'xh', 'zh-CN', 'zh-TW', 'zu')

You need to add your locale to the list, making sure to:

  • Respect the alphabetical order.
  • Use one space after commas, no comma after the last element.
  • Maximum 80 characters per line.
  • Enclose the locale code in single quotes.

In Atom there’s a vertical line to display where the 80 characters limit is. If your line is too long, like in this case, you can use a trick to make sure the text fits within the maximum length. First add the new locale code in the correct position, don’t worry about the line length.

PROD_LANGUAGES = ('ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'bg',
                  'bn-BD', 'bn-IN', 'br', 'bs', 'ca', 'cak', 'cs',
                  'cy', 'da', 'de', 'dsb', 'el', 'en-GB', 'en-US',
                  'en-ZA', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et',
                  'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd',
                  'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb',
                  'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ja-JP-mac',
                  'ka', 'kab', 'kk', 'km', 'kn', 'ko', 'lij', 'lt',
                  'ltg', 'lv', 'mai', 'mk', 'ml', 'mr', 'ms', 'my', 'nb-NO', 'ne-NP',
                  'nl', 'nn-NO', 'oc', 'or', 'pa-IN', 'pl', 'pt-BR', 'pt-PT',
                  'rm', 'ro', 'ru', 'si', 'sk', 'sl', 'son', 'sq',
                  'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'uk', 'ur',
                  'uz', 'vi', 'xh', 'zh-CN', 'zh-TW', 'zu')

Add an empty line two lines above and below the line you need to fix (more if needed, i.e. all lines are at near the maximum length).

PROD_LANGUAGES = ('ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'bg',
                  'bn-BD', 'bn-IN', 'br', 'bs', 'ca', 'cak', 'cs',
                  'cy', 'da', 'de', 'dsb', 'el', 'en-GB', 'en-US',
                  'en-ZA', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et',
                  'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd',
                  'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb',
                  'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ja-JP-mac',

                  'ka', 'kab', 'kk', 'km', 'kn', 'ko', 'lij', 'lt',
                  'ltg', 'lv', 'mai', 'mk', 'ml', 'mr', 'ms', 'my', 'nb-NO', 'ne-NP',
                  'nl', 'nn-NO', 'oc', 'or', 'pa-IN', 'pl', 'pt-BR', 'pt-PT',

                  'rm', 'ro', 'ru', 'si', 'sk', 'sl', 'son', 'sq',
                  'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'uk', 'ur',
                  'uz', 'vi', 'xh', 'zh-CN', 'zh-TW', 'zu')

Position the cursor in the block including the line you need to fix, and select Edit -> Reflow Selection. With this command lines are reorganized to fit the 80 characters limit, and this will be the result:

PROD_LANGUAGES = ('ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'bg',
                  'bn-BD', 'bn-IN', 'br', 'bs', 'ca', 'cak', 'cs',
                  'cy', 'da', 'de', 'dsb', 'el', 'en-GB', 'en-US',
                  'en-ZA', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et',
                  'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd',
                  'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb',
                  'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ja-JP-mac',

                  'ka', 'kab', 'kk', 'km', 'kn', 'ko', 'lij', 'lt', 'ltg', 'lv',
                  'mai', 'mk', 'ml', 'mr', 'ms', 'my', 'nb-NO', 'ne-NP', 'nl',
                  'nn-NO', 'oc', 'or', 'pa-IN', 'pl', 'pt-BR', 'pt-PT',

                  'rm', 'ro', 'ru', 'si', 'sk', 'sl', 'son', 'sq',
                  'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'uk', 'ur',
                  'uz', 'vi', 'xh', 'zh-CN', 'zh-TW', 'zu')

You can then remove the empty lines and save the file. These empty lines are needed to avoid affecting more content than needed, in particular the first line. Without them you would end up with this:

PROD_LANGUAGES = ('ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'bg', 'bn-BD',
'bn-IN', 'br', 'bs', 'ca', 'cak', 'cs', 'cy', 'da', 'de', 'dsb', 'el', 'en-GB',
'en-US', 'en-ZA', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et', 'eu', 'fa',
'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd', 'gl', 'gn', 'gu-IN', 'he', 'hi-IN',
'hr', 'hsb', 'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ja-JP-mac', 'ka', 'kab',
'kk', 'km', 'kn', 'ko', 'lij', 'lt', 'ltg', 'lv', 'mai', 'mk', 'ml', 'mr', 'ms',
'my', 'nb-NO', 'ne-NP', 'nl', 'nn-NO', 'oc', 'or', 'pa-IN', 'pl', 'pt-BR',
'pt-PT', 'rm', 'ro', 'ru', 'si', 'sk', 'sl', 'son', 'sq', 'sr', 'sv-SE', 'ta',
'te', 'th', 'tr', 'uk', 'ur', 'uz', 'vi', 'xh', 'zh-CN', 'zh-TW', 'zu')

Verify that there are no other changed files, and the diff looks good with git diff. Since you reorganized lines to fit the 80 characters limit, there will be more than one line changed.

$ git status
On branch bug1338759
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   bedrock/settings/base.py

no changes added to commit (use "git add" and/or "git commit -a")
$ git diff

If you notice trailing whitespaces after a line, you need to remove them. First make sure that Atom is set to trim whitespaces:

  • Open Edit->Preferences.
  • Select the Packages tab and search for whitespace.
  • Click on the package whitespace in Core Packages.
  • Check Remove Trailing Whitespace. If it’s already selected, uncheck and check it again (this should fix configuration problems).

Then open and save your file again.

Commit and open a pull request

You’re ready to commit your change. Remember to references your bug with "Fix", this will automatically close the bug when the change lands.

$ git commit -a -m "Fix Bug 1338759 - Enable ne-NP on production for mozilla.org"
$ git push origin bug1338759

Then visit Bedrock on GitHub and use the Compare & pull request button. Don’t forget to double check the diff:

  • Only one file should be included: bedrock/settings/base.py).
  • No trailing whitespaces added.
  • New locale should be there.

Set up searchplugins

Cloning/Updating the locale’s repository

You will be working on l10n-central for a new locale, and you need this locale’s Mercurial repository on your computer.

l10n repositories for Firefox live in https://hg.mozilla.org/l10n-central/. In this case let’s assume that l10n clones will be stored in ~/mozilla/mercurial/l10n, with a subfolder for each locale and each branch. So, if the locale is ur, the repository will be stored in ~/mozilla/mercurial/l10n/ur/l10n-central.

$ mkdir -p ~/mozilla/mercurial/l10n/ur
$ cd ~/mozilla/mercurial/l10n/ur

The first command makes sure that the path for ur exists, the second moves into the folder.

If you don’t have a clone yet, you need to create it.

$ hg clone ssh://hg.mozilla.org/l10n-central/ur l10n-central

A couple of details to note in this hg clone command:

  • The command uses ssh://, which means you need an active (and properly configured) SSH access to the repository.
  • It doesn’t use the default folder for the local clone, but specify a l10n-central directory. Without that it would create a ur folder, making it impossible to distinguish different branches.

At this point there’s a clone stored in ~/mozilla/mercurial/l10n/ur/l10n-central (remember that you already moved to ~/mozilla/mercurial/l10n/ur before running hg clone).

If you already have a clone on your computer, always make sure to update it before doing anything:

$ cd ~/mozilla/mercurial/l10n/ur/l10n-central
$ hg pull -r default -u

Setting up files for mozilla-unified

First of all make sure that your environment is correctly set up, and update your local mozilla-unified clone to be in sync with central:

$ cd ~/mozilla/mercurial/mozilla-unified
$ hg pull -u
$ hg up central

list.json

The list of searchplugins is stored in JSON format:

The default search engine is defined as searchDefault in a default key, together with a list of searchplugins and their searchOrder:

"default": {
  "searchDefault": "Google",
  "searchOrder": ["Google", "Bing"],
  "visibleDefaultEngines": [
    "google", "amazondotcom", "bing", "ddg", "ebay", "twitter", "wikipedia"
  ]
}

Unlike the list in visibleDefaultEngines, searchDefault and searchOrder require the searchplugin name, defined as ShortName in the corresponding XML file. For example, in google.xml:

<SearchPlugin xmlns="http://www.mozilla.org/2006/browser/search/">
<ShortName>Google</ShortName>
<Description>Google Search</Description>

If a locale is missing from list.json, or doesn’t provide any of these values, the default key is used as a fallback. Since Google is the default for most locales, this system allows to avoid defining the same searchDefault for each locale. The same is valid for searchOrder, using Google and Bing as first two searchplugins for most locales.

The basic structure for each locale is fairly simple. Assuming the locale code is ur, it will have a default key, with visibleDefaultEngines.

"ur": {
  "default": {
    "visibleDefaultEngines": [
      "google", "bing", "amazon-in", "ddg", "wikipedia-ur"
    ]
  }
},

Notes about the list:

  • If the searchplugin XML file is called google.xml, the item to add to visibleDefaultEngines will be only google, without the extension but also respecting the case. Only searchDefault and searchOrder use the ShortName of the searchplugin.
  • Order should reflect the actual order of searchplugins: if Google is default, and Bing 2nd, the list should start with "google", "bing". The remaining searchplugins can be in alphabetical order.
  • If a locale is missing from list.json, it will fall back to default, defined at the beginning of the file.

A locale can also have regional overrides, for example:

"be": {
  "default": {
    "visibleDefaultEngines": [
      "yandex-by", "google", "ddg", "wikipedia-be", "wikipedia-be-tarask"
    ]
  },
  "BY": {
    "searchDefault": "Яндекс"
  },
  "KZ": {
    "searchDefault": "Яндекс"
  },
  "RU": {
    "searchDefault": "Яндекс"
  },
  "TR": {
    "searchDefault": "Яндекс"
  }
}

This defines a different default engine for Belarussian in 4 countries, each one indicated by its country code.

A locale can also override the default search order:

"cs": {
  "default": {
    "searchOrder": ["Google", "Seznam"],
    "visibleDefaultEngines": [
      "google", "seznam-cz", "ddg", "heureka-cz", "mapy-cz", "wikipedia-cz"
    ]
  }
}

To make sure you’re not creating a broken JSON, you can test the final content with an online validator like jsonlint.

XML files

Searchplugins are stored:

For other searchplugins you will need to create the .xml file yourself, with some general rules to keep in mind:

  • Always have a MPL2 license header at the beginning.
  • On desktop icons are currently square 16px and 32px, both embedded in a multilayer .ico file.
  • On Android icons are currently 96px transparent PNG images, with rounded corners (radius 6px), preferably with a colored background since the UI is white, and a white background wouldn’t show the rounded corners.
  • Check if other locales already ship the same searchplugin, and copy their XML file to use as a base. If the searchplugin is brand new, it needs to be approved by BD and it will take some time, so it might be worth starting with a shorter list of searchplugins, and consider the new one in a follow-up bug. Icons will also be provided by the search provider after approval.

Icons

These are some useful tips to deal with new icons for searchplugins:

  • If the image is a PNG (for Android), make sure to optimize it before encoding it to base64 (for example using tinypng.com).
  • You can create an .ico file starting from separate 16px and 32px PNG icons. Create a 32px image in Gimp, add each icon to a separate layer, and export the file as .ico (make sure to select the option to compress each of the layers).
  • You can use online services (like this one) to quickly encode an image to base64.
  • You can test the image in the browser (copy and paste the data URI in the address bar). You will still have to download it to check that everything looks good, since only one image will be displayed for multilayer .ico files, and transparency is quite confusing in Firefox preview.

Wikipedia

URLs: If you copy the .xml file from another locale (en-US is always the best choice for Wikipedia), make sure to update all URLs. If the URL for English starts with en.wikipedia., for ur it should start with ur.wikipedia. (check if the domain actually exists).

Search template: There’s one special URL to keep in mind, in English it’s https://en.wikipedia.org/wiki/Special:Search It’s particular because the last part changes according to the locale, just try replacing the domain first from en. to ur.: https://ur.wikipedia.org/wiki/Special:Search You’ll notice that it redirects to https://ur.wikipedia.org/wiki/خاص:تلاش and that’s the URL you need to use in the .xml file.

You can’t copy the full URL from the address bar since it will encode UTF-8 characters, e.g. https://ur.wikipedia.org/wiki/%D8%AE%D8%A7%D8%B5:%D8%AA%D9%84%D8%A7%D8%B4, so it’s faster to just copy only part of the URL and fixing the rest by hand.

Description: You will need localizers to provide the translation for Wikipedia, the Free Encyclopedia.

shortName: In the .xml file there’s an attribute called shortName. For English is:

<ShortName>Wikipedia</ShortName>

When you visit a page that exposes a searchplugin, like Wikipedia, Firefox checks if you already have one installed with the same shortName: if you don’t, the UI will suggest you to install it by displaying a small green + sign on the magnifying glass icon in the search bar. To avoid this, the searchplugin shipping in Firefox needs to match the shortName of the searchplugin available from Wikipedia. Visit https://gn.wikipedia.org and press CTRL+U to view the source code of the page. At the beginning you will find a line looking like this:

<link rel="search" type="application/opensearchdescription+xml" href="/w/opensearch_desc.php" title="ویکیپیڈیا (ur)"/>

The shortName to use in your .xml file is ویکیپیڈیا (ur). Copying and pasting RTL languages is particularly tricky.

Creating a patch for review

Once files are ready, follow the instructions available in this document to create a patch, submit it for review, and land it.

Setting up files for locale’s repository

region.properties

region.properties is stored in /browser/chrome/browser-region for Firefox desktop (/mobile/chrome for Firefox for Android), and it contains information about protocol handlers. You can use these files as a base:

A few tips:

  • If you’re updating an existing file, make sure to not reset the gecko.handlerService.defaultHandlersVersion key. If, on the other hand, you’re adding a new handler, you will have to increment the existing numeric value.

Creating a patch for review (locale repository)

Once files are ready, follow the instructions available in this document to create a patch, submit it for review, and land it.

Working with Pontoon

Some documentation specific for common and advanced tasks on Pontoon

Adding a new project on Pontoon

Verify that the project is properly localizable

Project owners can follow the guidelines available in Pontoon Documentation to properly structure files inside the repository. Some things to check:

  • Files should be in a folder like locale(s)/ab-CD/somefile.extension and be in one of the supported formats (.ftl, .lang, .properties, .po, .xliff).
  • User mozilla-pontoon needs write access to the repository.

It’s important to also check the files for localization issues before exposing them to localizers: unclear strings, lack of localization comments, missing plural forms are some of the things to check.

Create the project in Pontoon STAGE instance

First you want to test that everything works using Pontoon staging server.

Access Pontoon’s admin console on the stage server and click ADD NEW PROJECT.

  • Name: name of the repository (it will be displayed in Pontoon’s project selector).
  • Slug: used in URLs, will be generated automatically based on the repository’s name.
  • Locales:
    • Select at least one locale. To make things faster it’s possible to copy supported locales from an existing project.
    • The Read-only column can be used to add languages in read-only mode. In this way, their translations will be available to other languages in the LOCALES tab when translating, but it won’t be possible to change or submit translations directly in Pontoon.
    • You can uncheck the Locales can opt-in checkbox to prevent localizers from requesting this specific project.
  • Repositories: select the type of repository and URL. Make sure to use SSH to allow write access. For example, if the repository is https://github.com/meandavejustice/min-vid, the URL should be git@github.com:meandavejustice/min-vid.git. You can use the Clone or download button in the repository page on GitHub, making sure that Clone with SSH is selected.
  • Leave the Branch field empty, unless developers asked to commit translations in a branch instead of master.
  • Download prefix: a URL prefix for downloading localized files. For GitHub repositories, select any localized file on GitHub, click Raw and replace locale code and the following bits in the URL with {locale_code}. For example, if the link is https://raw.githubusercontent.com/bwinton/TabCenter-l10n/master/locales/en-US/addon.properties, the field should be set to https://github.com/bwinton/TabCenter-l10n/blob/master/locales/{locale_code}.
  • Public Repository Website: displayed on dashboards. E.g. https://github.com/meandavejustice/min-vid. Pontoon will try to prefill it after you enter Repository URL.
  • Project info: provide some information about the project to help localizers with context or testing instructions. HTML is supported, so you can add external links. For example:
Localization for the <a href="https://testpilot.firefox.com/experiments/min-vid">Min Vid add-on</a>.
  • Internal admin notes: use them e.g. for developer contacts and information that other PMs will find useful when covering for you.
  • Deadline: if available, enter project deadline in the YYYY-MM-DD format.
  • Priority: select priority level from one of the 5 levels available (Lowest, Low, Normal, High, Highest).
  • Contact: select the L10n driver in charge of the project, probably yourself.
  • External Resources: provide links to external resources like l10n preview environment. You need to enter the name and the URL for each resource. You can also pick one of the predefined names: Development site, Production site, Development build, Production build, Screenshots, Language pack.

Click SAVE PROJECT at the bottom of the page, then click SYNC to run a test sync. In the Sync log you should be able to see if it succeeded or failed.

Tags

Tags can be used in a project to logically group resources, assigning them a priority. To enable tags for a project, check Tags enabled and save the project.

For each tag, it’s possible to define:

  • Name: it will be displayed in project (e.g. /projects/firefox/tags/) and localization dashboards (e.g. /it/firefox/tags/), but also in search filters.
  • Slug: used in URLs for tag dashboards, e.g. /projects/firefox/tags/devtools/.
  • Priority: like for a project, it’s possible to select a priority level from one of the 5 levels available (Lowest, Low, Normal, High, Highest).

Tags resources

Once you’ve created a new tag, you need to save the project in order to be able to manage the resources associated to the tag itself, using the button highlighted in red.

A few tips on using this section:

  • Use the selector on the right (highlighted in orange) to switch between Linked resources, i.e. resources already associated to the tag, and Unlinked resources.
  • It’s possible to use wildcards in the search box, e.g. devtools/* or */client/*.
  • The first checkbox on the left (highlighted in green) allows to select all displayed items. Note: if the search returns multiple pages, only those currently displayed are selected and will be linked if Link resources is clicked.
  • There is no need to save the project when adding resources to a tag: linked resources are stored in the database as soon as they’re added with the Link resources button.

Considering the amount of information required to properly set up tags in a project, it’s recommended to set them up directly in production, and use stage only for specific testing.

Resource deadline

Like for a project, it’s possible to set a deadline for a Resource.

Go to the resource section of the admin panel, then type the name of your project (e.g. engagement) and hit Enter. All the resources for your project should appear. Click on the one you want to edit, set the deadline in the Deadline field, then click SAVE.

Create the project in Pontoon PROD instance

At this point you need to repeat the same steps on the production server.

Access Pontoon’s admin console, add the same information you used on the staging server and make sure to select all supported locales for this project.

The new project will appear in the public list of Projects only after the next sync cycle.

Adding a new short-term project on Pontoon

Short-term projects are things like newsletters, marketing campaigns, surveys… They do not have a repository, and data is instead stored only in Pontoon’s database.

The process to create a short-term project is very similar to that of a regular one.

Create the project in Pontoon STAGE instance

First you want to test that everything works using Pontoon staging server.

Access Pontoon’s admin console on the stage server and click ADD NEW PROJECT.

  • Name: name of the project (it will be displayed in Pontoon’s project selector).
  • Slug: used in URLs, will be generated automatically based on the repository’s name.
  • Locales:
    • Select at least one locale. To make things faster it’s possible to copy supported locales from an existing project.
    • You can uncheck the Locales can opt-in checkbox to prevent localizers from requesting this specific project.
  • Data Source: select Database in the list of options. This will hide the Repositories section and show a Strings section instead.
  • Strings: you can enter the initial set of strings here. Strings must be separated by new lines. If you leave this empty, you’ll be able to enter strings as a batch again after creating the project. Strings must be in en-US, and they will become the entities on that project. A resource named database will automatically be created.
  • For every other option, please refer to the new project documentation.

Click SAVE PROJECT at the bottom of the page, and you’re done!

Create the project in Pontoon PROD instance

At this point you need to repeat the same steps on the production server.

Access Pontoon’s admin console, add the same information you used on the staging server and make sure to select all supported locales for this project.

The new project will immediately appear in the public list of Projects.

Managing strings

After you have created your project, you will be able to manage its source strings. On the admin project page (that you can find from the admin console), under the Strings section, you will find two links. The one called MANAGE STRINGS will lead you to a page where all strings are listed. There you can edit the string content, edit the string comment, add new strings (use the NEW STRING button) or remove existing strings (use the trashbin icon under the comment input). Once you’re done editing, click the SAVE STRINGS button to save your changes.

Downloading translations

On the project’s admin page and on the Manage Strings page, you’ll find a DOWNLOAD STRINGS link. Clicking it will download a CSV file that contains all the translations in all enabled locales. The file format looks as follow:

Source, fr, de
Hello, Salut, Hallo
World, Monde, Welt

Adding a new locale on Pontoon

Verify if the locale is already available

Access Django’s admin interface at https://pontoon.mozilla.org/a/ (note that this is not the usual admin interface), then click Locales. In the next page search for the locale you want to add (safer to search for the locale code).

Before moving forward:

  • If the locale is going to work on mozilla.org, files need to be set up in tools and GitHub before enabling the project in Pontoon.
  • If the locale is going to work on Firefox or Firefox for Android, and an official Mercurial repository is not available, a repository needs to be created in the mozilla-l10n organization on BitBucket. Currently :pike, :mathjazz and :flod have permissions to create such repository.

Add the new locale

If the locale you need is not available, click the ADD LOCALE+ button in the top right corner. For this example, let’s consider that the task is to add Amharic (am).

You will need to complete the following fields in the next page.

Code

It’s the locale code, in this case am.

Google Translate code

Google Translate maintains a list of supported locales in its own format. Choose one that matches the locale from a list of supported locales or leave it blank to disable support for Google Translate for this locale.

MS translator code

Microsoft Translator maintains a list of supported locales in its own format. Choose one that matches the locale from a list of supported locales or leave it blank to disable support for Microsoft Translator for this locale.

MS terminology code

Microsoft Terminology maintains a list of supported locales in its own format. Choose one that matches the locale from a list provided below the field or leave it blank to disable support for Microsoft Terminology for this locale.

Name

It’s the language name, in this case Amharic.

Plural rule

It’s the GetText plural rule. This document has plural rules for several languages. For example, for Amharic it would be:

nplurals=2; plural=(n > 1);

As explained in the note below the field, you only need to put the last part in the field:

(n > 1)

CLDR Plurals

You need to find the locale on CLDR. For Amharic, there are only two cardinal plural forms listed: one, other.

The mapping is:

0 -> zero
1 -> one
2 -> two
3 -> few
4 -> many
5 -> other

For Amharic you will need to set the field as 1,5, separating the available forms with a comma (no spaces).

Irish (ga-IE), for example, has all of them except for 0, so it will be 1,2,3,4,5.

Script

The script used by this locale. Find it in CLDR Languages and Scripts.

Direction

Writing direction of the script. Set to right-to-left if rtl value for the locale script is set to YES in CLDR scriptMetadata.json.

Population

This represents the number of native speakers. There are two ways to get this information from CLDR.

Using a script

Python needs to be available on the method to use this system: save this file on your computer as cldr_population.py and run it as python cldr_population.py LOCALE_CODE.

For example, this is the output of the script when checking data for sl:

$ python scripts/cldr_population.py sl

Adding HU: 4937 (0.05% of 9874780)
Adding IT: 105412 (0.17% of 62007500)
Adding SI: 1720886 (87% of 1978030)
Adding AT: 32233 (0.37% of 8711770)
--------
Territories: AT, HU, IT, SI
Population: 1863468
--------

Manual process

Find the locale code in CLDR territoryInfo.json and multiply its _populationPercent with the territory _population. Repeat if multiple occurrences of locale code exist and sum products.

At this point click SAVE in the bottom right corner to save and create the new locale. Then, from the standard admin interface, enable at least one “non system project” (i.e. not Tutorial or Pontoon Intro) and wait until Pontoon syncs for the locale dashboard to work. You can then check if the locale’s page, as well as the new project, are available - in this case at https://pontoon.mozilla.org/am/

Sending notifications in Pontoon

Pontoon allows you to send in-app notifications to contributors of a specific project. To do so, go to your project dashboard and select the Notifications tab.

Choose recipient teams

First you need to select project teams to send the notification to. You can select them manually or by using one of the available shortcuts:

  • All: all teams translating the project.
  • Complete: teams with all strings translated in the project.
  • Incomplete: teams that have untranslated strings in the project.

All project contributors from selected teams will receive the notification.

You need to select at least one team.

Enter notification message

Next, enter the message you’d like to send. You can use HTML tags. Try to be concise and use HTML wisely.

Send

To send the notification, simply click the Send button and wait for the confirmation message to appear. It might take a moment. Once notifications are sent, you’ll be redirected to the Sent notifications panel.

Renaming a localization file in a project without losing data

In some cases, project owners will want to rename a file for clarity, or for technical reasons. If it’s only done in the VCS repository, the file will be imported again and you will lose attribution, pending suggestions and history. Since this is a special case, we can work around this and rename the file in Pontoon while keeping existing data (translations, suggestions, attribution, history). The process is not complicated, but it has to be highly coordinated.

Get a Pull Request ready

Either you or the project owners will get a Pull Request ready that renames the file for all locales. Using a script is recommended as you will need to generate the patch not too long before the next steps to avoid conflicts.

Rename the resource in Pontoon admin panel

Wait for the current Pontoon sync cycle to end, and make sure you are already logged into Pontoon admin, then merge the Pull Request.

As soon as the Pull Request is merged, go to the resource section of the admin panel, then type the name of your project (e.g. thimble) and hit Enter. All the resources for your project should appear. Click on the one you want to rename, and in the path field, for instance rename messages.properties into server.properties then click SAVE.

It is crucial to do this step before the next sync cycle, otherwise the process won’t be effective.

After the next sync, open the file for one of the locales, you should be able to see all the existing data.

Stores Application

Overview of the Stores application

This web application is used to maintain localized content for mobile stores (Google Play, App Store):

  • Main listing (description).
  • What’s new content for new releases.
  • Keywords.
  • Other localizable content, like screenshots.

Localization files are stored in the appstores l10n repository (.lang format).

This app allows:

  • Localizers to see the various strings assembled in one page/template, and warnings for strings exceeding length limits.
  • L10n drivers to check the overall status of store localization for each product and channel. In the main view, completion cell turns green when localization is complete.
  • Release Engineering to get translations through API. This is currently happening only for Android, via mozapkpublisher, but there are plans to extend it to App Store content.
  • Other users, for example designers working on screenshots, to copy and paste translation without accessing directly repositories.

Refer to the README in the code repository for instructions on how to install this app.

Projects configuration

Configuration is split between app/config/product_sources.json and the Project class (app/classes/Stores/Project.php).

app/config/product_sources.json contains a list of supported products and channels, together with a URL that can be used to retrieve the list of shipping locales. JSON has the following structure:

{
  "product code name": [
    {
      "channel": "name of the channel",
      "format": "format of the remote list",
      "source": "URL of the remote list"
    }
  ]
}

app/scripts/update_shipping_locales.py can be used to generate, or update, the list of shipping locales which is stored in app/config/shipping_locales.json. It’s important to note that these lists are used in Langchecker, to determine which locales should see specific files (product pages on mozilla.org, store content).

$products_data stores all supported products, with the following structure for each of them:

'fx_android' =>
    [
        'channels' => ['beta', 'release'],
        'name'     => 'Firefox for Android',
        'store'    => 'google',
    ],

Relevant information in this array:

  • The ID of the project, used also for API calls, it’s the item’s key.
  • Supported channels.
  • Full name of the product, used in views and navigation.
  • In which store the project lives. Currently it can be apple or google.

$locales_mapping includes information on mappings between Mozilla’s locale codes, and each store’s internal codes.

$templates determines, for each product and channel, which template is used, and which .lang files are associated to a specific part of the template (What’s new, main listing, screenshots). This array is updated every time there is a new release.

In order to add a brand new project, you would need to:

  • Define the new project and supported channels in $products_data.
  • Define the list of supported locales for each channel in $supported_locales.
  • Define an entry in $templates.
  • Create files in /templates and /view for each supported channel.
  • Track the new .lang files in Langchecker for the appstores repository.

Template and View Structure

This is a portion of the template file used for Focus for iOS.

$screenshots = function ($translations) use ($_) {
    return <<<OUT
{$_('Automatically block ads<br>& other Web trackers')}

{$_('Browse Faster<br>Web pages may load faster<br>by removing trackers')}

{$_('Before')}

{$_('After')}

{$_('Tracker')}

{$_('From Mozilla<br>A brand you trust')}
OUT;
};

Similarly to a Django template, each string is wrapped in a translation function $_('') (defined in app/inc/utilities.php). Strings are grouped into a variable, like $screenshots in this case: this allows, for example, to determine the overall length of all strings included in a section, to see if it exceeds the limits imposed by stores. It’s important to note that, since these limits are per-section and not per-string, the standard webdashboard won’t display any warnings. Also, if a string exceeds the maximum length, it won’t be exposed via API and the locale will be marked as incomplete.

These variables are then used in views, for example the variable/function $description:

<h3>Description</h3>
<pre <?= $direction ?>><?= $description($translations) ?></pre>

It’s also possible to embed more logic into these view:

<?php
    /*
        Check if the file used for screenshots exists, display this section
        only in that case.
    */
    $screenshot_lang = $project->getLangFiles($request['locale'], $request['product'], $request['channel'], 'screenshots');
    if ($screenshot_lang) {
        $locale_file = LOCALES_PATH . $request['locale'] . '/' . array_shift($screenshot_lang);
        if (file_exists($locale_file)) {
            ?>
            <h3>Screenshots</h3>
            <pre <?= $direction ?> class="text-center"><?= $screenshots($translations) ?></pre>
<?php

        };
    }
?>

In this specific case the Screenshots section is displayed only if locale has the associated .lang file. This allows to use only one view for all languages, even if only some locales will have the screenshots .lang file.

How to add a new product to Stores

Before starting, make sure to read the overview document to get a general idea of what this web app does and its structure.

This document will explain how to start tracking a new project, using Focus for Android as an example.

Add new product to configuration

Configuration for the Stores web app is split between app/config/product_sources.json and the Project class (app/classes/Stores/Project.php). The first thing to do is to determine:

  • How many channels are supported.
  • Code name and displayed name for this product.
  • If the product needs to support App Store (apple) or Play Store (google).

Let’s start with app/config/product_sources.json. This is a JSON file with the following structure:

{
  "product code name": [
    {
      "channel": "name of the channel",
      "format": "format of the remote list",
      "source": "URL of the remote list"
    }
  ]
}

The only format supported at the moment is txt: a plain text file with one locale code per line.

If the project is tracked on Webstatus, you can use its API to get the list of all locales available (example for Focus for Android).

This is the configuration to add to the file for Focus for Android:

"focus_android": [
  {
    "channel": "release",
    "format": "txt",
    "source": "https://l10n.mozilla-community.org/webstatus/api/?product=focus-android&txt"
  }
]

If you’re not used to work with JSON files, once you have updated the configuration, you can use a validator to ensure that the syntax is correct.

Now that app/config/product_sources.json contains Focus for Android, you need to run apps/scripts/update_shipping_locales.py to generate the list of locales shipping in this new product. The list of locales will be stored in app/config/shipping_locales.json, make sure that the product and channel you’ve added are available, and spot check the list of locales generated.

At this point you need to work on app/classes/Stores/Project.php. This is the configuration for Focus for Android:

private $products_data = [
...
    'focus_android' =>
        [
            'channels' => ['release'],
            'name'     => 'Focus for Android',
            'store'    => 'google',
        ],
...
];

The next step is to add which templates and .lang files will be used for this project. In case of Focus for Android there is:

  • template: PHP file used to display the full information about this product.
  • listing: .lang file containing the strings used in template (description, keywords, etc.).
  • screenshots: .lang file used to localize screenshots. It’s distinct from the main .lang file in order to support only a subset of languages.

Since it’s the first launch for Focus for Android, a whatsnew entry won’t be needed, but it will be starting from the first update.

Important: follow the naming convention of existing products.

Here’s how the focus_android entry will look like:

private $$templates = [
...
    'focus_android' => [
        'release' => [
            'template'    => 'focus_android/release/listing_mar_2017.php',
            'listing'     => 'focus_android/description_release.lang',
            'screenshots' => 'focus_android/screenshots_v1.lang',
        ],
    ],
...
];

Add templates

Now that the configuration is done, you need to add the actual template file. Templates are stored in app/templates, and the easiest way to start is by copying an existing project. In this case you can copy templates from Focus for iOS, making sure to rename the folder to focus_android, and the template to listing_mar_2017.php (as specified in the configuration above).

This is the initial part of the template:

<?php
namespace Stores;

// Include closure needed in template
include INC . 'utilities.php';

$app_title = function ($translations) use ($_) {
    return $_('Firefox Focus: The privacy browser');
};

$description = function ($translations) use ($_) {
    return <<<OUT
{$_('Browse like no one’s watching.')} {$_('The new Firefox Focus automatically blocks a wide range of online trackers — from the moment you launch it to the second you leave it.')} {$_('Easily erase your history, passwords and cookies, so you won’t get followed by things like unwanted ads.')}

{$_('“Private browsing” on most browsers isn’t comprehensive or easy to use.')} {$_('Focus is next-level privacy that’s free, always on and always on your side — because it’s backed by Mozilla, the non-profit that fights for your rights on the Web.')}

{$_('AUTOMATIC PRIVACY')}
• {$_('Blocks a wide range of common Web trackers without any settings to set')}
• {$_('Easily erases your history — no passwords, no cookies, no trackers')}

{$_('BROWSE FASTER')}
• {$_('By removing trackers and ads, Web pages may require less data and load faster')}

{$_('MADE BY MOZILLA')}
• {$_('We believe everyone should have control over their lives online. That’s what we’ve been fighting for since 1998.')}
OUT;
};
...

The first lines are common to all templates: they define the namespace and include a set of helper functions. Then, each section of the template is defined as a variable that returns translations for selected strings. The simplest case is:

$app_title = function ($translations) use ($_) {
    return $_('Firefox Focus: The privacy browser');
};

$app_title is defined as a function that returns the translation, using the helper function $_(), for only one string.

In case there are more strings in a section, HEREDOC syntax can be used.

$screenshots = function ($translations) use ($_) {
    return <<<OUT
{$_('Automatically block ads<br>& other Web trackers')}

{$_('Browse Faster<br>Web pages may load faster<br>by removing trackers')}
OUT;
};

Associating a group of strings to a variable allows to count the total number of characters for a section, and see if it’s longer than allowed.

Pay attention to the log when testing a new template that you’ve copied from another product, some functions and variables are exclusive of a specific store, for example:

  • $keywords_warning is defined only for App Stores projects.
  • $short_desc is defined only for Play Store projects.

Add views

The next step is to add a view for each channel supported for this product. Views are stored in views/product_name/channel, in case of Focus for Android views/focus_android/release. There are 2 files to create:

  • locale_view.php: used for General view.
  • locale_view_escaped.php: used for Description raw HTML. It’s the same content as locale_view.php, but each element is included in <pre <?= $direction ?> contenteditable="true"></pre> elements.

As for templates, the easiest way to start is by copying files from an existing product.

This is how the first part of locale_view.php looks like:

<h1>Focus for Android Listing Copy (<?= $request['locale'] ?>)</h1>

<h3>Title &mdash; <?= $title_warning ?></h3>
<pre <?= $direction ?>><?= $app_title($translations) ?></pre>

<h3>Short Description &mdash; <?= $short_desc_warning ?></h3>
<pre <?= $direction ?>><?= $short_desc($translations) ?></pre>

As you can see, it simply calls the variables created inside the template. The interesting part is that you can add more complex logic in this file:

<?php
    /*
        Check if the file used for screenshots exists, display this section
        only in that case.
    */
    $screenshot_lang = $project->getLangFiles($request['locale'], $request['product'], $request['channel'], 'screenshots');
    if ($screenshot_lang) {
        $locale_file = LOCALES_PATH . $request['locale'] . '/' . array_shift($screenshot_lang);
        if (file_exists($locale_file)) {
            ?>
            <h3>Screenshots</h3>
            <pre <?= $direction ?> class="text-center"><?= $screenshots($translations) ?></pre>
<?php

        }
    }
?>

In this case, the screenshots session is displayed only if the requested locale actually has screenshots, i.e. if the file specified in the configuration is available.

locale_view_escaped.php is exactly the same as the default view, only using a different HTML structure to make copy and paste easier. For example, the same section displayed above becomes:

<h1>Focus for Android Listing Copy (<?= $request['locale'] ?>)</h1>

<h3>Title &mdash; <?= $title_warning ?></h3>
<pre <?= $direction ?> contenteditable="true"><?= htmlspecialchars($app_title($translations)) ?></pre>

<h3>Short Description &mdash; <?= $short_desc_warning ?></h3>
<pre <?= $direction ?> contenteditable="true"><?= $short_desc($translations) ?></pre>

As for the templates, pay attention to differences between a product shipping on Play Store and one shipping on App Stores:

  • For Android products there should be length warnings on title, short description, and description. For iOS, a warning should be displayed on keywords.
  • There are no keywords in Android, short description is not available for iOS products.

Tests

It’s not mandatory to add new tests, since they normally cover functionalities and not data. Adding a simple API test in tests/functional/api.php is highly suggested. For example:

$paths = [
...
    ['v1/focus_android/done/release/', 200, false],
...

Some tests will need to be updated, for example testGetSupportedProducts in tests/units/Store/Project.php. In case, unit tests are run as part of the automation when opening a pull request.

Track .lang files in Langchecker

The Stores web app is going to use .lang files from the appstores repository. This document won’t go into much detail about these configuration files, if you’re not familiar with Langchecker you should read this document first.

In the configuration two .lang file are associated to Focus for Android:

  • focus_android/description_release.lang for the main content.
  • focus_android/screenshots_v1.lang for screenshots.

First you need to create a new $focus_android_locales array in app/config/stores_locales.inc.php.

// Locales working on Focus for Android (from stores_l10n app)
$cache_id = 'focus_android_locales';
if (! $focus_android_locales = Cache::getKey($cache_id, 60 * 60)) {
    $focus_android_locales = $json_object
        ->setURI(STORES_L10N . 'focus_android/supportedlocales/release')
        ->fetchContent();
    $focus_android_locales = array_intersect($focus_android_locales, $mozilla);
    Cache::setKey($cache_id, $focus_android_locales);
}

This code retrieves, via API from the Stores web app, the list of locales supported for Focus for Android release. Then:

$focus_android_store = array_intersect($focus_android_locales, $google_play_locales);

This intersects the list of locales shipping in Focus for Android with the list of locales supported in Play Store, and stores the resulting array in $focus_android_store.

Then you need to start tracking the new files in app/config/sources.inc.php:

$appstores_lang = [
    'focus_android/description_release.lang' => [
        'supported_locales' => $focus_android_store,
    ],
    'focus_android/screenshots_v1.lang' => [
        'supported_locales' => [
            'es-ES', 'id', 'pt-BR', 'ru',
        ],
    ],
...
];

Once the configuration is complete, you’ll need to create the actual .lang files and add them to the appstores repository with lang_update.

Adding a What’s new section

As already explained, you won’t need a What’s new section for a newly launched product. For further updates, you will need to add a whatsnew entry in the project configuration.

private $$templates = [
...
    'focus_android' => [
        'release' => [
            'template'    => 'focus_android/release/listing_mar_2017.php',
            'listing'     => 'focus_android/description_release.lang',
            'screenshots' => 'focus_android/screenshots_v1.lang',
            'whatsnew'    => 'focus_android/whatsnew/focus_v2.lang',
        ],
    ],
...
];

Then manage this content in the template:

$whatsnew = function ($translations) use ($_) {
    return <<<OUT
* {$_('A brand new feature')}
OUT;

And use this $whatsnew variable in all available views.

<h3>What’s new &mdash; <?= $whatsnew_warning ?></h3>
<pre <?= $direction ?>><?= $whatsnew($translations) ?></pre>

Testing

When all tools and repositories are ready, make sure to test everything locally and check PHP logs before opening a pull request. It’s strongly recommended to use a virtual machine set up following these instructions running all needed tools.

A good way to test it is to provide a fake translation for one locale, slightly altering the translated strings, to check if all views and API calls work as expected.

Webdashboards

Structure

There are four interdependent elements in the current webdashboard system.

Webdashboard

Webdashboard is the main dashboard used by localizers. It contains information about files with missing strings, deadlines, and more. Data is also available via RSS, with information about missing strings and deadlines.

Webdashboard in itself doesn’t generate any data, it just displays information provided by Langchecker and Webstatus in JSON format.

Langchecker

Langchecker is used to analyze all projects using .lang files as data sources. It also stores all configuration information needed to manage these repositories: which projects are supported, which files are in each project, which locales are supported for each project or file, metadata like critical status or deadlines.

Langchecker’s scripts are used to propagate changes to all .lang files, or add news ones, in l10n repositories. It also provides an API to retrieve translations, and information about coverage, i.e how many locales translated a specific string or a page, and what percentage of the l10n population that represents.

Webstatus

Webstatus is used to analyze external projects, currently supporting the following formats: .po (Gettext), .properties, .xliff, and .ftl (l20n). It also provides an API to get list of locales (supported, complete), in JSON or TXT format, for a specific product.

Stores_l10n

Stores_l10n is a web application used to manage translations for Google Play Store and Apple App Store. It provides Langchecker with the list of locales supported in stores and iOS/Android products. Its API is used by release drivers in mozapkpublisher, a tool used to publish .apk on Google Play together with Store listing and whatsnew content. Public views are used to copy and paste content on Apple Store (no automation available at the moment).

This diagram describes the relation between each of the components, including relation with external entities.

Webdashboards diagram

Installation

See the Useful Links at the end of the document for links to the relevant GitHub code repositories. Follow the instructions provided in each README for system requirements and instructions.

Note that the instances on the l10n community server of all products use a GitHub’s webhook to update code automatically every time the repository is updated.

Tools

For detailed information check the specific pages for each project:

These are detailed instructions for the most common tasks:

For documentation specific to mozilla.org, see this page.

Contribution Guidelines

Always work on forks of the main repository and open pull requests if you’re going to update code. Automated tests are run with Travis on each pull request to check for syntax and functional errors.

Useful Links

Mozilla.org

  • Code repository (codename bedrock): https://github.com/mozilla/bedrock
  • Trunk localization: https://github.com/mozilla-l10n/www.mozilla.org
  • Production localization: https://github.com/mozilla-l10n/bedrock-l10n

Webdashboard

  • Production instance: https://l10n.mozilla-community.org/webdashboard/
  • Code: https://github.com/mozilla-l10n/webdashboard/

Langchecker

  • Production instance:https://l10n.mozilla-community.org/langchecker/
  • Code: https://github.com/mozilla-l10n/langchecker/

Webstatus

  • Production instance: https://l10n.mozilla-community.org/webstatus/
  • Code: https://github.com/mozilla-l10n/webstatus

Stores

  • Production instance: https://l10n.mozilla-community.org/stores_l10n/
  • Code: https://github.com/mozilla-l10n/stores_l10n

Langchecker

Langchecker is the main repository to update when adding or updating files. There are four different configuration files:

  • app/config/locales.inc.php: it contains several lists of locales, like locales supported in mozilla.org, locales only working in Fennec, etc. These lists are then used in the main configuration file to determine which locales are supported for each file. Normally you would need to update this file only when bootstrapping new languages.
  • app/config/store_locales.inc.php: this file is generated automatically from external data and tracks locales supported in online stores (Play Store, App Store).
  • app/config/websites.inc.php: it contains the configuration for websites (projects) supported in Langchecker.
  • app/config/sources.inc.php: it contains the configuration for all files supported in Langchecker.

Langchecker also includes several command line scripts to manage day-to-day operations on supported repositories. For example:

  • app/scripts/lang_update: reads the original en-US file, adds missing strings to requested files, reformat localized .lang files. It can also be used to import existing translations from another .lang file.
  • app/scripts/mark_active: mark complete files (all strings translated, no errors) as active.
  • app/scripts/add_tags: add l10n tags to a file. Some strings in a file can be bound to a tag, allowing to display them in production for mozilla.org only when they’re localized. This script examines if all the strings associated to a specific tag are localized, and adds tags to localized files.

Langchecker's wiki has a full list of the available views, API calls, and command line scripts.

Sources Structure

Each project is internally called website, and it’s identified by a numeric index. These are the projects currently supported.

IDCodenameDescription
0www.mozilla.orgUsed to track all files for www.mozilla.org
6engagementUsed for Engagement material, typically snippets.
12appstoresUsed to localize material for AppStore (iOS) and Google Play (Android).

Let’s consider a simple project like Engagement. This is how its definition looks like in the main $sites variable.

6 => [
    'engagement',
    $repositories['engagement']['local_path'],
    '',
    $engagement_locales,
    $engagement_lang,
    'en-US', // source locale
    $repositories['engagement']['public_path'],
    1,
    'lang',
    'engagement',
    ['tags'],
],

The structure of each item is:

  1. Name (used when displaying the website).
  2. Path to local repository.
  3. Folder containing locale files.
  4. Array of supported locale.
  5. Array of supported files with all associated data.
  6. Reference locale code.
  7. URL to public repo (used to create links to files).
  8. Default priority, used as fallback when files don't specify one.
  9. Type of files (lang, raw).
  10. Project name on Pontoon (used for edit links).
  11. Array of excluded error checks (tags).

For each website there is a list of supported locales, but each file might only use a subset of this list.

For this project, the list of supported files is stored in $engagement_lang.

$engagement_lang = [
    'ads/ios_android_apr2016.lang' => [
        'supported_locales' => ['de', 'es-ES', 'fr', 'pl'],
    ],
    'ads/ios_android_feb2017.lang' => [
        'deadline'          => '2017-02-02',
        'supported_locales' => ['zh-HK', 'zh-TW'],
    ],
    ...

Flags commonly used are:

  • obsolete: the file won’t be displayed in webdashboard at all, and its strings won’t be counted in stats. It’s usually used to mark a file that will be completely deleted later but needs to remain in the repository.
  • opt-in: the file was requested only for some locales, but others can decide to opt-in and request the page for translation through Bugzilla.

For more details on the structure of this file, check the [add_new_file.md](documentation to track new files).

Webdashboard

Webdashboard, getting its data from Langchecker, has only one configuration file, app/data/project.php, that is used to create special project views (e.g. Fall 2015).

The list of supported locales is taken directly from Langchecker.

Bootstrap a new locale

Langchecker

The first thing you need to do is to add the new locale code to the $mozilla array in app/config/locales.php. Note that all lists of locales are in alphabetical order.

If this new locale is going to localize only Fennec, you need to add it also to the $fennec_locales array. If it’s only working on mozilla.org, add it to $mozorg_locales. In this way desktop specific pages won’t be exposed for this language.

Example: let’s say you need to add the ab-CD locale. This is how the $mozilla array begins:

$mozilla = [
    'ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'be', 'bg',
    'bm', 'bn-BD', 'bn-IN', 'br', 'brx', 'bs', 'ca', 'cak',

And how it becomes after adding the new locale:

$mozilla = [
    'ab-CD', 'ach', 'af', 'an', 'ar', 'as', 'ast', 'az', 'be', 'bg',
    'bm', 'bn-BD', 'bn-IN', 'br', 'brx', 'bs', 'ca', 'cak',

Add files to l10n repositories (mozilla.org)

At this point you need to add the files to www.mozilla.org by using the lang_update script.

Run the following commands:

$ lang_update all 0 ab-CD

This will add all missing files to mozilla.org (ID: 0) for the ab-CD locale (you could also run it with the all parameter, but it might introduce changes not relevant for this task).

Move to the l10n repository for mozilla.org, check the status and commit (don’t forget to add the new folder). trunkst is available as a shortcut to move in the folder and run git status.

$ trunkst
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	ab-CD/

nothing added to commit but untracked files present (use "git add" to track)

$ git add ab-CD

$ git commit -m "Bootstrap locale ab-CD on mozilla.org"

$ git push

Add locales to an existing file

Example task: add sv-SE to the list of supported locales for foundation/index.lang.

Updating Langchecker sources

First you need to search for the file that you want to update in app/config/sources.inc.php.

Always remember to work on a fork of the main repository, and create a branch in your local clone. Assuming the langchecker installation is in ~/mozilla/git/langchecker, and that the branch name you want to create is TESTBRANCH, the commands to use would be:

$ cd ~/mozilla/git/langchecker
$ git branch TESTBRANCH
$ git checkout TESTBRANCH

At this point you’re ready to update the configuration file.

'foundation/index.lang' => [
    'flags' => [
        'opt-in' => ['all'],
    ],
    'supported_locales' => [
        'de', 'es-ES', 'fr', 'kab', 'pl', 'pt-BR', 'zh-TW',
    ],
],

Adding sv-SE would result in:

'foundation/index.lang' => [
    'flags' => [
        'opt-in' => ['all'],
    ],
    'supported_locales' => [
        'de', 'es-ES', 'fr', 'kab', 'pl', 'pt-BR', 'sv-SE', 'zh-TW',
    ],
],

Commit the changes and open a pull request on the main repository.

$ cd ~/mozilla/git/langchecker
$ git add app/config/sources.inc.php
$ git commit -m "Add sv-SE to foundation/index.lang"
$ git push origin TESTBRANCH

Adding the file to the repository

At this point you need to run lang_update to actually add the file to the repository.

$ lang_update all 0 sv-SE

This line updates all files, for website 0 (mozilla.org), for sv-SE.

Move in the l10n repository, make sure to add the file and commit.

$ trunkst
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	sv-SE/foundation/index.lang

nothing added to commit but untracked files present (use "git add" to track)

$ git add sv-SE
$ git commit -m "Add foundation/index.lang to sv-SE"
$ git push

Update an existing file

This task can be split into smaller tasks:

  • Add the new strings to the reference file (typically en-US).
  • Update the deadline (optional).
  • Update the file for all locales adding the new strings.

Add the new strings to the reference file

Edit the reference file and add new strings as needed.

If you’re adding a string behind a tag, always remember to bind the new string to the tag and to add the tag itself to the beginning of the file. Working with Bedrock has more details about it.

Always check the diff after completing, to make sure you’re not introducing unwanted changes. If you’re using the l10n-drivers virtual machine, you can use trunkst to move to the mozilla.org trunk folder and check the status (git status) and diff (git diff).

Also double check the Errors View, since it will warn if you’re introducing a duplicated string in the same file.

Update the deadline

If the file has a deadline set, you normally need to update it (e.g. plus 2 weeks from the the update).

Search for the filename in app/config/sources.inc.php and simply update the date associated to this file respecting the format YEAR-MM-DD.

Update the file for all locales

At this point you need to run lang_update to propagate the update to all locales. Assuming that the file is called mozorg/contribute/signup.lang, run:

$ lang_update mozorg/contribute/signup.lang 0 all

This line updates mozorg/contribute/signup.lang, for website 0 (mozilla.org), for all locales.

Move in the l10n repository (remember the trunkst shortcut) and spot check at least a couple of locales. For example, to check fr, run git diff fr. IMPORTANT: never run just git diff, otherwise you’ll get the diff for over a hundred locales.

After checking that everything looks correct, commit (notice the -aparameter to commit all changed files already tracked).

$ trunkst
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	(list of all locales and files added)

nothing added to commit but untracked files present (use "git add" to track)

$ git commit -a -m "Add new strings to mozorg/contribute/signup.lang"
$ git push

Make sure to check the commit on GitHub after pushing, especially to verify if the commit included any unwanted change.

Remove obsolete files from repositories

Once a file has been completely removed from Langchecker, it’s possible to remove files from the repository using a script available within the tool. Note that files are initially marked as obsolete, and only removed after a few weeks or months.

This document assumes that you have set up the system as explained in this document, so aliases like lang_update are available, repositories are already cloned in ~/mozilla/repositories, Atom is available and includes the syntax highlighter for LANG files.

Let’s assume that the file removed from Langchecker is called firefox/australis/firefox_tour.lang and belongs to the website mozilla.org (ID 0).

To check which locales still have this file, it’s enough to move into the local clone (since it’s mozilla.org, you can use the trunkst shortcut instead of cd ~/mozilla/repositories/mozilla_org) and run the script called untracked_files (full path to the script would be ~/mozilla/git/langchecker/app/scripts/untracked_files).

$ trunkst
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
$ untracked_files
$

If the output of the command is empty, like in this case, it means that there are no untracked files in the repository.

Without additional parameters, the script checks automatically mozilla.org. It’s possible to examine a different repository by passing the ID of the website within Langchecker, e.g. for engagement (ID 6):

$ untracked_files 6

If there are untracked files, the output will look like this:

$ untracked_files
ach/firefox/australis/firefox_tour.lang
af/firefox/australis/firefox_tour.lang
am/firefox/australis/firefox_tour.lang
an/firefox/australis/firefox_tour.lang
ar/firefox/australis/firefox_tour.lang
as/firefox/australis/firefox_tour.lang
...

The first step to remove these files is to create a new branch:

$ git branch my_branch
$ git checkout my_branch

Then pipe the output of untracked_files to a command that actually removes the files from the repository (make sure to be in the repository’s folder before running the command):

$ untracked_files | xargs git rm $1
rm 'ach/firefox/australis/firefox_tour.lang'
rm 'af/firefox/australis/firefox_tour.lang'
rm 'am/firefox/australis/firefox_tour.lang'
rm 'an/firefox/australis/firefox_tour.lang'
rm 'ar/firefox/australis/firefox_tour.lang'
rm 'as/firefox/australis/firefox_tour.lang'
...

Checking git status will show that files are staged to be removed. You only need to commit these changes and push them to your branch:

$ git status
On branch test
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	deleted:    ach/firefox/australis/firefox_tour.lang
	deleted:    af/firefox/australis/firefox_tour.lang
	deleted:    am/firefox/australis/firefox_tour.lang
	deleted:    an/firefox/australis/firefox_tour.lang
	deleted:    ar/firefox/australis/firefox_tour.lang
	deleted:    as/firefox/australis/firefox_tour.lang
...

$ git commit -a -m "Remove obsolete file firefox/australis/firefox_tour.lang"
$ git push origin my_branch

At this point create a pull request from this branch, review it to make sure it doesn’t include unnecessary changes, squash and merge it. In case of mozilla.org, you’ll also need to cherry-pick the merge commit to production.

One important detail to keep in mind is that Japanese (ja) is automatically excluded from checks for mozilla.org, since they localize several pages that are not tracked by our systems. It’s possible to check the list of untracked files in Japanese alone by running:

$ untracked_files 0 ja
ja/foundation/annualreport/2010/ahead.lang
ja/foundation/annualreport/2010/faq.lang
ja/foundation/annualreport/2010/index.lang
ja/foundation/annualreport/2010/opportunities.lang
ja/foundation/annualreport/2010/people.lang
ja/foundation/feed-icon-guidelines/faq.lang
ja/foundation/feed-icon-guidelines/index.lang
ja/foundation/trademarks/distribution-policy.lang
ja/foundation/trademarks/faq.lang
ja/foundation/trademarks/index.lang
ja/foundation/trademarks/l10n-website-policy.lang
ja/foundation/trademarks/list.lang
ja/foundation/trademarks/policy.lang
ja/mozorg/about/policy/lean-data.lang

If you want to remove a file for Japanese, you’ll need to do it manually with git rm before committing to the branch, i.e.

$ git rm ja/firefox/australis/firefox_tour.lang

Add a new file to an existing project

This task can be split into smaller tasks:

  • Add the file to the list of supported files specifying:
    • List of supported locales.
    • Priority (optional).
    • Flags (optional).
    • Deadline (optional).
  • Add the files to all locales in the l10n repository.

To analyze all steps, let’s consider a practical example: you need to add a file called mozorg/contribute/signup.lang to the project www.mozilla.org. The file is priority 1 only for French and German, opt-in for other locales, and deadline is May 30th, 2016. Note that most of the time a file will have the same priority for all locales, and won’t have any flags.

Add the file to the list of supported files

The file you need to update is app/config/sources.inc.php.

In app/config/websites.inc.php there is a global array $sites that defines all parameters for each project. For www.mozilla.org it looks like this:

0 => [
    'www.mozilla.org',
    $repositories['www.mozilla.org']['local_path'],
    '',
    $mozillaorg,
    $mozillaorg_lang,
    'en-US', // source locale
    $repositories['www.mozilla.org']['public_path'],
    3,
    'lang',
    [],
],

The array storing all supported files is the 5th element ($mozillaorg_lang). With time you won’t need to check the configuration array, it will be enough to search for a similar file in the same project.

Supported locales

Search for the definition of $mozillaorg_lang in app/config/sources.inc.php and add the new file, respecting the existing alphabetical order (snippets represent an exception to this rule). This is how the definition’s begin looks:

$mozillaorg_lang = [
    'download_button.lang' => [
        'deadline'          => '2016-04-29',
        'priority '         => 1,
        'supported_locales' => $mozillaorg,
    ],

Supported locales are defined in each file as an array associated to the key supported_locales. If this key is missing, file will fall back to the list of locales supported for the website (project) it belongs to. For clarity it’s highly suggested to always specify the list of supported locales.

For the list of locales you should check the definition for existing similar files. In this case, you can use the same settings as another contribute pages:

'mozorg/contribute/index.lang' => [
    'flags' => [
        'opt-in' => ['all'],
    ],
    'priority'          => 2,
    'supported_locales' => $getinvolved_locales,
],
'mozorg/contribute/signup.lang' => [
    'supported_locales' => $getinvolved_locales,
],

Priority

Priority is optional and can defined for each file. If the priority is the same for all locales, you can assign the integer value (from 1 to 5) to a priority key.

'mozorg/contribute/index.lang' => [
    'priority'          => 2,
    'supported_locales' => $getinvolved_locales,
],

It’s also possible to define a more complex set of priorities using an associative array, where to each priority (a numeric key) is associated an array of locales. all is a special locale to represent all supported locales for this file. For example:

'priorities'          => [
    1 => ['de', 'fr'],
    2 => ['all'],
],

If you don’t specify a priority, the file will fall back to the default priority specified for the project (in $sites). In this case the request is to set the file with priority 1 for French and German, while the default for mozilla.org is 3.

In this case, the request is to assign priority 1 only to French and German

'mozorg/contribute/signup.lang' => [
    'priority'          => [
        1 => ['de', 'fr'],
    ],
    'supported_locales' => $getinvolved_locales,
],

Flags

Flag are defined for each file as an array associated to the key flags:

'mozorg/contribute/index.lang' => [
    'flags' => [
        'opt-in' => ['all'],
    ],
    'supported_locales' => $getinvolved_locales,
],

To each flag (key) is associated an array of locales. all is a special locale to represent all supported locales for this file. Flags currently in use are: obsolete, opt-in.

In this case, the request is to flag the file as opt-in for all locales.

'mozorg/contribute/signup.lang' => [
    'flags'    => [
        'opt-in' => ['all'],
    ],
    'priority'          => [
        1 => ['de', 'fr'],
    ],
    'supported_locales' => $getinvolved_locales,
],

Deadline

If a file is critical, you also want to set a deadline for it: in the last week before deadline the date will be displayed in orange on the Webdashboard, after deadline it will be displayed in red.

If the deadline is the same for all locales, you can assign the date (as a string in ISO format YYYY-MM-DD) to a deadline key. In this case, deadline needs to be set to May 30th, 2016 (2016-05-30):

'mozorg/contribute/signup.lang' => [
    'deadline' => '2016-05-30',
    'flags'    => [
        'opt-in' => ['all'],
    ],
    'priority'          => [
        1 => ['de', 'fr'],
    ],
    'supported_locales' => $getinvolved_locales,
],

It’s also possible to define different deadlines for locales using an associative array, where to each deadline (date in ISO format YYYY-MM-DD) is associated an array of locales. all is a special locale to represent all supported locales for this file. For example, a deadline for German and French, and a later date for other languages:

'deadline' => [
    '2016-05-30' => ['de', 'fr'],
    '2016-06-15' => ['all'],
];

Or a deadline only for French:

'deadline' => [
    '2016-05-30' => ['fr'],
];

Add the files to all locales in the l10n repository

At this point you need to run lang_update to actually add the file to all locales.

lang_update mozorg/contribute/signup.lang 0 all

This line updates mozorg/contribute/signup.lang, for website 0 (mozilla.org), for all locales.

Move in the l10n repository, make sure to add the file and commit.

$ trunkst
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	(list of all locales and files added)

nothing added to commit but untracked files present (use "git add" to track)

$ git add .
$ git commit -m "Add new page mozorg/contribute/signup.lang"
$ git push

Note: git add . adds all the new files in the current repository. Make sure to check the list returner by git status, and to check the commit on GitHub after pushing.

Working with Mercurial

How to set up and work with Mercurial:

Set up the environment for Mercurial

IMPORTANT: This configuration has to be done only once on the computer. After that you will simply need to keep Mercurial up to date (a new version is released every month).

Mercurial

Most of the work needs to be done in Mercurial, so you need to install if first. An alternative way to install it on macOS is via homebrew with brew install hg (brew upgrade hg to update it later).

To check if Mercurial is available and up to date, run in the terminal hg --version, the output should look like this:

$ hg --version
Mercurial Distributed SCM (version 4.0.1)
(see https://mercurial-scm.org for more information)

Copyright (C) 2005-2016 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Unless you’re familiar with vi, you also want to setup nano as the default editor in the system.

$ export EDITOR=/usr/bin/nano

Mercurial configuration

Mercurial’s configuration is stored in a hidden file called .hgrc inside your user folder (~/.hgrc). You will need to create it if it’s not available on the system.

It’s divided into sections, with section names between square parentheses. For example:

[ui]
username = something@example.com

Set up SSH access for l10n repositories

In order to be able to commit directly to l10n repositories, you need SSH access via your LDAP account (both obtained through a bug, where you also provide your SSH key).

Assuming your SSH key is stored in ~/.ssh/id_rsa, your .ssh/config file should have a line that looks like this

Host hg.mozilla.org
User YOUR_LDAP_EMAIL_ADDRESS
IdentityFile ~/.ssh/id_rsa

Set up Arcanist for Phabricator

Detailed instructions and explanations are available in the official documentation.

On macOS, PHP is already shipping as part of the operating system, but you might need to install Git if not available. The following commands clone two repositories inside ~/mozilla/mercurial/. If you move them, you’ll need to update the PATH command accordingly.

$ git clone https://github.com/phacility/libphutil.git ~/mozilla/mercurial/libphutil
$ git clone https://github.com/phacility/arcanist.git ~/mozilla/mercurial/arcanist
$ echo -e '\n# Arcanist path for Phabricator\nPATH="$PATH:$HOME/mozilla/mercurial/arcanist/bin/"' >> ~/.bash_profile
$ source ~/.bash_profile

Check that the arc command is available:

$ arc --version
arcanist 875d018360374cb4b1287309782fcb9a75d4bcbf (9 Jul 2018)
libphutil 1613e68f474030e2c8de2797080a6872c140b1ef (20 Jul 2018)

You also need to create an account on Phabricator and log in.

At this point you still have to install Arcanist credentials, but you need a clone of mozilla-unified first.

Cloning and updating mozilla-unified

Since searchplugins are stored in mozilla-central, you will also need a clone of it on your computer. Using the mozilla-unified repository repository is a better solution, especially if you need to work on more than just mozilla-central.

$ cd ~/mozilla/mercurial/
$ hg clone https://hg.mozilla.org/mozilla-unified

To update the existing repository:

$ cd ~/mozilla/mercurial/mozilla-unified
$ hg pull -u
$ hg up central

The last command makes sure you’re working against central (to be more precise, it moves you to the central bookmark).

Now you can complete the setup for Arcanist. Follow the instructions on screen to complete, it will require you to connect to Phabricator and generate an API key to copy in the terminal:

$ cd ~/mozilla/mercurial/mozilla-unified
$ arc install-certificate

Mercurial configuration

Mercurial configuration is stored in the .hgrc file in the user’s home directory.

Useful extensions are Mercurial Queues Extension (queues), color (colorize output, for example diffs) and purge (to remove untracked files from the repository). If the .hgrc file doesn’t have an [extensions] section create one and add the following lines:

[extensions]
mq =
color =
purge =

Some aliases are also useful to work with bookmarks:

[alias]
shortlog = log --template "{node|short} | {date|isodatesec} | {author|user}: {desc|strip|firstline}\n"
wip = log --graph --rev=wip --template=wip

[templates]
wip = '{label("log.branch", branches)} {label("changeset.{phase}", rev)}{label("changeset.{phase}", ":")}{label("changeset.{phase}", short(node))} {label("grep.user", author|user)}{label("log.tag", if(tags," {tags}"))}{label("log.tag", if(fxheads," {fxheads}"))} {label("log.bookmark", if(bookmarks," {bookmarks}"))}\n{label(ifcontains(rev, revset("."), "desc.here"),desc|firstline)}'

Text editor

The suggestion is to use Atom with the optional Sort Lines package installed.

If you don’t like to use nano to edit commit messages, you can also use a graphical editor. A possible alternative on macOS is to use TextMate: after installing it, open Preferences and install the Shell support (Terminal tab), then add a editor preference to .hgrc.

[ui]
username = YOUR NAME <YOUR EMAIL>
ignore.other = ~/.hgignore
editor = mate -w

You can also use Atom, but it’s definitely slower.

[ui]
username = YOUR NAME <YOUR EMAIL>
ignore.other = ~/.hgignore
editor = atom --wait

In general, in order to be able to use it for commit messages, you’ll need a text editor available from the command line, and with a wait option.

Create a patch in Mercurial

There are currently two different methods available to create a patch for Mercurial repositories:

Phabricator is the preferred method for patches to main code repositories, like mozilla-central. On the other hand, Queues is the only tool available for patches to l10n repositories.

As a general rule, before creating a patch, make sure that your environment is correctly set up, and update your local clones.

Creating a patch using Phabricator

Creating a patch

Consider the following example: setting up productization for Urdu (ur) on Firefox desktop. After you’ve created or edited all the files you need, check the status of the repository.

$ hg status
M browser/locales/search/list.json
? browser/locales/searchplugins/amazon-in.xml
? browser/locales/searchplugins/wikipedia-ur.xml

These are all status codes relevant for working with patches:

  • M = modified. File was already tracked but has been modified.
  • A = added. File is new and was already added with hg add.
  • R = removed. File has been removed with hg remove.
  • ! = missing. File has been removed from the filesystem but it’s still tracked. Use hg remove to stop tracking it.
  • ? = not tracked. File is not tracked yet. You can start tracking it by using hg add.

Note that the file status will change once you commit, hg status only shows information about pending changes.

Now you need to add the new files. You can add them using the full path, or a parent folder, just make sure you’re not adding unnecessary files.

$ hg add browser/locales/searchplugins
$ hg status
M browser/locales/search/list.json
A browser/locales/searchplugins/amazon-in.xml
A browser/locales/searchplugins/wikipedia-ur.xml

Let’s create a bookmark for this pending work, for example bug1304757.

$ hg bookmark bug1304757

Commit the changes:

$ hg commit -m "Bug 1304757 - [ur] Search engine setup for Firefox for Urdu"

At this point you can check the status of the tree:

$ hg wip

And you should be able to identify your work and bookmark (press q to leave this view):

@   358728:c0b04112d4e6 flod tip  bug1304757
|  Bug 1304757 - [ur] Search engine setup for Firefox for Urdu, r?flod
| o   358726:f7834b7b4050 aosmond  inbound
|/:  Bug 1299498 - Keep a main thread only pointer to the underlying nsIURI for ImageURL. r=tnikkel
| : o   358715:6f79cece26e9 ffxbld  beta
| : |  No bug, Automated blocklist update from host bld-linux64-spot-245 - a=blocklist-update
| : ~
| : o   358714:f5f73390f439 ffxbld  release
| : |  No bug, Automated blocklist update from host bld-linux64-spot-246 - a=blocklist-update
| : ~
o :   358709:f8107cf96144 cbook  central
:/   merge mozilla-inbound to mozilla-central a=merge

Create a revision in Differential:

$ arc diff

An editor will open, asking for some information about the commit:

Bug 1304757 - [ur] Search engine setup for Firefox for Urdu

Summary:

Test Plan:

Reviewers: flod

Subscribers:

Bug #: 1304757

You need to provide:

  • Reviewers: use the Phabricator nickname (usually matches the nickname on Bugzilla). In this example: flod.
  • Bug #: provide the bug number. In this example: 1304757.

Once finished, save the editor and exit. At the end of the process, you should find a link to a Differential in Phrabricator:

Linting...
No lint engine configured for this project.
Running unit tests...
No unit test engine is configured for this project.
 SKIP STAGING  Phabricator does not support staging areas for this repository.
Updating commit message...
Created a new Differential revision:
        Revision URI: https://phabricator.services.mozilla.com/Dxxxx

Included changes:
  M       browser/locales/search/list.json
  A       browser/locales/searchplugins/amazon-in.xml
  A       browser/locales/searchplugins/wikipedia-ur.xml

Once published, the review request will be attached automatically to the bug, and the reviewer will be flagged. Note that you can also update information about the patch, like reviewer or bug, directly in Phrabricator after using arc diff.

Updating an existing patch

If you need to address review comments, you can restore your branch by switching to your bookmark, and start working on it again.

$ hg up bug1304757

You can check which bookmark is currently active with hg bookmarks:

$ hg bookmarks
   aurora                    359014:96503957841c
   beta                      359025:34c73c520f93
 * bug1304757                359034:bde380bc54ff
   central                   358999:a69583d2dbc6

After addressing the review comments and editing your local files, you have two choices to save your work:

  • Amend the last commit.
  • Create a new commit and squash history.

Amend the last commit

To amend the last commit, simply execute:

$ hg commit --amend

Then confirm (or edit) the commit message by saving with CTRL+O and exiting with CTRL+X (assuming the default editor is nano). Finally, update phabricator (you will need to provide a commit message):

$ arc diff

Create a new commit and squash history

If you prefer to have a separate commit, execute:

$ hg commit -m "Address review comments"

Then squash the commits together by editing history:

$ hg histedit

The following screen will look like this:

pick f6f70f6de69c 358597 Bug 123456 - [ur] Search engine setup for Firefox fo...
pick 8088fd8658fd 358598 Fix searchplugin name

# Edit history between f6f70f6de69c and 8088fd8658fd
#
# Commits are listed from least to most recent
#
# You can reorder changesets by reordering the lines
#
# Commands:
#
#  e, edit = use commit, but stop for amending
#  m, mess = edit commit message without changing commit content
#  p, pick = use commit
#  d, drop = remove commit from history
#  f, fold = use commit, but combine it with the one above
#  r, roll = like fold, but discard this commit's description
#

In this case you want to roll the second commit into the first one, so replace pick with roll (or r), save with CTRL+O and exit with CTRL+X (assuming the default editor is nano).

pick f6f70f6de69c 358597 Bug 123456 - [ur] Search engine setup for Firefox fo...
roll 8088fd8658fd 358598 Fix searchplugin name

...

Update Phabricator (you will need to provide a commit message):

$ arc diff

You can also use hg histedit to reword a commit message (set the commit line to edit). Just remember to complete the histedit after commit.

$ hg commit -m "Some changes"
$ hg histedit --continue

If necessary, you can rebase against another bookmark, like central or inbound

$ hg rebase -d central

More information about this workflow are available in the following pages:

  • https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/firefoxworkflow.html
  • https://www.mercurial-scm.org/wiki/Bookmarks

Landing the patch

Once the patch has been reviewed, you have two options:

  • If you have L3 access to the repository, you can use Lando to land your commit directly. If your reviewer has it, you can ask them to land.
  • You can set the checkin-needed keyword in the bug, and sheriffs will land it from you.

Creating a patch using Queues

Creating a patch

Consider the following example: setting up productization for Urdu (ur) on Firefox desktop (l10n repository). The first step is to create a region.properties file. Move into the repository folder and check its status:

$ cd ~/mozilla/mercurial/l10n/ur/l10n-central

$ hg status
? browser/chrome/browser-region/region.properties

File is brand new and needs to be added:

$ hg add browser

$ hg status
A browser/chrome/browser-region/region.properties

You need to assign a name to this patch, it’s easy to use a reference to the bug number: for example, if the bug number is 123456, the file could be called bug123456.patch (note the added extension .patch).

$ hg qnew bug123456.patch

At this point you will be asked to provide a commit message for your patch (in nano if you followed the instructions to set up the environment): write your commit message, then press CTRL+O to save the file, enter to confirm the proposed filename, and finally CTRL+X to exit the editor.

The commit message should be the same as the bug, for example Bug 123456 - Set up searchplugins for "ur" and Firefox desktop.

You are ready to pop the patch out of the queue. Since it’s likely that there are no other patches, you can pop them all with -a.

$ hg qpop -a
popping bug123456.patch
patch queue now empty

The patch is stored inside the .hg/patches folder in the root of the repository (in the suggested setup, the full path would be ~/mozilla/mercurial/l10n-central/.hg/patches). You can copy the file through the command line or the file explorer. For example, on macOS you can open the folder in Finder by typing:

$ open ~/mozilla/mercurial/l10n/ur/l10n-central/.hg/patches

Or you can copy the file on the Desktop with

$ cp ~/mozilla/mercurial/l10n/ur/l10n-central/.hg/patches/bug123456.patch ~/Desktop

Now you need to attach the file to Bugzilla and set an appropriate reviewer for it.

Updating an existing patch

Let’s assume that the review found some issues with the patch and you need to update it. The fastest way is to import the .patch file without committing, update the files as needed, and create a new patch using the same process explained above.

Assuming the file is called bug123456.patch and it’s in your desktop, you can move in the repository folder and import the file like this:

$ cd ~/mozilla/mercurial/l10n/ur/l10n-central

$ hg import --no-commit ~/Desktop/bug123456.patch

Don’t forget the --no-commit part. If you do, the patch will be added to your repository and you’ll need to clone the original repository again.

Note: You can drag and drop the patch file on the terminal on macOS to get its full path instead of typing it. In other words, type hg import --no-commit (leave an empty space at the end), then drag the icon of the patch on the Terminal’s window: its full path will appear automatically.

At this point you’re ready to modify the files, and create a new patch. The only difference is that you will need to use a different filename, for example bug123456v1.patch.

Landing the patch

Your patch got a r+, so you need to update the commit message to reference the review, import the patch and push it to the remote server.

Note: If the patch is for a repository where you don’t have write access, you don’t need to follow these instructions, only to set the checkin-needed flag in Bugzilla.

Open the .patch file in your editor, find the line with the commit message, and add r=NICKNAME to the commit message. For example, the last line in

# HG changeset patch
# User SOMENAME <SOME EMAIL>
# Parent  b45d23bc53df9227aa262acb2a5c6b0ab903b76e
Bug 123456 - Set up searchplugins for "ur" and Firefox for Android

Should become like this, assuming flod is the reviewer.

Bug 123456 - Set up searchplugins for "ur" and Firefox for Android, r=flod

Then you need to import the patch, this time without the no-commit parameter.

$ cd ~/mozilla/mercurial/l10n/ur/l10n-central

$ hg import ~/Desktop/bug123456.patch

The patch has been imported and committed. If you get an error while applying the patch, check if the editor you’re using hasn’t modified other lines of the patch, for example removing trailing whitespaces. In that case, using nano to edit the patch is probably your faster option.

Now you’re ready to push to the remote repository

$ hg push

The reply from the server will contain the URL to your changeset: copy and paste that URL in the bug. If the bug only contains a patch to region.properties, you can close the bug after landing; if it also contains a patch stored in MozReview, the bug will be automatically closed when the patch is merged to mozilla-central.

Useful Mercurial commands

Display log

To display the log you can use hg log -l NUMBER, where NUMBER is the number of commits. Note that Mercurial uses l instead of Git’s n. Use the space bar to move to the next page if the results don’t fit in the screen.

You can also browse the history by running hg serve within the repository folder, then opening http://localhost:8000

Display content of the last commit

To display the content of the last commit you can use:

$ hg log --patch --rev tip

Press the space bar to move to the next page, q to quit.

Revert staged changes

Let’s assume that you have staged (uncommitted) changes in your repositories but you want to go back to a clean state:

  • If you only want to reset one file, you can use hg revert FILENAME. Note that this will leave a FILENAME.orig in the repository, in case you decide to change your mind. To avoid that, use the -C option: hg revert -C FILENAME. You can use this command also to restore a file marked as ! (missing in the filesystem, but still tracked).
  • hg up -C will reset all changed files to the last commit. Note that it won’t remove untracked files.
  • To remove all untracked files, you can use the purge extension and run hg purge. It won’t remove staged changes, like modified files still uncommitted.

Revert changes already committed

If you want to revert changes already committed:

  • To backout a specific changeset use hg backout -r CHANGESET. This will prompt you directly with a request for the commit message to use in the backout.
  • To revert a file to a specific changeset, use hg revert -r CHANGESET FILENAME. This will revert the file without committing it.

Miscellaneous documentation

Creating a new repository for projects

Creating a new repository in the mozilla-l10n organization provides the advantage of letting l10n-drivers manage this repository directly, reducing delays when dealing with permissions. It also allows localizers to work directly on GitHub with pull requests.

Note that you need to be a manager of the organization in order to create a new repository.

Create the new repository

A new repository can be created in this page, or using the New button in the organization’s landing page.

Name: always include the -l10n suffix for localization repositories, to differentiate them from other repositories used for tools. Notable exceptions to this rule are appstores and www.mozilla.org.

For example, if the project is Focus for Android, a good repository name is focus-android-l10n.

Description: add a short description, e.g. Strings for the Focus (Android) project.

Set the project as Public and don’t initialize it with a README.

New repository

Add collaborators

Go into the project settings, panel Collaborators & teams:

  • Add the relevant l10n-drivers as Admin. You can either pick individuals or one of the pre-existing groups.
  • Only when the project is ready for localization, add the Localizers group with Write permissions. Note that Pontoon’s bot is included in the localizers group, so you will need to enable it for testing the sync process.
  • Add relevant developers as contributors if they need to make commits to the repository (that’s usually the case, in order to update strings).

New repository

Add a README.md to the project

Add a README with some useful information about the project. For example, for Focus for Android:

# Focus by Firefox localization
Localization for the Focus by Firefox project for Android.

The application code with build instructions can be found
at <https://github.com/mozilla-mobile/focus-android>.

# License
Translations in this repository are available under the
terms of the [Mozilla Public License v2.0](https://www.mozilla.org/MPL/2.0/).

Add a CODEOWNERS file

It’s possible to add a CODEOWNERS file to automatically ping the owner of the project in case of pull requests. File should be placed in the root of the repository, it’s called CODEOWNERS and uses the following format:

path/to/files/to/monitor @nickname_of_owner

For example, this is the content of the CODEOWNERS file for Focus for Android:

locales/templates/*.pot @delphine

Pull requests to update strings always modify locales/templates/app.pot, while delphine is the nickname of the PM in charge of this product. Any pull request trying to modify files matching that path will automatically send a review request to the code owner.

Add topics to the repository (optional)

In the main repository page you can add topics to make the repository more discoverable by potential contributors. For example, all localization projects have the following topics: mozilla-l10n, localization, translation.

Working with a Localization Agency

This document is a guideline for project manager who interacts with a localization agency that already successfully went through our vetting process. Our vetting process helps us ensure the vendors we work with are able to deliver translations in a timely manner, with the required level of quality and following our localization processes. We are also assessing their flexibility to work efficiently with our communities and our tools when it’s required.

Initial set up before sending localization requests

Ask for a rate sheet

  • Make sure you have received a rate sheet customized for Mozilla needs. This should have been sent as part of the vetting process, and it can be updated annually. An example of the official rate sheet that includes the following but not limited to:
    • Word rate for each of the interested locales as in new words, 100% match, fuzzy match for translation, review, proofreading.
    • Hourly rate: engineering, QA and testing, PM rate.
    • Minimum charge for a project and the thresholds by number of words per locale.
    • Rush rate, and its threshold.

Do the initial paperwork

A series of activities need to happen before an l10n project can start. The first step is doing the paperwork to be able to receive quotes for projects and pay bills for requests.

Unless there are changes in the signed document, this step is done once, for the whole organization, and only at the beginning of relationship engagement.

  • A master service agreement (or MSA) has to be signed between Mozilla and the vendor. The person signing on Mozilla side could be the PM or PM’s manager. This document shows the conditions that the contract is operated in between the service provider and Mozilla. Without this agreement, a service can’t start. Here is an example of MSA. Note that two separate MSA must be signed, one for Mozilla Corporation and another one for Mozilla Foundation.
  • An Account Payable (or AP) form has to be filled. It contains banking information, billing cycle (monthly, quarterly, or “as soon as”) and payment due terms (within 30 or 45 days), billing email address. Here is an example of AP form.

PO, or Purchase Order, may or may not be necessary in order to get vendor service. Some teams have ongoing need for l10n service and have a budget set aside annually. This is a good reason to have a PO established. For most projects that are one time need, or sporadic, it’s best to go without a PO, and bill to the serviced department as needed.

Set up accounts

Depending on the needs and where the source file can be accessed, it’s good to have an account set up for the vendor’s project manager in the following places:

  • Pontoon: set up a “translator” account when there is no community; or a basic “contributor” account when a community exists so that the vendor provides suggestions and the community can review them.
  • Bugzilla: so that the vendor’s project manager can interact in bugs. Bugzilla will allow us to have a central place to post project details and updates. Communication is generally handled via email with the vendor, but it’s useful to have a bug number to reference to, and the vendor can use the bug to quickly access all the details.
  • Google Drive: create a shared folder for the vendor to view, review, and translate directly spreadsheets or Google Docs.
  • GitHub: give commit permission if they need to commit directly in a repository (e.g. Legal docs).

Note: additional account setup may be needed if the project does not support our localization process (e.g. WordPress).

Once accounts are set up, allow the agency to explore the tools, flows and the product features to familiarize itself before actual work starts. Make sure to provide links to documentation as needed on how to use Pontoon and explain how to perform queries on Transvision. Setting up a meeting to share your screen might be a good idea, and use this opportunity to answer their questions about the project or the tools.

Localizing a project

Planning the request

Once outsourcing is a necessity for a project, before reaching out to the l10n agency to make an official request, make sure to check the following:

  • Required locales.
  • Scope of work: is the request asking for anything other than translation, review and testing?
  • Milestones and Deadlines: negotiate it so the agency has sufficient time to finish each of the required tasks, and the community has enough time to review and sign off.
  • Check with communities if they want to fully handle the request, handle only some parts (e.g. initial translation or QA), or can’t work on it at all and let the agency do the work.

Quotation

Now that you have all the initial info you need, you are good to go ahead and reach out to the vendor and provide what they need to generate a quote. Get a quote from the agency by communicating volume, locales, deadline, level of service needed, and type of content.

Setting everything up for the request

Now that you got a quote, you need to get it approved, send content to the vendor, and check permission.

  • Provide a quote/estimate to the project requestor for approval.
  • For all the target locales, make sure both agency and the community have the proper access to the l10n tool or platform where the localizable content is staged and roles properly assigned per agreement.
  • Information to share with both the vendor and the community:
    • Scope: volume, type of content being localized, list of locales and type of service.
    • Plan: schedule with milestones.
    • Expectations: how you want the translations to be delivered (as a single batch or as each locale is ready), the file format you want the translations to be delivered into, and any additional service you would need, like following a specific test plan.
  • Communicate the plan and the finalized schedule with the project stakeholders.
  • File a bug to track project progress. It can be a good idea to remind project specifics (repositories, deadline(s), locales…).

Executing the request

The initial quote has been approved, roles and permission are set accordingly to the request. You can now execute it, receive translations and deliver them to the project stakeholders.

  • Share the content with the agency:
    • Strings (whether it’s text files, a link to Pontoon, a link to a repository, a spreadsheet, a Google Doc…).
    • Locale specific style guides and product specific term list.
    • Terminology: Ask them to always refer to Transvision to check terminology.
  • Confirm with the vendor the following: word count, schedule and cost of the service.
  • Run testing/QA once you’ve received the translations, either by the community or the vendor, depending on the project agreement.
  • Communicate with communities for a chance to review and sign off on the project before releasing to the internal customer.

Post mortem

After the project is delivered, you need to handle payment with the vendor and feedback with the community. At the end of a calendar year, you would review the vendor working model and quality of paid service. You then provide feedback and evaluate if the service should be continued.

Steps to go through with the agency

  • Have the agency send an updated quote, in case there is a change of scope in the project.
  • PM reviews the quote and checks for discrepancies and then approves the quote.
  • The agency’s account payable contact sends invoice to mozilla@bill.com and accounting@mozilla.com.
  • In a few days, PM will be notified and asked to approve the invoice in bill.com system.
  • PM reviews the bill one more time, updates the Cost Center to the one corresponding the internal customer’s.

Steps to go through with the community

  • PM should check with community periodically to gather input on translation quality if needed but it’s not necessary to do it every time. A necessary case could be if the community wants to replace the translator for their language because there are quality concerns.
  • Gather feedback in a spreadsheet, with summary in category of the type of issues reported, and detailed examples to support the summary.
  • Mozilla localizers give an overall quality rating and suggest to keep using the same translator(s) or ask for a replacement. The PM then shares the feedback with the agency for rebuttal and further feedback if needed.

Terminology

Account Manager
Relationship, escalation of issues; handles complaints about quality or deadlines.
Project Manager
Handles day-to-day operations. Coordinates resources on the agency side; provides estimate of a project’s deadline and cost.
Account Payable Manager
Sends the bill to Mozilla Account Payable.

Documentation style guide

General rules

These are some of the general rules to follow when writing and reviewing documentation for this repository:

  • Address the reader as you, avoid using we.
  • Avoid using generic terms.
  • Avoid unnecessary empty lines in the document.

Typography

  • Lists: each list item should start with an uppercase letter, and end with a period (only exception when the item ends with a URL). Use the * markdown notation for each item.
  • Use lowercase after colons, only one space after a period.
  • Use proper quotes, like “example”, instead of straight double quotes.
  • Use typographical apostrophes instead of straight single quotes ' (code fragments are the only exception).
  • Use backticks `, bold, or italic to highlight text in a sentence, not quotes.
  • Use * instead of _ for bold and italic.
  • Try to use links instead of putting URLs directly in the text. For example, visit [this page](http://example.com) should be preferred to visit http://example.com.
  • Leave an empty line before and after code fragments, specify the language where possible. For example:
This is some text.

```PHP
$test = 'test';
```

This is some other text.

Titles

  • Use the dash syntax, leave one space after the dash, e.g. # This is a title.
  • Use sentence case, avoid title case.
  • Documents should always start with a 1st level title. There should be only one 1st level title across the document.
  • Leave an empty line before and after each title. The only exception is the 1st level title at the beginning of the page.
  • Make sure to follow a logic when using titles, e.g. a 3rd level title should be in a section starting with a 2nd level title of its own.

This is an example of a good structure:

# This is the main title for this document

## This is a section

This is some text.

### This is a subsection

This is some text.

## This is another section

This is some more text.

Bullet and numbered items

The first level has no indentation, and it’s completely aligned to the left. Sub levels should be indented using 4 spaces. Use * instead of - to create items. Example:

* The first level bullet point
    * A second level bullet point
        * A third level bullet point

Links

There are three kind of links when cross referencing a document: within a document, within a project group, and outside a project group. The [displayed text] should be to the point and within a pair of square brackets. The displayed text must be followed by (the link) enclosed in a pair of parenthesis. Follow these formats for each case:

  • Reference within the same document: link directly to the anchor, e.g. [General rules](#general-rules). Note that GitHub automatically creates anchors for titles: if the title is Title Example, the anchor will be lowercase, with spaces replaced by dashes, i.e. #title-example.
  • Reference within a project group: use relative links, instead of absolute links (starting with /) or full GitHub URLs. For example, to link to a document in the parent folder, use [other document](../other.md).
  • Reference outside the current repository: use the full GitHub URL, e.g. [OpenDesign GitHub repository](https://github.com/mozilla/OpenDesign/tree/master/2017).

Images

When adding images to a repository, make sure that the size is not too big for the content displayed on GitHub. In case of PNG files, make also sure to optimize files through online services like TinyPNG.

  • Create a folder that maps to the main folder in which your documented is stored.
  • Naming convention: if only a couple of images are needed, make the file name easily identifiable, e.g.: product_feature_type.png.
  • File name must not contain spaces and uppercase, and must use underscore to separate words as needed.
  • Image size: keep it under 800px. If necessary, create two versions: a small one and a bigger one to open when clicked.
  • Image format: ![Encoding bug](../assets/images/l10n_testing/encoding_bug.png).
  • When images contain personal information, such as email addresses, try to blur it out.

Tools

Atom is the best tool for editing Markdown files:

  • You can preview the content with CTRL+SHIFT+M. If you forget the shortcut, you can always use CMD+SHIFT+P to display the list of available commands, and search for Markdown Preview: Toggle.
  • You can install a package called smart-quotes-plus, which lets you swap straight quotes with curly ones. Once the package is installed, press CTRL+ALT+' to swap quotes on the entire document, or the text currently selected. Again, you can use CMD+SHIFT+P and search for Smart Quote Plus: Smartreplace. Only make sure to not swap " or ' in code fragments where these characters are wanted.

To install packages in Atom, open the Preferences, select the Install panel on the left and search for the package you’re interested in.

markdownlint, a Markdown linter, is run on automation (Travis) for each pull request. If you want to use this tool on your system, make sure to install Node.js, then run the following commands from the root of your repository (only once):

$ git clone https://github.com/flodolo/l10ndocs-linter
$ npm --prefix ./l10ndocs-linter install ./l10ndocs-linter

At this point you can launch the linter with this command:

$ node l10ndocs-linter/linter/markdownlint.js
Using config file: /your_path/documentation/l10ndocs-linter/linter/markdownlint.json
Searching path: /your_path/documentation

There are no linter errors.

Inquiries about joining Mozilla L10n

Here you’ll find a stockpile of templated responses to inquiries we frequently receive about Mozilla L10n. They’ve been centralized here so that you can simply copy and paste them into emails. While there is value to having these stock responses for common inquiries, we strongly feel that there is equal value to making them as personalized as possible. Take time to adjust the template in your email to include personalized information.

General inquiry

Hi [inquiree],

Thank you for your email and welcome to the Mozilla Community! We’re excited that you’re interested in joining our localization (L10n) effort.

Joining our L10n effort is easy. Within five easy steps you can start contributing to the project! We have a wiki page specially designed to help you get started. Please visit https://wiki.mozilla.org/L10n:Contribute and follow the five steps to get involved.

Could you also please provide me with some additional information about yourself, such as languages you speak, country you live in, and technical background?

You can also visit the following wiki pages to learn the details of how we localize:

[name]

Follow up to general inquiry

We’re very excited to have you join the Mozilla localization (l10n) effort! Your skills and dedication will be very valuable to your l10n team.

Next step, follow the steps outlined on https://wiki.mozilla.org/L10n:Contribute. When you contact your l10n team, please CC me on that email so that I can help to make sure you receive a prompt response.

If you have any questions or difficulties, please let me know.

Welcome!

Starting a new localization

Thank you for your email and welcome to the Mozilla Community! We’re excited that you’re interested in joining our localization effort.

Since you have stated that you would like to start a new localization effort, will be your mentor and point of contact.

Here’s how you can get started:

  • Check our list of existing localization communities to see if an effort for your language already exists. If an effort exists, contact the community to join the team and help drive the effort. If it doesn’t exist, introduce your new l10n community on the Mozilla localization mailing list.
  • Introduce yourself to the L10n community on the Mozilla localization mailing list by subscribing and posting an introduction. In your introduction, please add some brief information about yourself (e.g., professional background, interests, etc.) and what you hope to work on (e.g., "I saw that there was a xx-XX community and I’d like to help them out.").
  • Visit [L10n process doc needed] to become familiar with how we localize.

For general information on the Mozilla localization program, visit our [GitBook link]. You’re also welcome to read through and post to the localization discussion newsgroup. Hope to see more from you soon!

Localizing Thunderbird

We’re glad to see you’re interested in localizing Thunderbird. Please get in touch with the Thunderbird community to learn how to contribute to localization.

Having problems with Pontoon

Sorry to hear that you’re having problems with Pontoon. Please do the following to allow us to troubleshoot your problem(s):

  • Search Bugzilla to see if a bug has already been filed for your problem(s).
  • Check the l10n mailing list to see if anyone else has had the same problem.
  • If you can’t find a discussion thread about your problem, begin a discussion topic in those newsgroups asking for help. Include as much detail as possible.
  • If no one in the discussion group can offer a solution and there are no bugs currently filed, file a new bug providing as many details about the problem as possible.

If the first three steps don’t solve your problem and you have to file a bug, rest assured that the l10n-drivers will respond promptly.

Having problems with Mercurial (hg)

Sorry to hear that you’re having problems with hg. Please do the following to allow us to troubleshoot your problem(s):

  • Search Bugzilla to see if a bug has already been filed for your problem(s).
  • Check the l10n mailing list to see if anyone else has had the same problem.
  • If you can’t find a discussion thread about your problem, begin a discussion topic in those newsgroups asking for help. Include as much detail as possible.
  • If no one in the discussion group can offer a solution and there are no bugs currently filed, file a new bug providing as many details about the problem as possible.

If the first three steps don’t solve your problem and you have to file a bug, rest assured that the l10n-drivers will respond promptly.

Update your team wiki page

Subject: [locale code]Please update [locale name] wiki page

I’m writing you to verify that your L10n team’s information on your wiki page (https://wiki.mozilla.org/L10n:Teams:[your-locale-here]) is up-to-date and accurate. Please take a look at your page and update it as needed.

Please follow the format outlined in this wiki template.

If you have any questions, please feel free to contact me.

Thank you for your help!

Translation/l10n vendor sales pitches

It is a pleasure to meet you.

Mozilla is a non-profit, open source project. As such, all of our work, from development to marketing to localization, is performed by a dedicated, global community of volunteers. Our community is so thorough that it is very rare that we use vendors for translation, but when we do, we have a set of qualified vendors we rely on.

Of course, if you are interested in discussing a form of collaboration that fits with our volunteer, open source framework, I would be happy to arrange a meeting.

Thank you for your interest.

Responses to add

  • Cold Call Inquiry: Would you to contribute to XX localization?
  • Reactivation: Are you still interested in contributing to L10n? Here’s where we need your help.
  • Assigning first tasks based on user profiles/interests
  • Phasing out a locale

Setting up a Linux Virtual Machine for Webdashboards

This document describes how to configure a virtual machine based on Linux Ubuntu 18.04 LTS to manage tools like Langchecker, Webdashboard, Stores, Bedrock (mozilla.org). The assumption is that you’ve already installed a software to manage virtual machines, like VMWare Fusion on Mac, and installed Linux. The virtual machine should have at least 40 GB of disk, 2 cores and 4 GB or memory.

Update the system

First of all, make sure to update the installed packages. Search for the terminal and type:

sudo sh -c "apt update;apt dist-upgrade -y;apt autoremove -y"

The password for sudo is the same you use to login to the system with your user. Once finished, reboot the virtual machine to make sure that all packages are updated.

If you’re using VMWare, make also sure to install VMWare Tools:

sudo apt install open-vm-tools open-vm-tools-desktop -y

Restart the virtual machine at the end to make sure all packages are up to date.

Configure Git and repositories

Before starting the setup you need to make sure to setup Git to use SSH on this VM.

Install Git:

sudo apt install git -y

Generate a new SSH key for this VM (replace the email address placeholder), make sure to setup a passphrase when asked:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/github_ssh -C "YOUR_EMAIL_ADDRESS"

File will be stored in ~/.ssh/github_ssh. The next step is to add this key to your GitHub account.

cat ~/.ssh/github_ssh.pub

Add a new key in your profile and copy and paste the test displayed by the previous command, starting with ssh-rsa and finishing with the email address you provided.

At this point you’re ready to test your connection to GitHub (you will need to accept the certificate by typing yes)

ssh -T git@github.com
The authenticity of host 'github.com (192.30.253.112)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Hi USERNAME! You've successfully authenticated, but GitHub does not provide shell access.

It will ask for your passphrase, you can set the system to unlock this key whenever you’re logged in by selecting the checkbox under the input field.

You should also tweak your Git configuration with these commands (replace placeholders with your email address and name, they will be used for commit messages):

git config --global user.email "YOUR_EMAIL_ADDRESS"
git config --global user.name "YOUR_NAME"
git config --global push.default simple

Run the setup scripts

At this point you can run this command to download and execute a setup script. Note that this script must be run with sudo, since it needs to install several packages on the system.

sudo sh -c "wget -O - https://raw.githubusercontent.com/mozilla-l10n/vm_scripts/master/setup_vm/setup_vm.sh | bash"

Then run this script as standard user to clone all requested repositories:

~/mozilla/git/scripts/setup_vm/setup_repositories.sh

The entire procedure will take several minutes, depending on the speed of your Internet connection. During the process you will be asked, possibly more than once, to create a token on GitHub for Composer: follow the link to generate one and copy and paste it in the terminal.

All projects are now cloned using https and the original repository as origin (e.g. mozilla-l10n for langchecker), so you won’t be able to commit any changes. If you plan to make updates and commit them, make sure to create forks of the following repositories in your account:

  • langchecker: https://github.com/mozilla-l10n/langchecker
  • webdashboard: https://github.com/mozilla-l10n/webdashboard
  • stores_l10n: https://github.com/mozilla-l10n/stores_l10n
  • bedrock: https://github.com/mozilla/bedrock

Then run the following script providing your GitHub username (e.g. flodolo): it will setup the original repository as a remote called upstream, and your fork as origin. It will also switch relevant l10n repositories, like mozilla_org, to ssh to allow you to make direct commits.

~/mozilla/git/scripts/setup_vm/setup_remotes.sh GITHUB_USERNAME

Make sure to restart the terminal to enable the new command aliases. It’s recommended to use Terminator, installed as part of the setup script: it’s an improved terminal with support for tabs and split windows.