xkbcommon and my custom Russian phonetic layout

Custom layout - the naive way

The naive way to create a custom layout is by modifying an existing variant definition in one of the symbols files from /usr/share/X11/xkb/symbols. That's what I originally did — I modified the ru file and its phonetic section:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

partial alphanumeric_keys
xkb_symbols "phonetic" {

    name[Group1]= "Russia - Phonetic";

    key <TLDE>   {[       Cyrillic_yu,       Cyrillic_YU ]};
    key <AE01>   {[                 1,            exclam ]};
    key <AE02>   {[                 2,                at ]};
    key <AE03>   {[                 3,        numbersign ]};
    ...
    ...
}

Here, the first column contains the symbolic key names [1], the second column contains the corresponding key symbol for a key pressed without modifiers, and the last column contains the key symbol for a key pressed with Shift.

This works, BUT your user needs write permissions for the file, and the file will be overwritten anyway every time XKB gets updated by the system.

Can you believe that for over 15 years I used the same customised ru symbols file, and replaced the upstream file over and over? I did this manually every time the package responsible for /usr/share/X11/xkb got updated. The file followed me as I moved from Ubuntu to Debian, and from Debian to Arch. And honestly speaking, I never bothered to dig deeper and learn the mechanisms behind the command to set up my desired layouts:

setxkbmap -layout 'us,fi,ru,az' -variant ',,phonetic,' -option grp:shifts_toggle

That allowed me to cycle through US, Finnish, Russian phonetic and Azeri layouts by pressing both shifts.

This changed when I installed Arch on my new laptop. I finally decided to find a "smart way" and learn the mechanisms behind layouts in Linux graphical environments.

Bits about XKB

Imagine a system with a graphical environment backed by Wayland compositor. Let's find out how pressing a key on the keyboard results in a symbol appearing in an application. I'll omit deep technicalities to keep the narrative short.

1. Wayland compositor notifies the application about the current keymap configuration. This happens when application starts or the user changes keyboard layout. The application loads the keymap via xkbcommon library.
2. The keyboard device sends a hardware signal - a scancode to the Linux Kernel.
3. The kernel's evdev subsystem maps the scancode to a keycode - a representation abstracted from hardware.
4. Wayland compositor uses libinput to receive keyboard and other input device events from the kernel. It notifies the application (a Wayland client) about a pressed/released key with a keycode, and modifiers.
5. The application (using GTK, Qt, or other Wayland client implementations) passes the loaded keymap context, the keycode and modifiers to xkbcommon.
6. xkbcommon applies keycodes/compat/geometry/symbols/types (KcCGST) configuration [2], to map the input to the final symbol.

In a nutshell, it's xkbcommon's job to translate Shift + L keystroke to Cyrillic Л symbol, when I have my Russian phonetic layout activated.

As Daniel Stone, the maintainer of xkbcommon nicely puts it:

It’s about two things: parsing and loading keymaps, and managing their ongoing state. State, in terms of keyboards, is the usual suspects — modifiers (e.g. Shift, Alt), multiple layouts (mostly for multiple languages), and LEDs. In general, you only want one person keeping a canonical copy of the state, and distributing it to its clients, as both the X server and Wayland do today. xkbcommon allows for this mode of operation, and is indeed how all current Wayland clients handle keyboard input [3].

Custom layout - the smart way

The xkbcommon documentation comes with a tutorial on adding a custom keyboard layout [6]. I don't want to repeat that here, and rather encourage you to dive into it and get the information from the original source.

However, I'll briefly describe my Russian Alternative Phonetic setup.

1. Symbols file:

   I placed my custom layout variant in a new symbols file $HOME/.config/xkb/symbols/ru_alternative. This file contains translation of symbolic key codes into the desired key symbols (link).
2. Rules file:

   I added a rule to ~/.config/xkb/rules/evdev file (link). The purpose of the rules file is to map between user-friendly configuration, and the configuration used by xkbcomp keymap compiler [7].

3. I made the layout discoverable in GNOME by adding an entry to ~/.config/xkb/rules/evdev.xml (link).

That's it! A simple solution for a single-person machine, and no more half-baked solutions with broken layouts after system updates :)

References