Unlike current xkbcommon, X11’s xkbcomp allows to remove entries in
the modifiers’ map using “modifier_map None { … }”.
“None” is translated to the special value “XkbNoModifier” defined in
“X11/extensions/XKB.h”. Then it relies on the fact that in "CopyModMapDef",
the following code:
1U << entry->modifier
ends up being zero when “entry->modifier” is “XkbNoModifier” (i.e. 0xFF).
Indeed, it relies on the overflow behaviour of the left shift, which in
practice resolves to use only the 5 low bits of the shift amount, i.e.
0x1F here. Then the result of “1U << 0xFF” is cast to “char”, i.e. 0.
This is a good trick but too magical, so in libxkbcommon we will use
an explicit test against our new constant XKB_MOD_NONE.
`_MSC_VER` is specific to MSVC, but there can be other compilers targeting
windows. Hopefully they do define `_WIN32`, so let's use that.
Refs: https://github.com/xkbcommon/libxkbcommon/issues/305
Signed-off-by: Ran Benita <ran@unusedvar.com>
Not sure what it's doing here, but converting "€" to a keysym
doesn't work with this entry. 0x13a4 doesn't appear in
xkbcommon-keysyms.h. 0x20ac is the keysym documented in the
header (and it's the last entry in the table).
It's been in the table since it was introduced in e0524296d2
("Add API for getting unicode representation of a keysym").
Co-authored-by: Simon Ser <contact@emersion.fr>
lokid and hikid actually stores the sibling to current node, which
should not be cleared when override. This would break the sequence with
a common prefix when override another.
Fix#286
Signed-off-by: Weng Xuetian <wengxt@gmail.com>
This is hit when passing an empty string and XKB_KEYSYM_CASE_INSENSITIVE
to xkb_keysym_from_name currently if `(lo + hi) / 2` is 0 and `cmp < 0`,
causing mid to underflow and the the array access into name_to_keysym on
the next iteration of the loop to be out of bounds .
We *would* use ssize_t here as it is the appropriate type, but windows
unfortunately does not define it.
Add a skipError test function that can analyse rc, stdout, stderr to skip a test
even when we expect an error. We use that to skip if we couldn't find a keyboard
in the interactive-evdev test.
Fixes#235
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
tqdm prints to stderr by default but we're using that for failed keymap
compiles (which are the ones that really matter). Plus, whether we're using tqdm
is dependent on isatty(sys.stdout) anyway.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
With --keymap-output-dir, the given directory will contain a list of files named
after the layout + variant ('us', 'us(euro)', ...) that contain the keymaps for
each variant + option combination compiled.
This is still a lot, but better to sift through hundreds of keymaps than tens of
thousands.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The previous output is largely unusable. The result in the CI test runs is a 6GB
file with every compiled keymap in it and while we can grep for ERROR, it's not
particularly useful.
Let's change this and print out YAML instead - that can be machine-processed.
This patch adds a new parent class that prints itself in YAML format,
the tool invocations are child classes of that class. The result looks like this:
Example output:
- rmlvo: ["evdev", "pc105", "us", "haw", "grp:rwin_switch"]
cmd: "xkbcli-compile-keymap --verbose --rules evdev --model pc105 --layout us --variant haw --options grp:rwin_switch"
status: 0
- rmlvo: ["evdev", "pc105", "us", "foo", ""]
cmd: "xkbcli-compile-keymap --verbose --rules evdev --model pc105 --layout us --variant foo"
status: 1
error: "failed to compile keymap"
Special status codes are: 99 for "unrecognized keysym" and 90 for "Cannot open
display" in the setxkbmap case.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Relatedly, strtoul allows a lot of unwanted stuff (spaces, +/- sign,
thousand seperators), we really ought not use it. But that's for another
time.
Signed-off-by: Ran Benita <ran@unusedvar.com>
A simple script that creates a new layout with the given keysym replacing TLDE.
Then we compile a keymap and search for the keysym being assigned to TLDE and
bail if that fails.
The list of keysyms is manually maintained but we only need to add one or two to
spot-check whenever the xorgproto is updated.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
rmlvos is the parent list which then fails during a list join because, well,
it's a list of lists.
Fixes#206
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Makes this test easier to run from the commandline. Where either of top_srcdir
or top_builddir isn't set, fill them in from the CWD or fail otherwise.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Where resolve_keysym fails we warn but use the otherwise uninitialized variable
as our keysym. That later ends up in the keymap as random garbage hex value.
Simplest test case, set this in the 'us' keymap:
key <TLDE> { [ xyz ] };
And without this patch we get random garbage:
./build/xkbcli-compile-keymap --layout us | grep TLDE:
key <TLDE> { [ 0x018a5cf0 ] };
With this patch, we now get NoSymbol:
./build/xkbcli-compile-keymap --layout us | grep TLDE:
key <TLDE> { [ NoSymbol ] };
For me, installing pytest for libxkbcommon is a bit problematic, so I
end up skipping it which is not great.
Switch to unittest which is built in to Python. It's not as nice as
pytest but good enough in this case.
Note: I was too lazy to switch the plain asserts to unittest
assertions...
Signed-off-by: Ran Benita <ran@unusedvar.com>
mkdtmp, rmdir and unlink are in unstd.h on MacOS. Since including that it
doesn't hurt us on Linux, let's do it without ifdefs.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This completes the usual triplet of configuration locations available for most
processes:
- vendor-provided data files in /usr/share/X11/xkb
- system-specific data files in /etc/xkb
- user-specific data files in $XDG_CONFIG_HOME/xkb
The default lookup order user, system, vendor, just like everything else that
uses these conventions.
For include directives in rules files, the '%E' resolves to that path.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
'xkbcli compile-keymap' doesn't work unless we ninja install first. But for a
test that's to be run from the test directory, that's not a useful option so
let's call the binary directly. The script adds the meson builddir to the PATH
anyway.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Let's not have our tests fail if the user has an incompatible
$XDG_CONFIG_HOME/xkb directory.
libxkbcommon has fallbacks when XDG_CONFIG_HOME isn't set so we need to override
this with a real directory instead of just unsetting it.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The xkbcli tool usage help is ifdef'd out where the tool isn't built but the
man page always includes all tools. Easier that way.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Python leaks like crazy when run under valgrind. But if we make the script
executable **and** it has uses the env invocation (i.e. #!/usr/bin/env python3),
the leaks disappear. This is not the case for a shebang of /usr/bin/python3.
Why exactly this is the case I'm not sure but executables we plan to run
should have the exec bit set. So this is a janitor patch with the nice side
effect of fixing our valgrind runs.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The CI started installing some wrapper instead of a real bash which is
what gets found.
See:
https://github.com/actions/virtual-environments/pull/1081
Given meson is written in python, it should always be available
hopefully.
Disabled valgrind wrapper for now because it now also applies to the
python interpreter which leaks like a sieve.
Signed-off-by: Ran Benita <ran@unusedvar.com>
We only ever care about whether we error out or not, so let's wrap this into
something more sane.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This library is the replacement for clients parsing evdev.xml directly.
Instead, they should use the API here so that in the future we may even
be able to swap evdev.xml for a more suitable data format.
The library parses through evdev.xml (using libxml2) and - if requested -
through evdev.extras.xml as well. The merge approach is optimised for
the default case where we have a system-installed rules XML and another file in
$XDG_CONFIG_DIR that adds a few entries. We load the system file first, then
append any custom ones to that. It's not possible to overwrite the MLVO list
provided by the system files - if you want to do that, get the change upstream.
XML validation is handled through the DTD itself which means we only need to
check for a nonempty name, everything else the DTD validation should complain
about.
The logging system is effectively identical to xkbcommon.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Returns true on success or false on error _or_ truncation. Since truncation is
almost always an error anyway, we might as well make this easier to check.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
These were moved to tools/tools-common.c and now that all tools are switched
over, they're no longer needed.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Move (sometimes duplicate) the required bits into new shared files
tools-common.(c|h) that are compiled into the internal tools library. Rename the
test_foo() functions to tools_foo() and in one case just copy the code of the
keymap compile function to the tool.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Passing -errno around and having separate labels depending on failure types is
superfluous here. All the unref calls can handle NULL and nothing cares about
errno once we're out of the immediate scope. So let's simplify this and deal
with 0 and 1 only.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Using test helpers to init the context gives it fairly specific behavior; unless
the user sets the right environment variables and/or calls it from the right
PWD, it may or may not include the test data.
Let's drop this behavior, make it a default tool to compile a keymap. If there
is a specific need to modify the include paths, we can add this later.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Commit 16c84cdd81 removed the getopt handling for
RMLVO arguments, so now this tool only takes a keymap file and compiles it.
Using test helpers to init the context gives it fairly specific behavior; unless
the user sets the right environment variables and/or calls it from the right
PWD, it may or may not include the test data.
Let's drop this behavior, make it a default tool to compile a keymap. If there
is a specific need to modify the include paths, we can add this later.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The unref() functions take NULL as argument, so we don't need different labels
for every possible exit path.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
If the active set of modifiers doesn't match any explicit entry of the
key type, the resulting level is 0 (i.e. Level 1). Some key types don't
explicitly map Level 1, taking advantage of this fallback.
Previously, xkb_keymap_key_get_mods_for_level didn't consider this, and
only reported masks for explicit mappings. But this causes some glaring
omissions, like matching "a" in the "us" keymap returning not results.
Since every mask which isn't explicitly mapped falls back to 0, we can't
return the all. Almost always the best choice for this is the empty
mask, so return that, when applicable.
Fixes https://github.com/xkbcommon/libxkbcommon/issues/140.
Reported-by: https://github.com/AliKet
Signed-off-by: Ran Benita <ran@unusedvar.com>
The program takes a unicode codepoint and an RMLVO and prints out all
key + modifier combinations that would result in that codepoint.
The program was written to exercise the new
xkb_keymap_key_get_mods_for_level() function. It's handy and can be
extended in several ways, but enough for now.
Example:
$ ./build/how-to-type -l us,il,ru 0x41 | column -ts $'\t'
keysym: A (0x41)
KEYCODE KEY NAME LAYOUT# LAYOUT NAME LEVEL# MODIFIERS
38 AC01 1 English (US) 2 [ Shift ]
38 AC01 1 English (US) 2 [ Lock ]
38 AC01 2 Hebrew 2 [ Shift ]
38 AC01 2 Hebrew 2 [ Lock ]
$ ./build/how-to-type -l de -v neo 0x3b6 | column -ts $'\t'
keysym: Greek_zeta (0x7e6)
KEYCODE KEY NAME LAYOUT# LAYOUT NAME LEVEL# MODIFIERS
56 AB05 1 German (Neo 2) 4 [ Shift Mod5 ]
56 AB05 1 German (Neo 2) 4 [ Shift Mod2 Mod3 Mod5 ]
56 AB05 1 German (Neo 2) 4 [ Shift Lock Mod5 ]
56 AB05 1 German (Neo 2) 4 [ Lock Mod2 Mod3 Mod5 ]
Signed-off-by: Ran Benita <ran@unusedvar.com>
The new API is useful to implement features like auto-type and
desktop automation. Since the inputs for these features is usually
specified in terms of the symbols that need to be typed, the
implementation needs to be able to invert the keycode->keysym
transformation and produce a sequence of keycodes that can be used
to type the requested character(s).
In order to support features like auto-type and UI automation, the
relevant tools need to be able to invert the keycode->keysym->text
transformation. In order to facilitate that, a new API was added.
It allows querying the keysyms that correspond to particular Unicode
codepoints. For all practical purposes, it can be thought of as an
inverse of xkb_keysym_to_utf32().
This provides implementations of the test_enable_stdin_echo and
test_disable_stdin_echo which do not require <termios.h>, which is
not available on Windows.
Only the input/output functions from <unistd.h> options are used, so
using <io.h> when building with MSVC should be enough. The inclusion
of the header in context-priv.c does not seem to be needed (tested
on GNU/Linux) and so it is removed.
Signed-off-by: Ran Benita <ran@unusedvar.com>
The tests stringcomp and buffercomp do binary comparison on some files;
if the files are changed to CRLF on checkout, the tests fail.
Signed-off-by: Ran Benita <ran@unusedvar.com>
From MSVC:
test\atom.c(98): note: consider using '%zu' in the format string
test\atom.c(98): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(100): note: consider using '%zu' in the format string
test\atom.c(100): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(114): note: consider using '%zu' in the format string
test\atom.c(114): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(128): note: consider using '%zu' in the format string
test\atom.c(128): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(130): note: consider using '%zu' in the format string
test\atom.c(130): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(137): note: consider using '%zu' in the format string
test\atom.c(137): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 2 has type 'size_t'
Signed-off-by: Ran Benita <ran@unusedvar.com>
Previously we included it with an `-include` compiler directive. But
that's not portable. And it's better to be explicit anyway.
Every .c file should have `include "config.h"` first thing.
Signed-off-by: Ran Benita <ran@unusedvar.com>
The majority use-case for extending XKB on a machine is to override one or a
few keys with custom keycodes, not to define whole layouts.
Previously, we relied on the rules file to be a single file, making it hard to
extend. libxkbcommon parses $XDG_CONFIG_HOME/xkb/ but that only works as long
as there is a rule that matches the user-specified RMLVO. This works for MLV
but not for options which don't have a wildcard defined. Users have to copy
the whole rules file and then work from there - not something easy to extend
and maintain.
This patch adds a new ! include directive to rules files that allows including
another file. The file path must be without quotes and may not start with the
literal "include". Two directives are supported, %H to $HOME and %S for the
system-installed rules directory (usually /usr/share/X11/xkb/rules).
A user would typically use a custom rules file like this:
! option = symbols
custom:foo = +custom(foo)
custom:bar = +custom(baz)
! include %S/evdev
Where the above defines the two options and then includes the system-installed
evdev rule. Since most current implementations default to loading the "evdev"
ruleset, it's best to name this $XDG_CONFIG_HOME/xkb/rules/evdev, but any
valid name is allowed.
The include functionally replaces the line with the content of the included
file which means the behavior of rules files is maintained. Specifically,
custom options must be defined before including another file because the first
match usually wins. In other words, the following ruleset will not assign
my_model as one would expect:
! include %S/evdev
! model = symbols
my_model = +custom(foo)
The default evdev ruleset has wildcards for model and those match before the
my_model is hit.
The actual resolved components need only be in one of the XKB lookup
directories, e.g. for the example above:
$ cat $XDG_CONFIG_HOME/xkb/symbols/custom
partial alphanumeric_keys
xkb_symbols "foo" {
key <TLDE> { [ VoidSymbol ] };
};
partial alphanumeric_keys
xkb_symbols "baz" {
key <AB01> { [ k, K ] };
};
This can then be loaded with the XKB option "custom:foo,custom:bar".
The use of "custom" is just as an example, there are no naming requirements
beyond avoiding already-used ones. Also note the bar/baz above - the option
names don't have to match the component names.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This requires (well, at least implemented by) casting away `const` which
is undefined behavior, and clang started to warn about it.
The micro optimization didn't save too many allocations, anyway.
Signed-off-by: Ran Benita <ran@unusedvar.com>
A few lines above we check path_rel[0], so any null pointer will blow up
before we get here.
Found by coverity
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
python3 is always python3, but python could be python2 in some cases. Or just
missing (e.g. RHEL8).
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This way stdin/stdout of the process are opened in text mode and we don't need
manually decode.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
In python multiprocessing, each process needs to handle (and ignore) the
KeyboardInterrupt to avoid exception logging. This is a separate patch for
easier reviewing, the first hunks merely re-indent all of the
xkbcommontool/xkbcomp functions into a try/except KeyboardInterrupt block.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This is a change in behavior and requires any automated callers to adjust
accordingly. Still, much easier to get the errors that way rather than it
being mixed into a thousands-of-lines output file.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Collect all options into a dictionary, then process that as async actions
through a process pool. This of course requires collecting the various print
statements to avoid mangled output.
This dropped the time to completion from around 14 min to 8 min on my local
machine (unscientific single run only for the original timing).
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Wismill
Ran Benita
Sam Lantinga
Weng Xuetian
Peter Hutterer
M Hickford
Isaac Freund
Nils
Jaroslaw Kubik
Adrian Perez de Castro