Leading BOM is legal and is used as a signature — an indication that
an otherwise unmarked text file is in UTF-8.
See: https://www.unicode.org/faq/utf_bom.html#bom5 for further details.
- Update keymap to use reference keysym names.
- Fix x11comp test by handling old x11proto.
We need xkbcomp to be compiled with at least x11proto-dev 2023.2.
So we replace the unsupported keysyms with supported ones not
already in the keymap. This is kind of ugly, but it works. If we
ever want to restore the original keysyms with their supported names,
the substitute keysyms will be easy to spot.
The octal parser accepts the range `\1..\777`. The result is cast to
`char` which will silently overflow.
This commit prevents overlow and will treat `\400..\777` as invalid
escape sequences.
NULL usually terminates the strings; allowing to produce it via escape
sequences may lead to undefined behaviour.
- Make NULL escape sequences (e.g. `\0` and `\x0`) invalid.
- Add corresponding test.
- Introduce the new message: XKB_WARNING_INVALID_ESCAPE_SEQUENCE.
- Add a keymap test with decimal and hexadecimal keysyms.
- Reorganize code in `test/keysym.c` by parsing type: name, Unicode and
hexadecimal.
- Add more tests for edge cases. In particular:
- test decimal format (currently not supported);
- test the Unicode and hexadecimal ranges more thoroughly;
- test with wrong case without the XKB_KEYSYM_CASE_INSENSITIVE flag;
- test surrounding spaces.
- Document the tests.
Unlike current xkbcommon, X11’s xkbcomp allows to remove entries in
the modifiers’ map using “modifier_map None { … }”.
“None” is translated to the special value “XkbNoModifier” defined in
“X11/extensions/XKB.h”. Then it relies on the fact that in "CopyModMapDef",
the following code:
1U << entry->modifier
ends up being zero when “entry->modifier” is “XkbNoModifier” (i.e. 0xFF).
Indeed, it relies on the overflow behaviour of the left shift, which in
practice resolves to use only the 5 low bits of the shift amount, i.e.
0x1F here. Then the result of “1U << 0xFF” is cast to “char”, i.e. 0.
This is a good trick but too magical, so in libxkbcommon we will use
an explicit test against our new constant XKB_MOD_NONE.
Where resolve_keysym fails we warn but use the otherwise uninitialized variable
as our keysym. That later ends up in the keymap as random garbage hex value.
Simplest test case, set this in the 'us' keymap:
key <TLDE> { [ xyz ] };
And without this patch we get random garbage:
./build/xkbcli-compile-keymap --layout us | grep TLDE:
key <TLDE> { [ 0x018a5cf0 ] };
With this patch, we now get NoSymbol:
./build/xkbcli-compile-keymap --layout us | grep TLDE:
key <TLDE> { [ NoSymbol ] };
The tests stringcomp and buffercomp do binary comparison on some files;
if the files are changed to CRLF on checkout, the tests fail.
Signed-off-by: Ran Benita <ran@unusedvar.com>
The majority use-case for extending XKB on a machine is to override one or a
few keys with custom keycodes, not to define whole layouts.
Previously, we relied on the rules file to be a single file, making it hard to
extend. libxkbcommon parses $XDG_CONFIG_HOME/xkb/ but that only works as long
as there is a rule that matches the user-specified RMLVO. This works for MLV
but not for options which don't have a wildcard defined. Users have to copy
the whole rules file and then work from there - not something easy to extend
and maintain.
This patch adds a new ! include directive to rules files that allows including
another file. The file path must be without quotes and may not start with the
literal "include". Two directives are supported, %H to $HOME and %S for the
system-installed rules directory (usually /usr/share/X11/xkb/rules).
A user would typically use a custom rules file like this:
! option = symbols
custom:foo = +custom(foo)
custom:bar = +custom(baz)
! include %S/evdev
Where the above defines the two options and then includes the system-installed
evdev rule. Since most current implementations default to loading the "evdev"
ruleset, it's best to name this $XDG_CONFIG_HOME/xkb/rules/evdev, but any
valid name is allowed.
The include functionally replaces the line with the content of the included
file which means the behavior of rules files is maintained. Specifically,
custom options must be defined before including another file because the first
match usually wins. In other words, the following ruleset will not assign
my_model as one would expect:
! include %S/evdev
! model = symbols
my_model = +custom(foo)
The default evdev ruleset has wildcards for model and those match before the
my_model is hit.
The actual resolved components need only be in one of the XKB lookup
directories, e.g. for the example above:
$ cat $XDG_CONFIG_HOME/xkb/symbols/custom
partial alphanumeric_keys
xkb_symbols "foo" {
key <TLDE> { [ VoidSymbol ] };
};
partial alphanumeric_keys
xkb_symbols "baz" {
key <AB01> { [ k, K ] };
};
This can then be loaded with the XKB option "custom:foo,custom:bar".
The use of "custom" is just as an example, there are no naming requirements
beyond avoiding already-used ones. Also note the bar/baz above - the option
names don't have to match the component names.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
xkbcomp only accepts the "Level" prefix for a level name for levels 1 to
8, but the keymap dumping code added it always, e.g. "Level15".
The plain integer, e.g. "8", "15" is always accepted, so just use that.
Fixes https://github.com/xkbcommon/libxkbcommon/issues/113
Signed-off-by: Ran Benita <ran@unusedvar.com>
Reported-by: progandy
The `test/data/keymaps/host.xkb` file contains a duplicate definition of
this type. On my computer (linux, xkbcomp 1.3.0, xserver 1.17.2), the
test passes as is, but if I remove the duplicate definition, the
roundtrip brings it back and the test fails. I can also reproduce it
without relation to the test, by loading `test/data/keymaps/host.xkb`
(without the duplicate) using
xkbcomp -I $(pwd)/test/data/keymaps/host.xkb $DISPLAY
and downloading it again using
xkbcomp $DISPLAY out.xkb
the duplicate is added. On Mac OS X however, the duplicate is removed
(correctly), so the test fails there.
xkbcommon itself, which was forked from xkbcomp, doesn't have this bug;
in fact, doing
./test/print-compiled-keymap -k keymaps/host.xkb
removes the duplicate if it is present.
This is (probably) a regression in xkbcomp or xserver compared to the
versions used in Mac OS X. Since getting a patch for any of these two is
hopeless from my experience, I did not try to investigate further.
I am not sure why, but if I also add a `PC_SUPER_LEVEL2` type, the
duplicate of `FOUR_LEVEL_KEYPAD` doesn't show up. Hopefully the test
will work on all platforms now.
https://github.com/xkbcommon/libxkbcommon/issues/26
Reported-by: @nuko8
Signed-off-by: Ran Benita <ran234@gmail.com>
Some results from the benchmark (compilation of en_US.UTF-8/Compose):
$ grep 'model name' /proc/cpuinfo
model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
$ uname -a
Linux ran 3.16.1-1-ARCH #1 SMP PREEMPT Thu Aug 14 07:40:19 CEST 2014 x86_64 GNU/Linux
$ ./test/compose bench
compiled 1000 compose tables in 7.776488331s
So according to the above benchmark and valgrind --tool=massif, an
xkb_compose_table adds an overhead of about ~8ms time and ~130KB
resident memory.
For contrast, a plain US keymap adds an overhead of ~3ms time and 90KB
resident memory.
Signed-off-by: Ran Benita <ran234@gmail.com>
If the keymap doesn't have any key-aliases (which is certainly
possible), the calloc(num_key_aliases, ...) is allowed to return NULL
according to the C standard, but this is not an error.
Signed-off-by: Ran Benita <ran234@gmail.com>
It is like test/stringcomp, only instead of using
xkb_keymap_new_from_string(), it uses xkbcomp to upload the keymap to a
dummy Xvfb X server and then xkb_x11_keymap_new_from_device().
If any of these components are not present or fails, the test is shown
as skipped.
The test is messy, fragile, limited and depends on external tools, but I
will improve on that later -- it's better to have a test.
Signed-off-by: Ran Benita <ran234@gmail.com>
With the following two rules:
InterpretDecl : INTERPRET InterpretMatch OBRACE
VarDeclList
CBRACE SEMI
{ $2->def = $4; $$ = $2; }
;
InterpretMatch : KeySym PLUS Expr
{ $$ = InterpCreate($1, $3); }
| KeySym
{ $$ = InterpCreate($1, NULL); }
;
And the fact that InterpCreate doesn't initialize ->def, if the
VarDeclList fails, the %destructor tries to recursively free the
uninitialized ->def VarDef. So always initialize it.
That was the only problematic code in the parser for %destructor (I'm
pretty sure).
Signed-off-by: Ran Benita <ran234@gmail.com>
We didn't really have any. It also a exposes a memory leak, since the
parser doesn't clean up the AST nodes of the discarded symbols.
Signed-off-by: Ran Benita <ran234@gmail.com>
It's the same as no flags, so might as well not print it.
(In fact it is slightly harmful, because it actively *clears* the affect
flags, which might have been set in some other manner. But in practice
this cannot happen).
Signed-off-by: Ran Benita <ran234@gmail.com>
This is too strict, and causes symbols/cz to fail parsing. Instead, just
emit a warning (not shown by default):
xkbcommon: WARNING: cz:75:19: unknown escape sequence in string literal
https://bugs.freedesktop.org/show_bug.cgi?id=68056
Reported-By: Gatis Paeglis <gatis.paeglis@digia.com>
Signed-off-by: Ran Benita <ran234@gmail.com>
Some keymaps actually have this, like the quartz.xkb which is tested. We
need to support these.
https://bugs.freedesktop.org/show_bug.cgi?id=67654
Reported-By: Gatis Paeglis <gatis.paeglis@digia.com>
Signed-off-by: Ran Benita <ran234@gmail.com>
We should handle empty xkb_keycode and xkb_symbol sections, since
xkbcomp handles them, and apparently XQuartz uses it. There are also
files for it in xkeyboard-config (rules=base model=empty layout=empty,
which translate to keycodes/empty and symbols/empty).
https://bugs.freedesktop.org/show_bug.cgi?id=67654
Reported-By: Gatis Paeglis <gatis.paeglis@digia.com>
Signed-off-by: Ran Benita <ran234@gmail.com>
Needed for some tests. The tests need some adjustment, mostly because of
the resolution of xkeyboard-config bug
https://bugs.freedesktop.org/show_bug.cgi?id=50935
Also add the 'ch' symbols file for future tests.
Signed-off-by: Ran Benita <ran234@gmail.com>
If the keycode range is smaller than 8 → 255, artifically widen it when
dumping the keymap as not to displease X.
Signed-off-by: Daniel Stone <daniel@fooishbar.org>
xkbcomp doesn't indent there, so it's easier to diff.
Also saves some horizontal space which is sorely needed when looking at
these files (especially the xkb_symbols).
Signed-off-by: Ran Benita <ran234@gmail.com>
Recent xkeyboard-config introduced the following line in symbols/level3:
vmods = LevelThree,
However, the XKM format which xkbcomp produces for the X server can't
handle explicit virtual modifiers such as this:
https://bugs.freedesktop.org/show_bug.cgi?id=4927
So by doing the following, for example:
setxkbmap -layout de (or another 3-level layouts)
xkbcomp $DISPLAY out.xkb
xkbcomp out.xkb $DISPLAY
The modifier is lost and can't be used for switching to Level3 (see the
included test).
We, however, are affected worse by this bug when we load the out.xkb
keymap. First, the FOUR_LEVEL_ALPHABETIC key type has these entries:
map[None] = Level1;
map[Shift] = Level2;
map[Lock] = Level2;
map[LevelThree] = Level3;
[...]
Now, because the LevelThree virtual modifier is not bound to anything,
the effective mask of the "map[LevelThree]" entry is just 0. So when
the modifier state is empty (initial state), this entry is chosen, and
we get Level3, instead of failing to match any entry and getting the
default Level1.
The difference in behavior from the xserver stems from this commit:
acdad6058d
Which removed the entry->active field. Without bugs, this would be
correct; however, it seems in this case we should just follow the
server's behavior.
The server sets the entry->active field like so in XKBMisc.c:
/* entry is active if vmods are bound */
entry->active = (mask != 0);
The xkblib spec explains this field, but does not specify how to
initialize it. This commit does the same as above but more directly.
Signed-off-by: Ran Benita <ran234@gmail.com>
Sync the files again from xkeyboard-config 2.8, since there have been
some changes we should test against.
Also added a script test/data/sync.sh if we want to do it again in the
future.
Signed-off-by: Ran Benita <ran234@gmail.com>
This rule allows you to put several xkb_keymaps in one file.
This doesn't make any sense: only the default/first can ever be used,
yet the others are fully parsed as well.
Different keymaps should just be put in different files.
Signed-off-by: Ran Benita <ran234@gmail.com>
This is a proof-of-concept for the long key names. The keycodes in the
file evdev-xkbcommon are autogenerated from linux/input.h, and uses the
names given there; all of the previous names are aliased to the new
names, so they continue to work with the symbols files, etc.
You can try it with 'sudo ./test/interactive -r evdev-xkbcommon -n 0'
The -n 0 means that we don't offset the evdev scan codes - just feed
them directly. The -r evdev-xkbcommon just means to use a new rules file
which makes us use the new keycodes file. (The only problem I can see is
with the MENU and LSGT names which has some conflicts).
Maybe some day xkeyboard-config could ship something similar, so that
the 8 offset is unneeded.
Signed-off-by: Ran Benita <ran234@gmail.com>
These statements are pretty pointless for us; we don't restrict keycodes
like X does, and if someone writes e.g. maximum = 255 but only has 100
keys, we currently happily alloc all those empty keys. xkbcomp already
handles the case when these statements aren't given, and uses a computed
min/max instead. We should just always use that.
(Of course since keycodes/evdev currently uses almost all of the
keycodes in the range it declares, including 255, this doesn't save any
memory for the common user right now).
Signed-off-by: Ran Benita <ran234@gmail.com>
There are two ways to separate multiple files in XKB include statements:
'+' will cause the later file to override the first in case of conflict,
while '|' will cause it augment it (this is done by xkbcomp). '!' is
unrelated here.
Since '|' is practically never used, this wasn't noticed.
In the modified test, the '|some_compat' previously was just ignored.
Signed-off-by: Ran Benita <ran234@gmail.com>
Avoids a warning, from xkeyboard-config:
commit 6676053f2c93596c2aaa9905151a5c76355a1540
Author: Peter Hutterer <peter.hutterer@who-t.net>
Date: Fri Jun 29 09:53:45 2012 +1000
symbols: keypad can only have one default section
Warning: Multiple default components in keypad
Using x11, ignoring pointerkeys
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Signed-off-by: Ran Benita <ran234@gmail.com>
Group compatibility statements are like the following:
group 3 = AltGr;
This currently results in:
keymap->groups[2].mask = <real mod mapped from AltGr vmod>
And we don't do any thing with this value later. The reason it exists in
XKB is to support non-XKB clients (i.e. XKB support disabled entirely in
the server), which do not know the concept of "group", and use some
modifier to distinguish between the first and second keyboard layouts
(usually with the AltGr key). We don't care about all of that, so we can
forget about it.
One artifact of this removal is that xkb_map_num_groups no longer
works, because it counted through keymap->groups (this wasn't entirely
correct BTW). Instead we add a new num_groups member to the keymap,
which just hold the maximum among the xkb_key's num_groups. This also
means we don't have to compute anything just to get the number of
groups.
Signed-off-by: Ran Benita <ran234@gmail.com>
Pierre Le Marre
Ran Benita
Peter Hutterer
Ran Benita
Daniel Stone