In commit 1638409b22, the number of
compose nodes was limited to 65535 to enable "future optimizations",
which apparently means slightly reduced memory usage due to fitting in
a uint16_t. At this time, it was mentioned that the author was not
aware of "any compose files which come close".
However, I'm one of the users that actually do require a larger number
of nodes for their compose file. Thus, use a uint32_t again and raise
the limit significantly.
There is no feedback that they are both replaced with default values.
Fix it by adding a warning informing about missing layout and show the
defaults for both.
As the documentation for xkb_keymap_new_from_buffer() states, the "input string
does not have to be zero-terminated". The actual implementation however failed
with "unrecognized token/syntax error" when it encountered a null byte.
Fix this by allowing a null byte at the last position of the buffer. Anything
else is likely a client error anyway.
Fixes#307
strdup() is the least likely call to fail here, let's move it to the bottom so
we don't need to worry about the allocated string.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Unlike current xkbcommon, X11’s xkbcomp allows to remove entries in
the modifiers’ map using “modifier_map None { … }”.
“None” is translated to the special value “XkbNoModifier” defined in
“X11/extensions/XKB.h”. Then it relies on the fact that in "CopyModMapDef",
the following code:
1U << entry->modifier
ends up being zero when “entry->modifier” is “XkbNoModifier” (i.e. 0xFF).
Indeed, it relies on the overflow behaviour of the left shift, which in
practice resolves to use only the 5 low bits of the shift amount, i.e.
0x1F here. Then the result of “1U << 0xFF” is cast to “char”, i.e. 0.
This is a good trick but too magical, so in libxkbcommon we will use
an explicit test against our new constant XKB_MOD_NONE.
`_MSC_VER` is specific to MSVC, but there can be other compilers targeting
windows. Hopefully they do define `_WIN32`, so let's use that.
Refs: https://github.com/xkbcommon/libxkbcommon/issues/305
Signed-off-by: Ran Benita <ran@unusedvar.com>
This entry maps a non existing special keysym 0x20a9.
The correct mapping for XKB_KEY_Korean_Won (0x0eff) already exists.
Signed-off-by: Ran Benita <ran@unusedvar.com>
Not sure what it's doing here, but converting "€" to a keysym
doesn't work with this entry. 0x13a4 doesn't appear in
xkbcommon-keysyms.h. 0x20ac is the keysym documented in the
header (and it's the last entry in the table).
It's been in the table since it was introduced in e0524296d2
("Add API for getting unicode representation of a keysym").
Co-authored-by: Simon Ser <contact@emersion.fr>
lokid and hikid actually stores the sibling to current node, which
should not be cleared when override. This would break the sequence with
a common prefix when override another.
Fix#286
Signed-off-by: Weng Xuetian <wengxt@gmail.com>
From the documentation:
> It does not clean up parser state, it cleans up memory allocated by the library
> itself. It is a cleanup function for the XML library. It tries to reclaim all
> related global memory allocated for the library processing. [...]
> One should call xmlCleanupParser() only when the process has finished using the library.
http://xmlsoft.org/html/libxml-parser.html#xmlCleanupParser
Since we're a library ourselves we cannot know if something else in the same
proces uses the parser, so we must not call this.
Reported-by: M Hickford
In 1b3a1c277a we changed the error
handling in this code to not bail out immediately but only after
everything has been processed, to simplify the code. But I suspect the
code isn't prepared for this and that's what causing the crash reported
in the issue.
Bring back the short-circuit error handling which would hopefully fix
such crashes.
Fixes: https://github.com/xkbcommon/libxkbcommon/issues/252
Signed-off-by: Ran Benita <ran@unusedvar.com>
While the previous 1987-style[0] scheme was fun (and I reasonably
optimized it for a fair comparison), this task is more suited to a hash
table. Even a simple implementation beats the old one.
[0] Seems to have first appeared in X11R1, released September 1987.
See server/dix/atom.c here: https://www.x.org/releases/X11R1/X.V11R1.tar.gz
Signed-off-by: Ran Benita <ran@unusedvar.com>
This is hit when passing an empty string and XKB_KEYSYM_CASE_INSENSITIVE
to xkb_keysym_from_name currently if `(lo + hi) / 2` is 0 and `cmp < 0`,
causing mid to underflow and the the array access into name_to_keysym on
the next iteration of the loop to be out of bounds .
We *would* use ssize_t here as it is the appropriate type, but windows
unfortunately does not define it.
The numpad:mac option doesn't specify a name for the first level:
// On Mac keypads, level 1 and 2 are swapped.
partial xkb_types "mac" {
type "KEYPAD" {
modifiers = None;
map[None] = Level2;
level_name[Level2] = "Number";
};
include "extra(keypad)"
};
This means the atom for level name is XCB_ATOM_NONE. We tried to get its
name, which fails. This regressed in 40c00b4721.
Instead, translate it to XKB_ATOM_NONE, same as the previous behavior.
Fixes: https://github.com/xkbcommon/libxkbcommon/issues/229
Signed-off-by: Ran Benita <ran@unusedvar.com>
Make it use a bit operation instead of an expensive modulo.
perf diff:
Baseline Delta Abs Shared Object Symbol
........ ......... ................. ...................................
28.15% -6.57% bench-compose [.] xkb_keysym_from_name
Signed-off-by: Ran Benita <ran@unusedvar.com>
In 7d84809fdc I added a fast path for the
case-sensitive case, but it is still slowing down Compose parsing.
Instead of the binary search, use a perfect hash function, computed with
a simple python module I found (vendored).
It is faster -- perf diff is:
Baseline Delta Abs Shared Object Symbol
........ ......... ................. ...................................
22.35% -14.04% libc-2.33.so [.] __strcmp_avx2
16.75% +10.28% bench-compose [.] xkb_keysym_from_name
20.72% +2.40% bench-compose [.] parse.constprop.0
2.29% -1.97% bench-compose [.] strcmp@plt
2.56% +1.81% bench-compose [.] resolve_name
2.37% +0.92% libc-2.33.so [.] __GI_____strtoull_l_internal
26.19% -0.63% bench-compose [.] lex
1.45% +0.56% libc-2.33.so [.] __memchr_avx2
1.13% -0.31% libc-2.33.so [.] __strcpy_avx2
Also reduces the binary size:
Before:
text data bss dec hex filename
341111 5064 8 346183 54847 build/libxkbcommon.so.0.0.0
After:
text data bss dec hex filename
330215 5064 8 335287 51db7 build/libxkbcommon.so.0.0.0
Note however that it's still larger than before 7d84809fdccbb5898d08388:
text data bss dec hex filename
320617 5168 8 325793 4f8a1 build/libxkbcommon.so.0.0.0
Signed-off-by: Ran Benita <ran@unusedvar.com>
Previously we used a simple trie with a linked list for each chain.
Unfortunately most compose files have very long chains which means the
constructions performs an almost quadratic number of comparisons.
Switch to using a ternary search tree instead. This is very similar to a
trie, only the linked list is essentially replaced with a binary tree.
On the en_US/Compose file, the perf diff is the following (the modified
function is `parse`):
Event 'cycles:u'
Baseline Delta Abs Shared Object Symbol
........ ......... ................ .................................
39.91% -17.62% bench-compose [.] parse.constprop.0
20.54% +6.47% bench-compose [.] lex
17.28% +5.55% libc-2.33.so [.] __strcmp_avx2
12.78% +4.01% bench-compose [.] xkb_keysym_from_name
2.30% +0.83% libc-2.33.so [.] __GI_____strtoull_l_internal
3.36% +0.78% bench-compose [.] strcmp@plt
Thanks to some careful packing, the memory usage is pretty much the
same.
Signed-off-by: Ran Benita <ran@unusedvar.com>
Relatedly, strtoul allows a lot of unwanted stuff (spaces, +/- sign,
thousand seperators), we really ought not use it. But that's for another
time.
Signed-off-by: Ran Benita <ran@unusedvar.com>
Fits in uint16_t, which enables some future optimizations. But also a
good idea to have some limit. Not aware of any compose files which come
close.
Signed-off-by: Ran Benita <ran@unusedvar.com>
../../../src/xkbcomp/compat.c:693:16: warning: Although the value stored to
'merge' is used in the enclosing expression, the value is never actually read
from 'merge' [deadcode.DeadStores]
si.merge = merge = (def->merge == MERGE_DEFAULT ? merge : def->merge);
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The target buffer is 7 bytes long, null-termination is optional (as the comment
already suggests). Coverity is unhappy about this though so let's use memset and
memcpy instead.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
xkb_keysym_from_name() is called a lot in Compose file parsing. The
lower case handling slows things down a lot (particularly given we can't
use the optimized strcasecmp() due to locale issues). So add separate
handling for the non-case-sensitive case which is used by Compose.
To do this we need to add another version of the ks_tables table. This
adds ~20kb to the shared library binary. We can probably do something
better here but I think it's fine.
Signed-off-by: Ran Benita <ran@unusedvar.com>
src/x11/keymap.c:980:26: warning: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
980 | for (size_t i = 0; i < length; i++) {
| ^
Signed-off-by: Ran Benita <ran@unusedvar.com>
libxkbcommon-1.0.3/src/xkbcomp/ast-build.c:526: leaked_storage: Variable "file"
going out of scope leaks the storage it points to.
Where we exit the loop early, we don't release the various allocated memory.
Make this patch more obvious my moving the declaration for those into the loop
as well, this way we know that they aren't used outside the loop anywhere.
Found by coverity
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
If the name is missing in a configItem, we'd fail and leak the memory for
description, brief and vendor.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Both get_atom_name() and the new atom interner required a round trip. Move
get_atom_name() into the atom interner to save one more round trip. This brings
xkb_x11_keymap_new_from_device() down to two round trips, which is the minimum
possible number.
(Also, I think the new code in keymap.c is more readable than the mess I
previously created)
With this last commit in the series, this definitely:
Fixes: https://github.com/xkbcommon/libxkbcommon/pull/217
Signed-off-by: Uli Schlachter <psychon@znc.in>
There are a number of XKB requests needed to request all the information from
the X11 server. So far, the code was sending one request and waiting for the
reply. This commit starts batching the request so that we get multiple replies
with one round trip.
This removes three round trips.
Only the simple requests are converted. get_map() and get_names() use some
bitmasks that are needed for both the request and the reply. These will be dealt
with separately.
Signed-off-by: Uli Schlachter <psychon@znc.in>
Instead of asking for an atom name and waiting for the reply four times, this
now sends four GetAtomName requests and waits for all the replies at once. Thus,
this saves three round trips.
Signed-off-by: Uli Schlachter <psychon@znc.in>
On my system, calling xkb_x11_keymap_new_from_device() did 78 round trips to the
X11 server, which seems excessive. This commit brings this number down to about
9 to 10 round trips.
The existing functions adopt_atom() and adopt_atoms() guarantee that the atom
was adopted by the time they return. Thus, each call to these functions must do
a round-trip. However, none of the callers need this guarantee.
This commit makes "atom adopting" asynchronous: Only some time later is the atom
actually adopted. Until then, it is in some pending "limbo" state.
This actually fixes a TODO in the comments.
Fixes: https://github.com/xkbcommon/libxkbcommon/issues/216
Signed-off-by: Uli Schlachter <psychon@znc.in>
On every keymap notify event, the keymap should be refreshed, which
fetches the required X11 atoms. A big keymap might have a few hundred of
atoms.
A profile by a user has shown this *might* be slow when some intensive
amount of keymap activity is occurring. It might also be slow on a
remote X server.
While I'm not really sure this is the actual bottleneck, caching the
atoms is easy enough and only needs a couple kb of memory, so do that.
On the added bench-x11:
Before: retrieved 2500 keymaps from X in 11.233237s
After : retrieved 2500 keymaps from X in 1.592339s
Signed-off-by: Ran Benita <ran@unusedvar.com>
Where resolve_keysym fails we warn but use the otherwise uninitialized variable
as our keysym. That later ends up in the keymap as random garbage hex value.
Simplest test case, set this in the 'us' keymap:
key <TLDE> { [ xyz ] };
And without this patch we get random garbage:
./build/xkbcli-compile-keymap --layout us | grep TLDE:
key <TLDE> { [ 0x018a5cf0 ] };
With this patch, we now get NoSymbol:
./build/xkbcli-compile-keymap --layout us | grep TLDE:
key <TLDE> { [ NoSymbol ] };
MacOS doesn't have eaccess/euidaccess but it does have unistd.h, so let's
include it to silence the R_OK redefinition compiler warnings.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
../src/context.c:57:9: warning: variable 'err' is used uninitialized whenever
'if' condition is true [-Wsometimes-uninitialized]
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This completes the usual triplet of configuration locations available for most
processes:
- vendor-provided data files in /usr/share/X11/xkb
- system-specific data files in /etc/xkb
- user-specific data files in $XDG_CONFIG_HOME/xkb
The default lookup order user, system, vendor, just like everything else that
uses these conventions.
For include directives in rules files, the '%E' resolves to that path.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Previously, a 'symbols/us' file in path A would shadow the same file in path B.
This is suboptimal, we rarely need to hide the system files - we care mostly
about *extending* them. By continuing to check other lookup paths, we make it
possible for a XDG_CONFIG_HOME/xkb/symbols/us file to have sections including
those from /usr/share/X11/xkb/symbols/us.
Note that this is not possible for rules files which need to be manually
controlled to get the right bits resolved.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Streamline the code a bit - instead of handling all the if (!file) conditions
handle the case of where we have a file and jump to the end.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Before reading ~/.XCompose, try to read $XDG_CONFIG_HOME/XCompose
(falling back to ~/.config/XCompose).
This helps unclutter the home directory of users who want that.
Don't do the realloc dance, just asprintf to the buffer and move on. The check
is likely pointless anyway, if we run out of asprintf size, log_error will
probably blow up as well.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
We only ever care about whether we error out or not, so let's wrap this into
something more sane.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Now that we're relying on various different include paths, let's log that ones
we use and the ones we failed to use.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This library is the replacement for clients parsing evdev.xml directly.
Instead, they should use the API here so that in the future we may even
be able to swap evdev.xml for a more suitable data format.
The library parses through evdev.xml (using libxml2) and - if requested -
through evdev.extras.xml as well. The merge approach is optimised for
the default case where we have a system-installed rules XML and another file in
$XDG_CONFIG_DIR that adds a few entries. We load the system file first, then
append any custom ones to that. It's not possible to overwrite the MLVO list
provided by the system files - if you want to do that, get the change upstream.
XML validation is handled through the DTD itself which means we only need to
check for a nonempty name, everything else the DTD validation should complain
about.
The logging system is effectively identical to xkbcommon.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Returns true on success or false on error _or_ truncation. Since truncation is
almost always an error anyway, we might as well make this easier to check.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
If the active set of modifiers doesn't match any explicit entry of the
key type, the resulting level is 0 (i.e. Level 1). Some key types don't
explicitly map Level 1, taking advantage of this fallback.
Previously, xkb_keymap_key_get_mods_for_level didn't consider this, and
only reported masks for explicit mappings. But this causes some glaring
omissions, like matching "a" in the "us" keymap returning not results.
Since every mask which isn't explicitly mapped falls back to 0, we can't
return the all. Almost always the best choice for this is the empty
mask, so return that, when applicable.
Fixes https://github.com/xkbcommon/libxkbcommon/issues/140.
Reported-by: https://github.com/AliKet
Signed-off-by: Ran Benita <ran@unusedvar.com>
The new API is useful to implement features like auto-type and
desktop automation. Since the inputs for these features is usually
specified in terms of the symbols that need to be typed, the
implementation needs to be able to invert the keycode->keysym
transformation and produce a sequence of keycodes that can be used
to type the requested character(s).
In order to support features like auto-type and UI automation, the
relevant tools need to be able to invert the keycode->keysym->text
transformation. In order to facilitate that, a new API was added.
It allows querying the keysyms that correspond to particular Unicode
codepoints. For all practical purposes, it can be thought of as an
inverse of xkb_keysym_to_utf32().
Only the input/output functions from <unistd.h> options are used, so
using <io.h> when building with MSVC should be enough. The inclusion
of the header in context-priv.c does not seem to be needed (tested
on GNU/Linux) and so it is removed.
Signed-off-by: Ran Benita <ran@unusedvar.com>
Detected by MSVC:
xkbcomp\xkbcomp.c(111): warning C4047: 'return': 'bool' differs in levels of indirection from 'void *'
Signed-off-by: Ran Benita <ran@unusedvar.com>
Don't use int which can have different size on different machines.
Also avoid some warnings from MSVC:
xkbcomp/parser.y(760): warning C4244: '=': conversion from 'int64_t' to 'int', possible loss of data
xkbcomp/parser.y(761): warning C4244: '=': conversion from 'int64_t' to 'int', possible loss of data
xkbcomp/parser.y(767): warning C4244: '=': conversion from 'int64_t' to 'int', possible loss of data
Signed-off-by: Ran Benita <ran@unusedvar.com>
This increases the size of the struct a bit but it's not very important.
Fixes these MSVC warnings:
src\scanner-utils.h(112): warning C4267: '+=': conversion from 'size_t' to 'unsigned int', possible loss of data
src\scanner-utils.h(147): warning C4267: '+=': conversion from 'size_t' to 'unsigned int', possible loss of data
Signed-off-by: Ran Benita <ran@unusedvar.com>
Previously we included it with an `-include` compiler directive. But
that's not portable. And it's better to be explicit anyway.
Every .c file should have `include "config.h"` first thing.
Signed-off-by: Ran Benita <ran@unusedvar.com>
Some environments (e.g. Windows + MSVC) do not provide asprintf() or
vasprintf(). This tries to detect their presence, and provides suitable
fallback implementations when not available.
alois31
Pierre Le Marre
Ran Benita
Peter Hutterer
Ronan Pigott
Sam Lantinga
Weng Xuetian
M Hickford
Isaac Freund
Uli Schlachter
hhb
Emmanuel Gil Peyrot
Jaroslaw Kubik
Adrian Perez de Castro