0.9.12 Release
- guid: F2193578-1773-43A9-85CA-79EA8CE48D7B
- author: Mihael Pranjić
- pubDate: Fri, 03 Mar 2023 08:07:28 +0100
- shortDesc: New Backend Logic, Copy-on-Write, FLOSS Course
We are proud to release Elektra 0.9.12.
What is Elektra?
Elektra serves as a universal and secure framework to access configuration settings in a global, hierarchical key database.
For more information, visit https://libelektra.org.
You can also read the news on our website.
You can try out the latest Elektra release using our docker image elektra/elektra.
This is the quickest way to get started with Elektra without compiling and other obstacles, simply run:
docker pull elektra/elektra
docker run -it elektra/elektra
Highlights
- New Backend
- Copy-on-Write
- FLOSS
New Backend
The entire logic for backends has been rewritten, to allow for more flexibility und an unlimited number of plugins.
Instead of calling plugins directly, libelektra-kdb
now only calls so-called backend plugins and special hook plugins.
There is a contract between libelektra-kdb
and the backend plugins.
All backend plugins must adhere to this contract.
To achieve this goal, most backend plugins will call other plugins (like libelektra-kdb
did previously).
The logic previously implemented in libelektra-kdb
was moved to the new default backend plugin backend
.
It works like the old system, but now also allows an unlimited number of plugins in positions where that makes sense.
For example, you can have unlimited postgetstorage
plugins, but only a single getresolver
.
There have also been slight changes to kdbGet
and kdbSet
.
Please read their API docs to find out, if you rely on any behavior that has been altered.
You can also read the new low-level docs to find out all the intricate details.
The structure of system:/elektra/mountpoints
changed as well.
Take a look at the new docs, if you need to know details.
Updating config
The mountpoint configuration format contains breaking changes and a manual upgrade process is needed.
Follow these steps to upgrade the old mountpoint configuration to the new format:
Warning: BACK UP YOUR CONFIG FILES BEFORE UPDATING!
We recommend making a backup of the file printed by kdb file system:/elektra/mountpoints
before updating your installation.
In the unlikely case that the migration script fails, you can still use the information from the backup to manually recreate your mountpoints.
To update your existing system:/elektra/mountpoints
data you can use the migration script.
Note: To run the script you must have Elektra, Python (>= 3.7) and the Python binding installed.
The script uses the Python binding to manipulate Key
s and KeySet
s, but it does not use the kdb
CLI tool, or the KDB
API.
It is safe to run this script before, or after you update your Elektra installation.
By default, the script loads the file /etc/kdb/elektra.ecf
.
If you changed where system:/elektra/mountpoints
is stored, you can provide an alternative path:
./migrate-mountpoints.py /path/to/your/mountpoints/config.file
Note: Because the script does not use the KDB
API it only works, if the mountpoints config file uses the default dump
format.
If your mountpoints config file is not using the dump
format, you may still be able to use the migration script.
However, in that case, you will have to use the script before updating your Elektra installation:
- Run
kdb export system:/elektra/mountpoints dump
to get a copy of your mountpoints config in dump
format
- Write this data to a file and run the migration script on the file.
- To get the data back in your original format you can use
./migrate-mountpoints.py /path/to/file/from/step2 | kdb convert dump your-format > /path/to/converted/file
- Run
kdb file system:/elektra/mountpoints
to find out where your mountpoint config is stored.
Make sure to back up this file, before upgrading your installation.
- Now upgrade your Elektra installation.
- Copy the file
/path/to/converted/file
from step 3 to the location you got in step 4.
The script will read the old mountpoint configuration from the given file.
It will convert the configuration and print the new version to stdout
.
You can inspect the output to make sure, everything is in order.
When you are ready to commit the changes, you can manually edit the config file, or use:
./migrate-mountpoints.py --output=/etc/kdb/elektra.ecf /etc/kdb/elektra.ecf
Individual changes
- Implement hooks. (Maximilian Irlinger @atmaxinger)
- Removed old global plugins code. (Maximilian Irlinger @atmaxinger)
- New backend logic, based on PR #2969 by @vLesk. (@kodebach)
- Add script to migrate
system:/elektra/mountpoints
to new format. (@kodebach)
Copy-on-Write
Thanks to (Maximilian Irlinger @atmaxinger) our Key
and KeySet
datastructures are now fully copy-on-write!
This means noticeably reduced memory usage for cases where keys and keysets are copied and/or duplicated!
We ran some very promising benchmarks, each were performed with 400,000 keys.
All benchmarks were executed using valgrind --tool=massif --time-unit=B --max-snapshots=200 --threshold=0.1
.
Benchmark |
Old Implementation |
Copy-on-Write |
Size Reduction |
Remarks |
createkeys.c |
5.3 MiB |
6.5 MiB |
-22 % |
|
deepdup.c |
10.5 MiB |
8.2 MiB |
22 % |
|
large.c |
18.9 MiB |
15.3 MiB |
19 % |
|
kdb.c |
23.5 MiB |
17.8 MiB |
24 % |
|
kdbget.c |
11.0 MiB |
8.8 MiB |
20 % |
|
kdbmodify.c |
11.0 MiB |
8.8 MiB |
20 % |
Same results as kdbget.c |
First, it should be noted that a single key, without counting payload, is about 50% larger with the copy-on-write implementation.
This explains why the createkeys.c
benchmark yields a negative reduction result.
This benchmark only allocates keys, so not much improvement can be expected there.
Still, as other stuff also uses heap memory, the overall memory consumption only increased by 22%, which is far less than 50%.
All other benchmarks saw meaningful reductions of heap memory used.
One interesting observation is that kdbget.c
and kdbmodify.c
used exactly the same memory.
This can most likely be explained by internal caching within the memory allocator of glibc
.
We also performed runtime tests on the same benchmarks using perf stat --repeat 13
to ensure no major performance regressions occur.
Benchmark |
Old Implementation |
Deviation |
Copy-on-Write |
Deviation |
Runtime Increase |
createkeys.c |
0.209572 s |
0.36 % |
0.21987 s |
0.77 % |
4.9 % |
deepdup.c |
0.23025 s |
0.47 % |
0.231804 s |
0.32 % |
0.6 % |
large.c |
1.14038 s |
0.21 % |
1.14837 s |
0.21 % |
0.7 % |
kdb.c |
1.9270 s |
2.63 % |
1.93354 s |
0.17 % |
0.3 % |
kdbget.c |
0.145663 s |
0.17 % |
0.15763 s |
0.70 % |
8.2 % |
kdbmodify.c |
0.146506 s |
0.19 % |
0.156347 s |
0.15 % |
6.7 % |
Overall, the runtime performance hit is less than 10%.
The more a program does, the less the additional overhead of the copy-on-write algorithms matter.
One interesting detail is that keyCopy
and keyDup
have become quite a bit faster.
This can be seen by comparing the differences between createkeys.c
and deepdup.c
.
The differences are 21 ms for the old implementation and 12 ms for the copy-on-write implementation.
FLOSS
A Free/Libre and Open Source Software (FLOSS) Initiative couldn't survive with many small contributions fixing annoying problems.
This release also contains all contributions done via one term of the FLOSS course.
The success was tremendous, as shown in the rest of the release notes.
A big thanks to the students for their contributions!
Plugins
The following text lists news about the plugins we updated in this release.
yajl
- Fix an issue where trying to set invalid meta-keys won't show an error. (Juri Schreib @Bujuhu)
list
- Removed the outdated
list
plugin. (Maximilian Irlinger @atmaxinger)
(Was only needed for global plugins, which are now replaced by hooks.)
logchange
- Made logchange a notification-send hook plugin. (Maximilian Irlinger @atmaxinger)
toml
- Fix bug, where meta-keys that cannot be inserted don't report an error. (@Bujuhu)
uname
- Add error handling if uname call fails. (Richard Stöckl @Eiskasten)
quickdump
- elektraQuickdumpSet: don't fclose if stdout. (@hannes99)
blockresolver
- Add encoding test for blockresolver read. (@dtdirect)
- Refactor and restructure blockresolver. (@dtdirect)
desktop
- Add a unit test. (Richard Stöckl @Eiskasten)
mini
- Fix a bug where writing meta-keys will fail silently. (Juri Schreib @Bujuhu)
mmapstorage
- Remove code duplication in the data block calculation. (Richard Stöckl @Eiskasten)
network
- Add a retry mechanism. (Richard Stöckl @Eiskasten)
xfconf
- Add xfconf storage plugin with the ability to read and write to xfconf channels. (Richard Stöckl @Eiskasten)
- Make xfconf valgrind suppressions more flexible to lib updates. (Mihael Pranjić @mpranj)
date
- Fix an issue with validationg RFC 822 date-times. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Improve Code Coverage. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
csvstorage
- Fix a bug where writing unkown meta-keys will fail silently. (Juri Schreib @Bujuhu)
Libraries
The text below summarizes updates to the C (and C++)-based libraries of Elektra.
Compatibility
- Global plugins do not work anymore, use hooks instead.
Core
- The Key and KeySet datastructures are now fully copy-on-write. (Maximilian Irlinger @atmaxinger)
keyCopy
now only allocates additional memory if KEY_CP_META
or KEY_CP_ALL
is used. (Maximilian Irlinger @atmaxinger)
- Check for circular links (overrides). (@0x6178656c)
io
- Check file flags for
elektraIoFdSetFlags
: file flags must be exactly one of: read only, write only or read write. (Richard Stöckl @Eiskasten)
Merge
- Add methods
elektraMergeGetConflictingKeys
and elektraMergeIsKeyConflicting
to check which keys were causing a merge conflict. (Maximilian Irlinger @atmaxinger)
Bindings
Bindings allow you to utilize Elektra using various programming languages.
This section keeps you up-to-date with the multi-language support provided by Elektra.
intercept/env
- Remove fallback code. (Markus Raab)
- Command-line functionality is broken due to new-backend differences.
intercept/fs
- Use
KDB_MAX_PATH_LENGTH
for better portability. (Markus Raab)
jna
- Documentation: Improved build instructions. (@Bujuhu)
- Add validation on get for whitelist plugin. (@Bujuhu)
- Upgrade Gradle to 8.0.1. (Mihael Pranjić @mpranj)
rust
- Fix "feature
resolver
is required" error. (Markus Raab)
elixir
- Initial release of the Elixir binding. (@0x6178656c)
- Mark tests as
memleak
. (@0x6178656c)
kdb
- Removed
global-mount
and global-umount
commands. (Maximilian Irlinger @atmaxinger)
- Fixed SIGSEGV when using
kdb find
without argument. (Christian Jonak-Moechel @joni1993)
elektrad
- Removed leftover package-lock.json file. (stefnotch)
Scripts
- Added automatic spelling corrections for changeset. (Maximilian Irlinger @atmaxinger)
- Introduce shebang-conventions. (@0x6178656c)
- Apply auto-fixes from shellcheck. (@0x6178656c)
- Use prettier 2.8.4. (Mihael Pranjić @mpranj)
- Add scripts to enable and disable pre-commit hooks. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Only let http links pass the check if whitelisted. (Richard Stöckl @Eiskasten)
- Link Checker: Add documentation for the usage and how it behaves. (Richard Stöckl @Eiskasten)
- Sed: Add spelling correction for "key-value storage". (@Bujuhu)
- Fix/extends some shell recorder tests. (@Joni1993)
- Use clang-format v15. (Mihael Pranjić @mpranj)
- Fix warning parsing in shell recorder. (@Joni1993)
- Replaced
egrep
by grep -E
. (@0x6178656c and @janldeboer)
- Add audit-dependencies script to check for vulnerabilities for npm depndencies. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
Documentation
- Restructured contrib/api. (Markus Raab).
- Improve page on compilation. (@0x6178656c)
- Improve page for bindings. (@0x6178656c)
- Improve page for getting started. (@stefnotch)
- Remove version number from docker README and replace it with latest. (@Joni1993)
- Fix grammar for
elektra-granularity.md
. (@dtdirect)
- Rephrase sections in
doc/dev/error-\*
. (@dtdirect)
- Improve Git.md. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Unify spelling of man pages. (@stefnotch) (@janldeboer)
- Extend consistency check
check_doc.sh
to work for contrib, dev and tutorials. (@Joni1993)
- Fix internal links. (@0x6178656c)
- Update AUR Link from
elektra
to libelektra
package. (@Bujuhu)
- Update example Ansible playbook in VISION.md. (@Bujuhu)
- Harmonize spelling of Git. (@Joni1993)
- Update packaging instructions for Fedora. (@0x6178656c)
- Improve use of gender. (@0x6178656c)
- Fix some minor mistakes in CONTRIBUTING.md. (@Joni1993)
- Fix various spelling errors. (@Joni1993)
- Denoted package names & global variable names in INSTALL.md as
Code
. (@janldeboer)
- Improve readability of doc/tutorials/highlevel.md. (@deoknats861)
- Improve reference to Podman documentation. (@0x6178656c)
- Unify spelling. (@Joni1993)
- Fix typo in dev/hooks.md. (@dtdirect)
- Remove unused images from doc/images. (@dtdirect)
- Fixed Coverage Badge Link. (@janldeboer)
- Improve CONTRIBUTING doc. (Juri Schreib @Bujuhu) and (Nikola Prvulovic @Dynamichost96)
- Update Doxyfile with Doxygen 1.9.4. (@0x6178656c)
- Add project logo to Doxygen in Doxyfile. (@dtdirect)
- Add mermaid.js to the project using doxygen-mermaid. (@dtdirect)
- Create diagrams in mermaid.js to use in doxygen. (@dtdirect)
- Create README for Doxygen and Mermaid JS. (@dtdirect)
- Tutorial: Add automatic validation to Docker tutorial (Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Add mention of audit-dependencies script in doc/todo/RELEASE.md. (@Bujuhu)
- Move note in GETSTARTED.md. (@Joni1993)
- Use
code
blocks to prevent Markdown from falsy rendering LaTeX. (@stefnotch), (@janldeboer)
- Fix broken links in use cases for KDB after files were renamed. (Florian Lindner @flo91)
- Replace http links with https. (Richard Stöckl @Eiskasten)
- Enhance notifications.md in doc/tutorial. (@dtdirect)
- Add tutorial how to suppress memleaks in plugins from dependencies. (Richard Stöckl @Eiskasten)
- Write about history to make plans of Elektra's adoption more clear. (Markus Raab)
Use Cases
- Improve use cases Template. (@kodebach and Markus Raab)
- Use cases for KDB. (@kodebach)
- Use cases for
libelektra-core
. (@kodebach)
Decisions
- Decide and implement decision process. (Markus Raab)
- Decided future library split. (@kodebach)
- Decided decision process. (Markus Raab)
- Draft for man pages. (Markus Raab)
- Add decision for change tracking. (Maximilian Irlinger @atmaxinger)
- Create decision for allowed and prohibited operation seqences. (Maximilian Irlinger @atmaxinger)
- Add decisions about location of headers and use of
#include
in the repo. (@kodebach)
- Add decision about metadata semantics. (@kodebach)
- Many small fixes to adapt to documentation guidelines and new decision process. (Markus Raab)
- Add decision for read-only keynames. (Maximilian Irlinger @atmaxinger)
- Revive keyname decision. (@kodebach)
- Add decisions for constructor functions and builder functions. (@kodebach)
- Add decision for copy-on-write and provide implementation suggestions. (Maximilian Irlinger @atmaxinger)
- Update internal cache. (Markus Raab) (@kodebach)
- Create transformations. (Maximilian Irlinger @atmaxinger)
- Replace TOC-style README.md with folders and generate HTML for website. (@kodebach)
- Restructured decisions directories based on new agreed-upon steps. (Maximilian Irlinger @atmaxinger)
- Decision for types of
KeySet
s. (@kodebach)
- Added Documentation Guidelines. (Markus Raab)
- Add links and formatting to documents affected by PR#4492 (Document Guidelines) and rephrase some parts. (Florian Lindner @flo91)
- Decisions for changes to
keyIsBelow
and new keyGetNextPart
functions. (@kodebach)
- Apply fix spelling to more files. (Markus Raab)
Tutorials
- Add tutorial for manual installation from the AUR on Arch Linux. (@Bujuhu)
- Add Markdown shell recorder validation to install.webui.md. (@deoknats861)
- Fix the outdated array tutorial. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Reinstate mounting tutorial. (@Bujuhu)
- Make namespaces tutorial verifiable. (@0x6178656c)
- Move Podman-related information to a dedicated page. (@0x6178656c)
Man Pages
- Update man page (patch) as suggested by the CI to fix CI error on master. (Florian Lindner @flo91)
- Added links to the website & webui after further reading. (Philipp Nirnberger @nirnberger)
- Upgrade
ronn-ng
to 0.10.1.pre3. (Mihael Pranjić @mpranj)
Tests
- Fix an Issue where scripts/dev/fix-spelling does not work, if a resolved path contains whitespaces. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Rename scripts/sed to scripts/spelling.sed. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Add
memleak
label to test_getenv
. (@0x6178656c)
- Add test using shellcheck. (@0x6178656c)
- Remove
--rerun-failed
from run_*
scripts. (@kodebach)
- Fix paths for icheck test. (Mihael Pranjić @mpranj)
Shell Recorder
- Add check if file exists. (@0x6178656c)
Packaging
- Add missing new
backend
plugin to components of libelektra package. (Mihael Pranjić @mpranj)
Build
CMake
- Fix warning for CMP0115. (0x6178656c)
- Change Doxygen configuration for LaTeX. (0x6178656c)
- Fix developer warning for package DISCOUNT. (Dennis Toth @dtdirect)
- Pass
--stacktrace
to gradle for the JNA builds. (Maximilian Irlinger @atmaxinger)
- Adapt npm build flags to remove reproducability issues. (Juri Schreib @Bujuhu) (Nikola Prvulovic @Dynamichost96)
- Fix creation of shell recorder tests. (@0x6178656c)
Docker
- Update packagename
libpcrec++-dev
to libpcrecpp0v5
in Debian Sid. (Richard Stöckl @Eiskasten)
- Add shellcheck to Debian containers. (@0x6178656c)
- Use
openjdk-17-jdk
in Debian Sid. (Maximilian Irlinger @atmaxinger)
- Add Fedora 37 images. (Mihael Pranjić @mpranj)
- Update Debian Sid image to use repository Python modules instead of installing with
pip3
due to upstream debian changes. (Mihael Pranjić @mpranj)
- Debian Bullsye: use clang 13. (Mihael Pranjić @mpranj)
- Update Alpine Linux to 3.17.2. (Mihael Pranjić @mpranj)
Gradle
- Use Gradle 7.5.1. (Mihael Pranjić @mpranj)
- Update
java-library.gradle
to use archiveClassifier
(Maximilian Irlinger @atmaxinger)
Infrastructure
Jenkins
- Add Fedora 37 builds, drop Fedora 35 builds. (Mihael Pranjić @mpranj)
- Run more tests also on Master. (Markus Raab)
- Move doc to main build stage. (Markus Raab)
- Disable parallel test runs. (Maximilian Irlinger @atmaxinger)
- Upgrade Jenkins node container to Debian bullseye. (@0x6178656c)
- Undo previous change that added automatic
ctest --rerun-failed
to Jenkins CI. (@kodebach)
Cirrus
- Use Fedora 37. (Mihael Pranjić @mpranj)
- Fix
macos_instance
reference, upgrade to macOS Ventura (by default), use Python 3.11 and Ruby 3.x. (Mihael Pranjić @mpranj)
- Automatically rerun
testmod_dbus*
tests on macOS. (@kodebach)
- Fix dbus not starting on macOS. (Maximilian Irlinger @atmaxinger)
GitHub Actions
- Add auto-cancellation-running action. (Tomislav Makar @tmakar)
- Automatically rerun
testmod_dbus*
tests on macOS. (@kodebach)
- Fix dbus not starting on macOS. (Maximilian Irlinger @atmaxinger)
- Change stale issue/PR checking to GitHub action. (@0x6178656c)
- Update configuration of stale issue/PR action. (@0x6178656c)
- Upgrade actions to recent versions and remove deprecated ruby-setup action. (Mihael Pranjić @mpranj)
Website
The website is generated from the repository, so all information about plugins, bindings and tools are always up-to-date. Furthermore, we changed:
- Fix broken
/pythongen
link on homepage. (@stefnotch)
- Fix redirect logic to not cause loops. (@stefnotch)
- Remove duplicated link to
TESTING.md
file. (@stefnotch), (@janldeboer)
- Restructure parts of the links on the website. (@stefnotch), (@janldeboer)
- Removed broken links to packages for Linux distributions. (@Dynamichost96)
- Update npm packages. (Mihael Pranjić @mpranj)
- Change URLs to say man-page with a dash. (@stefnotch) (@janldeboer)
Outlook
We are currently working on following topics:
- 1.0 API (Klemens Böswirth @kodebach) and (Stefan Hanreich)
- Session recording and better Ansible integration (Maximilian Irlinger @atmaxinger)
- Change tracking (Maximilian Irlinger @atmaxinger)
- Rewriting tools in C (@hannes99)
- Elektrify KDE and GNOME (Mihael Pranjić @mpranj)
- Elektrify XFCE (Richard Stöckl @Eiskasten)
- Mounting SQL databases (Florian Lindner @flo91)
- Recording Configuration (Maximilian Irlinger)
- Ansible-Elektra (Lukas Hartl) and (Maximilian Irlinger)
- Configure Olimex Base Images (Maximilian Irlinger)
- Improving Build Server Infrastructure (Lukas Hartl) and (Maximilian Irlinger)
- Improve Java Development Experience (Michael Tucek)
Statistics
We closed about 150 issues for this release.
About 28 authors changed 960 files with 29400 insertions(+) and 20927 deletions(-) in 1421 commits.
Thanks to all authors for making this release possible!
Join the Initiative!
We welcome new contributors!
Read here about how to get started.
As first step, you could give us feedback about these release notes.
Contact us via our issue tracker.
Get the Release!
You can download the release from
The hashsums are:
- name: elektra-0.9.12.tar.gz
- size: 9297913
- md5sum: a6de9401709283b69ec211681f2a7757
- sha1: cb4e282d1346fda771de7510663652555f8e6c7d
- sha256: 38238ba4a5318f999dc3045da06467abf529344dc46ad3fdf42bdca0155e149c
The release tarball is also available signed using GnuPG from
The following GPG Key was used to sign this release: 12CC44541E1B8AD9B66AFAD55262E7353324914A
Already built API documentation can be found
Stay tuned!
Subscribe to the RSS feed to always get the release notifications.
If you also want to participate, or for any questions and comments, please contact us via our issue tracker on GitHub.
Permalink to this NEWS entry
For more information, see https://libelektra.org.
Best regards,
Elektra Initiative