Elektra: 0.8.20 Release

Elektra 0.8.24

0.8.20 Release

- guid: 547d48e2-c044-4a8e-9d32-ca6b6fb914d9
- author: Markus Raab
- pubDate: Thu, 31 Oct 2017 23:08:07 +0200
- shortDesc: New Website, puppet-libelektra, New Plugins

## What is Elektra?

Elektra serves as a universal and secure framework to access configuration settings in a global, hierarchical key database. For more information, visit https://libelektra.org.

## Highlights

This is by far the largest release in Elektra's history. In 2813 commits, 19 authors changed 1714 files with 92462 insertions(+) and 21532 deletions(-). The highlights are:

- libelektra.org: new website and puppet-libelektra
- plugin+bindings for Haskell and Ruby
- improved shell completion
- new plugins: yamlcpp, camel, mini, date, file, range, multifile, xerces, ipaddr

### libelektra.org

Unfortunately this release was delayed. The reason for the delay is that our community server (build server, web site,...) was compromised and we needed to reinstall everything from scratch.

We took advantage of the situation, and reinstalled everything properly managed by puppet-libelektra. With puppet-libelektra, you can directly set keys, specifications (validations), and even mount new configuration files from within Puppet.

Our community server is now completely managed by libelektra.

Thanks to Bernhard Denner, for rescuing us from the difficult situation, especially for the sprint shortly before the release.

As already already announced in https://master.libelektra.org/doc/news/2016-12-17_website_release.md "December 2016" we completely reimplemented our website. Now all our websites are available via https. This release is the first one that includes the source code of the website and its snippet sharing functionality.

The backend for this snippet sharing website uses Elektra itself: both for its configuration and for the configuration snippets.

Thanks again to Marvin Mall for the awesome website.

#### Haskell and Ruby

The Ruby binding, created by Bernhard Denner, was greatly improved and now includes libtools bindings. It is the first binding that goes beyond Elektra's main API. Bernhard Denner also added many examples that demonstrate how you can take advantage of the Ruby bindings.

Armin Wurzinger created a new binding for the functional language Haskell. He also added support for Haskell plugins. Due to generic CMake and C Code, plugins can be written exclusively in Haskell, without any glue code. Several Haskell examples already exist. The Haskell support is currently experimental.

### Shell Completion

René Schwaiger added completion support for Fish in this release. We also extended our support for other shells: The new tool `kdb complete` suggests how to complete an Elektra path. It considers mountpoints and also takes bookmarks into account. Thanks to Armin Wurzinger for creating this useful utility. Our Zsh and fish completions already take advantage of `kdb complete`. Thanks to Sebastian Bachmann for taking the time to update the `zsh` completions.

### New Plugins

See plugin overview to get an overview of the ever-growing number of plugins.

The yamlcpp plugin and camel plugin add first support for YAML.

The mini plugin is yet another minimal INI plugin.

Thanks to René Schwaiger.

The date plugin supports validation of dates according to three standards:

- `RFC2822`
- `ISO8601`
- `POSIX`

The multifile plugin allows us to integrate many configuration files via globbing with a single mount command. It supports `.d` configuration directories as often used today.

The file plugin interprets the content of a file as configuration value.

The ipaddr plugin adds support for IP address validation on systems that do not support `getaddrinfo`.

Thanks to Thomas Waser for creating these useful plugins.

The xerces plugin supplants the xmltool plugin and allows us to use XML files not following a specific schemata. Attributes are mapped to Elektra's metadata, multiple keys with the same names are mapped to arrays.

Thanks to Armin Wurzinger.

## Documentation

The documentation was greatly improved within this release.

- Added "Hello, Elektra" and logging tutorial, thanks to René Schwaiger
- extended https://master.libelektra.org/doc/help/elektra-faq.md "FAQ"
- Christoph Weber () improved some tutorials
- options are passed to PDFLaTeX compiler, thanks to René Schwaiger
- small fixes, thanks to Dominik Hofer
- fix many spelling mistakes, use American english, fix formatting, fix+add links, unify title style, fix code blocks, add titles and fix the PDF manual a big thanks to René Schwaiger

## Features

We added even more functionality, which could not make it to the highlights:

- DBUS support for qt-gui (listening to configuration changes): qt-gui gets a viewer-mode where configuration settings are immediately updated via DBus notifications, thanks to Raffael Pancheri With the new qt-gui and newer qt releases (~5.7) the qtquick experience is much smoother, for example, the tree view does not collapse on syncs anymore.
- Armin Wurzinger greatly improved the JNA binding. The build system now uses Maven to build them. Armin also added Doxygen documentation and a script to test the JNA binding using Randoop.
- The improved curlget plugin, is now able to upload configuration files, thanks to Thomas Waser and Peter Nirschl (CMake fixes).
- New command `kdb rmmeta`, thanks to Bernhard Denner
- crypto plugin and fcrypt plugin
  - The configuration option `gpg/key` was renamed to `encrypt/key`
  - The plugins now make sure that you configured them properly by validating key IDs
  - thanks to Peter Nirschl
- fcrypt plugin:
  - The plugin now list available GPG keys when config is missing
  - You can now specify signatures via the configuration option `sign/key`
  - New text mode, enabled by default (disable by setting `fcrypt/textmode` to `0`)
  - New option `fcrypt/tmpdir` allows you to specify the output directory of `gpg`
  - If you want to learn how to use the plugin please check out our new ASCII cast
  - thanks to Peter Nirschl
- Thomas Waser added useful scripts:
  - https://master.libelektra.org/doc/help/kdb-mount-list-all-files.md "mount-list-all-files" to list all mounted files.
  - https://master.libelektra.org/doc/help/kdb-mountpoint-info.md "mountpoint-info" to provide more info about mountpoints.
  - https://master.libelektra.org/doc/help/kdb-stash.md "stash" to stash away Elektra's configuration, to be restored using `restore`.
  - https://master.libelektra.org/doc/help/kdb-backup.md "backup" to backup Elektra's configuration.
  - https://master.libelektra.org/doc/help/kdb-restore.md "restore" to restore a backup or stash.
  - https://master.libelektra.org/doc/help/kdb-check-env-dep.md "check-env-dep" allows users to check if environment has influence on configuration settings.
  - https://master.libelektra.org/doc/help/kdb-change-resolver-symlink.md "change-resolver-symlink" allows users to change the default resolver.
  - https://master.libelektra.org/doc/help/kdb-change-storage-symlink.md "change-storage-symlink" allows users to change the default storage.
- limit min/max depth for `kdb ls` (`-mM`), thanks to Armin Wurzinger.
- conditionals: allow multiple assigns and conditions
- base64 also works as filter for binary data (not just encrypted data), thanks to René Schwaiger
- csvstorage plugin: compatibility with RFC 4180, thanks to Thomas Waser
- gitresolver plugin: improvements and update of libgit version, thanks to Thomas Waser
- curlget plugin: also allow uploading of configuration files, thanks to Thomas Waser

## Compatibility

As always, the ABI and API of kdb.h is fully compatible, i.e. programs compiled against an older 0.8 version of Elektra will continue to work (ABI) and you will be able to recompile programs without errors (API).

We added `explicit` to some C++ constructors in libtools and internally moved some typedefs. ModulesPluginDatabase now has protected members (instead of private). This might break code in special cases, but should not affect binary compatibility. As always we tested for binary compatibility. This time we had to revert some changes to keep libelektra-tools ABI compatible.

Furthermore:

- scripts now work on macOS (readlink and sed), thanks to Armin Wurzinger and René Schwaiger
- generalized error #60, "invalid line encountered"
- added new errors #164 - #187
- added private headerfiles `kdbglobal.h`, `kdbinvoke.h`

## Notes for Package Maintainer

These notes are of interest for people maintaining packages of Elektra:

- LICENSE.md now contains only the BSD 3-Clause License (without any additional non-license text)
- AUTHORS renamed to AUTHORS.md
- NEWS.md is now a generated file containing all news concatenated
- CMake 2.8.8 is no longer supported, CMake 3.0 is now needed
- fix macOS `RPATH`, remove old policies, thanks to René Schwaiger
- new option `BUILD_DOCSET` to build DocSet, thanks to René Schwaiger
- new option `ENABLE_OPTIMIZATIONS` for `OPMPHM` preparation work, thanks to Kurt Micheli For this release, please keep `ENABLE_OPTIMIZATIONS` turned off. Currently the flag increases memory usage, without being faster.
- add `TARGET_TOOL_DATA_FOLDER` for installation of tool data (for rest-backend and rest-frontend)

The following files are new:

- Libs: `libelektra-utility.so` ,`libelektra4j-0.8.20.pom.xml`, `libelektra-invoke` (needed by plugins: curlget, gitresolver, dini, blockresolver, multifile)
- Plugins: `libelektra-camel.so`, `libelektra-date.so`, `libelektra-file.so`, `libelektra-ipaddr.so`, `libelektra-mini.so`, `libelektra-multifile.so`, `libelektra-range.so`, `libelektra-xerces.so`, `libelektra-yamlcpp.so`
- Tools: `backup`, `mount-list-all-files`, `mountpoint-info`, `restore`, `stash`, `update-snippet-repository`
- Tests: `change-resolver-symlink`, `change-storage-symlink`, `check-env-dep`, `check_bashisms`, `check_doc`, `check_meta`, `testmod_camel`, `testmod_crypto_openssl`, `testmod_date`, `testmod_file`, `testmod_ipaddr`, `testmod_jni`, `testmod_mini`, `testmod_range`, `testmod_simpleini`, `testmod_xerces`, `testmod_yamlcpp`, `testtool_plugindatabase`, `test_utility`

The following files were removed: `testmod_curlget`, `testmod_dpkg`, `testmod_profile`, `testmod_shell`, `testmod_spec`, `test_opmphm_vheap`, `test_opmphm_vstack`

The following files were renamed: `libelektra-1.jar` → `libelektra4j-0.8.19.jar`

In the Debian branch of the git repo, we now build upon the work of Pino Toscano. The branch allows building Debian packages of the release for Debian Stretch and Jessie.

Thanks to Pino Toscano for the high-quality packages.

## Notes for Elektra's Developers

These notes are of interest for people developing Elektra:

- Added macros, thanks to René Schwaiger:
  - `ELEKTRA_NOT_NULL` is an assertion against null pointers
  - `ELEKTRA_MALLOC_ERROR` sets an error when allocation failed
  - `ELEKTRA_STRINGIFY` to quote a macro value
  - `ELEKTRA_PLUGIN_STATUS_ERROR`, `ELEKTRA_PLUGIN_STATUS_SUCCESS`, `ELEKTRA_PLUGIN_STATUS_NO_UPDATE` for return values of plugins.
- `ELEKTRA_STRINGIFY` used throughout, thanks to René Schwaiger
- use `(void)` instead of `()` (added `-Wstrict-prototypes`)
- new positions for global plugins, thanks to Mihael Pranjic
- Kurt Micheli added `generateKeySet` to randomly generate large key sets
- add vagrant and docker support, thanks to Christoph Weber ()
- improve support for CLion, NetBeans and `oclint`
- portability improvements for logger, thanks to René Schwaiger
- metadata consistency check within source repo, thanks to Thomas Waser
- `ELEKTRA_PLUGIN_EXPORT` accepts macro as argument
- fallthroughs in switch statements are now marked with `FALLTHROUGH`
- introduce `print_result` to unify test output, thanks to René Schwaiger
- export `validateKey` as preparation for type plugin

## Other

Various other changes happened in the code repository:

- kdb: errors are more colorful, add infos to report issues, catch signals for `kdb` tools to print errors on crashes, use `$EDITOR` if `sensible-editor` and `editor` is not found. René Schwaiger fixed preposition and format of the messages.
- added Spanish translation for qt-gui thanks to AdanGQ ()
- augeas plugin: error messages improved, export genconf (for WebUI to list all lenses)
- improvements for CentOS and Debian Packages, thanks to Sebastian Bachmann
- Travis improvements, fixes, and many build variants added, thanks to Mihael Pranjic and René Schwaiger
- `ronn` now always uses UTF-8 as encoded and is no longer required as essential dependency to get man pages, thanks to René Schwaiger
- GitHub now recognizes that we have a BSD licence, thanks to René Schwaiger
- uninstallation Script also uninstalls directories and Python files, thanks to René Schwaiger
- Kurt Micheli created a benchmark tool to generate large KeySets
- added/reformatted use cases, thanks to Daniel Bugl
- Thomas Wahringer prepared for a new theme on the website
- Arfon Smith updated meta data for Elektra's journal entry

## Quality

In this release we had a focus on quality improvements:

- fixed all remaining ASAN problems, thanks to René Schwaiger and Armin Wurzinger (some tests are excluded when compiled with ASAN)
- fix many compilation warnings, thanks to René Schwaiger and Armin Wurzinger
- fixed many potential out-of-bound errors, thanks to René Schwaiger
- fixed warnings of Clang's static analyzers, thanks to René Schwaiger
- fixed cppcheck warnings, thanks to Armin Wurzinger
- fixed strict prototypes, thanks to Armin Wurzinger
- fixed and use scan-build (clang)
- fixed potential memory leaks on errors
- added assertions
- generate Java API tests with randoop which revealed bugs in jna bindings that were fixed, thanks to Armin Wurzinger
- Numerous fixes in the shell recorder, which does regression tests on Elektra's tutorials and READMEs, thanks to René Schwaiger and Thomas Waser

## Fixes

Many problems were resolved with the following fixes:

- `kdb file`: never print errors, thanks to René Schwaiger
- plugin mathcheck: fixed regex #1094, thanks to Thomas Waser
- dbus: properly do unref and document how to integrate DBus, thanks to Kai-Uwe Behrmann
- dbus accepts announce=once which is used for `kdb mount-openicc` It protects against message floods in large configuration files, thanks to Kai-Uwe Behrmann for reporting
- plugin desktop: fix crash if `DESKTOP_SESSION` is missing
- shell-recorder: many fixes and improvements, thanks to Thomas Waser and René Schwaiger
- fix getopt positional parameters, thanks to Armin Wurzinger
- resolver: avoid silent errors of fchown/fchmod
- plugin fcrypt: fixes in file name handling to make leaks less likely (still needs tmpfs to be secure!), thanks to Peter Nirschl
- plugin jni: fix segfaults on errors, plugin is nevertheless tagged as experimental due to other problems
- plugin type: reject integers if garbage follows
- `kdb`: fix memleak when listing plugins
- many spelling fixes and fix typo in source of qt-gui: thanks to klemens (ka7)
- dpkg, fix file leakage, thanks to Armin Wurzinger
- plugin line: only skip parentKey if present
- plugin resolver: avoid failure after commit for files that cannot be removed
- plugin simpleini: handle more errors, make format parameter more robust thanks to Bernhard Denner
- plugin crypto: fix compilation errors for openssl versions on Debian 9, thanks to Peter Nirschl
- `kdb mv`: fail without keys also in recurse mode
- fix bashism, thanks to Armin Wurzinger
- qtgui: fix crash on unhandled exception on binary values, thanks to Raffael Pancheri

## Outlook

We are currently working on following topics:

- Migration of LCDproc, OpenIcc, machinekit, ... to Elektra.
- A reimplementation of Elektra's core API in Rust (next to implementation in C).
- A user interface which generates restricted input fields based on the specification.
- YAML as configuration file format (next to INI, XML, JSON, TCL, ...).
- An mmap persistent cache.
- Improvements for the specification language.
- New APIs to be directly used by applications.
- An order-preserving minimal hash for O(1) lookup and iteration.
- Mainloop migration for notifications (currently only DBus, to be extended to Redis, ZeroMq).
- Improvements on the Website and snippet sharing to also handle misconfiguration.

## Get It!

You can download the release from here or GitHub

The hashsums are:

- name: elektra-0.8.20.tar.gz
- size: 4740032
- md5sum: 0e906f1a1677a8bfb31d144e1eaeb3cf
- sha1: 5e33c49ae6e3b890c9267288fb9f321289910eb5
- sha256: e9cbc796e175685ccb6221f1dd5ea5c43832f545c40557c32b764ff5d567b312

The release tarball is also available signed by me using gpg from here or GitHub

Already built API-Docu can be found online or GitHub.

## Stay tuned!

Subscribe to the RSS feed to always get the release notifications.

For any questions and comments, please contact the issue tracker on GitHub or me by email using elekt.nosp@m.ra@m.nosp@m.arkus.nosp@m.-raa.nosp@m.b.org.

Permalink to this NEWS entry

For more information, see https://libelektra.org

Best regards, Markus