Elektra: Plugin: yamlcpp

Elektra 0.8.24

Plugin: yamlcpp

- infos = Information about the yamlcpp plugin is in keys below
- infos/author = René Schwaiger sanss.nosp@m.ecou.nosp@m.rs@me.nosp@m..com
- infos/licence = BSD
- infos/needs = base64 directoryvalue
- infos/provides = storage/yaml
- infos/recommends =
- infos/placements = getstorage setstorage
- infos/status = maintained unittest preview unfinished concept discouraged
- infos/metadata =
- infos/description = This storage plugin reads and writes data in the YAML format

# YAML CPP

## Introduction

The YAML CPP plugin reads and writes configuration data via the yaml-cpp library.

## Usage

You can mount this plugin via `kdb mount`:

sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

. To unmount the plugin use `kdb umount`:

sudo kdb umount /tests/yamlcpp

. The following examples show how you can store and retrieve data via `yamlcpp`.

```<h1>Mount yamlcpp plugin to cascading namespace`/tests/yamlcpp` sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

# Manually add a mapping to the database

echo "🔑 : 🐳" > `kdb file /tests/yamlcpp`

# Retrieve the value of the manually added key

kdb get /tests/yamlcpp/🔑 #> 🐳

# Manually add syntactically incorrect data

echo "some key: @some value" >> `kdb file /tests/yamlcpp` kdb get "/tests/yamlcpp/some key"

# STDERR: .*yaml-cpp: error at line 2, column 11: unknown token.*

# ERROR: 10

# RET: 5

# Overwrite incorrect data

echo "🔑: value" > `kdb file /tests/yamlcpp`

# Add some values via `kdb set`

kdb set /tests/yamlcpp 🎵 kdb set /tests/yamlcpp/fleetwood mac kdb set /tests/yamlcpp/the chain

# Retrieve the new values

kdb get /tests/yamlcpp #> 🎵 kdb get /tests/yamlcpp/the #> chain kdb get /tests/yamlcpp/fleetwood #> mac

# Undo modifications

kdb rm -r /tests/yamlcpp sudo kdb umount /tests/yamlcpp

## Arrays
YAML CPP provides basic support for Elektra’s array data type.

# Mount yamlcpp plugin to cascading namespace `/tests/yamlcpp`

sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

# Manually add an array to the database

echo 'sunny:' > `kdb file /tests/yamlcpp` echo ' - Charlie' >> `kdb file /tests/yamlcpp` echo ' - Dee' >> `kdb file /tests/yamlcpp`

# List the array entries

kdb ls /tests/yamlcpp #> user/tests/yamlcpp/sunny #> user/tests/yamlcpp/sunny/#0 #> user/tests/yamlcpp/sunny/#1

# Read an array entry

kdb get user/tests/yamlcpp/sunny/#1 #> Dee

# You can retrieve the last index of an array by reading the metakey `array`

kdb getmeta /tests/yamlcpp/sunny array

# 1

# Extend the array

kdb set user/tests/yamlcpp/sunny/#2 Dennis kdb set user/tests/yamlcpp/sunny/#3 Frank kdb set user/tests/yamlcpp/sunny/#4 Mac

# The plugin supports empty array fields

kdb set user/tests/yamlcpp/sunny/#_10 'The Waitress' kdb getmeta user/tests/yamlcpp/sunny array #> #_10 kdb get user/tests/yamlcpp/sunny/#_9

# RET: 11

# Retrieve the last array entry

kdb get user/tests/yamlcpp/sunny/$(kdb getmeta user/tests/yamlcpp/sunny array) #> The Waitress

# The plugin also supports empty arrays (arrays without any elements)

kdb setmeta user/tests/yamlcpp/empty array '' kdb export user/tests/yamlcpp/empty yamlcpp #> []

# For arrays with at least one value we do not need to set the type `array`

kdb set user/tests/yamlcpp/movies kdb set user/tests/yamlcpp/movies/#0 'A Silent Voice' kdb getmeta user/tests/yamlcpp/movies array #> #0 kdb export user/tests/yamlcpp/movies yamlcpp #> - A Silent Voice

# Undo modifications to the key database

kdb rm -r /tests/yamlcpp sudo kdb umount /tests/yamlcpp

The plugin also supports nested arrays.

# Mount yamlcpp plugin to cascading namespace `/tests/yamlcpp`

sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

# Add some key value pairs

kdb set /tests/yamlcpp/key value kdb set /tests/yamlcpp/array/#0 scalar kdb set /tests/yamlcpp/array/#1/key value kdb set /tests/yamlcpp/array/#1/🔑 🙈

kdb ls /tests/yamlcpp #> user/tests/yamlcpp/array #> user/tests/yamlcpp/array/#0 #> user/tests/yamlcpp/array/#1/key #> user/tests/yamlcpp/array/#1/🔑 #> user/tests/yamlcpp/key

# Retrieve part of an array value

kdb get /tests/yamlcpp/array/#1/key #> value

# Since an array saves a list of values, an array parent

# - which represent the array - does not store a value!

echo "/tests/yamlcpp/array: “`kdb get /tests/yamlcpp/array`”" #> /tests/yamlcpp/array: “”

# Remove part of an array value

kdb rm /tests/yamlcpp/array/#1/key

kdb ls /tests/yamlcpp #> user/tests/yamlcpp/array #> user/tests/yamlcpp/array/#0 #> user/tests/yamlcpp/array/#1/🔑 #> user/tests/yamlcpp/key

# The plugin stores array keys using YAML sequences.

# Since yaml-cpp stores keys in arbitrary order -

# either `key` or `array` could be the “first” key -

# we remove `key` before we retrieve the data. This way

# we make sure that the output below will always look

# the same.

kdb rm /tests/yamlcpp/key kdb file /tests/yamlcpp | xargs cat #> array: #> - scalar #> - 🔑: 🙈

# Undo modifications to the key database

kdb rm -r /tests/yamlcpp sudo kdb umount /tests/yamlcpp

## Metadata
The plugin supports metadata. The example below shows how a basic `Key` including some metadata, looks inside the YAML configuration file:
```yaml
key without metadata: value
key with metadata:
!elektra/meta
- value2
- metakey: metavalue
empty metakey:
another metakey: another metavalue

. As we can see above the value containing metadata is marked by the tag handle `!elektra/meta`. The data type contains a list with two elements. The first element of this list specifies the value of the key, while the second element contains a map saving the metadata for the key. The data above represents the following key set in Elektra if we mount the file directly to the namespace `user`:

Name | Value | Metaname | Metavalue
--- | --- | --- | ---
user/key without metadata | value1 | — | —
user/key with metadata | value2 | metakey | metavalue
|  | empty metakey | —
|  | another metakey | another metavalue

. The example below shows how we can read and write metadata using the `yamlcpp` plugin via `kdb`.

```<h1>Mount yamlcpp plugin to cascading namespace`/tests/yamlcpp` sudo kdb mount config.yaml user/tests/yamlcpp yamlcpp

# Manually add a key including metadata to the database

echo "🔑: !elektra/meta [🦄, {comment: Unicorn}]" > `kdb file user/tests/yamlcpp` kdb lsmeta user/tests/yamlcpp/🔑 #> comment kdb getmeta user/tests/yamlcpp/🔑 comment #> Unicorn

# Add a new key and add some metadata to the new key

kdb set user/tests/yamlcpp/brand new kdb setmeta user/tests/yamlcpp/brand comment "The Devil And God Are Raging Inside Me" kdb setmeta user/tests/yamlcpp/brand rationale "Because I Love It"

# Retrieve metadata

kdb lsmeta user/tests/yamlcpp/brand #> comment #> rationale kdb getmeta user/tests/yamlcpp/brand rationale #> Because I Love It

# Undo modifications to the key database

kdb rm -r user/tests/yamlcpp sudo kdb umount user/tests/yamlcpp

We can also invoke additional plugins that use metadata like `type`.

sudo kdb mount config.yaml /tests/yamlcpp yamlcpp type kdb set /tests/yamlcpp/typetest/number 21 kdb setmeta /tests/yamlcpp/typetest/number check/type short

kdb set /tests/yamlcpp/typetest/number "One"

# RET: 5

# STDERR: .*The type short failed to match for .*/number with string: One.*

# ERROR: 52

kdb get /tests/yamlcpp/typetest/number #> 21

# Undo modifications to the key database

kdb rm -r /tests/yamlcpp sudo kdb umount /tests/yamlcpp

## Binary Data
YAML CPP also supports [base64](https://tools.ietf.org/html/rfc4648) encoded data via the [Base64](@ref src_plugins_base64_README_md) plugin.

# Mount YAML CPP plugin at cascading namespace `/tests/binary`

sudo kdb mount test.yaml /tests/binary yamlcpp

# Manually add binary data

echo 'bin: !!binary aGk=' > `kdb file /tests/binary`

# Base 64 decodes the data `aGk=` to `hi` and stores the value in binary form.

# The command `kdb get` prints the data as hexadecimal byte values.

kdb get /tests/binary/bin #>

# We can use `bash` to convert the hexadecimal value returned by `kdb get`

# to its ASCII representation.

bash -c 'printf `kdb get /tests/binary/bin`' #> hi

# Add a string value to the database

kdb set /tests/binary/text mate

# Base 64 does not modify textual values

kdb get /tests/binary/text #> mate

# The Base 64 plugin re-encodes binary data before YAML CPP stores the key set. Hence the

# configuration file contains the value `aGk=` even after YAML CPP wrote a new configuration.

grep -q 'bin: !.* aGk=' `kdb file user/tests/binary`

# RET: 0

# Undo modifications to the database

kdb rm -r /tests/binary sudo kdb umount /tests/binary

## Null & Empty
Sometimes you only want to save a key without a value (null key) or a key with an empty value. The commands below show that YAML CPP supports this scenario properly.

# Mount YAML CPP plugin at cascading namespace `user/tests/yamlcpp`

sudo kdb mount test.yaml user/tests/yamlcpp yamlcpp

# Check if the plugin saves null keys correctly

kdb set user/tests/yamlcpp/null kdb set user/tests/yamlcpp/null/level1/level2 kdb setmeta user/tests/yamlcpp/null/level1/level2 comment 'Null key'

kdb ls user/tests/yamlcpp/null #> user/tests/yamlcpp/null #> user/tests/yamlcpp/null/level1/level2 kdb get -v user/tests/yamlcpp/null | grep -q 'The key is null.' kdb get -v user/tests/yamlcpp/null/level1/level2 | grep -q 'The key is null.'

# Check if the plugin saves empty keys correctly

kdb set user/tests/yamlcpp/empty "" kdb set user/tests/yamlcpp/empty/level1/level2

kdb ls user/tests/yamlcpp/empty #> user/tests/yamlcpp/empty #> user/tests/yamlcpp/empty/level1/level2 kdb get -v user/tests/yamlcpp/empty | grep -vq 'The key is null.'

# Undo modifications to the database

kdb rm -r user/tests/yamlcpp sudo kdb umount user/tests/yamlcpp

## Dependencies
This plugin requires [yaml-cpp][]. On a Debian based OS the package for the library is called `libyaml-cpp-dev` . On macOS you can install the package `yaml-cpp` via [HomeBrew](https://brew.sh).
## Limitations
### Leaf Values
One of the limitations of this plugin is, that it only supports values inside [leaf nodes](https://github.com/ElektraInitiative/libelektra/issues/106). Let us look at an example to show what that means. The YAML file below:
```yaml
root:
subtree: 🍂
below root: leaf
level 1:
level 2:
level 3: 🍁

stores all of the values (`🍂`, `leaf` and `🍁`) in the leaves of the mapping. The drawing below makes this situation a little bit clearer.

The key set that this plugin creates using the data above looks like this (assuming we mount the plugin to `user/tests/yamlcpp`):

Name | Value
--- | ---
user/tests/yamlcpp/level |
user/tests/yamlcpp/level 1/level 2 |
user/tests/yamlcpp/level 1/level 2/level 3 | 🍁
user/tests/yamlcpp/root |
user/tests/yamlcpp/root/below root | leaf
user/tests/yamlcpp/root/subtree | 🍂

. Now why is this plugin unable to store values outside leaf nodes? For example, why can we not store a value inside `user/tests/yamlcpp/level 1/level 2`? To answer this question we need to look at the YAML representation:

level 1:
level 2:
level 3: 🍁

. In a naive approach we might just try to add a value e.g. `🙈` right next to level 2:

level 1:
level 2: 🙈
level 3: 🍁

. This however would be not correct, since then the YAML node `level 2` would contain both a scalar value (`🙈`) and a mapping (`{ level 3: 🍁 }`). We could solve this dilemma using a list:

level 1:
level 2:
- 🙈
- level 3: 🍁

. However, if we use this approach we are not able to support Elektra’s array type properly.

#### Directory Values

To overcome the limitation described above, the YAML CPP plugin requires the Directory Value plugin. This plugin converts the value of a non-leaf node to a leaf node with the name `___dirdata`. For example, let us assume we have the following key set:

directory = Directory Data
directory/file = Leaf Data

. The Directory Value plugin will convert the key set in the set (write) direction to

directory =
directory/___dirdata = Directory Data
directory/file = Leaf Data

. Consequently the YAML plugin will store the key set as

directory:
___dirdata = Directory Data
file = Leaf Data

. A user of the YAML plugin will not notice this feature unless he edits the configuration file by hand, as the following example shows:

```<h1>Mount YAML CPP plugin at`user/tests/yamlcpp` sudo kdb mount test.yaml user/tests/yamlcpp yamlcpp

kdb set user/tests/yamlcpp/directory 'Directory Data' kdb setmeta user/tests/yamlcpp/directory comment 'Directory Metadata' kdb set user/tests/yamlcpp/directory/file 'Leaf Data'

kdb ls user/tests/yamlcpp/directory #> user/tests/yamlcpp/directory #> user/tests/yamlcpp/directory/file

kdb get user/tests/yamlcpp/directory #> Directory Data kdb getmeta user/tests/yamlcpp/directory comment #> Directory Metadata kdb get user/tests/yamlcpp/directory/file #> Leaf Data

# Undo modifications to the database

kdb rm -r user/tests/yamlcpp sudo kdb umount user/tests/yamlcpp

.
### Other Limitations
- Adding and removing keys does remove **comments** inside the configuration file
- The plugin currently lacks proper **type support** for scalars.
- If Elektra uses YAML CPP as **default storage** plugin, multiple tests of the test suite fail. However, if you mount YAML CPP at `/`:

kdb mount default.yaml / yamlcpp ```

all tests should work correctly. The problem here is that Elektra does not load additional required plugins (`infos/needs`) for a default storage plugin.