LXD - ArchWiki

Jump to content

# LXD

From ArchWiki

LXD is a manager/hypervisor for containers (via LXC) and virtual-machines (via QEMU).

Related articles

- Linux Containers

## Setup

### Required software

Install the lxd package, then enable `lxd.service`.

### Setup for unprivileged containers

It is recommended to use unprivileged containers (See Linux Containers#Privileged containers or unprivileged containers for an explanation of the difference).

In order to use them, you need to enable support to run unprivileged containers.

Once enabled, every container will be started `unprivileged` by default.

For the alternative see howto set up privileged containers.

### Configure LXD

On first start LXD needs to be configured.
Run as root:

    # lxd init

This will start an interactive configuration guide in the terminal, that covers different topics like storages, networks etc.
You can find an overview in the official Getting Started Guide.

### Accessing LXD as an unprivileged user

By default the LXD daemon allows users in the `lxd` group access, so add your user to the group:

    # usermod -a -G lxd username

Warning: Anyone added to the `lxd` group is root equivalent. For more information see [1] and [2].

## Usage

LXD consists of two parts:

- the daemon (the lxd binary)
- the client (the lxc binary)

Note: lxc is not LXC; the naming is a bit confusing, you can read the forum post on comparing LXD vs LXC regarding the difference.

The client is used to control one or multiple daemon(s).

The client can also be used to control remote LXD servers.

### Overview of commands

You can get an overview of all available commands by typing:

    $ lxc

### Create a container

You can create a container with `lxc launch`, for example:

    $ lxc launch ubuntu:20.04

Container are based on images, that are downloaded from image servers or remote LXD servers.
You can see the list of already added servers with:

    $ lxc remote list

You can list all images on a server with `lxc image list`, for example:

    $ lxc image list images:

This will show you all images on one of the default servers: images.linuxcontainers.org

You can also search for images by adding terms like the distribution name:

    $ lxc image list images:debian

Launch a container with an image from a specific server with:

    $ lxc launch servername:imagename

For example:

    $ lxc launch images:centos/8/amd64 centos

To create an amd64 Arch container:

    $ lxc launch images:archlinux/current/amd64 arch

### Create a virtual machine

Just add the flag `--vm` to `lxc launch`:

    $ lxc launch ubuntu:20.04 --vm

Note:

- For now virtual machines support less features than containers (see Difference between containers and virtual machines for example).
- Only `cloud` variants of the official images enable the lxd-agent out-of-the-box (which is needed for the usual lxc commands like `lxc exec`).
  You can search for cloud images with `lxc image list images: cloud` or `lxc image list images: distribution-name cloud`.
  If you use other images or encounter problems take a look at #lxd-agent inside a virtual machine.

### Use and manage a container or VM

See Instance managament in the official Getting Started Guide of LXD.

### Container/VM configuration (optional)

You can add various options to instances (containers and VMs).
See Configuration of instances in the official Advanced Guide of LXD for details.

## Tips and tricks

### Access the containers by name on the host

This assumes that you are using the default bridge, that it is named lxdbr0 and that you are using systemd-resolved.

     # systemd-resolve --interface lxdbr0 --set-domain '~lxd' --set-dns $(lxc network get lxdbr0 ipv4.address | cut -d / -f 1)

You can now access the containers by name:

     $ ping containername.lxd

#### Other solution

It seems that the systemd-resolve solution stops working after some time.

Another solution is to create a `/etc/systemd/network/lxd.network` that contains (replace x and y to match your bridge IP):

     [Match]
     Name=lxdbr0
     [Network]
     DNS=10.x.y.1
     Domains=~lxd
     IgnoreCarrierLoss=yes
     [Address]
     Address=10.x.y.1/24
     Gateway=10.x.y.1

And then enable and start `systemd-networkd.service`.

### Use Wayland and Xorg applications

Note: Always consider security implications, as some of the described methods may weaken the seperation between container and host.

There are multiple methods to use GUI applications inside containers.

You can find an overview in the official Forum of LXD: https://discuss.linuxcontainers.org/t/overview-gui-inside-containers/8767

#### Method 1: Use the host's Wayland or Xorg Server

Note: Using Xorg might weaken the seperation between container and host, because Xorg allows applications to access other applications windows. So container applications might have access to host applications windows.
Use Wayland instead (but be aware that Xorgs downsides also apply to XWayland).

Summary: In this method we grant containers access to the host's sockets of Wayland (+XWayland) or Xorg.

1. Add the following devices to a containers profile.

See also: LXD-Documentation regarding Devices

General device for the GPU:

    mygpu:
       type: gpu

Note: The path under "listen" is different, because /run and /tmp folders might be overridden, see: https://github.com/lxc/lxd/issues/4540

Device for the Wayland Socket:
Notes:

- Adjust the Display (wayland-0) accordingly.
- Add the folders in /mnt and /tmp inside the container, if they do not already exist.

    Waylandsocket:
        bind: container
        connect: unix:/run/user/1000/wayland-0
        listen: unix:/mnt/wayland1/wayland-0
        uid: "1000"
        gid: "1000"
        security.gid: "1000"
        security.uid: "1000"
        mode: "0777"
        type: proxy

Device for the Xorg (or XWayland) Socket:
Note: Adjust the Display Number accordingly (for example X1 instead of X0).

    Xsocket:
        bind: container
        connect: unix:/tmp/.X11-unix/X0
        listen: unix:/mnt/xorg1/X0
        uid: "1000"
        gid: "1000"
        security.gid: "1000"
        security.uid: "1000"
        mode: "0777"
        type: proxy

2. Link the sockets to the right location inside the container.

Note: These Scripts need to be run after each start of the container; you can automate this with systemd for example.

Shell-Script to link the Wayland socket:

    #!/bin/sh
    mkdir /run/user/1000
    ln -s /mnt/wayland1/wayland-0 /run/user/1000/wayland-0

Link the Xorg (or XWayland) socket:

    #!/bin/sh
    ln -s /mnt/xorg1/X0 /tmp/.X11-unix/X0

3. Add Environment variables to the users config inside the container.

Note: Adjust the Display Numbers and/or the filename (.profile) accordingly.

For Wayland:

    $ echo "export XDG_RUNTIME_DIR=/run/user/1000" >> ~/.profile
    $ echo "export WAYLAND_DISPLAY=wayland-0" >> ~/.profile
    $ echo "export QT_QPA_PLATFORM=wayland" >> ~/.profile

For Xorg (or XWayland):

    $ echo "export DISPLAY=:0" >> .profile

Reload the .profile:

    $ . .profile

4. Install necessary software in the container.

Note: Necessary software needs to be added. For now you can install an example GUI application, this will probably install all necessary packages as well.

5. Start GUI applications.

Now you should be able to start GUI applications inside the container (via terminal for example) and make them appear as a window on your hosts display.

You can try out "glxgears" for example.

### Privileged containers

Note: Privileged containers are not isolated from the host!

The root user in the container is the root user on the host.

Use unprivileged containers instead whenever possible.

If you want to set up a privileged container, you must provide the config key `security.privileged=true`.

Either during container creation:

    $ lxc launch ubuntu:20.04 ubuntu -c security.privileged=true

Or for an already existing container you may edit the configuration:

    $ lxc config edit ubuntu

    name: ubuntu
    profiles:
    - default
    config:
      ...
      security.privileged: "true"
      ...

## Troubleshooting

### lxd-agent inside a virtual machine

Inside some virtual machine images the `lxd-agent` is not enabled by default.
In this case you have to enable it manually, for example by mounting a `9p` network share. This requires console access with a valid user.

1. Login with `lxc console`:
Replace `virtualmachine-name` accordingly.

    $ lxc console virtualmachine-name

Login as root:

Note: On some systems you have to setup a root password first to be able to login as root.
You can use cloud-init for this for example.

    $ su root

Mount the network share:

    $ mount -t 9p config /mnt/

Go into the folder and run the install script (this will enable the lxd-agent inside the VM):

    $ cd /mnt/
    $ ./install.sh

After sucessful install, reboot with:

    $ reboot

Afterwards the `lxd-agent` is available and `lxc exec` should work.

### Check kernel config

By default Arch Linux kernel is compiled correctly for Linux Containers and its frontend LXD. However, if you are using a custom kernel, or changed kernel options the kernel might be configured incorrectly. Verify that your kernel is properly configured:

    $ lxc-checkconfig

### Resource limits are not applied when viewed from inside a container

Install lxcfs and start `lxcfs.service`.

lxd will need to be restarted. Enable `lxcfs.service` for the service to be started at boot time.

### Starting a virtual machine fails

If you see the error: `Error: Required EFI firmware settings file missing: /usr/share/ovmf/x64/OVMF_VARS.ms.fd`

Arch Linux does not distribute secure boot signed ovmf firmware, to boot virtual machines you need to disable secure boot for the time being.

    $ lxc launch ubuntu:18.04 test-vm --vm -c security.secureboot=false

This can also be added to the default profile by doing:

    $ lxc profile set default security.secureboot=false

### No IPv4 with systemd-networkd

Starting with version version 244.1, systemd detects if `/sys` is writable by containers. If it is, udev is automatically started and breaks IPv4 in unprivileged containers. See commit bf331d8 and discussion on linuxcontainers.

On containers created past 2020, there should already be a `systemd.networkd.service` override to work around this issue, create it if it is not:

    /etc/systemd/system/systemd-networkd.service.d/lxc.conf

    [Service]
    BindReadOnlyPaths=/sys

You could also work around this issue by setting `raw.lxc: lxc.mount.auto = proc:rw sys:ro` in the profile of the container to ensure `/sys` is read-only for the entire container, although this may be problematic, as per the linked discussion above.

## Uninstall

Stop and disable `lxd.service` and `lxd.socket`. Then uninstall the lxd package.

If you uninstalled the package without disabling the service, you might have a lingering broken symlink at `/etc/systemd/system/multi-user.wants/lxd.service`.

If you want to remove all data:

    # rm -r /var/lib/lxd

If you used any of the example networking configuration, you should remove those as well.

## See also

- The official LXD homepage
- Official documentation
- Getting Started Guide
- Advanced Guide
- Official Forum
- The LXD GitHub page
- Tutorials in LXD Forum
- Release Notes in LXD Forum

Category:

- Virtualization