diff --git a/README.md b/README.md index 41104b5..d8eac58 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Universal Blue - Forge -On-premises Universal Blue. This repository is intended to provide the service units +On-premises Universal Blue. This projects intended is to provide the service units necessary to set up a self-hosted OS forge for custom images. > **Warning** @@ -30,8 +30,8 @@ then made available to all other components and are valid for 2 years and 30 day ### Reverse Proxy -As an entry point for all components we use [Traefik](https://doc.traefik.io/traefik/) as -a reverse proxy. Based on URL routing it will redirect the traffic to the +As an entry point for all web components we use [Traefik](https://doc.traefik.io/traefik/) +as a reverse proxy. Based on URL routing it will redirect the traffic to the right container instance. The reverse proxy dashboard is available at @@ -39,7 +39,7 @@ The reverse proxy dashboard is available at ### Container Registry As container registry we make use of the [Docker Registry 2.0](https://hub.docker.com/_/registry/) -implementation for storing and distributing container images +implementation for storing and distributing container images. The container registry API is available at @@ -47,11 +47,39 @@ The container registry API is available at The blacksmith's work is done with [Ansible](https://docs.ansible.com/ansible/latest/index.html). -The shiny GUI is missing but this should not shy us away. See [usage](#usage) for instructions. +There are two methods of operating the forge, either via a [GUI](./docs/gui.md) available +at or via [just](./docs/just.md) command runner. -## Handling the forge +Details about the project and usage instructions are available in the [documentation](./docs/index.md) +section. -You can use the `forge.sh` to **setup**, **heat-up** and **cool-down** the forge. +## Installation + +### Pre-requisites + +As many tools as possible are built-in but still we rely on some pre-requisites. +These tools and service are necessary to get started: + +- [Podman](https://podman.io/) + Must be installed and a [podman socket](https://github.com/containers/podman/blob/main/docs/tutorials/socket_activation.md) + in the user space must be active. + +- [jq](https://jqlang.github.io/jq/) + Must be installed. It it currently needed in the setup process to parse certain parameters + automatically for you + +- The kernel parameter `net.ipv4.ip_unprivileged_port_start` must be set to `80` + This is because of podman's [shortcoming](https://github.com/containers/podman/blob/main/rootless.md#shortcomings-of-rootless-podman) + to bind to ports `< 1024`. Our reverse-proxy is listening on port `80` and `443` + for incoming traffic. + +- [OpenSSH](https://www.openssh.com/) + Must be installed and the service activated. Ansible needs this to execute all the fancy + commands on your host for you. + +### Setup / Heat-Up / Cool-Down + +For the initial setup and maintenance of the forge you can use the [forge.sh](forge.sh) script: @@ -63,32 +91,4 @@ You can use the `forge.sh` to **setup**, **heat-up** and **cool-down** the forge -### Usage - -Once the forge has been setup the following recipes are available via [just command runner](https://github.com/casey/just). - - - -| Just recipe | Input argument | Default argument value | Description | -| --------------------- | ----------------------- | ------------------------------------------- | -------------------------------------------- | -| `forge_project-clone` | `forge_config_var_file` | $HOME/ublue-os_forge/forge_default_vars.env | Clone git project repository | -| `forge_project-build` | `forge_config_var_file` | $HOME/ublue-os_forge/forge_default_vars.env | Build container image and upload to registry | - - - -All available settings for the `forge_config_var_file` are documented in the [variables.md](./docs/variables.md) -file. To launch a recipe you simple run: - -```sh -just -f forge.just {{ recipe_name }} {{ forge_config_var_file }} -``` - -**_Example:_** - -```sh -just -f forge.just forge_project-clone ~/ublue-os_forge/my-forge-project.env -``` - -In case you don't have [just command runner](https://github.com/casey/just) available. -Have a look at the [forge.just](./forge.just) file. It easy enough to understand which commands -are executed via the just recipes. +Details about what the setup does can be found [here](./docs/setup.md). diff --git a/docs/assets/gui_ansible.png b/docs/assets/gui_ansible.png new file mode 100644 index 0000000..087e9b8 Binary files /dev/null and b/docs/assets/gui_ansible.png differ diff --git a/docs/assets/gui_navigation.png b/docs/assets/gui_navigation.png new file mode 100644 index 0000000..307268d Binary files /dev/null and b/docs/assets/gui_navigation.png differ diff --git a/docs/gui.md b/docs/gui.md new file mode 100644 index 0000000..22444f7 --- /dev/null +++ b/docs/gui.md @@ -0,0 +1,36 @@ +# Usage with GUI + +We have simple UI available at . + +## Navigation + +For navigation you can use the navigation bar on the top left. + +![navigation](./assets/gui_navigation.png) + +### Home + +Currently nothing to see here. The goal is to display some +general information here. Links to documentation and important +news etc. + +### Ansible + +Here you can load your forge [configuration](./index.md#configuration) file from the data store, +clone projects and build container images. The buttons should be self explanatory. +The "Deploy VM" button is currently a mock-up, sorry :wink: +we are [working](https://github.com/ublue-os/forge/issues/35) on it. + +![ansible](./assets/gui_ansible.png) + +### Registry + +Currently nothing to see here. The goal is to display some +information about the docker registry here. +Like a list of all available images you have uploaded etc. + +### About + +Currently nothing to see here. The goal is to display some +information here about the software and versions used in this project +and maybe some acknowledgments etc. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..d06b925 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,65 @@ +# Universal Blue - Forge Documentation + +On-premises Universal Blue provides the service units necessary to set up a self-hosted +OS forge for custom images. + +## Configuration + +Since we are using ansible in the background, all user configurations are loaded via +[`extra-vars`](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_variables.html#defining-variables-at-runtime) +at runtime. No worries on how things are plugged together, you only have to know where +to create your configuration files. That's all. + +So, during forge setup a podman volume `ublue-os_forge-data` is created. All user configuration +etc. is stored there. You can inspect the volume with: + +```sh +❯ podman volume inspect ublue-os_forge-data +[ + { + "Name": "ublue-os_forge-data", + "Driver": "local", + "Mountpoint": "/var/home/stephan/.local/share/containers/storage/volumes/ublue-os_forge-data/_data", + "CreatedAt": "2024-05-06T21:25:28.818751853+02:00", + "Labels": {}, + "Scope": "local", + "Options": {}, + "MountCount": 0, + "NeedsCopyUp": true, + "LockNumber": 98 + } +] +``` + +Under the `"Mountpoint:"` you find the actual path on your file system. I would recommend you +symlink this folder to a path more rememberable. For example you can do the following: + +```sh +FORGE_POD_DATA_DIR="$(podman volume inspect ublue-os_forge-data | jq -r '.[0].Mountpoint')" +ln -s $FORGE_POD_DATA_DIR $HOME/ublue-os_forge-data +``` + +With this the data folder would be available in your home directory under `~/ublue-os_forge-data` + +In that folder you will find an example configuration similar to this: + +```yaml +## ublue-os forge extra-vars example configuration +## For more details got to https://github.com/ublue-os/forge/blob/main/docs/variables.md +--- +forge_git_repository_url: https://github.com/ublue-os/bluefin.git +forge_git_repository_destination: /var/home/stephan/.local/share/containers/storage/volumes/ublue-os_forge-data/_data/data/bluefin +forge_git_repository_version: main +forge_registry_url: registry.ublue.local +``` + +This file acts as a template. It has all available settings outlined for you. Simple copy +it and modify it to your liking. For each project you want to handle with the forge you can +create a dedicated file. + +Details about the available variables are documented [here](./variables.md). + +## Usage + +There are two methods of operating the forge, either via a [GUI](./gui.md) available +at or via [just](./just.md) command runner. diff --git a/docs/just.md b/docs/just.md new file mode 100644 index 0000000..5859f7d --- /dev/null +++ b/docs/just.md @@ -0,0 +1,31 @@ +# Usage with just command runner + +If you don't want to use the [GUI](./gui.md) we provide the following recipes +via [just command runner](https://github.com/casey/just). + + + +| Just recipe | Input argument | Description | +| --------------------- | ----------------------- | -------------------------------------------- | +| `forge_project-clone` | `forge_config_var_file` | Clone git project repository | +| `forge_project-build` | `forge_config_var_file` | Build container image and upload to registry | + + + +The input argument expects a [configuration](./index.md#configuration) file from the data store. + +To launch a recipe you simple run: + +```sh +just -f forge.just {{ recipe_name }} {{ forge_config_var_file }} +``` + +**_Example:_** + +```sh +just -f forge.just forge_project-clone ~/ublue-os_forge/my-forge-project.env +``` + +In case you don't have [just command runner](https://github.com/casey/just) available. +Have a look at the [forge.just](../forge.just) file. It easy enough to understand which commands +are executed via the just recipes. diff --git a/docs/setup.md b/docs/setup.md new file mode 100644 index 0000000..66b7354 --- /dev/null +++ b/docs/setup.md @@ -0,0 +1,43 @@ +# Forge setup + +The initial setup and maintenance of the forge you can use the `forge.sh` script: + + + +| Command | Description | +| ---------------------- | ----------------------------------------------------------- | +| `./forge.sh setup` | Setup the forge for the first time or update existing setup | +| `./forge.sh heat-up` | Start the forge | +| `./forge.sh cool-down` | Stop the forge | + + + +## What happens during setup + +Just in case you haven't have a look at the [forge.sh](../forge.sh) script. The following +things happen during the setup: + +1. The [pre-requisites](../README.md#pre-requisites) are checked + +2. You will be asked for your user credentials + This is not to spy on you. We need them to execute the [configure_host.yml](../anvil/ansible/playbooks/configure_host.yml). + It will configure the necessary settings so you don't have to do that manually. + See step 7. + +3. A podman network `ublue-os_forge` is created. All containers from this setup will be + attached to it in the next step + +4. All containers are built and spun up according to the [forge-pod.yml](../forge-pod.yml) + specification. + +5. The forge SSH key is added to your `~/.ssh/authorized_keys` file. + The root certificate and key are created during container build mentioned in the + previous step. + +6. The forge root certificate is copied to the [data](index.md#configuration) directory. + +7. The [configure_host.yml](../anvil/ansible/playbooks/configure_host.yml) playbook mentioned + in step 2 is executed. + This will created the necessary host entries in `/etc/hosts`, copy the forge root certificate + to `/etc/pki/ca-trust/source/anchors/`, `update-ca-trust` and create the example configuration + in the [data](index.md#configuration) directory. diff --git a/docs/variables.md b/docs/variables.md index 5252b53..420e36e 100644 --- a/docs/variables.md +++ b/docs/variables.md @@ -1,52 +1,23 @@ # Variables -The following sections contains all important variables defined for daily usage. -All variables mentioned here can be declared in a line-delimited file of environment variables. +On this page all important variables defined for daily usage are documented. +All variables mentioned here can be declared in a `yaml` configuration file. -An example file on the host system with all variables available will be created on setup -for you. By default it can be found under `$HOME/ublue-os_forge/forge_default_vars.env`. +Have a look at the [configuration](./index.md#configuration) chapter for details +on where to find the configuration directory. -On playbook launch the variable file will be imported into the ansible container so that -the settings are available during playbook execution. - -## group_vars/all/data.yml - -In the [data.yml](../ansible/group_vars/all/data.yml) all variables are defined -which are used in the context of the data handling. +The following configuration variables are available and can be set to your liking: -| name | type | environment variable | default value | description | -| ---------------------------------------- | ---- | -------------------- | ------------------------------------------- | --------------------------------------------- | -| `forge_data_path` | str | `FORGE_DATA_PATH` | $HOME/ublue-os_forge | Path where forge will store files per default | -| `forge_data_default_variables_file_path` | str | | $HOME/ublue-os_forge/forge_default_vars.env | Path to default configuration file | +| Name | Type | Default value | Description | +| ---------------------------------- | ---- | ------------------------------------------------- | -------------------------------------------------------------------------------- | +| `forge_git_repository_url` | str | | Git repository url | +| `forge_git_repository_destination` | str | `{{ forge_data_volume_mountpoint }}`/data/bluefin | Git destination where repository is cloned to. Can be any directory on your host | +| `forge_git_repository_version` | str | main | Git repository branch or tag or commit version | +| `forge_registry_url` | str | registry.ublue.local | Container registry url | - + -## group_vars/all/git.yml - -In the [git.yml](../ansible/group_vars/all/git.yml/) all variables are defined which are -used in the context of the git repositories. - - - -| name | type | environment variable | default value | description | -| ---------------------------------- | ---- | ---------------------------------- | ----------------------------------------- | ---------------------------------------------- | -| `forge_git_repository_url` | str | `FORGE_GIT_REPOSITORY_URL` | | Git repository url | -| `forge_git_repository_destination` | str | `FORGE_GIT_REPOSITORY_DESTINATION` | $HOME/ublue-os/forge/bluefin | Git destination where repository is cloned to | -| `forge_git_repository_version` | str | `FORGE_GIT_REPOSITORY_VERSION` | main | Git repository branch or tag or commit version | - - - -## group_vars/all/registry.yml - -In the [registry.yml](../ansible/group_vars/all/registry.yml) all variables are defined -which are used in the context of the container registry. - - - -| name | type | environment variable | default value | description | -| -------------------- | ---- | -------------------- | -------------------- | ---------------------- | -| `forge_registry_url` | str | `FORGE_REGISTRY_URL` | registry.ublue.local | Container registry url | - - +**_Note:_** The `{{ forge_data_volume_mountpoint }}` points to your ublue-os_forge-data +podman volume.