M4

Alteeve Wiki :: How To :: M4

Warning: This is little more than a collection of notes at this point. Do not consider anything here to be complete or accurate.

How to build an offline/ait-gapped, minimal, highly available Open Shift cluster.

Setting Up A Test Environment

This is a guide to setting up a bare iron machine to run open shift and kcli.

Install

whoami
# digimer
sudo dnf -y install libvirt libvirt-daemon-driver-qemu qemu-kvm tar
sudo usermod -aG qemu,libvirt $(id -un)
sudo newgrp libvirt
sudo systemctl enable --now libvirtd
sudo dnf -y copr enable karmab/kcli

sudo dnf -y install kcli </syntaxhighlight>

Configure

sudo kcli create pool -p /var/lib/libvirt/images default

Creating pool default...

sudo setfacl -m u:$(id -un):rwx /var/lib/libvirt/images
sudo virsh net-destroy default

Network default destroyed

sudo virsh net-undefine default

Network default has been undefined

kcli create network  -c 192.168.0.0/16 default

Network default deployed

Create the config;

kcli create host kvm -H 127.0.0.1 local

Using local as hostname
Host local created

Note: Use 'pull-secret', not rhn stuff.

cat ~/openshift-config.yml

info: Madi's Test Plan on os-02
cluster: mk-anvil-02
domain: 'digimer.ca'
version: stable
tag: 4.19
ctlplanes: 3
workers: 3
memory: 16384
numcpus: 16
pull_secret: /home/digimer/pull-secret.txt

Make sure there's a key for root and for the user running 'kcli';

if [ -e ~/.ssh/id_ed25519.pub ]; then echo key exists; else echo key needed; ssh-keygen -f ~/.ssh/id_ed25519 -P ""; fi

sudo kcli create kube openshift --paramfile ~/openshift-config.yml mk-anvil-02

Deploying on client local
Deploying cluster mk-anvil-02
Using stable version
Using 192.168.255.253 as api_ip
Downloading openshift-install from https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/stable-4.19
Move downloaded openshift-install somewhere in your PATH if you want to reuse it
Using installer version 4.19.7
Grabbing image rhcos-9.6.20250523-0-openstack.x86_64.qcow2 from url https://rhcos.mirror.openshift.com/art/storage/prod/streams/rhel-9.6/builds/9.6.20250523-0/x86_64/rhcos-9.6.20250523-0-openstack.x86_64.qcow2.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 40 1196M   40  480M    0     0  25.6M      0  0:00:46  0:00:18  0:00:28 25.7M
....

Go have a coffee or a nap, this will take a while...

More info;

https://kcli.readthedocs.io/en/latest/#openshift-cluster-creation
vim mytest.yml
kcli create kube openshift --paramfile mytest.yml mk-anvil-01

Notes

• Bare iron OS is hardly relevant, it gets rebuilt.

defaultimg=""
ctlplanes="1"

kcli create plan --inputfile "$(dirname $0)/deployers/kcli-plan.yml" --threaded --param image=$defaultimg --param ctlplanes=$controllers --param workers=$workers $extregparam "$1"

parameters:
  info: kubesan kcli test plan
  cluster: kubesan-test
  ctlplanes: 3
  workers: 3
  image: fedora40


kubesan-test:   # replace with 'an-anvil-01'
  type: kube
  ctlplanes: {{ ctlplanes }}
  workers: {{ workers }}
  image: {{ image }}     # remove this
  domain: ''             
  <add pull-secret file>

mycluster:
  type: cluster
  kubetype: openshift
  okd: true
  ctlplanes: 3
  workers: 3

Test mk-anvil cluster:

parameters:
  info: kubesan kcli test plan
  cluster: mk-anvil
  ctlplanes: 3
  workers: 3
  image: fedora40

mk-anvil:
  type: kube
  ctlplanes: {{ ctlplanes }}
  workers: {{ workers }}
  image: {{ image }}
  domain: ''
  registry: true
  cmds:
    - yum -y install podman lvm2-lockd sanlock
    - sed -i "s|# use_watchdog = 1|use_watchdog = 0|" /etc/sanlock/sanlock.conf
    - >-
      sed -i "
      s|# validate_metadata = \"full\"|validate_metadata = \"none\"|;
      s|# multipath_component_detection = 1|multipath_component_detection = 0|;
      s|# md_component_detection = 1|md_component_detection = 0|;
      s|# backup = 1|backup = 0|;
      s|# archive = 1|archive = 0|;
      s|# use_lvmlockd = 0|use_lvmlockd = 1|;
      s|# thin_check_options = \[.*\]|thin_check_options = \[ \"-q\", \"--clear-needs-check-flag\", \"--skip-mappings\" \]|;
      s|# io_memory_size = 8192|io_memory_size = 65536|;
      s|# reserved_memory = 8192|reserved_memory = 0|
      " /etc/lvm/lvm.conf
{%for node in cluster|kubenodes(ctlplanes, workers) %}
    - if [ "$(hostname)" == "{{ node }}" ]; then sed -i "s|# host_id = 0|host_id = {{ loop.index }}|" /etc/lvm/lvmlocal.conf; fi
{%endfor%}
    - systemctl enable --now podman lvmlockd sanlock
# TODO: paramaterize shared storage
kubesan-test-shared-1.img:
  type: disk
  thin: false
  size: 5
  pool: default
  vms: {{ cluster|kubenodes(ctlplanes, workers) }}

References

• RHN OpenShift
• kcli os deploy
• Deploying Red Hat OpenShift Operators in a disconnected environment
• kcli
• example
• Aman's link

 

Any questions, feedback, advice, complaints or meanderings are welcome.
Alteeve's Niche!	Alteeve Enterprise Support	Community Support
© 2025 Alteeve. Intelligent Availability® is a registered trademark of Alteeve's Niche! Inc. 1997-2025
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions.