Read time 2 minutes
Overview of some services on my network. The setup is fully redundant as high availability cluster. Driven by open source and powered with renewable energy. Using Terraform, Cloud-init and Ansible for IaC. And K8s, Gitlab, Flux2 and Renovate for GitOps. Network Diagram DMZ Service Description Postfix, Dovecot, SpamAssassin Email Server with DKIM, DMARC, SPF, DANE,Read more
Read time 2 minutes
I’ve been managing my cluster using IaC tools for quite some while now and while with K8S I try to always go with an Immutable approach and therefore replace whole VM’s or Containers there are still some edge cases where an in place upgrades is necessary, e. g. hypervisor, gateways, etc. That’s where my APT-UpgradeRead more
Read time 2 minutes
Error tracking is an essential monitoring tool if you develop software. Sentry is basically the only tool for that, but it comes at a heavy cost: resource usage. A default install of Sentry already needs at least 14GB of RAM. Basically you have to provide 24-32GB for even the smallest instance and If you everRead more
Read time < 1 minute
ZFS with it’s default options is not great for SWAP usage. If you run “Root on ZFS” you want to create a ZVOL with the right options. This means limiting arc to metadata, disabling l2arc, forcing sync writes, setting logbias to throughput and disabling autosnapshots. Adjust 8G to the swap size you want. Adjust rpoolRead more
Read time 2 minutes
We will use the dashboard that we previously created for reference see https://herold.space/proxmox-grafana-dashboard Prepare Use Etcher to copy Raspberry Pi OS with desktop onto a SD Card. Change the password for “pi” and setup a static IP. Also run updates. Kiosk First enable auto login and disable screen blanking via raspi config cli. The defaultRead more
Read time 2 minutes
InfluxDB First we have to create a time series database to store the metrics. Proxmox only supports Graphite and InfluxDB. I will run InfluxDB via Docker tagged to v1.8 for now. To create a default config run influxd. Proxmox only supports UDP to send metrics which is not enabled by default. We create a UDPRead more
Read time 3 minutes
Although ZFS also offers encryption there is no support for full disk encryption yet so we will use LUKS instead. Requirements: ZFS mirror (raid1) install In order to encrypt a running system rpool must be a ZFS mirror. To verify you are running a ZFS mirror execute the following command. Note: If you don’t wantRead more
Read time < 1 minute
I wanted a way to know who authenticates on my servers. This is especially useful if you work in a team or want to have some additional security in place. Rsyslogd is a service that ships with most distros which proceeds logs before they are written to disk. We will create a simple script whichRead more
Read time 2 minutes
Install SSH Start by installing SSH. Sudo For best practices you should disable root login and create a new user with sudo access. Add the user to the sudo group. Whenever needed you should use sudo to execute binaries with root privileges. To pipe text use sudo in combination with tee. If you know whatRead more
Read time 2 minutes
Besides VPN and SOCKS5 proxies using the Tor anonymity network to scan servers and run brute force attacks is a common choice. It is especially useful to circumvent rate limiting given that you have easy access to a set of over 1500 IP addresses. So if you have your own rate limiting implementation make sureRead more
Read time 2 minutes
Most Linux distros ship with a longterm kernel that can be quite old. If you happen to buy new hardware the driver support can be troublesome. So in order to get things like network interfaces, wifi, bluetooth, audio etc. working you might have to build the kernel yourself or switch to a rolling release distro.Read more