# Mikkel's Homelab Infrastructure

## Overview

This document describes the complete homelab infrastructure spanning Hetzner cloud servers, home NAS, and future expansion plans.

---

## Core Infrastructure

### core.georgsen.dk (Primary Server)

| Attribute | Value |
|-----------|-------|
| Provider | Hetzner (AX52) |
| Location | Helsinki, Finland |
| CPU | AMD Ryzen 7 3700X |
| RAM | 64GB ECC |
| Storage | 2x 1TB NVMe (RAID0) - ZFS (rpool) |
| OS | Proxmox VE |
| Public IP | 65.108.14.165/26 |
| IPv6 | 2a01:4f9:6a:4a4f::/64 |

**Warning:** Storage is RAID0 — no local redundancy. PBS backups to Synology are critical.

### Network Bridges

| Bridge | Purpose | Subnet | Gateway |
|--------|---------|--------|---------|
| vmbr0 | Public internet | 65.108.14.165/26 | 65.108.14.129 |
| vmbr1 | Internal services | 10.5.0.0/24 | 10.5.0.254 |
| vmbr2 | Hetzner vSwitch | 10.9.1.0/24 | 10.9.1.1 |

### vSwitch Details
- Cloud Network: #11828854
- Subnet: 10.9.1.0/24
- Routes to: 10.9.0.0/16 via 10.9.1.1
- MTU: 1400

---

## Network Configuration

### /etc/network/interfaces (core)

```
auto lo
iface lo inet loopback

auto enp9s0
iface enp9s0 inet manual

auto enp9s0.4000
iface enp9s0.4000 inet manual
    mtu 1400

auto vmbr0
iface vmbr0 inet static
    address 65.108.14.165/26
    gateway 65.108.14.129
    bridge-ports enp9s0
    bridge-stp off
    bridge-fd 0

iface vmbr0 inet6 static
    address 2a01:4f9:6a:4a4f::/64
    gateway fe80::1

auto vmbr1
iface vmbr1 inet static
    address 10.5.0.254/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0

auto vmbr2
iface vmbr2 inet static
    address 10.9.1.2/24
    bridge-ports enp9s0.4000
    bridge-stp off
    bridge-fd 0
    mtu 1400
    up ip route add 10.9.0.0/16 via 10.9.1.1 dev vmbr2
    down ip route del 10.9.0.0/16 via 10.9.1.1 dev vmbr2
```

### Port Forwarding (iptables)

| External Port | Internal Destination | Purpose |
|---------------|---------------------|---------|
| 80 | 10.5.0.1:80 | NPM HTTP |
| 443 | 10.5.0.1:443 | NPM HTTPS |

NAT masquerade enabled for 10.5.0.0/24 → vmbr0

### Firewall Rules (INPUT on vmbr0)

**Home IP (83.89.248.247) is whitelisted - always allowed.**

| Protocol | Port | Action | Purpose |
|----------|------|--------|---------|
| ALL | * | ACCEPT | Allow home IP (83.89.248.247) |
| TCP/UDP | 111 | DROP | Block portmapper from internet |
| TCP/UDP | 53 | DROP | Block DNS (prevent amplification attacks) |
| TCP | 3128 | DROP | Block spiceproxy |
| TCP | 8006 | DROP | Block Proxmox UI (use home IP or Tailscale) |
| TCP | 8008 | DROP | Block Proxmox console |

Saved with: `netfilter-persistent save`

### DHCP (dnsmasq)
- Range: 10.5.0.100 - 10.5.0.200
- Lease time: 24h

---

## Virtual Machines

### VM 200: mail.georgsen.dk (Stalwart Mail Server)

| Attribute | Value |
|-----------|-------|
| Type | VM (KVM) |
| IP | 65.108.14.164 (dedicated public IP) |
| Bridge | vmbr0 (direct) |
| Software | Stalwart Mail Server |
| Webmail | Snappymail (via dockge) |

**Current domains:** dataloes.dk (building reputation before adding more)

**Planned domains:** georgsen.dk, microsux.dk, dataloes.dk

---

## LXC Containers

### Container Overview

| VMID | Name | IP | Purpose | Status |
|------|------|-----|---------|--------|
| 100 | npm | 10.5.0.1 | Nginx Proxy Manager | Running |
| 101 | dockge | 10.5.0.10 | Docker Compose Manager | Running |
| 102 | mgmt | 10.5.0.102 | Management/Automation (Claude Code) | Running |
| 103 | postgresql01 | DHCP | PostgreSQL (community) | Running |
| 104 | redis01 | DHCP | Redis (community) | Running |
| 105 | sentry | DHCP | Defense Intelligence System | Running |
| 106 | pbs | 10.5.0.6 | Proxmox Backup Server | Running |
| 107 | pve-scripts-local | DHCP | Community Scripts Web UI | Running |
| 108 | jukebox | DHCP (→10.5.0.184) | Music Player (custom project) | Running |
| 110 | sense.microsux.dk | DHCP | CBD Vendor Locator | Stopped |
| 111 | dev | DHCP | Development container | Running |
| 112 | dataloes | 10.5.0.112 | dataloes.dk website | Stopped |
| 113 | general | 10.5.0.113 | Decomissioned | Stopped |
| 114 | forgejo | 10.5.0.14 | Git server (Forgejo) | Running |
| 115 | dns | 10.5.0.2 | DNS server (Technitium) | Running |
| 1000 | tailscale | 10.5.0.x + 10.9.1.10 | Tailscale relay | Running |

### Container Details

#### 100: NPM (Nginx Proxy Manager)

- **Purpose:** Reverse proxy with automatic Let's Encrypt certificates
- **IP:** 10.5.0.1
- **Ports:** 80, 443, 81 (admin)
- **SSL:** Let's Encrypt (HTTP challenge)
- **Runtime:** Docker (docker-compose at /opt/npm)

**Update NPM:**
```bash
cd /opt/npm && docker compose pull && docker compose up -d
```

**Proxy Hosts:**

| Domain | Destination | SSL |
|--------|-------------|-----|
| dataloes.dk, www.dataloes.dk | http://10.5.0.112:80 | Let's Encrypt |
| dev01.georgsen.dk | http://10.5.0.113:3001 | Let's Encrypt |
| dns.georgsen.dk | http://10.5.0.2:5380 | Let's Encrypt |
| dockge.georgsen.dk | http://10.5.0.10:5001 | Let's Encrypt |
| git.georgsen.dk | http://10.5.0.14:3000 | Let's Encrypt |
| jukebox.georgsen.dk | http://10.5.0.184:4000 | Let's Encrypt |
| pbs.georgsen.dk | https://10.5.0.6:8007 | Let's Encrypt |
| status.georgsen.dk | http://10.5.0.10:3001 | Let's Encrypt |
| webmail.georgsen.dk | http://10.5.0.10:8888 | Let's Encrypt |

#### 101: Dockge

- **Purpose:** Docker Compose stack management
- **IP:** 10.5.0.10
- **Port:** 5001

**Running Stacks:**
```yaml
# Snappymail (webmail for Stalwart)
services:
  snappymail:
    image: djmaze/snappymail:latest
    container_name: snappymail
    restart: unless-stopped
    ports:
      - 8888:8888
    volumes:
      - ./data:/var/lib/snappymail

# Uptime Kuma (status monitoring)
services:
  uptime-kuma:
    image: louislam/uptime-kuma:1
    container_name: uptime-kuma
    restart: unless-stopped
    ports:
      - 3001:3001
    volumes:
      - ./data:/app/data
```

#### 105: Sentry (Defense Intelligence)

- **Purpose:** Custom defense intelligence system for monitoring military contracting opportunities
- **Focus:** Ground equipment (EV conversion, diesel upgrades, specialty vehicles, bomb loaders, Humvee engines)
- **Databases:** Uses postgresql01 and redis01

#### 106: PBS (Proxmox Backup Server)

- **Purpose:** Centralized backup for all PVE nodes
- **IP (vmbr1):** 10.5.0.6
- **Tailscale IP:** 100.115.85.120
- **Web UI:** https://10.5.0.6:8007

**Configuration:**
- Container type: Privileged (required for NFS/CIFS)
- Features: nesting=1
- AppArmor: unconfined
- TUN device enabled (for Tailscale)

**Datastore:**
- Name: `synology`
- Path: `/mnt/synology/datastore`
- Backend: Synology NAS via CIFS over Tailscale

**Namespaces:**
- `core.georgsen.dk`
- `pve01.warradejendomme.dk`
- `pve02.warradejendomme.dk`

**Retention Policy:**
- Keep Daily: 7
- Keep Weekly: 4
- Keep Monthly: 3

**Schedule:**
- Prune: 21:00
- GC: 22:30
- core backup: 01:00
- pve01 backup: 01:30
- pve02 backup: 02:00

#### 108: JukeBox

- **Purpose:** Custom music player
- **Source:** https://github.com/mikl0s/JukeBox
- **Port:** 4000

#### 114: Forgejo (Git Server)

- **Purpose:** Self-hosted Git server (GitHub alternative)
- **IP:** 10.5.0.14
- **Port:** 3000
- **URL:** https://git.georgsen.dk
- **Software:** Forgejo 10.0.1
- **Storage:** 60GB

#### 115: Technitium DNS

- **Purpose:** Local DNS server with web UI and API
- **IP:** 10.5.0.2
- **Ports:** 53 (DNS), 5380 (Web UI)
- **URL:** https://dns.georgsen.dk
- **Features:** Authoritative DNS for internal zone, upstream forwarder

#### 1000: Tailscale

- **Purpose:** Tailscale access to vmbr1 and vmbr2 networks
- **IPs:** vmbr1 (DHCP) + 10.9.1.10 (vmbr2)
- **Tailscale IP:** 100.99.33.100

---

## Backup Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│  BACKUP FLOW                                                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  core.georgsen.dk ────────┐                                     │
│  (01:00 daily)            │                                     │
│                           ▼                                     │
│  pve01.warradejendomme.dk ──► PBS (10.5.0.6)                    │
│  (01:30 daily)            │        │                            │
│                           │        │ CIFS over Tailscale        │
│  pve02.warradejendomme.dk ┘        ▼                            │
│  (02:00 daily)                Synology NAS                      │
│                               (100.105.26.130)                  │
│                                                                 │
│  Retention: 7 daily, 4 weekly, 3 monthly                        │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
```

### PBS Container Config (/etc/pve/lxc/106.conf)

```
arch: amd64
cores: 2
features: nesting=1
hostname: pbs
memory: 2048
net0: name=eth0,bridge=vmbr1,gw=10.5.0.254,hwaddr=BC:24:11:BF:BD:74,ip=10.5.0.6/24,type=veth
onboot: 1
ostype: debian
rootfs: local-zfs:subvol-106-disk-0,size=50G
swap: 512
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
lxc.cap.drop:
lxc.apparmor.profile: unconfined
```

### PBS fstab (/etc/fstab on PBS)

```
//100.105.26.130/pbs-backup /mnt/synology cifs credentials=/etc/pbs-smb.creds,uid=34,gid=34,vers=3.0,_netdev,x-systemd.automount,x-systemd.after=tailscaled.service,x-systemd.requires=tailscaled.service 0 0
```

### PBS Systemd Dependencies

```
# /etc/systemd/system/proxmox-backup-proxy.service.d/wait-for-mount.conf
[Unit]
After=mnt-synology.mount tailscaled.service
Requires=mnt-synology.mount

# /etc/systemd/system/proxmox-backup.service.d/wait-for-mount.conf
[Unit]
After=mnt-synology.mount tailscaled.service
Requires=mnt-synology.mount
```

### Synology NFS/CIFS Settings

- Share: `/volume1/pbs-backup`
- Protocol: CIFS (SMB 3.0)
- Squash: Map all users to admin
- Tailscale IP allowed: 100.115.85.120

---

## Other PVE Nodes

### pve01.warradejendomme.dk

- **Purpose:** Friend's server (former job), needs backups
- **Backup namespace:** `pve01.warradejendomme.dk`
- **PBS connection:** Via Tailscale (100.115.85.120:8007)

### pve02.warradejendomme.dk

- **Purpose:** Friend's server (former job), needs backups
- **Backup namespace:** `pve02.warradejendomme.dk`
- **PBS connection:** Via Tailscale (100.115.85.120:8007)

---

## Tailscale Network

| Device | Tailscale IP | Notes |
|--------|--------------|-------|
| pbs-1 (PBS) | 100.115.85.120 | PBS container |
| nas01 (Synology) | 100.105.26.130 | Home NAS, backup target |
| tailscale-pve01 | 100.99.33.100 | core's Tailscale LXC |
| pve01 | 100.99.118.54 | Friend's server |
| pve02 | 100.82.87.108 | Friend's server |
| dev | 100.85.227.17 | |
| sentry | 100.83.236.113 | |
| mge-t14 | 100.112.71.15 | |
| mikflix | 100.84.253.6 | |
| xanderryzen | 100.71.118.78 | |
| nvr01 | 100.118.17.103 | Exit node |
| tailscalemg | 100.115.101.65 | Exit node |

**Tailscale config:** SSH enabled on all devices where possible

---

## DNS

### External DNS
**Provider:** dns.services (Danish, free, has API for future automation)

**Domains:**
- georgsen.dk
- dataloes.dk
- microsux.dk
- warradejendomme.dk

### Internal DNS (Technitium)
- **Server:** 10.5.0.2
- **Web UI:** https://dns.georgsen.dk or http://10.5.0.2:5380
- **API:** http://10.5.0.2:5380/api/
- **Internal zone:** lab.georgsen.dk
- **Upstream:** Cloudflare (1.1.1.1), Google (8.8.8.8), Quad9 (9.9.9.9)

**Internal DNS Records (lab.georgsen.dk):**

| Name | IP |
|------|-----|
| npm | 10.5.0.1 |
| dns | 10.5.0.2 |
| pbs | 10.5.0.6 |
| dockge | 10.5.0.10 |
| forgejo | 10.5.0.14 |
| git | 10.5.0.14 |
| mgmt | 10.5.0.102 |

---

## Home Infrastructure

### Synology NAS (nas01)

- **Tailscale IP:** 100.105.26.130
- **Purpose:** Backup target, general NAS
- **PBS Share:** /volume1/pbs-backup
- **Note:** To be replaced in 2026 (noisy)

### Future: Intel NUC Cluster (3 nodes)

Planned home PVE cluster:

| Node | RAM | NVMe | SATA SSD | HDD |
|------|-----|------|----------|-----|
| Node 1 | 16GB | 500GB | 512GB | - |
| Node 2 | 16GB | 500GB | 512GB | - |
| Node 3 | 16GB | 2TB | 512GB | 5TB (CCTV) |

---

## User Conventions

### Standard User Setup

When creating containers/VMs:

```bash
# Create group and user with UID/GID 1000
groupadd -g 1000 georgsen
useradd -u 1000 -g 1000 -m -s /bin/bash mikkel

# Passwordless sudo
apt install -y sudo
echo "mikkel ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/mikkel
chmod 440 /etc/sudoers.d/mikkel

# SSH key
mkdir -p /root/.ssh /home/mikkel/.ssh
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIOQrK06zVkfY6C1ec69kEZYjf8tC98icCcBju4V751i mikkel@georgsen.dk" | tee /root/.ssh/authorized_keys /home/mikkel/.ssh/authorized_keys
chmod 700 /root/.ssh /home/mikkel/.ssh
chmod 600 /root/.ssh/authorized_keys /home/mikkel/.ssh/authorized_keys
chown -R mikkel:georgsen /home/mikkel/.ssh

# Enable ping in unprivileged containers (required because cap_net_raw is dropped)
# Without this, ping fails with "Operation not permitted"
# Must be re-applied if iputils-ping package is upgraded
setcap cap_net_raw+ep /bin/ping
```

---

## Projects

### Sentry (Defense Intelligence)

Custom system monitoring military contracting news for business opportunities.
- Focus: Ground equipment (NOT aircraft/naval)
- Services: EV conversion, diesel upgrades, specialty vehicles, bomb loaders, Humvee engine upgrades

### JukeBox

Custom music player: https://github.com/mikl0s/JukeBox

### Sense Vendor Locator

CBD retailer locator for Denmark (sense.microsux.dk)

### dataloes.dk

Personal company website

---

## Preferences

- **Coding:** Python, Batch
- **UI Style:** 256-color terminal retro aesthetic
- **Development:** Ask clarifying questions, prefer understanding over workarounds
- **Tools:** Claude Code for development projects
- **Home IP:** 83.89.248.247 (static)

---

## Security

### NPM Access Lists

| ID | Name | IPs | Applied To |
|----|------|-----|------------|
| 1 | home_only | 83.89.248.247 | dns.georgsen.dk, dockge.georgsen.dk, pbs.georgsen.dk |

### Fail2ban

**core.georgsen.dk (PVE host):**
- Config: `/etc/fail2ban/jail.local`
- Jail: sshd
- Max retries: 5
- Ban time: 24 hours
- Whitelisted: 127.0.0.1, 10.5.0.0/24, 83.89.248.247

**Forgejo (VMID 114):**
- Config: `/etc/fail2ban/jail.local`
- Jail: forgejo
- Max retries: 5
- Ban time: 24 hours
- Log: `/var/lib/forgejo/log/forgejo.log`

---

## Maintenance Notes

### Cleanup Tasks

1. **Remove orphaned iptables rules** (Coolify is gone):
   ```bash
   iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 6001 -j DNAT --to-destination 10.5.0.1:6001
   iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 6002 -j DNAT --to-destination 10.5.0.1:6002
   netfilter-persistent save
   ```

2. **Containers to evaluate:**
   - 110 (sense.microsux.dk) - Consider consolidating
   - 112 (dataloes) - Stopped
   - 113 (general) - Decomissioned, can remove

3. **DHCP vs Static IPs:**
   - Containers .112 and .113 have static IPs inside DHCP range (100-200)
   - Consider moving static assignments to .2-.99 range

---

## Quick Reference

### Service URLs

| Service | URL |
|---------|-----|
| PVE (core) | https://65.108.14.165:8006 |
| PBS | https://pbs.georgsen.dk or https://10.5.0.6:8007 |
| NPM Admin | http://10.5.0.1:81 |
| Dockge | https://dockge.georgsen.dk |
| DNS Admin | https://dns.georgsen.dk or http://10.5.0.2:5380 |
| Forgejo | https://git.georgsen.dk or http://10.5.0.14:3000 |
| Status | https://status.georgsen.dk |
| Webmail | https://webmail.georgsen.dk |
| JukeBox | https://jukebox.georgsen.dk |

### Important IPs

| Resource | IP |
|----------|-----|
| core public | 65.108.14.165 |
| mail public | 65.108.14.164 |
| NPM | 10.5.0.1 |
| DNS | 10.5.0.2 |
| PBS | 10.5.0.6 |
| Dockge | 10.5.0.10 |
| Forgejo | 10.5.0.14 |
| Synology (Tailscale) | 100.105.26.130 |
| PBS (Tailscale) | 100.115.85.120 |

### PBS Credentials

- **Web UI:** root@pam
- **API Token:** root@pam!pve
- **Fingerprint:** f6:5d:3f:63:0e:15:38:10:38:7a:33:82:3a:c0:b0:59:98:a7:04:7d:8d:ed:64:7d:fc:85:ab:8f:87:a4:48:eb

### PVE API (from mgmt container)

- **Token:** root@pam!mgmt
- **Config:** ~/homelab/pve/credentials (symlinked to ~/.config/pve)
- **Helper:** ~/bin/pve (list, status, start, stop, create-ct)

### NPM API (from mgmt container)

- **Config:** ~/homelab/npm/npm-api.conf (symlinked)
- **Helper:** ~/bin/npm-api (--host-list, --host-create, --host-delete, --cert-list)

### DNS API (from mgmt container)

- **Config:** ~/homelab/dns/credentials (symlinked to ~/.config/dns)
- **Helper:** ~/bin/dns (list, records, add, delete, lookup)
- **Default zone:** lab.georgsen.dk

### Forgejo API (from mgmt container)

- **Config:** ~/homelab/forgejo/credentials (symlinked to ~/.config/forgejo)
- **Repo:** git@10.5.0.14:mikkel/homelab.git

---

*Last updated: 2026-01-14*