Networking
Cilium was selected over Flannel and Calico because it matches the networking model I am building experience with professionally. The cluster passed the Cilium connectivity validation suite after the Helm installation was corrected.
CLOUD-NATIVE PLATFORM CASE STUDY
Kubernetes Platform and Gateway Engineering
I built a three-node Kubernetes cluster to move beyond isolated container exercises and operate a small platform with real networking, GitOps, persistent storage, monitoring, and application routing. The project has changed direction several times as I learned what each layer actually does. That decision history is part of the work, not something I want to hide.
CURRENT PLATFORM
The cluster now has a working operating model, not just three joined nodes.
The current platform runs Kubernetes across one control-plane node and two workers. Cilium handles cluster networking. Argo CD reconciles the GitOps repository. Helm is the default installation method for platform services, and each service keeps a full commented defaults file beside the values I actually change.
flowchart LR
GIT[GitHub GitOps Repository] --> ARGO[Argo CD]
ARGO --> CILIUM[Cilium Networking]
ARGO --> STORAGE[Local Path Provisioner]
ARGO --> KUMA[Uptime Kuma]
KUMA --> PVC[Persistent Volume]
KUMA --> DISCORD[Discord Alerts]
FUTURE[Envoy Gateway] --> ROUTES[Gateway API Routes]
ROUTES --> KUMA
GitOps
Argo CD now manages namespaces, storage, and applications through an app-of-apps pattern. The dashboard shows the root application and child applications as synchronized and healthy.
Storage
Local Path Provisioner supplies working dynamic storage for the lab. I accepted that it is node-local and not resilient because the immediate requirement was functional persistence, not distributed storage.
Monitoring
Uptime Kuma runs with a bound persistent volume and sends successful alert tests into the lab Discord. It is the first application used to prove the GitOps, storage, and alerting path end to end.
BUILD TIMELINE
The platform grew in layers, and each layer exposed the next dependency.
01. Cluster bootstrap
Configured three Ubuntu 24.04 virtual machines, containerd, kubeadm,
kubelet, kubectl, host resolution, and node joins.
02. Cilium networking
Evaluated Flannel, Calico, and Cilium. Installed Cilium with Helm,
corrected the deployment, and completed the connectivity tests.
03. Firewall path
Moved cluster traffic behind OPNsense, then worked through outbound NAT,
allow rules, and DNS behavior until the nodes had reliable external access.
04. Argo CD bootstrap
Created the GitOps repository, root application, namespace application,
and the first infrastructure applications.
05. Persistent storage
Added Local Path Provisioner after the first stateful workload showed that
the cluster had no default StorageClass.
06. First application
Deployed Uptime Kuma, bound its PVC, initialized SQLite, and connected
alerting to Discord.
07. Gateway direction
Evaluated Traefik, Istio, and kGateway. Selected Envoy Gateway for the next
implementation phase because it aligns with Gateway API and the platform
direction I am supporting professionally.
WHAT BROKE
The useful parts of this project came from the failures, not the happy path.
No default StorageClass
When I deployed Uptime Kuma, its PVC could not bind because I had never added dynamic storage. That forced the storage decision at the correct time. I added Local Path Provisioner and documented the resilience tradeoff.
Git repository authentication
The private repository path initially failed because the SSH agent was expected but SSH_AUTH_SOCK was not set. I standardized the repository URL and corrected the authentication path instead of working around it manually.
Discord alert configuration
The first alert test failed because thread mode expected a thread ID. The next attempt returned an invalid webhook token error. Recreating the webhook and using the normal channel mode resolved the path.
Gateway session behavior
An earlier gateway test could reach the Proxmox management page but the console path failed because sessions moved between backend nodes. The available persistence option depended on experimental behavior, so I removed that implementation instead of forcing it into the design.
DECISION RECORD
I am using Envoy Gateway next, but I am not adding a load balancer yet.
I initially leaned toward Istio because it provides a broad service-mesh path. Envoy Gateway aligned better with the platform direction I am developing professionally. For this lab, direct professional relevance is a better reason than installing a broader stack just because it has more features.
The first Envoy Gateway implementation will use a fixed NodePort. That lets me validate the GatewayClass, Gateway, HTTPRoute, and backend path without debugging a gateway and a bare-metal load balancer at the same time. A stable LAN VIP through MetalLB or Cilium load-balancer features can be added after the routing layer is proven.
Cilium
Pod networking, service networking, network policy, and eBPF visibility.
Envoy Gateway
Gateway API control plane, north-south routing, TLS, and traffic policy.
Argo CD
Deployment state, reconciliation, and change history through Git.
Load balancer
Deferred until a stable VIP and clean DNS path are operational requirements.
VALIDATION
I treat a component as working only after the complete path is visible.
Cluster validation
Verified the control plane and both workers were Ready, then ran the Cilium connectivity tests instead of treating successful installation output as proof of working networking.
GitOps validation
Confirmed the root and child applications were synchronized and healthy, and checked that the repository, chart, path, namespace, and destination matched the intended state.
Storage validation
Confirmed the local-path StorageClass was default, the provisioner pod was running, and the Uptime Kuma claim was Bound with the expected size and access mode.
Alert validation
Sent a real test notification to the lab Discord and retained the successful result as evidence that the monitoring path worked.
WHAT I WOULD DO DIFFERENTLY
I would establish a few platform dependencies earlier, but I would keep the same learning sequence.
I would create the StorageClass before the first stateful application and define the repository authentication path before bootstrapping Argo CD. Those changes would remove avoidable friction. I would still build the cluster, CNI, GitOps, storage, and application layers separately because isolating each layer made failures easier to reason through.
I would also decide earlier whether the objective was ingress, Gateway API, or a full service mesh. Testing several products was useful, but the project became clearer once the requirement was stated as a work-aligned Gateway API implementation rather than a general search for the most capable tool.
RELATED ENGINEERING JOURNAL
The command history and screenshots are documented separately.
The project page captures the architecture and decisions. The journal entries preserve the hands-on bootstrap work, Cilium decision, GitOps structure, storage failure, application deployment, and Discord alert troubleshooting.
NEXT PHASE
The next milestone is one clean application route through Envoy Gateway.
Implementation sequence
- Confirm Envoy Gateway compatibility with the cluster version.
- Install Envoy Gateway through Helm and Argo CD.
- Expose the managed proxy through a fixed NodePort.
- Create one Gateway and one HTTPRoute for Uptime Kuma.
- Validate the complete request path before adding TLS.
- Add a stable VIP only when DNS and cleaner LAN access are needed.
Current project summary
Built and operated a three-node Kubernetes platform using Cilium, Helm, Argo CD, dynamic storage, persistent applications, Discord alerting, and Git-driven configuration. Evaluated multiple gateway approaches, removed implementations that did not meet the requirement cleanly, and selected Envoy Gateway for the next work-aligned Gateway API phase.