https://blog.gene.nz/Recent content on Gene Marikhin - Tech BlogHugoen-USgene@marikhin.com (Gene Marikhin)gene@marikhin.com (Gene Marikhin)Mon, 28 Sep 2026 00:00:00 +0000https://blog.gene.nz/posts/vso_vault/Mon, 28 Sep 2026 00:00:00 +0000gene@marikhin.com (Gene Marikhin)https://blog.gene.nz/posts/vso_vault/<h1 id="how-a-vault-secret-becomes-a-kubernetes-secret">How a Vault secret becomes a Kubernetes Secret</h1> <p><em>A homelab deep dive. Bare-metal Kubernetes v1.36.4, Vault running inside the same cluster via the official Helm chart, and the Vault Secrets Operator (VSO) syncing secrets for GitLab runners.</em></p> <p>Right after I upgraded my homelab cluster to Kubernetes v1.36.4, every <code>VaultStaticSecret</code> started logging the same error, over a thousand times:</p> <pre tabindex="0"><code>Failed to get Vault auth login: Error making API request. URL: PUT http://vault.hvault.svc.cluster.local:8200/v1/auth/kubernetes/login Code: 403. Errors: * permission denied </code></pre><p>That message tells you almost nothing. To debug it you need a clear picture of what happens between &ldquo;VSO wants a secret&rdquo; and &ldquo;a Kubernetes <code>Secret</code> exists in etcd&rdquo;. This post walks through that chain.</p>https://blog.gene.nz/about/Mon, 01 Jan 0001 00:00:00 +0000gene@marikhin.com (Gene Marikhin)https://blog.gene.nz/about/<figure><img src="https://blog.gene.nz/img/gene.jpg" alt="Gene"> </figure> <p>Hi, I&rsquo;m Gene, a DevOps engineer focused on Kubernetes, CI/CD pipelines and infrastructure as code. I like automating the boring parts, keeping systems observable, and making deployments uneventful.</p> <p>Currently working with GitLab, Vault, Helm and Kubernetes.</p> <p>Based in Auckland, New Zealand</p>