Level up Security Management with HashiCorp Vault and Flux
HashiCorp Vault is now a native Flux extension enabling secret management for all environments including on premise and hybrid. Secure GitOps automated pipelines with rotating encryption keys in HashiCorp Vault, re-encryption of secrets and audit any changes in git.
Flux’s Native Support for HashiCorp Vault
If you love HashiCorp Vault, then you know its benefits and popularity as a way to secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.
If you love the CNCF Flux project for its enterprise-grade and secure GitOps capabilities, you'll be excited to know that Flux provides native support for Hashicorp Vault token-based authentication when decrypting SOPS encrypted secrets!
How Does this Help you?
Secure and automated GitOps
HashiCorp Vault is one of the most popular places to store your secrets. Being able to use it with Flux means that you can set up a fully automated GitOps workflow that uses secrets and doesn't compromise the tenets of secure GitOps. You can store your secrets with a secure enterprise solution instead as a Kubernetes secret.
Native decryption for on-premise and hybrid cloud
Another big benefit here is that you can natively decrypt with Flux on-premises or when you have an hybrid cloud setup. This is particularly important because you likely are doing this because you want to rely on HashiCorp Vault instead of on a cloud KMS.
What’s Going on Under the Hood?
Flux is able to work with your encrypted secrets securely using Mozilla SOPS and HashiCorp Vault. When you make changes to the manifests in your Git repo, which is the single source of truth in GitOps workflows, Flux will decrypt your secrets and apply changes to the cluster accordingly.
This is important as you can rotate your encryption keys in HashiCorp Vault, re-encrypt your secret, and reflect this change in git. Rotating keys increases security and lowers the management overhead when it's done with Vault.
Flux natively supports HashiCorp Vault. When you make changes to your manifests or secrets, you can automatically deploy your apps to the state that you declare automatically. Because of GitOps, Flux will continue to keep the cluster and your apps updated based on your definitions and secrets. Advanced GitOps practitioners experience these core benefits through Flux: automation, along with security, velocity, reliability, reduction in manual work, and the potential of mistakes.
Flux relies on SOPS to encrypt secret data. SOPS uses an envelope encryption mechanism to create a data key which will be used to encrypt your secret. You can use a Key group to make sure that any key in the group can be used to encrypt/decrypt the data key. SOPS also provides a Key Rotation command to rotate your data key and re-encrypt the secret. You then have to push the encrypted secret to Git. From there, Flux steps in to decrypt your secret and says "hey this secret has not changed"! Flux has a microservices architecture of various controllers and the kustomize-controller does this work.
To summarize, this setup gives you the ability to decrypt on-premises or with a hybrid cloud setup using HashiCorp Vault and Flux.
To get started, here’s an overview of the kustomize-controller that does this work for Vault.
In order to store secrets safely in a public or private Git repository, you can use Mozilla SOPS and encrypt your Kubernetes Secrets data with age and OpenPGP keys, or using provider implementations such as Hashicorp Vault.
To configure what keys must be used for decryption, a .decryption.secretRef can be specified with a reference to a Secret in the same namespace as the Kustomization.
--- apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 kind: Kustomization metadata: name: sops-encrypted namespace: default spec: interval: 5m path: "./" sourceRef: kind: GitRepository name: repository-with-secrets decryption: provider: sops secretRef: name: sops-keys
The Secret’s .data section is expected to contain entries with decryption keys (for age and OpenPGP), or credentials (for any of the supported provider implementations). The controller identifies the type of the entry by the suffix of the key (e.g. .agekey), or a fixed key (e.g. sops.vault-token).
--- apiVersion: v1 kind: Secret metadata: name: sops-keys namespace: default data: # Exemplary age private key identity.agekey: <BASE64> # Exemplary Hashicorp Vault token sops.vault-token: <BASE64>
Hashicorp Vault Secret Entry
To specify credentials for Hashicorp Vault in a Kubernetes Secret, append a .data entry with a fixed sops.vault-token key and the token as value.
--- apiVersion: v1 kind: Secret metadata: name: sops-keys namespace: default data: # Exemplary Hashicorp Vault Secret token sops.vault-token: <BASE64>
To configure a global default for Hashicorp Vault, patch the controller’s Deployment with a VAULT_TOKEN environment variable.
--- apiVersion: apps/v1 kind: Deployment metadata: name: kustomize-controller namespace: flux-system spec: template: spec: containers: - name: manager env: - name: VAULT_TOKEN value: <token
For the most updated version of these instructions, check
Join us on slack if you need help!