KO
|
EN
gitlite — search
Search
#javascript
#java
#python
#php
#bioinformatics
#nodejs
#chrome-extension
#c-plus-plus
#hacktoberfest
#api
#rest
#go
puppet-vault_lookup
★ 29
Open GitHub ↗
No description available.
Download README (.md)
Explore Similar Repositories
puppet-snmp
:
Puppet module to manage Net-SNMP.
puppet-hiera
:
Hiera hierarchy module for templating `hiera.yaml`
puppet-fail2ban
:
This module installs, configures and manages the Fail2ban service.
puppet-network
:
Puppet module to manage networking on Suse, Debian and RedHat.
puppet-kafka
:
The kafka module for managing the installation and configuration of Apache Kafka
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
puppet-vault_lookup
?
Download (.md)
# vault_lookup [](https://github.com/voxpupuli/puppet-vault_lookup/actions?query=workflow%3ACI) [](https://github.com/voxpupuli/puppet-vault_lookup/actions/workflows/release.yml) [](https://forge.puppetlabs.com/puppet/vault_lookup) [](https://forge.puppetlabs.com/puppet/vault_lookup) [](https://forge.puppetlabs.com/puppet/vault_lookup) [](https://forge.puppetlabs.com/puppet/vault_lookup) [](https://www.puppetmodule.info/m/puppet-vault_lookup) [](https://raw.githubusercontent.com/voxpupuli/puppet-vault_lookup/master/LICENSE) Module to integrate Puppet agents with Hashicorp Vault. #### Table of Contents 1. [Description](#description) 2. [Requirements](#setup) 3. [Usage, Configuration, and Examples](#usage) 4. [Authentication Methods](#authentication-methods) ## Description For Puppet 6+ or Puppet Enterprise 2019+ users wanting to use secrets from [Hashicorp Vault](https://www.vaultproject.io/) on their Puppet agents, this Puppet module provides the `vault_lookup::lookup()` function. When used with Puppet 6's [`Deferred` type](https://puppet.com/docs/puppet/7/deferring_functions.html), the function allows agents to retrieve secrets from Vault when a catalog is applied rather than compiled. In this way, the secret data is not embedded in the catalog and the Puppetserver does not need permissions to read all your Vault secrets. ## Requirements This modules assumes the following: 1. Puppet (for supported versions, check metadata.json) 2. An existing [Vault](https://www.vaultproject.io/) infrastructure The `vault_lookup::lookup()` function is expected to be run with the `Deferred` type; as such, Puppet 6 or later is required. And as this function is meant to read secrets from Vault, an existing Vault infrastructure is assumed to be up and reachable by your Puppet agents. ## Usage Install this module as you would in any other; the necessary code will be distributed to Puppet agents via pluginsync. In your manifests, call the `vault_lookup::lookup()` function using the Deferred type. For example: ```puppet $d = Deferred('vault_lookup::lookup', ["secret/test", 'https://vault.hostname:8200']) node default { notify { example : message => $d } } ``` The lookup function will be run on the agent and the value of `$d` will be resolved when the catalog is applied. This will make a call to `https://vault.hostname:8200/v1/secret/test` and wrap the result in Puppet's `Sensitive` type, which prevents the value from being logged. You can also choose not to specify the Vault URL, and then Puppet will use the `VAULT_ADDR` environment variable. This will be either set on the command line, or set in the service config file for Puppet, on Debian `/etc/default/puppet`, on RedHat `/etc/sysconfig/puppet`: ``` $d = Deferred('vault_lookup::lookup', ["secret/test"]) node default { notify { example : message => $d } } ``` ### Configuring the Vault lookup The lookup done by `vault_lookup::lookup()` can be configured in three ways: positional arguments, a hash of options, and/or environment variables. In all cases, the path to the secret is the first positional argument and is required. All other arguments are optional. Arguments in `[square brackets]` below are optional. #### Positional Arguments ``` vault_lookup::lookup( <path>, [<vault_addr>], [<cert_path_segment>], [<cert_role>], [<namespace>], [<auth_namespace>], [<field>], [<auth_method>], [<role_id>], [<secret_id>], [<approle_path_segment>], [<agent_sink_file>] ) ``` #### Usage in multi-tenant Vault setup Some Vault setups allows to obtain the authentication token in a defined namespace and to use it for secret retrieval in other namespaces. In this case you have to provide the namespace for authentication as an additional option: ``` $vault_conf = { 'approle_path_segment' => 'v1/auth/approle/', 'namespace' => 'customer_1', 'auth_namespace' => 'provider_1', } ``` #### Options Hash ``` vault_lookup::lookup( <path>, [<options_hash>] ) ``` #### Environment Variables Not all options can be set with environment variables. Use the table below to find the matching env var, if available. Also note that environment variables are only used if the option is not supplied to the function. | Option Name | Environment Variable | | ----------- | -------------------- | | `vault_addr` | `VAULT_ADDR` | | `cert_path_segment` | ---- | | `cert_role` | ---- | | `namespace` | `VAULT_NAMESPACE` | | `field` | ---- | | `auth_method` | `VAULT_AUTH_METHOD` | | `role_id` | `VAULT_ROLE_ID` | | `secret_id` | `VAULT_SECRET_ID` | | `approle_path_segment` | ---- | | `agent_sink_file` | `VAULT_AGENT_SINK_FILE` | #### Environment variables on Puppet server When not using deferred function, the compiler will execute the function and needs either the specific settings or can make use of environment variables. Please note that environment variables must be set in `/etc/puppetlabs/puppetserver/conf.d/puppet-server.conf` within the `jruby-puppet` section. Please find a simple example on how to add environment variables to the conf file using hocon: ```shell /opt/puppetlabs/puppet/lib/ruby/vendor_gems/bin/hocon \ -f /etc/puppetlabs/puppetserver/conf.d/pe-puppet-server.conf \ set jruby-puppet.environment-vars "{\ VAULT_ADDR: \"https://FQDN\",\ VAULT_ROLE_ID: 12345678-1234-1234-1234-1234567890ab,\ VAULT_SECRET_ID: 12345678-1234-1234-1234-1234567890ab\ }" ``` ### Usage Examples Here are some examples of each method: ```puppet # Positional arguments ## Using the default 'cert' auth method. $data_1a = vault_lookup::lookup('secret/db/password', 'https://vault.corp.net:8200') ## Using the 'approle' auth method. $data_2a = vault_lookup::lookup('secret/db/blah', 'https://vault.corp.net:8200', undef, undef, undef, undef, 'approle', 'team_a', 'abcd1234!@#') ## Pulling out a specific field. $password = vault_lookup::lookup('secret/test', 'http://vault.corp.net:8200', undef, undef, undef, 'password') ``` ```puppet # Options hash ## Using the default 'cert' auth method. $data_1b = vault_lookup::lookup('secret/db/password', { 'vault_addr' => 'https://vault.corp.net:8200' }) ## Using the 'approle' auth method. $data_2b = vault_lookup::lookup('secret/db/blah', { 'vault_addr' => 'https://vault.corp.net:8200', 'auth_method' => 'approle', 'role_id' => 'team_a', 'secret_id' => 'abcd1234!@#', }) # Using 'field' to pull out a specific field from the data. $password = vault_lookup::lookup('secret/test', {'vault_addr' => 'http://127.0.0.1:8200', 'field' => 'password'}) # Using Deferred is simpler with the options hash. $password_deferred = Deferred('vault_lookup::lookup', ["secret/test", { vault_addr => 'http://127.0.0.1:8200', field => 'password', }]) ``` ### A note about caching The `vault_lookup::lookup()` function caches the result of a lookup and will use that cached result for the life of the catalog application (when using `Deferred`) or catalog compilation (when not using `Deferred`). Looked up values are cached based on a combination of their: * Path in the Vault URI * Vault Address * Namespace * Field This means that you can call `vault_lookup::lookup()` multiple times for the same piece of data or refer to the same `Deferred` value multiple times and there will only be a single fetch from Vault. This helps to reduce the amount of back-and-forth network traffic to your Vault cluster. For example, in the code below, due to caching, the `secret/db/password` value is only looked up once even though the function is called twice: ```puppet # Wrap the function in Deferred, and save it to a variable. # # Since the path, vault_addr, and namespace don't change, only one Vault lookup # will be made regardless of how many times the $db_password variable is used. # $db_password = Deferred('vault_lookup::lookup', [ 'secret/db/password', {'vault_addr' => 'https://vault.corp.net:8200'}, ]) # Call the deferred function once. file { '/etc/db.conf': ensure => file, content => $db_password, } # Call the deferred function twice. notify { 'show the DB password': message => $db_password, } ``` But if the path, the Vault address, or the namespace change, a new lookup to Vault will happen. For example, in the code below, even though the path is the same in both of these lookups (`secret/db/password`), the namespace is different, so a separate lookup will be made rather than the cached value from the first lookup of `secret/db/password` being used. ```puppet # Fetch a value from Vault without using a namespace. $db_password = Deferred('vault_lookup::lookup', [ 'secret/db/password', {'vault_addr' => 'https://vault.corp.net:8200'}, ]) # Fetch a value from Vault in the 'dev' namespace. $db_password_namespaced = Deferred('vault_lookup::lookup', [ 'secret/db/password', {'vault_addr' => 'https://vault.corp.net:8200', 'namespace' => 'dev'}, ]) file { '/etc/db.conf': ensure => file, content => $db_password, } notify { 'show the dev namespace DB password': message => $db_password_namespaced, } ``` ### A note about spec testing Puppet code that uses this function When spec testing Puppet code that uses the `vault_lookup::lookup()` function, you'll probably want to stub the function so that it doesn't hit your real Vault servers. Below is an example of how to do that. This is particularly useful when spec testing a class or define that defers the function call. Two things that will need to be configured are 1) a `require_relative` of the internal puppet_x module. Note that this assumes you're pulling the module down into a `spec/fixtures/` directory. ```ruby require_relative '../fixtures/modules/vault_lookup/lib/puppet_x/vault_lookup/lookup' ``` and 2) a stub on the `PuppetX::VaultLookup::Lookup` class's `:lookup` method: ```ruby before(:each) do allow(PuppetX::VaultLookup::Lookup).to receive(:lookup) .and_return(Puppet::Pops::Types::PSensitiveType::Sensitive.new('hello world')) end ``` Here's a complete example: ```ruby require 'spec_helper' require_relative '../fixtures/modules/vault_lookup/lib/puppet_x/vault_lookup/lookup' describe 'some::class' do on_supported_os.each do |os, os_facts| context "on #{os}" do let(:facts) { os_facts } before(:each) do allow(PuppetX::VaultLookup::Lookup).to receive(:lookup) .and_return(sensitive('hello world')) end context 'with all defaults' do it { is_expected.to compile } it { is_expected.to contain_file('/etc/credentials.txt').with_content(sensitive('hello world')) } end end end end ``` ## Authentication Methods The `vault_lookup::lookup()` function can authenticate to Vault in a number of ways. This table shows the currently supported `auth_method` types: | `auth_method` | Description | | --- | --- | | `cert` | (this is the default) Uses the Puppet agent's certificate via the [TLS Certificates](https://developer.hashicorp.com/vault/docs/auth/cert) auth method. | | `approle` | Uses the [AppRole](https://developer.hashicorp.com/vault/docs/auth/approle) auth method. | | `agent` | Uses a local Vault Agent's [auto-auth token](https://developer.hashicorp.com/vault/docs/agent/caching#using-auto-auth-token) and caching proxy. | | `agent_sink` | Uses a local Vault Agent's [auto-auth file sink](https://developer.hashicorp.com/vault/docs/agent/autoauth/sinks/file). | ### Puppetserver CA and agent certificates The `vault_lookup::lookup()` function by default will use the Puppet agent's certificates to authenticate to the Vault server. This means that before any agents contact a Vault server, you must configure the Vault server with the Puppet Server's CA certificate, and Vault must be part of the same certificate infrastructure. 1. Set up Vault using Puppet certs (if not already set up this way). If the Vault host has a Puppet agent on it then you can just its existing host certificates. Otherwise generate a new certificate with `puppetserver ca` and copy the files. ``` $ puppetserver ca generate --certname my-vault.my-domain.me ``` In the Vault listener configuration, set `tls_client_ca_file` as the Puppet CA cert, `tls_cert_file` as the agent's or generated certificate, and `tls_key_file` as the agent's or generated private key. 2. Enable the `cert` auth backend in Vault. ``` $ vault auth enable cert ``` 3. Upload the Puppet Server CA certificate to Vault. After `cert` auth has been enabled for Vault, upload the CA certificate from your Puppet Server to Vault, and add it as a trusted certificate. ``` $ vault write auth/cert/certs/puppetserver \ display_name=puppet \ policies=prod,test \ certificate=@/path/to/puppetserver/ca.pem \ ttl=3600 ``` Once the certificate has been uploaded, any Puppet agent with a signed certificate will be able to authenticate with Vault. ### AppRole `vault:vault_lookup()` can also use AppRole authentication to authenticate against Vault with a valid `role_id` and `secret_id`. See [The Approle Vault Documentation](https://www.vaultproject.io/docs/auth/approle) for detailed explanations of creating and obtaining the security credentials. You will need the Role ID (non sensitive) and the Secret ID (sensitive!). The Secret ID can be provided as an argument to the `vault:vault_lookup()` function but it is recommended to pass this as an environment variable and not bake this into code. Example: ``` $ vault read auth/approle/role/puppet/role-id Key Value --- ----- role_id XXXXX-XXXX-XXX-XX-XXXXXXXXXX ``` ``` # vault write -f auth/approle/role/puppet/secret-id Key Value --- ----- secret_id YYYYY-YYYY-YYY-YY-YYYYYYYYYYY secret_id_accessor ZZZZZ-ZZZZZZ-ZZZZZZ-ZZZZZZZZ-ZZZZ secret_id_ttl 0s ``` In order to use the AppRole auth method, either set the `VAULT_AUTH_METHOD` environment variable on the Puppet process to `approle` or set the `auth_method` option to `approle` when calling the function: ```shell export VAULT_AUTH_METHOD=approle export VAULT_ROLE_ID=XXXXX-XXXX-XXX-XX-XXXXXXXXXX export VAULT_SECRET_ID=YYYYY-YYYY-YYY-YY-YYYYYYYYYYY ``` ### Vault Agent: auto-auth token This method of authentication relies on a local Vault Agent running on the Puppet agent host. The Vault Agent handles authenticating to your Vault server, and the `vault_lookup::lookup()` function just needs to make requests through the local Vault Agent's caching proxy. The Vault Agent in this scenario must be using Auto Auth, have Caching enabled, and have `use_auto_auth_token` set to `true`. <https://developer.hashicorp.com/vault/docs/agent/caching#using-auto-auth-token> An example Vault Agent config for this scenario is shown below: ```hcl vault { address = "https://vault.corp.net:8200" } listener "tcp" { address = "127.0.0.1:8100" tls_disable = true } auto_auth { # Some type of auto_auth configuration from: # https://developer.hashicorp.com/vault/docs/agent/autoauth } cache { use_auto_auth_token = true } ``` And here's how the `vault_lookup::lookup()` function can be used to talk to the local Vault agent and use its token for authentication: ```puppet # Talk to the local Vault Agent that has "use_auto_auth_token = true" $data = Deferred('vault_lookup::lookup', ["secret/test", { vault_addr => 'http://127.0.0.1:8200', auth_method => 'agent', field => 'password', }]) file { '/tmp/secret_data.txt': ensure => file, owner => 'app', group => 'app', mode => '0440', content => $data, } ``` A benefit of this method is that is uses the Vault Agent's cached token rather than generating a new token for each call of the function. This reduces the load on your Vault servers as token generation can be an expensive operation. ### Vault Agent: auto-auth file sink This method of authentication relies on a local Vault Agent running on the Puppet agent host. The Vault Agent handles authenticating to your Vault server, and the `vault_lookup::lookup()` function reads the cached token from a sink file managed by the Vault Agent. Optionally, the lookup could also talk through your Vault Agent's caching proxy if enabled. The Vault Agent in this scenario must be using Auto Auth and an **unencrypted, non-response-wrapped** File Sink for the token. <https://developer.hashicorp.com/vault/docs/agent/autoauth/sinks/file> An example Vault Agent config for this scenario is shown below: ```hcl vault { address = "https://vault.corp.net:8200" } # The listener is optional here, but could be used for the 'vault_addr' in # the vault_lookup::lookup() Puppet function. listener "tcp" { address = "127.0.0.1:8100" tls_disable = true } auto_auth { # Some type of auto_auth method from: # https://developer.hashicorp.com/vault/docs/agent/autoauth/methods method { } sink { type = "file" config = { path = "/path/to/vault-token } } } ``` And here's how the `vault_lookup::lookup()` function can be configured to use the token from the auto-auth file sink for authentication: ```puppet # Use the token from the local Vault Agent's auto-auth file sink. $data = Deferred('vault_lookup::lookup', ["secret/test", { # This doesn't have to be the local Vault agent's proxy, but using it can # provide additional caching. vault_addr => 'http://127.0.0.1:8200', auth_method => 'agent_sink', agent_sink_file => '/path/to/vault-token', field => 'password', }]) file { '/tmp/secret_data.txt': ensure => file, owner => 'app', group => 'app', mode => '0440', content => $data, } ``` A benefit of this method is that is uses the Vault Agent's cached token rather than generating a new token for each call of the function. This reduces the load on your Vault servers as token generation can be an expensive operation.
