431 lines
11 KiB
Markdown
431 lines
11 KiB
Markdown
BlackBox
|
|
========
|
|
|
|
Safely store secrets in Git/Hg for use by Puppet.
|
|
|
|
|
|
|
|
|
|
Overview
|
|
========
|
|
|
|
The goal is to have secret bits (passwords, private keys, and such) in your VCS repo but encrypted so that
|
|
it is safe. On the puppet masters they sit on disk unencrypted but only readable by Puppet Master.
|
|
|
|
How does this work?
|
|
===================
|
|
|
|
**Private keys (and anything that is the entire file):**
|
|
|
|
Files are kept in git/hg encrypted (foo.txt is stored as foo.txt.gpg).
|
|
|
|
After deploying an update to your Puppet Master, the master runs a script that decrypts them. The sit unencrypted on the master, which should already be locked down.
|
|
|
|
**Passwords (and any short string):**
|
|
Passwords are kept in hieradata/blackbox.yaml.gpg, which is decrypted to become hieradata/blackbox.yaml. This data can be read by hiera. This file is encrypted/decrypted just like any other blackbox file.
|
|
|
|
**Key management:**
|
|
The Puppet Masters have GPG keys with no passphrase so that they can decrypt the file unattended. That means having root access on a puppet master gives you the ability to find out all our secrets. That's ok because if you have root access to the puppet master, you own the world anyway.
|
|
|
|
The secret files are encrypted such that any one key on a list of keys can decrypt them. That is, when encrypting it is is "encrypted for multiple users". Each person that should have acecss to the secrets should have a key and be on the key list. There should also be a key for account that deploys new code to the Puppet master.
|
|
|
|
What does this look like to the typical sysadmin?
|
|
================================
|
|
|
|
* If you need to, start the GPG Agent:
|
|
|
|
``eval $(gpg-agent --daemon)``
|
|
|
|
* Decrypt the file so it is editable:
|
|
|
|
``bin/blackbox_edit_start.sh FILENAME``
|
|
|
|
(You will need to enter your GPG passphrase.)
|
|
|
|
* Edit FILENAME as you desire.
|
|
|
|
``vim FILENAME``
|
|
|
|
* Re-encrypt the file:
|
|
|
|
``bin/blackbox_edit_end.sh FILENAME``
|
|
|
|
|
|
* Commit the changes.
|
|
|
|
```
|
|
git commit -a
|
|
# or
|
|
hg commit
|
|
```
|
|
|
|
|
|
This content is released under the MIT License. See the LICENSE.txt file.
|
|
|
|
How to use the secrets with Puppet?
|
|
================================
|
|
|
|
### Small strings:
|
|
|
|
Small strings, such as passwords and API keys, are stored in a hiera yaml file. You can access them using the hiera() function.
|
|
|
|
Puppet example for a single password:
|
|
|
|
```
|
|
$the_password = hiera('module::test_password', 'fail')
|
|
file {'/tmp/debug-blackbox.txt':
|
|
content => $the_password,
|
|
owner => 'root',
|
|
group => 'root',
|
|
mode => '0600',
|
|
}
|
|
```
|
|
|
|
### Entire files:
|
|
|
|
Entire files, such as SSL certs and private keys, are treated just like files.
|
|
|
|
Puppet example for an encrypted file:
|
|
|
|
```
|
|
file { '/etc/my_little_secret.key':
|
|
ensure => 'file',
|
|
owner => 'root',
|
|
group => 'puppet',
|
|
mode => '0760',
|
|
source => "puppet:///modules/${module_name}/secret_file.key",
|
|
}
|
|
```
|
|
|
|
|
|
How to enroll a new file into the system?
|
|
============================
|
|
|
|
* If you need to, start the GPG Agent:
|
|
|
|
``eval $(gpg-agent --daemon)``
|
|
|
|
* Add the file to the system:
|
|
|
|
```
|
|
bin/blackbox_register_new_file.sh path/to/file.name.key
|
|
```
|
|
|
|
How to indoctrinate a new user into the system?
|
|
============================
|
|
|
|
``keyrings/live/blackbox-admins.txt`` is a file that
|
|
lists which users are able to decrypt files.
|
|
(More pedantically, it is a list of the GnuPG key
|
|
names that the file is encrypted for.)
|
|
|
|
To join the list of people that can edit the file requires three steps; You create a GPG key and add it to the key ring. Then, someone that already has access adds you to the system. Lastly, you should test your access.
|
|
|
|
### Step 1: YOU create a GPG key pair on a secure machine and add to public keychain.
|
|
|
|
|
|
```
|
|
KEYNAME=$USER@$DOMAINNAME
|
|
# For example...
|
|
KEYNAME=myusername@example.com
|
|
gpg --gen-key
|
|
```
|
|
|
|
Pick defaults for encryption settings, 0 expiration. Pick a VERY GOOD passphrase.
|
|
|
|
When GPG is generating entropy, consider running this on the machine in another window:
|
|
|
|
```
|
|
dd if=/dev/sda of=/dev/null
|
|
```
|
|
|
|
Add your public key to the public key-ring.
|
|
|
|
```
|
|
gpg --export -a $KEYNAME >~/.gnupg/pubkey.txt
|
|
wc -l ~/.gnupg/pubkey.txt
|
|
```
|
|
|
|
The output of "wc" should be non-zero (usually it is 30 or more)
|
|
|
|
Add your keyname to the list of keys:
|
|
|
|
```
|
|
cd keyrings/live
|
|
gpg --homedir=. --import ~/.gnupg/pubkey.txt
|
|
cd ../..
|
|
blackbox_addadmin.sh $KEYNAME
|
|
```
|
|
|
|
Check all these updates into the VCS:
|
|
|
|
```
|
|
git add pubring.gpg trustdb.gpg blackbox-admins.txt
|
|
git commit -m"Adding my gpg key" pubring.gpg trustdb.gpg blackbox-admins.txt
|
|
|
|
or
|
|
|
|
hg commit -m"Adding my gpg key" pubring.gpg trustdb.gpg blackbox-admins.txt
|
|
```
|
|
|
|
|
|
### Step 2: SOMEONE ELSE adds you to the system.
|
|
|
|
Ask someone that already has access to re-encrypt the data files. This gives you access. They simply decrypt and re-encrypt the data without making any changes:
|
|
|
|
```
|
|
gpg --import keyrings/live/pubring.gpg
|
|
bin/blackbox_update_all_files.sh
|
|
```
|
|
|
|
Push the re-encrypted files:
|
|
|
|
```
|
|
git push
|
|
|
|
or
|
|
|
|
hg push
|
|
```
|
|
|
|
### Step 3: YOU test.
|
|
|
|
Make sure you can decrypt a file. (NOTE: It is a good idea to keep a dummy file in VCS just for new people to practice on.)
|
|
|
|
First Time Setup
|
|
===========================
|
|
|
|
Overview:
|
|
|
|
To add "blackbox" to a git repo, you'll need to do the following:
|
|
|
|
1. Create some directories
|
|
2. For each user, have them createa GPG key and add it to the key ring.
|
|
3. For any automated user (one that must be able to decrypt without a passphrase), create a GPG key and create a subkey with an empty passphrase.
|
|
4. Add
|
|
|
|
### Create some directories
|
|
|
|
You'll want to include blackbox's binaries in your PATH:
|
|
```
|
|
export PATH=$PATH:/the/path/to/blackbox/bin
|
|
```
|
|
|
|
In the git repo you plan on using blackbox, add these two lines to .gitignore
|
|
|
|
```
|
|
pubring.gpg~
|
|
secring.gpg
|
|
```
|
|
|
|
Create this directory. It is where the pubkeys will be stored:
|
|
```
|
|
mkdir -p keyrings/live
|
|
```
|
|
|
|
And commit the change:
|
|
|
|
```
|
|
git add keyrings
|
|
git add .gitignore
|
|
git commit -m'Update .gitignore' .gitignore keyrings
|
|
```
|
|
|
|
|
|
### For each user, have them create a GPG key and add it to the key ring.
|
|
|
|
Follow the instructions for "How to indoctrinate a new user into
|
|
the system?". For the first user, you only have to do Step 1.
|
|
|
|
Once that is done, is a good idea to test the system by making sure
|
|
a file can be added to the system (see "How to enroll a new file
|
|
into the system?"), and a different user can decrypt the file.
|
|
|
|
Make a new file and register it:
|
|
|
|
```
|
|
rm -f foo.txt.gpg foo.txt
|
|
echo This is a test. >foo.txt
|
|
blackbox_register_new_file.sh foo.txt
|
|
```
|
|
|
|
Decrypt it:
|
|
|
|
```
|
|
blackbox_edit_start.sh foo.txt.gpg
|
|
cat foo.txt
|
|
echo This is the new file contents. >foo.txt
|
|
```
|
|
|
|
Re-encrypt it:
|
|
```
|
|
blackbox_edit_end.sh foo.txt.gpg
|
|
ls -l foo.txt*
|
|
```
|
|
|
|
Push these changes to the repo. Make sure another user can
|
|
check out and change the contents of the file.
|
|
|
|
|
|
### For any automated user create a key and subkey.
|
|
|
|
An automated user (a "role account") is one that that must be able
|
|
to decrypt without a passphrase. In general you'll want to do this
|
|
for the user that pulls the files from the repo to the master. This
|
|
may be automated with Jenkins CI or other CI system.
|
|
|
|
GPG keys have to have a passphrase. However, passphrases are optional
|
|
on subkeys. Therefore, we will create a key with a passphrase then
|
|
create a subkey without a passphrase.
|
|
Since the subkey is very powerful, it should be created on a very
|
|
secure machine.
|
|
|
|
There's another catch. The role account probably can't check files
|
|
into Git. It probably only has read-only access to the repo. That's
|
|
a good security policy. This means that the role account can't
|
|
be used to upload the subkey public bits into the repo.
|
|
|
|
Therefore, we will create the key/subkey on a secure machine
|
|
as yourself. From there we can commit the public portions into
|
|
the repo. Also from this account we will export the parts
|
|
that the role account needs, copy them to where the role account
|
|
can access them, and import them as the role account.
|
|
|
|
ProTip: If asked to generate entropy, consider running this on the same machine in another window:`sudo dd if=/dev/sda of=/dev/null`
|
|
|
|
For the rest of this doc, you'll need to make the following substittions:
|
|
|
|
- ROLEUSER: svc_deployacct or whatever your role account's name is.
|
|
- NEWMASTER: the machine this role account exists on.
|
|
- SECUREHOST: The machine you use to create the keys.
|
|
|
|
NOTE: This should be more automated. Patches welcome.
|
|
|
|
On SECUREHOST, create thew puppet master's keys:
|
|
|
|
```
|
|
$ mkdir /tmp/NEWMASTER
|
|
$ cd /tmp/NEWMASTER
|
|
$ gpg --homedir . --gen-key
|
|
Your selection?
|
|
(1) RSA and RSA (default)
|
|
What keysize do you want? (2048) DEFAULT
|
|
Key is valid for? (0) DEFAULT
|
|
|
|
# Real name: Puppet CI Deploy Account
|
|
# Email address: svc_deployacct@hostname.domain.name
|
|
```
|
|
|
|
NOTE: Rather than a real email address, use the username@FQDN of
|
|
the host the key will be used on. If you use this role account on
|
|
many machines, each should have its own key. By using the FQDN of
|
|
the host, you will be able to know which key is which.
|
|
In this doc, we'll refer to username@FQDN as $KEYNAME
|
|
|
|
Save the passphrase somewhere safe!
|
|
|
|
ProTip: If asked to generate entropy, consider running this on the same machine in another window:`sudo dd if=/dev/sda of=/dev/null`
|
|
|
|
Create a sub-key that has no password:
|
|
|
|
```
|
|
$ gpg --homedir . --edit-key svc_deployacct
|
|
gpg> addkey
|
|
(enter passphrase)
|
|
Please select what kind of key you want:
|
|
(3) DSA (sign only)
|
|
(4) RSA (sign only)
|
|
(5) Elgamal (encrypt only)
|
|
(6) RSA (encrypt only)
|
|
Your selection? 6
|
|
What keysize do you want? (2048)
|
|
Key is valid for? (0)
|
|
Command> key 2
|
|
(the new subkey has a "*" next to it)
|
|
Command> passwd
|
|
(enter the main key's passphrase)
|
|
(enter an empty passphrase for the subkey... confirm you want to do this)
|
|
Command> save
|
|
```
|
|
|
|
Now securely export this directory to NEWMASTER:
|
|
|
|
```
|
|
$ gpg --homedir . --export -a svc_sadeploy >/tmp/NEWMACHINE/pubkey.txt
|
|
$ tar cvf /tmp/keys.tar .
|
|
$ rsync -avP /tmp/keys.tar NEWMASTER:/tmp/.
|
|
```
|
|
|
|
On NEWMASTER, receive the new GnuPG config:
|
|
|
|
```
|
|
sudo -u svc_deployacct bash
|
|
mkdir -m 0700 -p ~/.gnupg
|
|
cd ~/.gnupg && tar xpvf /tmp/keys.tar
|
|
```
|
|
|
|
<!---
|
|
Back on SECUREHOST, import the pubkey into the repository.
|
|
|
|
```
|
|
$ cd keyrings/live
|
|
$ gpg --homedir . --import /tmp/NEWMACHINE/pubkey.txt
|
|
```
|
|
-->
|
|
|
|
Back on SECUREHOST, add the new email address to keyrings/live/blackbox-admins.txt:
|
|
|
|
```
|
|
cd /path/to/the/repo
|
|
blackbox_addadmin.sh $KEYNAME
|
|
```
|
|
|
|
Verify that secring.gpg is a zero-length file. If it isn't, you have
|
|
somehow added a private key to the keyring. Start over.
|
|
|
|
```
|
|
$ cd keyrings/live
|
|
$ ls -l secring.gpg
|
|
```
|
|
|
|
Commit the recent changes:
|
|
|
|
```
|
|
$ cd keyrings/live
|
|
git commit -m"Adding key for KEYNAME" pubring.gpg trustdb.gpg blackbox-admins.txt
|
|
```
|
|
|
|
Regenerate all encrypted files with the new key:
|
|
|
|
```
|
|
blackbox_update_all_files.sh
|
|
git status
|
|
git commit -m"updated encryption" -a
|
|
git push
|
|
```
|
|
|
|
On NEWMASTER, import the keys and decrypt the files:
|
|
|
|
```
|
|
sudo -u svc_sadeploy bash # Become the role account.
|
|
gpg --import /etc/puppet/keyrings/live/pubring.gpg
|
|
export PATH=$PATH:/path/to/blackbox/bin
|
|
blackbox_postinstall.sh
|
|
sudo -u puppet cat /etc/puppet/hieradata/blackbox.yaml # or any encrypted file.
|
|
```
|
|
|
|
ProTip: If you get "gpg: decryption failed: No secret key" then you forgot to re-encrypt blackbox.yaml with the new key.
|
|
|
|
On SECUREHOST, securerly delete your files:
|
|
|
|
```
|
|
cd /tmp/NEWMASTER
|
|
# On machines with the "shred" command:
|
|
shred -u /tmp/keys.tar
|
|
find . -type f -print0 | xargs -0 shred -u
|
|
# All else:
|
|
rm -rf /tmp/NEWMASTER
|
|
```
|
|
|
|
Also shred any other temporary files you may have made.
|