|
|
|
@@ -1,122 +1,78 @@
|
|
|
|
BlackBox
|
|
|
|
BlackBox
|
|
|
|
========
|
|
|
|
========
|
|
|
|
|
|
|
|
|
|
|
|
Safely store secrets in a VCS repo (i.e. Git, Mercurial, Subversion or Perforce). These
|
|
|
|
Safely store secrets in a VCS repo (i.e. Git, Mercurial, Subversion or Perforce). These commands make it easy for you to Gnu Privacy Guard (GPG) encrypt specific files in a repo so they are "encrypted at rest" in your repository. However, the scripts make it easy to decrypt them when you need to view or edit them, and decrypt them for use in production. Originally written for Puppet, BlackBox now works with any Git or Mercurial repository.
|
|
|
|
commands make it easy
|
|
|
|
|
|
|
|
for you to Gnu Privacy Guard (GPG) encrypt specific files in a repo so they are
|
|
|
|
|
|
|
|
"encrypted at rest" in your repository. However, the scripts
|
|
|
|
|
|
|
|
make it easy to decrypt them when you need to view or edit them,
|
|
|
|
|
|
|
|
and decrypt them for use in production. Originally written
|
|
|
|
|
|
|
|
for Puppet, BlackBox now works with any Git or Mercurial repository.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A slide presentation about an older release [is on SlideShare](http://www.slideshare.net/TomLimoncelli/the-blackbox-project-sfae).
|
|
|
|
A slide presentation about an older release [is on SlideShare](http://www.slideshare.net/TomLimoncelli/the-blackbox-project-sfae).
|
|
|
|
|
|
|
|
|
|
|
|
Table of Contents
|
|
|
|
Table of Contents
|
|
|
|
===============
|
|
|
|
=================
|
|
|
|
|
|
|
|
|
|
|
|
* [Overview](#overview)
|
|
|
|
- [BlackBox](#blackbox)
|
|
|
|
* [Why is this important?](#why-is-this-important)
|
|
|
|
- [Table of Contents](#table-of-contents)
|
|
|
|
* [Installation Instructions](#installation-instructions)
|
|
|
|
- [Overview](#overview)
|
|
|
|
* [Commands](#commands)
|
|
|
|
- [Why is this important?](#why-is-this-important)
|
|
|
|
* [Compatibility](#compatibility)
|
|
|
|
- [Installation Instructions:](#installation-instructions)
|
|
|
|
* [How is the encryption done?](#how-is-the-encryption-done)
|
|
|
|
- [Commands:](#commands)
|
|
|
|
* [What does this look like to the typical user?](#what-does-this-look-like-to-the-typical-user)
|
|
|
|
- [Compatibility:](#compatibility)
|
|
|
|
* [How to use the secrets with Puppet?](#how-to-use-the-secrets-with-puppet)
|
|
|
|
- [How is the encryption done?](#how-is-the-encryption-done)
|
|
|
|
* [How to enroll a new file into the system?](#how-to-enroll-a-new-file-into-the-system)
|
|
|
|
- [What does this look like to the typical user?](#what-does-this-look-like-to-the-typical-user)
|
|
|
|
* [How to remove a file from the system?](#how-to-remove-a-file-from-the-system)
|
|
|
|
- [How to use the secrets with Puppet?](#how-to-use-the-secrets-with-puppet)
|
|
|
|
* [How to indoctrinate a new user into the system?](#how-to-indoctrinate-a-new-user-into-the-system)
|
|
|
|
- [Entire files:](#entire-files)
|
|
|
|
* [How to remove a user from the system?](#how-to-remove-a-user-from-the-system)
|
|
|
|
- [Small strings:](#small-strings)
|
|
|
|
* [Enabling Blackbox For a Repo](#enabling-blackbox-for-a-repo)
|
|
|
|
- [How to enroll a new file into the system?](#how-to-enroll-a-new-file-into-the-system)
|
|
|
|
* [Set up automated users or "role accounts"](#set-up-automated-users-or-role-accounts)
|
|
|
|
- [How to remove a file from the system?](#how-to-remove-a-file-from-the-system)
|
|
|
|
* [Some common errors](#some-common-errors)
|
|
|
|
- [How to indoctrinate a new user into the system?](#how-to-indoctrinate-a-new-user-into-the-system)
|
|
|
|
* [Using Blackbox without a repo](#using-blackbox-without-a-repo)
|
|
|
|
- [How to remove a user from the system?](#how-to-remove-a-user-from-the-system)
|
|
|
|
* [How to submit bugs or ask questions?](#how-to-submit-bugs-or-ask-questions)
|
|
|
|
- [Enabling Blackbox For a Repo](#enabling-blackbox-for-a-repo)
|
|
|
|
* [Developer Info](#developer-info)
|
|
|
|
- [Set up automated users or “role accounts”](#set-up-automated-users-or-role-accounts)
|
|
|
|
* [Alternatives](#alternatives)
|
|
|
|
- [Replace expired keys:](#replace-expired-keys)
|
|
|
|
* [License](#license)
|
|
|
|
- [Some common errors:](#some-common-errors)
|
|
|
|
|
|
|
|
- [Using Blackbox without a repo](#using-blackbox-without-a-repo)
|
|
|
|
|
|
|
|
- [How to submit bugs or ask questions?](#how-to-submit-bugs-or-ask-questions)
|
|
|
|
|
|
|
|
- [Developer Info](#developer-info)
|
|
|
|
|
|
|
|
- [Alternatives](#alternatives)
|
|
|
|
|
|
|
|
- [License](#license)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Overview
|
|
|
|
Overview
|
|
|
|
========
|
|
|
|
========
|
|
|
|
|
|
|
|
|
|
|
|
Suppose you have a VCS repository (i.e. a Git or Mercurial repo)
|
|
|
|
Suppose you have a VCS repository (i.e. a Git or Mercurial repo) and certain files contain secrets such as passwords or SSL private keys. Often people just store such files "and hope that nobody finds them in the repo". That's not safe.
|
|
|
|
and certain files contain secrets such as passwords or SSL private
|
|
|
|
|
|
|
|
keys. Often people just store such files "and hope that nobody finds
|
|
|
|
|
|
|
|
them in the repo". That's not safe.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
With BlackBox, those files are stored encrypted using GPG. Access to
|
|
|
|
With BlackBox, those files are stored encrypted using GPG. Access to the VCS repo without also having the right GPG keys makes it worthless to have the files. As long as you keep your GPG keys safe, you don't have to worry about storing your VCS repo on an untrusted server. Heck, even if you trust your server, now you don't have to trust the people that do backups of that server, or the people that handle the backup tapes!
|
|
|
|
the VCS repo without also having the right GPG keys
|
|
|
|
|
|
|
|
makes it worthless to have the files. As long as you keep your GPG keys
|
|
|
|
|
|
|
|
safe, you don't have to worry about storing your VCS repo on an untrusted
|
|
|
|
|
|
|
|
server. Heck, even if you trust your server, now you don't have to trust
|
|
|
|
|
|
|
|
the people that do backups of that server, or the people that handle the
|
|
|
|
|
|
|
|
backup tapes!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Rather than one GPG passphrase for all the files, each person with access
|
|
|
|
Rather than one GPG passphrase for all the files, each person with access has their own GPG keys in the system. Any file can be decrypted by anyone with their GPG key. This way, if one person leaves the company, you don't have to communicate a new password to everyone with access. Simply disable the one key that should no longer have access. The process for doing this is as easy as running 2 commands (1 to disable their key, 1 to re-encrypt all files.)
|
|
|
|
has their own GPG keys in the system. Any file can be decrypted by
|
|
|
|
|
|
|
|
anyone with their GPG key. This way, if one person leaves the company,
|
|
|
|
|
|
|
|
you don't have to communicate a new password to everyone with access.
|
|
|
|
|
|
|
|
Simply disable the one key that should no longer have access.
|
|
|
|
|
|
|
|
The process for doing this is as easy as running 2 commands (1 to
|
|
|
|
|
|
|
|
disable their key, 1 to re-encrypt all files.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Automated processes often need access to all the decrypted files.
|
|
|
|
Automated processes often need access to all the decrypted files. This is easy too. For example, suppose Git is being used for Puppet files. The master needs access to the decrypted version of all the files. Simply set up a GPG key for the Puppet master (or the role account that pushes new files to the Puppet master) and have that user run `blackbox_postdeploy` after any files are updated.
|
|
|
|
This is easy too. For example, suppose Git is being used for Puppet
|
|
|
|
|
|
|
|
files. The master needs access to the decrypted version of all the
|
|
|
|
|
|
|
|
files. Simply set up a GPG key for the Puppet master (or the role
|
|
|
|
|
|
|
|
account that pushes new files to the Puppet master) and have that
|
|
|
|
|
|
|
|
user run `blackbox_postdeploy` after any files are updated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Getting started is easy. Just `cd` into a Git, Mercurial, Subversion or
|
|
|
|
|
|
|
|
Perforce repository and run `blackbox_initialize`. After that, if a file is to
|
|
|
|
|
|
|
|
be encrypted, run `blackbox_register_new_file` and you are done. Add
|
|
|
|
|
|
|
|
and remove keys with `blackbox_addadmin` and `blackbox_removeadmin`.
|
|
|
|
|
|
|
|
To view and/or edit a file, run `blackbox_edit`; this will decrypt the
|
|
|
|
|
|
|
|
file and open with whatever is specified by your $EDITOR environment
|
|
|
|
|
|
|
|
variable. When you close the editor the file will automatically be
|
|
|
|
|
|
|
|
encrypted again and the temporary plaintext file will be shredded. If
|
|
|
|
|
|
|
|
you need to leave the file decrypted while you update you can use the
|
|
|
|
|
|
|
|
`blackbox_edit_start` to decrypt the file and `blackbox_edit_end` when
|
|
|
|
|
|
|
|
you want to "put it back in the box."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Getting started is easy. Just `cd` into a Git, Mercurial, Subversion or Perforce repository and run `blackbox_initialize`. After that, if a file is to be encrypted, run `blackbox_register_new_file` and you are done. Add and remove keys with `blackbox_addadmin` and `blackbox_removeadmin`. To view and/or edit a file, run `blackbox_edit`; this will decrypt the file and open with whatever is specified by your $EDITOR environment variable. When you close the editor the file will automatically be encrypted again and the temporary plaintext file will be shredded. If you need to leave the file decrypted while you update you can use the`blackbox_edit_start` to decrypt the file and `blackbox_edit_end` when you want to "put it back in the box."
|
|
|
|
|
|
|
|
|
|
|
|
Why is this important?
|
|
|
|
Why is this important?
|
|
|
|
============================
|
|
|
|
======================
|
|
|
|
|
|
|
|
|
|
|
|
OBVIOUSLY we don't want secret things like SSL private keys
|
|
|
|
OBVIOUSLY we don't want secret things like SSL private keys and passwords to be leaked.
|
|
|
|
and passwords to be leaked.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
NOT SO OBVIOUSLY when we store "secrets" in a VCS repo like Git or
|
|
|
|
NOT SO OBVIOUSLY when we store "secrets" in a VCS repo like Git or Mercurial, suddenly we are less able to share our code with other people. Communication between subteams of an organization is hurt. You can't collaborate as well. Either you find yourself emailing individual files around (yuck!), making a special repo with just the files needed by your collaborators (yuck!!), or just deciding that collaboration isn't worth all that effort (yuck!!!).
|
|
|
|
Mercurial, suddenly we are less able to share our code with other
|
|
|
|
|
|
|
|
people. Communication between subteams of an organization is hurt.
|
|
|
|
|
|
|
|
You can't collaborate as well. Either you find yourself emailing
|
|
|
|
|
|
|
|
individual files around (yuck!), making a special repo with just
|
|
|
|
|
|
|
|
the files needed by your collaborators (yuck!!), or just deciding that
|
|
|
|
|
|
|
|
collaboration isn't worth all that effort (yuck!!!).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The ability to be open and transparent about our code, with the
|
|
|
|
The ability to be open and transparent about our code, with the exception of a few specific files, is key to the kind of collaboration that DevOps and modern IT practitioniers need to do.
|
|
|
|
exception of a few specific files, is key to the kind of
|
|
|
|
|
|
|
|
collaboration that DevOps and modern IT practitioniers
|
|
|
|
|
|
|
|
need to do.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Installation Instructions:
|
|
|
|
Installation Instructions:
|
|
|
|
==========================
|
|
|
|
==========================
|
|
|
|
|
|
|
|
|
|
|
|
* *The MacPorts Way*: `sudo port install vcs_blackbox`
|
|
|
|
- *The MacPorts Way*: `sudo port install vcs_blackbox`
|
|
|
|
* *The RPM way*: Check out the repo and make an RPM via `make packages-rpm`; now you can distribute the RPM via local methods.
|
|
|
|
- *The RPM way*: Check out the repo and make an RPM via `make packages-rpm`; now you can distribute the RPM via local methods.
|
|
|
|
* *The Debian/Ubuntu way*: Check out the repo and install [fpm](https://github.com/jordansissel/fpm). Now you can make a DEB `make packages-deb` that can be distributed via local methods.
|
|
|
|
- *The Debian/Ubuntu way*: Check out the repo and install [fpm](https://github.com/jordansissel/fpm). Now you can make a DEB `make packages-deb` that can be distributed via local methods.
|
|
|
|
* *The hard way*: Copy all the files in "bin" to your "bin".
|
|
|
|
- *The hard way*: Copy all the files in "bin" to your "bin".
|
|
|
|
* *The manual way*: `make manual-install` to install. `make manual-uninstall` to uninstall.
|
|
|
|
- *The manual way*: `make manual-install` to install. `make manual-uninstall` to uninstall.
|
|
|
|
* *The Antigen Way*: Add `antigen bundle StackExchange/blackbox` to your .zshrc
|
|
|
|
- *The Antigen Way*: Add `antigen bundle StackExchange/blackbox` to your .zshrc
|
|
|
|
* *The Zgen Way*: Add `zgen load StackExchange/blackbox` to your .zshrc where you're loading your other plugins.
|
|
|
|
- *The Zgen Way*: Add `zgen load StackExchange/blackbox` to your .zshrc where you're loading your other plugins.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Commands:
|
|
|
|
Commands:
|
|
|
|
============================
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Name: | Description: |
|
|
|
|
| Name: | Description: |
|
|
|
|
| --- | --- |
|
|
|
|
|------------------------------|-------------------------------------------------------------------------|
|
|
|
|
| `blackbox_edit` | Decrypt, run $EDITOR, re-encrypt a file |
|
|
|
|
| `blackbox_edit` | Decrypt, run $EDITOR, re-encrypt a file |
|
|
|
|
| `blackbox_edit_start` | Decrypt a file so it can be updated |
|
|
|
|
| `blackbox_edit_start` | Decrypt a file so it can be updated |
|
|
|
|
| `blackbox_edit_end` | Encrypt a file after blackbox_edit_start was used |
|
|
|
|
| `blackbox_edit_end` | Encrypt a file after blackbox_edit_start was used |
|
|
|
|
@@ -134,122 +90,76 @@ Commands:
|
|
|
|
| `blackbox_update_all_files` | Decrypt then re-encrypt all files. Useful after keys are changed |
|
|
|
|
| `blackbox_update_all_files` | Decrypt then re-encrypt all files. Useful after keys are changed |
|
|
|
|
| `blackbox_whatsnew` | show what has changed in the last commit for a given file |
|
|
|
|
| `blackbox_whatsnew` | show what has changed in the last commit for a given file |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Compatibility:
|
|
|
|
Compatibility:
|
|
|
|
============================
|
|
|
|
==============
|
|
|
|
|
|
|
|
|
|
|
|
Blackbox automatically determines which VCS you are using
|
|
|
|
Blackbox automatically determines which VCS you are using and does the right thing. It has a plug-in architecture to make it easy to extend to work with other systems. It has been tested to work with many operating systems.
|
|
|
|
and does the right thing. It has a plug-in architecture
|
|
|
|
|
|
|
|
to make it easy to extend to work with other systems.
|
|
|
|
|
|
|
|
It has been tested to work with many operating systems.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Version Control systems
|
|
|
|
- Version Control systems
|
|
|
|
* `git` -- The Git
|
|
|
|
- `git` -- The Git
|
|
|
|
* `hg` -- Mercurial
|
|
|
|
- `hg` -- Mercurial
|
|
|
|
* `svn` -- SubVersion (Thanks, Ben Drasin!)
|
|
|
|
- `svn` -- SubVersion (Thanks, Ben Drasin!)
|
|
|
|
* `p4` -- Perforce
|
|
|
|
- `p4` -- Perforce
|
|
|
|
* none -- The files can be decrypted outside of a repo if the keyrings directory is intact
|
|
|
|
- none -- The files can be decrypted outside of a repo if the keyrings directory is intact
|
|
|
|
* Operating system
|
|
|
|
- Operating system
|
|
|
|
* CentOS / RedHat
|
|
|
|
- CentOS / RedHat
|
|
|
|
* MacOS X
|
|
|
|
- MacOS X
|
|
|
|
* Cygwin (Thanks, Ben Drasin!)
|
|
|
|
- Cygwin (Thanks, Ben Drasin!)
|
|
|
|
|
|
|
|
|
|
|
|
To add or fix support for a VCS system, look for code at the end
|
|
|
|
To add or fix support for a VCS system, look for code at the end of `bin/_blackbox_common.sh`
|
|
|
|
of `bin/_blackbox_common.sh`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To add or fix support for a new operating system, look for the case
|
|
|
|
To add or fix support for a new operating system, look for the case statements in `bin/_blackbox_common.sh` and `bin/_stack_lib.sh` and maybe `tools/confidence_test.sh`
|
|
|
|
statements in `bin/_blackbox_common.sh` and `bin/_stack_lib.sh` and
|
|
|
|
|
|
|
|
maybe `tools/confidence_test.sh`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note: Cywin support requires the following packages:
|
|
|
|
Note: Cywin support requires the following packages:
|
|
|
|
|
|
|
|
|
|
|
|
* Normal operation:
|
|
|
|
- Normal operation:
|
|
|
|
* gnupg
|
|
|
|
- gnupg
|
|
|
|
* git or mercurial or subversion or perforce (as appropriate)
|
|
|
|
- git or mercurial or subversion or perforce (as appropriate)
|
|
|
|
* Development (if you will be adding code and want to run the confidence test)
|
|
|
|
- Development (if you will be adding code and want to run the confidence test)
|
|
|
|
* procps
|
|
|
|
- procps
|
|
|
|
* make
|
|
|
|
- make
|
|
|
|
* git (the confidence test currently only tests git)
|
|
|
|
- git (the confidence test currently only tests git)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
How is the encryption done?
|
|
|
|
How is the encryption done?
|
|
|
|
============================
|
|
|
|
===========================
|
|
|
|
|
|
|
|
|
|
|
|
GPG has many different ways to encrypt a file. BlackBox uses
|
|
|
|
GPG has many different ways to encrypt a file. BlackBox uses the mode that lets you specify a list of keys that can decrypt the messsage.
|
|
|
|
the mode that lets you specify a list of keys that can decrypt
|
|
|
|
|
|
|
|
the messsage.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If you have 5 people ("admins") that should be able to access
|
|
|
|
If you have 5 people ("admins") that should be able to access the secrets, each creates a GPG key and adds their public key to the keychain. The GPG command used to encrypt the file lists all 5 key names, and therefore any 1 key can decrypt the file.
|
|
|
|
the secrets, each creates a GPG key and adds their public key
|
|
|
|
|
|
|
|
to the keychain. The GPG command used to encrypt the file lists
|
|
|
|
|
|
|
|
all 5 key names, and therefore any 1 key can decrypt the file.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To remove someone's access, remove that admin's key name (i.e. email
|
|
|
|
To remove someone's access, remove that admin's key name (i.e. email address) from the list of admins and re-encrypt all the files. They can still read the .gpg file (assuming they have access to the repository) but they can't decrypt it any more.
|
|
|
|
address) from the list of admins and re-encrypt all the files.
|
|
|
|
|
|
|
|
They can still read the .gpg file (assuming they have access
|
|
|
|
|
|
|
|
to the repository) but they can't decrypt it any more.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
*What if they kept a copy of the old repo before you removed
|
|
|
|
*What if they kept a copy of the old repo before you removed access?* Yes, they can decrypt old versions of the file. This is why when an admin leaves the team, you should change all your passwords, SSL certs, and so on. You should have been doing that before BlackBox, right?
|
|
|
|
access?* Yes, they can decrypt old versions of the file. This
|
|
|
|
|
|
|
|
is why when an admin leaves the team, you should change all
|
|
|
|
|
|
|
|
your passwords, SSL certs, and so on. You should have been
|
|
|
|
|
|
|
|
doing that before BlackBox, right?
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
*Why don't you use symmetric keys?* In other words, why mess
|
|
|
|
*Why don't you use symmetric keys?* In other words, why mess with all this GPG key stuff and instead why don't we just encrypt all the files with a single passphrase. Yes, GPG supports that, but then we are managing a shared password, which is fraught with problems. If someone "leaves the team" we would have to communicate to everyone a new password. Now we just have to remove their key. This scales better.
|
|
|
|
with all this GPG key stuff and instead why don't we just encrypt
|
|
|
|
|
|
|
|
all the files with a single passphrase. Yes, GPG supports that,
|
|
|
|
|
|
|
|
but then we are managing a shared password, which is fraught with problems.
|
|
|
|
|
|
|
|
If someone "leaves the team" we would have to communicate to everyone
|
|
|
|
|
|
|
|
a new password. Now we just have to remove their key. This scales
|
|
|
|
|
|
|
|
better.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
*How do automated processes decrypt without asking for a password?*
|
|
|
|
*How do automated processes decrypt without asking for a password?* GPG requires a passphrase on a private key. However, it permits the creation of subkeys that have no passphrase. For automated processes, create a subkey that is only stored on the machine that needs to decrypt the files. For example, at Stack Exchange, when our Continuous Integration (CI) system pushes a code change to our Puppet masters, they run `blackbox_postdeploy` to decrypt all the files. The user that runs this code has a subkey that doesn't require a passphrase. Since we have many masters, each has its own key. And, yes, this means our Puppet Masters have to be very secure. However, they were already secure because, like, dude... if you can break into someone's puppet master you own their network.
|
|
|
|
GPG requires a passphrase on a private key. However, it permits
|
|
|
|
|
|
|
|
the creation of subkeys that have no passphrase. For automated
|
|
|
|
|
|
|
|
processes, create a subkey that is only stored on the machine
|
|
|
|
|
|
|
|
that needs to decrypt the files. For example, at Stack Exchange,
|
|
|
|
|
|
|
|
when our Continuous Integration (CI) system pushes
|
|
|
|
|
|
|
|
a code change to our Puppet masters, they run `blackbox_postdeploy`
|
|
|
|
|
|
|
|
to decrypt all the files. The user that runs this code has a subkey
|
|
|
|
|
|
|
|
that doesn't require a passphrase. Since we have many masters,
|
|
|
|
|
|
|
|
each has its own key. And, yes, this means our Puppet Masters
|
|
|
|
|
|
|
|
have to be very secure. However, they were already secure because,
|
|
|
|
|
|
|
|
like, dude... if you can break into someone's puppet master you own
|
|
|
|
|
|
|
|
their network.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
*If you use Puppet, why didn't you just use hiera-eyaml?*
|
|
|
|
*If you use Puppet, why didn't you just use hiera-eyaml?* There are 4 reasons:
|
|
|
|
There are 4 reasons:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1. This works with any Git or Mercurial repo, even if you aren't using Puppet.
|
|
|
|
|
|
|
|
2. hiera-eyaml decrypts "on demand" which means your Puppet Master now uses a lot of CPU to decrypt keys every time it is contacted. It slows down your master, which, in my case, is already slow enough.
|
|
|
|
|
|
|
|
3. This works with binary files, without having to ASCIIify them and paste them into a YAML file. Have you tried to do this with a cert that is 10K long and changes every few weeks? Ick.
|
|
|
|
|
|
|
|
4. hiera-eyaml didn't exist when I wrote this.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1. This works with any Git or Mercurial repo, even if you aren't using Puppet.
|
|
|
|
|
|
|
|
2. hiera-eyaml decrypts "on demand" which means your Puppet Master now uses a lot of CPU to decrypt keys every time it is contacted. It slows down your master, which, in my case, is already slow enough.
|
|
|
|
|
|
|
|
3. This works with binary files, without having to ASCIIify them and paste them into a YAML file. Have you tried to do this with a cert that is 10K long and changes every few weeks? Ick.
|
|
|
|
|
|
|
|
4. hiera-eyaml didn't exist when I wrote this.
|
|
|
|
|
|
|
|
|
|
|
|
What does this look like to the typical user?
|
|
|
|
What does this look like to the typical user?
|
|
|
|
================================
|
|
|
|
=============================================
|
|
|
|
|
|
|
|
|
|
|
|
* If you need to, start the GPG Agent: `eval $(gpg-agent --daemon)`
|
|
|
|
- If you need to, start the GPG Agent: `eval $(gpg-agent --daemon)`
|
|
|
|
* Decrypt the file so it is editable: `blackbox_edit_start FILENAME`
|
|
|
|
- Decrypt the file so it is editable: `blackbox_edit_start FILENAME`
|
|
|
|
* (You will need to enter your GPG passphrase.)
|
|
|
|
- (You will need to enter your GPG passphrase.)
|
|
|
|
* Edit FILENAME as you desire: `vim FILENAME`
|
|
|
|
- Edit FILENAME as you desire: `vim FILENAME`
|
|
|
|
* Re-encrypt the file: `blackbox_edit_end FILENAME`
|
|
|
|
- Re-encrypt the file: `blackbox_edit_end FILENAME`
|
|
|
|
* Commit the changes. `git commit -a` or `hg commit`
|
|
|
|
- Commit the changes. `git commit -a` or `hg commit`
|
|
|
|
|
|
|
|
|
|
|
|
Wait... it can be even easier than that!
|
|
|
|
|
|
|
|
Run `blackbox_edit FILENAME`, and it'll decrypt the file
|
|
|
|
|
|
|
|
in a temp file and call `$EDITOR` on it, re-encrypting again after the editor
|
|
|
|
|
|
|
|
is closed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Wait... it can be even easier than that! Run `blackbox_edit FILENAME`, and it'll decrypt the file in a temp file and call `$EDITOR` on it, re-encrypting again after the editor is closed.
|
|
|
|
|
|
|
|
|
|
|
|
How to use the secrets with Puppet?
|
|
|
|
How to use the secrets with Puppet?
|
|
|
|
================================
|
|
|
|
===================================
|
|
|
|
|
|
|
|
|
|
|
|
### Entire files:
|
|
|
|
### Entire files:
|
|
|
|
|
|
|
|
|
|
|
|
Entire files, such as SSL certs and private keys, are treated just like
|
|
|
|
Entire files, such as SSL certs and private keys, are treated just like regular files. You decrypt them any time you push a new release to the puppet master.
|
|
|
|
regular files. You decrypt them any time you push a new release
|
|
|
|
|
|
|
|
to the puppet master.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Puppet example for an encrypted file: `secret_file.key.gpg`
|
|
|
|
Puppet example for an encrypted file: `secret_file.key.gpg`
|
|
|
|
|
|
|
|
|
|
|
|
@@ -263,13 +173,9 @@ file { '/etc/my_little_secret.key':
|
|
|
|
}
|
|
|
|
}
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Small strings:
|
|
|
|
### Small strings:
|
|
|
|
|
|
|
|
|
|
|
|
Small strings, such as passwords and API keys, are stored in a hiera
|
|
|
|
Small strings, such as passwords and API keys, are stored in a hiera yaml file, which you encrypt with `blackbox_register_new_file`. For example, we use a file called `blackbox.yaml`. You can access them using the hiera() function.
|
|
|
|
yaml file, which you encrypt with `blackbox_register_new_file`. For
|
|
|
|
|
|
|
|
example, we use a file called `blackbox.yaml`. You can access them
|
|
|
|
|
|
|
|
using the hiera() function.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
*Setup:* Configure `hiera.yaml` by adding "blackbox" to the search hierarchy:
|
|
|
|
*Setup:* Configure `hiera.yaml` by adding "blackbox" to the search hierarchy:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -300,15 +206,14 @@ file {'/tmp/debug-blackbox.txt':
|
|
|
|
}
|
|
|
|
}
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
The variable `$the_password` will contain "my secret password" and
|
|
|
|
The variable `$the_password` will contain "my secret password" and can be used anywhere strings are used.
|
|
|
|
can be used anywhere strings are used.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
How to enroll a new file into the system?
|
|
|
|
How to enroll a new file into the system?
|
|
|
|
============================
|
|
|
|
=========================================
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- If you need to, start the GPG Agent: `eval $(gpg-agent --daemon)`
|
|
|
|
|
|
|
|
- Add the file to the system:
|
|
|
|
|
|
|
|
|
|
|
|
* If you need to, start the GPG Agent: `eval $(gpg-agent --daemon)`
|
|
|
|
|
|
|
|
* Add the file to the system:
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
blackbox_register_new_file path/to/file.name.key
|
|
|
|
blackbox_register_new_file path/to/file.name.key
|
|
|
|
```
|
|
|
|
```
|
|
|
|
@@ -317,28 +222,29 @@ Multiple file names can be specified on the command line:
|
|
|
|
|
|
|
|
|
|
|
|
Example 1: Register 2 files:
|
|
|
|
Example 1: Register 2 files:
|
|
|
|
|
|
|
|
|
|
|
|
blackbox_register_new_file file1.txt file2.txt
|
|
|
|
```
|
|
|
|
|
|
|
|
blackbox_register_new_file file1.txt file2.txt
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Example 2: Register all the files in `$DIR`:
|
|
|
|
Example 2: Register all the files in `$DIR`:
|
|
|
|
|
|
|
|
|
|
|
|
find $DIR -type f -not -name '*.gpg' -print0 | xargs -0 blackbox_register_new_file
|
|
|
|
```
|
|
|
|
|
|
|
|
find $DIR -type f -not -name '*.gpg' -print0 | xargs -0 blackbox_register_new_file
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
How to remove a file from the system?
|
|
|
|
How to remove a file from the system?
|
|
|
|
============================
|
|
|
|
=====================================
|
|
|
|
|
|
|
|
|
|
|
|
This happens quite rarely, but we've got it covered:
|
|
|
|
This happens quite rarely, but we've got it covered:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
blackbox_deregister_file path/to/file.name.key
|
|
|
|
blackbox_deregister_file path/to/file.name.key
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
How to indoctrinate a new user into the system?
|
|
|
|
How to indoctrinate a new user into the system?
|
|
|
|
============================
|
|
|
|
===============================================
|
|
|
|
|
|
|
|
|
|
|
|
``keyrings/live/blackbox-admins.txt`` is a file that
|
|
|
|
`keyrings/live/blackbox-admins.txt` is a file that lists which users are able to decrypt files. (More pedantically, it is a list of the GnuPG key names that the file is encrypted for.)
|
|
|
|
lists which users are able to decrypt files.
|
|
|
|
|
|
|
|
(More pedantically, it is a list of the GnuPG key
|
|
|
|
|
|
|
|
names that the file is encrypted for.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
To join the list of people that can edit the file requires three steps; You create a GPG key and add it to the key ring. Then, someone that already has access adds you to the system. Lastly, you should test your access.
|
|
|
|
To join the list of people that can edit the file requires three steps; You create a GPG key and add it to the key ring. Then, someone that already has access adds you to the system. Lastly, you should test your access.
|
|
|
|
|
|
|
|
|
|
|
|
@@ -384,7 +290,6 @@ ht push
|
|
|
|
|
|
|
|
|
|
|
|
NOTE: Creating a Role Account? If you are adding the pubring.gpg of a role account, you can specify the directory where the pubring.gpg file can be found as a 2nd parameter: `blackbox_addadmin puppetmaster@puppet-master-1.example.com /path/to/the/dir`
|
|
|
|
NOTE: Creating a Role Account? If you are adding the pubring.gpg of a role account, you can specify the directory where the pubring.gpg file can be found as a 2nd parameter: `blackbox_addadmin puppetmaster@puppet-master-1.example.com /path/to/the/dir`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Step 2: SOMEONE ELSE adds you to the system.
|
|
|
|
### Step 2: SOMEONE ELSE adds you to the system.
|
|
|
|
|
|
|
|
|
|
|
|
Ask someone that already has access to re-encrypt the data files. This gives you access. They simply decrypt and re-encrypt the data without making any changes.
|
|
|
|
Ask someone that already has access to re-encrypt the data files. This gives you access. They simply decrypt and re-encrypt the data without making any changes.
|
|
|
|
@@ -395,8 +300,7 @@ Pre-check: Verify the new keys look good.
|
|
|
|
$ gpg --homedir=keyrings/live --list-keys
|
|
|
|
$ gpg --homedir=keyrings/live --list-keys
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
For example, examine the key name (email address) to make sure
|
|
|
|
For example, examine the key name (email address) to make sure it conforms to corporate standards.
|
|
|
|
it conforms to corporate standards.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Import the keychain into your personal keychain and reencrypt:
|
|
|
|
Import the keychain into your personal keychain and reencrypt:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -422,7 +326,7 @@ hg push
|
|
|
|
Make sure you can decrypt a file. (Suggestion: Keep a dummy file in VCS just for new people to practice on.)
|
|
|
|
Make sure you can decrypt a file. (Suggestion: Keep a dummy file in VCS just for new people to practice on.)
|
|
|
|
|
|
|
|
|
|
|
|
How to remove a user from the system?
|
|
|
|
How to remove a user from the system?
|
|
|
|
============================
|
|
|
|
=====================================
|
|
|
|
|
|
|
|
|
|
|
|
Simply run `blackbox_removeadmin` with their keyname then re-encrypt:
|
|
|
|
Simply run `blackbox_removeadmin` with their keyname then re-encrypt:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -435,9 +339,7 @@ blackbox_update_all_files
|
|
|
|
|
|
|
|
|
|
|
|
When the command completes, you will be given a reminder to check in the change and push it.
|
|
|
|
When the command completes, you will be given a reminder to check in the change and push it.
|
|
|
|
|
|
|
|
|
|
|
|
Note that their keys will still be in the key ring, but they will
|
|
|
|
Note that their keys will still be in the key ring, but they will go unused. If you'd like to clean up the keyring, use the normal GPG commands and check in the file.
|
|
|
|
go unused. If you'd like to clean up the keyring, use the normal
|
|
|
|
|
|
|
|
GPG commands and check in the file.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
gpg --homedir=keyrings/live --list-keys
|
|
|
|
gpg --homedir=keyrings/live --list-keys
|
|
|
|
@@ -447,24 +349,19 @@ git commit -m'Cleaned olduser@example.com from keyring' keyrings/live/*
|
|
|
|
|
|
|
|
|
|
|
|
The key ring only has public keys. There are no secret keys to delete.
|
|
|
|
The key ring only has public keys. There are no secret keys to delete.
|
|
|
|
|
|
|
|
|
|
|
|
Remember that this person did have access to all the secrets at one
|
|
|
|
Remember that this person did have access to all the secrets at one time. They could have made a copy. Therefore, to be completely secure, you should change all passwords, generate new SSL keys, and so on just like when anyone that had privileged access leaves an organization.
|
|
|
|
time. They could have made a copy. Therefore, to be completely
|
|
|
|
|
|
|
|
secure, you should change all passwords, generate new SSL keys, and
|
|
|
|
|
|
|
|
so on just like when anyone that had privileged access leaves an
|
|
|
|
|
|
|
|
organization.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Enabling Blackbox For a Repo
|
|
|
|
Enabling Blackbox For a Repo
|
|
|
|
===========================
|
|
|
|
============================
|
|
|
|
|
|
|
|
|
|
|
|
Overview:
|
|
|
|
Overview:
|
|
|
|
|
|
|
|
|
|
|
|
To add "blackbox" to a git or mercurial repo, you'll need to do the following:
|
|
|
|
To add "blackbox" to a git or mercurial repo, you'll need to do the following:
|
|
|
|
|
|
|
|
|
|
|
|
1. Run the initialize script. This adds a few files to your repo in a directory called "keyrings".
|
|
|
|
1. Run the initialize script. This adds a few files to your repo in a directory called "keyrings".
|
|
|
|
2. For the first user, create a GPG key and add it to the key ring.
|
|
|
|
2. For the first user, create a GPG key and add it to the key ring.
|
|
|
|
3. Encrypt the files you want to be "secret".
|
|
|
|
3. Encrypt the files you want to be "secret".
|
|
|
|
4. For any automated user (one that must be able to decrypt without a passphrase), create a GPG key and create a subkey with an empty passphrase.
|
|
|
|
4. For any automated user (one that must be able to decrypt without a passphrase), create a GPG key and create a subkey with an empty passphrase.
|
|
|
|
|
|
|
|
|
|
|
|
### Run the initialize script.
|
|
|
|
### Run the initialize script.
|
|
|
|
|
|
|
|
|
|
|
|
@@ -475,17 +372,13 @@ export PATH=$PATH:/the/path/to/blackbox/bin
|
|
|
|
blackbox_initialize
|
|
|
|
blackbox_initialize
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
If you're using antigen, adding `antigen bundle StackExchange/blackbox` to
|
|
|
|
If you're using antigen, adding `antigen bundle StackExchange/blackbox` to your .zshrc will download this repository and add it to your $PATH.
|
|
|
|
your .zshrc will download this repository and add it to your $PATH.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### For the first user, create a GPG key and add it to the key ring.
|
|
|
|
### For the first user, create a GPG key and add it to the key ring.
|
|
|
|
|
|
|
|
|
|
|
|
Follow the instructions for "How to indoctrinate a new user into
|
|
|
|
Follow the instructions for "[How to indoctrinate a new user into the system?](#how-to-indoctrinate-a-new-user-into-the-system)". Only do Step 1.
|
|
|
|
the system?". Only do Step 1.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Once that is done, is a good idea to test the system by making sure
|
|
|
|
Once that is done, is a good idea to test the system by making sure a file can be added to the system (see "How to enroll a new file into the system?"), and a different user can decrypt the file.
|
|
|
|
a file can be added to the system (see "How to enroll a new file
|
|
|
|
|
|
|
|
into the system?"), and a different user can decrypt the file.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Make a new file and register it:
|
|
|
|
Make a new file and register it:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -504,51 +397,36 @@ echo This is the new file contents. >foo.txt
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Re-encrypt it:
|
|
|
|
Re-encrypt it:
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
blackbox_edit_end foo.txt.gpg
|
|
|
|
blackbox_edit_end foo.txt.gpg
|
|
|
|
ls -l foo.txt*
|
|
|
|
ls -l foo.txt*
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Push these changes to the repo. Make sure another user can
|
|
|
|
You should only see `foo.txt.gpg` as `foo.txt` should be gone.
|
|
|
|
check out and change the contents of the file.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The next step is to commit `foo.txt.gpg` and make sure another user can check out, view, and change the contents of the file. That is left as an exercise for the reader. If you are feel like taking a risk, don't commit `foo.txt.gpg` and delete it instead.
|
|
|
|
|
|
|
|
|
|
|
|
Set up automated users or "role accounts"
|
|
|
|
Set up automated users or "role accounts"
|
|
|
|
=========================================
|
|
|
|
=========================================
|
|
|
|
|
|
|
|
|
|
|
|
i.e. This is how a Puppet Master can have access to the unencrypted data.
|
|
|
|
i.e. This is how a Puppet Master can have access to the unencrypted data.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
An automated user (a "role account") is one that that must be able to decrypt without a passphrase. In general you'll want to do this for the user that pulls the files from the repo to the master. This may be automated with Jenkins CI or other CI system.
|
|
|
|
|
|
|
|
|
|
|
|
An automated user (a "role account") is one that that must be able
|
|
|
|
GPG keys have to have a passphrase. However, passphrases are optional on subkeys. Therefore, we will create a key with a passphrase then create a subkey without a passphrase. Since the subkey is very powerful, it should be created on a very secure machine.
|
|
|
|
to decrypt without a passphrase. In general you'll want to do this
|
|
|
|
|
|
|
|
for the user that pulls the files from the repo to the master. This
|
|
|
|
|
|
|
|
may be automated with Jenkins CI or other CI system.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
GPG keys have to have a passphrase. However, passphrases are optional
|
|
|
|
There's another catch. The role account probably can't check files into Git/Mercurial. It probably only has read-only access to the repo. That's a good security policy. This means that the role account can't be used to upload the subkey public bits into the repo.
|
|
|
|
on subkeys. Therefore, we will create a key with a passphrase then
|
|
|
|
|
|
|
|
create a subkey without a passphrase.
|
|
|
|
|
|
|
|
Since the subkey is very powerful, it should be created on a very
|
|
|
|
|
|
|
|
secure machine.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
There's another catch. The role account probably can't check files
|
|
|
|
Therefore, we will create the key/subkey on a secure machine as yourself. From there we can commit the public portions into the repo. Also from this account we will export the parts that the role account needs, copy them to where the role account can access them, and import them as the role account.
|
|
|
|
into Git/Mercurial. It probably only has read-only access to the repo. That's
|
|
|
|
|
|
|
|
a good security policy. This means that the role account can't
|
|
|
|
|
|
|
|
be used to upload the subkey public bits into the repo.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Therefore, we will create the key/subkey on a secure machine
|
|
|
|
ProTip: If asked to generate entropy, consider running this on the same machine in another window: `sudo dd if=/dev/sda of=/dev/null`
|
|
|
|
as yourself. From there we can commit the public portions into
|
|
|
|
|
|
|
|
the repo. Also from this account we will export the parts
|
|
|
|
|
|
|
|
that the role account needs, copy them to where the role account
|
|
|
|
|
|
|
|
can access them, and import them as the role account.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ProTip: If asked to generate entropy, consider running this on the
|
|
|
|
|
|
|
|
same machine in another window: `sudo dd if=/dev/sda of=/dev/null`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For the rest of this doc, you'll need to make the following substitutions:
|
|
|
|
For the rest of this doc, you'll need to make the following substitutions:
|
|
|
|
|
|
|
|
|
|
|
|
- ROLEUSER: svc_deployacct or whatever your role account's name is.
|
|
|
|
- ROLEUSER: svc_deployacct or whatever your role account's name is.
|
|
|
|
- NEWMASTER: the machine this role account exists on.
|
|
|
|
- NEWMASTER: the machine this role account exists on.
|
|
|
|
- SECUREHOST: The machine you use to create the keys.
|
|
|
|
- SECUREHOST: The machine you use to create the keys.
|
|
|
|
|
|
|
|
|
|
|
|
NOTE: This should be more automated/scripted. Patches welcome.
|
|
|
|
NOTE: This should be more automated/scripted. Patches welcome.
|
|
|
|
|
|
|
|
|
|
|
|
@@ -567,11 +445,7 @@ Key is valid for? (0) DEFAULT
|
|
|
|
# Email address: svc_deployacct@hostname.domain.name
|
|
|
|
# Email address: svc_deployacct@hostname.domain.name
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
NOTE: Rather than a real email address, use the username@FQDN of
|
|
|
|
NOTE: Rather than a real email address, use the username@FQDN of the host the key will be used on. If you use this role account on many machines, each should have its own key. By using the FQDN of the host, you will be able to know which key is which. In this doc, we'll refer to username@FQDN as $KEYNAME
|
|
|
|
the host the key will be used on. If you use this role account on
|
|
|
|
|
|
|
|
many machines, each should have its own key. By using the FQDN of
|
|
|
|
|
|
|
|
the host, you will be able to know which key is which.
|
|
|
|
|
|
|
|
In this doc, we'll refer to username@FQDN as $KEYNAME
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Save the passphrase somewhere safe!
|
|
|
|
Save the passphrase somewhere safe!
|
|
|
|
|
|
|
|
|
|
|
|
@@ -629,8 +503,7 @@ cd /path/to/the/repo
|
|
|
|
blackbox_addadmin $KEYNAME /tmp/NEWMASTER
|
|
|
|
blackbox_addadmin $KEYNAME /tmp/NEWMASTER
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Verify that secring.gpg is a zero-length file. If it isn't, you have
|
|
|
|
Verify that secring.gpg is a zero-length file. If it isn't, you have somehow added a private key to the keyring. Start over.
|
|
|
|
somehow added a private key to the keyring. Start over.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
$ cd keyrings/live
|
|
|
|
$ cd keyrings/live
|
|
|
|
@@ -678,50 +551,88 @@ rm -rf /tmp/NEWMASTER
|
|
|
|
|
|
|
|
|
|
|
|
Also shred any other temporary files you may have made.
|
|
|
|
Also shred any other temporary files you may have made.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Replace expired keys:
|
|
|
|
|
|
|
|
=====================
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If any one admin's key expires, you can no longer encrypt files. You will need to replace the key and re-encrypt.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Step 0: You see this error:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
$ blackbox_edit_end modified_file.txt
|
|
|
|
|
|
|
|
--> Error: can't re-encrypt because a key has expired.
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Step 1. Administrator removes expired user:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Warning: This process will erase any unencrypted files that you were in the process of editing. Copy them elsewhere and restore the changes when done.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
blackbox_removeadmin expired_user@example.com
|
|
|
|
|
|
|
|
# This next command overwrites any changed unencrypted files. See warning above.
|
|
|
|
|
|
|
|
blackbox_update_all_files
|
|
|
|
|
|
|
|
git commit -m "Re-encrypt all files"
|
|
|
|
|
|
|
|
gpg --homedir=keyrings/live --delete-key expired_user@example.com
|
|
|
|
|
|
|
|
git commit -m 'Cleaned expired_user@example.com from keyring' keyrings/live/*
|
|
|
|
|
|
|
|
git push
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Step 2. Expired user adds an updated key:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
git pull
|
|
|
|
|
|
|
|
blackbox_addadmin updated_user@example.com
|
|
|
|
|
|
|
|
git commit -m'NEW ADMIN: updated_user@example.com keyrings/live/pubring.gpg keyrings/live/trustdb.gpg keyrings/live/blackbox-admins.txt
|
|
|
|
|
|
|
|
git push
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Step 3. Administrator re-encrypts all files with the updated key of the expired user:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
git pull
|
|
|
|
|
|
|
|
gpg --import keyrings/live/pubring.gpg
|
|
|
|
|
|
|
|
blackbox_update_all_files
|
|
|
|
|
|
|
|
git commit -m "Re-encrypt all files"
|
|
|
|
|
|
|
|
git push
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Step 4: Clean up:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Any files that were temporarily copied in the first step so as to not be overwritten can now be copied back and re-encrypted with the `blackbox_edit_end` command.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(Thanks to @chishaku for finding a solution to this problem!)
|
|
|
|
|
|
|
|
|
|
|
|
Some common errors:
|
|
|
|
Some common errors:
|
|
|
|
=========================================
|
|
|
|
===================
|
|
|
|
|
|
|
|
|
|
|
|
`gpg: filename: skipped: No public key` -- Usually this means there
|
|
|
|
`gpg: filename: skipped: No public key` -- Usually this means there is an item in `keyrings/live/blackbox-admins.txt` that is not the name of the key. Either something invalid was inserted (like a filename instead of a username) or a user has left the organization and their key was removed from the keychain, but their name wasn't removed from the blackbox-admins.txt file.
|
|
|
|
is an item in `keyrings/live/blackbox-admins.txt` that is not the
|
|
|
|
|
|
|
|
name of the key. Either something invalid was inserted (like a
|
|
|
|
|
|
|
|
filename instead of a username) or a user has left the organization
|
|
|
|
|
|
|
|
and their key was removed from the keychain, but their name wasn't
|
|
|
|
|
|
|
|
removed from the blackbox-admins.txt file.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`gpg: decryption failed: No secret key` -- Usually means you forgot
|
|
|
|
`gpg: decryption failed: No secret key` -- Usually means you forgot to re-encrypt the file with the new key.
|
|
|
|
to re-encrypt the file with the new key.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`Error: can't re-encrypt because a key has expired.` -- A user's key has expired and can't be used to encrypt any more. Follow the[Replace expired keys](#replace-expired-keys) tip.
|
|
|
|
|
|
|
|
|
|
|
|
Using Blackbox without a repo
|
|
|
|
Using Blackbox without a repo
|
|
|
|
===========================
|
|
|
|
=============================
|
|
|
|
|
|
|
|
|
|
|
|
If the files are copied out of a repo they can still be decrypted
|
|
|
|
If the files are copied out of a repo they can still be decrypted and edited. Obviously edits, changes to keys, and such will be lost if they are made outside the repo. Also note that commands are most likely to only work if run from the base directory (i.e. the parent to the keyrings directory).
|
|
|
|
and edited. Obviously edits, changes to keys, and such will be lost
|
|
|
|
|
|
|
|
if they are made outside the repo. Also note that commands are most
|
|
|
|
|
|
|
|
likely to only work if run from the base directory (i.e. the parent to
|
|
|
|
|
|
|
|
the keyrings directory).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The following commands have been tested outside a repo:
|
|
|
|
The following commands have been tested outside a repo:
|
|
|
|
|
|
|
|
|
|
|
|
* `blackbox_postdeploy`
|
|
|
|
- `blackbox_postdeploy`
|
|
|
|
* `blackbox_edit_start`
|
|
|
|
- `blackbox_edit_start`
|
|
|
|
* `blackbox_edit_end`
|
|
|
|
- `blackbox_edit_end`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
How to submit bugs or ask questions?
|
|
|
|
How to submit bugs or ask questions?
|
|
|
|
============
|
|
|
|
====================================
|
|
|
|
|
|
|
|
|
|
|
|
We welcome questions, bug reports and feedback!
|
|
|
|
We welcome questions, bug reports and feedback!
|
|
|
|
|
|
|
|
|
|
|
|
* https://github.com/StackExchange/blackbox/issues
|
|
|
|
- https://github.com/StackExchange/blackbox/issues
|
|
|
|
|
|
|
|
|
|
|
|
Developer Info
|
|
|
|
Developer Info
|
|
|
|
============
|
|
|
|
==============
|
|
|
|
|
|
|
|
|
|
|
|
Code submissions are gladly welcomed! The code is
|
|
|
|
|
|
|
|
fairly easy to read.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Code submissions are gladly welcomed! The code is fairly easy to read.
|
|
|
|
|
|
|
|
|
|
|
|
Get the code:
|
|
|
|
Get the code:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -735,39 +646,27 @@ Test your changes:
|
|
|
|
make confidence
|
|
|
|
make confidence
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
This runs through a number of system tests. It
|
|
|
|
This runs through a number of system tests. It creates a repo, encrypts files, decrypts files, and so on. You can run these tests to verify that the changes you made didn't break anything. You can also use these tests to verify that the system works with a new operating system.
|
|
|
|
creates a repo, encrypts files, decrypts files, and so on.
|
|
|
|
|
|
|
|
You can run these tests to verify that the changes you made
|
|
|
|
|
|
|
|
didn't break anything. You can also use these tests to
|
|
|
|
|
|
|
|
verify that the system works with a new operating system.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Please submit tests with code changes:
|
|
|
|
Please submit tests with code changes:
|
|
|
|
|
|
|
|
|
|
|
|
The best way to change Blackbox is via Test Driven Development.
|
|
|
|
The best way to change Blackbox is via Test Driven Development. First add a test to `tools/confidence.sh`. This test should fail, and demonstrate the need for the change you are about to make. Then fix the bug or add the feature you want. When you are done, `make confidence` should pass all tests. The PR you submit should include your code as well as the new test. This way the confidence tests accumulate as the system grows as we know future changes don't break old features.
|
|
|
|
First add a test to `tools/confidence.sh`. This test should
|
|
|
|
|
|
|
|
fail, and demonstrate the need for the change you are about to
|
|
|
|
|
|
|
|
make. Then fix the bug or add the feature you want. When
|
|
|
|
|
|
|
|
you are done, `make confidence` should pass all tests.
|
|
|
|
|
|
|
|
The PR you submit should include your code as well as the new
|
|
|
|
|
|
|
|
test. This way the confidence tests accumulate as the system
|
|
|
|
|
|
|
|
grows as we know future changes don't break old features.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note: The tests currently assume "git" and have been tested
|
|
|
|
|
|
|
|
only on CentOS, Mac OS X, and Cygwin. Patches welcome!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note: The tests currently assume "git" and have been tested only on CentOS, Mac OS X, and Cygwin. Patches welcome!
|
|
|
|
|
|
|
|
|
|
|
|
Alternatives
|
|
|
|
Alternatives
|
|
|
|
============
|
|
|
|
============
|
|
|
|
|
|
|
|
|
|
|
|
Here are other open source packages that do something similar to Blackbox. If you like them better than Blackbox, please use them.
|
|
|
|
Here are other open source packages that do something similar to Blackbox. If you like them better than Blackbox, please use them.
|
|
|
|
|
|
|
|
|
|
|
|
* git-crypt: https://www.agwa.name/projects/git-crypt/
|
|
|
|
- git-crypt: https://www.agwa.name/projects/git-crypt/
|
|
|
|
* Pass: http://www.zx2c4.com/projects/password-store/
|
|
|
|
- Pass: http://www.zx2c4.com/projects/password-store/
|
|
|
|
* Transcrypt: https://github.com/elasticdog/transcrypt
|
|
|
|
- Transcrypt: https://github.com/elasticdog/transcrypt
|
|
|
|
|
|
|
|
- Keyringer: https://keyringer.pw/
|
|
|
|
|
|
|
|
|
|
|
|
git-crypt has the best git integration. Once set up it is nearly transparent to the users. However it only works with git.
|
|
|
|
git-crypt has the best git integration. Once set up it is nearly transparent to the users. However it only works with git.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
License
|
|
|
|
License
|
|
|
|
=======
|
|
|
|
=======
|
|
|
|
|
|
|
|
|
|
|
|
This content is released under the MIT License. See the LICENSE.txt file.
|
|
|
|
This content is released under the MIT License. See the LICENSE.txt file.
|
|
|
|
|
