Note this article was written in May of 2016, there have been new features introduced for Puppet Orchestration in October 2016 with the release of PE 2016.4. When I have a chance I may write a follow-up post to this one with any new thoughts, etc.
Sorry for the delay in getting this blog post done, been busy with project work for clients, but enough of that on to the good stuff.
Puppet has a great tool to help you automate the deployment of your Puppet code called Code Manager. In this blog post I am going to show you how to setup Puppet’s Code Manager with GitLab as our git repository, and automate Code Manager utilizing Webhooks with GitLab.
With Code Manager you need to setup a control repository with separate branches for each Puppet environment to be setup under the /etc/puppetlabs/code/environments directory. What main data that will reside in these branches is your hieradata, Puppetfile, and environment.conf for each environment. There will also be blank folders for modules and manifests. I prefer to keep my modules controlled by the Puppetfile, either they be ones from the Puppet Forge or custom modules I keep in GitLab. The modules folder will be populated when Code Manager runs, based on what is in the Puppetfile.
Also when Code Manager runs it will overwrite any existing data in the environments subdirectories with what is stored in the repository and Puppetfile. So no changes should be made directly on Puppet Master because it will be overwritten by what is in source control.
First in GitLab I created a project called “puppet-code” with a development and production branch:
Next lets take a look at the file and folder structure of our control repository:
Taking a look at our Puppetfile content below (note capitalization of the filename “Puppetfile”) I have a few modules from the Puppet Forge I want Code Manager to download for me and keep at this version. If I wanted Code Manager to update the modules to the latest version each time it runs, I would append to the module “, :latest” so for the apache module it would be “mod ‘puppetlabs/apache’, :latest” I am also having Code Manager manage a git repo that is holding my code for example my corporate website. More can be read on the Puppetfile from the Puppet Documentation.
Each branch will have appropriate data for each environment, such as environment specific hieradata.
First prerequisite before setting up Code Manager on Puppet is we need to create a deployment user in GitLab for Code Manager to use to connect to the git repositories. In GitLab I created a user called Puppet, and Impersonated that user to add an SSH public key to the user account. Use whatever tool you like to create an SSH keypair, I use puttygen.exe. Add the public key to GitLab and save the private key that we will put on the Puppet Master when we enable Code Manager. Next we add the Puppet user in GitLab to have Reporter permissions to be able to Pull project code from GitLab.
To enable Code Manager perform the following steps:
- Enter chown -R pe-puppet:pe-puppet /etc/puppetlabs/code on the command line. This ensures that the pe-puppet user owns the code directory and can make changes as needed.
- In the console, in the _puppet_enterprise::profile::master_ class, set the following parameters:
- _code_manager_auto_configure_ to true. This enables and automatically configures both Code Manager and file sync.
- _r10k_remote_: This is the location of your control repository. Enter a string that is a valid URL for your Git control repository. For example: ‘git@<YOUR.GIT.SERVER.COM>:puppet/control.git’.
- _r10k_private_key_: This is the path to the private key that permits the pe-puppet user to access all Git repositories. This file must be owned by the pe-puppet user. Enter a string, such as _’/etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa’_.
- Run Puppet on all of your masters.
Testing connectivity to the control repo is pretty straightforward because Code Manager is running r10k in the background, you can test the connection to the control repo with a read-only r10k command. To make sure that Code Manager can connect to the control repo, enter the following on the command line:
r10k deploy display –fetch
If the control repo is set up properly, this command fetches and displays a list of the environments in the control repo.
Code Manager needs an authentication token for both authentication and authorization. This token allows Code Manager to securely deploy the requested environment.
These steps assume that you have already configured the Puppet Access command line tool.
To generate a token for Code Manager, first you create a deployment role and user, and then you request an authentication token.
Before you request a token, you must assign a user the correct permissions with role-based access control (RBAC).
To create the deployment user and user role:
- Create a new role named “Deploy Environments”.
- Assign this role the following permissions:
- Add the Puppet Environment type.
- Set Permissions for this type to Deploy code.
- Set Object for this type to All.
- Add the Tokens type.
- Set Permissions for this type to Override default expiry.
- Add the Puppet Environment type.
- Create a deployment user.
- Add the deployment user to the Deploy Environments role.
- Before requesting the token we need to set the password for the deployment user. Click Generate password reset, and copy and paste the Password Reset Link into another browser and change the user’s password.
Next we need to request an authentication token. Note that by default, authentication tokens have a five-minute lifetime. With the Override default expiry permission set, you can change the lifetime to a duration better suited for a long-running, automated process.
Generate the authentication token using the puppet-access command:
- On the command line on the master, run puppet-access login –service-url https://
:4433/rbac-api –lifetime 720d.
- Enter the username and password of the deployment user when prompted.
The generated token is stored in a file for later use. The default location for storing the token is ~/.puppetlabs/token. To view the token, run puppet-access show.
To make sure Code Manager deploys your environments correctly, you should test it. To test, trigger Code Manager on the command line to deploy a single environment:
puppet-code deploy production –wait
Check to make sure the environment was deployed. If so, you’ve set up Code Manager correctly.
To have Code Manager automatically run every time a push event is made to the control repository, we can create a webhook in GitLab to call the Puppet webhook endpoint. In GitLab go to the control repository project, then Settings. Go to the Webhooks page and add the following URL:
The token is the token of the deployment user created above.
Now when you push changes to either branch of the project this will run code manager and update the code. More information on Webhooks and Code Manager can be found here in the Puppet Documentation.
That’s pretty much it for setting up Code Manager in Puppet, it definitely makes your life easier managing your Puppet Code, and forcing the good habit of keeping all Puppet Code in source control.
My next topic I will be covering is Puppet’s Application Orchestration, which I have been really excited to finally start playing with.
Until next time.