User:OrenBochman/Labs 2

Puppet and Labs

So you have learned about labs and how to create an instance. Behind labs interface there is a configuration management system called puppet. Puppet is important in another more crucial sense - it is also used to manage MediaWiki's production cluster. A necessary step to bringing your work to production will require writing puppet manifests to manage it. This study unit covers basic Puppet usage as well as a number of best practices.

What is puppet?

Puppet allows users to:

Setup and manage multiple virtual machines with predetermined configurations.
It is a DSL built on top of Ruby and uses a declarative model.
It allows to work using a recipe based approach.
It provides an abstraction layer for configuration.

Getting Started

To proceed with this tutorial you will require sysadmin access to the testing lab - which will enable you to define variables within your instance. .

Create a new instance and configure it as follows:
- name it e.g. demo4
- small memory
- availability zone does not matter
- image - use either lucid or precise - all options are ubuntu instances.
- default security group enabled.
- add Puppetmasterself c.f. https://labsconsole.wikimedia.org/wiki/Help:SelfHostedPuppet
This last one configures the instance to install a Puppetmaster. Puppetmaster is the server used for running puppet - which means that your system will not be able to work locally without having to access an external puppet server.

now wait for a bit for the lab to go online, it takes time since it requires puppet to compile the definitions, create the vm and boot it. Now:

login to bastion

"ssh bastion"

access the lab

ssh puppet-demo4 (accept the first lack of certificate message)
sudo puppetd -tv

TODO: insert screenshot of puppetd -tv output

The last command invokes a puppet run by the puppet demons to run and updates the local configuration

add a puppet master to local instance.
adds the operations repository.
and configures the instance to talk to to itself rather than the puppet master.

without a puppet master self - you need to work with a centerelized puppet master configuration as used in production by WM operations . To make changes you need to push changes into gerrit, get them code reviewed and use them from there.

by using the what is

Directory Layout

\puppet
|\manifests
|+site.pp
|\files
|\private
|\modules

Resources Definitions

Puppet simplifies configuration management by abstracting resources. In puppet a resource is defined by a type, title and a list of attributes. Further more resources can be bundled together into collections. Some of the built-in types include:

package
service
file
cron

an example is

     file {
           "/etc/apache2/sites-enabled/000-default/":
                 ensure => absent
                 notify => Service["apache2"];
     }

this defines a file resource of type file at "/etc/apache2/sites-enabled/000-default/" which should be removed if available.

Creating a Class Definition in Puppet

Puppet is declarative and highly reusable. This allows working in a recipe like approach.

nodes (only used in productions)

variables can be used to parametrize the installation.

TODO: screenshot she used icinga which is a opensource monitoring application which is a fork of ngios

#import ...

class apache::demo {
...
}

Packages

the advantage of packages is that they can be used to combine different types together.

class name::monitor::packages {

     #this will install the latest version of the apache web server

      package { [ "apache2"]:
              ensure => latest;
      }

}

Services

services:

   service{
         "apache2":
              ensure => running,
              require => Package["apache2"];
              subscibe => [File[$....],
                           File[$....],
                           File[$....],
                          ]
   }

this spells out explicitly that the service should not be started until apache has been installed.

subscribe here means that the service must be restarted if the configuration files are changed.

$ before a file names indicates that these are variables.

Files


     file {
           "/etc/apache2/sites-enabled/000-default":
                 ensure => absent,
                 notify => Service["apache2"];
     }

which removes the file and then request apache server to restart or refresh it's execution context.

Dependencies

To organize the order of operation on resources four metaparameters are used. These are:

before
require
notify
subscribe

they all accept a resource refrence or an array of resources.

Before and require are very similar. before is used to declate the earlier resource, while require is used to declare the later resource.

notify and subscribe are analogous to before and require, but also declare refresh relationships that they define

Templates variables and Conditionals

It is possible to use templates to parametrize a puppet manifest. For this however we will need to define variables, set the variables and use them in conditional constructs.


     file {
              "/etc/apache2/sites-enabled/000-default":
                    ensure => absent
                    notify => Service["apache2"];
              #this file name is deined in the template apache demo
              "/etc/apache2/sites-enabled/apachedemo":
                    owner => root,
                    group => root,
                    mode => 0444,
                    content => template('apache/sites/apachedemo.wmflabs.org');
                    notify => Service["apache2"];
     }

to set it we go to the Labs UI and

TODO: add screen shot of using

The variables can then be set using:

an external file (in production) or
using the labs UI in labs.

Using Modules

  MODULEPATH/
    MODULE_NAME/
      files/
      templates/
      manifests/
      ...
      README

http://cloudsmith.github.com/geppetto/

Debugging, Tracing, Logging & Troubleshooting

Running in debug mode

Run a test file:


 puppet --debug --verbose test.pp

grep mysql
Check if mysql is installed: `dpkg --list`

Tracing the value of a variable
`notice("The value is: ${varName}")`

Logging the value of a variable
`notify{"The value is: ${varName}": }`

Logging the value of a variable
`$arr = [1, 2, 3] define print() { notice("The value is: '${name}'") } print{$arr:`

Dry run

This is how to ask puppet to list it's actions instead of running them


puppetd –test –noop

problem with ganglia
err: /Stage[main]/Ganglia/Service[gmond]/ensure: change from stopped to running failed: Could not start Service[gmond]: Execution of '/etc/init.d/ganglia-monitor start' returned 1: at /etc/puppet/manifests/ganglia.pp:241

solution
sudo apt-get install ganglia-monitor

Dependencies
One main source of problems is that puppet will install and run things out-of-order and unless the dependencies are defined explicitly it will break things.

Hint
add explicit definitions

Solution
is to use the require and subscribe directives.

Java
The most common difficulty is install java automatically under Ubuntu, because apt requires user interaction to accept the license agreement.

Hint
apt will also accept an appropriate file instead.

Solution

The exec below will lay down the file indicating acceptance (via require). Same methodology can be used for other packages which enforce user interaction.


 case $operatingsystem {
   debian,ubuntu: {
     exec {"pre-accept-sun-dlj":
       path => "/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin",
       cwd => "/tmp",
       command => 'echo "sun-java6-bin shared/accepted-sun-dlj-v1-1 boolean true"

Trouble Shooting & Error Logs

Finding and fixing errors

TODO: Add error log location

Style

quote resource names
quote parameter values which are not reserved words in Puppet.
include curly braces {} around variable names when referring to them in strings.
end lines that declare parameters with a comma.
when declaring a resource with a single parameter, use a single line declaration without a trailing comma.
multiple parameters of a single resource are placed each on their own lines.
arrows should be lined up within each resource.
more at http://projects.puppetlabs.com/projects/puppet/wiki/Style_Guide

Discussion

Any questions or would you like to take the test?