How I use Puppet at work

These are my notes for a presentation at my university on how I am using Puppet at work.

About me
Studying at Haaga-Helia (BITe)
Linux project on Automated System Configuration Management (with Cobbler and Fabric)
http://awaseconfigurations.wordpress.com/
https://github.com/AwaseConfigurations/main

Company
http://www.conformiq.com/
http://www.conformiq.com/products/conformiq-designer/
Infrastructure (rough):
– 15 / 12 / 0
– 25 / 20 / 15

Responsibilities
– Infrastructure development / maintenance (most recent)
— Setup Nagios
— Setup apt-cacher
— Web-app development (php)
— virtualization..
— pbx
– Support our support
— Mimicikng environments (e.g. RHEL3 + WebShpere 5.1.1.8 + HP Quality Center 9.2), small automation scripts
– SW licenses, HW procurement / installation / maintenance, backups etc.

Why puppet?
– Centralized configuration management
— Less manual work
— Less ‘human’ mistakes
— More time for other tasks
— Logs, documentation

What we do with puppet? (examples in cases)

Case A: Deploy new machines
– prepare dhcp and dns entries for the new machine
– automated installation of base system (Debian) from CD / .iso (with preseed (puppet package and dependencies included))
– issue / sign puppet certs
– pull the configuration
— packages
— services and their configuration
— users, mounts etc.

Case B: Add new user
We have a setup where users home directories on all the Debian servers are mounted from one certain server over the network. Also on each server there are local directories for each user. So, to add a new user I:
– sudo adduser theuser
– grep theuser’s uid, password hash and add them in my user-related modules, which look similar to these:

# /etc/puppet/modules/users/manifests/init.pp

class users {
    include users::virtual
}

class users::virtual {
        define localuser ($uid,$gid,$pass,$shell="/bin/bash",$home="/home/$title") {
                user { $title:
                        ensure  =>      "present",
                        uid     =>      $uid,
                        groups  =>      $gid,
                        shell   =>      $shell,
                        home    =>      "$home",
                        comment =>      $realname,
                        password =>     $pass,
                        managehome =>   true,
                        require => Class["home_mounted"],
                }

class mysite::users {
        include users::virtual

        @users::virtual::localuser { "armen":
                uid     =>      "1003",
                gid     =>      ["users","sudo"],
                pass    =>      '$6$4sB7vCY4$ywzet3qSIQYa/pUikJjaO00VEp6n2koifoo/'
        }
        @users::virtual::localuser { "brmen":
                uid     =>      "1004",
                gid     =>      ["users","sudo"],
                pass    =>      '$6$4sB7vCYnKzTVuG6pgFF7aVRnG64NGfMN5KzZl8UACbar/',
                shell   =>      "/bin/zsh",
        }
}
# /etc/puppet/modules/userdirs/manifests/init.pp

class userdirs {
        file{ "/data/users":
                ensure => "directory",
                owner => "root",
                group => "users",
                mode => "755",
                require => Class["users"]
        }

        file{ "/data/users/armen":
                ensure => "directory",
                owner => "armen",
                group => "users",
                mode => "755"
        }       
        file{ "/data/users/brmen":
                ensure => "directory",
                owner => "brmen",
                group => "users",
                mode => "755"
        }
}

- to make sure that home directories are mounted over nfs I am using a module like this:

# /etc/puppet/modules/home_mounted/manifests/init.pp

class home_mounted { 
    mount { "/home": 
        device  => "thenfsserver:/home", 
        fstype  => "nfs", 
        ensure  => "mounted", 
        options => "defaults", 
        atboot  => "true", 
    } 
}

- (these modules are then applied in the nodes.pp as foll):

# /etc/puppet/manifests/nodes.pp

node 'lin1.conformiq.com' {
    include users
    include home_mounted
}

class users {
    include users
    include mysite::users
    realize (
    Users::Virtual::Localuser[ "armen", "brmen", ],
    )
}

Case C: Install and configure new service, say nagios-nrpe-server (recent)
– Manually install and configure on one machine (take notes), make sure it works with the monitoring server

ssh lin1
sudo apt-get install nagios-nrpe-server nagios-plugins    # installs the packages
sudo vim /etc/nagios/nrpe.cfg       # edit the config file as needed
sudo /etc/init.d/nagios-nrpe-server restart       # restart the service

- Based on above write a module:

# /etc/puppet/modules/nagios_client/manifests/init.pp

class nagios_client {
	include nagios_client::install, nagios_client::config, nagios_client::service
}
class nagios_client::install {
	package { ["nagios-nrpe-server", "nagios-plugins"]:
		ensure => present,
	}
}
class nagios_client::config {
	file { "/etc/nagios/nrpe.cfg":
		ensure => present,
		owner => 'root',
		group => 'root',
		mode => 0644,
		source => "puppet:///modules/nagios_client/etc/nagios/nrpe.cfg",
		require => Class["nagios_client::install"],
		notify => Class["nagios_client::service"],
	}
}
class nagios_client::service {
	service { "nagios-nrpe-server":
		ensure => running,
		hasstatus => true,
		hasrestart => true,
		enable => true,
		require => Class["nagios_client::config"],
	}
}

- include the edited configuration file to the module
– include the module to the nodes.pp

Case D: Someone in r&d wants to add a package to the pool of packages that need to be present on the servers.
– That person (or me) edits the module which makes sure that packages are installed.

Case E: Someone in r&d writes a module to limit memory address space for a certain process/user. I develop it further.
– initial module can look like this:

# /etc/puppet/modules/limits/manifests/init.pp

class limits {
    file { "/etc/security/limits.conf":
        ensure => "file",
        owner => "root",
        group => "root",
        mode => 0644,
        content => '
# limit address space of any process spawned by user jenkins-slave to 30GB max
jenkins-slave hard as 31457280
',
    }
}

- because we have machines with various amounts of RAM, the hard-coded 30GB limit will not always help, so I rewrite the module:

# /etc/puppet/modules/limits/manifests/init.pp

class limits {
    file { "/etc/security/limits.conf":
        ensure => "file",
        owner => "root",
        group => "root",
        mode => 0644,
        content => template("limits/limits.tmpl")
    }
}

- the content of the file goes into the template which does the calculation:

# /etc/puppet/modules/limits/templates/limits.tmpl

# limit address space of any process spawned by user jenkins-slave to 95% or total RAM

jenkins-slave hard as <%= "%.0f" % [memorysizeinbytes.to_f / 1024 * 0.95] %>

- memorysizeinbytes is derived from a custom fact written by a third party:

# /etc/puppet$ cat modules/limits/lib/facter/meminbytes.rb 
# meminbytes.rb
# Additional Facts for memory/swap usage in bytes
#
# Original file:
# Copyright (C) 2006 Mooter Media Ltd
# Author: Matthew Palmer 
#
# Modifications by: Rutger Spiertz, Kumina BV
#
require 'thread'

{   :MemorySizeInBytes => "MemTotal",
    :MemoryFreeInBytes => "MemFree",
    :SwapSizeInBytes   => "SwapTotal",
    :SwapFreeInBytes   => "SwapFree"
}.each do |fact, name|
    Facter.add(fact) do
        confine :kernel => :linux
        setcode do
            meminfo_number(name)
        end
    end
end

def meminfo_number(tag)
    memsize = ""
    Thread::exclusive do
        size, scale = [0, ""]
        File.readlines("/proc/meminfo").each do |l|
            size, scale = [$1.to_f, $2] if l =~ /^#{tag}:\s+(\d+)\s+(\S+)/
            # MemoryFree == memfree + cached + buffers
            #  (assume scales are all the same as memfree)
            if tag == "MemFree" &&
                l =~ /^(?:Buffers|Cached):\s+(\d+)\s+(?:\S+)/
                size += $1.to_f
            end
        end
        memsize = scale_number(size, scale)
    end

    memsize
end

def scale_number(size, multiplier)
    suffixes = ['', 'kB', 'MB', 'GB', 'TB']
    size *= 1024 ** suffixes.index(multiplier)
    return "%.0f" % size
end

Still to do:
– set up testing (pre-production) environment
– follow the liveliness of puppet’s service in nagios

About these ads

2 responses to “How I use Puppet at work

  1. armen movsesjans November 16, 2012 at 10:21 am

    During the presentation there was a question about looking into the source code of puppet modules. I did not understand the question correctly, so here comes the authoritative answer. No, I am not looking at the source code of the 3rd party modules since I have not been using any. All the modules that I have been using until now are the “internal” puppet functionality shipped with the Puppet. Thank you for this brilliant question, it implies the concern before stupid stuff happens!

  2. Rahul.Patilul Patil November 16, 2013 at 5:04 pm

    Nice post!!… thanks for sharing your experience..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: