Showing posts from 2016

Rsync: Transferring files between two hosts

Rsync is a versatile file copying tool. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync also has options to delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized.

Configuring Rsync:

rsync -av --exclude 'share/web/moodle/sessions/' --exclude 'share/wordpress/uploads/wpcf7_captcha/' --delete --rsync-path "sudo rsync" /data/ 172.31.X.X:/data/ &>/home/fsync/rsyncjob/output.

Options Description:

--archive : Copies recursively (all directories and  
            subdirectories) while preserving symbolic links, 
            permissions, file ownerships and timestamps.

--exclude : This option allows you to add rules to selectively  
            exclude certain files from the list of files to be   
            transferred …

Puppet 4 : Using Metaparameters

Puppet Meta-parameters are attributes that work with any resource type, including custom types and defined types.

Meta-parameters do things like
 -> add metadata to a resource (alias, tag),
 -> set limits on when the resource should be synced
 -> prevent Puppet from making changes (noop), 
 -> and change logging verbosity (loglevel).

For resource ordering, puppet offers the meta-parameters, require and before. Both take one or more references to a declared resource as their value. Puppet references have a special syntax, as was discussed in previous post.


Let's take an example below to understand resource ordering

package { 'apache':
       ensure => 'installed',

   file {'/etc/httpd/httpd.conf':
      ensure => file,
      owner  => 'root',
      group  => 'root',
      mode   => '0644',
      source => 'puppet:///modules/apache/ht…

Puppet Fundamentals:Manifests and Resources

Puppet Fundamentals:Manifests and Resources


Puppet is driven by manifests, the equivalent of scripts or programs, written in Puppet'sdomain-specific language (DSL).
Let's start with the obligatory Hello, world! manifest:

# hello_world.pp notify { 'Hello, world!': }
To put the manifest to work, use the following command with root or with sudo . To control system processes, boot options, and the like, Puppet needs to be run with root privileges .

root@puppetmaster:~# puppet apply hello_world.ppNotice: Compiled catalog for in environment production in 0.45 secondsNotice: Hello, world!Notice: /Stage[main]/Main/Notify[Hello, world!]/message: defined 'message' as 'Hello, world!'Notice: Applied catalog in 0.03 secondsLet's interpret the above output:Notice => This keyword at the beginning of the line represents the log level.  Other levels include Warning, Error, and Debug Main      => Defau…

Puppet: Writing multiple manifest for multiple clients

How to use multiple manifests in Puppet ?

Puppet always starts compiling with either a single manifest file or a directory of manifests that get treated like a single file. This main starting point is called the main manifest or site manifest.

If we need to write multiple files, then for  earlier versions of Puppet , we can use the import function to call  the other manifests in site.pp

import "../classes/*"
             import "../nodes/*"

In Puppet 4, the import function is deprecated . I was getting the error below

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Use of 'import' has been discontinued in favor of a manifest directory. See at /apps/wps/puppetlabs/code/environments/production/manifests/site.pp:2:1 on node

The solution is to put all the main manifests in Puppet manifests directory as per output of below command. 

# puppet con…

Understanding Puppet Modules

Using Modules

Modules are self-contained bundles of code and data. You can download pre-built modules from the Puppet Forge or you can write your own modules .

Modules are how Puppet finds the classes and types it can use — it automatically loads any class or defined type stored in its modules. Any of these classes or defines can be declared by name within a manifest or from an external node classifier (ENC) . 

Module layoutOn disk, a module is simply a directory tree with a specific, predictable structure:
manifestsfilestemplateslibfacts.dexamplesspec files/ — Contains static files, which managed nodes can download.
service.conf— This file’s source => URL would be puppet:///modules/my_module/service.conf. Its contents can also be accessed with the file function, like content => file('my_module/service.conf'). Puppet will be looking for service.conf in  ${codedir}/environments/production/modules/my_module/files/service.conf . templates/-- Contains templates, which th…

Using fileserver in Puppet

Puppet Server includes a file server for transferring static file content to agents; this is what’s used whenever a file resource has a source => puppet:///... attribute specified.
 Mostly files can be served via modules but if we need to serve larger files that shouldn’t be with a module, you can make a custom file server mount point and let Puppet serve those files from another directory.

Creating a File Server:

1. Create a directory which puppet Can access and copy files which needs to be exported  to this directory. 

2. Create filserver.conf  and define directory and hosts which can access this directory. 

$ touch   /etc/puppetlabs/puppet/fileserver.conf

Defining File Server:

fileserver.conf uses a one-off format that resembles an INI file without the equals (=) signs. It is a series of mount-point stanzas, where each stanza consists of:

A [mount_point_name] surrounded by square brackets. This will become the name used in puppet:/// URLs for files in this mount point.A path directiv…

Puppet manifests to automate Tomcat Deployment

To automate tomcat deployments using Puppet , we have to write manifests for it. Our tomcat deployments uses the following steps

1. unlink war from existing war
2. link to latest war
3. restart application

/data/dist/app is the location for us which contains all the war files.Once a new war file is available we copy the war file to this location and update the link from current war to the latest war. The location which contains the current working war is/data/tomcat/app/deploy/ .

$ls  -lrt  /data/tomcat/jacplus/deploy/
drwxrwsr-x  5 tomcat  admin 4096 May 22  2012 manager
drwxrwsr-x  4 tomcat  admin 4096 Oct 23  2012 whoami
lrwxrwxrwx  1 tomcat  admin    7    Dec 12  2012 ROOT -> app
lrwxrwxrwx  1 user      admin   35   Jun  1    23:11 app.war -> /data/dist/app/app-4.41.war

So, during deployment we unlink current war from /data/tomcat/jacplus/deploy/ and point it to latest war in /data/dist/app/directory.

$ cd /data/tomcat/jacplus/deploy/
          $ unlink app.war
          $ …

MCollective Puppet Configuration

Install and configure Puppet using the following link

Generate/Signing of  Certificates

How MCollective Works:
MCollective uses publish/subscribe middleware to transport requests between clients and servers. The mcollectived server registers with the middleware broker and remains in a listening or IDLE state. Whenever a client sends a request to the middleware, each server receives and evaluates the request immediately and independently. mcollectived validates the request and then hands it off to an agent to process the request. The agent processes the request and sends the reply back.  

Configuring MCollective : 

Puppet Server will  act as MCollective Client and All Puppet Client will act as MCollective Servers.

# puppet cert generate "mcollective-client"
Notice: mcollective-client has a waiting certificate requ…