MCollective Puppet Configuration


Install and configure Puppet using the following link 
http://zaman4linux.blogspot.in/2014/11/quick-howto-puppet-installation-and.html

Generate/Signing of  Certificates 
http://zaman4linux.blogspot.in/2016/03/installing-and-configuring-puppet-from.html

How MCollective Works:
MCollective uses publish/subscribe middleware to transport requests between clients and servers. The mcollectived server registers with the middleware broker and remains in a listening or IDLE state. Whenever a client sends a request to the middleware, each server receives and evaluates the request immediately and independently. mcollectived validates the request and then hands it off to an agent to process the request. The agent processes the request and sends the reply back.  

Configuring MCollective : 

Puppet Server will  act as MCollective Client and All Puppet Client will act as MCollective Servers.

# puppet cert generate "mcollective-client"
Notice: mcollective-client has a waiting certificate request
Notice: Signed certificate request for mcollective-client
Notice: Removing file Puppet::SSL::CertificateRequest mcollective-client at '/etc/puppetlabs/puppet/ssl/ca/requests/mcollective-client.pem'
Notice: Removing file Puppet::SSL::CertificateRequest mcollective-client at '/etc/puppetlabs/puppet/ssl/certificate_requests/mcollective-client.pem'

# puppet cert generate "mcollective-servers"
Notice: mcollective-servers has a waiting certificate request
Notice: Signed certificate request for mcollective-servers
Notice: Removing file Puppet::SSL::CertificateRequest mcollective-servers at '/etc/puppetlabs/puppet/ssl/ca/requests/mcollective-servers.pem'
Notice: Removing file Puppet::SSL::CertificateRequest mcollective-servers at '/etc/puppetlabs/puppet/ssl/certificate_requests/mcollective-servers.pem'


# ls -l /etc/puppetlabs/puppet/ssl/certs/
total 20
-rw-r--r-- 1 puppet puppet 1980 Jun 10 19:53 activemq.example.com.pem
-rw-r--r-- 1 puppet puppet 1992 Apr 15 11:22 ca.pem
-rw-r--r-- 1 puppet puppet 1972 Jun 10 19:57 mcollective-client.pem
-rw-r--r-- 1 puppet puppet 1976 Jun 10 19:59 mcollective-servers.pem


# mkdir server_private ; mkdir server_public ; mkdir clients

# cp /etc/puppetlabs/puppet/ssl/private_keys/mcollective-servers.pem /etc/puppetlabs/mcollective/server_private/
# cp /etc/puppetlabs/puppet/ssl/public_keys/mcollective-servers.pem /etc/puppetlabs/mcollective/server_public/
# cp /etc/puppetlabs/puppet/ssl/certs/mcollective-client.pem /etc/puppetlabs/mcollective/clients/




Copy MCollective certificates to Agent Nodes 


The MCollective certificates generated above need to be copied to each of the agent nodes as shown below

scp /etc/puppetlabs/mcollective/clients/mcollective-client.pem bzaman@node-znxd-003:/etc/puppetlabs/mcollective/clients/
scp /etc/puppetlabs/mcollective/server_private/mcollective-servers.pem bzaman@node-znxd-003:/etc/puppetlabs/mcollective/server_private/
scp /etc/puppetlabs/mcollective/server_public/mcollective-servers.pem bzaman@node-znxd-003:/etc/puppetlabs/mcollective/server_public/


Copy Host Certificates from Puppet Server to Agent Nodes
  
Copy the certificates for each of  the Agent Nodes from /etc/puppetlabs/puppet/ssl/ca/signed/  location on Puppet Server to  /etc/puppetlabs/puppet/ssl/certs/  on each of the Puppet Agent.



[user@puppetserver]$ sudo ls -l /etc/puppetlabs/puppet/ssl/ca/signed/
total 48
-rw-r--r-- 1 puppet puppet 1960 Jun 27 15:56 activemq.example.com.pem
-rw-r--r-- 1 puppet puppet 2000 Jun 27 22:30 puppetserver.pem
-rw-r--r-- 1 puppet puppet 1956 Jun 27 16:34 mcollective-client.pem
-rw-r--r-- 1 puppet puppet 1956 Jun 27 16:35 mcollective-servers.pem
-rw-r--r-- 1 puppet puppet 1944 Jun 30 11:38 node-znxd-003.pem
-rw-r--r-- 1 puppet puppet 1944 Jun 30 16:23 node-znxd-003.pem
-rw-r--r-- 1 puppet puppet 1944 Jun 30 16:41 node-znxq-001.pem
-rw-r--r-- 1 puppet puppet 1944 Jun 30 16:56 node-znxq-004.pem




[user@node-znxd-003]$ ls -l /etc/puppetlabs/puppet/ssl/certs
total 8
-rw-r--r-- 1 puppet puppet 1988 Mar 23 13:21 ca.pem
-rw-r--r-- 1 puppet puppet 1976 Mar 23 13:24 node-znxd-003.example.com

In the above example , node-znxd-003.pem from PuppetServer is copied to /etc/puppetlabs/puppet/ssl/certs on Agent Nodes. This needs to be followed for each of the agent Nodes. 

Configure MCollective Server Configuration file 

Update MCollective server.cfg (/etc/puppetlabs/mcollective/server.cfg ) on all  Puppet Agent Nodes with the following entries.

connector = activemq
plugin.activemq.pool.size = 1
plugin.activemq.pool.1.host =
plugin.activemq.pool.1.port = 61614
plugin.activemq.pool.1.user =mcollective
plugin.activemq.pool.1.password =  
plugin.activemq.pool.1.ssl = true
plugin.activemq.pool.1.ssl.ca = /etc/puppetlabs/puppet/ssl/certs/ca.pem
plugin.activemq.pool.1.ssl.cert =/etc/puppetlabs/puppet/ssl/certs/node1.pem
plugin.activemq.pool.1.ssl.key =/etc/puppetlabs/puppet/ssl/private_keys/node1.pem
plugin.activemq.pool.1.ssl.fallback = false

# SSL security plugin settings:
securityprovider = ssl
plugin.ssl_client_cert_dir = /etc/puppetlabs/mcollective/clients
plugin.ssl_server_private = /etc/puppetlabs/mcollective/server_private/mcollective-servers.pem
plugin.ssl_server_public = /etc/puppetlabs/mcollective/server_public/mcollective-servers.pem



Configure MCollective Client Configuration file
 
Update MCollective client.cfg ( /etc/puppetlabs/mcollective/client.cfg )  on the Puppet Server which is MCollective Client  

connector = activemq
plugin.activemq.heartbeat_interval = 30
plugin.activemq.pool.size = 1
plugin.activemq.pool.1.host =

plugin.activemq.pool.1.port = 61614
plugin.activemq.pool.1.user = mcollective
plugin.activemq.pool.1.password =
 
plugin.activemq.pool.1.ssl = true
plugin.activemq.pool.1.ssl.ca = /etc/puppetlabs/puppet/ssl/certs/ca.pem
plugin.activemq.pool.1.ssl.cert = /etc/puppetlabs/puppet/ssl/certs/mcollective-client.pem
plugin.activemq.pool.1.ssl.key = /etc/puppetlabs/puppet/ssl/private_keys/mcollective-client.pem
plugin.activemq.pool.1.ssl.fallback = false

# SSL security plugin settings:
securityprovider = ssl
plugin.ssl_server_public = /etc/puppetlabs/puppet/ssl/certs/mcollective-servers.pem
plugin.ssl_client_private = /etc/puppetlabs/puppet/ssl/private_keys/mcollective-client.pem
plugin.ssl_client_public = /etc/puppetlabs/puppet/ssl/certs/mcollective-client.pem


Installing MCollective Plugins

Install the following MCollective plugins on all Nodes including Puppet Client and Server. 

Puppet Server:

mcollective-puppet-client
mcollective-service-agent
mcollective-puppet-agent


Puppet Client:

mcollective-service-agent
mcollective-puppet-agent


For installing the plugins enable the following repos ( puppetlabs.repo ) :

$ rpm -qf /etc/yum.repos.d/puppetlabs.repo
puppetlabs-release-6-11.noarch
$ rpm -qf /etc/yum.repos.d/puppetlabs-pc1.repo
puppetlabs-release-pc1-1.0.0-1.el6.noarch
 

or download from the below Puppet Repository

https://yum.puppetlabs.com/el/6.5/products/x86_64/


Starting MCollective

Start mcollective on both Puppet Agent/Server using the below command

                        /etc/init.d/mcollective restart
           
Status is as follows:

MCollective Server:

$ ps -ef | grep mcollective
         root     11950     1  0 Jul10 ?        00:02:20 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived --pid=/var/run/puppetlabs/mcollectived.pid --config=/etc/puppetlabs/mcollective/server.cfg --daemonize

MCollective Client:

$  ps -ef | grep mcollective
root     23881     1  0 Jul10 ?        00:01:50 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived --pid=/var/run/puppetlabs/mcollectived.pid --config=/etc/puppetlabs/mcollective/server.cfg --daemonize

For troubleshooting  issues , check MCollective Log  file

              tail -f /var/log/puppetlabs/mcollective.log


Testing MCollective

# mco ping
Node-ZNXD-003.example.com                   time=102.68 ms
Node-ZNXD-002.
example.com                   time=108.90 ms
Node-ZNXQ-010.
example.com                   time=110.91 ms
Node-ZNXQ-003.
example.com                   time=116.95 ms
Node-ZNXQ-002.
example.com                   time=126.92 ms
Node-ZNXQ-009.
example.com                   time=128.95 ms

---- ping statistics ----
6 replies max: 128.95 min: 102.68 avg: 115.88


All the puppet agent nodes should respond , we are fine then .


# mco rpc service status service=cups
Discovering hosts using the mc method for 2 second(s) .... 6
 * [ ============================================================> ] 6 / 6
Node-ZNXQ-003.example.com
   Service Status: running
Node-ZNXD-003.example.com
   Service Status: running
Node-ZNXQ-010.example.com
   Service Status: running
Node-ZNXQ-009.example.com
   Service Status: running
Node-ZNXD-002.example.com
   Service Status: running
Node-ZNXQ-002.example.com
   Service Status: running

Summary of Service Status:
   running = 6
Finished processing 6 / 6 hosts in 102.94 ms

Above we are checking the status of cups service using discovery method. In the above example , we checked the status for all Nodes. If we want to check the status for the hosts which ends with ZNXQ, then we can use host filter as follows:

# mco rpc service status service=cups  -I /ZNXD/
Discovering hosts using the mc method for 2 second(s) .... 2
 * [ ============================================================> ] 2 / 2
Node-ZNXD-003.example.com
   Service Status: running
Node-ZNXD-002.example.com
   Service Status: running

Summary of Service Status:
   running = 2
Finished processing 2 / 2 hosts in 139.99 ms


Applying Puppet Manifests

Write a temporary manifests to create a test file to test the setup


# cat /data/puppetlabs/code/environments/production/manifests/site.pp

node default{

}

node "node-znxd-001.pem","node-znxd-002.pem","node-znxd-003.pem","node-znxd-004.pem" {

            file { "/tmp/puppet.txt":

                             ensure =>; file,

                             owner  =>; "root",

                             group  =>; "root",

                             mode   =>; "0644",

                             content =>; "Node1.example.com"

                   }

           }


Now , apply the manifest using the below command




# mco puppet runonce
* [ ============================================================> ] 8 / 8
 Finished processing 6 / 6 hosts in 104.50 ms



Comments

Popular posts from this blog

PSSH : Parallel SSH to execute commands on a number of hosts

How to add check_http as a service in Nagios Monitoring using NRPE

Configuring Nagios to monitor services using NRPE