Configuring GridDB with Puppet

Configuring many nodes can be very challenging with lots of manual work — even just one small typo on one node can lead to hours of debugging. Puppet and other configuration management systems solve this by deploying configurations to multiple nodes thus ensuring that the values are consistent while also helping to prevent human error. We’ve created a Puppet module available here: https://griddb.net/en/resources/griddb-puppet-module.tgz (and soon on puppetforge) that can configure GridDB Standard Edition with either MULTICAST or FIXED_LIST modes.

First a bit of terminology, a puppet master is both the server and the process that stores all of the configuration management settings. A puppet agent is the process that connects to the master and then applies the configuration. A puppet module is a third party set of definitions and templates that enable the end user to use puppet with a particular application. Hiera are YAML definitions that can be referenced within Puppet recipes.

To deploy our puppet module, we’re going to start a fresh installation of GridDB on Centos 7 on three nodes, essentially what the Amazon AWS instances are. One of the nodes will be the puppet master where the GridDB configuration is stored and needs to be accessible to the other hosts by its fully qualified domain name:

Installing and configuring Puppet

On all three hosts:

# echo "192.168.1.77 griddb1.example.com" >> /etc/hosts

To install Puppet, we install Puppetlabs Repo and then Puppet.

# rpm -Uvh https://yum.puppetlabs.com/puppet5/puppet5-release-el-7.noarch.rpm

On the master:

# yum -y install puppetserver

On the other nodes:

# yum -y install puppet-agent

Start the puppet master:

# systemctl enable puppetmaster
# systemctl start puppetmaster

You will need to either disable the firewall or allow access to port 8140 for the puppet agents to connect to the master:

# firewall-cmd --zone=public --port 8140/tcp

Now that puppet is installed we can configure the agents. The hostname of the master needs to be added to /etc/puppetlabs/puppet/puppet.conf, the snippet looks like the following:

[main]
server=griddb1.example.com

Now we can generate the certificates for the puppet agents, run the following command on all three nodes:

# puppet agent --test

Then on the puppet master, you can run the follow commands to sign the certificates:

# puppet cert sign griddb1.example.com
# puppet cert sign griddb2.example.com
# puppet cert sign griddb3.example.com

Now, you can re-run “puppet agent –test” on all the nodes which should have the following output:

Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for griddb1.example.com
Info: Applying configuration version '1503328741'
Notice: Applied catalog in 0.20 seconds

Now that the agent is also working, the background process can also be enabled:

# systemctl enable puppet
# systemctl start puppet

Now puppet is set up on your system and we can set up the GridDB module.

Setting Up GridDB with Puppet

First download the GridDB puppet module from https://griddb.net/en/resources/griddb-puppet-module.tgz and place its unzipped contents in /etc/puppetlabs/code/environments/production/modules/.

We will then edit /etc/puppetlabs/code/environment/production/data/common.yaml to add all of our GridDB settings. If your network supports multicast networking, you can set griddb_mode to “multi” or use “fixed” as needed on most Cloud services or with containers as follows:

griddb_clusterName: "defaultCluster"
griddb_user: "admin"
griddb_pass: "admin"
griddb_minNodeNum: "3"
griddb_replicationNum: "2"
griddb_concurrency: "2"
griddb_storeMemoryLimit: "1024MB"
griddb_mode: "fixed"
griddb_nodes: 
        - "192.168.1.74"
        - "192.168.1.75"
        - "192.168.1.77"

Since common.yaml will contain the GridDB username and password, it is a good idea to lockdown its permissions:

# chown puppet.puppet common.yaml
# chmod 640 common.yaml

Since the actual password file is machine generated by gs_passwd, we first create it and then copy it into the modules templates so that it can be propagated to the other nodes:

# su - gsadm
$ gs_passwd admin
Password: 
Retype password:
$ exit
# cp /var/lib/gridstore/conf/password /etc/puppet/code/environment/production/modules/griddb/templates/password.erb

The last thing to do is either disable the firewall or configure it to allow access on the required ports.

# systemctl stop firewalld

or

# firewall-cmd --zone=public --port 10001/tcp
# firewall-cmd --zone=public --port 10010/tcp
# firewall-cmd --zone=public --port 10020/tcp
# firewall-cmd --zone=public --port 10040/tcp

Now, if you invoke “puppet agent –test” on all nodes simultaneously, Puppet will deploy the configuration files and then start the Gridstore service on all of the nodes. The above instructions will work using the GridDB SE in both local or cloud environments and can also be adapted for use with GridDB CE. The only additional step you will need to use with Community Edition is to disable the “service” section from the GridDB puppet manifest and manually run gs_startnode and gs_joincluster.

 

 

If you have any questions about the blog, please create a Stack Overflow post here https://stackoverflow.com/questions/ask?tags=griddb .
Make sure that you use the “griddb” tag so our engineers can quickly reply to your questions.