GridDB quick start guide

Revision: 1538

Table of Contents

1 Introduction

1.1 Aim and composition of this document

This manual describes the basic operating procedure of GridDB.

It is written for system designer and system administrators responsible for GridDB system's construction and operation management respectively.

Two main subjects are covered in this manual:

  • System design and setup
    • Basic setup of GridDB post-installation is explained in this section.
  • Operation
    • Basic operation such as starting and stopping of GridDB, operation management, and troubleshooting are explained in this section.

1.2 What is GridDB?

1.2.1 Overview

GridDB is a distributed NoSQL database that manages a group of data called a row, which contains a key and multiple values. There are currently three commercial version of GridDB in offer.

  • GridDB Standard Edition
  • GridDB Advanced Edition
  • GridDB Vector Edition

GridDB Advanced Edition adds SQL processing engine on top of the high performance offered in GridDB Standard Edition.

GridDB Vector Edition is a product added to GridDB AE for high speed matching of high dimensional vector data used for image recognition, machine learning in big data analysis and large scale media analysis.

Even though the library used during application development is different, system construction and operation remains the same.

GridDB’s merits are as follows:

  • Allows data management to be carried out in memory at a high speed.
    • Groups of rows are stored in memory, allowing high-speed updates and searches to be carried out.
  • Allows in-memory processing to be carried out and its capacity to be increased.
    • Capacity can be increased by dispersing and placing data in multiple machines. Furthermore, data management using disks is also possible (this operation is not covered in this manual). As a result, even when operating with individual nodes, the capacity can be increased without being limited by the memory size.
  • Able to achieve high availability.
    • Data is replicated within a cluster, thus allowing uninterrupted process in the case of a failure in any of the cluster’s node. In addition, each node continuously updates data to the disk, allowing restoration of data when a failure occurs.
  • Can scale out up to 1000 units.
    • In transactions involving container as the unit, high scalability is achieved by improving the degree of parallelism of the cluster process.
  • Does not require manual cluster management.
    • Communications are carried out using a decentralized protocol among mutual nodes, with cluster control carried out autonomously by GridDB itself.
  • Supports non-standard data used in social infrastructure.
    • It supports non-standard data such as time series data and spatial data etc. that is used in social infrastructure.
  • In the GridDB Advanced Edition / Vector Edition, ODBC/JDBC I/F is supported.
  • In GridDB Vector Edition, extended SQL that supports vector data operation is supported.

1.2.2 Features

The features of GridDB are summarized below.

RequirementsDescription
[Basic requirements]
Large capacity (petabyte class)In order to achieve both high speed and high capacity, data is stored in memory and SSD devices.
High speed (in-memory)In-memory processing
High scalabilityThe number of servers can be increased up to 1000 units
High availabilityAvailability is further improved by replicating data to HDD in multiple servers.
High autonomyAutomation of data replication and data placement balancing. Server expansion can be easily done online.
Operation control functionsMonitoring, security, backup etc.
[Social infrastructure requirements]
Time series dataA lossless compression function that could reduce the data size yet keeping the original features unchanged is provided.
Spatial data typeThe index can be tuned to increase the search speed. 2D and 3D data type are supported.
Consistency guaranteeSupports ACID transactions within the same container

1.3 Explanation of terms

The following table explains the GridDB related terms commonly found in this manual.

TermMeaning
NodeIndividual server process to perform data management in GridDB.
ClusterSingle or a set of nodes that perform data management together in an integrated manner.
PartitionLogical area where data is stored. It exists only within GridDB, and not visible directly to the user.
RowA chunk of data managed in GridDB, which consists of a key and multiple values.
ContainerVessel that manages a set of rows. 2 types of container exist; collection and time-series.
CollectionA type of container that manages rows containing general key.
Time-series containerA type of container that manages rows containing time-series key. It has a special function to handle time-series data.
Master nodeThe node that performs cluster management.
Follower nodeAll other nodes in the cluster other than the master node.
Owner nodeWithin the replicated containers, it records the location of the master container.
Backup nodeWithin the replicated containers, it records the location of the containers’ replica.

2 System design and construction

The basic system design and construction flow is shown in this chapter.

The design and construction of GridDB nodes and clusters are carried out as follows.

  1. Check that the required resources are sufficient
  2. Perform node installation and setup
  3. Set the environment-dependent parameters
  4. Set the tuning parameters
  5. Distribute the definition file to each node

For client setting, please refer to the following item.

2.1 Checking the required resources

GridDB is a scale-out database that unlike a conventional DB, careful system design and sizing is not required in order to achieve a non-stop operation. However, the following points should be considered as guidelines in the initial system design.

  • Memory usage
  • Number of nodes in a cluster
  • Disk usage

The estimation method is explained in sequence below.

However, functions to increase the capacity by using external storage devices such as SSDs etc. have not been considered in calculating the memory size below. Please check with our service desk for estimation if these functions are used.

2.1.1 Total memory usage

Predict the amount of data to be stored in the container and then estimate the memory usage.

First, predict the amount of data to be stored in the application. Estimate the following:

  • Data size of row
  • Number of rows to be registered

Next, estimate the memory required to store those estimated data. The formula to calculate the memory usage differs by container type, as shown below.

  • Collection
    • Memory usage (byte) = 600 × no. of containers + 1.8 × row size × no. of all rows + 24 × no. of all rows × no. of indices
  • Time-series container
    • Memory usage (byte) = 700 × no. of containers + 1.5 × row size × no. of all rows + 24 × no. of all rows × no. of indices

Perform a similar estimation for all containers created and used in the application. The total sum becomes the amount of memory required in the GridDB cluster.

  • Total memory usage = the sum of memory usage in all containers

However, please consider this number as the minimum requirement, as the memory usage also depends on update frequency.

2.1.2 No. of nodes constituting a cluster

Estimate the required no. of nodes used in GridDB. In the example below, it is assumed that one node is executed in one machine.

First, make an assumption of the memory required per machine.

  • Memory size of machine

In addition, make an assumption of the no. of replicas to create. The no. of replicas is given as a GridDB configuration value.

  • No. of replicas

Default no. of replicas is 2.

  • No. of nodes = (Total memory usage ÷ machine memory size) × no. of replicas (units)

However, please consider this as the minimum requirement as it is preferable to have a greater number of spare units to take into account the load distribution and availability improvement.

2.1.3 Disk usage

Estimate the size of the file to be created in GridDB, and the disk capacity required for the machine to execute a node. Two types of files should be created; a checkpoint file and a transaction log file.

The memory usage of individual node is determined as follows.

  • Individual memory usage = (total memory usage × no. of replicas) ÷ no. of nodes (byte)

The size of the checkpoint file is estimated as follows based on this numerical value.

  • File size = Individual memory usage × 2 (byte)

In addition, as the transaction log file size is dependent on the update frequency of the application, the following data is thus predicted.

  • Row update frequency (times/sec)

Furthermore, the checkpoint interval is assumed. Checkpoint interval is given as a GridDB configuration value.

  • Checkpoint interval

Default value of checkpoint interval is 1200 sec (20 minutes).

The transaction log file size is estimated as follows based on these numerical values.

  • File size = row data size × row update frequency × checkpoint interval (byte)

The individual disk usage is estimated as follows.

  • Individual disk usage = Transaction log file size + checkpoint file size

[Points to note]

  • The size of checkpoint file expands depending on the data capacity. However, please note that once it has expanded, the file size will not be reduced even if some of data in the container or row get deleted. Empty space can be re-used after data is deleted.

2.2 Installation and setup (node)

This chapter explains the installation of a node in a single machine. See the chapter on operations for the cluster configuration.

2.2.1 Environmental check

Check that the version is RHEL 6.2/6.3/6.4/6.5/6.6/6.7/6.8/7.2, or CentOS 6.2/6.3/6.4/6.5/6.6/6.7/6.8/7.2.

$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

[Points to note]

  • When choosing the package group during OS installation, please choose the minimum version or lower.
    • Basic Server

2.2.2 Installing a node

The following 3 RPM packages are needed when installing a GridDB node. Place these packages anywhere in the machine.

Package nameFile nameDescription
griddb-xx-servergriddb-xx-server-X.X.X-linux.x86_64.rpmThe start and other commands for the GridDB node module and server are included.
griddb-xx-clientgriddb-xx-client-X.X.X-linux.x86_64.rpmOne set of operating commands except start node is included.
griddb-xx-docsgriddb-xx-docs-X.X.X-linux.x86_64.rpmGridDB manual and program samples are included.
*: xx is the GridDB edition (se, ae, ve) *: X.X.X is the GridDB version

Install using the rpm command as a root user.

$ su
# rpm -Uvh griddb-se-server-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
User gsadm and group gridstore have been registered.
GridDB uses new user and group.
   1:griddb-se-server                 ########################################### [100%]
# rpm -Uvh griddb-se-client-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
User and group has already been registered correctly.
GridDB uses existing user and group.
   1:griddb-se-client                  ########################################### [100%]
# rpm -Uvh griddb-se-docs-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-se-docs                   ########################################### [100%]

When you install the package, the following group and user are created in the OS. This OS user is set as the operator of GridDB.

GroupUserHome directory
gridstoregsadm/var/lib/gridstore

The following environment variables are defined in this gsadm user.

Environment variablesValueMeaning
GS_HOME/var/lib/gridstoregsadm/GridDB home directory
GS_LOG/var/lib/gridstore/logEvent log file output directory

[Points to note]

  • These environment variables are used in the operating commands described later.
  • The password of the gsadm user has not been set. With the root user privilege, please set the password appropriately.
    • Some of the functions of the operation tools may be necessary.

In addition, when the GridDB node module is installed, services that are executed automatically upon startup of the OS will be registered.

Service nameRun level
gridstore3,4,5

The service registration data can be checked with the following command.

# /sbin/chkconfig --list | grep gridstore
gridstore 0:off 1:off 2:off 3:on 4:on 5:on 6:off

The GridDB node will be started automatically by this service during OS startup.

[Points to note]

  • Services will not start automatically immediately after installation.

To stop auto startup of a service, use the command below.

# /sbin/chkconfig gridstore off

To start the service, it is necessary to set the startup configuration file (/etc/sysconfig/gridstore/gridstore.conf). See the chapter on services in “GridDB Operational Management Guide” (GridDB_OperationGuide.html) for details of the services.

2.2.3 Post-installation checks

Check the directory configuration of the installed GridDB node.

First, check that the GridDB home directory, and related directory and files have been created.

GridDB home directory

/var/lib/gridstore/                      # GridDB home directory
                   admin/                # integrated operational management GUI home directory
                   backup/               # backup directory
                   conf/                 # definition file directory
                        gs_cluster.json  # cluster definition file
                        gs_node.json     # node definition file
                        password         # user definition file
                   data/                 # database file directory
                   log/                  # log directory

Confirm with the following command.

$ ls /var/lib/gridstore/
admin backup conf data log

Next, check that the installation directory has been created

Installation directory

/usr/gridstore-X.X.X/              # installation directory
                     Fixlist.pdf   # revision record
                     Readme.txt    # release instructions
                     bin/          # operation command, module directory
                     conf/         # definition file sample directory
                     docs/         # document directory
                     etc/
                     lib/          # library directory
                     license/      # license directory
                     prop/         # configuration file directory
                     web/          # integrated operational management GUI file directory

Confirm with the following command.

$ ls /usr/griddb/
Fixlist.pdf Readme.txt bin conf etc lib license prop web

All documents have been compressed into a single ZIP file. Decompress and refer to the documents where appropriate as shown below.

$ cd /usr/griddb/docs
$ unzip griddb-documents-X.X.X.zip

Lastly, confirm the version of the server module installed with the following command.

$ gsserver --version
GridDB version X.X.X build XXXXX ***** Edition

Points to note

The following files are created when GridDB is operated according to the following procedure.

[Database file]

/var/lib/gridstore/                     # GridDB home directory
                   data/                # database file directory
                        gs_log_n_m.log  # log file to record the transaction logs (n, m are numbers)
                        gs_cp_n_p.dat   # checkpoint file to record data regularly (n, p are numbers)

[Log file]

/var/lib/gridstore/                            # GridDB home directory
                   log/                        # log directory
                       gridstore-%Y%m%d-n.log  # event log file
                       gs_XXXX.log             # operation tool log file

The directory where these files are created can be changed by the parameter settings in the node definition file.

*: gs_XXXX is an operation tool name. (Example: gs_startnode.log)

2.2.4 Setting up an administrator user

Administrator privilege is used for authentication related matter within the nodes and clusters. Creation date of administrator user is saved in the user definition file. The default file is as shown below.

  • /var/lib/gridstore/conf/password

The default user below exists immediately after installation.

UserPasswordExample of proper use
adminadminFor authentication of operation administrator user, operation commands
systemmanagerFor authentication of application user, client execution

Administrator user information including the above-mentioned default users can be changed using the user administration command in the operating commands.

CommandFunction
gs_adduserAdd an administrator user
gs_deluserDelete an administrator user
gs_passwdChange the password of administrator user

Change the password as shown below when using a default user. The password is encrypted during registration.

$ gs_passwd admin
Password: (enter password)
Retype password: (re-enter password)

When adding a new administrator user except a default user, the user name has to start with gs#.

One or more ASCII alphanumeric characters and the underscore sign “_” can be used after gs#.

An example on adding a new administrator user is shown below.

$ gs_adduser gs#newuser
Password: (enter password)
Retype password: (re-enter password)

[Points to note]

  • A GridDB administrator user is different from the OS user gsadm created during installation.
  • A change in the administrator user information using a user administration command becomes valid when a node is restarted.
  • In order to use it for authentication purposes in the client, the same user data needs to be registered in all the nodes. Copy the user definition file and make sure the same user data can be referred to in all the nodes.
  • Execute the operating command as a gsadm user.

[Memo]

2.3 Setting the environment-dependent parameters

After installation, the following settings are necessary in order to operate GridDB.

  1. Network environment settings
  2. Cluster name settings

GridDB settings are configured by editing 2 types of definition files.

  • Cluster definition file (gs_cluster.json)
  • Node definition file (gs_node.json)

The cluster definition file defines the parameters that are common in the entire clusters.

The node definition files define the parameters for the different settings in each node.

These definition file samples are installed as follows.

/usr/griddb/                     # installation directory
               conf/                # definition file directory
                    gs_cluster.json # cluster definition file sample
                    gs_node.json    # node definition file sample

In a new installation, the same files are also placed in the conf directory under the GridDB home directory.

/var/lib/gridstore/                     # GridDB home directory
                   conf/                # definition file directory
                        gs_cluster.json # (edited) cluster definition file
                        gs_node.json    # (edited) node definition file

During operations, edit these definition files.

[Points to note]

  • When the GridDB version is upgraded, compare the newly installed sample with these definition files to adequately reflect the parameters added.
  • A cluster definition file defines the parameters that are common in the entire clusters. As a result, the settings must be the same in all of the nodes in the cluster. Nodes with different settings will get an error upon joining the cluster and prevented from participating in the cluster. Further details will be explained in the later chapter.

2.3.1 Network environment settings (essential)

First, set up the network environment.

An explanation of the recommended configuration method in an environment that allows a multicast to be used is given below. In an environment which does not allow a multicast to be used, or an environment in which communications between fellow nodes cannot be established in a multicast, a cluster configuration method other than the multicast method has to be used. See "GridDB Operational Management Guide" (GridDB_OperationGuide.html) for the details.

The configuration items can be broadly divided as follows.

  • (1) Address information serving as an interface with the client
  • (2) Address information for cluster administration and processing
  • (3) Address information serving as an interface with the JDBC client (GridDB Advanced Edition / Vector Edition only)

Although these settings need to be set to match the environment, basically default settings will also work.

However, an IP address derived in reverse from the host name of the machine needs to be an address that allows it to be connected from the outside regardless of whether the GridDB cluster has a multiple node configuration or a single node configuration.

Normally, this can be set by stating the host name and the corresponding IP address in the /etc/hosts file.

/etc/hosts setting

First, check with the following command to see whether the setting has been configured. If the IP address appears, it means that the setting has already been configured.

$ hostname -i
192.168.11.10

The setting has not been configured in the following cases.

$ hostname -i
hostname: no address corresponding to name

In addition, a loopback address that cannot be connected from the outside may appear.

$ hostname -i
127.0.0.1

If the setting has not been configured or if a loopback address appears, use the following example as a reference to configure /etc/hosts. The host name and IP address, and the appropriate network interface card (NIC) differ depending on the environment.

  1. Check the host name and IP address.
    $ hostname
    GS_HOST
    $ ip route | grep eth0 | cut -f 12 -d " " | tr -d "\n"
    192.168.11.10
    
  2. Add the IP address and corresponding host name checked by the root user to the /etc/hosts file.
    192.168.11.10 GS_HOST
    
  3. Check that the settings have been configured correctly.
    $ hostname -i
    192.168.11.10
    

*If the displayed setting remains the same as before, it means that a setting higher in priority is given in the /etc/hosts file. Change the priority order appropriately.

Proceed to the next setting after you have confirmed that /etc/hosts has been configured correctly.

(1) Address information serving as an interface with the client

In the address data serving as an interface with the client, there are configuration items in the node definition file and cluster definition file.

Node definition file

ParametersData typeMeaning
/transaction/serviceAddressstringReception address of transaction process
/transaction/servicePortstringReception port of transaction process
/system/serviceAddressstringConnection address of operation command
/system/servicePortstringConnection port of operation command

The reception address and port of transaction processes are used to connect individual client to the nodes in the cluster, and to request for the transaction process from the cluster. This address is used when configuring a cluster with a single node, but in the case where multiple nodes are present through API, the address is not used explicitly.

The connection address and port of the operating command are used to specify the process request destination of the operation command, as well as the repository information of the integrated operation control GUI.

These reception/connection addresses need not be set so long as there is no need to use/separate the use of multiple interfaces.

Cluster definition file

ParametersData typeMeaning
/transaction/notificationAddressstringInterface address between client and cluster
/transaction/notificationPortstringInterface port between client and cluster

A multi-cast address and port are specified in the interface address between a client and cluster. This is used by a GridDB cluster to send cluster information to its clients and for the clients to send processing requests via the API to the cluster. See the description of GridStoreFactory class/method in “GridDB API reference” (GridDB_API_Reference.html) for details.

It is also used as a connection destination address of the export/import tool, or as repository data of the integrated operation control GUI.

(2) Address information for cluster administration and processing

In the address data for a cluster to autonomously perform cluster administration and processing, there are configuration items in the node definition file and cluster definition file. These addresses are used internally by GridDB to exchange the heart beat (live check among clusters) and information among the clusters. These settings are not necessary so long as the address used is not duplicated with other systems on the same network or when using multiple network interface cards.

Node definition file

ParametersData typeMeaning
/cluster/serviceAddressstringReception address used for cluster administration
/cluster/servicePortstringReception port used for cluster administration

Cluster definition file

ParametersData typeMeaning
/cluster/notificationAddressstringMulticast address for cluster administration
/cluster/notificationPortstringMulticast port for cluster administration
  • Although a synchronization process is carried out with a replica when the cluster configuration is changed, a timeout time can be set for the process.
    • /sync/timeoutInterval

[Points to note]

  • An address or port that is not in use except in GridDB has to be set.
  • The same address can be set for the node definition file gs_node.json /transaction/serviceAddress, /system/serviceAddress, and /cluster/serviceAddress for operations to be performed. If a machine has multiple network interfaces, the bandwidth can be increased by assigning a separate address to each respective interface.

The following settings are applicable in the GridDB Advanced Edition / Vector Edition only.

(3) Address information serving as an interface with the JDBC client

In the address data serving as an interface with the JDBC/ODBC client, there are configuration items in the node definition file and cluster definition file.

Node definition file

ParametersData typeMeaning
/sql/serviceAddressstringReception address for JDBC/ODBC client connection
/sql/servicePortintReception port for JDBC/ODBC client connection

The reception address and port of JDBC/ODBC client connection are used to connect JDBC/ODBC individual client to the nodes in the cluster, and to access the cluster data in SQL. This address is used when configuring a cluster with a single node, but in the case where multiple nodes are present through API, the address is not used explicitly.

Cluster definition file

ParametersData typeMeaning
/sql/notificationAddressstringAddress for multi-cast distribution to JDBC/ODBC client
/sql/notificationPortintMulticast port to JDBC/ODBC client

The address and port used for multicast distribution to a JDBC/ODBC client are used for the GridDB cluster to notify the JDBC/ODBC client of cluster data, and to access the cluster data in SQL with the JDBC/ODBC client.

Refer to Annex Parameter List for the other parameters and default values.

2.3.2 Cluster name settings (essential)

Set the name of the cluster to be composed by the target nodes in advance. The name set will be checked to see if it matches the value specified in the command to compose the cluster. As a result, this prevents a different node and cluster from being composed when there is an error in specifying the command.

The cluster name is specified in the following configuration items of the cluster definition file.

Cluster definition file

ParametersData typeMeaning
/cluster/clusterNamestringName of cluster to create

[Points to note]

  • Node failed to start with default value ("").
  • A unique name on the sub-network is recommended.
  • A cluster name is a string composed of 1 or more ASCII alphanumeric characters and the underscore “_”. However, the first character cannot be a number. The name is also not case-sensitive. In addition, it has to be specified within 64 characters.

2.3.3 Settings of other cluster configuration methods

In an environment which does not allow the multicast method to be used, configure the cluster using the fixed list method or provider method. An explanation of the respective network settings in the fixed list method and provider method is given below.

When using the multicast method, proceed to Setting the tuning parameters.

(1) Fixed list method

When a fixed address list is given to start a node, the list is used to compose the cluster.

When composing a cluster using the fixed list method, configure the parameters in the cluster definition file.

Cluster definition file

ParametersData typeMeaning
/cluster/notificationMemberstringSpecify the address list when using the fixed list method as the cluster configuration method.

A configuration example of a cluster definition file is shown below.

{
                             :
                             :
    "cluster":{
        "clusterName":"yourClusterName",
        "replicationNum":2,
        "heartbeatInterval":"5s",
        "loadbalanceCheckInterval":"180s",
        "notificationMember": [
            {
                "cluster": {"address":"172.17.0.44", "port":10010},
                "sync": {"address":"172.17.0.44", "port":10020},
                "system": {"address":"172.17.0.44", "port":10040},
                "transaction": {"address":"172.17.0.44", "port":10001},
                "sql": {"address":"172.17.0.44", "port":20001}
            },
            {
                "cluster": {"address":"172.17.0.45", "port":10010},
                "sync": {"address":"172.17.0.45", "port":10020},
                "system": {"address":"172.17.0.45", "port":10040},
                "transaction": {"address":"172.17.0.45", "port":10001},
                "sql": {"address":"172.17.0.45", "port":20001}
            },
            {
                "cluster": {"address":"172.17.0.46", "port":10010},
                "sync": {"address":"172.17.0.46", "port":10020},
                "system": {"address":"172.17.0.46", "port":10040},
                "transaction": {"address":"172.17.0.46", "port":10001},
                "sql": {"address":"172.17.0.46", "port":20001}
            }
        ]
    },
                             :
                             :
}

(2) Provider method

Get the address list supplied by the address provider to perform cluster configuration.

When composing a cluster using the provider method, configure the parameters in the cluster definition file.

Cluster definition file

ParametersData typeMeaning
/cluster/notificationProvider/urlstringSpecify the URL of the address provider when using the provider method as the cluster configuration method.
/cluster/notificationProvider/updateIntervalstringSpecify the interval to get the list from the address provider. Specify a value that is 1s or higher and less than 2^31s.

A configuration example of a cluster definition file is shown below.

{
                             :
                             :
    "cluster":{
        "clusterName":"yourClusterName",
        "replicationNum":2,
        "heartbeatInterval":"5s",
        "loadbalanceCheckInterval":"180s",
        "notificationProvider":{
            "url":"http://example.com/notification/provider",
            "updateInterval":"30s"
        }
    },
                             :
                             :
}

The address provider can be configured as a Web service or as a static content. The specifications below need to be satisfied.

  • Compatible with the GET method.
  • When accessing the URL, the node address list of the cluster containing the cluster definition file in which the URL is written is returned as a response.
    • Response body: Same JSON as the contents of the node list specified in the fixed list method
    • Response header: Including Content-Type:application/json

An example of a response sent from the address provider is as follows.

$ curl http://example.com/notification/provider
[
    {
        "cluster": {"address":"172.17.0.44", "port":10010},
        "sync": {"address":"172.17.0.44", "port":10020},
        "system": {"address":"172.17.0.44", "port":10040},
        "transaction": {"address":"172.17.0.44", "port":10001},
        "sql": {"address":"172.17.0.44", "port":20001}
    },
    {
        "cluster": {"address":"172.17.0.45", "port":10010},
        "sync": {"address":"172.17.0.45", "port":10020},
        "system": {"address":"172.17.0.45", "port":10040},
        "transaction": {"address":"172.17.0.45", "port":10001},
        "sql": {"address":"172.17.0.45", "port":20001}
    },
    {
        "cluster": {"address":"172.17.0.46", "port":10010},
        "sync": {"address":"172.17.0.46", "port":10020},
        "system": {"address":"172.17.0.46", "port":10040},
        "transaction": {"address":"172.17.0.46", "port":10001},
        "sql": {"address":"172.17.0.46", "port":20001}
    }
]

[Memo]

  • Specify the serviceAddress and servicePort of the node definition file in each module (cluster,sync etc.) for each address and port.
  • sql items are required in the GridDB Advanced Edition only.
  • Set either the /cluster/notificationAddress, /cluster/notificationMember, /cluster/notificationProvider in the cluster definition file to match the cluster configuration method used.
  • See “GridDB technical reference” (GridDB_TechnicalReference.html) for details on the cluster configuration method.

2.4 Setting the tuning parameters

The main tuning parameters are described here. These parameters are not mandatory but affect the processing performance of the cluster.

2.4.1 Parameter related to update performance

GridDB creates a transaction log file and checkpoint file for persistency purposes. As data written in these files has an impact on the update performance, the method of creation can be changed by the following parameters. However, the disadvantage is that the possibility of data being lost during a failure is higher.

The related parameters are shown below.

Node definition file

ParametersData typeMeaning
/dataStore/persistencyModestringPersistency mode
/dataStore/logWriteModeintLog write mode

The persistency mode specifies whether to write data to a file during a data update. The log write mode specifies the timing to write data to the transaction log file.

Either one of the values below can be set in the persistency mode.

  • "NORMAL"
  • "KEEP_ALL_LOGS"

A "NORMAL" writes data to the transaction log file and checkpoint file every time there is an update. A transaction log file which is no longer required will be deleted by a checkpoint. The write timing of "KEEP_ALL_LOGS" is the same as "NORMAL" but all transaction log files are retained. Default value is "NORMAL".

[Points to note]

  • When performing a differential backup, set the persistency mode to "NORMAL".

Either one of the values below can be set in the log write mode.

  • 0: SYNC
  • An integer value of 1 or higher1: DELAYED_SYNC

In the "SYNC" mode, log writing is carried out synchronously every time an update transaction is committed or aborted. In the "DELAYED_SYNC" mode, log writing during an update is carried out at a specified delay of several seconds regardless of the update timing. Default value is "1 (DELAYED_SYNC 1 sec)".

2.4.2 Parameter related to performance and availability

By composing a cluster, GridDB can replicate data in multiple nodes to improve the search performance and availability. As replication of these data has an impact on the update performance, the method of creation can be changed by the following parameters. However, the disadvantage is that the possibility of data being lost during a failure is higher.

The related parameters are shown below.

Cluster definition file

ParametersData typeMeaning
/transaction/replicationModeintReplication mode

The replication mode specifies how to create a replica. The replication method has to be same for all nodes in the cluster .

  • "0": Asynchronous replication
  • "1": Semi-synchronous replication

"Asynchronous replication" performs replication without synchronizing the timing of the update process. "Semi-synchronous replication" performs replication synchronously at the timing of the update process timing but makes no appointment at the end of the replication. Default is "0".

2.4.3 Parameter related to access performance immediately after startup

Data perpetuated on a disk etc. can be loaded into the memory at the same time when a node is started (warm start process).

The warm start process can be enabled/disabled by the parameter below.

Node definition file

ParametersData typeMeaning
/dataStore/storeWarmStartbooleanStart processing mode
  • false: non-warm start mode
  • true: warm start mode

Default is false (invalid).

2.4.4 Other parameters

An explanation of the other parameters is given. See Annex Parameters list for the default values.

Node definition file

ParametersData typeMeaning
/dataStore/dbPathstringDatabase file directory
/dataStore/backupPathstringBackup file directory
/dataStore/storeMemoryLimitstringMemory buffer size
/dataStore/concurrencyintProcessing parallelism
/dataStore/affinityGroupSizeintNo. of data affinity groups
/checkpoint/checkpointIntervalintCheckpoint execution interval (unit: sec)
/system/eventLogPathstringOutput directory of event log file
/transaction/connectionLimitintNo. of connections upper limit value
/trace/categorystringEvent log output level
  • A database file directory is created when the data registered in the in-memory is perpetuated. In this directory, the transaction log file and checkpoint files are created.
  • A backup file directory is created when a backup is executed (further details will be explained in the subsequent chapters). In this directory, the backup file is stored.
  • The memory buffer size is used for data management. To set the memory size, simply use numbers and text to specify the memory size and its unit respectively. E.g. “2048MB”.
  • Processing parallelism refers to the upper limit value for GridDB to perform I/O processing of the secondary memory device in parallel.
  • With regards to data affinity, related data are collected, and the number of groups is specified during layout management.
  • Any number from 1 to 64 can be selected when specifying the number of groups. Please note though that the larger the group, the lower the memory utilization efficiency becomes.
  • The checkpoint execution interval is the interval to execute an internal checkpoint process regularly (related to the perpetuation of data).
  • The output directory of event log is the directory where messages (event message files) related to event data occurring inside a node such as exceptions etc, will be saved into.
  • As a rule of thumb, please set the no. of connection upper limit to at least twice the number of expected clients.
  • The event log output level is the output level for each category of the event log.

2.5 Distributing the definition file to each node

Both of the user and cluster definition files need to have the same settings in all of the nodes within the GridDB cluster.

Therefore, when configuring a cluster with 2 or more nodes, please follow these steps to set up all the nodes at once. (When configuring a cluster with a single node, the following steps can be skipped. )

  1. Set up the administrator user settings and environment-dependent parameters on either of the machines installed with nodes.
  2. Copy the already configured cluster definition file and user definition file to the definition file directory in the other nodes.
  3. Copy the node definition file too when the configuration setting is common to all of the nodes.
  4. Configure the nodes with different settings separately. (Network environment settings etc.)

2.6 Installation and setup (client)

This chapter explains the installation procedure of the client library. There are 2 types of client library in GridDB, Java and C. Only the Java version supports the GridDB NewSQL interface.

2.6.1 Environmental check

Check that the version is RHEL 6.2/6.3/6.4/6.5/6.6/6.7/6.8/7.2, or CentOS 6.2/6.3/6.4/6.5/6.6/6.7/6.8/7.2.

$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

[Points to note]

  • When choosing the OS package group during OS installation, please choose the minimum version or lower.
    • Software Development WorkStation

The following needs to be installed as a Java language development environment.

  • Oracle Java 6/7/8
  • Only 64-bit Java is supported by the GridDB NewSQL interface.
$ java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

2.6.2 Installing the client library

The following 4 RPM packages are required for the installation of the GridDB client library. Place the packages anywhere in the machine.

The griddb-xx-newsql package is only available in GridDB Advanced Edition / Vector Edition.

Package nameFile nameDescription
griddb-xx-java_libgriddb-xx-java_lib-X.X.X-linux.x86_64.rpmJava library is included.
(/usr/share/java/gridstore.jar)
griddb-xx-c_libgriddb-xx-c_lib-X.X.X-linux.x86_64.rpmC header file and library are included.
(/usr/include/gridstore.h and /usr/lib64/libgridstore.so)
griddb-xx-docsgriddb-xx-docs-X.X.X-linux.x86_64.rpmGridDB manual and program samples are included.
griddb-xx-newsqlgriddb-xx-newsql-X.X.X-linux.x86_64.rpmNewSQL interface library is included.

Install using the rpm command as a root user.

$ su
# rpm -ivh griddb-xx-c_lib-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-xx-c_lib                  ########################################### [100%]
# rpm -ivh griddb-xx-java_lib-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-xx-java_lib               ########################################### [100%]
# rpm -ivh griddb-xx-docs-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-xx-docs                   ########################################### [100%]
# rpm -ivh griddb-xx-newsql-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-xx-newsql                 ########################################### [100%]

2.6.3 Post-installation checks

After installation, check the directory configuration of the GridDB client library. The directory below is created when the installation is completed normally.

Installation directory

/usr/griddb/              # installation directory
                     docs/         # document directory
                     lib/          # library directory

The directory below is also created when griddb-xx-newsql package is installed.

/usr/griddb-xx-newsql-X.X.X/       # NewSQL interface installation directory
                            lib/   # library directory

In addition, the symbolic link below is created.

Symbolic link

/usr/lib64/libgridstore.so -> /usr/lib64/libgridstore.so.0
/usr/lib64/libgridstore.so.0 -> /usr/lib64/libgridstore.so.0.0.0
/usr/lib64/libgridstore.so.0.0.0 -> /usr/griddb-X.X.X/lib/libgridstore.so.0.0.0

/usr/share/java/gridstore.jar -> /usr/griddb-X.X.X/lib/gridstore-X.X.X.jar

The file below is also created when griddb-xx-newsql package is installed.

/usr/share/java/gridstore-jdbc.jar -> /usr/griddb-xx-newsql-X.X.X/lib/gridstore-jdbc-X.X.X.jar

2.6.4 Setting up a library

When the Java version of the client is used, add the client library to CLASSPATH.

$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore.jar

When the C version is used instead, add the client library to LD_LIBRARY_PATH.

$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib64

2.6.5 Configuring client settings

There is no definition file for configuring client settings. Specify the connection point and user/password in the client program.

See “GridDB API reference” (GridDB_API_Reference.html) for specification details on the GridDB NoSQL interface, and “GridDB Advanced Edition JDBC driver instructions” (GridDB_AE_JDBC_Driver_Guide.pdf) for specification details on the GridDB NewSQL interface.

2.7 Uninstall

Uninstall all packages if GridDB is no longer required. Follow the procedure below to uninstall the packages.

$ su
# rpm -e griddb-xx-server
# rpm -e griddb-xx-client
# rpm -e griddb-xx-java_lib
# rpm -e griddb-xx-c_lib
# rpm -e griddb-xx-newsql
# rpm -e griddb-xx-docs
*: xx is the GridDB edition (se, ae, ve)

Files such as definition files and data files etc. that are stored under the GridDB home directory will not be uninstalled.

Manually delete them if they are no longer required.

3 Operations

GridDB operating procedures are explained in this chapter.

The chapter explains the following cases in sequence.

  • Operations from start to stop
  • Operations when a cluster is in operation
  • Troubleshooting

Use the following commands during operations.

[Command list]

CommandFunction
gs_startnodeStarting a node
gs_joinclusterCreating a cluster/joining a node
gs_stopclusterStopping a cluster (stopping a process)
gs_stopnodeStop a node (shutdown)
gs_leaveclusterDetach a node from a cluster
gs_appendclusterExpand a cluster
gs_configGet cluster configuration node data
gs_statGet node internal data
gs_paramconfChange the memory used by a service
gs_logsGet node event log
gs_logconfChange node log output level
gs_backupBackup node data
gs_backuplistCheck backup data
gs_restoreRestore backup data
gs_importImport data
gs_exportExport data

[Points to note]

  • Execute the operating command as a gsadm user.
  • If a proxy variable (http_proxy) has been set up, set the address (group) in no_proxy and exclude it from the proxy. As an operating command will perform REST/http communications, the proxy server may be connected by mistake, thereby deactivating the operating command.
  • For commands which has the optional setting “connection server: port”, there is no need to specify the option if the port setting has not been changed from the default. In addition, if the optional setting “connection server: port” is specified, this command can be executed from a machine that is different from the one that started the node.

The methods used are explained in sequence below. See “GridDB Operational Management Guide” (GridDB_OperationGuide.html) for export/import details.

3.1 Operations from start to stop

3.1.1 Basic flow

The normal operating flow from commencement until termination of the GridDB cluster (post installation and setup of GridDB nodes) is as follows.

  1. Start each node.
  2. Configure the cluster.
  3. Use the GridDB cluster.
  4. Stop a cluster.
  5. Stop each node.

[Points to note]

  • For the procedure below, the operation administrator should be aware of the list of host names (or addresses) of all the machines running a node.
  • Similarly, the total number of nodes in the cluster should also be known.
  • User “admin” and password “admin” are used as examples in the user authentication option (-u).

3.1.2 Starting each node

Execute the start node command on the machine running the node. This command needs to be executed for each node.

However, if the GridDB node process (gsserver) is started automatically by the service, the following start operation is not necessary. Start from the “Compose a cluster” step described in the next section.

Use the following command to start a node.

  • gs_startnode

Use the node definition file, cluster definition file and user definition file settings under the conf director of GridDB home directory file to start the node. A command execution example is shown below.

[Command execution example]

$ gs_startnode

The nodes in all the machines that make up the cluster need to be started.

[Points to note]

  • When composing a cluster, the cluster definition file of every participant nodes needs to be the same. Prior to starting up the cluster, ensure that every node has the same cluster definition file.
  • Similarly, the user definition file of each node also needs to be made the same.

3.1.3 Composing a cluster

Prior to configuring the cluster, start the nodes first. This step is also necessary for a cluster with only a single node. (The cluster is not made up of multiple nodes.)

Execute the following command to add a node into the cluster.

  • gs_joincluster [-s connection server: port] -n|--nodeNum number of nodes constituting a cluster -c|--clusterName cluster name -u user name/password

Provide the “cluster name” and “Number of nodes constituting a cluster” as options and execute the command.

Specify the number of nodes in the cluster (“Number of nodes constituting a cluster”) as an option. When starting GridDB for the first time, the number is used as a threshold value for the various services to be started.

An example of the command execution is shown below. In this command, a cluster consisted of “1” node with the name “configured cluster name” is created.

[Command execution example]

$ gs_joincluster -c configured cluster name -n 1 -u admin/admin

An example of the command execution to be performed on a machine that is different from the one in which the nodes have been started is shown below. Another scenario where “3” nodes are added into the “example_three_nodes_cluster” cluster, where the address of the machine is “192.168.10.11”, is shown below.

[Command execution example]

$ gs_joincluster -s 192.168.10.11:10040 -c example_three_nodes_cluster -n 3 -u admin/admin

A cluster will be configured when the correct cluster name is specified and executed for each of the 3 machines making up the cluster. In addition, the cluster service will start only when the number of nodes added into the cluster is equal to the number of nodes specified for the cluster. Once service is started, you will be able to access the cluster from the application.

However, this command will return control immediately after receiving a request. Since the connection from the application could fail before the cluster is fully configured, add “-w” at to the end of the command and wait until the “Cluster is constituted” message appears, as shown in the example below.

[Command execution example]

$ gs_joincluster -s 192.168.10.12:10040 -c example_three_nodes_cluster -n 3 -u admin/admin
$ gs_joincluster -s 192.168.10.13:10040 -c example_three_nodes_cluster -n 3 -u admin/admin -w
...
Cluster is constituted.

[Points to note]

  • State “1” for the no. of nodes constituting a cluster when a single node configuration is used.
  • If the cluster participation command ends in an error, it means that there is a discrepancy in the cluster definition file of the node. Check the cluster definition file again and adopt the same definition.
  • The cluster service will not start when the number of nodes participating in a cluster does not reach the number of nodes constituting the cluster. When service is not started, check whether the number of nodes is correct.

Separate the nodes from the cluster if a wrong number of nodes constituting a cluster is specified. Execute the following cluster detachment command.

  • gs_leavecluster [-s connection server: port] -u user name/password

An example of the command execution in a machine in which the nodes to be separated from the cluster have been started is shown below.

[Command execution example]

$ gs_leavecluster -u admin/admin

[Points to note]

  • If this command is used for the purpose of stopping the cluster, there is a possibility that the data may no longer be viewable after the cluster comes into operation again.
  • If the cluster is already in operation, use the cluster stop command (gs_stopcluster).

3.1.4 Using a cluster

After a cluster is composed, data registration and search can be carried out in GridDB from a client program using a registered user.

See the “GridDB API Reference” (GridDB_API_Reference.html) and“GridDB Programming Tutorial” (GridDB_ProgrammingTutorial.html) for details on the creation of client programs.

3.1.5 Changing the memory used by a cluster

The memory used by GridDB is defined in the node definition file of the node that constitutes GridDB. This value can be changed online without having to restart the node or cluster.

Node definition file

ParametersData typeMeaning
/dataStore/storeMemoryLimitstringAvailable memory size

Execute the following command.

  • gs_paramconf [-s connection server: port] -u user name/password --show storeMemoryLimit | --set storeMemoryLimit value

An example to execute a command in a machine in which the nodes have been started is shown below.

[Command execution example]

$ gs_paramconf -u admin/admin --set storeMemoryLimit 2048MB
$ gs_paramconf -u admin/admin --show storeMemoryLimit
"2048MB"

[Points to note]

  • This operation is carried out on a single node unit. If you want to carry out the same changes for all the nodes, perform the operation above for each node.
  • When a node is shutdown, changed settings will not be saved. To perpetuate the value, change the node definition file.

3.1.6 Stopping a cluster

Stop a GridDB cluster. To stop the nodes, each node has to be stopped in sequence after the GridDB cluster management process is stopped .

First, stop the cluster management by executing the following stop-cluster command in one of the nodes in the cluster. Execute the following command in one of the nodes participating in the cluster.

  • gs_stopcluster [-s connection server: port] -u user name/password

The command line example is shown below.

[Command execution example]

$ gs_stopcluster -u admin/admin

At this point, data registration and search services in all nodes in the cluster will be stopped.

To stop/ shut down the nodes, execute the following command. For this purpose, execute a node stop command.

  • gs_stopnode [-w [WAIT_TIME]][-s connection server: port] [-f|--force] -u user name/password

The command line example is shown below.

[Command execution example]

$ gs_stopnode -w -u admin/admin

During nodes termination, it may take a while for the node process to be fully terminated due to the checkpoint (file writing of the memory data) process. It is recommended to add the “-w” command to wait until the process is fully completed.

3.1.7 Restarting a stopped cluster

To restart the GridDB cluster, follow the normal startup procedure below.

  • Make a note of the number of nodes in the cluster prior to shutting it down.
  • Start the nodes.
  • Re-add the specified number of nodes to the cluster.

The command line example is shown below.

[Command execution example]

$ gs_startnode
...
$ gs_joincluster -c configured cluster name -n 1 -u admin/admin
...
  • Use the same cluster name as stated in the cluster definition file.
  • For a single node configuration, write “1” for the number of nodes in the cluster, whereas in the case of multiple nodes configuration, use the same number of nodes used prior to the shutdown.
  • Information about the number of nodes participating in the cluster is written to the event log file during shutdown.

When restarted, the GridDB will read the database files (transaction log file, checkpoint file) and restore to the original state at the shutdown time. The service will be restarted only once all the “number of nodes constituting a cluster” has been added back to the cluster.

[Points to note]

  • The number of nodes at the shutdown point needs to be correctly specified in the “number of nodes constituting a cluster”. The cluster service will not start when the number of nodes added to the cluster is not the same as the number of nodes constituting the cluster. When the service does not start, please check that the correct number of nodes has been used.
  • If incorrect “Number of nodes constituting a cluster” was specified, separate the nodes from the cluster using the cluster detachment command while the cluster is not in operation. Ensure that the correct “Number of nodes constituting a cluster” is specified this time prior to re-adding the nodes back into the cluster.
  • In addition, when the incorrect number of nodes is specified, there is a possibility that the service starts in the wrong state when the cluster starts operating. In this case, stop and restart the cluster.
  • If the current number of nodes becomes less than the original (at the shutdown point), perhaps due to machine failures, etc., specify the number of nodes that can be used and restart the process. Data will be re-arranged as if a failure had occurred during the operation. However, if the nodes get reduced drastically, data may no longer be accessible.
  • When adding a new node, add it after the original cluster has been restarted. Please see the “Adding a node to a cluster in operation” section for details.
  • It is possible to change the IP address and port (/xxx/serviceAdress, /xxx/servicePort of the node definition file) of the machine when restarting the cluster.

3.2 Getting various data

3.2.1 Getting cluster configuration data

To get the cluster configuration data (data containing the list of nodes participating in a cluster), execute the following command.

  • gs_config [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_config -u admin/admin
{
    "follower": [],
    "master": {
        "address": "192.168.1.10", 
        "port": 10040
    }, 
    "multicast": {
        "address": "239.0.0.1",
        "port": 31999
    },
    "self": {
        "address": "192.168.1.10", 
        "port": 10040, 
        "status": "ACTIVE"
    }
}
  • A list of nodes (address and port) in the cluster excluding the master node will be displayed in “follower”. There may be several units available. This data is displayed in the master node only.
  • The address and port of the master node, which manages the cluster, will be displayed in “master”. Make sure it is 1 node only.
  • The multicast address and port of the cluster will be displayed in “multicast”.
  • The address and port of the node itself which executed the command will be displayed in “self”.

The meaning of system status (Status) is as follows.

  • INACTIVE: Stop
  • ACTIVATING: Start operation
  • ACTIVE: In operation
  • DEACTIVATING: Start stop
  • ABNORMAL: Abnormal stop
  • NORMAL_SHUTDOWN: Start normal shutdown

3.2.2 Getting cluster data

To get the cluster data (cluster configuration data and internal data), execute the following.

  • gs_stat [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_stat -u admin/admin
{
        :
        :
    "cluster": {
        "activeCount": 3,
        "clusterName": "defaultCluster",
        "clusterStatus": "MASTER",
        :
        :
}

The meaning of the cluster status (clusterStatus) is as follows.

  • MASTER: Master
  • SUB_MASTER: Candidate serving as the master during a master failure
  • FOLLOWER: Follower
  • SUB_FOLLOWER: Candidate serving as the follower during a master failure
  • SUB_CLUSTER: Cluster is not in operation

The meaning of system status (nodeStatus) is as follows.

  • INACTIVE: Stop
  • ACTIVATING: Start operation
  • ACTIVE: In operation
  • DEACTIVATING: Start stop
  • ABNORMAL: Abnormal stop
  • NORMAL_SHUTDOWN: Start normal shutdown

See Parameter list for the description of the other items.

3.2.3 Displaying the event log

To get the most recent event log, use the command below.

  • gs_logs [-s connection server: port] -u user name/password --lines No. of rows to acquire [first key word [second key word]]

An example of the command execution is shown below.

[Command execution example]

$ gs_logs -u admin/admin --lines 3 WARNING
2015-02-23T05:28:47.780+0900 host1 728 WARNING EVENT_ENGINE [130901:EE_WAIT_COMPLETION] (queueingElapsed=0, handlerElapsed=10000, watcherEngine=CHECKPOINT_SERVICE, watchedEngine=TRANSACTION_SERVICE, eventType=3004)
2015-02-23T05:29:12.437+0900 host1 726 WARNING IO_MONITOR [1900:CM_LONG_IO] [LONG I/O] sync time,34656,fileName,data/gs_log_0_60.log
2015-02-23T05:29:12.438+0900 host1 726 WARNING IO_MONITOR [LONG EVENT] eventType=PARTITION_GROUP_END, pId=0, pgId=0, time=34658

Event log contains a list of character strings on event data. The format of event data is as follows.

  • Time, host name, thread no., event level, generation module, event no., event name, message

Check with the support desk for details.

3.2.4 Displaying and changing the event log output level

Use the command below to display a list of the output levels of the event log.

  • gs_logconf [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_logconf -u admin/admin
{
    "levels": {
        "CHECKPOINT_SERVICE": "INFO",
        "CHECKPOINT_SERVICE_DETAIL": "ERROR",
        "CHUNK_MANAGER": "ERROR",
        "CLUSTER_OPERATION": "INFO",
        :
        :
    }
}

Use the command below to change the output level of the event log.

  • gs_logconf [-s connection server: port] -u user name/password category output level

An example of the command execution is shown below.

[Command execution example]

$ gs_logconf -u admin/admin CHUNK_MANAGER INFO
$ gs_logconf -u admin/admin
{
    "levels": {
        "CHECKPOINT_SERVICE": "INFO",
        "CHECKPOINT_SERVICE_DETAIL": "ERROR",
        "CHUNK_MANAGER": "INFO",
        "CLUSTER_OPERATION": "INFO",
        :
        ;
    }
}

A list of the output levels from high to low is shown below.

  • ERROR: Error
  • WARNING: Warning
  • INFO: Info
  • DEBUG: Debug

When the output level is set to low, all logs with an output level higher than that level will be produced. For example, if the level is set at INFO, the INFO, WARNING and ERROR logs will be created.

[Points to note]

  • When a node is shutdown, changed settings will not be saved.
  • The log output level is either the default value given in gs_node.json of the sample, or a level lower than that is recommended to be set. In addition, the default value is given in Annex Parameter list.

3.3 Backup and restoration

3.3.1 Data backup

In GridDB, a hot backup of node units can be carried out.

Data of a node in operation can be backed up by executing the command below.

  • gs_backup [-s connection server: port] -u user name/password backup name

An example of the command execution is shown below.

[Command execution example]

$ cat /var/lib/gridstore/conf/gs_node.json         # configuration check
{
        "dataStore":{
                "dbPath":"/var/lib/gridstore/data",
                "backupPath":"/var/lib/gridstore/backup",  # backup directory
                "storeMemoryLimit":"1024MB",
                "storeWarmStart":true,
                "concurrency":1,
                "logWriteMode":1,
                "persistencyMode":"NORMAL",
      :
      :
}
$ gs_backup -u admin/admin 20130301_backup        # backup execution
...

As a result, the following process will be executed.

  1. Create a 20130301_backup directory under the backup directory (/var/lib/gridstore/backup.
  2. Create a checkpoint file (gs_cp_n_p.dat), (single or multiple) transaction log file (gs_log_n_m.log), and a backup data file (gs_backup_info.json,gs_backup_info_digest.json) (hereinafter known as a backup file group).

Control returns after commencing backup. Depending on the data size and online processing load, it may several hours or more for the backup to be completed.

The progress status of the backup can be acquired with a gs_stat command.

The progress status of the backup can be backed up by executing the command below.

  • gs_stat -t backup [-s connection server: port] -u user name/password

[Command execution example]

$ gs_stat -t backup -u admin/admin 20130301_backup        
BackupStatus: Processing                          # backup in progress

$ gs_stat -t backup -u admin/admin 20130301_backup  
BackupStatus: -                                   # backup completed or not in operation                            

[Points to note]

  • See “GridDB backup guide” (GridDB_BackupGuide.html) for the backup details.
  • To perform a hot backup of the whole cluster while service is being continued, the above-mentioned backup operation needs to be carried out for all nodes constituting the cluster.
  • In this example, the backupPath is /var/lib/gridstore/backup as a matter of convenience but in an actual operation, change the directory to a suitable directory to match the system configuration.
  • When the system is restored using backup data, the status is restored to the status immediately before the completion of the backup.
  • If a failure occurs during backup, the system cannot be restored using this as the backup performed is incomplete.
  • When executing a hot backup, the backup may be created with the entire cluster in a non-conforming state if multiple containers have been created. Where necessary, ban transaction services so that the backup can be executed in the static state.
  • In GridDB, data will be automatically re-arranged when a failure occurs. As a result, when a failure occurs during a backup, it may no longer be possible to back up the necessary data. When a failure occurs, perform the backup again starting with the first node.

3.3.2 Restoring backup data

To restore backup data to a node,

follow the procedure below (this will restore an entire cluster from backup data).

  • Check that no node has been started.
    • Check that the cluster definition file is the same as the other nodes in the cluster that the node is joining.
  • Check whether past transaction log files and checkpoint files have been left behind in the database file directory (/var/lib/gridstore/data by default) of the node.
    • Delete if unnecessary and move to another directory if required.
  • Execute the restore command on the machine executing the node.
  • Start a node.
  • Join a cluster.

Execute the following command.

  • gs_backuplist -u user name/password

Below is a specific example to display a list of the backup names. A list of the backup names can be displayed regardless of the startup status of the nodes. The Status appears as Processing if the backup process is in progress with the nodes started.

[Command execution example]

$ gs_backuplist -u admin/admin

BackupName Status StartTime EndTime
------------------------------------------------------------------------
 20141025NO2     P   2014-10-25T06:37:10+0900 -
 20141025 NG 2014-10-25T02:13:34+0900 -
 20140925 OK 2014-09-25T05:30:02+0900 2014-09-25T05:59:03+0900
 20140825 OK 2014-08-25T04:35:02+0900 2014-08-25T04:55:03+0900

The backup status (Status) will be one of the following.

  • OK: Normal
  • NG: Error
  • P: In progress

[Points to note]

  • If the status displayed is NG, the backup file may be damaged and so restoration is not possible.

An execution example to restore backup data is shown below. Stop the nodes first prior to executing the restoration command.

[Command execution example]

$ mv ${GS_HOME}/data/*.{dat,log} ${GS_HOME}/temp    # Move the database file
$ gs_restore 20130521_backup                        # restore

As a result, the following process will be executed.

  1. Copy the backup file group from the 20130521_backup directory under the backup directory (/var/lib/gridstore/backup) to the data directory (/var/lib/gridstore/data).

In this example, the backup directory is /var/lib/gridstore/backup and the data directory is /var/lib/gridstore/data as a matter of convenience but in an actual operation, change the directory to a suitable directory to match the system configuration. (See Other Parameters)

After restoration is completed, restart the nodes and re-add them to the original cluster.

[Command execution example]

$ gs_startnode
...
$ gs_joincluster -c [Configured cluster name] -n 1 -u admin/admin
...

Once restarted, the node will import the database file (backup file group) and then start the services back on. After import ends, the node will start the services.

[Points to note]

  • Take note of the no. of partitions and the parameter of the processing parallelism in the cluster definition file. Set the configuration value of the node to restore to be the same as the configuration value of the backup node. The node cannot start correctly if it is not the same.
  • If you want to restore the backup state correctly, the backup and restoration tasks need to be carried out for the entire cluster.
  • For example, even if some of the nodes are restored, these nodes cannot be returned to the state they were in at the time of the backup. After restoration, it is necessary to let the nodes join the cluster in operation in order to use the data. However, if the data is updated in the cluster after backup, the restored data will be updated by the (updated) cluster data.
  • In particular, if the cluster configuration changes from the time the backup was created, there will be no restoration effect. As the data will be autonomously re-arranged if the node is forced to join a cluster, the data will become invalid with a high probability even if restored.
  • If data is missing in the backup data file, or if the contents have been revised, the node will not be able to start services.

3.4 Adding/detaching nodes in a cluster

3.4.1 Adding a node to a cluster in operation

Additional nodes can be added to a running cluster having a specified number of nodes (the number is specified in the cluster configuration gs_joincluster).

Follow the procedure below to add additional nodes to a running cluster.

  • Ensure that the cluster is running.
  • Check the status of the cluster.
  • Start the nodes that you want to add.
    • Check the cluster definition file of the node you want to add is the same as that of the other nodes of the cluster which you want to add the node to.
  • Execute the node addition command on the “node to be added”.
    • Get the cluster information of the nodes to be added with a gs_stat command, and if the cluster status turns to FOLLOWER, the node will be able to join the cluster.

To increase the number of nodes, execute the following command.

  • gs_appendcluster --cluster connection server: port [-s connection server: port] -u user name/password

Specify the server address and port (for the operating command) of “any one of the nodes constituting the cluster where the node is to be added” in the cluster option. A specific example on adding a new node to a cluster is shown below.

Check the status of the cluster to add the node.

[Command execution example]

$ gs_stat -s 192.168.33.29:10040  -u admin/admin
{
    :
    "cluster":{                          // cluster-related
        "activeCount":5,                   // number of nodes already participating in a cluster
        "clusterName":"function_1",        // cluster name
        "clusterStatus":"MASTER",          // cluster status
        "designatedCount":5,               // number of nodes constituting a cluster (predetermined number of nodes)
        :
        

A cluster can be added if the number of nodes already participating in a cluster (number of nodes currently joined to a cluster) is equal to the number of nodes constituting a cluster. If the number of nodes constituting a cluster > number of nodes already participating in a cluster, use gs_joincluster (join a cluster configuration) to add a node to the cluster.

Execute the following command on the machine to which the node will be added. Specify the server address and port (for operating command) of any of the nodes in the cluster (the node does not have to be a master node).

[Command execution example]

$ gs_startnode
$ gs_appendcluster --cluster 192.168.33.29:10040 -u admin/admin

After adding the nodes, the number of nodes constituting a cluster and the number of nodes already participating in a cluster will be changed.

[Command execution example]

$ gs_stat -u admin/admin
{
      :
    "cluster":{                                 // cluster-related
        "activeCount":6,                        // number of nodes already participating in a cluster
        "clusterName":"function_1",             // cluster name
        "clusterStatus":"MASTER",               // cluster status
        "designatedCount":6,                    // number of nodes constituting a cluster (predetermined number of nodes)
      :
}

[Points to note]

  • Since the number of nodes constituting a cluster is required during cluster restart, make a note of this number using the gs_stat command when cluster expansion is carried out.
  • Non-stop expansion of a GridDB cluster (node increase) will be carried out 1 unit at a time.
  • In the case of large scale expansion, stop and re-configure the cluster instead.

3.4.2 Detaching a node from a cluster in operation (shrinking a cluster)

In the case where a single node needs to be removed from a running GridDB cluster,

follow the procedure below.

  • Check that the cluster is running.
  • Execute the cluster detachment command on the node which needs to be detached.

Execute the following cluster detachment command.

  • gs_leavecluster [-s connection server: port] [-f] -u user name/password

    [Command execution example]

    $ gs_leavecluster -u admin/admin
    

    [Points to note]

    • If there is a possibility of data lost resulted from node detachment, cluster reduction will not be able to be carried out. To force node detachment, use the ?f command.
    • Continuous node reduction will be carried out 1 node at a time.
    • A cluster will be terminated if the number of nodes existing in the cluster is less than half of the number of nodes constituting the cluster. When a large scale node reduction is required, stop and re-configure the cluster with the new number of nodes. However, please note that when a large scale reduction is carried out, the possibility of data lost increases.

3.5 Software update

3.5.1 Updating the software

Follow the procedure below to update the software.

  1. Stop the cluster
  2. Stop the node
  3. Make a backup copy of the definition file, database file and event log file
  4. Update the software
  5. Start the node
  6. Configure the cluster

An example of the command execution in a machine in which the nodes have been started is shown below.

[Command execution example]

$ gs_stopcluster -u admin/admin
$ gs_stopnode -u admin/admin
$ cp -rp /var/lib/gridstore/data /xxx/shelter  # copy just in case
$ cp -rp /var/lib/gridstore/log /xxx/shelter   # copy just in case
$ cp -rp /var/lib/gridstore/conf /xxx/shelter  # copy just in case
$ su
# rpm -Uvh griddb-xx-server-Y.Y.Y-linux.x86_64.rpm
# rpm -Uvh griddb-xx-client-Y.Y.Y-linux.x86_64.rpm
# rpm -Uvh griddb-xx-docs-Y.Y.Y-linux.x86_64.rpm
# exit
$ gs_startnode
$ gs_joincluster -c configured cluster name -u admin/admin

*xx: Edition of GridDB to update (se, ae, ve)

*Y.Y.Y: Version of GridDB to update

3.6 Response in the event of a failure

See “GridDB Backup Guide” (GridDB_BackupGuide.html) for details on the failure type and troubleshooting methods.

3.6.1 Basic flow

When a failure occurs, the flow of operations carried out by the administrator will be as follows.

  1. First, check the cluster operating status and identify which node has been withdrawn from the cluster (failed node).
  2. Get the failed node event log data.
  3. Analyze the event log data and check the cause of failure.
  4. Remove the failed node and replace with a new node.

Even if a failure occurs, GridDB will perform a failover in the cluster to continue services. As long as there is a backup node (replica) in operation, failover will be performed repeatedly. An error will result when there are no more backup nodes.

The following action is carried when a failure occurs in GridDB.

  1. The failed node is automatically detached from the cluster.
  2. Failover is carried out in the backup node to replace the failed node.

3.6.2 Checking the operating status of a cluster

The node status can be checked using the get node data command. Check the status individually for all nodes.

Command execution example

[Command execution example]

$ gs_stat -u admin/admin
{
        :
    "cluster": {
        :
        "nodeStatus": "ACTIVE",
        :
}
# Repeat execution for all nodes

Check the value of /cluster/nodeStatus output. Symptoms of a failed node are as follows.

  • Detaches itself unexpectedly from the cluster
    • When the status is INACTIVE, DEACTIVATING, ABNORMAL, NORMAL_SHUTDOWN.
  • Does not return results

3.6.3 Getting event log data from the failed node

To identify the cause of node failure, get the most recent event log from the failed node using the command below.

  • gs_logs [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_logs -u admin/admin
2015-03-20T10:07:47.219+0900 host01 22962 INFO CHECKPOINT_SERVICE [30902:CP_STATUS] [CP_START] mode=NORMAL_CHECKPOINT, backupPath=
 :
 :

Event log contains a list of character strings on event data. The format of event data is as follows.

  • Time, machine name, thread no., event type, event category, generation source file name, generation method, generation row number: followed by any character string (*)

(*) If a new line is returned, a space is assigned to the first character of the new line

Although the cause of failure can be identified directly from the event log data, please check with the service desk for detailed analysis.

[Points to note]

  • Due to log rotation, multiple event log files are created. However, only data of the current event log file will be displayed.

3.6.4 Replacing a failed node

Remove the machine running the faulty node from the network. Then, install and set up a GridDB node in a new machine, and add to the new cluster. As explained in the “Adding a node to a cluster in operation” section, the cluster will then automatically redistribute the data and services will be started by the newly added node once redistribution is completed.

4 Annexes

X.X.X below indicates the GridDB version.

4.1 Parameter list

The list of parameters in the node definition file and cluster definition file in GridDB is shown below.

4.1.1 Node definition file (gs_node.json)

ParametersData typeMeaningDefault
/dataStore/dbPathstringDatabase file directory"data"
/dataStore/backupPathstringBackup file directory"backup"
/dataStore/storeMemoryLimitstringMemory buffer size"1024MB"
/dataStore/storeWarmStartbooleanWarm start during restart (false: invalid, true: valid)false
/dataStore/concurrencyintProcessing parallelism1
/dataStore/logWriteModeintLog write mode1
/dataStore/persistencyModestringPersistency mode"NORMAL"
/dataStore/affinityGroupSizeintNumber of affinity groups4
/dataStore/storeCompressionModestringMode of block data compression"NO_COMPRESSION"
/checkpoint/checkpointIntervalstringCheckpoint execution interval"1200s"
/checkpoint/checkpointMemoryLimitstringCheckpoint memory buffer size"1024MB"
/checkpoint/useParallelModebooleanCheckpoint parallel operation (false: invalid, true: valid)false
/cluster/serviceAddressstringReception address used for cluster administration"127.0.0.1"
/cluster/servicePortintReception port used for cluster administration10010
/sync/serviceAddressstringReception address used in data synchronization"127.0.0.1"
/sync/servicePortintReception port used in data synchronization10020
/system/serviceAddressstringConnection address of operation command"127.0.0.1"
/system/servicePortintConnection port of operation command10040
/system/eventLogPathstringOutput directory of event log file"log"
/transaction/serviceAddressstringReception address of transaction process"127.0.0.1"
/transaction/servicePortintReception port of transaction process10001
/transaction/connectionLimitintNo. of connections upper limit value5000
/trace/defaultstringEvent log output level"LEVEL_ERROR"
/trace/dataStorestring"LEVEL_ERROR"
/trace/collectionstring"LEVEL_ERROR"
/trace/timeSeriesstring"LEVEL_ERROR"
/trace/chunkManagerstring"LEVEL_ERROR"
/trace/objectManagerstring"LEVEL_INFO"
/trace/checkpointFilestring"LEVEL_ERROR"
/trace/checkpointServicestring"LEVEL_INFO"
/trace/logManagerstring"LEVEL_WARNING"
/trace/clusterOperationstring"LEVEL_INFO"
/trace/clusterServicestring"LEVEL_ERROR"
/trace/syncServicestring"LEVEL_ERROR"
/trace/systemServicestring"LEVEL_INFO"
/trace/transactionManagerstring"LEVEL_ERROR"
/trace/transactionServicestring"LEVEL_ERROR"
/trace/transactionTimeoutstring"LEVEL_WARNING"
/trace/sessionTimeoutstring"LEVEL_WARNING"
/trace/replicationTimeoutstring"LEVEL_WARNING"
/trace/recoveryManagerstring"LEVEL_INFO"
/trace/eventEnginestring"LEVEL_WARNING"
/trace/triggerServicestring"LEVEL_ERROR"

The parameters used in the GridDB Advanced Edition / Vector Edition in addition to those mentioned above are as follows.

ParametersData typeMeaningDefault
/sql/serviceAddressstringReception address for JDBC/ODBC client connection"127.0.0.1"
/sql/servicePortintReception port for JDBC/ODBC client connection20001
/sql/connectionLimitintNo. of connections upper limit value5000
/sql/concurrencyintProcessing parallelism5
/trace/sqlServicestringEvent log output level"LEVEL_ERROR"

4.1.2 Cluster definition file (gs_cluster.json)

ParametersData typeMeaningDefault
/dataStore/partitionNumintNumber of partitions128
/dataStore/storeBlockSizestringBlock size ("64KB", "1MB")"64KB"
/cluster/clusterNamestringCluster name""
/cluster/replicationNumintNo. of replicas2
/cluster/notificationAddressstringMulticast address for cluster administration"239.0.0.1"
/cluster/notificationPortintMulticast port for cluster administration20000
/cluster/notificationIntervalstringInterval of multicast for cluster administration"5s"
/cluster/heartbeatIntervalstringHeartbeat interval"5s"
/cluster/loadbalanceCheckIntervalstringLoad balance check interval"180s"
/cluster/notificationMemberarrayAddress list used in the fixed list method
/cluster/notificationProvider/urlstringURL of address provider used in the provider method
/cluster/notificationProvider/updateIntervalintInterval to get list from address provider"5s"
/sync/timeoutIntervalstringShort-term synchronization timeout time"30s"
/transaction/notificationAddressstringAddress for multi-cast distribution to client"239.0.0.1"
/transaction/notificationPortintPort for multi-cast distribution to client31999
/transaction/notificationIntervalstringInterval of multicast distribution to client"5s"
/transaction/replicationTimeoutIntervalstringReplication/timeout time"10s"
/transaction/replicationModeintReplication method (0: asynchronous, 1: semi-synchronous)0
/transaction/authenticationTimeoutIntervalstringAuthentication timeout time"5s"

The parameters used in the GridDB Advanced Edition / Vector Edition in addition to those mentioned above are as follows.

ParametersData typeMeaningDefault
/sql/notificationAddressstringAddress for multi-cast distribution to JDBC/ODBC client"239.0.0.1"
/sql/notificationPortintMulticast port to JDBC/ODBC client41999
/sql/notificationIntervalstringInterval of multicast distribution to JDBC/ODBC client"5s"

4.2 Build/execution method

An example on how to build and execute a program is covered in this section.

Since the sample program is stored inside the ZIP file in the document package, decompress first using the appropriate program.

[Points to note]

  • The user and password in the sample program need to be changed appropriately.

(1) Login as gsadm.

(2) Build and execute the program.

[For GridDB NoSQL interface]

For Java

  1. Set the environmental variables
  2. Copy the sample program to the gsSample directory
  3. Build
  4. Execute
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore.jar
$ mkdir gsSample
$ cp /usr/griddb/docs/sample/program/Sample1.java gsSample/.
$ javac gsSample/Sample1.java
$ java gsSample/Sample1 239.0.0.1 31999 configured cluster name

For C language

  1. Set the environmental variables
  2. Copy the sample program
  3. Addvoid main(int argc,char *argv[]){ sample2(argv[1],argv[2],argv[3]);} to the last line in the sample2.c file
  4. Build
  5. Execute

[Output example]

$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib64
$ cp /usr/griddb/docs/sample/program/sample2.c .
$ echo "void main(int argc,char *argv[]){ sample2(argv[1],argv[2],argv[3]);}" >> sample2.c
$ gcc -I/usr/griddb/lib sample2.c -lgridstore
$ a.out 239.0.0.1 31999 configured cluster name

Specify the 3 arguments, namely the interface address between the client and cluster, the port and the cluster name.

[For GridDB NewSQL interface]

  1. Set the environmental variables
  2. Copy the sample program
  3. Build
  4. Execute
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore-jdbc.jar
$ cp /usr/griddb/docs/sample/program/SampleJDBC.java .
$ javac SampleJDBC.java
$ java SampleJDBC 239.0.0.1 41999 configured cluster name

Specify the 3 arguments, namely the address for multicast distribution to JDBC/ODBC clients, the port and the cluster name.

[Points to note]

  • Execution of sample2 needs to be completed.

[Output example]

DB Connection Start
Start
1414054670561|0|100.0|
End
DB Connection Close

5 Trademarks

  • GridDB is a trademark of Toshiba Corporation.
  • Oracle and Java are registered trademarks of Oracle and/or its affiliates.
  • Linux is a trademark of Linus Torvalds.
  • Red Hat is a registered trademark or trademark of Red Hat, Inc. in the US and other countries.
  • Other product names are trademarks or registered trademarks of the respective owners.

              Copyright Toshiba Solutions Corporation 2013 - 2016