GridDB^® quick start guide

GridDB is a distributed NoSQL database that manages a group of data called a row, which contains a key and multiple values. There are currently two commercial version of GridDB in offer; GridDB Advanced Edition and GridDB Standard Edition. In the former edition, support for SQL as the inquiry language has been added. Even though the library used during application development is different, system construction and operation remains the same.

GridDB’s merits are as follows:

• Allows data management to be carried out in memory at a high speed.
  • Groups of rows are stored in memory, allowing high-speed updates and searches to be carried out.
• Allows in-memory processing to be carried out and its capacity to be increased.
  • Capacity can be increased by dispersing and placing data in multiple machines. Furthermore, data management using disks is also possible (this operation is not covered in this manual). As a result, even when operating with individual nodes, the capacity can be increased without being limited by the memory size.
• Able to achieve high availability.
  • Data is replicated within a cluster, thus allowing uninterrupted process in the case of a failure in any of the cluster’s node. In addition, each node continuously updates data to the disk, allowing restoration of data when a failure occurs.
• Can scale out up to 1000 units.
  • In transactions involving container as the unit, high scalability is achieved by improving the degree of parallelism of the cluster process.
• Does not require manual cluster management.
  • Communications are carried out using a decentralized protocol among mutual nodes, with cluster control carried out autonomously by GridDB itself.
• Supports non-standard data used in social infrastructure.
  • It supports non-standard data such as time series data and spatial data etc. that is used in social infrastructure.
• In the GridDB Advanced Edition, ODBC/JDBC I/F is supported.

The features of GridDB are summarized below.

Requirements	Description
[Basic requirements]
Large capacity (petabyte class)	In order to achieve both high speed and high capacity, data is stored in memory and SSD devices.
High speed (in-memory)	In-memory processing
High scalability	The number of servers can be increased up to 1000 units
High availability	Availability is further improved by replicating data to HDD in multiple servers.
High autonomy	Automation of data replication and data placement balancing. Server expansion can be easily done online.
Operation control functions	Monitoring, security, backup etc.
[Social infrastructure requirements]
Time series data	A lossless compression function that could reduce the data size yet keeping the original features unchanged is provided.
Spatial data type	The index can be tuned to increase the search speed. 2D and 3D data type are supported.
Consistency guarantee	Supports ACID transactions within the same container

Predict the amount of data to be stored in the container and then estimate the memory usage.

First, predict the amount of data to be stored in the application. Estimate the following:

• Data size of row
• Number of rows to be registered

Next, estimate the memory required to store those estimated data.

• Memory capacity used = row data size × no. of registered rows ÷ 0.75 + 8 × no. of registered rows × (assigned index number + 2) ÷ 0.66 (byte)

Perform a similar estimation for all collections created and used in the application. The total sum becomes the amount of memory required in the GridDB cluster.

• Total memory usage = the sum of memory usage in all collections

However, please consider this number as the minimum requirement, as the memory usage also depends on update frequency.

Estimate the required no. of nodes used in GridDB. In the example below, it is assumed that one node is executed in one machine.

First, make an assumption of the memory required per machine.

• Memory size of machine

In addition, make an assumption of the no. of replicas to create. The no. of replicas is given as a GridDB configuration value.

• No. of replicas

Default no. of replicas is 2.

• No. of nodes = (Total memory usage ÷ machine memory size) × no. of replicas (units)

However, please consider this as the minimum requirement as it is preferable to have a greater number of spare units to take into account the load distribution and availability improvement.

Estimate the size of the file to be created in GridDB, and the disk capacity required for the machine to execute a node. Two types of files should be created; a checkpoint file and a transaction log file.

The memory usage of individual node is determined as follows.

• Individual memory usage = (total memory usage × no. of replicas) ÷ no. of nodes (byte)

The size of the checkpoint file is estimated as follows based on this numerical value.

• File size = Individual memory usage × 2 (byte)

In addition, as the transaction log file size is dependent on the update frequency of the application, the following data is thus predicted.

• Row update frequency (times/sec)

Furthermore, the checkpoint interval is assumed. Checkpoint interval is given as a GridDB configuration value.

• Checkpoint interval

Default value of checkpoint interval is 1200 sec (20 minutes).

The transaction log file size is estimated as follows based on these numerical values.

• File size = row data size × row update frequency × checkpoint interval (byte)

The individual disk usage is estimated as follows.

• Individual disk usage = Transaction log file size + checkpoint file size

[Points to note]

• The size of checkpoint file expands depending on the data capacity. However, please note that once it has expanded, the file size will not be reduced even if some of data in the container or row get deleted. Empty space can be re-used after data is deleted.

Check that the version is RHEL 6.2/6.3/6.4/6.5, or CentOS 6.2/6.3/6.4/6.5.

$ lsb_release -id
Distributor ID: CentOS
Description: CentOS release 6.3 (Final)

[Points to note]

• When choosing the package group during OS installation, please choose the minimum version or lower.
  • Basic Server

The following 3 RPM packages are needed when installing a GridDB node. Place these packages anywhere in the machine.

Package name	File name	Description
griddb-server	griddb-server-X.X.X-linux.x86_64.rpm	The start and other commands for the GridDB node module and server are included.
griddb-client	griddb-client-X.X.X-linux.x86_64.rpm	One set of operating commands except start node is included.
griddb-docs	griddb-docs-X.X.X-linux.x86_64.rpm	GridDB manual and program samples are included.

*: X.X.X is the GridDB version

Install using the rpm command as a root user.

$ su
# rpm -Uvh griddb-server-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
User gsadm and group gridstore have been registered.
GridStore uses new user and group.
   1:griddb-server                 ########################################### [100%]
# rpm -Uvh griddb-client-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
User and group has already been registered correctly.
GridStore uses existing user and group.
   1:griddb-client                  ########################################### [100%]
# rpm -Uvh griddb-docs-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-docs                   ########################################### [100%]

When you install the package, the following group and user are created in the OS. This OS user is set as the operator of GridDB.

Group	User	Home directory
gridstore	gsadm	/var/lib/gridstore

The following environment variables are defined in this gsadm user.

Environment variables	Value	Meaning
GS_HOME	/var/lib/gridstore	gsadm/GridDB home directory
GS_LOG	/var/lib/gridstore/log	Event log file output directory

[Points to note]

• These environment variables are used in the operating commands described later.
• The password of the gsadm user has not been set. With the root user privilege, please set the password appropriately.
  • Some of the functions of the operation tools may be necessary.

In addition, when the GridDB node module is installed, services that are executed automatically upon startup of the OS will be registered.

Service name	Run level
gridstore	3,4,5

The service registration data can be checked with the following command.

# /sbin/chkconfig --list | grep gridstore
gridstore 0:off 1:off 2:off 3:on 4:on 5:on 6:off

The GridDB node will be started automatically by this service during OS startup.

[Points to note]

• Services will not start automatically immediately after installation.

To stop auto startup of a service, use the command below.

# /sbin/chkconfig gridstore off

See the chapter on services in “GridDB Operational Management Guide” (GridDB_OperationGuide.html) for details of the services.

Check the directory configuration of the installed GridDB node.

First, check that the GridDB home directory, and related directory and files have been created.

GridDB home directory

/var/lib/gridstore/                      # GridDB home directory
                   admin/                # integrated operational management GUI home directory
                   backup/               # backup directory
                   conf/                 # definition file directory
                        gs_cluster.json  # cluster definition file
                        gs_node.json     # node definition file
                        password         # user definition file
                   data/                 # database file directory
                   log/                  # log directory

Confirm with the following command.

$ ls /var/lib/gridstore/
admin backup conf data log

Next, check that the installation directory has been created

Installation directory

/usr/gridstore-X.X.X/              # installation directory
                     Fixlist.pdf   # revision record
                     Readme.txt    # release instructions
                     bin/          # operation command, module directory
                     conf/         # definition file sample directory
                     docs/         # document directory
                     etc/
                     lib/          # library directory
                     license/      # license directory
                     prop/         # configuration file directory
                     web/          # integrated operational management GUI file directory

Confirm with the following command.

$ ls /usr/gridstore-X.X.X/
Fixlist.pdf Readme.txt bin conf etc lib license prop web

All documents have been compressed into a single ZIP file. Decompress and refer to the documents where appropriate as shown below.

$ cd /usr/gridstore-X.X.X/docs
$ unzip griddb-documents-X.X.X.zip

In addition, the following symbolic links are created as shown below in a few directories under the installation directory for greater convenience.

$ ls /usr/gridstore/
conf lib prop web

Lastly, confirm the version of the server module installed with the following command.

$ gsserver --version
GridDB version X.X.X build XXXXX

Points to note

The following files are created when GridDB is operated according to the following procedure.

[Database file]

/var/lib/gridstore/                     # GridDB home directory
                   data/                # database file directory
                        gs_log_n_m.log  # log file to record the transaction logs (n, m are numbers)
                        gs_cp_n_p.dat   # checkpoint file to record data regularly (n, p are numbers)

[Log file]

/var/lib/gridstore/                            # GridDB home directory
                   log/                        # log directory
                       gridstore-%Y%m%d-n.log  # event log file
                       gs_XXXX.log             # operation tool log file

The directory where these files are created can be changed by the parameter settings in the node definition file.

*: gs_XXXX is an operation tool name. (Example: gs_startnode.log)

Administrator privilege is used for authentication related matter within the nodes and clusters. Creation date of administrator user is saved in the user definition file. The default file is as shown below.

• /var/lib/gridstore/conf/password

The default user below exists immediately after installation.

User	Password	Example of proper use
admin	admin	For authentication of operation administrator user, operation commands
system	manager	For authentication of application user, client execution

Administrator user information including the above-mentioned default users can be changed using the user administration command in the operating commands.

Command	Function
gs_adduser	Add an administrator user
gs_deluser	Delete an administrator user
gs_passwd	Change the password of administrator user

Change the password as shown below when using a default user. The password is encrypted during registration.

$ gs_passwd admin
Password: (enter password)
Retype password: (re-enter password)

When adding a new administrator user except a default user, the user name has to start with gs#.

One or more ASCII alphanumeric characters and the underscore sign “_” can be used after gs#.

An example on adding a new administrator user is shown below.

$ gs_adduser gs#newuser
Password: (enter password)
Retype password: (re-enter password)

[Points to note]

• A GridDB administrator user is different from the OS user gsadm created during installation.
• A change in the administrator user information using a user administration command becomes valid when a node is restarted.
• In order to use it for authentication purposes in the client, the same user data needs to be registered in all the nodes. Copy the user definition file and make sure the same user data can be referred to in all the nodes.
• Execute the operating command as a gsadm user.

[Memo]

• See “GridDB Operational Management Guide” (GridDB_OperationGuide.html) for details of the user management commands.

First, set up the network environment.

An explanation of the recommended configuration method in an environment that allows a multicast to be used is given below. In an environment which does not allow a multicast to be used, or an environment in which communications between fellow nodes cannot be established in a multicast, a cluster configuration method other than the multicast method has to be used. See Other cluster configuration method settings for the details.

The configuration items can be broadly divided as follows.

• (1) Address information serving as an interface with the client
• (2) Address information for cluster administration and processing
• (3) Address information serving as an interface with the JDBC client (GridDB Advanced Edition only)

Although these settings need to be set to match the environment, basically default settings will also work.

However, an IP address derived in reverse from the host name of the machine needs to be an address that allows it to be connected from the outside regardless of whether the GridDB cluster has a multiple node configuration or a single node configuration.

Normally, this can be set by stating the host name and the corresponding IP address in the /etc/hosts file.

/etc/hosts setting

First, check with the following command to see whether the setting has been configured. If the IP address appears, it means that the setting has already been configured.

$ hostname -i
192.168.11.10

The setting has not been configured in the following cases.

$ hostname -i
hostname: no address corresponding to name

In addition, a loopback address that cannot be connected from the outside may appear.

$ hostname -i
127.0.0.1

If the setting has not been configured or if a loopback address appears, use the following example as a reference to configure /etc/hosts. The host name and IP address, and the appropriate network interface card (NIC) differ depending on the environment.

1. Check the host name and IP address.

   $ hostname
   GS_HOST
   $ ip route | grep eth0 | cut -f 12 -d " " | tr -d "\n"
   192.168.11.10
   

2. Add the IP address and corresponding host name checked by the root user to the /etc/hosts file.

   192.168.11.10 GS_HOST
   

3. Check that the settings have been configured correctly.

   $ hostname -i
   192.168.11.10
   

*If the displayed setting remains the same as before, it means that a setting higher in priority is given in the /etc/hosts file. Change the priority order appropriately.

Proceed to the next setting after you have confirmed that /etc/hosts has been configured correctly.

(1) Address information serving as an interface with the client

In the address data serving as an interface with the client, there are configuration items in the node definition file and cluster definition file.

Node definition file

Parameters	Data type	Meaning
/transaction/serviceAddress	string	Reception address of transaction process
/transaction/servicePort	string	Reception port of transaction process
/system/serviceAddress	string	Connection address of operation command
/system/servicePort	string	Connection port of operation command

The reception address and port of transaction processes are used to connect individual client to the nodes in the cluster, and to request for the transaction process from the cluster. This address is used when configuring a cluster with a single node, but in the case where multiple nodes are present through API, the address is not used explicitly.

The connection address and port of the operating command are used to specify the process request destination of the operation command, as well as the repository information of the integrated operation control GUI.

These reception/connection addresses need not be set so long as there is no need to use/separate the use of multiple interfaces.

Cluster definition file

Parameters	Data type	Meaning
/transaction/notificationAddress	string	Interface address between client and cluster
/transaction/notificationPort	string	Interface port between client and cluster

A multi-cast address and port are specified in the interface address between a client and cluster. This is used by a GridDB cluster to send cluster information to its clients and for the clients to send processing requests via the API to the cluster. See the description of GridStoreFactory class/method in “GridDB API reference” (GridDB_API_Reference.html) for details.

It is also used as a connection destination address of the export/import tool, or as repository data of the integrated operation control GUI.

(2) Address information for cluster administration and processing

In the address data for a cluster to autonomously perform cluster administration and processing, there are configuration items in the node definition file and cluster definition file. These addresses are used internally by GridDB to exchange the heart beat (live check among clusters) and information among the clusters. These settings are not necessary so long as the address used is not duplicated with other systems on the same network or when using multiple network interface cards.

Node definition file

Parameters	Data type	Meaning
/cluster/serviceAddress	string	Reception address used for cluster administration
/cluster/servicePort	string	Reception port used for cluster administration

Cluster definition file

Parameters	Data type	Meaning
/cluster/notificationAddress	string	Multicast address for cluster administration
/cluster/notificationPort	string	Multicast port for cluster administration

• Although a synchronization process is carried out with a replica when the cluster configuration is changed, a timeout time can be set for the process.
  • /sync/timeoutInterval

[Points to note]

• An address or port that is not in use except in GridDB has to be set.
• The same address can be set for the node definition file gs_node.json /transaction/serviceAddress, /system/serviceAddress, and /cluster/serviceAddress for operations to be performed. If a machine has multiple network interfaces, the bandwidth can be increased by assigning a separate address to each respective interface.

The following settings are applicable in the GridDB Advanced Edition only.

(3) Address information serving as an interface with the JDBC client

In the address data serving as an interface with the JDBC/ODBC client, there are configuration items in the node definition file and cluster definition file.

Node definition file

Parameters	Data type	Meaning
/sql/serviceAddress	string	Reception address for JDBC/ODBC client connection
/sql/servicePort	int	Reception port for JDBC/ODBC client connection

The reception address and port of JDBC/ODBC client connection are used to connect JDBC/ODBC individual client to the nodes in the cluster, and to access the cluster data in SQL. This address is used when configuring a cluster with a single node, but in the case where multiple nodes are present through API, the address is not used explicitly.

Cluster definition file

Parameters	Data type	Meaning
/sql/notificationAddress	string	Address for multi-cast distribution to JDBC/ODBC client
/sql/notificationPort	int	Multicast port to JDBC/ODBC client

The address and port used for multicast distribution to a JDBC/ODBC client are used for the GridDB cluster to notify the JDBC/ODBC client of cluster data, and to access the cluster data in SQL with the JDBC/ODBC client.

Refer to Annex Parameter List for the other parameters and default values.

Set the name of the cluster to be composed by the target nodes in advance. The name set will be checked to see if it matches the value specified in the command to compose the cluster. As a result, this prevents a different node and cluster from being composed when there is an error in specifying the command.

The cluster name is specified in the following configuration items of the cluster definition file.

Cluster definition file

Parameters	Data type	Meaning
/cluster/clusterName	string	Name of cluster to create

[Points to note]

• Node failed to start with default value ("").
• A unique name on the sub-network is recommended.
• A cluster name is a string composed of 1 or more ASCII alphanumeric characters and the underscore “_”. However, the first character cannot be a number. The name is also not case-sensitive. In addition, it has to be specified within 64 characters.

In an environment which does not allow the multicast method to be used, configure the cluster using the fixed list method or provider method. An explanation of the respective network settings in the fixed list method and provider method is given below.

When using the multicast method, proceed to Setting the tuning parameters.

(1) Fixed list method

When a fixed address list is given to start a node, the list is used to compose the cluster.

When composing a cluster using the fixed list method, configure the parameters in the cluster definition file.

Cluster definition file

Parameters	Data type	Meaning
/cluster/notificationMember	string	Specify the address list when using the fixed list method as the cluster configuration method.

A configuration example of a cluster definition file is shown below.

{
                             :
                             :
    "cluster":{
        "clusterName":"yourClusterName",
        "replicationNum":2,
        "heartbeatInterval":"5s",
        "loadbalanceCheckInterval":"180s",
        "notificationMember": [
            {
                "cluster": {"address":"172.17.0.44", "port":10010},
                "sync": {"address":"172.17.0.44", "port":10020},
                "system": {"address":"172.17.0.44", "port":10040},
                "transaction": {"address":"172.17.0.44", "port":10001},
                "sql": {"address":"172.17.0.44", "port":20001}
            },
            {
                "cluster": {"address":"172.17.0.45", "port":10010},
                "sync": {"address":"172.17.0.45", "port":10020},
                "system": {"address":"172.17.0.45", "port":10040},
                "transaction": {"address":"172.17.0.45", "port":10001},
                "sql": {"address":"172.17.0.45", "port":20001}
            },
            {
                "cluster": {"address":"172.17.0.46", "port":10010},
                "sync": {"address":"172.17.0.46", "port":10020},
                "system": {"address":"172.17.0.46", "port":10040},
                "transaction": {"address":"172.17.0.46", "port":10001},
                "sql": {"address":"172.17.0.46", "port":20001}
            }
        ]
    },
                             :
                             :
}

(2) Provider method

Get the address list supplied by the address provider to perform cluster configuration.

When composing a cluster using the provider method, configure the parameters in the cluster definition file.

Cluster definition file

Parameters	Data type	Meaning
/cluster/notificationProvider/url	string	Specify the URL of the address provider when using the provider method as the cluster configuration method.
/cluster/notificationProvider/updateInterval	string	Specify the interval to get the list from the address provider. Specify a value that is 1s or higher and less than 2^31s.

A configuration example of a cluster definition file is shown below.

{
                             :
                             :
    "cluster":{
        "clusterName":"yourClusterName",
        "replicationNum":2,
        "heartbeatInterval":"5s",
        "loadbalanceCheckInterval":"180s",
        "notificationProvider":{
            "url":"http://example.com/notification/provider",
            "updateInterval":"30s"
        }
    },
                             :
                             :
}

The address provider can be configured as a Web service or as a static content. The specifications below need to be satisfied.

• Compatible with the GET method.
• When accessing the URL, the node address list of the cluster containing the cluster definition file in which the URL is written is returned as a response.
  • Response body: Same JSON as the contents of the node list specified in the fixed list method
  • Response header: Including Content-Type:application/json

An example of a response sent from the address provider is as follows.

$ curl http://example.com/notification/provider
[
    {
        "cluster": {"address":"172.17.0.44", "port":10010},
        "sync": {"address":"172.17.0.44", "port":10020},
        "system": {"address":"172.17.0.44", "port":10040},
        "transaction": {"address":"172.17.0.44", "port":10001},
        "sql": {"address":"172.17.0.44", "port":20001}
    },
    {
        "cluster": {"address":"172.17.0.45", "port":10010},
        "sync": {"address":"172.17.0.45", "port":10020},
        "system": {"address":"172.17.0.45", "port":10040},
        "transaction": {"address":"172.17.0.45", "port":10001},
        "sql": {"address":"172.17.0.45", "port":20001}
    },
    {
        "cluster": {"address":"172.17.0.46", "port":10010},
        "sync": {"address":"172.17.0.46", "port":10020},
        "system": {"address":"172.17.0.46", "port":10040},
        "transaction": {"address":"172.17.0.46", "port":10001},
        "sql": {"address":"172.17.0.46", "port":20001}
    }
]

[Memo]

• Specify the serviceAddress and servicePort of the node definition file in each module (cluster,sync etc.) for each address and port.
• sql items are required in the GridDB Advanced Edition only.
• Set either the /cluster/notificationAddress, /cluster/notificationMember, /cluster/notificationProvider in the cluster definition file to match the cluster configuration method used.
• See “GridDB technical reference” (GridDB_TechnicalReference.html) for details on the cluster configuration method.

GridDB creates a transaction log file and checkpoint file for persistency purposes. As data written in these files has an impact on the update performance, the method of creation can be changed by the following parameters. However, the disadvantage is that the possibility of data being lost during a failure is higher.

The related parameters are shown below.

Node definition file

Parameters	Data type	Meaning
/dataStore/persistencyMode	string	Persistency mode
/dataStore/logWriteMode	int	Log write mode

The persistency mode specifies whether to write data to a file during a data update. The log write mode specifies the timing to write data to the transaction log file.

Either one of the values below can be set in the persistency mode.

• "NORMAL"
• "KEEP_ALL_LOGS"

A "NORMAL" writes data to the transaction log file and checkpoint file every time there is an update. A transaction log file which is no longer required will be deleted by a checkpoint. The write timing of "KEEP_ALL_LOGS" is the same as "NORMAL" but all transaction log files are retained. Default value is "NORMAL".

[Points to note]

• When performing a differential backup, set the persistency mode to "NORMAL".

Either one of the values below can be set in the log write mode.

• 0: SYNC
• An integer value of 1 or higher1: DELAYED_SYNC

In the "SYNC" mode, log writing is carried out synchronously every time an update transaction is committed or aborted. In the "DELAYED_SYNC" mode, log writing during an update is carried out at a specified delay of several seconds regardless of the update timing. Default value is "1 (DELAYED_SYNC 1 sec)".

By composing a cluster, GridDB can replicate data in multiple nodes to improve the search performance and availability. As replication of these data has an impact on the update performance, the method of creation can be changed by the following parameters. However, the disadvantage is that the possibility of data being lost during a failure is higher.

The related parameters are shown below.

Cluster definition file

Parameters	Data type	Meaning
/transaction/replicationMode	int	Replication mode

The replication mode specifies how to create a replica. The replication method has to be same for all nodes in the cluster .

• "0": Asynchronous replication
• "1": Semi-synchronous replication

"Asynchronous replication" performs replication without synchronizing the timing of the update process. "Semi-synchronous replication" performs replication synchronously at the timing of the update process timing but makes no appointment at the end of the replication. Default is "0".

Data perpetuated on a disk etc. can be loaded into the memory at the same time when a node is started (warm start process).

The warm start process can be enabled/disabled by the parameter below.

Node definition file

Parameters	Data type	Meaning
/dataStore/storeWarmStart	boolean	Start processing mode

• false: non-warm start mode
• true: warm start mode

Default is true (valid).

An explanation of the other parameters is given. See Annex Parameters list for the default values.

Node definition file

Parameters	Data type	Meaning
/dataStore/dbPath	string	Database file directory
/dataStore/backupPath	string	Backup file directory
/dataStore/storeMemoryLimit	string	Memory buffer size
/dataStore/concurrency	int	Processing parallelism
/dataStore/affinityGroupSize	int	No. of data affinity groups
/checkpoint/checkpointInterval	int	Checkpoint execution interval (unit: sec)
/system/eventLogPath	string	Output directory of event log file
/transaction/connectionLimit	int	No. of connections upper limit value
/trace/category	string	Event log output level

• A database file directory is created when the data registered in the in-memory is perpetuated. In this directory, the transaction log file and checkpoint files are created.
• A backup file directory is created when a backup is executed (further details will be explained in the subsequent chapters). In this directory, the backup file is stored.
• The memory buffer size is used for data management. To set the memory size, simply use numbers and text to specify the memory size and its unit respectively. E.g. “2048MB”.
• Processing parallelism refers to the upper limit value for GridDB to perform I/O processing of the secondary memory device in parallel.
• With regards to data affinity, related data are collected, and the number of groups is specified during layout management.
• Any number from 1 to 64 can be selected when specifying the number of groups. Please note though that the larger the group, the lower the memory utilization efficiency becomes.
• The checkpoint execution interval is the interval to execute an internal checkpoint process regularly (related to the perpetuation of data).
• The output directory of event log is the directory where messages (event message files) related to event data occurring inside a node such as exceptions etc, will be saved into.
• As a rule of thumb, please set the no. of connection upper limit to at least twice the number of expected clients.
• The event log output level is the output level for each category of the event log.

Check that the version is RHEL 6.2/6.3/6.4/6.5, or CentOS 6.2/6.3/6.4/6.5.

$ lsb_release -id
Distributor ID: CentOS
Description: CentOS release 6.3 (Final)

[Points to note]

• When choosing the OS package group during OS installation, please choose the minimum version or lower.
  • Software Development WorkStation

The following needs to be installed as a Java language development environment.

• Oracle Java 6/7/8
• Only 64-bit Java is supported by the GridDB Advanced Edition NewSQL interface.

$ java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

The following 4 RPM packages are required for the installation of the GridDB client library. Place the packages anywhere in the machine.

The griddb-newsql package is only available in GridDB Advanced Edition.

Package name	File name	Description
griddb-java_lib	griddb-java_lib-X.X.X-linux.x86_64.rpm	Java library is included.
		(/usr/share/java/gridstore.jar)
griddb-c_lib	griddb-c_lib-X.X.X-linux.x86_64.rpm	C header file and library are included.
		(/usr/include/gridstore.h and /usr/lib64/libgridstore.so)
griddb-docs	griddb-docs-X.X.X-linux.x86_64.rpm	GridDB manual and program samples are included.
griddb-newsql	griddb-newsql-X.X.X-linux.x86_64.rpm	NewSQL interface library is included.

Install using the rpm command as a root user.

$ su
# rpm -ivh griddb-c_lib-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-c_lib                  ########################################### [100%]
# rpm -ivh griddb-java_lib-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-java_lib               ########################################### [100%]
# rpm -ivh griddb-docs-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-docs                   ########################################### [100%]
# rpm -ivh griddb-newsql-X.X.X-linux.x86_64.rpm
Under preparation...           ########################################### [100%]
   1:griddb-newsql                 ########################################### [100%]

After installation, check the directory configuration of the GridDB client library. The directory below is created when the installation is completed normally.

Installation directory

/usr/gridstore-X.X.X/              # installation directory
                     docs/         # document directory
                     lib/          # library directory

The directory below is also created when GridDB Advanced Edition is installed.

/usr/gridstore-newsql-X.X.X/       # NewSQL interface installation directory
                            lib/   # library directory

In addition, the symbolic link below is created.

Symbolic link

/usr/lib64/libgridstore.so -> /usr/lib64/libgridstore.so.0
/usr/lib64/libgridstore.so.0 -> /usr/lib64/libgridstore.so.0.0.0
/usr/lib64/libgridstore.so.0.0.0 -> /usr/gridstore-X.X.X/lib/libgridstore.so.0.0.0

/usr/share/java/gridstore.jar -> /usr/gridstore-X.X.X/lib/gridstore-X.X.X.jar

The file below is also created when GridDB Advanced Edition is installed.

/usr/share/java/gridstore-jdbc.jar -> /usr/gridstore-newsql-X.X.X/lib/gridstore-jdbc-X.X.X.jar

When the Java version of the client is used, add the client library to CLASSPATH.

$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore.jar

When the C version is used instead, add the client library to LD_LIBRARY_PATH.

$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib64

There is no definition file for configuring client settings. Specify the connection point and user/password in the client program.

See “GridDB API reference” (GridDB_API_Reference.html) for specification details on the GridDB Standard Edition NoSQL interface, and “GridDB Advanced Edition JDBC driver instructions” (GridDB_AE_JDBC_Driver_Guide.pdf) for specification details on the GridDB Advanced Edition NewSQL interface.

The normal operating flow from commencement until termination of the GridDB cluster (post installation and setup of GridDB nodes) is as follows.

1. Start each node.
2. Configure the cluster.
3. Use the GridDB service.
4. Stop a cluster.
5. Stop each node.

[Points to note]

• For the procedure below, the operation administrator should be aware of the list of host names (or addresses) of all the machines running a node.
• Similarly, the total number of nodes in the cluster should also be known.
• User “admin” and password “admin” are used as examples in the user authentication option (-u).

Execute the start node command on the machine running the node. This command needs to be executed for each node.

However, if the GridDB node process (gsserver) is started automatically by the service, the following start operation is not necessary. Start from the “Compose a cluster” step described in the next section.

Use the following command to start a node.

• gs_startnode

Use the node definition file, cluster definition file and user definition file settings under the conf director of GridDB home directory file to start the node. A command execution example is shown below.

[Command execution example]

$ gs_startnode

The nodes in all the machines that make up the cluster need to be started.

[Points to note]

• When composing a cluster, the cluster definition file of every participant nodes needs to be the same. Prior to starting up the cluster, ensure that every node has the same cluster definition file.
• Similarly, the user definition file of each node also needs to be made the same.

Prior to configuring the cluster, start the nodes first. This step is also necessary for a cluster with only a single node. (The cluster is not made up of multiple nodes.)

Execute the following command to add a node into the cluster.

• gs_joincluster [-s connection server: port] -n|--nodeNum number of nodes constituting a cluster -c|--clusterName cluster name -u user name/password

Provide the “cluster name” and “Number of nodes constituting a cluster” as options and execute the command.

Specify the number of nodes in the cluster (“Number of nodes constituting a cluster”) as an option. When starting GridDB for the first time, the number is used as a threshold value for the various services to be started.

An example of the command execution is shown below. In this command, a cluster consisted of “1” node with the name “configured cluster name” is created.

[Command execution example]

$ gs_joincluster -c configured cluster name -n 1 -u admin/admin

An example of the command execution to be performed on a machine that is different from the one in which the nodes have been started is shown below. Another scenario where “3” nodes are added into the “example_three_nodes_cluster” cluster, where the address of the machine is “192.168.10.11”, is shown below.

[Command execution example]

$ gs_joincluster -s 192.168.10.11:10040 -c example_three_nodes_cluster -n 3 -u admin/admin

A cluster will be configured when the correct cluster name is specified and executed for each of the 3 machines making up the cluster. In addition, the cluster service will start only when the number of nodes added into the cluster is equal to the number of nodes specified for the cluster. Once service is started, you will be able to access the cluster from the application.

However, this command will return control immediately after receiving a request. Since the connection from the application could fail before the cluster is fully configured, add “-w” at to the end of the command and wait until the “Cluster is constituted” message appears, as shown in the example below.

[Command execution example]

$ gs_joincluster -s 192.168.10.12:10040 -c example_three_nodes_cluster -n 3 -u admin/admin
$ gs_joincluster -s 192.168.10.13:10040 -c example_three_nodes_cluster -n 3 -u admin/admin -w
...
Cluster is constituted.

[Points to note]

• State “1” for the no. of nodes constituting a cluster when a single node configuration is used.
• If the cluster participation command ends in an error, it means that there is a discrepancy in the cluster definition file of the node. Check the cluster definition file again and adopt the same definition.
• The cluster service will not start when the number of nodes participating in a cluster does not reach the number of nodes constituting the cluster. When service is not started, check whether the number of nodes is correct.

Separate the nodes from the cluster if a wrong number of nodes constituting a cluster is specified. Execute the following cluster detachment command.

• gs_leavecluster [-s connection server: port] -u user name/password

An example of the command execution in a machine in which the nodes to be separated from the cluster have been started is shown below.

[Command execution example]

$ gs_leavecluster -u admin/admin

[Points to note]

• If this command is used for the purpose of stopping the cluster, there is a possibility that the data may no longer be viewable after the cluster comes into operation again.
• If the cluster is already in operation, use the cluster stop command (gs_stopcluster).

After a cluster is composed, data registration and search can be carried out in GridDB from a client program using a registered user.

See the “GridDB API Reference” (GridDB_API_Reference.html) and“GridDB Programming Tutorial” (GridDB_ProgrammingTutorial.html) for details on the creation of client programs.

The memory used by GridDB is defined in the node definition file of the node that constitutes GridDB. This value can be changed online without having to restart the node or cluster.

Node definition file

Parameters	Data type	Meaning
/dataStore/storeMemoryLimit	string	Available memory size

Execute the following command.

• gs_paramconf [-s connection server: port] -u user name/password --show storeMemoryLimit | --set storeMemoryLimit value

An example to execute a command in a machine in which the nodes have been started is shown below.

[Command execution example]

$ gs_paramconf -u admin/admin --set storeMemoryLimit 2048MB
$ gs_paramconf -u admin/admin --show storeMemoryLimit
"2048MB"

[Points to note]

• This operation is carried out on a single node unit. If you want to carry out the same changes for all the nodes, perform the operation above for each node.
• When a node is shutdown, changed settings will not be saved. To perpetuate the value, change the node definition file.

Stop a GridDB cluster. To stop the nodes, each node has to be stopped in sequence after the GridDB cluster management process is stopped .

First, stop the cluster management by executing the following stop-cluster command in one of the nodes in the cluster. Execute the following command in one of the nodes participating in the cluster.

• gs_stopcluster [-s connection server: port] -u user name/password

The command line example is shown below.

[Command execution example]

$ gs_stopcluster -u admin/admin

At this point, data registration and search services in all nodes in the cluster will be stopped.

To stop/ shut down the nodes, execute the following command. For this purpose, execute a node stop command.

• gs_stopnode [-w [WAIT_TIME]][-s connection server: port] [-f|--force] -u user name/password

The command line example is shown below.

[Command execution example]

$ gs_stopnode -w -u admin/admin

During nodes termination, it may take a while for the node process to be fully terminated due to the checkpoint (file writing of the memory data) process. It is recommended to add the “-w” command to wait until the process is fully completed.

To restart the GridDB cluster, follow the normal startup procedure below.

• Make a note of the number of nodes in the cluster prior to shutting it down.
• Start the nodes.
• Re-add the specified number of nodes to the cluster.

The command line example is shown below.

[Command execution example]

$ gs_startnode
...
$ gs_joincluster -c configured cluster name -n 1 -u admin/admin
...

• Use the same cluster name as stated in the cluster definition file.
• For a single node configuration, write “1” for the number of nodes in the cluster, whereas in the case of multiple nodes configuration, use the same number of nodes used prior to the shutdown.
• Information about the number of nodes participating in the cluster is written to the event log file during shutdown.

When restarted, the GridDB will read the database files (transaction log file, checkpoint file) and restore to the original state at the shutdown time. The service will be restarted only once all the “number of nodes constituting a cluster” has been added back to the cluster.

[Points to note]

• The number of nodes at the shutdown point needs to be correctly specified in the “number of nodes constituting a cluster”. The cluster service will not start when the number of nodes added to the cluster is not the same as the number of nodes constituting the cluster. When the service does not start, please check that the correct number of nodes has been used.
• If incorrect “Number of nodes constituting a cluster” was specified, separate the nodes from the cluster using the cluster detachment command while the cluster is not in operation. Ensure that the correct “Number of nodes constituting a cluster” is specified this time prior to re-adding the nodes back into the cluster.
• In addition, when the incorrect number of nodes is specified, there is a possibility that the service starts in the wrong state when the cluster starts operating. In this case, stop and restart the cluster.
• If the current number of nodes becomes less than the original (at the shutdown point), perhaps due to machine failures, etc., specify the number of nodes that can be used and restart the process. Data will be re-arranged as if a failure had occurred during the operation. However, if the nodes get reduced drastically, data may no longer be accessible.
• When adding a new node, add it after the original cluster has been restarted. Please see the “Adding a node to a cluster in operation” section for details.
• It is possible to change the IP address and port (/xxx/serviceAdress, /xxx/servicePort of the node definition file) of the machine when restarting the cluster.

To get the cluster configuration data (data containing the list of nodes participating in a cluster), execute the following command.

• gs_config [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_config -u admin/admin
{
    "follower": [],
    "master": {
        "address": "192.168.1.10", 
        "port": 10040
    }, 
    "multicast": {
        "address": "239.0.0.1",
        "port": 31999
    },
    "self": {
        "address": "192.168.1.10", 
        "port": 10040, 
        "status": "ACTIVE"
    }
}

• A list of nodes (address and port) in the cluster excluding the master node will be displayed in “follower”. There may be several units available. This data is displayed in the master node only.
• The address and port of the master node, which manages the cluster, will be displayed in “master”. Make sure it is 1 node only.
• The multicast address and port of the cluster will be displayed in “multicast”.
• The address and port of the node itself which executed the command will be displayed in “self”.

The meaning of system status (Status) is as follows.

• INACTIVE: Stop
• ACTIVATING: Start operation
• ACTIVE: In operation
• DEACTIVATING: Start stop
• ABNORMAL: Abnormal stop
• NORMAL_SHUTDOWN: Start normal shutdown

To get the cluster data (cluster configuration data and internal data), execute the following.

• gs_stat [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_stat -u admin/admin
{
　　　　　　　　:
　　　　　　　　:
    "cluster": {
        "activeCount": 3,
        "clusterName": "defaultCluster",
        "clusterStatus": "MASTER",
　　　　　　　　:
　　　　　　　　:
}

The meaning of the cluster status (clusterStatus) is as follows.

• MASTER: Master
• SUB_MASTER: Candidate serving as the master during a master failure
• FOLLOWER: Follower
• SUB_FOLLOWER: Candidate serving as the follower during a master failure
• SUB_CLUSTER: Cluster is not in operation

The meaning of system status (nodeStatus) is as follows.

• INACTIVE: Stop
• ACTIVATING: Start operation
• ACTIVE: In operation
• DEACTIVATING: Start stop
• ABNORMAL: Abnormal stop
• NORMAL_SHUTDOWN: Start normal shutdown

See Parameter list for the description of the other items.

To get the most recent event log, use the command below.

• gs_logs [-s connection server: port] -u user name/password --lines No. of rows to acquire [first key word [second key word]]

An example of the command execution is shown below.

[Command execution example]

$ gs_logs -u admin/admin --lines 3 WARNING
2015-02-23T05:28:47.780+0900 host1 728 WARNING EVENT_ENGINE [130901:EE_WAIT_COMPLETION] (queueingElapsed=0, handlerElapsed=10000, watcherEngine=CHECKPOINT_SERVICE, watchedEngine=TRANSACTION_SERVICE, eventType=3004)
2015-02-23T05:29:12.437+0900 host1 726 WARNING IO_MONITOR [1900:CM_LONG_IO] [LONG I/O] sync time,34656,fileName,data/gs_log_0_60.log
2015-02-23T05:29:12.438+0900 host1 726 WARNING IO_MONITOR [LONG EVENT] eventType=PARTITION_GROUP_END, pId=0, pgId=0, time=34658

Event log contains a list of character strings on event data. The format of event data is as follows.

• Time, host name, thread no., event level, generation module, event no., event name, message

Check with the support desk for details.

Use the command below to display a list of the output levels of the event log.

• gs_logconf [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_logconf -u admin/admin
{
    "levels": {
        "CHECKPOINT_SERVICE": "INFO",
        "CHECKPOINT_SERVICE_DETAIL": "ERROR",
        "CHUNK_MANAGER": "ERROR",
        "CLUSTER_OPERATION": "INFO",
　　　　　　　　:
　　　　　　　　:
    }
}

Use the command below to change the output level of the event log.

• gs_logconf [-s connection server: port] -u user name/password category output level

An example of the command execution is shown below.

[Command execution example]

$ gs_logconf -u admin/admin CHUNK_MANAGER INFO
$ gs_logconf -u admin/admin
{
    "levels": {
        "CHECKPOINT_SERVICE": "INFO",
        "CHECKPOINT_SERVICE_DETAIL": "ERROR",
        "CHUNK_MANAGER": "INFO",
        "CLUSTER_OPERATION": "INFO",
　　　　　　　　:
　　　　　　　　;
    }
}

A list of the output levels from high to low is shown below.

• ERROR: Error
• WARNING: Warning
• INFO: Info
• DEBUG: Debug

When the output level is set to low, all logs with an output level higher than that level will be produced. For example, if the level is set at INFO, the INFO, WARNING and ERROR logs will be created.

[Points to note]

• When a node is shutdown, changed settings will not be saved.
• The log output level is either the default value given in gs_node.json of the sample, or a level lower than that is recommended to be set. In addition, the default value is given in Annex Parameter list.

In GridDB, a hot backup of node units can be carried out.

Data of a node in operation can be backed up by executing the command below.

• gs_backup [-s connection server: port] -u user name/password backup name

An example of the command execution is shown below.

[Command execution example]

$ cat /var/lib/gridstore/conf/gs_node.json         # configuration check
{
        "dataStore":{
                "dbPath":"/var/lib/gridstore/data",
                "backupPath":"/var/lib/gridstore/backup",  # backup directory
                "storeMemoryLimit":"1024MB",
                "storeWarmStart":true,
                "concurrency":1,
                "logWriteMode":1,
                "persistencyMode":"NORMAL",
　　　　　　:
　　　　　　:
}
$ gs_backup -u admin/admin 20130301_backup        # backup execution
...

As a result, the following process will be executed.

1. Create a 20130301_backup directory under the backup directory (/var/lib/gridstore/backup.
2. Create a checkpoint file (gs_cp_n_p.dat), (single or multiple) transaction log file (gs_log_n_m.log), and a backup data file (gs_backup_info.json,gs_backup_info_digest.json) (hereinafter known as a backup file group).

Control returns after commencing backup. Depending on the data size and online processing load, it may several hours or more for the backup to be completed.

The progress status of the backup can be acquired with a gs_stat command.

The progress status of the backup can be backed up by executing the command below.

• gs_stat -t backup [-s connection server: port] -u user name/password

[Command execution example]

$ gs_stat -t backup -u admin/admin 20130301_backup        
BackupStatus: Processing                          # backup in progress

$ gs_stat -t backup -u admin/admin 20130301_backup  
BackupStatus: -                                   # backup completed or not in operation                            

[Points to note]

• See “GridDB backup guide” (GridDB_BackupGuide.html) for the backup details.
• To perform a hot backup of the whole cluster while service is being continued, the above-mentioned backup operation needs to be carried out for all nodes constituting the cluster.
• In this example, the backupPath is /var/lib/gridstore/backup as a matter of convenience but in an actual operation, change the directory to a suitable directory to match the system configuration.
• When the system is restored using backup data, the status is restored to the status immediately before the completion of the backup.
• If a failure occurs during backup, the system cannot be restored using this as the backup performed is incomplete.
• When executing a hot backup, the backup may be created with the entire cluster in a non-conforming state if multiple containers have been created. Where necessary, ban transaction services so that the backup can be executed in the static state.
• In GridDB, data will be automatically re-arranged when a failure occurs. As a result, when a failure occurs during a backup, it may no longer be possible to back up the necessary data. When a failure occurs, perform the backup again starting with the first node.

To restore backup data to a node,

follow the procedure below (this will restore an entire cluster from backup data).

• Check that no node has been started.
  • Check that the cluster definition file is the same as the other nodes in the cluster that the node is joining.
• Check whether past transaction log files and checkpoint files have been left behind in the database file directory (/var/lib/gridstore/data by default) of the node.
  • Delete if unnecessary and move to another directory if required.
• Execute the restore command on the machine executing the node.
• Start a node.
• Join a cluster.

Execute the following command.

• gs_backuplist -u user name/password

Below is a specific example to display a list of the backup names. A list of the backup names can be displayed regardless of the startup status of the nodes. The Status appears as Processing if the backup process is in progress with the nodes started.

[Command execution example]

$ gs_backuplist -u admin/admin

BackupName Status StartTime EndTime
------------------------------------------------------------------------
 20141025NO2     P   2014-10-25T06:37:10+0900 -
 20141025 NG 2014-10-25T02:13:34+0900 -
 20140925 OK 2014-09-25T05:30:02+0900 2014-09-25T05:59:03+0900
 20140825 OK 2014-08-25T04:35:02+0900 2014-08-25T04:55:03+0900

The backup status (Status) will be one of the following.

• OK: Normal
• NG: Error
• P: In progress

[Points to note]

• If the status displayed is NG, the backup file may be damaged and so restoration is not possible.

An execution example to restore backup data is shown below. Stop the nodes first prior to executing the restoration command.

[Command execution example]

$ mv ${GS_HOME}/data/*.{dat,log} ${GS_HOME}/temp    # Move the database file
$ gs_restore 20130521_backup                        # restore

As a result, the following process will be executed.

1. Copy the backup file group from the 20130521_backup directory under the backup directory (/var/lib/gridstore/backup) to the data directory (/var/lib/gridstore/data).

In this example, the backup directory is /var/lib/gridstore/backup and the data directory is /var/lib/gridstore/data as a matter of convenience but in an actual operation, change the directory to a suitable directory to match the system configuration. (See Other Parameters)

After restoration is completed, restart the nodes and re-add them to the original cluster.

[Command execution example]

$ gs_startnode
...
$ gs_joincluster -c [Configured cluster name] -n 1 -u admin/admin
...

Once restarted, the node will import the database file (backup file group) and then start the services back on. After import ends, the node will start the services.

[Points to note]

• Take note of the no. of partitions and the parameter of the processing parallelism in the cluster definition file. Set the configuration value of the node to restore to be the same as the configuration value of the backup node. The node cannot start correctly if it is not the same.
• If you want to restore the backup state correctly, the backup and restoration tasks need to be carried out for the entire cluster.
• For example, even if some of the nodes are restored, these nodes cannot be returned to the state they were in at the time of the backup. After restoration, it is necessary to let the nodes join the cluster in operation in order to use the data. However, if the data is updated in the cluster after backup, the restored data will be updated by the (updated) cluster data.
• In particular, if the cluster configuration changes from the time the backup was created, there will be no restoration effect. As the data will be autonomously re-arranged if the node is forced to join a cluster, the data will become invalid with a high probability even if restored.
• If data is missing in the backup data file, or if the contents have been revised, the node will not be able to start services.

Additional nodes can be added to a running cluster having a specified number of nodes (the number is specified in the cluster configuration gs_joincluster).

Follow the procedure below to add additional nodes to a running cluster.

• Ensure that the cluster is running.
• Check the status of the cluster.
• Start the nodes that you want to add.
  • Check the cluster definition file of the node you want to add is the same as that of the other nodes of the cluster which you want to add the node to.
• Execute the node addition command on the “node to be added”.
  • Get the cluster information of the nodes to be added with a gs_stat command, and if the cluster status turns to FOLLOWER, the node will be able to join the cluster.

To increase the number of nodes, execute the following command.

• gs_appendcluster --cluster connection server: port [-s connection server: port] -u user name/password

Specify the server address and port (for the operating command) of “any one of the nodes constituting the cluster where the node is to be added” in the cluster option. A specific example on adding a new node to a cluster is shown below.

Check the status of the cluster to add the node.

[Command execution example]

$ gs_stat -s 192.168.33.29:10040  -u admin/admin
{
　　　　:
    "cluster":{                          // cluster-related
        "activeCount":5,                   // number of nodes already participating in a cluster
        "clusterName":"function_1",        // cluster name
        "clusterStatus":"MASTER",          // cluster status
        "designatedCount":5,               // number of nodes constituting a cluster (predetermined number of nodes)
        :
        

A cluster can be added if the number of nodes already participating in a cluster (number of nodes currently joined to a cluster) is equal to the number of nodes constituting a cluster. If the number of nodes constituting a cluster > number of nodes already participating in a cluster, use gs_joincluster (join a cluster configuration) to add a node to the cluster.

Execute the following command on the machine to which the node will be added. Specify the server address and port (for operating command) of any of the nodes in the cluster (the node does not have to be a master node).

[Command execution example]

$ gs_startnode
$ gs_appendcluster --cluster 192.168.33.29:10040 -u admin/admin

After adding the nodes, the number of nodes constituting a cluster and the number of nodes already participating in a cluster will be changed.

[Command execution example]

$ gs_stat -u admin/admin
{
　　　　　　:
    "cluster":{                                 // cluster-related
        "activeCount":6,                        // number of nodes already participating in a cluster
        "clusterName":"function_1",             // cluster name
        "clusterStatus":"MASTER",               // cluster status
        "designatedCount":6,                    // number of nodes constituting a cluster (predetermined number of nodes)
　　　　　　:
}

[Points to note]

• Since the number of nodes constituting a cluster is required during cluster restart, make a note of this number using the gs_stat command when cluster expansion is carried out.
• Non-stop expansion of a GridDB cluster (node increase) will be carried out 1 unit at a time.
• In the case of large scale expansion, stop and re-configure the cluster instead.

In the case where a single node needs to be removed from a running GridDB cluster,

follow the procedure below.

• Check that the cluster is running.
• Execute the cluster detachment command on the node which needs to be detached.

Execute the following cluster detachment command.

• gs_leavecluster [-s connection server: port] [-f] -u user name/password

  [Command execution example]

  $ gs_leavecluster -u admin/admin
  

  [Points to note]

  • If there is a possibility of data lost resulted from node detachment, cluster reduction will not be able to be carried out. To force node detachment, use the –f command.
  • Continuous node reduction will be carried out 1 node at a time.
  • A cluster will be terminated if the number of nodes existing in the cluster is less than half of the number of nodes constituting the cluster. When a large scale node reduction is required, stop and re-configure the cluster with the new number of nodes. However, please note that when a large scale reduction is carried out, the possibility of data lost increases.

Follow the procedure below to update the software.

1. Stop the cluster
2. Stop the node
3. Make a backup copy of the definition file, database file and event log file
4. Update the software
5. Start the node
6. Configure the cluster

An example of the command execution in a machine in which the nodes have been started is shown below.

[Command execution example]

$ gs_stopcluster -u admin/admin
$ gs_stopnode -u admin/admin
$ cp -rp /var/lib/gridstore/data /xxx/shelter  # copy just in case
$ cp -rp /var/lib/gridstore/log /xxx/shelter   # copy just in case
$ cp -rp /var/lib/gridstore/conf /xxx/shelter  # copy just in case
$ su
# rpm -Uvh griddb-server-Y.Y.Y-linux.x86_64.rpm
# rpm -Uvh griddb-client-Y.Y.Y-linux.x86_64.rpm
# rpm -Uvh griddb-docs-Y.Y.Y-linux.x86_64.rpm
# exit
$ gs_startnode
$ gs_joincluster -c configured cluster name -u admin/admin

*Y.Y.Y: Version of GridDB to update

When a failure occurs, the flow of operations carried out by the administrator will be as follows.

1. First, check the cluster operating status and identify which node has been withdrawn from the cluster (failed node).
2. Get the failed node event log data.
3. Analyze the event log data and check the cause of failure.
4. Remove the failed node and replace with a new node.

Even if a failure occurs, GridDB will perform a failover in the cluster to continue services. As long as there is a backup node (replica) in operation, failover will be performed repeatedly. An error will result when there are no more backup nodes.

The following action is carried when a failure occurs in GridDB.

1. The failed node is automatically detached from the cluster.
2. Failover is carried out in the backup node to replace the failed node.

The node status can be checked using the get node data command. Check the status individually for all nodes.

Command execution example

[Command execution example]

$ gs_stat -u admin/admin
{
　　　　　　　　:
    "cluster": {
　　　　　　　　:
        "nodeStatus": "ACTIVE",
　　　　　　　　:
}
# Repeat execution for all nodes

Check the value of /cluster/nodeStatus output. Symptoms of a failed node are as follows.

• Detaches itself unexpectedly from the cluster
  • When the status is INACTIVE, DEACTIVATING, ABNORMAL, NORMAL_SHUTDOWN.
• Does not return results

To identify the cause of node failure, get the most recent event log from the failed node using the command below.

• gs_logs [-s connection server: port] -u user name/password

An example of the command execution is shown below.

[Command execution example]

$ gs_logs -u admin/admin
2015-03-20T10:07:47.219+0900 host01 22962 INFO CHECKPOINT_SERVICE [30902:CP_STATUS] [CP_START] mode=NORMAL_CHECKPOINT, backupPath=
　:
　:

Event log contains a list of character strings on event data. The format of event data is as follows.

• Time, machine name, thread no., event type, event category, generation source file name, generation method, generation row number: followed by any character string (*)

(*) If a new line is returned, a space is assigned to the first character of the new line

Although the cause of failure can be identified directly from the event log data, please check with the service desk for detailed analysis.

[Points to note]

• Due to log rotation, multiple event log files are created. However, only data of the current event log file will be displayed.

Remove the machine running the faulty node from the network. Then, install and set up a GridDB node in a new machine, and add to the new cluster. As explained in the “Adding a node to a cluster in operation” section, the cluster will then automatically redistribute the data and services will be started by the newly added node once redistribution is completed.

Parameters	Data type	Meaning	Default
/dataStore/dbPath	string	Database file directory	"data"
/dataStore/backupPath	string	Backup file directory	"backup"
/dataStore/storeMemoryLimit	string	Memory buffer size	"1024MB"
/dataStore/storeWarmStart	boolean	Warm start during restart (false: invalid, true: valid)	true
/dataStore/concurrency	int	Processing parallelism	1
/dataStore/logWriteMode	int	Log write mode	1
/dataStore/persistencyMode	string	Persistency mode	"NORMAL"
/dataStore/affinityGroupSize	int	Number of affinity groups	4
/checkpoint/checkpointInterval	string	Checkpoint execution interval	"1200s"
/checkpoint/checkpointMemoryLimit	string	Checkpoint memory buffer size	"1024MB"
/checkpoint/useParallelMode	boolean	Checkpoint parallel operation (false: invalid, true: valid)	false
/cluster/serviceAddress	string	Reception address used for cluster administration	"127.0.0.1"
/cluster/servicePort	int	Reception port used for cluster administration	10010
/sync/serviceAddress	string	Reception address used in data synchronization	"127.0.0.1"
/sync/servicePort	int	Reception port used in data synchronization	10020
/system/serviceAddress	string	Connection address of operation command	"127.0.0.1"
/system/servicePort	int	Connection port of operation command	10040
/system/eventLogPath	string	Output directory of event log file	"log"
/transaction/serviceAddress	string	Reception address of transaction process	"127.0.0.1"
/transaction/servicePort	int	Reception port of transaction process	10001
/transaction/connectionLimit	int	No. of connections upper limit value	5000
/trace/default	string	Event log output level	"LEVEL_ERROR"
/trace/dataStore	string		"LEVEL_ERROR"
/trace/collection	string		"LEVEL_ERROR"
/trace/timeSeries	string		"LEVEL_ERROR"
/trace/chunkManager	string		"LEVEL_ERROR"
/trace/objectManager	string		"LEVEL_INFO"
/trace/checkpointFile	string		"LEVEL_ERROR"
/trace/checkpointService	string		"LEVEL_INFO"
/trace/logManager	string		"LEVEL_WARNING"
/trace/clusterOperation	string		"LEVEL_INFO"
/trace/clusterService	string		"LEVEL_ERROR"
/trace/syncService	string		"LEVEL_ERROR"
/trace/systemService	string		"LEVEL_INFO"
/trace/transactionManager	string		"LEVEL_ERROR"
/trace/transactionService	string		"LEVEL_ERROR"
/trace/transactionTimeout	string		"LEVEL_WARNING"
/trace/sessionTimeout	string		"LEVEL_WARNING"
/trace/replicationTimeout	string		"LEVEL_WARNING"
/trace/recoveryManager	string		"LEVEL_INFO"
/trace/eventEngine	string		"LEVEL_WARNING"
/trace/triggerService	string		"LEVEL_ERROR"

The parameters used in the GridDB Advanced Edition in addition to those mentioned above are as follows.

Parameters	Data type	Meaning	Default
/sql/serviceAddress	string	Reception address for JDBC/ODBC client connection	"127.0.0.1"
/sql/servicePort	int	Reception port for JDBC/ODBC client connection	20001
/sql/connectionLimit	int	No. of connections upper limit value	5000
/sql/concurrency	int	Processing parallelism	5
/trace/sqlService	string	Event log output level	"LEVEL_ERROR"

Parameters	Data type	Meaning	Default
/dataStore/partitionNum	int	Number of partitions	128
/dataStore/storeBlockSize	string	Block size ("64KB", "1MB")	"64KB"
/cluster/clusterName	string	Cluster name	""
/cluster/replicationNum	int	No. of replicas	2
/cluster/notificationAddress	string	Multicast address for cluster administration	"239.0.0.1"
/cluster/notificationPort	int	Multicast port for cluster administration	20000
/cluster/notificationInterval	string	Interval of multicast for cluster administration	"5s"
/cluster/heartbeatInterval	string	Heartbeat interval	"5s"
/cluster/loadbalanceCheckInterval	string	Load balance check interval	"180s"
/cluster/notificationMember	array	Address list used in the fixed list method
/cluster/notificationProvider/url	string	URL of address provider used in the provider method
/cluster/notificationProvider/updateInterval	int	Interval to get list from address provider	"5s"
/sync/timeoutInterval	string	Short-term synchronization timeout time	"30s"
/transaction/notificationAddress	string	Address for multi-cast distribution to client	"239.0.0.1"
/transaction/notificationPort	int	Port for multi-cast distribution to client	31999
/transaction/notificationInterval	string	Interval of multicast distribution to client	"5s"
/transaction/replicationTimeoutInterval	string	Replication/timeout time	"10s"
/transaction/replicationMode	int	Replication method (0: asynchronous, 1: semi-synchronous)	0
/transaction/authenticationTimeoutInterval	string	Authentication timeout time	"5s"

The parameters used in the GridDB Advanced Edition in addition to those mentioned above are as follows.

Parameters	Data type	Meaning	Default
/sql/notificationAddress	string	Address for multi-cast distribution to JDBC/ODBC client	"239.0.0.1"
/sql/notificationPort	int	Multicast port to JDBC/ODBC client	41999
/sql/notificationInterval	string	Interval of multicast distribution to JDBC/ODBC client	"5s"