Using GridDB’s C/Python/Ruby APIs

Introduction

Most databases today have support for many frameworks and programming languages. Some of the commonly supported languages are Java, C, Ruby, and Python. GridDB is no different. GridDB now provides toolkits to developers with APIs to C, Python, and Ruby programming languages.

Setting Up the C API

C lang

GridDB’s C client API has support for CentOS 6.7 and CentOS 7.3. The client is compatible gcc version-4.8.5 and can be used with either Community or Standard edition. The C client’s Standard Edition version supports all the functionality as its Java counterpart with support for Geometry columns and operations etc. The files and instructions to setup the C client for Community Edition can all be found on the GridDB Github page

If for some reason, either the configure or bootstrap.sh scripts throw errors, install the following libraries with these commands.

$ sudo yum groupinstall ‘Development Tools’
$ sudo yum makecache fast

When running the make command, if the error **recursive error is thrown, a quick fix is to ensure the above libraries are installed. Then remove the c_client directory by obtaining and cloning a new one from Github C repository.

From there you can test the client by running a few sample programs provided in the c_client repository to test the client and your GridDB connection.

Some things to note about compiling and linking C client files with GridDB. When compiling with gcc, the include (-I) flag should point to an include directory with the gridstore.h file. The library, or (-L) flag should always point to the bin directory in the c-client folder that has .so file and symbolic links.

$ ls c_client/bin
libgridstore.so
libgridstore.so.0
libgridstore.so.0.0.0

You should use the -lgridstore option as well to successfully compile the file.

Also for your compiled executables to successfully run, the environment variable known as LD_LIBRARY_PATH needs to be set. This variable needs to point to the bin folder in the GridDB C-client directory.

Example compilation of a C source code file.

$ export LD_LIBRARY_PATH=/path/to/c_client/bin
$ gcc -o cSample cSample.c -I./include -L./bin -lgridstore
$ ./cSample 239.0.0.1 31999 defaultCluster admin admin

A more detailed for the C API reference can be found here.

Python/Ruby API Setup

GridDB gives support for Python 2.6, 2.7, and now Python 3.6.

One important thing to successfully build the Python library you will need to have the C client successfully built. You will also need the development kits for Python and Ruby installed as well.

$ sudo yum install ruby-devel
$ sudo yum install python-devel

You will also need the SWIG and pcre libraries as well. Installation instructions and sample programs can be found at the griddb_client page as well.

An important thing to know about using the Python or Ruby library is that the C client has to be installed and setup because variables in the client’s Makefile point to the c_client library.

If you have the C client setup then your environment variable LD_LIBRARY_PATH should be set. You will need this variable’s value to successfully build the client library files for both Ruby and Python.

Simply edit this line in the Makefile.

LDFLAGS = -L/path/c_client/library/bin -lpthread -lrt -lgridstore

In the Makefile, the default version of Python used is Python 2.6. If you want to use 2.7 or Python 3.6 simply edit line 11 of the Makefile so the make command can successfully run.

Example of changing Makefile Python version from 2.6 to 2.7:
Line 11 of Makefile:

INCLUDES_PYTHON = $(INCLUDES) -I/usr/include/python2.6 ## Before Makefile used (Python 2.6)
## Change it to
INCLUDES_PYTHON = $(INCLUDES) -I/usr/include/python2.7 ## After, now uses (Python 2.7)

In your source code, simply import the griddb_python_client file so you have access to the gridstore functions.

A reference for Python’s API can be found here.

Ruby API Features

When cloning the repository for the GridDB Python client, you will also get all the files needed to build and run the Ruby client as well. GridDB supports Ruby version 1.8 and version 2.4. Details on how to build and run the Ruby client can be found in the ReadME.md of the griddb_client page.

Similar rules apply to the Ruby client as the Python client. The one difference is that in your source code simply require the ‘griddb_ruby_client’ which refers to the griddb_ruby_client.so file that contains the Ruby client library.

A reference for Ruby’s API can be found here.

Connecting to your GridDB Cluster

For the most part connecting to a GridDB cluster in the C/Python/Ruby APIs are similar to connecting to GridDB in the Java API. Create a Properties object or similar data structure and fill in the fields with the host, port, and configuration settings of the database you wish to connect to. Then simply use a StoreFactory object to fetch a connection with those properties to retrieve a Gridstore instance which will represent the connection to GridDB.

C

#include "gridstore.h"
// (snip)
GSGridStore* gridstore;
const GSPropertyEntry properties[] = {
	{"notificationAddress",host},
	{"notificationPort",port},
	{"clusterName",clusterName},
	{"user",username},
	{"password",password}
};
size_t propertyCount = sizeof(properties) / sizeof(*properties);
gsGetGridStore(gsGetDefaultFactory(),properties,propertyCount,&gridstore);

Python

#!/usr/bin/python
import sys
import griddb_python_client
# (snip)
griddb = griddb_python_client
factory = griddb.StoreFactory.get_default()
try:
     gridstore = factory.get_store({
	"notificationAddress":host,
	"notificationPort": port,
	"clusterName":cluster,
	"user":user,
	"password":password
	})

Ruby

#!/usr/bin/ruby
$:.unshift File.dirname(__FILE__)
require 'griddb_ruby_client'
Griddb = Griddb_ruby_client
factory = Griddb::StoreFactory.get_default()
gridstore = factory.get_store({
                        "notificationAddress"=> ARGV[0],
                        "notificationPort"=> ARGV[1],
                        "clusterName"=> ARGV[2],
                        "user"=> ARGV[3],
                        "password"=> ARGV[4]
                      }) 

Issuing TQL Queries

In GridDB’s C API there are a variety of built-in functions for querying and aggregating rows in a container. For the most part though the simplest and easiest way to make and issue a query to a container is to use a TQL string. TQL is GridDB’s simplified form of SQL querying.

The query begins as a TQL string. From there a query object is formed by issuing that TQL to a container which can then be fetched to retrieve a resulting set of rows. The resulting row set can then be iterated over to get individual row and column values. Aggregation results are also retrieved from row sets. This flow is fairly the same for the C, Python, and Ruby APIs.

TQL Query in C

Like many other querying languages, you can perform string operations like length checking, substring slicing, and pattern recognition with GridDB’s TQL.

Device nextDevice;
GSQuery* generalQuery;
GSRowSet* generalRowSet;

gsQuery(collection, \ 
"select * where (SUBSTRING(name,8,3)='r12' OR name LIKE '%LUE%R55_') and CHAR_LENGTH(name) > 4", \ &generalQuery);

gsFetch(generalQuery,GS_FALSE,&generalRowSet);
printf("Performing a String Operation on Collection ampCollection151 in TQL\n\n");
while(gsHasNextRow(generalRowSet)){
	GSChar timeString[GS_TIME_STRING_SIZE_MAX];
	gsGetNextRow(generalRowSet,&nextDevice); // Obtain  row object
	gsFormatTime(nextDevice.timestamp,timeString,sizeof(timeString)); // Format timestamp number

	printf("Row in collection ampMeter151: Name=%s",nextDevice.name);
	printf(" Amperage = %.2lf",nextDevice.amperage);
	printf(" Count = %d",nextDevice.count);
	printf(" Timestamp = %s\n",timeString);

Output

Performing a String Operation on Collection ampCollection151 in TQL

Row in collection ampMeter151: Name=ampMeter12 Amperage = 1.45 \ 
Count = 3 Timestamp = 2017-09-15T18:13:52.710Z
Row in collection ampMeter151: Name=BLUESEAMETER555 Amperage = 2.43 \ 
Count = 5 Timestamp = 2017-09-15T18:13:52.710Z

Python

With the exception of Geometry-specific queries and functions, TQL can be used to make any query in both the Python and Ruby API. For example, while there are no built-in functions to perform Timeseries aggregations in Python, they can be issued in TQL. These include time-specific queries and functions in GridDB like time-sampling, time-averages, and interpolation.

TQL Time Query in Python

query = timeseries.query(“select * where timestamp > TIMESTAMPADD(HOUR,NOW(),-12)”)
rowSet = query.fetch(False)
while rowSet.has_next():
	rowSet.get_next(row)
	time = row.get_field_as_timestamp(0)
	voltage = row.get_field_as_double(2)
	print(“Voltage at timestamp: {0} is {1} volts in Timeseries voltmeter2.”.format(time,voltage)

Output

Voltage at timestamp: 1505398224644 is 12.34 volts in Timeseries voltmeter2

Ruby

You can also combine certain aggregation functions with time-specific functions in TQL as well.

Aggregation TQL Query in Ruby

update = false
timestamp = Griddb::Timestamp.current()
aggCommand = "select AVG(voltage) from voltmeter502 where timestamp > TIMESTAMPADD(MINUTE, \
TO_TIMESTAMP_MS(#{timestamp}), -10) AND timestamp < TIMESTAMPADD(MINUTE, \ 
TO_TIMESTAMP_MS(#{timestamp}), 10)"
aggQuery = timeseries.query(aggCommand) 
aggRowSet = aggQuery.fetch(update)
while aggRowSet.has_next()
      aggResult = aggRowSet.get_next_aggregation()
      print "Aggregation Result from voltmeter502: Average voltage = #{aggResult.get_double()}\n"
end

Output

Aggregation result from voltmeter502: Average voltage = 10.5

Inserting Collection into GridDB with C API

In C, schemas for containers can be created and updated into GridDB using either static schemas or dynamic schemas. Static schemas come in the form of struct bindings. Dynamic schemas come in the form of columnInfo and containerInfo objects that can be used to update or insert containers.

Creating a Static Schema with a C-Struct

typedef struct {
	const GSChar* name;
	double amperage;
	int count;
	GSTimestamp timestamp;
} Device; // Static Column Schema in GridDB C API

To use the struct as a schema for a container in GridDB, you need to create a struct-binding for it where you define what is the row-key and explicitly assign a type to each of the fields in the struct.

Creating a Column Schema from a Struct

GS_STRUCT_BINDING(Device,
	GS_STRUCT_BINDING_KEY(name,GS_TYPE_STRING)
	GS_STRUCT_BINDING_ELEMENT(amperage,GS_TYPE_DOUBLE)
	GS_STRUCT_BINDING_ELEMENT(count,GS_TYPE_INTEGER)
	GS_STRUCT_BINDING_ELEMENT(timestamp,GS_TYPE_TIMESTAMP));

From there once the struct has been bound and made into a row schema it can be used in containers.

Inserting a Struct Binding Schema into a Container

GSCollection* collection;
gsPutCollection(gridstore,"ampCollection151",GS_GET_STRUCT_BINDING(Device),NULL,GS_FALSE,&collection);

From there the Device struct can be used as a row object to fetch and insert rows.

Inserting a Struct as a GSRow

Device insertionDevice;
insertionDevice.name = "ampMeter12";
insertionDevice.amperage = 1.45;
insertionDevice.count = 3;
insertionDevice.timestamp = gsCurrentTime();

Having a Struct Represent a Row Schema

Device device;
while(gsHasNextRow(rowSet)){
	gsGetNextRow(rowSet,&device);
	GSChar timeStr[GS_TIME_STRING_SIZE_MAX];
	gsFormatTime(device.timestamp,timeStr,sizeof(timeStr));
	printf("Device in ampCollection151: ");
	printf("name=%s",device.name);
	printf(" amperage=%.2lf",device.amperage);
	printf(" count=%d",device.count);
	printf(" timestamp=%s\n",timeStr);

Output

Device in ampCollection151: name=ampMeter12 amperage=1.45 count=3 \ 
timestamp=2017-09-14T20:05:06.285Z

Python

In both Python and Ruby, the approach to creating schemas and inserting containers is similar to the dynamic approach shown with the C API. Simply give your columns and container a name and define their column types. Determine whether they are TimeSeries or Collections and what indexes they should have. The column types available can be found in the GridDB documentation and API references.

Inserting a Timeseries in Python

timeseries = gridstore.put_container("voltmeter2",[
		("timestamp",griddb.GS_TYPE_TIMESTAMP),
		("active",griddb.GS_TYPE_BOOL),
		("voltage",griddb.GS_TYPE_DOUBLE)
		],griddb.GS_CONTAINER_TIME_SERIES)

In Python since there is no need to explicitly state the types of variables. Row fields can be inserted and fetched a lot more fluidly. All that needs to be known is the column type and index. The created row object will have its row schema set by the container it is based off of. Inserting a row can be done as below.

Inserting a Row in Python

## Create rows and set all Row Fields
insertionRow = timeseries.create_row()
insertionRow.set_field_by_timestamp(0,griddb.Timestamp_add_time(griddb.Timestamp_current(),-6, \ 
 griddb.GS_TIME_UNIT_HOUR))

insertionRow.set_field_by_bool(1,True)
insertionRow.set_field_by_double(2, 12.34)

## Insert Row into Timeseries
timeseries.put_row(insertionRow)

Ruby

The same process is used to create schemas and containers in Ruby as in Python. Simply name the column and set the types and set the type of the container, collection or timeseries.

Inserting a Collection in Ruby

Collection = gridstore.put_container(“collection121”,[
{“name”=>Grid db::GS_TYPE_STRING},
{“status”=>Grid db::GS_TYPE_BOOL},
{“count”=>Grid db::GS_TYPE_LONG},
{“lob”=>Grid db::GS_TYPE_BLOB }
],CONTAINER_TYPE_COLLECTION)

From there the rows can be fetched, inserted, and updated in the same manner as in Python.

Fetching and Updating Rows in Ruby

rowSet.get_next(row)
name = row.get_field_as_string(0)
status = row.get_field_as_bool(1)
count = row.get_field_as_long(2) + 1
lob = row.get_field_as_blob(3)
print "Row in Collection: collection121, Person: name=#{name} status=#{status} count=#{count} lob="
p lob.unpack("U*")
row.set_field_as_long(2,count)
rowSet.update_current(row)

Output

Row in Collection: collection121, Person: name=secondRow status=false \
 count=6 lob=[65, 66, 67, 68, 69, 70, 71, 72, 73, 74]

As seen from the above examples, interactions with GridDB can be understood and used by many languages. This illustrates how GridDB provides wide support through easy to use APIs, it can be used as a base to create a wide variety of applications.

Source Code

All the code snippets seen here were from sample programs that you can download and run from the link below. There are 3 in total, one in Ruby, one in Python, and one in C.
api_samples.tar.gz (156 downloads)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.