Getting Started with GridDB’s Golang Client

Introduction

GridDB has released a database connector for Google’s Go programming language. Like the Python and Ruby apis for GridDB, it was built using the SWIG library. The connector supports versions of Go 1.9 or higher and can work on CentOS versions 6 and 7.
Go or Golang is an open-source programming language developed by Google. It is optimized for parallel and concurrent processing through the use of lightweight threads known as goroutines. This makes Go a high performance programming language that pairs well with a fast, in-memory oriented database like GridDB.
In this post, we will go through common steps involved in setting up and testing a database connector for GridDB, in this case it will be for our Go client.

Go Client Installation & Setup

Begin by cloning the Github repository for the Go API.

$ git clone https://github.com/griddb/go_client

To build the Go client on your system, you will need to have the GridDB C client built and installed. You can follow this blog post on how to set up and test the C client if you are not familiar.

• NOTE: before you make and build the Go client on your system, ensure that you have Golang version 1.9 or higher. You can follow instructions on this page which explains how to get and configure Golang version 1.9.

Now that you have obtained the source code from Github, you can simply follow the instructions in the README on the Go Client’s Github page.

Possible Issues When Building the Go Client

Occasionally, this error may occur when issuing make on the Go client package:

go install github.com/griddb/go_client
can't load package: package github.com/griddb/go_client:
  cannot find package "github.com/griddb/go_client"

This could likely be due to issues with your GOPATH or GODEBUGenvironment variables. A quick correction is to clean the GridDB Go client project directory and then setting them to their proper values.

$ make clean
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<Path to GridDB c_client bin/ directory>
$ export GOPATH=$GOPATH:<Path to go_client directory>
$ export GODEBUG=cgocheck=0
$ make

NOTE: Ensure you always have GODEBUG equal to cgocheck=0, to prevent any unexpected runtime errors.

Connecting to GridDB with Golang

If you wish to connect to GridDB in a Golang program, simply import the Go API with: import statement ( "github.com/griddb/go_client" ). The name of the GridDB package for Golang is griddb_go.
From there you can obtain a StoreFactoryInstance from the GridDB library and connect it to your GridDB cluster with the factory instance’s .GetStore() method. The parameters you give this method are a map[string]interface{}{} object (equivalent of a HashMap or Python dictionary with string-keys). The keys are simply the configuration fields you want to specify to connect to GridDB.
Note that the port must be specified as an integer while all the other database configuration values like host or cluster name can be strings.

package main
import (
	"github.com/griddb/go_client"
	"fmt"
	"os"
	"strconv"
)
func main(){
	factory := griddb_go.StoreFactoryGetInstance() //Get a GridStoreFactory instance
        //The 'notificationPort' value to connect to GridDB must be an integer
        //when using the StoreFactory.GetStore() method to connect to GridDB
        //Read notification-port from command-line
        //Parse port reading to integer
	port, err := strconv.Atoi(os.Args[2])
	if err != nil {
		fmt.Println(err)
		os.Exit(2)
	}
        //Connect to GridDB cluster
        //Note that port must be an integer
        gridstore := factory.GetStore(map[string]interface{}{
                "host": os.Args[1], // "239.0.0.1"
                "port": os.Args[2], // 31999
                "cluster_name": os.Args[3], //"defaultCluster"
                "user": os.Args[4], // "admin"
                "password": os.Args[5], // "admin"
                //"notification_member": os.Args[6],
                //"notification_provide": os.Args[7],
        })
        //Close access to GridDB
        gridstore.Close(0)
        //Note that you must specify an integer flag when using the .Close() function call
}

Creating Schemas and Containers

Schemas for containers and rows can be created using the interface types in Golang.
You can simply use a two-dimensional array of interfaces to map the column types for your container. From there you can create a container schema (ContainerInfo Type) by using the StoreFactory.CreateContainerInfo() method. You can use and insert these schemas to create containers.
Rows can be modeled as interface arrays ( []interface{} ). To create a row just create an interface array and have each index’s value map to the corresponding type it is supposed to store in the container.
For example, if in a container the second column stores an integer named 'count', then the value 3 from index 1 in the array row := []interface{}{"A_1",3,true} would be mapped to the 'count' column.

//Container schema for our timeseries container
//The first column is always the row-key
row_schema := [][]interface{}{
		{"timestamp",griddb_go.TYPE_TIMESTAMP},
		{"water_temperature",griddb_go.TYPE_FLOAT},
		{"turbidity",griddb_go.TYPE_FLOAT},
		{"depth",griddb_go.TYPE_FLOAT},
		{"wave_height",griddb_go.TYPE_FLOAT},
		{"wave_period",griddb_go.TYPE_FLOAT},
		{"battery_life",griddb_go.TYPE_FLOAT},
		{"beach_name",griddb_go.TYPE_STRING}}
timeseries_name := "water_monitor_ts_2" //Name for our container
//Create a container info to use to insert containers into GridDB
containerInfo, err := griddb_go.CreateContainerInfo(timeseries_name,
                        row_schema,
                        griddb_go.CONTAINER_TIME_SERIES,
                        true)
timeseries := gridstore.PutContainer(containerInfo,false)
//Use an interface array to create and insert rows
row := []interface{}{time.Now().UTC(),16.2,1.26,1.514,0.147,4.0,11.7,"Calumet Beach"}
// You can also initialize an empty row with ( row := make([]interface{},8) )
//Insert row into Timeseries container
err = timeseries.Put(row)

You can also obtain a Row with a simple call to the .Get() method on the container with its row-key. As mentioned before, a row is just an array of generic Golang interfaces, all one needs to do to access the column value they want to retrieve is to reference the row-index of the column.
For example, let us have a container with a column 'longitude' of type float that is the fourth column in the container’s schema. When you retrieve a row from this container, to access the longitude column all that needs to be done to access the fields is access index 4.
You can also remove rows in a container by row-key as well.

 //Schema of our collection container
schema, err := griddb_go.CreateContainerInfo(collection_name,[][]interface{}{
		{"sid",griddb_go.TYPE_STRING},
		{"name",griddb_go.TYPE_STRING},
		{"type",griddb_go.TYPE_STRING},
		{"latitude",griddb_go.TYPE_DOUBLE},
		{"longitude",griddb_go.TYPE_DOUBLE},
		{"installation",griddb_go.TYPE_TIMESTAMP}},
		griddb_go.CONTAINER_COLLECTION,
		true)
collection, err := gridstore.PutContainer(schema,false)
// (snip)
//Find a row in the collection container with row-key ('sid') "W_1003"
retrieved, err := collection.Get("W_1003")
if len(retrieved) > 0 {
                //Access and display longitude column of row (is column 4 in row)
                fmt.Printf("Longitude value of row with row-key %s is %f\n","W_1003",retrieved[4]
}
//We can also remove the "W_1003" row by calling .remove()
collection.Remove("W_1003")
//Attempt to access row again, this should return an empty row (an empty array) if the removal was successful
attempt, err := collection.Get("W_1003")
if len(attempt) == 0 {
                fmt.Printf("Row with key %s removed successfully\n","W_1003")
}

Querying and Retrieving Data

Once you have your containers populated with data and inserted into GridDB, you are ready to query and fetch your data. Similar to the Python or Java APIs, all you need to is to construct a Query object (or griddb_go.Query type) with the TQL query you would like to issue to your container. Once that is done, simply fetch the query’s results and store it in a griddb_go.RowSet object.

• Note: , if you want to select specific rows to update, you must set the commit-mode of the container you are trying update to false.

Once you have finished with updating row values using either griddb_go.Container.Put() or griddb_go.RowSet.Update() methods, you can Commit() the changes.

Standard TQL Query

//Set commit-mode to false so you can update the rows
collection.SetAutoCommit(false)
//Search for a row with row-key that ends in '2' and has a column value of 'weather' in the type column
query,err := collection.Query("select * WHERE CHAR_LENGTH(sid) = 6 AND sid LIKE 'W%2' AND LOWER(type) = 'weather'")
//Fetch the rows and make them available for update
rowSet,err := query.Fetch(true)
for rowSet.HasNext(){
        //Retrieve and iterate through the rowset row-by-row
	rrow, err := rowSet.NextRow()
	if err != nil{
		fmt.Println("Error retrieving row")
	}
        //Update the fourth column value of the row
	rrow[4] = -88.0
        //Save updated row back to the rowSet
	rowSet.Update(rrow)
}
//Commit and save the updated RowSet back to the container
collection.Commit()

Aggregation TQL

You can also use TQL to create and issue aggregation queries. The methods of performing and fetching griddb_go.AggregationResult from containers is fairly similar to the Python API. Once you have obtained the RowSet fetched from your query, instead of using rowSet.NextRow() we simply change it to .NextAggregation(). With that AggregationResult object, we use the .Get() to get the type of number we need.

column := "turbidity"
aggQuery, err := timeseries.Query(fmt.Sprintf("SELECT TIME_AVG(%s)",column)) //Get the time-weighted average of all turbidity values in our container
if err != nil {
	fmt.Printf("Failed to issue query to timeseries container %s\n",container_name)
	fmt.Println(err)
	os.Exit(2)
}
aggRowSet, err := aggQuery.Fetch(false) //Issue query and fetch results
if err != nil {
	fmt.Printf("Error retrieving query results from timeseries container %s\n",container_name)
	fmt.Println(err)
	os.Exit(2)
}
for aggRowSet.HasNext(){
	aggregation, err := aggRowSet.NextAggregation() //Get the aggregation result from the rowset.
	if err != nil {
		fmt.Println("Failed to get aggregation result!")
	} else {
		avg, err := aggregation.Get(griddb_go.TYPE_DOUBLE) //Get the aggregation result's numerical value
		if err != nil {
			fmt.Printf("Error parsing aggreagtion result\n")
		}
		fmt.Printf("The time average for the %s column in the %s container is %f\n",column,container_name,avg)
	}
}

Handling Timestamp and Binary Data

Timestamp Data

The Golang client for GridDB provides the same offerings as the other API clients (Python, C) for timestamp manipulation. An important thing to note is that in GridDB, to insert a time or date value into a GridDB container, the value has to able to be cast to a Time.time type in Golang.
What this means is that one cannot insert timestamp values as pure numerical value. If one does, this means that the call to container.Put(row) will fail if the timestamp value is in a row-key column and the data will not be inserted into the container.
Another thing to NOTE is that the GridDB timestamps have a precision in milliseconds.
A way to convert the GridDB timestamp number into type time.Time is to convert the timestamp number into nanoseconds and use time.Unix() to convert it into the time Type.

import (
     "time"
     "github.com/griddb/go_client"
)
// (snip)
//A way to express the current time as a time Type in Golang
now := time.Now().UTC() //This has less precision than GridDB TimestampUtils
// GridDB method of getting the current time as a number
now_ts :=  time.Unix(griddb_go.TimestampUtilsCurrent() * 1000000)// The number is then converted into a UTC time
// Converting GridDB to Time.time values allows for precision and lowers the chance that
// a row will inadvertently replace another row with the same row-key.
row := []interface{}{now_ts.UTC(),true,5.6}
err = timeseries.Put(row)

Binary or Blob Data

In GridDB, binary or BLOB data, can be represented as bytearray. In the case of having to read image data or large buffers simply allocate an empty byte array or [] byte.
The function below parseBlob reads the data from an image into a bytearray.

func parseBlob(filename string) (blob []byte){
	file, err := os.Open(filename) //Open image file
	if err != nil {
		fmt.Printf("Error opening file %s\n",filename)
		fmt.Println(err)
		os.Exit(1)
	}
	defer file.Close()
	fileInfo, _ := file.Stat() //Get meta-information related to the image file
	var size int64 = fileInfo.Size() //Obtain file size
	outputStream := make([]byte,size) //Allocate a byte array that is the size of the image's data
	buffer := bufio.NewReader(file) //Create a file reader for reading byte data from image file
	_, err = buffer.Read(outputStream) //Read image file's data into output byte array
	return outputStream
}

We can then use the data from parseBlob to be the values of our BLOB columns

image_blob := parseBlob("liveimage1.jpg") //Get bytearray for image file's data
now := time.Now().UTC() //Obtain current time
//Create a Row containing our BLOB data (byte array) and insert it into Timeseries container
first_row := []interface{}{now,"Bright Sky",500,"liveimage1.jpg",image_blob)
err = timeseries.Put(first_row)
blob :=  []byte{65, 66, 67, 68, 69, 70, 71, 72, 73, 74} //Create Blob as a simple byte array
third_row := []interface{}{time.Now().UTC(),"Generic",10,"NULL",blob}
//Create and insert GridDB row containing smaller byte array
err = timeseries.Put(third_row)

Conclusion

It should now be quite easy to connect and manage your GridDB cluster using Go. With all the various utilities offered by the GridDB Go client, storing data and developing high-performance Go applications with GridDB should now be a rather fun proposition.

Reference

• Golang version 1.9.2 linux/amd64 was used for this post on a CentOS 7.3 operating system.

• GridDB Community Edition 3.0.1 was used as the GridDB database for this tutorial

• The source code for the GridDB Go client can be found on the official Github repository.

• This reference page lists all the objects and resources from GridDB that can be accessed with the Go client

• Once the Go library for GridDB is built, all the methods and attributes used in the griddb_go package can be found in the src/griddb_go.go files.

• If you want to learn more about how to use Golang, check out the official documentation.

Source Code

All the source code used in this post can be downloaded below.

• [download id=”24823″]

If you have any questions about the blog, please create a Stack Overflow post here https://stackoverflow.com/questions/ask?tags=griddb .
Make sure that you use the “griddb” tag so our engineers can quickly reply to your questions.