Build and Run Project Components
Source Code
Complete source code used for the GridDB Kafka and MQTT Application can be downloaded from the following:
Download: griddb-kafka-mqtt-application.tar.gz
Mqtt Client
To test the client you will need to have the Kafka connector and mqtt-kafka-connector set up and running beforehand.
From the mqttclient directory, you can run the following commands to build and start the simple mqttclient. The mqttclient mimics a sensor reading and uses libmosquitto to publish the data to the mosquitto server where it can be read by subscribers to that server such as our Kafka mqtt-source-connector.
$ make $ ./mqttclient
If all components are operating correctly, the kafka-consumer process will be running and you will see data in this form
{"schema": {"type":"string","optional": false}, "payload": "{light":42.0 , "sound": 24.0 }" }
This means the mosquitto client program can successfully publish its data and send it to the kafka process through the connector
GridDB Sink Connector
To build the sink connector, gridstore.jar needs to be placed into a local maven repository.
$ cp /usr/share/java/gridstore.jar ~/.m2/repository/com/toshiba/mwcloud/gs/gridstore/3.0/gridstore-3.0.jar
Now, you can build the connector:
$ ./gradlew build
To run the connector, you’ll also need to download the jsonic JSON parser and include it and the connector JAR and gridstore.jar in your CLASSPATH.
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore..jar:$PWD/build/libs/griddb-kafka-sink.jar:$PATH_TO_JSONIC/jsonic-1.3.0.jar
Now you can run the connector:
$ connect-standalone.sh connect-standalone.properties griddb-sink.properties
If the Sink is able to connect GridDB and the other components are still running normally, then the terminal should output messages such as:
Putting Timestamp (Current Time and Date) Light : 42 Sound : 24 into Test
GridDB Data Viewer
Chang into your griddb-kafka-sink repository and run
$ export CLASSPATH=${CLASSPATH}:/usr/griddb-3.0.1/bin/gridstore-3.0.1.jar
$ cd utils
$ javac DataViewer/*.java
$ java DataViewer/DataViewer 239.0.0.1 31999 YOUR_CLUSTER_NAME admin password
If the kafka-consumer, the kafka-mqtt-connector, the griddb-kafka-sink, and mqttclient processes are all running properly, then this terminal should print the output of what timestamps and data the mqttclient added to the database following the below format:
Timestamp Mon Jun 19 16:10:10 EDT Light: 42 Sound: 24