Using Multiple Servers
Now that the basic scenario is working, you can expand the scenario to include multiple MQTT servers. For example, these MQTT servers may collect data in different buildings or factories while Kafka and GridDB run on servers located in a data center or they may be in the same data center as the Kafka and GridDB instances and are exposed to the outside sensors via a DNS round robin or other load balancing mechanism.
Following the instructions from the simple scenario, you can learn how to set up Kafka, Mosquitto, Kafka MQTT Source Connector with a few exceptions:
Do not start the Kafka and Zookeeper servers as they will cause a conflict and the MQTT Source Connector will not deliver messages to the centralized Kafka Server. The libraries and other components are still required for the MQTT Source Connector to run.
On the centralized Kafka server, make sure port
9092 is accessible from the remote MQTT instances completely disable the firewall on the centralized server.
Mqtt Source ConnectorYou will need to edit
bootstrap.serversto the IP address of your Kafka Server.
Now you can run the MqttClient program on all of the remote MQTT instances and be able to view the logged messages either with the kafka-console-consumer or GridDB Sink Connector.
Complete source code used for the GridDB Kafka and MQTT Application can be downloaded from the following: