Follow the instructions on the GridDB Python Package Index (Pypi)<\/a> page to install these clients. The scripts below will help you install and import the necessary libraries for running the code in this article.<\/p>\n We will use stock market data from Yahoo Finance to train our time series forecasting model. The Output:<\/strong><\/p>\n To connect to GridDB you need to call the The following script shows how to connect to GridDB and test your GridB connection:<\/p>\n Output:<\/strong><\/p>\n A GridDB container is a fundamental data structure in used for storing and managing data in GridDB. To create a container you first need to call the Next, call the Output:<\/strong><\/p>\n The last step is to insert the Yahoo Finance data from the Pandas DataFrame into the GridDB container you created in the previous script. The script below shows how to store Yahoo Finance Data in a GridDB container.<\/p>\n Output:<\/strong><\/p>\n We have now Successfully stored our time series stock market data in GriDB, next we will train a TensorFlow Keras model for time series forecasting.<\/p>\n First we will retrieve data from our GridDB container and store it in a Pandas DataFrame. Output:<\/strong><\/p>\n We will use a Transformer model from TensorFlow Keras<\/a> for time series forecasting in this article. You can also use a long short term memory (LSTM)<\/a> or one-dimensional convolutional neural networks (1D-CNN)<\/a> as well. However, transformers, being the state of the art are likely to outperform the other models.<\/p>\n We will use the The following script preprocesses and normalizes the dataset.<\/p>\n Next, we will define our transformer model architecture. Our model will consist of a multiheaded attention layer<\/a>, followed by two 1-D convolutional neural network layers. We will also add dropout and layer normalization to avoid overfitting. You can modify the model architecture if you want.<\/p>\n =Subsequently, we will define the Next, we pass the model configurations to the Next, we define call backs for early stopping, storing the best model weights, and reducing the learning rate.<\/p>\n Finally, we call the Output:<\/strong><\/p>\n The script below shows the training and validation losses for our model. The curves show that our model is not overfitting.<\/p>\n Output:<\/strong><\/p>\n Let’s evaluate our model’s performance on the training and test sets. Next, we will plot the actual and predicted stock prices side-by-side on a line plot. It is important to note that we have to inverse the effect of data scaling that we did during the data preprocessing step. Output:<\/strong><\/p>\n Finally, the script below plots the actual stock prices for the training set, and the actual and predicted stock prices for the test set.<\/p>\n
\nYou will also need to install TensorFlow, yfinance, Numpy, Pandas, and Matplotlib libraries.<\/p>\npip install yfinance\npython3 -m pip install tensorflow[and-cuda]<\/code><\/pre>\n<\/div>\nimport os\nimport absl.logging\n\nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'\nos.environ['TF_ENABLE_ONEDNN_OPTS'] = '0'\nabsl.logging.set_verbosity(absl.logging.ERROR)\nos.environ['CUDA_VISIBLE_DEVICES'] = '-1' \n\nimport yfinance as yf\nimport pandas as pd\nimport griddb_python as griddb\nimport numpy as np\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nfrom tensorflow.keras.utils import disable_interactive_logging\nimport matplotlib.pyplot as plt\nfrom sklearn.preprocessing import MinMaxScaler<\/code><\/pre>\n<\/div>\nInserting Stock Market Data into GridDB<\/h2>\n
\nIn this section, you will see how to fetch stock market data from Yahoo Finance<\/a>, create a connection with GridDB, and insert the Yahoo finance data into a GridDB container.<\/p>\nFetch Data from Yahoo Finance<\/h3>\n
yfinance.download()<\/code> method allows you to retrieve data from Yahoo Finance into a Pandas dataframe.
\nIn the script below we retrieve Apple’s stock prices for the full year of 2023:<\/p>\nticker = \"AAPL\"\nstart_date = \"2023-01-01\"\nend_date = \"2023-12-31\"\ndata = yf.download(ticker, start=start_date, end=end_date)\nprint(f\"Fetched {len(data)} rows of data for {ticker}\")\ndata.head()\n<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2025\/04\/img1-AAPL-yahoo-finance-stock-data.png\")
<\/a><\/p>\nConnect to GridDB<\/h3>\n
griddb.StoreFactory.get_instance()<\/code> method to get a GridDB factory instance object.
\nNext, you need to create a GridDB factory store object using the get_store()<\/code> method. You will need to pass your GridDB host name, cluster, name and user and password to the get_store()<\/code> method. Finally, you can test your connection by randomly retrieve a GridDB container using the get_container()<\/code> method.<\/p>\n# GridDB connection details\nDB_HOST = \"127.0.0.1:10001\"\nDB_CLUSTER = \"myCluster\"\nDB_USER = \"admin\"\nDB_PASS = \"admin\"\n\n# creating a connection\n\nfactory = griddb.StoreFactory.get_instance()\n\ntry:\n gridstore = factory.get_store(\n notification_member = DB_HOST,\n cluster_name = DB_CLUSTER,\n username = DB_USER,\n password = DB_PASS\n )\n\n container1 = gridstore.get_container(\"container1\")\n if container1 == None:\n print(\"Container does not exist\")\n print(\"Successfully connected to GridDB\")\n\nexcept griddb.GSException as e:\n for i in range(e.get_error_stack_size()):\n print(\"[\", i, \"]\")\n print(e.get_error_code(i))\n print(e.get_location(i))\n print(e.get_message(i))<\/code><\/pre>\n<\/div>\nContainer does not exist\nSuccessfully connected to GridDB<\/code><\/pre>\n<\/div>\nCreate Container for Stock Data in GridDB<\/h3>\n
\nWe will store the Yahoo Finance data we retrieved in a time series type container.<\/p>\nContainerInfo()<\/code> method and pass it the container name, a list of lists containing data columns and types, and the container type which in our case will be griddb.ContainerType.TIME_SERIES<\/code>.<\/p>\nput_container()<\/code> method and pass it as a parameter the container info object you previously created.
\nThe script below shows how to create the AAPL_stock_data<\/code> container in GridDB.<\/p>\ncontainer_name = f\"{ticker}_stock_data\"\ncolumn_info = [\n [\"Timestamp\", griddb.Type.TIMESTAMP],\n [\"Open\", griddb.Type.DOUBLE],\n [\"High\", griddb.Type.DOUBLE],\n [\"Low\", griddb.Type.DOUBLE],\n [\"Close\", griddb.Type.DOUBLE],\n [\"Volume\", griddb.Type.LONG]\n]\ncontainer_info = griddb.ContainerInfo(container_name, column_info, griddb.ContainerType.TIME_SERIES)\n\ntry:\n gridstore.put_container(container_info)\n container = gridstore.get_container(container_name)\n if container is None:\n print(f\"Failed to create or retrieve container: {container_name}\")\n else:\n print(f\"Successfully created and retrieved container: {container_name}\")\nexcept griddb.GSException as e:\n print(f\"Error creating or retrieving container {container_name}:\")\n for i in range(e.get_error_stack_size()):\n print(f\"[{i}]\")\n print(f\"Error code: {e.get_error_code(i)}\")\n print(f\"Location: {e.get_location(i)}\")\n print(f\"Message: {e.get_message(i)}\")<\/code><\/pre>\n<\/div>\nSuccessfully created and retrieved container: AAPL_stock_data<\/code><\/pre>\n<\/div>\nInsert Data into GridDB Container<\/h3>\n
\nTo do so, you can iterate through all the rows of a Pandas DataFrame, call the container’s put()<\/code> method and pass it the data you want to store in the container.<\/p>\ntry:\n for index, row in data.iterrows():\n container.put([index.to_pydatetime(), row['Open'], row['High'], row['Low'], row['Close'], int(row['Volume'])])\n print(f\"Successfully inserted {len(data)} rows of data into {container_name}\")\nexcept griddb.GSException as e:\n print(f\"Error inserting data into container {container_name}:\")\n for i in range(e.get_error_stack_size()):\n print(f\"[{i}]\")\n print(f\"Error code: {e.get_error_code(i)}\")\n print(f\"Location: {e.get_location(i)}\")\n print(f\"Message: {e.get_message(i)}\")<\/code><\/pre>\n<\/div>\nSuccessfully inserted 250 rows of data into AAPL_stock_data<\/code><\/pre>\n<\/div>\nCreating a Stock Market Forecasting Model Using TensorFlow Keras<\/h2>\n
Retrieving Data from GridDB<\/h3>\n
\nTo do so, call the get_container()<\/code> method and pass to it the name of the container you want to retrieve.
\nNext, call SELECT *<\/code> query on the container using the query()<\/code> method.
\nCall the fetch()<\/code> method to run the query and finally the fetch_rows()<\/code> function to store returned records in a Pandas DataFrame.<\/p>\ndef retrieve_data_from_griddb(container_name):\n\n try:\n stock_data_container = gridstore.get_container(container_name)\n\n # Query all data from the container\n query = stock_data_container.query(\"select *\")\n rs = query.fetch() # Adjust the number based on your data size\n\n data = rs.fetch_rows()\n data .set_index(\"Timestamp\", inplace=True)\n return data\n\n except griddb.GSException as e:\n print(f\"Error retrieving data from GridDB: {e.get_message()}\")\n return none\n\nstock_data = retrieve_data_from_griddb(\"AAPL_stock_data\")\nstock_data.head()\n<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2025\/04\/img2-data-retrieved-from-griddb.png\")
<\/a><\/p>\nData Preprocessing for TensorFlow Keras Transformer Model<\/h3>\n
Open<\/code> and Volume<\/code> stock prices for the last seven days to predict the Open<\/code> stock price for the next day.
\nTo do so, we will divide our data into into feature (X<\/code>) and labels (y<\/code>) set, and the into training (80%) and test(20%) sets. We will also normalize our data since deep learning models are known to work better with the normalized data.<\/p>\nfeatures = ['Open', 'Volume']\ndata = stock_data[features].values\n\n# Initialize the scaler\nscaler = MinMaxScaler(feature_range=(0, 1))\n\n# Fit the scaler to the data and transform\ndata_normalized = scaler.fit_transform(data)\n\n# Create sequences\ndef create_sequences(data, seq_length):\n X, y = [], []\n for i in range(len(data) - seq_length):\n X.append(data[i:(i + seq_length), :])\n y.append(data[i + seq_length, 0]) # Predicting next day's Open price\n return np.array(X), np.array(y)\n\n\nseq_length = 7 # stock prices of last 7 days\nX, y = create_sequences(data_normalized, seq_length)\n\n# Split the data into training and testing sets\nsplit = int(0.8 * len(X))\nX_train, X_test = X[:split], X[split:]\ny_train, y_test = y[:split], y[split:]\n<\/code><\/pre>\n<\/div>\nCreating a TensorFlow Keras Transformer Model<\/h3>\n
# Define the Transformer block\ndef transformer_encoder(inputs, head_size, num_heads, ff_dim, dropout=0):\n # Attention and Normalization\n x = layers.MultiHeadAttention(\n key_dim=head_size, num_heads=num_heads, dropout=dropout\n )(inputs, inputs)\n x = layers.Dropout(dropout)(x)\n x = layers.LayerNormalization(epsilon=1e-6)(x)\n res = x + inputs\n\n # Feed Forward Part\n x = layers.Conv1D(filters=ff_dim, kernel_size=1, activation=\"relu\")(res)\n x = layers.Dropout(dropout)(x)\n x = layers.Conv1D(filters=inputs.shape[-1], kernel_size=1)(x)\n x = layers.LayerNormalization(epsilon=1e-6)(x)\n return x + res<\/code><\/pre>\n<\/div>\nbuild_model()<\/code> method that builds our model. The model takes our data features and labels as inputs, pass the data through transformer model we just defined. The output of the transformer model is passed through a global average pooling layer<\/a>, followed by three dense layers to get the final model output.<\/p>\n# Build the model\ndef build_model(\n input_shape,\n head_size,\n num_heads,\n ff_dim,\n num_transformer_blocks,\n mlp_units,\n dropout=0,\n mlp_dropout=0,\n):\n inputs = keras.Input(shape=input_shape)\n x = inputs\n for _ in range(num_transformer_blocks):\n x = transformer_encoder(x, head_size, num_heads, ff_dim, dropout)\n\n x = layers.GlobalAveragePooling1D(data_format=\"channels_first\")(x)\n for dim in mlp_units:\n x = layers.Dense(dim, activation=\"relu\")(x)\n x = layers.Dropout(mlp_dropout)(x)\n outputs = layers.Dense(1)(x)\n return keras.Model(inputs, outputs)<\/code><\/pre>\n<\/div>\nbuild_model()<\/code> function and get the model object back from the function. We call the compile()<\/code> method to compile the model.<\/p>\n# Create the model\ninput_shape = X_train.shape[1:]\nmodel = build_model(\n input_shape,\n head_size=256,\n num_heads=4,\n ff_dim=4,\n num_transformer_blocks=4,\n mlp_units=[128],\n mlp_dropout=0.4,\n dropout=0.25,\n)\n# Compile the model\nmodel.compile(\n optimizer=keras.optimizers.Adam(learning_rate=1e-4),\n loss=\"mse\",\n metrics=[\"mae\"]\n)<\/code><\/pre>\n<\/div>\nfit()<\/code> method an pass it our training data to start model training.<\/p>\n# Define callbacks\ncallbacks = [\n keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True),\n keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=5, min_lr=1e-6),\n]\n\n# Train the model\nhistory = model.fit(\n X_train,\n y_train,\n validation_split=0.2,\n epochs=100,\n batch_size=32,\n callbacks=callbacks,\n)<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2025\/04\/img3-keras-transformer-training-results.png\"){kind=spacer}
<\/a><\/p>\nplt.figure(figsize=(12, 6))\nplt.plot(history.history['loss'], label='Training Loss')\nplt.plot(history.history['val_loss'], label='Validation Loss')\nplt.title('Model Training History')\nplt.ylabel('Loss')\nplt.xlabel('Epoch')\nplt.legend()\nplt.show()<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2025\/04\/img4-model-training-history.png\")
<\/a><\/p>\nEvaluating the Model Performance<\/h3>\n
\nThe output of the following script shows that we receive a mean absolute error score of 0.1596 which is greater than 0.0788. This shows that our model is overfitting on training set.<\/p>\n
\nThe following script also does that.<\/p>\n\ntest_loss, test_mae = model.evaluate(X_test, y_test)\nprint(f\"Test MAE: {test_mae:.4f}\")\n\n# When making predictions and inverse transforming:\ntrain_predictions = model.predict(X_train)\ntest_predictions = model.predict(X_test)\n\n# Inverse transform predictions (for Open price only)\ntrain_predictions = scaler.inverse_transform(np.column_stack((train_predictions, np.zeros_like(train_predictions))))[:, 0]\ntest_predictions = scaler.inverse_transform(np.column_stack((test_predictions, np.zeros_like(test_predictions))))[:, 0]\n\n# Inverse transform actual values\ny_train_actual = scaler.inverse_transform(np.column_stack((y_train.reshape(-1, 1), np.zeros_like(y_train.reshape(-1, 1)))))[:, 0]\ny_test_actual = scaler.inverse_transform(np.column_stack((y_test.reshape(-1, 1), np.zeros_like(y_test.reshape(-1, 1)))))[:, 0]<\/code><\/pre>\n<\/div>\nTest MAE: 0.1596<\/code><\/pre>\n<\/div>\n# Get the actual prices for the entire dataset\ny_actual = np.concatenate([y_train_actual, y_test_actual])\n\n# Create a date range for the entire dataset\nfull_date_range = pd.date_range(start=stock_data.index[seq_length], periods=len(y_actual), freq='B')\n\n# Get the date range for the test set\ntest_date_range = full_date_range[-len(y_test_actual):]\n\n# Plot results\nplt.figure(figsize=(20, 10))\n\n# Plot the entire actual price series\nplt.plot(full_date_range, y_actual, label='Actual', color='blue')\n\n# Plot only the test predictions\nplt.plot(test_date_range, test_predictions, label='Predicted (Test Set)', color='red', linestyle='--')\n\n# Add a vertical line to indicate the start of the test set\nsplit_date = full_date_range[-len(y_test_actual)]\nplt.axvline(x=split_date, color='green', linestyle='--', label='Test Set Start')\n\nplt.title('Stock Open Price - Actual vs Predicted (Test Set)', fontsize=20)\nplt.xlabel('Date', fontsize=16)\nplt.ylabel('Open Price', fontsize=16)\nplt.legend(fontsize=14)\nplt.grid(True, which='both', linestyle='--', linewidth=0.5)\n\n# Rotate and align the tick labels so they look better\nplt.gcf().autofmt_xdate()\n\n\n# Show the plot\nplt.show()<\/code><\/pre>\n<\/div>\n
![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2025\/04\/img5-line-plots-for-actual-predicted-stock-prices.png\")
<\/a><\/p>\n