{"id":46692,"date":"2022-03-23T00:00:00","date_gmt":"2022-03-23T07:00:00","guid":{"rendered":"https:\/\/griddb-linux-hte8hndjf8cka8ht.westus-01.azurewebsites.net\/blog\/modelling-sp-500-index-price-based-on-u-s-economic-indicators-using-python-and-griddb\/"},"modified":"2025-11-13T12:55:52","modified_gmt":"2025-11-13T20:55:52","slug":"modelling-sp-500-index-price-based-on-u-s-economic-indicators-using-python-and-griddb","status":"publish","type":"post","link":"https:\/\/griddb.net\/en\/blog\/modelling-sp-500-index-price-based-on-u-s-economic-indicators-using-python-and-griddb\/","title":{"rendered":"Modelling S&P 500 Index Price Based on U.S. Economic Indicators using Python and GridDB"},"content":{"rendered":"

Economic indicators have been used in numerous studies to forecast stock prices using well-known statistical methods. The rising power of computers since *2010 has resulted in the development of new methods of machine learning. To discover whether those algorithms can predict the stock market using a variety of mathematical and statistical methods would be interesting. The purpose of this article is to model the monthly price of the S&P 500 index based on U.S. economic indicators using GridDB to extract the data, followed by performing the statistical tests and finally building the machine learning model.<\/p>\n

The outline of the tutorial is as follows:<\/p>\n

    \n
  1. Prerequisites and Environment setup<\/li>\n
  2. Dataset overview<\/li>\n
  3. Importing required libraries<\/li>\n
  4. Loading the dataset<\/li>\n
  5. Exploratory Data Analysis & Feature Selection<\/li>\n
  6. Building and Training a Machine Learning Model<\/li>\n
  7. Conclusion<\/li>\n<\/ol>\n

    Prerequisites and Environment setup<\/h1>\n

    This tutorial is carried out in Anaconda Navigator (Python version \u2013 3.8.3) on Windows Operating System. The following packages need to be installed before you continue with the tutorial \u2013<\/p>\n

      \n
    1. \n

      Pandas<\/p>\n<\/li>\n

    2. \n

      NumPy<\/p>\n<\/li>\n

    3. \n

      Scikit-learn<\/p>\n<\/li>\n

    4. \n

      Matplotlib<\/p>\n<\/li>\n

    5. \n

      Statsmodels<\/p>\n<\/li>\n

    6. \n

      griddb_python<\/p>\n<\/li>\n<\/ol>\n

      You can install these packages in Conda\u2019s virtual environment using conda install package-name<\/code>. In case you are using Python directly via terminal\/command prompt, pip install package-name<\/code> will do the work.<\/p>\n

      GridDB installation<\/h3>\n

      While loading the dataset, this tutorial will cover two methods \u2013 Using GridDB as well as Using Pandas. To access GridDB using Python, the following packages also need to be installed beforehand:<\/p>\n

        \n
      1. GridDB C-client<\/a><\/li>\n
      2. SWIG (Simplified Wrapper and Interface Generator)<\/li>\n
      3. GridDB Python Client<\/a><\/li>\n<\/ol>\n

        1. Dataset Overview<\/h1>\n

        The indicators that are recognised to have the most significant impact on stock market return in general and on S&P 500 in particular can be attributed to the following categories: general macroeconomic indicators, labour market indicators (unemployment rate and jobs reports), real estate indicators, credit market indicators, monetary supply indicators, consumer (household) financial behaviour indicators and commodity market indicators.<\/p>\n

        The S&P 500 index close price was modelling in this paper. The study used a python programming language with many libraries in Google Colab environment. The analysis period between 1970-01-01 \/ 2018-04-01, and both S&P 500 index close price and U.S economic indicators data frequency are month.<\/p>\n

        Feature Description:<\/p>\n

        1) SP500<\/code> – Price of S&P 500 at respective month (Units: U.S. Dollars)<\/p>\n

        2) INTDSRUSM193N<\/code> – Interest Rates, Discount Rate for United States (Units: Percent per Annum)<\/p>\n

        3) BUSLOANS<\/code> – Commercial and Industrial Loans, All Commercial Banks (Units: Billions of U.S. Dollars)<\/p>\n

        4) MPRIME<\/code> – Bank Prime Loan Rate (Units: Percent)<\/p>\n

        5) FEDFUNDS<\/code> – Federal Funds Effective Rate (The federal funds rate is the interest rate at which depository institutions trade federal funds with each other overnight.) (Units: Percent)<\/p>\n

        6) CURRCIR<\/code> – Currency in Circulation (Units: Billions of Dollars)<\/p>\n

        7) PSAVERT<\/code> – Personal Saving Rate (Personal saving as a percentage of disposable personal income (DPI), frequently referred to as “the personal saving rate,” is calculated as the ratio of personal saving to DPI.)(Units: Percent)<\/p>\n

        8) PERMIT<\/code> – New Privately-Owned Housing Units Authorized in Permit-Issuing Places: Total Units (Units: Billions of Dollars)<\/p>\n

        9) INDPRO<\/code> – Industrial Production: Total Index (The Industrial Production Index (INDPRO) is an economic indicator that measures real output for all facilities located in the United States manufacturing, mining, and electric, and gas utilities) (Units: Index 2017=100)<\/p>\n

        10) PMSAVE<\/code> – Personal Saving (Units: Billions of Dollars)<\/p>\n

        11) DAUTOSAAR<\/code> – Motor Vehicle Retail Sales: Domestic Autos (Units: Millions of Units)<\/p>\n

        12) UNEMPLOY<\/code> – Unemployment Level (Units: Thousands of Persons)<\/p>\n

        13) CPIAUCSL<\/code> – Consumer Price Index for All Urban Consumers: All Items in U.S. City Average (Units: Index 1982-1984=100)<\/p>\n

        The dataset is available publicly and can be downloaded from finance.yahoo.com<\/code> and fred.stlouisfed.org<\/code> website. The obtained data were combined into one data set, which was checked for missing values.<\/p>\n

        2. Importing Required Libraries<\/h1>\n
        \n
        import griddb_python as griddb\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport datetime\n\nimport statsmodels.api as sm\nfrom statsmodels.stats.outliers_influence import variance_inflation_factor\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.ensemble import RandomForestRegressor\nfrom sklearn.metrics import r2_score\nfrom sklearn.metrics import mean_squared_error\n\nimport warnings\nwarnings.filterwarnings('ignore')\n%matplotlib inline<\/code><\/pre>\n<\/div>\n
        3. Loading the Dataset<\/h1>\n
        Let\u2019s proceed and load the dataset into our notebook.<\/p>\n
        3.a Using GridDB<\/h2>\n
        Toshiba GridDB\u2122 is a highly scalable NoSQL database best suited for IoT and Big Data. The foundation of GridDB\u2019s principles is based upon offering a versatile data store that is optimized for IoT, provides high scalability, tuned for high performance, and ensures high reliability.<\/p>\n
        To store large amounts of data, a CSV file can be cumbersome. GridDB serves as a perfect alternative as it in open-source and a highly scalable database. GridDB is a scalable, in-memory, No SQL database which makes it easier for you to store large amounts of data. If you are new to GridDB, a tutorial on reading and writing to GridDB<\/a> can be useful.<\/p>\n
        Assuming that you have already set up your database, we will now write the SQL query in python to load our dataset.<\/p>\n
        \n
        sql_statement = ('SELECT * FROM us_economic_data')\ndataset = pd.read_sql_query(sql_statement, cont)<\/code><\/pre>\n<\/div>\n
        Note that the cont<\/code> variable has the container information where our data is stored. Replace the credit_card_dataset<\/code> with the name of your container. More info can be found in this tutorial reading and writing to GridDB<\/a>.<\/p>\n
        When it comes to IoT and Big Data use cases, GridDB clearly stands out among other databases in the Relational and NoSQL space. Overall, GridDB offers multiple reliability features for mission-critical applications that require high availability and data retention.<\/p>\n
        3.b Using Pandas<\/h2>\n
        We can also use Pandas’ read_csv<\/code> function to load our data. Both of the above methods will lead to the same output as the data is loaded in the form of a pandas dataframe using either of the methods.<\/p>\n
        \n
        us_economic_data = pd.read_csv('us_economic_data.csv')<\/code><\/pre>\n<\/div>\n
        4. Exploratory Data Analysis & Feature Selection<\/h1>\n
        Once the dataset is loaded, let us now explore the dataset. We’ll print the first 10 rows of this dataset using head() function.<\/p>\n
        \n
        us_economic_data.head()<\/code><\/pre>\n<\/div>\n
        \n