Financial markets experience periods of growth, corrections and extreme uncertainty. Understanding how risks interact with market performance. This project investigates whether volatility driven by fear or stress is key to assessing the risk of investing in the main US stock market.
The VIX Index and S&P500 historical datasets were downloaded from the Kaggle website. Both datasets were composed of multiple columns, such as high, low, open, volume, adjusted close and date.
Firstly, I ran the Exploratory Data Analysis (EDA) to study if the selected raw data could answer the main question and, right after, I imported it into MySQL for the preliminary cleaning step. In order to optimize the BI process, I used SQL queries to join both tables, creating a single one, which constitutes the foundation of all the visualizations.
SQL queries used during this process are shown below:
-- Creation of the table structures:
CREATE TABLE sp500_index(
Point_in_time DATE PRIMARY KEY NOT NULL UNIQUE,
Open DECIMAL(10,6) NOT NULL,
High DECIMAL(10,6) NOT NULL,
Low DECIMAL(10,6) NOT NULL,
Close DECIMAL(10,6) NOT NULL,
Adj_close DECIMAL(10,6) NOT NULL,
Volume INT NOT NULL
);
CREATE TABLE vix(
Point_in_time DATE PRIMARY KEY NOT NULL UNIQUE,
VIX_open DECIMAL(4,2) NOT NULL,
VIX_high DECIMAL(4,2) NOT NULL,
VIX_Low DECIMAL(4,2) NOT NULL,
VIX_close DECIMAL(4,2) NOT NULL
);
-- Exploratory Data Analysis:
SELECT COUNT(*) FROM sp500_index WHERE point_in_time IS NULL;
SELECT
COUNT(Point_in_time) AS total,
COUNT(DISTINCT Point_in_time) AS unique_dates
FROM sp500_index;
-- Joining both tables to create a single one:
CREATE TABLE market_data AS
SELECT
sp500_index.point_in_time,
sp500_index.open,
sp500_index.high,
sp500_index.low,
sp500_index.close,
sp500_index.adj_close,
sp500_index.volume,
vix.vix_open,
vix.vix_high,
vix.vix_low,
vix.vix_close
FROM sp500_index
JOIN vix
ON sp500_index.point_in_time = vix.point_in_time;
After conducting data transformation through SQL, as you can observe below, I created a dashboard updating visualizations with key metrics such as the correlation between the S&P 500 index and the VIX index or the trendline of the S&P 500 over a long-term period (2004-2020) to find insights, which could lead to data-driven recommendations.
| Observations | Findings | Insights | Recommendations |
|---|---|---|---|
| The correlation between the VIX index and the S&P index is slightly negative and not perfectly linear. | When the market experiences a period of stress, the S&P 500 tends to decline. | High uncertainty could give important returns after a stable recovery. | Periods of elevated VIX may create buying opportunities for investors with a high-risk tolerance. |
| Periods of bigger stress are associated with the financial crisis in 2008 and the pandemic in 2020. | Major VIX spikes coincide with periods of market stress (2008 and 2020). | Market volatility tends to increase during market stress periods. | Volatility driven by stress isn’t essential for long-term investors, as the S&P 500 trendline is positive and all short-term downward trends are habitually followed by a stable recovery. However, the situation changes for a short-term investor. In this case, periods of volatility are dangerous, as assets could be sold before the recovery of the stock market. Depending on the investor profile, they need to evaluate whether to invest. |
| The VIX generally remains stable within the green area. | It represents a supported stress level by most investors. | S&P 500 trendline is positive despite the VIX index behavior. | Based on these results, fear from investors can have an impact on daily fluctuations, but it doesn’t change the long-term market performance. |
| The S&P 500 trendline is positive while the slope is well defined. | S&P 500 showed a steady upward trend from 2004 to 2020. | These upward trends demonstrate that investments in this market are lucrative, regardless of stressful events. | Investing in the S&P 500 is recommended to create value and earn money. |
| S&P 500 performance rocketed from 676 points to 3426 points | The S&P 500 shows a strong long-term upward trend. | The S&P 500 has a lucrative performance. | The added value from the S&P 500 is safe, and it’s recommended to invest in it. |
In conclusion, to answer the question “Financial markets experience periods of growth, corrections and extreme uncertainty. Understanding how risks interact with market performance. This project investigates whether volatility driven by fear or stress is key to assessing the risk of investing in the main US stock market”. The answer based on my data analysis is that volatility driven by fear or stress could be something to consider for a short-term investor like a speculator or a trader. On the opposite side, volatility is not key to consider investing in the S&P 500 for long-term investors, as trends are usually followed by a stable recovery.