Market-Data-Exploration-And-Factor-Analysis

Financial Data Science

Yield curve modeling with Nelson-Siegel and Cubic Spline, PCA on Treasury yield changes, and factor analysis of XLI sector ETF holdings via eigendecomposition and SVD.

This project applies statistical and machine learning techniques to two distinct financial datasets: US Treasury yield curves and US Industrial sector equity returns. The analysis spans yield curve fitting, principal component decomposition of rate movements, equity return characterization, and a direct comparison of PCA eigendecomposition versus SVD on standardized returns.

• Yield curve: Static US Treasury par yield snapshot (late 2024) embedded in the notebook
• Treasury yield changes: FRED DGS series (1Y, 2Y, 5Y, 10Y, 30Y) from 2023-10-01; synthetic fallback if network unavailable
• XLI holdings: yfinance adjusted closing prices for 30 XLI holdings (Jul-Dec 2024); synthetic fallback if network unavailable

Nelson-Siegel model:

y(t) = beta0 + beta1 * [(1 - exp(-t/tau)) / (t/tau)]
             + beta2 * [(1 - exp(-t/tau)) / (t/tau) - exp(-t/tau)]

The three factors map to economically interpretable components: level (beta0), slope (beta1), and curvature (beta2), making Nelson-Siegel the standard in central bank yield curve modeling. Parameters are estimated by nonlinear least squares via lmfit with constrained bounds (beta0 > 0, tau > 0.1).

Cubic Spline: Piecewise polynomial interpolation that passes exactly through each observed maturity point. Unlike Nelson-Siegel, the cubic spline is purely a fitting device with no economic interpretation, but it achieves near-zero residuals at observed maturities.

Data: US Treasury par yields at maturities 0.25, 0.5, 1, 2, 3, 5, 7, 10, 20, and 30 years.

Evaluation: RMSE computed at observed maturities for both models. Nelson-Siegel trades off fit quality for parsimony and interpretability; Cubic Spline achieves exact fit at knot points.

Synthetic uncorrelated data:

Five independent Gaussian yield changes (mean=0, sigma=1%) are generated and subjected to PCA. Since the variables are uncorrelated by construction, all five principal components explain approximately equal variance and no single factor dominates. This establishes a theoretical baseline: PCA only finds structure when structure exists.

Real FRED Treasury yield changes:

Daily first differences of 1Y, 2Y, 5Y, 10Y, and 30Y US Treasury yields sourced from FRED (DGS series). Two PCA variants are compared:

• Covariance PCA: operates on raw yield changes, weighting by variance. Short-term yields typically dominate due to higher absolute volatility.
• Correlation PCA: standardizes each tenor to unit variance before decomposing, treating all maturities equally.

In real yield data, PC1 (level factor) typically explains over 80% of variance, PC2 (slope factor) another 10-15%, and PC3 (curvature factor) most of the remainder, consistent with the empirical literature on term structure factor decomposition.

Universe: 30 largest holdings of XLI (iShares US Industrials ETF) by market cap as of mid-2024, covering Jul 2024 to Dec 2024 daily adjusted closing prices from Yahoo Finance.

Returns analysis:

• Daily percentage returns computed via first difference of log prices
• Holdings sorted by mean return and by volatility
• Grouped into 6 bands of 5 stocks each by ascending volatility
• Box plots and time series panels illustrate cross-sectional return dispersion

Covariance heatmap: Full 30x30 covariance matrix of daily returns in percentage space, revealing clusters of co-movement among industrial sub-sectors (aerospace and defense, transportation, diversified industrials).

PCA via eigendecomposition:

The standardized returns matrix R_std has covariance matrix C. Eigendecomposition:

C = V * Lambda * V^T

where Lambda is the diagonal matrix of eigenvalues and V contains eigenvectors (loadings). Principal component scores are computed as R_std @ V. The explained variance proportion of each PC equals lambda_k / sum(lambda). In industrial equities, the first 3-5 components typically explain 60-75% of total variance, reflecting the broad market factor, sector rotation, and sub-industry groupings.

SVD verification:

For normalized returns matrix B = R_std / sqrt(n-1):

B = U * S * V^T

The squared singular values of B equal the eigenvalues of C, providing an algebraic verification of the PCA eigendecomposition. The comparison plot confirms numerical equivalence.

• Nelson-Siegel achieves low but non-zero RMSE; cubic spline interpolates exactly at observed maturities but may oscillate between knots
• PCA on synthetic uncorrelated variables confirms uniform variance distribution, serving as a useful null hypothesis baseline
• The first PC of real Treasury yield changes captures the parallel shift factor, consistent with the Litterman-Scheinkman (1991) three-factor decomposition
• XLI returns show strong co-movement; the first principal component likely represents broad equity market beta
• Squared SVD singular values numerically match PCA eigenvalues, confirming the algebraic equivalence of the two decompositions

Python 3.x
numpy
pandas
scipy
matplotlib
seaborn
scikit-learn
lmfit (Nelson-Siegel fitting)
fredapi (FRED data, with synthetic fallback)
yfinance (equity data, with synthetic fallback)

git clone https://github.com/QuantSingularity/Market-Data-Exploration-And-Factor-Analysis.git
cd Market-Data-Exploration-And-Factor-Analysis
pip install numpy pandas scipy matplotlib seaborn scikit-learn lmfit fredapi yfinance
jupyter notebook Market-Data-Exploration-And-Factor-Analysis.ipynb

• Nelson, C.R., and Siegel, A.F. (1987). Parsimonious Modeling of Yield Curves. Journal of Business, 60(4), 473-489.
• Litterman, R., and Scheinkman, J. (1991). Common Factors Affecting Bond Returns. Journal of Fixed Income, 1(1), 54-61.
• Jolliffe, I.T. (2002). Principal Component Analysis (2nd ed.). Springer.
• Lopez de Prado, M. (2018). Advances in Financial Machine Learning. Wiley.