Hierarchical Risk Parity (HRP) is an advanced investment portfolio optimization framework developed in 2016 by Marcos López de Prado. HRP is a probabilistic graph-based alternative to the prevailing mean-variance optimization (MVO) framework developed by Harry Markowitz. HRP algorithms apply discrete mathematics and machine learning techniques to create diversified and robust investment portfolios that outperform MVO methods out-of-sample. HRP aims to address the limitations of traditional portfolio construction methods, particularly when dealing with highly correlated assets. Following its publication, HRP has been implemented in numerous open-source libraries, and received multiple extensions.
Key features HRP portfolios have been proposed as a robust alternative to traditional quadratic optimization methods, including the Critical Line Algorithm (CLA) of Markowitz. HRP addresses three central issues commonly associated with quadratic optimizers: numerical instability, excessive concentration in a small number of assets, and poor out-of-sample performance.
HRP leverages techniques from graph theory and machine learning to construct diversified portfolios using only the information embedded in the covariance matrix. Unlike quadratic programming methods, HRP does not require the covariance matrix to be invertible. Consequently, HRP remains applicable even in cases where the covariance matrix is ill-conditioned or singular—conditions under which standard optimizers fail.
Monte Carlo simulations indicate that HRP achieves lower out-of-sample variance than CLA, despite the fact that minimizing variance is the explicit optimization objective of CLA. Furthermore, HRP portfolios exhibit lower realized risk compared to those generated by traditional risk parity methodologies. Empirical backtests have demonstrated that HRP would have historically outperformed conventional portfolio construction techniques.
Algorithms within the HRP framework are characterized by the following features:
Machine Learning Approach: HRP employs hierarchical clustering, a machine learning technique, to group similar assets based on their correlations. This allows the algorithm to identify the underlying hierarchical structure of the portfolio, and avoid that errors spread through the entire network. Risk-Based Allocation: The algorithm allocates capital based on risk, ensuring that assets only compete with similar assets for representation in the portfolio. This approach leads to better diversification across different risk sources, while avoiding the instability associated with noisy returns estimates. Covariance Matrix Handling: Unlike traditional methods like Mean-Variance Optimization, HRP does not require inverting the covariance matrix. This makes it more stable and applicable to portfolios with a large number of assets, particularly when the covariance matrix's condition number is high.
The problem: Markowitz's Curse Portfolio construction is perhaps the most recurrent financial problem. On a daily basis, investment managers must build portfolios that incorporate their views and forecasts on risks and returns. Despite the theoretical elegance of Markowitz's mean-variance framework, its practical implementation is hindered by several limitations that undermine the reliability of solutions derived from the Critical Line Algorithm (CLA). A principal concern is the high sensitivity of optimal portfolios to small perturbations in expected returns: even minor forecasting errors can result in significantly different allocations (Michaud, 1998). Given the inherent difficulty of producing accurate return forecasts, numerous researchers have advocated for approaches that forgo expected returns entirely and instead rely solely on the covariance structure of asset returns. This has given rise to risk-based allocation methods, among which risk parity is a widely cited example (Jurczenko, 2015).
While eliminating return forecasts mitigates some instability, it does not eliminate it. Quadratic programming techniques employed in portfolio optimization require the inversion of a positive-definite covariance matrix, meaning all eigenvalues must be strictly positive. When the matrix is numerically ill-conditioned—that is, when the ratio of its largest to smallest eigenvalue (its condition number) is large—matrix inversion becomes unreliable and prone to significant numerical errors (Bailey and López de Prado, 2012).
The condition number of a covariance, correlation, or any symmetric (and thus diagonalizable) matrix is defined as the absolute value of the ratio between its largest and smallest eigenvalues in modulus. The figure on the right presents the sorted eigenvalues of several correlation matrices; the condition number is represented by the ratio of the first to last eigenvalues in each sequence. A diagonal correlation matrix, which is equal to its own inverse, exhibits the minimum possible condition number.
As the number of correlated (or multicollinear) assets in a portfolio increases, the condition number rises. At high levels, this leads to severe numerical instability, whereby slight modifications in any matrix entry may result in drastically different inverses. This phenomenon, often referred to as Markowitz’s curse, encapsulates the paradox wherein increased correlation among assets heightens the theoretical need for diversification, yet simultaneously increases the likelihood of unstable optimization outcomes. Consequently, the potential benefits of diversification are frequently overshadowed by estimation errors.
These problems are exacerbated as the dimensionality of the covariance matrix increases. The estimation of each covariance term consumes degrees of freedom, and in general, a minimum of independent and identically distributed (IID) observations is required to estimate a non-singular covariance matrix of dimension . For example, constructing an invertible covariance matrix of dimension 50 necessitates at least five years of daily IID observations. However, empirical evidence suggests that the correlation structure of financial assets is highly unstable over such extended periods. These difficulties are highlighted by the observation that even naïve allocation strategies—such as equally weighted portfolios—have frequently outperformed both mean-variance and risk-based optimizations in out-of-sample tests (De Miguel et al., 2009).
The solution: Hierarchical Risk Parity The HRP algorithm addresses Markowitz's curse in three steps:
Hierarchical Clustering: Assets are grouped into clusters based on their correlations, forming a hierarchical tree structure. Quasi-Diagonalization: The correlation matrix is reordered based on the clustering results, revealing a block diagonal structure. Recursive Bisection: Weights are assigned to assets through a top-down approach, splitting the portfolio into smaller sub-portfolios and allocating capital based on inverse variance.
Step 1: Hierarchical clustering
Given a matrix of asset returns , where each column represents a time series of returns for one of assets over time periods, a hierarchical clustering process can be used to construct a tree-based representation of asset relationships. First, we compute the correlation matrix , where . From this, a pairwise distance matrix is defined using the transformation:
This distance function defines a proper metric space, satisfying non-negativity, identity of indiscernibles, symmetry, and the triangle inequality.
Next, a secondary distance matrix is computed, where each entry measures the Euclidean distance between the distance profiles of two assets:
While reflects correlation-based proximity between two assets, quantifies dissimilarity across the entire system, as it depends on all pairwise distances.
Hierarchical clustering proceeds by identifying the pair with the smallest value of (for ), and forming a new cluster . The distance between this new cluster and all remaining items is updated using a linkage criterion. One example is the single linkage (nearest neighbor) method:
The algorithm is repeated recursively: the pair with minimum distance is clustered, the matrix is updated, and clustering continues until all items belong to a single cluster. This produces clusters and a binary tree (dendrogram) that encodes the hierarchical structure.
The clustering process is recorded in a linkage matrix , where each row represents a clustering step: and are the merged clusters, is the distance between them, and is the total number of original items included in the cluster. HRP accepts a wide range of clustering metrics and linkage criteria. For further discussion, see Rokach and Maimon (2005), Brualdi (2010), and the documentation for SciPy’s hierarchical clustering tools.
Step 2: Quasi-diagonalization
At this stage, the rows and columns of the covariance matrix are reordered such that the largest values are concentrated along the diagonal. This process is referred to as a quasi-diagonalization of the covariance matrix. Unlike full diagonalization, this method does not require a change of basis. The resulting structure places similar investments near each other and dissimilar investments farther apart, facilitating block-wise portfolio construction. (See illustrative examples on the right)
The procedure relies on the structure of the linkage matrix generated during hierarchical clustering. Each row in the linkage matrix represents the merger of two branches into a cluster. To obtain the ordering of original items, the final row is recursively expanded by replacing clusters with their constituents, in the order dictated by the hierarchical tree. This produces a sorted list of indices corresponding to the unclustered items.
This ordering is used in the final step of the HRP algorithm to allocate capital proportionally to the hierarchical structure of asset relationships.
Step 3: Recursive bisection This final stage of the Hierarchical Risk Parity (HRP) algorithm computes portfolio weights using the quasi-diagonal covariance matrix. When the covariance matrix is diagonal, inverse-variance weighting is optimal (see Appendix 16.A.2). HRP uses this insight in both bottom-up and top-down directions:
Bottom-up: estimate the variance of a cluster using inverse-variance weights.
Top-down: split capital between clusters inversely proportional to their estimated variances.
The recursive algorithm proceeds as follows:
Initialize a list L with all asset indices: L = {1, 2, ..., N}. Assign initial weights: w_n = 1 for all assets. While any subset in L contains more than one element: Bisect the subset into two halves: L1 and L2. For each subset Lj, compute its covariance matrix Vj and define weights proportional to the inverse of its diagonal entries. Normalize those weights so they sum to 1, and compute the variance of the cluster as w' V w. Compute a split factor α = 1 - (V1 / (V1 + V2)). Rescale the weights of L1 by α and of L2 by 1 - α.
This ensures that all weights remain between 0 and 1 and sum to 1 at all times.
The following Python function implements this process, where sort_ix is the list of ordered assets returned by Step 2 (Quasi-Diagonalization), cov is the covariance matrix:
import numpy as np import pandas as pd
def get_rec_bipart(cov, sort_ix): # Compute HRP allocation w = pd.Series(1.0, index=sort_ix) clusters = [sort_ix] while len(clusters) > 0: # Bisect each cluster new_clusters = [] for cluster in clusters: if len(cluster) > 1: split = int(len(cluster) / 2) new_clusters.append(cluster[:split]) new_clusters.append(cluster[split:]) clusters = new_clusters # Update weights for i in range(0, len(clusters), 2): c1 = clusters[i] c2 = clusters[i + 1] var1 = get_cluster_var(cov, c1) var2 = get_cluster_var(cov, c2) alpha = 1.0 - var1 / (var1 + var2) w[c1] *= alpha w[c2] *= 1.0 - alpha return w
def get_cluster_var(cov, cItems):