Please rotate your device to landscape mode to view the charts.

Evolutionary multi-objective optimisation for large-scale portfolio selection with both random and uncertain returns

Journal: IEEE Transactions on Evolutionary Computation

Date: 2024

Author: Weilong Liu, Yong Zhang, Kailong Liu, Barry Quinn, Xingyu Yang, Qiao Peng

Abstract:
With the advent of Big Data, managing large-scale portfolios of thousands of securities is one of the most challenging tasks in the asset management industry. This study uses an evolutionary multi-objective technique to solve large-scale portfolio optimisation problems with both long-term listed and newly listed securities. The future returns of long-term listed securities are defined as random variables whose probability distributions are estimated based on sufficient historical data, while the returns of newly listed securities are defined as uncertain variables whose uncertainty distributions are estimated based on experts’ knowledge. Our approach defines security returns as theoretically uncertain random variables and proposes a three-moment optimisation model with practical trading constraints. In this study, a framework for applying arbitrary multi-objective evolutionary algorithms to portfolio optimisation is established, and a novel evolutionary algorithm based on large-scale optimisation techniques is developed to solve the proposed model. The experimental results show that the proposed algorithm outperforms state-of-the-art evolutionary algorithms in large-scale portfolio optimisation.

Link: Google Scholar

Background and Context

The Challenge

Managing large-scale portfolios of thousands of securities is becoming increasingly complex with the advent of Big Data in the asset management industry.

Innovative Approach

This study introduces an evolutionary multiobjective technique to optimize portfolios containing both established securities (with historical data) and newly listed securities (without sufficient historical data).

Methodology

A novel evolutionary algorithm employing large-scale optimization techniques is developed to solve complex portfolio optimization problems with practical trading constraints.

Traditional Portfolio Methods Fail to Account for Different Security Types

Traditional Approach Long-term Securities Random Variables Only Historical Data Mean-Variance Model Proposed Approach Long-term + Newly Listed Random + Uncertain Variables Data + Expert Knowledge Three-Moment Model
  • Traditional portfolio models focus only on securities with sufficient historical data, limiting portfolio diversity.
  • The proposed approach combines both random variables (historical data) and uncertain variables (expert knowledge).
  • Adding skewness as a third moment captures the asymmetry of returns, important for comprehensive risk assessment.

Novel LSWOEA Algorithm Significantly Outperforms Existing Methods

  • The proposed LSWOEA algorithm consistently outperforms other state-of-the-art algorithms across all portfolio sizes.
  • Performance gap widens as the number of securities increases, demonstrating superior scalability for large portfolios.
  • Higher Hypervolume (HV) values indicate better overall optimization of return, risk, and skewness objectives simultaneously.

Innovative Three-Step Optimization Framework for Large-Scale Portfolios

Initial Portfolio Model Constraint Handling Method Unconstrained Model LSWOEA Algorithm Normal Optimization Step Weighting Optimization Step Dispersed Target-guided Search Strategy Pareto Efficient Solutions
  • The algorithm transforms complex constrained problems into unconstrained ones through a novel encoder-decoder method.
  • Three key steps work together: normal optimization, weighting optimization, and dispersed target-guided search strategy.
  • This framework allows efficient handling of realistic constraints like cardinality, transaction lots, and shorting prohibition.

Effectiveness of Constraint Handling Method Across Algorithms

  • The proposed constraint handling method significantly improves performance across most evolutionary algorithms tested.
  • LSWOEA shows the most dramatic improvement, with its hypervolume value tripling after constraint elimination.
  • This demonstrates that the constraint handling approach can benefit various multiobjective optimization algorithms in portfolio selection.

Contribution and Implications

  • The approach enables large institutional investors to efficiently manage portfolios with thousands of diverse securities.
  • Portfolio managers can now incorporate newly listed securities without sufficient historical data into optimal allocations.
  • The framework provides tailored solutions aligned with different investor preferences for return, risk, and skewness.
  • The encoder-decoder method opens possibilities for applying various evolutionary algorithms to complex constrained portfolio problems.
  • The entire optimization process takes only about 15 seconds for 1000 securities, making it practical for real-world implementation.

Data Sources

  • Visualization 1 draws on the theoretical framework introduced in Sections III and IV of the paper.
  • Visualization 2 is based on Figure 4 from the paper, showing Hypervolume (HV) values across different portfolio sizes.
  • Visualization 3 is based on Figure 3, showing three investment strategies with different objective preferences.
  • Visualization 4 represents the optimization framework described in Figure 2 and Section V of the paper.
  • Visualization 5 is based on Figure 5, showing effectiveness of constraint handling across different algorithms.