Gamma and vega hedging using deep distributional reinforcement learning

Abstract

We show how reinforcement learning can be used in conjunction with quantile regression to develop a hedging strategy for a trader responsible for derivatives that arrive stochastically and depend on a single underlying asset. We assume that the trader makes the portfolio delta-neutral at the end of each day by taking a position in the underlying asset. We focus on how trades in options can be used to manage gamma and vega. The option trades are subject to transaction costs. We consider three different objective functions. We reach conclusions on how the optimal hedging strategy depends on the trader's objective function, the level of transaction costs, and the maturity of the options used for hedging. We also investigate the robustness of the hedging strategy to the process assumed for the underlying asset.

Keywords: D4PG; delta-neutral; derivatives; gamma; hedging; quantile regression; reinforcement learning; vega.

Figure 1

RL architecture for actor-critic learning and proposed networks.

Figure 2

Comparison of gain distribution for delta hedging, delta-gamma-neutral hedging, and hedging by the VaR95 RL agent when transaction cost = 1%. Note that the 5^th percentile of the gain distribution corresponds to the 95th percentile loss in Table 1.

Figure 3

Risk-return trade-offs that are possible using RL with different objective functions. The positive average cost of hedging reported in previous tables means the average return of the agent is negative.