Recommendation of 5 papers on the application of reinforcement learning in the financial field

 source ：DeepHub IMBA This article is about 1500 word , Recommended reading 5 This article recommends 5 A paper on the application of reinforcement learning in the financial field .

In recent years, machine learning has been applied in all aspects of various financial fields , In fact, the scenario in the financial field is very suitable for reinforcement learning application , However, due to the of real gold and silver in the financial field , Based on the current learning efficiency of reinforcement learning, it is estimated that few people are willing to try , But it does not prevent us from learning and understanding this knowledge .

Reinforcement learning in market games(arxiv 0710.0114)

Edward W. Piotrowski, Jan Sladkowski, Anna Szczypinska

Financial market investment is like many multiplayer games —— Must interact with other agents to achieve their goals . These include factors directly related to activities in the market , And other aspects that affect human decision-making and its performance as an investor . If all sub games are distinguished, it is usually beyond the hope and resource consumption . In this paper, we study how investors face many different choices 、 Collect information and make decisions without understanding the complete structure of the game . This paper applies reinforcement learning method to the theoretical model of market information (ITMM). Try to distinguish between i A kind of game and possible actions of agents （ Strategy ）. Any agent divides the entire game class into her / He thinks subclasses , Therefore, the same strategy is adopted for a given subclass . The division criteria are based on profit and cost analysis . Analogy classes and strategies are updated at all stages through the learning process .

Dreaming machine learning: Lipschitz extensions for reinforcement learning on financial markets(arXiv 1909.03278)

J. M. Calabuig, H. Falciani, E. A. Sánchez-Pérez

This paper considers a quasi metric topology for constructing a new reinforcement learning model within the framework of financial market . It is based on the reward function defined in the metric space Lipschitz Type extension . say concretely ,McShane and Whitney Used for reward function , This function is defined by the total evaluation of the income generated by the investment decision at a given time . The metric is defined as the linear combination of Euclidean distance and angle metric components . All the information about the system evolution from the time interval is used to support the extension of the reward function , And enrich the data set by adding some artificially generated States . The paper says , The main novelty of this method is that （ In the paper, it is called “dreams”） In a way that enriches learning . Using some known states of dynamic systems representing the evolution of financial markets , Using the existing technology, we can simulate the new state by inserting the real state and introducing some random variables . These new states are used to provide training data for learning algorithms , The purpose of this algorithm is to improve the investment strategy by following a typical reinforcement learning scheme .

Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning(arXiv 1909.03278)

Autonomous trading agent is one of the most active research fields of artificial intelligence to solve the problem of portfolio management in capital market . The two main objectives of portfolio management are to maximize profits and curb risks . Most solutions to this problem only consider maximizing returns . However, this paper proposes a transaction agent based on deep reinforcement learning , When it manages its portfolio , Not only consider profit maximization , Also consider risk constraints . A new target strategy is also proposed in this paper , Let trading agents learn to prefer low-risk actions . This new goal strategy can reduce the risk of action by adjusting the greedy degree of the optimal behavior through hyperparameters . The proposed trading agent verifies its performance through the data of encrypted money market , Because the cryptocurrency market is the best testing ground for trading agents , Because the amount of data accumulated per minute is huge , The market is highly volatile . As an experimental result , During the test , The agent implements 1800% The return of , It also provides the investment strategy with the least risk in the existing methods . And another experiment shows that , Even if the market fluctuates greatly or the training cycle is very short , Trading agents can also maintain robust generalization performance .

Application of deep reinforcement learning for Indian stock trading automation(arXiv 2106.16088)

Author : Supriya Bajpai

In stock trading , Feature extraction and transaction strategy design are two important tasks to realize long-term benefits by using machine learning technology . By obtaining the trading signal to design the trading strategy, we can maximize the trading income . This paper applies the deep reinforcement learning theory to the stock trading strategy and investment decision-making in the Indian market . Using three classical deep reinforcement learning models Deep Q-Network、Double Deep Q-Network and Dueling Double Deep Q-Network Yes 10 A systematic experiment was carried out on an Indian stock data set . The performance of the model is evaluated and compared .

Robo-Advising: Enhancing Investment with Inverse Optimization and Deep ReinforcementLearning(arXiv 2105.09264)

Author : Haoran Wang, Shi Yu

machine learning (ML) It has been regarded as a powerful tool by the financial industry , It has significant applications in various fields such as investment management . This paper proposes a full cycle data-driven investment robot consulting framework , By two ML The composition of the agency . The first agent is an inverse portfolio optimization agent , It uses online inverse optimization method to infer investors' risk preference and expected return directly from investors' historical allocation data . The second is deep reinforcement learning (deep reinforcement learning, RL) agent , It aggregates the inferred expected return series , Form a new multi period mean - Variance portfolio optimization problem , In this way, the deep reinforcement learning method can be used to solve . The investment plan in this paper is applied to 2016 year 4 month 1 solstice 2021 year 2 month 1 Actual market data for the day , Performance continues to outperform S & P, which represents the optimal allocation of the overall market 500 Benchmark portfolio . This excellent performance may be attributed to multi cycle planning ( Compared with single cycle planning ) And data driven RL Method ( Compared with classical estimation methods ).

edit ： Wang Jing

proofreading ： Yang Xuejun