PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).

I'm having trouble reconciling what I read in the paper and what I read in the code.

The reward function in a single period in the paper (Eq. (10)) is \log(\mu_t y_t \cdot w_{t-1}). But in the code, it seems that the reward is instead \log(mu_t y_{t+1} \cdot w_{t}). Am I correct?

Because __pack_samples (in datamatrices.py) makes the price tensor X using M[..., :-1] and the relative price vector y using M[...,-1]/M[...,-2], so y is one period ahead of X.

Running on Python 3.4.3, after I call python3 main.py --mode=train --processes=1, I get the following error:

ValueError: the length of selected coins 0 is not equal to expected 11

Perhaps this an issue with my version of Python?

Hello

Thanks for the wonderful work, i read your paper and almost studied most of the code. However, i don't get the concept of append_experience and agent train in the rolling_train method I have some questions if i may ask 1- what is the format of the saved experience and how does it affect the model ? 2- how is that different from training the model directly using self._agent.train() ? 3- is the experience mentioned here the same as the mini-batches mentioned in the paper for online learning section 5.3 for example ?

thanks in advance Sarah Ahmed

Im trying reproduce result plotted in User Guide (10^2), but with default config getting much worse results. Which config was used in example? Thanks!

Figure 2 in Paper (Attached image): Shouldn't the convolutional filters be 3 dimensional? I mean, in the original convolution how do we go from 3 feature maps to 2 feature maps. I believe this would make sense if the filter was of dimension 2x1x3 (same as described but with additional depth of 2). And then the second convolution would be 2x48 to get the 20 11x1 feature maps.

net_config.json: In ConvLayer, I don't understand how {"filter_shape":[1,2],"filter_number":3} corresponds to the filters outlined in the paper as described in my above question. (Excuse my ignorance of tflearn, but the params to conv2d() are not well explained in the documentation)

Hello,

In the code, i don't understand what is the difference between reversed_USDT and the cash (BTC).

I supposed (USDT_BTC) which is actually BTC/USD is a mapping to just holding some weight in BTC

Am i wrong ?

In BackTest, "omega" seems to be the vector wT storing the recommended new portfolio distribution at each step, "_last_omega" the latest/previous portfolio screenhost wT-1. So the system assumes to be able to sell at each step all the current coins of the portfolio and buy all "omega" reco, or at least the delta between omega & last_omega. This strong hypothesis (slippage/liquidity) is in your paper but shouldn't it check at least whether any coin qty adjustment would not cost more transaction fees than the expected value adjustment ?

Looks like the Poloniex API is no longer accessible programmatically. I'll look into alternative APIs and will try to follow up with a pull request for this.

Hey @ZhengyaoJiang I've updated the readme and user guide to reflect the current version of the library. Please have a look and let me know if I missed anything or if there are other things that need improvement.

I mostly simplified the explanation and made it clearer where I thought there were ambiguities.

Hi, thank you for your excellent work, this is very interesting stuff.

I am eager to test this on the live market, but having trouble moving from backtesting to forwardtesting. Any chance that an update with a ForwardTest class is on the way, or that you could advise on how to implement it? I understand it roughly, i.e. the generate_history_matrix( ) function needs to update the datamatrix with the newest market data (with "online" = True in the config file), and return that. And the trade_by_strategy( ) clearly needs a slight rewriting compared to BackTest as we don't know the future price. Any help on how to correctly return the newest market data would be appreciated.

Hello again!

May I ask here for more details about learning procedure, because I'm not really in shape to understand all the code, may be with your guides here I'll go through it again with more success.

1. During training phase how many times CNN learns on the same batch? Do you use epochs to learn or CNN passes through the data only once?
2. During CV and Test phases rolling learning is used. On what data do CNN weights get updated? After all orders have been completed in current period we add price history into local DB. Do we select N periods before current period into learning batch? Or we update weights only using last price window?

Sorry if it's newbie questions, I just want to understand how this magic works.

Hi,

I've tried several configurations of my anaconda environment, at first, I managed to make the python main.py --mode=download_data part work, but then I ran into the issues with the update of pandas mentioned in other issues. Trying to fix that I cannot come back to my initial progress even though I've made a new environment and forked the repo once again.

The error I get is:

Traceback (most recent call last): File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\main.py", line 132, in main() File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\main.py", line 71, in main DataMatrices(start=start, File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\pgportfolio\marketdata\datamatrices.py", line 44, in init self.__history_manager = gdm.HistoryManager(coin_number=coin_filter, end=self.__end, File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\pgportfolio\marketdata\globaldatamatrix.py", line 24, in init self._coin_list = CoinList(end, volume_average_days, volume_forward) File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\pgportfolio\marketdata\coinlist.py", line 35, in init prices.append(1.0 / float(ticker[k]['last'])) KeyError: 'BTS_BTC'

I've no clue how to solve this. Have any others experienced the issue?

Thanks

Hi! i am trying to run your code on Visual Studio 2017, I have downloaded and installed all necessary libraries and dependencies. I attach the main.py file and run it and a console window opens, which I will attach below. I am not native to python so some step by step procedure would be extremely helpful.

Hello,

When I run this code:

python main.py --mode=train --processes=1

I get this error: TypeError: init() got multiple values for argument 'dtype'

I changed only the start and end times in the configuration file. Are there any recommendations?

Here's the logfile:

INFO:root:select coin online from 2021-10-12 00:00 to 2021-11-11 00:00 DEBUG:root:Selected coins are: ['reversed_USDT', 'reversed_USDC', 'ETH', 'LTC', 'XRP', 'SRM', 'DOGE', 'XMR', 'BCH', 'DOT', 'EOS'] INFO:root:fill SRM data from 2021-03-26 00:00 to 2021-06-24 11:59 INFO:root:fill SRM data from 2021-06-24 12:00 to 2021-09-22 23:59 INFO:root:fill SRM data from 2021-09-23 00:00 to 2021-12-01 00:00 INFO:root:fill DOT data from 2021-03-26 00:00 to 2021-06-24 11:59 INFO:root:fill DOT data from 2021-06-24 12:00 to 2021-09-22 23:59 INFO:root:fill DOT data from 2021-09-23 00:00 to 2021-12-01 00:00 INFO:root:fill EOS data from 2021-11-30 23:00 to 2021-12-01 00:00 INFO:root:feature type list is ['close', 'high', 'low'] DEBUG:root:buffer_bias is 0.000050 INFO:root:the number of training examples is 11008, of test examples is 929 DEBUG:root:the training set is from 0 to 11007 DEBUG:root:the test set is from 11040 to 12000

Thanks.

when you already know I think normalization on open price is incorrect for this task. In real life, you can not buy on open price, when you already know high and low, from my point of view for real testing you should normalize for the close price (open for next candle) - if you do this - results will be significant worst. Have I made a mistake in my reasoning?

Hello,

I was not getting error in train phase however train_summary.csv not generated either. However, when i backtest i was getting error "train_summary.csv not found".

I found the solution. The problem is related to indentation in tradertrainer.py(__log_result_csv method). At the end of the method replace the lines with indented ones as attached.

That will produce the necessary csv file.