How To Use Echo State Network In Forex Trading?

Algorithmic trading is the future. Today more than 80% of the trades that are being placed at Wall Street are being made by algorithmic trading systems. Did you read the post on EURNZD swing trade that made 1000 pips in 7 days with a small SL of 20 pips? How did I know this that EURNZD will move big time? Perhaps by using a neural network. Neural Networks are powerful tools. But only in the hands of experienced data scientists and artificial intelligence experts. If you want to master forex trading start thinking about becoming a data scientist and an artificial intelligence expert. The days of manual traders making a killing in the market are over. Today algorithms are trading against algorithms without any human intervention. Do you remember the British Pound Flash Crash?  Reportedly the British Pound Flash was caused by rogue algorithm.

What Is An Echo State Network (ESN)?

So let’s start our discussion on how to design neural networks for forex trading. Echo State Network (ESN) is a Recurrent Neural Network (RNN) model that is far superior to the traditional Feed Forward Neural Network (FFNN). When we use the traditional RNN models, it takes a long time to train them and then use them to make the prediction. ESN is much superior. It is simple conceptually and trains very fast. Did you take a look at our Deep Learning For Traders Course? Deep Learning is being called the technological breakthrough of the 21st century. Deep Learning has solved many complex problems encountered in computer vision and robotics. Can we use Deep Learning in forex trading? I believe we can.

Candlestick charts are very popular with forex traders. Candlestick patterns can be good leading signals. Can we make a deep learning candlestick patterns model? Take a look at my course in which I discuss how we can make deep learning candlestick pattern models that can be used for make good predictions for daily trading. The trick lies in combining fuzzy logic with deep learning. More on that in future posts. Did you read the post on how to make candlestick chart for custom timeframe? Candlestick patterns can help us lower risk.

My journey into the world of developing neural networks started last year when I was looking for a method to predict the trend in forex market. I started looking for methods that could be used to predict the price after a certain interval of time. I came across this word neural network frequently. Some people said neural networks are very good at predicting price. Others said neural networks are not good. I came across many forex neural network software that were being sold from $50-42500. At that point I decided to learn everything I need to know about neural networks and use them in my trading system. So my journey started. Did you read the post on EURUSD swing trade that made 270%?

Now it is being claimed that neural networks have been designed to work like the biological neurons that make our brain work. This is just a statement. Human brain is too complex an entity that is very difficult to model. Neural Networks have nothing to do with how the human brain works. It is just that the first developers thought that they are modelling human neurons. After that the analogy ends. But every textbook on neural networks still starts with human neurons and then tries to shows as if a neural network is trying to approximate it. As said above this is not the case. Neural networks are mathematical models that have been developed to model non linear phenomenon. Did you watch the video on a powerful method of capturing trending markets?

The first neural networks were Feed Forward Neural Networks (FFNN). FFNNs have an input layer and an output layer. In between are a few hidden layers. These hidden layers are used to introduce non linear behavior in the model. Input layer is fed into the first hidden layer which is then fed into the second hidden layer and so on. The last hidden layer feeds into the output layer. FFNNs work well in most cases. Training is done through the Back Propagation (BP) Algorithm. But when it comes to predicting financial time series, FFNNs fail miserably. This happens because FFNNs don’t look backward when doing the training.

We know from statistical time series analysis that time series have serial correlations. Serial correlations means that the present time series value is being affected by the past values. Autoregressive models have been developed that try to cater for this serial correlation with the past values. If we want our neural network to get better at predicting financial time series data we need them to look back as well. Now there are external shocks. Central banks are the ones that cause these external shocks most of the time. Currency trading is all about keep an eye on the central banks and know what they are doing. Did you read the post on how central banks control the currency market?

In forex day trading, we are dealing with high frequency data mostly on M15, M30 or M60 timeframe. This data has got its own special peculiarities. We cannot use the Gaussian assumption that is pivotal in building Autoregressive model or the Autoregressive Integrated Model (ARIMA). It is because of this reason that predictions made by ARIMA model on high frequency forex data are not good. Did you read the post that shows that high frequency forex data is not normally distributed? Now this makes the Kalman Filter model also not good as it also heavily depends on the normal distribution assumption. Neural Networks don’t have this problem as they don’t make any normality assumption about the data. So we overcome the problem and hope to build a neural network model that will make good predictions.

Finance professors have developed a random walk model for the stock market. This model stipulates that the financial time series is a random walk. Did you read the post on a random walk with GBPUSD one fine morning? According to the random walk model, price cannot be predicted. But we know now for sure that there are several price patterns in the market that can make good short term as well as long term predictions. So the random walk model got rejected. Today we have many machine learning and artificial intelligence algorithms that are being used for predicting the markets. One of the artificial intelligence algorithms widely being used is neural networks. Deep learning are just advanced neural networks.

As said above, FFNNs are not good at predicting financial time series. This problem was solved with the introduction of a Recurrent Neural Network (RNN). RNNs have the output layer connected with in between hidden layers. This introduces an element of feedback between the input and the output. But another problem arose. This problem was it took a long time to train these models and then make predictions. Back Propagation Algorithm that worked well for training FFNNs was taking too long to train a RNN. Echo State Network (ESN) solves this problem beautifully as it does not use Back Propagation Algorithm for training the neural network. It has a reservoir. You should watch the video below in which Professor Geoffrey Hinton explains what is an Echo State Network (ESN). Watch this video if you don’t understand what I am saying!

https://www.youtube.com/watch?v=j-Mj_YuTa70

Echo State Network Python Code

We will be using python to develop our Echo State Network model. You should have Anaconda installed. We will be using Spyder as our IDE. Spyder is a bit slow. I need to find a good python IDE. Now this is the code for the Echo State Network that we will use: We will be using GBPUSD 30 minute high frequency data. You can download this data from MT4 in a csv file.

#import numpy module
#import numpy
import numpy as np
#import ESN module
from pyESN import ESN
#import matplotlib
import matplotlib.pyplot as plt

#read the open, high, low and closing price from the csv files
o, h, l, c=np.loadtxt("E:/MarketData/GBPUSD30.csv", delimiter=',', 
                      usecols=(2,3,4,5), unpack=True)

##build an Echo State Network
esn = ESN(n_inputs = 1,
          n_outputs = 1,
          n_reservoir = 500,
          spectral_radius = 1.5,
          random_state=42)
#choose the training set
trainlen = 500
future = 20
#start training the model
pred_training = esn.fit(np.ones(trainlen),c[len(c)-trainlen: len(c)])
#make the predictions
prediction = esn.predict(np.ones(future))
print("test error: \n"+str(np.sqrt(np.mean((prediction.flatten() \
- c[trainlen:trainlen+future])**2))))

#print the predicted values of the closing price
prediction
#plot the predictions
plt.figure(figsize=(11,1.5))
plt.plot(range(0,trainlen),c[len(c)-trainlen:len(c)],'k',label="target system")
plt.plot(range(trainlen,trainlen+future),prediction,'r', label="free running ESN")
lo,hi = plt.ylim()
plt.plot([trainlen,trainlen],[lo+np.spacing(1),hi-np.spacing(1)],'k:')
plt.legend(loc=(0.61,1.1),fontsize='x-small')

In the above code we have different parameters. The important ones are the reservoir size, the spectral radius and the random states. Reservoir size should be big for better approximation. We will try to tune these parameter values and see if we get good predictions. Below is the plot of the ESN predictions with reservoir size equal to 500.

Echo State Network

In the above plot we can see closing price going up and down wildly. The predictions are not good. This is another plot of the ESN with reservoir size of 500.

Echo State Network

You can see in the above plot the predictions are not good. We change the reservoir size to 1000 and see if we get better results.  Below is the plot of the predicted values with reservoir size of 1000.

Echo State Network

Once again we don’t get good predictions. But the predictions are getting better. So lets increase the reservoir size to 5000. These are the predictions that we get:

>>
… esn = ESN(n_inputs = 1,
n_outputs = 1,
test error:
0.297942855198>>>
n_reservoir = 5000,
>>>           spectral_radius = 1.5,
array([[ 1.23347641],
[ 1.23476635],
[ 1.23500126],
…,
[ 1.22824547],
[ 1.22697307],
[ 1.22551418]])>>>
random_state=42)
>>>

In the above output you can see MSE is 0.2979. The predictions are good this time.  So by increasing the size of the reservoir to 5000 we have improved the predictions.

Echo State Network

As you can see from the above plots the predictions have improved by increasing the reservoir size. By using machine learning and artificial intelligence we can make good market predictions. Python is a powerful machine learning and data science language. Ypu can do algorithmic trading using Python. Another language that is popular with many brokers for algorithmic trading is C#. C# is a general purpose modern object oriented programming language developed by Microsoft Corporation. I have developed few courses on how to use C# in algorithmic trading. You can start with C# For Traders.  This is course for absolute beginners. I make everything easy so that you don’t face any problem. After that you can take this course C# Machine Learning For Traders. Once you have mastered C# Machine Learning, you can take C# For Algorithmic Trading. Remember what I said in the start. Future belongs to Algorithmic Trading Systems. Days of manual trading systems are over.