Within my first article of the year I talked about net trader positioning and how this could provide useful information for deciding whether there is a trend or not and whether there might be opportunities for entry if a trader predicts that a trend is more likely to stop-and-reverse or continue. However ForexFactory and myfxbook both lack access to significant amounts of historical net positioning data, making back-testing and analysis using this information much more difficult. Today I want to talk about how you can download and constantly update this information from the Oanda servers using the Oanda REST API, giving you the possibility to obtain daily updated information that you can then use to create back-tests and evaluate ideas making use of this historical data. You can also obtain the data when doing live trading as well so you can easily implement the same setup in live trading after you figure out what to do with this information.
–
#!/usr/bin/python
import subprocess, os, csv
import json, httplib, math, calendar, datetime, requests
from time import *
import pandas as pd
import sys
global OandaSymbol
OandaSymbol = {
'EURUSD': 'EUR_USD',
'GBPUSD': 'GBP_USD',
'USDJPY': 'USD_JPY',
'AUDUSD': 'AUD_USD',
'USDCHF': 'USD_CHF',
'USDCAD': 'USD_CAD',
'EURCHF': 'EUR_CHF',
'EURGBP': 'EUR_GBP',
'EURJPY': 'EUR_JPY',
'AUDJPY': 'AUD_JPY',
'AUDCAD': 'AUD_CAD',
'GBPJPY': 'GBP_JPY',
'GBPCHF': 'GBP_CHF',
'GBPAUD': 'GBP_AUD',
'GBPCAD': 'GBP_CAD',
'EURAUD': 'EUR_AUD',
'EURCAD': 'EUR_CAD',
'CHFJPY': 'CHF_JPY',
'CADJPY': 'CAD_JPY'
}
def update_positioning_data(symbol, tokenValue):
print "getting data for {}".format(symbol)
j = 0
token = "Bearer %s" % (tokenValue)
server = "api-fxtrade.oanda.com"
headers = {'Authorization': token, 'Connection': 'Keep-Alive', 'Accept-Encoding': 'gzip, deflate', 'Content-type': 'application/x-www-form-urlencoded'}
global OandaSymbol
url = "https://" + server + "/labs/v1/historical_position_ratios?instrument=" + OandaSymbol[symbol] + "&period=31536000"
try:
req = requests.get(url, headers = headers)
resp = req.json()
except Exception:
e = Exception
print "ERROR GETTING DATA. ABORTING."
quit()
position_data = resp['data'][OandaSymbol[symbol]]['data']
for p in range(len(position_data)):
position_data[p][0] = datetime.datetime.fromtimestamp(position_data[p][0])
headers = ["timestamp", "lpr", "rate"]
df = pd.DataFrame(position_data, columns=headers)
df = df.set_index(df['timestamp'])
df = df.drop(['timestamp', 'rate'], axis=1)
df = df[:-1]
try:
old_df = pd.DataFrame.from_csv("OANDA_POSITION_"+symbol + '.csv', index_col=0)
df = pd.concat([df,old_df])
except:
print "No old datafile found. Generating new one."
df = df[~df.index.duplicated(keep='first')]
df.to_csv("OANDA_POSITION_"+symbol + '.csv', date_format='%Y-%m-%d %H:%M:%S')
def main():
symbolsToUpdate = ["EURUSD","GBPUSD","USDJPY","USDCHF","EURJPY","AUDUSD","AUDJPY","EURAUD","USDCAD","GBPJPY","EURGBP","GBPCHF", "CADJPY"]
for symbol in symbolsToUpdate:
update_positioning_data(symbol, "INSERT_YOUR_LIVE_ACCOUNT_REST_API_TOKEN_HERE")
##################################
### MAIN ####
##################################
if __name__ == "__main__": main()
–
The above python script allows you to download net positioning information from the Oanda servers that tells you what percentage of retail traders within this broker are holding net long and short positions within live accounts. To use this script you need to insert your Oanda live account REST API token where indicated. After you run the script it will create a set of files using all available historical positioning data – excluding the current day – which means you will always have files with at least one year worth of data. Sadly Oanda does not allow you to download more of this data so you will be restricted to a max of one year of data each time you start the files from scratch. Each time you re-run the script it will search for any already downloaded files and it will append new net positioning information so you can build your own net positioning database by using this simple script.
The symbols that the script uses are contained within the “symbolsToUpdate” variable. You can add symbols to this python list if you want to include other currency pairs or CFDs. However bear in mind that any symbols you add will also need to be added to the OandaSymbol dictionary in the beginning of the script or otherwise the script will give you an error when it gets to that symbol. If you don’t want data for any of the included symbols you can also remove the symbols from the list to avoid downloading their data.
–
–
After downloading the data each CSV file will contain two columns, one with the timestamp and another with the long positioning for that symbol. Since the short positioning is just 100 minus the long positioning you can easily obtain all the positioning information from this single data point. The positioning data includes timestamp information in YYYY-MM-DD HH:MM:SS information and each day of net positioning data is closed at 11AM UTC time. If you want to use this data for back-testing make sure that you do not introduce look-ahead bias by using positioning information that wouldn’t have been finished by the time you use the data. If your back-testing data has a different GMT shift and/or DST make sure you convert your data to match this timestamp information or you will not get accurate results.
With this data now in a well organized CSV format we can perform some basic analysis and plotting using something like R. The image above shows you a simple plot of the daily close and the net positioning for the GBPJPY over the past year. As you can see the positioning remained higher than 50% while the GBPJPY dropped through most of the year – in alignment with my previous observations about retail positioning always being against trending movements -and later it dipped below 50% as the GBPJPY started a strong uptrend in late 2016. Using R you can carry out all sorts of studies using this data and the currency information. For example if predictability and trends are of interest you can figure out whether the net positioning is related to variables like the fractal dimension and the Hurst exponent.
–
–
The above is clearly just the tip of the iceberg. Using this script you can carry out your own daily-updated analysis of the net positioning information and you can then do interesting analysis to see if this can yield interesting information for trading. The net positioning data – especially historical positioning data – is seldom used by retail traders for trading so it might be able to yield some interesting systems or at least some interesting insights about currency pair behavior. If you would like to learn more about trading systems and how you too can build your own algorithmic trading strategies please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.strategies.
