Category Archives: Python

State of Trend Following – Part 1

Clenow’s Trend Following System

Its a new decade! Time to see what’s up with Trend Following.

I am a huge fan of Andreas Clenow’s books, and how he demonstrated that a typical trader could replicate the performance of most large Trend Following CTAs and not pay the 2% / 20% management/incentive combo fees.  So. I felt the system that he described in his book would be a great representation of The State of Trend Following.  At the same time I am going to demonstrate TradingSimula18 (the software included in my latest book).

System Description

Take a look at my last post.  I provide the EasyLanguage and a pretty good description of Clenow’s strategy.

TradingSimula18 Code [Python]

#---------------------------------------------------------------------------------------------------
# Start programming your great trading ideas below here - don't touch stuff above
#---------------------------------------------------------------------------------------------------
# Define Long, Short, ExitLong and ExitShort Levels - mind your indentations
ATR = sAverage(myTrueRange,30,curBar,1)
posSize = 2000/(ATR*myBPV)
posSize = max(int(posSize),1)
posSize = min(posSize,20)
avg1 = xAverage(myClose,marketVal5[curMarket],50,curBar,1)
avg2 = xAverage(myClose,marketVal6[curMarket],100,curBar,1)
marketVal5[curMarket] = avg1
marketVal6[curMarket] = avg2
donchHi = highest(myHigh,50,curBar,1)
donchLo = lowest(myLow,50,curBar,1)

if mp == 1 : marketVal1[curMarket] = max(marketVal1[curMarket],myHigh[curBar-1]- 3 * ATR)
if mp ==-1 : marketVal2[curMarket] = min(marketVal2[curMarket],myLow[curBar-1]+ 3 * ATR)
# Long Entry
if avg1 > avg2 and myHigh[curBar-1] == donchHi and mp !=1:
price = myOpen[curBar]
tradeName = "TFClenowB";numShares = posSize
marketVal1[curMarket] = price - 3 * ATR
if mp <= -1:
profit,curShares,trades = bookTrade(entry,buy,price,myDate[curBar],tradeName,numShares)
barsSinceEntry = 1
marketMonitorList[curMarket].setSysMarkTrackingInfo(tradeName,cumuProfit,mp,barsSinceEntry,curShares,trades)
# Long Exit
if mp == 1 and myClose[curBar-1] <= marketVal1[curMarket] and barsSinceEntry > 1:
price = myOpen[curBar]
tradeName = "Lxit";numShares = curShares
profit,curShares,trades = bookTrade(exit,ignore,price,myDate[curBar],tradeName,numShares)
todaysCTE = profit;barsSinceEntry = 0
marketMonitorList[curMarket].setSysMarkTrackingInfo(tradeName,cumuProfit,mp,barsSinceEntry,curShares,trades)
# Short Entry
if avg1 < avg2 and myLow[curBar-1] == donchLo and mp !=-1:
price = myOpen[curBar];numShares = posSize
marketVal2[curMarket] = price + 3 * ATR
if mp >= 1:
tradeName = "TFClenowS"
profit,curShares,trades = bookTrade(entry,sell,price,myDate[curBar],tradeName,numShares)
barsSinceEntry = 1
marketMonitorList[curMarket].setSysMarkTrackingInfo(tradeName,cumuProfit,mp,barsSinceEntry,curShares,trades)
# Short Exit
if mp == -1 and myClose[curBar-1] >= marketVal2[curMarket] and barsSinceEntry > 1:
price = myOpen[curBar]
tradeName = "Sxit"; numShares = curShares
profit,curShares,trades = bookTrade(exit,ignore,price,myDate[curBar],tradeName,numShares)
todaysCTE = profit;barsSinceEntry = 0
marketMonitorList[curMarket].setSysMarkTrackingInfo(tradeName,cumuProfit,mp,barsSinceEntry,curShares,trades)
#----------------------------------------------------------------------------------------------------------------------------
# - Do not change code below - trade, portfolio accounting - our great idea should stop here
#----------------------------------------------------------------------------------------------------------------------------
TradingSimula18 Python System Testing Environment

I am going to go over this very briefly.   I know that many of the readers of my blog have attempted to use Python and the various packages out there and have given up on it.  Quantopia and QuantConnect are great websites, but I feel they approach back-testing with a programmer in mind.  This was the main reason I created TS-18 – don’t get me wrong its not a walk in the park either, but it doesn’t rely on external libraries to get the job done.  All the reports I show here are generated from the data created solely by TS-18.  Plus it is very modular – Step 1 leads to Step2 and on and on.   Referring to the code I calculate the ATR (average true range) by calling the simple average function sAverage.  I pass it myTrueRanges, 30, curBar and 1.   I am looking for the average true range over the last 30 days.  I then move onto my position sizing – posSize = $2,000 / ATR in $s.  PosSize must fit between 1 and 20 contracts.  The ATR calculation can get rather small for some markets and the posSize can get rather large.  Avg1 and Avg2 are exponential moving averages of length 50 and 100DonchHi and donchLo are the highest high and lowest low of the past 50 days.   If mp == 1 (long position) then a trailing stop (marketVal1) is set to whichever is higher – the current marketVal1 or the yesterday’s High – 3 X ATR;  the trailing stop tracks new intra-trade highs.  The trailing stop for the short side, marketVal2 is calculated in a similar manner, but low prices are used as well as a positive offset of 3 X ATR.  

Now the next section of code is quite a bit different than say EasyLanguage, but parallels some of the online Python paradigms. Here you must test the current bar’s extremes against the donchHi if you are flat and marketVal1 (the trailing stop variable) if you are long.  If flat you also test the low of the bar against donchLo.  The relationship between avg1 and avg2 are also examined.  If the testing criteria is true, then its up to you to assign the correct price, posSize and tradeName.  So you have four independent if-then constructs:

  • Long Entry – if flat test to see if a long position should be initiated
  • Long Exit – if Long then test to see if a liquidation should be initiated
  • Short Entry – if flat test to see if a short position should be initiated
  • Short Exit – if Short then test to see if a liquidation should be initiated

That’s it – all of the other things are handled by TS-18.  Now that I have completely bored you out of your mind, let’s move onto some results.

Results from 2000 – risking $2,000 per trade:

Roller Coaster Ride for most CTAs, Last one out turn off the lights!

Sector Performance from 2000

Sector Performance from 2000

From this chart it doesn’t make much sense to trade MEATS, SOFTS or GRAINS with a Trend Following approach or does it?

In the next post, I will go over the results with more in depth and possibly propose some ideas that might or might not help.  Stay Tuned!

 

Get My Latest Book-TrendFollowing Systems: A DIY Project – Batteries Included

Just wanted to let you know that my latest book has just been published.

Trend Following Systems: A DIY Project – Batteries Included: Can You Reboot and Fix Yesterday’s Algorithms to Work with Today’s Markets? 

Trend Following Systems: A DIY Project – Batteries Included

This book introduces my new Python back-tester, TradingSimula-18.  It is completely and I mean completely self contained.  All you need is the latest version of Python and you will be up and running trading systems in less than 5 minutes.  Fifteen years of data on 30 futures is included (data from Quandl).  I have included more than 20 scripts that you can test and build on.   This back-tester is different than the one I published in the Ultimate Algorithmic Trading System Toolbox.  It utilizes what I call the horizontal portfolio spanning paradigm.  Instead of sequentially testing different markets in a portfoio:

It process data in the following manner:

This form of testing allows for decisions to be made on a portfolio basis at the end of any historic bar.   Things like inputting portfolio performance into an allocation formula is super simple.  However, this paradigm opens up a lot of different “what-if” scenarios.

  1. What If I Limit 2 Markets Per Sector
  2. What If I Turn Off A Certain Sector
  3. What If I Liquidate The Largest OTE loser
  4. What If I Liquidate The Largest OTE winner
  5. What If I Only Trade The Ten Markets With The Highest ADX Values

All the data and market performance and portfolio performance is right at your fingertips.  Your testing is only limited by your creativity.

The best part is you get to learn raw Python without having to install complicated libraries like SciKit, Numpy or Pandas.  You don’t even need to install distributions of commercial products – like Anaconda.  Don’t get me wrong I think Anaconda is awesome but many times it is overkill.  If you want to do machine learning then that is the way to go.  If you want to test simple Trend Following algorithms and make portfolio level decisions you don’t need a data science application.

There isn’t a complicated interface to learn.  Its all command line driven from Python’s IDLE.  90% of the source code is revealed for the back-testing software.  Its like one of those see-thru calculators.  You see all the circuits and semiconductors, but in Python.  So you will need to flow through the code to get to the sections that pertain to your test.  Here is a small sample of how you set up the testing parameters for a Donchian Script.

 

#--------------------------------------------------------------------------------
# If you want to ignore a bunch of non-eseential stuff then
# S C R O L L A L M O S T H A L F W A Y D O W N
#--------------------------------------------------------------------------------
#TradingSimula18.py - programmed by George Pruitt
#Built on the code and ideas from "The Ultimate Algorithmic Tradins System T-Box"
#Code is broken into sections
#Most sections can and should be ignored
#Each trading algorithm must be programmed with this template
#This is the main entry into the platform
#--------------------------------------------------------------------------------
#Import Section - inlcude functions, classes, variables from external modules
#--------------------------------------------------------------------------------
# --- Do not change below here
from getData import getData
from equityDataClass import equityClass
from tradeClass import tradeInfo
from systemMarket import systemMarketClass
from indicators import highest,lowest,rsiClass,stochClass,sAverage,bollingerBands
from indicators import highest,lowest,rsiClass,stochClass,sAverage,bollingerBands,\
adxClass,sAverage2
from portfolio import portfolioClass
from systemAnalytics import calcSystemResults
from utilityFunctions import getDataAtribs,getDataLists,roundToNearestTick,calcTodaysOTE
from utilityFunctions import setDataLists,removeDuplicates
from portManager import portManagerClass,systemMarkTrackerClass
from positionMatrixClass import positionMatrixClass
from barCountCalc import barCountCalc

from sectorClass import sectorClass, parseSectors, numPosCurrentSector,getCurrentSector
#-------------------------------------------------------------------------------------------------
# Pay no attention to these two functions - unless you want to
#-------------------------------------------------------------------------------------------------
def exitPos(myExitPrice,myExitDate,tempName,myCurShares):
global tradeName,entryPrice,entryQuant,exitPrice,numShares,myBPV,cumuProfit
if mp < 0:
trades = tradeInfo('liqShort',myExitDate,tempName,myExitPrice,myCurShares,0)
profit = trades.calcTradeProfit('liqShort',mp,entryPrice,myExitPrice,entryQuant,myCurShares) * myBPV
profit = profit - myCurShares *commission;trades.tradeProfit = profit;cumuProfit += profit
trades.cumuProfit = cumuProfit
if mp > 0:
trades = tradeInfo('liqLong',myExitDate,tempName,myExitPrice,myCurShares,0)
profit = trades.calcTradeProfit('liqLong',mp,entryPrice,myExitPrice,entryQuant,myCurShares) * myBPV
profit = profit - myCurShares * commission;trades.tradeProfit = profit;cumuProfit += profit
trades.cumuProfit = cumuProfit
curShares = 0
for remShares in range(0,len(entryQuant)):curShares += entryQuant[remShares]
return (profit,trades,curShares)

def bookTrade(entryOrExit,lOrS,price,date,tradeName,shares):
global mp,commission,totProfit,curShares,barsSinceEntry,listOfTrades
global entryPrice,entryQuant,exitPrice,numShares,myBPV,cumuProfit
if entryOrExit == -1:
profit,trades,curShares = exitPos(price,date,tradeName,shares);mp = 0
else:
profit = 0;curShares = curShares + shares;barsSinceEntry = 1;entryPrice.append(price);entryQuant.append(shares)
if lOrS == 1:mp += 1;trades = tradeInfo('buy',date,tradeName,entryPrice[-1],shares,1)
if lOrS ==-1:mp -= 1;trades = tradeInfo('sell',date,tradeName,entryPrice[-1],shares,1)
return(profit,curShares,trades)

dataClassList = list()

marketMonitorList,masterDateList,masterDateGlob,entryPrice = ([] for i in range(4))
buy = entry = 1; sell = exit = -1; ignore = 0;
entryQuant,exitQuant,trueRanges,myBPVList = ([] for i in range(4))
myComNameList,myMinMoveList,systemMarketList = ([] for i in range(3))
cond1,cond2,cond3,cond4 = ([] for i in range(4))
marketVal1,marketVal2,marketVal3,marketVal4 = ([] for i in range(4))
portManager = portManagerClass();marketList = getData();portfolio = portfolioClass()
numMarkets = len(marketList);positionMatrix = positionMatrixClass();positionMatrix.numMarkets = numMarkets
firstMarketLoop = True

#----------------------------------------------------------------------------------
# Set up algo parameters here
#----------------------------------------------------------------------------------
startTestDate = 20100101 #must be in yyyymmdd
stopTestDate = 20190228 #must be in yyyymmdd
rampUp = 100 # need this minimum of bars to calculate indicators
sysName = 'Donch-MAX2NSect' #System Name here
initCapital = 500000
commission = 100
Ignore Most Of This Code

Everything is batched processed: set up, pick market or portfolio, run.  Then examine all of the reports.  Here is an example of the sector analysis report.

          Total Profit  Max DrawDown
Currency -------------------------------
BP -14800 19062
SF -8600 53575
AD 4670 11480
DX 10180 10279
EC -9000 16775
JY 10025 18913
CD -19720 21830
-------------------------------------------
Totals: -27245 69223
-------------------------------------------
Energies -------------------------------
CL -40400 55830
HO 80197 23382
NG -14870 28920
RB -45429 61419
-------------------------------------------
Totals: -20502 75957
-------------------------------------------
Metals -------------------------------
GC 27210 36610
SI -1848 2389
HG -2402 2438
PL -16750 25030
PA 27230 38615
-------------------------------------------
Totals: 33440 61472
-------------------------------------------
Grains -------------------------------
S_ 27312 9088
W_ -25538 32600
C_ -1838 12212
BO -8460 9544
SM 390 11250
RR -1390 12060
-------------------------------------------
Totals: -9523 34135
-------------------------------------------
Financials -------------------------------
US 29488 18375
TY 969 12678
TU -2020 3397
FV -2616 4531
ED -4519 4869
-------------------------------------------
Totals: 21302 30178
-------------------------------------------
Softs -------------------------------
SB -1716 19717
KC 15475 44413
CC 540 8090
CT -8705 35660
LB 22269 16586
OJ 4720 8262
-------------------------------------------
Totals: 32583 57976
-------------------------------------------
Meats -------------------------------
LC -18910 24020
LH -31270 35640
FC 14600 25737
-------------------------------------------
Totals: -35580 59550
-------------------------------------------
Sector Analysis

Plus I include EasyLanguage for the majority of the scripts.  Of course without the portfolio level management.  I am working on a new website that will support the new book at TrendFollowingSystems.com.

Please take a look at my latest book – it would make an awesome Christmas present.

 

Python Script To Import List of Trades into TradeStation’s EasyLanguage – Sort of

Converting A List of Trades, Dates and Prices Into EasyLanguage Arrays:

As the old saying goes “a picture is worth a thousand words!”  Have you ever been given a list of trades like this:

Sell Short,20010622,1178.50 
Buy to Cover,20010626,1159.75
Sell Short,20010801,1150.00
Buy to Cover,20010807,1139.75
Sell Short,20010814,1129.00
Buy to Cover,20010816,1117.25
Sell Short,20011001,976.75
Buy to Cover,20011004,1016.75
Sell Short,20011107,1053.00
Buy to Cover,20011123,1069.50
Sell Short,20011219,1076.25
Buy to Cover,20020102,1075.00
Sell Short,20020129,1067.25
Buy to Cover,20020131,1046.75
Sell Short,20020131,1046.75
Buy to Cover,20020205,1026.75
Sell Short,20020520,1033.25
Buy to Cover,20020522,1011.50
Sell Short,20020731,832.00
Buy to Cover,20020805,792.50
Sell Short,20020812,834.00
Buy to Cover,20020814,811.75
Sell Short,20020911,838.50
Buy to Cover,20020913,816.75
List of Trades : Order, Date, Price

But really wanted to see this:

I have created a small Python script that will take a list of trades like those listed in table above and create the following EasyLanguage:

arrays: DateArr[500](0),TradeArr[500](""),PriceArr[500](0);
DateArr[0]=1010622;TradeArr[0]="SS";PriceArr[0]=1178.5;
DateArr[1]=1010626;TradeArr[1]="SX";PriceArr[1]=1159.75;
DateArr[2]=1010801;TradeArr[2]="SS";PriceArr[2]=1150.0;
DateArr[3]=1010807;TradeArr[3]="SX";PriceArr[3]=1139.75;
DateArr[4]=1010814;TradeArr[4]="SS";PriceArr[4]=1129.0;
DateArr[5]=1010816;TradeArr[5]="SX";PriceArr[5]=1117.25;
DateArr[6]=1011001;TradeArr[6]="SS";PriceArr[6]=976.75;
DateArr[7]=1011004;TradeArr[7]="SX";PriceArr[7]=1016.75;
DateArr[8]=1011107;TradeArr[8]="SS";PriceArr[8]=1053.0;
DateArr[9]=1011123;TradeArr[9]="SX";PriceArr[9]=1069.5;
DateArr[10]=1011219;TradeArr[10]="SS";PriceArr[10]=1076.25;
DateArr[11]=1020102;TradeArr[11]="SX";PriceArr[11]=1075.0;
DateArr[12]=1020129;TradeArr[12]="SS";PriceArr[12]=1067.25;
DateArr[13]=1020131;TradeArr[13]="SX";PriceArr[13]=1046.75;
DateArr[14]=1020131;TradeArr[14]="SS";PriceArr[14]=1046.75;
DateArr[15]=1020205;TradeArr[15]="SX";PriceArr[15]=1026.75;
DateArr[16]=1020520;TradeArr[16]="SS";PriceArr[16]=1033.25;
DateArr[17]=1020522;TradeArr[17]="SX";PriceArr[17]=1011.5;
Converting list of trades to EasyLanguage

This just creates the arrays that you can use to graph the trades on a chart.  If you are using exact prices you got to make sure your data aligns with the prices in the list of trades.  If you are only entering on the open or the close of the bar then the price array isn’t necessary.

The following Python script will also be helpful if you want to learn how to open a file in csv format, read it into lists, convert it and then save the output to a file.

#-------------------------------------------------------------------------------
# Name: Read csv file via askOpen and save txt file via askSave
# Purpose: Read the trade metrics from a TradeStation csv format
# and build arrays from the information to display on charts in
# TradeStation
# Author: georg
#
# Created: 29/08/2018
# Copyright: (c) georg 2018
#-------------------------------------------------------------------------------
import csv
import tkinter as tk
import os.path
from tkinter.filedialog import askopenfilenames
from tkinter.filedialog import asksaveasfilename

tradeType = list()
tradeDate = list()
tradePrice = list()

def main():
root = tk.Tk()
root.withdraw()
files = askopenfilenames(filetypes=(('CSV files', '*.csv'),
('TXT files', '*.txt')),
title='Select CSV format only!')
fileList = root.tk.splitlist(files)
fileListLen = len(fileList)


# make sure you know the format ahead of time
# I know "Buy",20180828,2745.75
#
cnt = 0
for files in range(0,fileListLen):
head,tail = os.path.split(fileList[files])
with open(fileList[files]) as f:
f_csv = csv.reader(f)
for row in f_csv:
numCols = len(row)
tradeType.append(row[0])
tradeDate.append(int(row[1]))
tradePrice.append(float(row[2]))
cnt += 1
f.close


filename = asksaveasfilename(title="Will Save File with '.txt'",defaultextension=".txt")
# filename = filename + '.txt'
target1 = open(filename,'w')
outString = 'arrays: DateArr[500](0),TradeArr[500](0),PriceArr[500](0);\n'
target1.write(outString)
for x in range(0,cnt):
if tradeType[x] == "Sell Short": tradeType[x] = "SS"
if tradeType[x] == "Buy": tradeType[x] = "B"
if tradeType[x] == "Buy to Cover": tradeType[x] = "SX"
if tradeType[x] == "Sell": tradeType[x] = "LX"
outString = 'DateArr['+str(x)+']='+str(tradeDate[x]-19000000)+';TradeArr['+str(x)+']="'+tradeType[x]+'";PriceArr['+str(x)+']='+str(tradePrice[x])+';\n'
target1.write(outString)
target1.close


if __name__ == '__main__':
main()
Python Script Open, Read, Convert and Write A File Using TK Dialogs

And here is the EasyLanguage code that will step through the arrays and place the trades accordingly.  I noticed that sometimes two trades could occur on the same bar, but only two and you will notice in the code where I programmed this occurrence.

vars: cnt(0);

If date of tomorrow = DateArr[cnt] then
Begin
print("Inside: ",d," ",dateArr[cnt]);
If tradeArr[cnt] = "B" then
begin
buy next bar at PriceArr[cnt] stop;
end;
If tradeArr[cnt] = "LX" then
begin
sell next bar at PriceArr[cnt] stop;
end;
If tradeArr[cnt] = "SS" then
begin
sellShort next bar at PriceArr[cnt] stop;
end;
If tradeArr[cnt] = "SX" then
begin
buyToCover next bar at PriceArr[cnt] stop;
end;
cnt = cnt + 1;
If DateArr[cnt] = DateArr[cnt-1] then
Begin
print("two trades same day ",d," ",dateArr[cnt]);
If tradeArr[cnt] = "B" then
begin
buy next bar at PriceArr[cnt] stop;
end;
If tradeArr[cnt] = "LX" then
begin
sell next bar at PriceArr[cnt] stop;
end;
If tradeArr[cnt] = "SS" then
begin
print("looking to go short at ",PriceArr[cnt]);
sellShort next bar at PriceArr[cnt] stop;
end;
If tradeArr[cnt] = "SX" then
begin
buyToCover next bar at PriceArr[cnt] stop;
end;
cnt = cnt + 1;
end;
end;
EasyLanguage Snippet To Execute Trades Stored in Arrays

 

 

How to Create a Dominant Cycle Class in Python

John Ehlers used the following EasyLanguage code to calculate the Dominant Cycle in a small sample of data.  If you are interested in cycles and noise reduction, definitely check out the books by John Ehlers – “Rocket Science for Traders” or “Cybernetic Analysis for Stocks and Futures.”  I am doing some research in this area and wanted to share how I programmed the indicator/function in Python.  I refer you to his books or online resources for an explanation of the code.  I can tell you it involves an elegantly simplified approach using the Hilbert Transform.

 

Inputs:	Price((H+L)/2);

Vars: Imult(.635),
Qmult (.338),
InPhase(0),
Quadrature(0),
count(0),
Re(0),
Im(0),
DeltaPhase(0),
InstPeriod(0),
Period(0);

If CurrentBar > 8 then begin
Value1 = Price - Price[7];
Inphase = 1.25*(Value1[4] - Imult*Value1[2]) + Imult*InPhase[3];

// print(price," ",price[7]," ",value1," ",inPhase," ",Quadrature," ",self.im[-1]," ",self.re[-1])
// print(d," ",h," ",l," ",c," ",Value1[4]," ",Imult*Value1[2]," ", Imult*InPhase[3]," ",Inphase);
Quadrature = Value1[2] - Qmult*Value1 + Qmult*Quadrature[2];
Re = .2*(InPhase*InPhase[1] + Quadrature*Quadrature[1]) + .8*Re[1];
Im = .2*(InPhase*Quadrature[1] - InPhase[1]*Quadrature) + .8*Im[1];
print(d," ",o," ",h," ",l," ",c," ",value1," ",inPhase," ",Quadrature," ",Re," ",Im);
If Re <> 0 then DeltaPhase = ArcTangent(Im/Re);

{Sum DeltaPhases to reach 360 degrees. The sum is the instantaneous period.}
InstPeriod = 0;
Value4 = 0;
For count = 0 to 50 begin
Value4 = Value4 + DeltaPhase[count];
If Value4 > 360 and InstPeriod = 0 then begin
InstPeriod = count;
end;
end;

{Resolve Instantaneous Period errors and smooth}
If InstPeriod = 0 then InstPeriod = InstPeriod[1];
Period = .25*InstPeriod + .75*Period[1];

Plot1(Period, "DC");
EasyLanguage Code For Calculating Dominant Cycle

In my Python based back tester an indicator of this type is best programmed by using a class.  A class is really a simple construct, especially in Python, once you familiarize yourself with the syntax.   This indicator requires you to refer to historical values to calculate the next value in the equation:  Value1[4], inPhase[1], re[2], etc.,.  In EasyLanguage these values are readily accessible as every variable is defined as a BarArray – the complete history of a variable is accessible by using indexing.  In my PSB I used lists to store values for those variables most often used such as Open, High, Low, Close.  When you need to store the values of let’s say the last five bars its best to just create a list on the fly or build them into a class structure.  A Class stores data and data structures and includes the methods (functions) that the data will be pumped into.  The follow code describes the class in two sections:  1) data declaration and instantiation and 2) the function to calculate the Dominant Cycle.  First off I create the variables that will hold the constant values: imult and qmult.  By using the word self I make these variables class members and can access them using “.” notation.  I will show you later what this means.  I also make the rest of the variables class members, but this time I make them lists and instantiate the first five values to zero.  I use list comprehension to create the lists and zero out the first five elements – all in one line of code.  This is really just a neat short cut, but can be used for much more powerful applications.  Once you create a dominantCycleClass object the object is constructed and all of the data is connected to this particular object.  You can create many dominantCycleClass objects and each one would maintain its own data.  Remember a class is just a template that is used to create an object.

class dominantCycleClass(object):
def __init__(self):
self.imult = 0.635
self.qmult = 0.338
self.value1 = [0 for i in range(5)]
self.inPhase = [0 for i in range(5)]
self.quadrature = [0 for i in range(5)]
self.re = [0 for i in range(5)]
self.im = [0 for i in range(5)]
self.deltaPhase = [0 for i in range(5)]
self.instPeriod = [0 for i in range(5)]
self.period = [0 for i in range(5)]
Data Portion of Class

 

The second part of the class template contains the method or function for calculating the Dominant Cycle.  Notice how I index into the lists to extract prior values.  You will also see the word self. preceding the variable names used in the calculations Initially I felt like this redundancy hurt the readability of the code and in this case it might.  But by using self. I know I am dealing with a class member.  This is an example of the ” . ” notation I referred to earlier.  Basically this ties the variable to the class.

def calcDomCycle(self,dates,hPrices,lPrices,cPrices,curBar,offset):
tempVal1 = (hPrices[curBar - offset] + lPrices[curBar-offset])/2
tempVal2 = (hPrices[curBar - offset - 7] + lPrices[curBar-offset - 7])/2
self.value1.append(tempVal1 - tempVal2)
self.inPhase.append(1.25*(self.value1[-5] - self.imult*self.value1[-3]) + self.imult*self.inPhase[-3])
self.quadrature.append(self.value1[-3] - self.qmult*self.value1[-1] + self.qmult*self.quadrature[-2])
self.re.append(.2*(self.inPhase[-1]*self.inPhase[-2]+self.quadrature[-1]*self.quadrature[-2])+ 0.8*self.re[-1])
self.im.append(.2*(self.inPhase[-1]*self.quadrature[-2] - self.inPhase[-2]*self.quadrature[-1]) +.8*self.im[-1])
if self.re[-1] != 0.0: self.deltaPhase.append(degrees(atan(self.im[-1]/self.re[-1])))
if len(self.deltaPhase) > 51:
self.instPeriod.append(0)
value4 = 0
for count in range(1,51):
value4 += self.deltaPhase[-count]
if value4 > 360 and self.instPeriod[-1] == 0:
self.instPeriod.append(count)
if self.instPeriod[-1] == 0: self.instPeriod.append(self.instPeriod[-1])
self.period.append(.25*self.instPeriod[-1]+.75*self.period[-1])
return(self.period[-1])
Dominant Cycle Method

Okay we now have the class template to calculate the Dominant Cycle but how do we us it?

#---------------------------------------------------------------------------------
#Instantiate Indicator Classes if you need them
#---------------------------------------------------------------------------------
# rsiStudy = rsiClass()
# stochStudy = stochClass()
domCycle = dominantCycleClass()
#---------------------------------------------------------------------------------
#Call the dominantCycleClass method using " . " notation.
tempVal1 = domCycle.calcDomCycle(myDate,myHigh,myLow,myClose,i,0)
#Notice how I can access class members by using " . " notation as well!
tempVal2 = domCycle.imult
Dominant Cycle Object Creation

Here I assign domCycle the object created by calling the dominantCycleClass constructor.  TempVal1 is assigned the Dominant Cycle when the function or method is called using the objects name (domCycle) and the now familiar ” . ” notation.  See how you can also access the imult variable using the same notation.

Here is the code in its entirety.  I put this in the indicator module of the PSB.

class dominantCycleClass(object):
def __init__(self):
self.imult = 0.635
self.qmult = 0.338
self.value1 = [0 for i in range(5)]
self.inPhase = [0 for i in range(5)]
self.quadrature = [0 for i in range(5)]
self.re = [0 for i in range(5)]
self.im = [0 for i in range(5)]
self.deltaPhase = [0 for i in range(5)]
self.instPeriod = [0 for i in range(5)]
self.period = [0 for i in range(5)]

def calcDomCycle(self,dates,hPrices,lPrices,cPrices,curBar,offset):
tempVal1 = (hPrices[curBar - offset] + lPrices[curBar-offset])/2
tempVal2 = (hPrices[curBar - offset - 7] + lPrices[curBar-offset - 7])/2
self.value1.append(tempVal1 - tempVal2)
self.inPhase.append(1.25*(self.value1[-5] - self.imult*self.value1[-3]) + self.imult*self.inPhase[-3])
self.quadrature.append(self.value1[-3] - self.qmult*self.value1[-1] + self.qmult*self.quadrature[-2])
self.re.append(.2*(self.inPhase[-1]*self.inPhase[-2]+self.quadrature[-1]*self.quadrature[-2])+ 0.8*self.re[-1])
self.im.append(.2*(self.inPhase[-1]*self.quadrature[-2] - self.inPhase[-2]*self.quadrature[-1]) +.8*self.im[-1])
if self.re[-1] != 0.0: self.deltaPhase.append(degrees(atan(self.im[-1]/self.re[-1])))
if len(self.deltaPhase) > 51:
self.instPeriod.append(0)
value4 = 0
for count in range(1,51):
value4 += self.deltaPhase[-count]
if value4 > 360 and self.instPeriod[-1] == 0:
self.instPeriod.append(count)
if self.instPeriod[-1] == 0: self.instPeriod.append(self.instPeriod[-1])
self.period.append(.25*self.instPeriod[-1]+.75*self.period[-1])
return(self.period[-1])
Dominant Cycle Class - Python

 

Pyramiding with Python

A reader of the “UATSTB” asked for an example of how to pyramid in the Python System BackTester (PSB).  I had original bug where you couldn’t peel off all of the positions at once.  If you tried then the PSB would crash.  I have now fixed that problem and here are the necessary lines to add on another position after the initial positions is put on.   I am using a simple moving average crossover to initiate the first position and then if the longer term moving average is positive for 4 days in a row I add on another position.  That’s it.  A max of two position only.  The system either gets reversed to the other side, stopped out or takes profits.  Attached in this post is the module that fixes the pyramiding problem and the code for this system in its entirety.   I will also put the fix in the version 2.0 download.

        upCnt = 0
dnCnt = 0
for j in range(4):
if sAverage(myClose,39,i,j) > sAverage(myClose,39,i,j+1):
upCnt += 1
if sAverage(myClose,39,i,j) < sAverage(myClose,39,i,j+1):
dnCnt += 1


#Long Entry Logic
if (mp != 1) and avg1 > avg2 and prevAvg1 < prevAvg2:
profit = 0
price = myClose[i]
tradeName = "DMA Buy"

#Long Pyramid Logic
if (mp == 1) and upCnt == 4:
profit = 0
price = myClose[i]
tradeName = "LongPyra"
Pyramiding the Long Side of a Dual Moving Average

I only put in the summary of code that will create the initial position and then add 1 more.  Notice the highlighted code from line 3 to 7.  Here I am using a simple loop to determine if the the 39-day moving average is increasing/decreasing consecutively over the past four days.  If the upCnt == 4 then I add a long position.  Notice how I test in the Long Pyramid Logic the values of mp and upCnt.  I only go long another position if I am already long 1 and upCnt == 4.  

Here is a print out of some of the trades:

20040923      DMA Buy  1 127.79000       0.00       0.00
20040923 LongPyra 1 127.79000 0.00 0.00
20041006 L-Prof 2 130.79000 5800.00 14560.00
20041116 DMA Short 1 126.14000 0.00 0.00
20041119 ShortPyra 1 128.64000 0.00 0.00
20041201 S-Prof 2 125.64000 3300.00 17860.00
20050121 DMA Buy 1 127.85000 0.00 0.00
20050202 LongPyra 1 126.01000 0.00 0.00
20050222 L-Prof 2 129.01000 3960.00 21820.00
20050418 DMA Short 1 128.18000 0.00 0.00
20050419 S-MMLoss 1 130.28000 -2200.00 19620.00
20050616 DMA Buy 1 131.41000 0.00 0.00
20050621 LongPyra 1 133.02000 0.00 0.00
20050630 L-MMLoss 2 130.48000 -3670.00 15950.00
20050809 DMA Short 1 135.95000 0.00 0.00
20050810 RevShrtLiq 1 137.78000 -1930.00 14020.00
20050810 DMA Buy 1 137.78000 0.00 0.00
20050810 LongPyra 1 137.78000 0.00 0.00
20050829 L-Prof 2 140.98000 6200.00 20220.00
Trade Listing Of DMA with Pyramiding

While I was posting the trades I found something that struck my funny – look at line 5 (highlighted).  The short pyramid trade occurs at a higher price than the initial short – at first I thought I had made a programming error.  So I thought I would double check the code and then do some debugging.  Instead of invoking the debugger which is really cool and easy to use I decided to just print out the results to the console.

        for j in range(4):
if sAverage(myClose,39,i,j) > sAverage(myClose,39,i,j+1):
upCnt += 1
if sAverage(myClose,39,i,j) < sAverage(myClose,39,i,j+1):
dnCnt += 1
if tempDate == 20041119:
print(myDate[i]," ",j," ",sAverage(myClose,39,i,j))

'''Output of debugging using a print statement
20041119 0 130.68692307692308
20041119 1 130.69538461538463
20041119 2 130.74871794871794
20041119 3 130.7723076923077'''
Debugging using Print Statements

As you can see the longer term moving average is moving down event though price has increased.  Take a look at this chart and you can see multiple occurrences of this.

Examples of Divergence of Moving Average and Price

Remember to copy and replace the tradeClass.py in you PSB2.0 directory – this fixes the pyramid bug.  Also copy the DMAPyra.py to the same directory.  If you haven’t as of yet download the PSB from this website and buy the book for a very good description.

PostPythonCode

Restructuring Trade Entry with PSB

It has been brought to my attention by a very astute reader of the book that the ordering of the buy/sell/longliq/shortliq directives creates a potential error by giving preference to entries.  The code in the book and my examples thus far test for a true condition first in the entry logic and then in the exit logic.  Let’s say your long exit stop in the Euros is set at 11020, but your reversal is set at 11000.  The python back tester will skip the exit at 11020 and reverse at 11000.  This only happens if both stops are hit on the same day.  If this happens you will incur a 20 point additional loss.  You can prevent this by using logic similar to:

if ((mp == 0 or (mp == -1 and stb < stopb)) and myHigh[D0] >= stb) :
Eliminate Reversal Bias

Notice bow I compare the price level of stb and stopb [stb – reversal and stopb – liquidation].  I added this to the long entry logic – the code is only executed if the entry is less than the exit (closer to the current market).  I am in the process of restructuring the flow so that all orders will be examined on a bar by bar basis and the ones that should take place (chronologically speaking) will do so and be reflected in the performance metrics.  This can cause multiple trades on a single bar.  This is what happens in real trading and should be reflected in the PSB.  This is where the lack of a GOTO creates a small headache in Python.  The ESB already takes this into consieration.  I will post when I finalize the code.

Version 2.0 of Python System Back-tester Available

I have just wrapped up the latest version of the Python System Back-tester (PSB).

I have added some more portfolio performance metrics and you will see these in the performance reports.  The most useful addition is the concept of the .POR file when you run a system.  Instead of having to select your data files each time you run a system, you can build a .POR file with a list of files/markets you want to batch run.

Here is an example of a Portfolio file:

TY.CSV

CU.CSV

SB.CSV

S2.CSV

QG.CSV

QM.CSV

C2.CSV

Just make sure you put the .POR file inside the same folder that contains your testing data.  I have included a TestPortfolio.por file in version 2.0.  I would treat the different versions as completely separate applications.  Your existing .py algorithm files will need to be slightly modified to work with version 2.0.

This line of code needs to be modified from this:

systemMarket.setSysMarkInfo(sysName,myComName,listOfTrades,equity)

to this:

systemMarket.setSysMarkInfo(sysName,myComName,listOfTrades,equity,initCapital)

This line is near the bottom of the overall loop.  I added the initCapital variable so you could do position sizing and the performance metrics would reflect this initial value.

And set initCapital to a pertinent value.  I put it right below the sysName variable:

sysName = ‘BollingerBandSys’ #System Name here

initCapital = 100000 #starting account balance

Also I corrected a small bug in the main loop.  You should change this:

for i in range(len(myDate) – numBarsToGoBack,len(myDate)):

to

for i in range(len(myDate) – (numBarsToGoBack-rampUp),len(myDate)):

In another post I will show how the portfolio performance metrics have changed.  I hope you like the new version.  I will be adding a library of trading systems utilizing this new version in a few days.

If you want to download Version 2.0 – just go the the following link

PSBVersion2.0

If You Can Do This – You Can Test Any Algorithm!

All the unnecessary lines of the Python System Back-Testing Module have been hidden.  Only the lines that you need to develop the next great algorithm are included.  Reads sort  of like English.  This snippet introduces you to the use of functions and lists – two major components of the Python language.  If you buy my latest book – “The Ultimate Algorithmic Trading System Toolbox” then simply email me or sign up through the contact form and you will get version 2.0 for free!

PythonReduction

Turtle Volatility Loss in Python Back Tester – Part 3 in Series

The Turtle N or Volatility is basically a 20-day Average True Range in terms of  Dollars.  This amount is considered the market volatility.  In other words the market, based on the average, can either move up or down by this amount.  It can move much less or much further; this is just an estimate.  If the market moves 2 N against a position, then as a Turtle you were to liquidate your stake in that commodity.  The logic indicates that if the market has a break out and then moves 2 N in the opposite direction, then the break out has failed.  First the code must be defined to represent the market volatility.  This is simple enough by using the sAverage function call and passing it the trueRanges and 20 days.  There’s no use in converting this to dollars because what we want is a price offset.  Once a position is entered the turtleN  is either added to the price [short position] or subtracted from the price [long position] to determine the respective stop levels.  Look at lines 2, 8 and 17 to see how this is handled.  An additional  trade code block must be added to facilitate this stop.  Lines 17 to 28 takes care of exiting a long position when the market moves 2 N in the opposite direction.   This new stop is in addition to the highest/lowest high/low stops for the past 10 -20 days.

        atrVal = sAverage(trueRanges,20,i,1)

turtleN = atrVal*2


if lastTradeLoser == True :
tradeName = "Turt20Buy"
mp += 1
longNExitStop = price - turtleN
marketPosition[i] = mp
entryPrice.append(price)
entryQuant.append(numShares)
curShares = curShares + numShares
trades = tradeInfo('buy',myDate[i],tradeName,entryPrice[-1],numShares,1)
barsSinceEntry = 1
listOfTrades.append(trades)
#long Exit - 2 N Loss
if mp >= 1 and myLow[i] <= longNExitStop and barsSinceEntry > 1:
price = min(myOpen[i],longNExitStop)
tradeName = "LongNExitLoss"
exitDate =myDate[i]
numShares = curShares
exitQuant.append(numShares)
profit,trades,curShares = exitPos(price,myDate[i],tradeName,numShares)
if curShares == 0 : mp = marketPosition[i] = 0
totProfit += profit
todaysCTE = profit
listOfTrades.append(trades)
maxPositionL = maxPositionL - 1
Turtle Part 3