Don’t Fool Yourself – Limitations of Back Testing with Daily Data [EasyLanguage]

Which equity curve do you like best? (created with EasyLanguage script) This one…

Or this one?

Obviously the first one.  Even though it had a substantial draw down late in the test.  What if I told you that the exact same system logic generated both curves?  Here is the EasyLanguage code for this simple system.

Buy next bar at open of next bar + .25 *avgTrueRange(10) stop;
Sellshort next bar at open of next bar - .25*avgTrueRange(10) stop;

setStopLoss(500);
setProfitTarget(1000);
Open Range Break Out with Profit and Loss Objective

This algorithm relies heavily on needing to know which occurred first: the high or the low of the day.   The second chart tells the true story because it looks inside the daily bar to see what really happened.  The first chart uses an algorithm to try to determine which happened first and applies this to the trades.  In some instances,  the market looks like it opens then has a slight pull back and then goes up all day.  As a result the system buys and holds the trade through the close and onto the next day, but in reality the market opens, goes up and triggers a long entry, then retraces and you get stopped out.  What was a nice winner turns into a bad loss.  Here is an example of what might have happened during a few trades:

Nice flow – sold, bought, sold, bought, sold again and finally a nice profit.  But this is what really happened:

Sold, bought, reversed short on same day and stopped out on same day.  Then sold and reversed long on same day and finally sold and took profit.   TradeStation’s Look Inside Bar feature helps out when your system needs to know the exact path the market made during the day.  In many cases, simply clicking this feature to on will take care of most of your testing needs.  However, this simple algorithm needs to place or replace orders based on what happens during the course of the day.  With daily bars you are sitting on the close of the prior day spouting off orders.  So once the new day starts all of your orders are set.  You can’t see this initially on the surface, because it seems the algorithm is so simple.   Here is another consequence of day bar testing when the intra-day market movement is paramount:

Here the computer is doing exactly what you told it!  Sell short and then take a profit and sell short 25% of the ATR below the open.  Well once the system exited the short it realized it was well below the sell entry point so it immediately goes short at the exact same price (remember TS doesn’t allow stop limit orders).  You told the computer that you wanted to be short if the market moves a certain amount below the open.  These were the orders that were place on yesterday’s close  This may not be exactly what you wanted, right?  You probably wanted to take the profit and then wait for the next day to enter a new trade.  Even if you did want to still be short after the profit level was obtained you wouldn’t want to exit and then reenter at the same price (practically impossible) and be levied a round-turn slip and commission.   You could fiddle around with the code and try to make it work, but I guarantee you that a system like this can only be tested properly on intra-day data.  Let’s drop down to a lower time frame, program the system and see what the real results look like:

Looks very similar to the daily bar chart with Look Inside Bar turned on.  However, it is different.  If you wan’t to gauge a systems potential with a quick program, then go ahead and test on daily bars with LIB turned on.  If it shows promise, then invest the time and program the intra-day version just to validate your results.  What do you mean spend the time?  Can’t you simply turn your chart from daily bars to five minute bars and be done with it.  Unfortunately no!  You have to switch paradigms and this requires quite a bit more programming.  Here is our simple system now in EasyLanguage:

Vars:stb(0),sts(0),atr(0),icnt(0);
Vars:buysToday(0),sellsToday(0),mp(0);

{Use highD() and XXXXD(0) functions to capture the highs, lows, and closes for the past 10 days.
I could have just used a daily bar as data2.
I am looking at five minute bars so we know how the market flows through the day.
}

{This loop kicks out a warning message, but seems to work
Just do this once at the beginning of the day - faster}

{remember true range is either the higher of todays high
Or yesterdays close minus the lower of todays low or
yesterdays close}

{ tradeStation time stamps at the close of the bar so
we capture the opening of the open time plus the bar interval -
in this case 5 minute - so at 1800 + 5 (1805) I capture the open
of the day}

if time = sess1StartTime + barInterval then
begin
Value1 = 0.0;
for icnt = 1 to 10
begin
Value1 = value1 + maxList(closeD(icnt-1),highD(icnt)) - minList(closeD(icnt-1),lowD(icnt));
end;
atr = value1/10.0;
stb = open + .25* atr;
sts = open - .25* atr;
buysToday = 0;
sellsToday = 0;
end;

mp = marketPosition; {The ole mp trick}

If mp = 1 and mp[1] <> 1 then buysToday = buysToday + 1;
If mp =-1 and mp[1] <> -1 then sellsToday = sellsToday + 1;

if buysToday = 0 and time < sess1EndTime and close <= stb then buy next bar at stb stop;
if sellsToday = 0 and time < sess1EndTime and close >= sts then sellshort next bar at sts stop;

setStopLoss(500);
setProfitTarget(1000);
Open Range Break Out Utilizing Five Minute Bars

Here is a validation that Look Inside Bar does work:

This is the trade from June 1st.  Scroll back up to the second chart where LIB is turned on.