In May of 2018 Morgan Stanley presented a report from their research department. The report was written by Tony Small and Matthew Hornbach. The information presented validated the idea of seasonality in the markets. The writers/researchers gathered all the data going back to 1984 on the ten-year futures and all the federal holidays going back over the same span of time. They then analyzed either going long or short M days prior to the holiday and exiting N days before the holidays. They also did the same thing after the holiday – entering M days after and exiting N days after. The boundary was set to five days before and after. This report was sent to me by one of my clients. The conclusion of the article was that there seemed to be a seasonality component to the days prior and after a major holiday. And that it seemed this component did indeed provide a technical advantage (on the surface.) Here is their conclusion.
“Given the risk of data mining, investors should accept this analysis with some degree of caution,” Small and Hornbach said. “There is no guarantee that historical seasonality and past price performance will continue in the future.”
How would one test this idea on their own?
Tools needed:
Python with NumPy installed.
An AI chatbot such as ChatGPT.
Access to TY futures data going back to 1984 (actual non-Panama adjusted data would be preferable.)
A testing platform that allows future leak such as George’s own TradingSimula-18. Excel would work too but would require a bunch of work.
Some programming knowledge.
Create output for all of the federal holidays going back to January 1984. This list must also include Good Fridays. This list should have two fields – the date and the name of the holiday.
TS-18 came in handy because I read the file that was created from step 1 and whenever I came across or spanned a holiday date, I created a loop that calculated the ROC for the prior five days and also for the five days that followed. I created a nomenclature to quickly recognize the ROC span.
B54 – ROC between 5 days before and 4 days before – 1 day
B53 – ROC between 5 days before and 3 days before – 2 days
B52 – ROC between 5 days before and 2 days before – 3 days
B51 – ROC between 5 days before and 1 days before – 4 days
B43 – ROC between 4 days before and 3 days before – 1 day
B42 – ROC between 4 days before and 2 days before – 2 days
B41 – ROC between 4 days before and 1 days before – 3 days
B32 – ROC between 3 days before and 2 days before – 1 day
B31 – ROC between 3 days before and 1 days before – 2 days
B21 – ROC between 2 days before and 1 days before – 1 day
A12 – ROC between 1 days after and 2 days after – 1 day
A13 – ROC between 1 days after and 3 days after – 2 days
A14 – ROC between 1 days after and 4 days after – 3 days
A15 – ROC between 1 days after and 5 days after – 4 days
A23 – ROC between 2 days after and 3 days after – 1 day
A24 – ROC between 2 days after and 4 days after – 2 days
A25 – ROC between 2 days after and 5 days after – 3 days
A34 – ROC between 3 days after and 4 days after – 1 day
A35 – ROC between 3 days after and 5 days after – 2 days
A45 – ROC between 4 days after and 5 days after – 1 day
This process created a file that looked like this:
Output of ROC with nested for loops
With this data I summed up each ROC calculation (B54 thru A45). From this you could pick an entry and exit day combination. The best results were wrapped around Memorial Day. Here was where I ran into a small but highly aggravating scenario. Most of the holidays had similar numbers, but there were a few that had some really strange values. I knew I couldn’t use regular adjusted futures data, because many times these values can go below zero and when doing division with these price adjustments, the values are simply not accurate. So, I used continuous non adjusted data, but ran into a problem with the rollover gap. Some of the holidays occurred really close to rollover dates. The calculations that bridged a rollover was not accurate. I then remembered Pinnacle data also includes Ratio Adjusted Data or RAD. I applied the looping algorithm to this data, and it SEEMED to fix the problem.
I was able to pick a B(n,m) or A(m,n) for each market. If the magnitude of ROC was sufficiently negative, you could use the dates to short the market.
ChatGPT and Python and Numpy to the rescue
I asked ChatGPT to provide python code to generate a list of all federal holidays from 1984.
from pandas.tseries.holiday import USFederalHolidayCalendar from datetime import datetime,timedelta import numpy as np # Create a calendar for U.S. federal holidays cal = USFederalHolidayCalendar()
# Get the holidays for the year 2014 holidays_2014 = cal.holidays(start='1984-01-01', end='2024-12-31').to_pydatetime()
federalHolidays = [] date_list = holidays_2014.astype(datetime).tolist() for n in range(len(date_list)): dateString = f'{date_list[n]:%Y%m%d}' federalHolidays.append(int(dateString))
Create a list of federal holidays
This worked great, but I had to convert the NumPy datetime construct to a string. I like to keep things simple. You will see this in the code. I started to get to work on the second part of the task when I discovered there were no dates for GOOD FRIDAYS – argh! ChatGPT to rescue once again.
def good_friday_dates(start_year, end_year): good_friday_dates = [] for year in range(start_year, end_year + 1): # Calculate the date of Easter Sunday for the given year easter_sunday = calculate_easter_date(year)
# Good Friday is two days before Easter Sunday good_friday = easter_sunday - timedelta(days=2)
# Add the Good Friday date to the list good_friday_dates.append(good_friday)
return good_friday_dates
def calculate_easter_date(year): # Algorithm to calculate the date of Easter Sunday a = year % 19 b = year // 100 c = year % 100 d = b // 4 e = b % 4 f = (b + 8) // 25 g = (b - f + 1) // 3 h = (19 * a + b - d - g + 15) % 30 i = c // 4 k = c % 4 l = (32 + 2 * e + 2 * i - h - k) % 7 m = (a + 11 * h + 22 * l) // 451 month = (h + l - 7 * m + 114) // 31 day = ((h + l - 7 * m + 114) % 31) + 1 easter_date = datetime(year, month, day) return easter_date
# Generate Good Friday dates from 1984 to 2024 good_friday_dates_1984_to_2024 = good_friday_dates(1984, 2024)
Hope this code is right for Good Fridays
Good Friday, as Easter, is all over the map. Tomorrow is Good Friday. The code looks pretty gross, so I just accepted it. It may not be 100% accurate, but if ChatGPT says so…. Once I had a list of all holidays, I was able to start cooking (MLK was added in 1986 and Juneteenth in 2021.) Since Juneteenth is relatively new it did not offer a statistically significant number of observations.
Future Leak comes in Handy.
Every quant needs some software that will look through data and do some calculations. I did this project with EasyLanguage, but it was a little difficult. I had to do a two-pass process (well I did the same with Python too.) I first created an indicator that imported the holiday dates and then outputted the close (or open of next bar) for close[5], close[4], close[3], close[2], close[1] and then waited 1, 2, 3, 4 and 5 bars after the date and printed out those respective closing prices. I did not do this step in EasyLanguage. I actually create EasyLanguage as my output. Here is what it looked like:
Future leak allows you to peak into the future and this comes in super handy when doing research like this. Here is some python code in TS-18 that does all these calculations in one sweep.
# The date of the holiday and the holiday name are stored in lists # Each Holiday has its own date and 20 ROC calculations # These calculations are stored in lists as well tempDate = int(extraData[extraCnt][0]) #first field in the csv file whichHoliday = extraData[extraCnt][1] #second field in the csv file if date[D] == tempDate or (date[D] > tempDate and date[D1] < tempDate): foundJ = 99 for j in range(12): if whichHoliday == listOfHolidayClass[j].holidayName: foundJ = j listOfHolidayClass[foundJ].holidayDates.append(tempDate); for j in range(-5,-1): #Before Holiday for k in range(j+1,0): denom = close[D+j] listOfHolidayClass[foundJ].rocList.append((close[D+k] - close[D+j])/denom) for j in range(0,4): #After Holiday - here is the future leak for k in range(j+1,5): denom = close[D+j] listOfHolidayClass[foundJ].rocList.append((close[D+k] - close[D+j])/denom) extraCnt = extraCnt + 1
Python code using future leak to calculate the 20 ROC from before and after holiday
I then printed this out to a csv file and imported into Excel (see above) and found out the best combination of days for each holiday. Now that I had the B(n,m) or the A(m,n) values, I need a process to tell TS-18 which day to buy/short and exit. Here is some code that, again uses future leak, to print out the five dates before the holiday and the five dates after the holiday.
tempDate = int(extraData[extraCnt][0]) whichHoliday = extraData[extraCnt][1] if date[D] == tempDate or (date[D] > tempDate and date[D1] < tempDate): for j in range(12): if whichHoliday == listOfHolidayClass[j].holidayName: foundJ = j print(date[D],end=',') print(whichHoliday,end=',') for j in range(-5,6): print(date[D+j],end=',') print("",end = '\n') extraCnt = extraCnt + 1
# Here is what the file looked like 19840103,NewYear,19831223,19831227,19831228,19831229,19831230,19840103,19840104,19840105,19840106,19840109,19840110, 19840221,PresidentsDay,19840213,19840214,19840215,19840216,19840217,19840221,19840222,19840223,19840224,19840227,19840228, 19840423,GoodFriday,19840413,19840416,19840417,19840418,19840419,19840423,19840424,19840425,19840426,19840427,19840430, 19840529,MemorialDay,19840521,19840522,19840523,19840524,19840525,19840529,19840530,19840531,19840601,19840604,19840605, 19840705,July4th,19840627,19840628,19840629,19840702,19840703,19840705,19840706,19840709,19840710,19840711,19840712, 19840904,LaborDay,19840827,19840828,19840829,19840830,19840831,19840904,19840905,19840906,19840907,19840910,19840911, 19841008,ColumbusDay,19841001,19841002,19841003,19841004,19841005,19841008,19841009,19841010,19841011,19841012,19841015, 19841112,VeteransDay,19841105,19841106,19841107,19841108,19841109,19841112,19841113,19841114,19841115,19841116,19841119, 19841123,Thanksgiving,19841115,19841116,19841119,19841120,19841121,19841123,19841126,19841127,19841128,19841129,19841130,
Create the dates that occur in the date B4 and after holiday
Now read the data in and pick how many days before to b/s and before to exit or after to b/s and to exit
#structure of holidayDatesData list - all strings # 0.) Holiday Date # 1.) Holiday Name # 2.) 5-Before # 3.) 4-Before # 4.) 3-Before # 5.) 2-Before # 6.) 1-Before # 7.) 0-Before/After # 8.) 1-After # 9.) 2-After # 10.) 3-After # 11.) 4-After # 12.) 5-After #Below I pick 1 day after to buy and 4 days after to exit [8,11] for j in range(numHolidayDates): if holidayDatesData[j][1] == "MemorialDay": holidayBuyDates.append(int(holidayDatesData[j][8])) holidayXitDates.append(int(holidayDatesData[j][11]))
# Okay Let's put in some logic to create a long position if not(long) and canTrade and date[D] == holidayBuyDates[gfCnt]: price = close[D] tradeName = "Hol-A1Buy" posSize = 1 tradeTicket.append([buy,tradeName,posSize,price,mkt]) # Long Exits if long and canTrade and date[D] == holidayXitDates[gfCnt]: price = longExit tradeName = "Hol-A4Xit" tradeTicket.append([exitLong,tradeName,curShares,price,mkt]) gfCnt = gfCnt + 1
Here are the results for the Morgan Stanley analysis in 2018
Best combos for entry and exit for each holiday.
My analysis covered the same period plus all of 2018, 2019, 2020, 2021, 2022, and 2023. My combos were very similar in most cases, but the last few years moved the needle a little bit. However, the results still showed a technical edge.
The Fixed Fractional position sizing scheme is the most popular, so why does it need fixed?
Problems solved with Fixed Fractional:
Efficient usage of trading capital
Trade size normalization between different futures contracts
Trade size normalization across different market environments
These are very good reasons why you should use positions sizing. Problem #2 doesn’t apply if you are only trading one market. This sounds idyllic, right? It solves these two problems, but it introduces a rather bothersome side effect – huge draw down. Well, huge draw down in absolute terms. Draw downs when using a fixed fractional approach are proportional to the prior run up. If you make a ton of money on a prior trade, then your position sizing reflects that big blip in the equity curve. So, if you have a large loser following a large winner, the draw down will be a function of the run up. In most cases, a winning trading system using fixed fractional position sizing will scale profit up as well as draw down. A professional money manager will look at the profit to draw down ratio instead of the absolute draw down value. The efficient use of capital will reflect a smaller position size after a draw down, so that is good right? It is unless you believe in a Martingale betting algorithm – double up on losses and halve winners. Are we just stuck with large “absolute” draw downs when using this size scheme?
Possible solutions to fixing Fixed Fractional (FF)
The first thing you can do is risk less than the industry standard 2% per trade. Using 1% will cause equity to grow at a slower rate and also reduce the inevitable draw down. But this doesn’t really solve the problem as we are giving up the upside. And that might be okay with you. Profit will increase and you are using an algorithm for size normalization. In this blog I am going to propose a trading equity adjustment feature while using FF. What if we act like money managers, and you should even if you are trading your own personal money, and at the end of the year or month we take a little off the table (theoretically – we are not removing funds from the account just from the position sizing calculation) that is if there is any extra on the table. This way we are getting the benefit of FF while removing a portion of the compounding effect, which reduces our allocation for the next time interval. How do you program such a thing? Well first off let’s code up the FF scheme.
Nothing new here. Simply multiply tradingCapital by the riskPerTrade (1 or 2%) and then divide by a formula that defines current and inherent market risk. This is where you can become very creative. You could risk the distance between entry and exit if you know those values ahead of time or you can use a value in terms of the current market. Here I have chosen the 30-day average true range. This value gives a value that predicts the market movement into the future. However, this value only gives the expected market movement for a short period of time into the future. You could us a multiplier since you will probably remain in a trade for more than a few days – that is if you are trying to capture the trend. In my experiment I just use one as my multiplier.
Capture and store the prior year/month NetProfit
When I come up with a trading idea I usually just jump in and program it. I don’t usually take time to see if Easy Language already provides a solution for my problem. Many hours have been used to reinvent the wheel, which isn’t always a bad thing. So, I try to take time and search the functions to see if the wheel already exists. This time it looks like I need to create the wheel. I will show the code first and then explain afterward.
if useAllocateYearly then begin value1 = year(d); value2 = year(d[1]); end;
if useAllocateMonthly then //remember make sure your array is 12XNumYears begin value1 = month(d); value2 = month(d[1]); end;
if useAllocateYearly or useAllocateMonthly then begin if value1 <> value2 then begin if ypIndex > 0 then yearProfit[ypIndex] = prevNetProfit - snapShotNetProfit[ypIndex-1] else yearProfit[ypIndex] = prevNetProfit;
Capture either the prior years or months net profit
I wanted to test the idea of profit retention on a monthly and yearly basis to see if it made a difference. I also wanted to just use the vanilla version of FF. The use of Arrays may not be necessary, but I didn’t know ahead of time. When you program on the fly, which is also called “ad hoc” programming you create first and then refine later. Many times, the “ad hoc” version turns out to be the best approach but may not be the most efficient. Like writing a book, many times your code needs revisions. When applying a study or strategy that uses dates to a chart, you don’t know exactly when the data starts so you always need to assume you are starting in the middle of a year. If you are storing yearly data into an array, make sure you dimension you array sufficiently. You will need 12X the number of years as the size you need to dimension your array if you want to store monthly data.
//250 element array will contain more than 20 years of monthly data //You could increase these if you like just to be safe arrays: yearProfit[250](0),snapShotNetProfit[250](0); //Remember you dimension you arrray variable first and then //Pass it a value that you want to initiate all the values //in the array to equal vars: ypIndex(0);
Dimension and Initiate Your Arrays
The first thing we need to do is capture the beginning of the year or month. We can do this by using the year and month function. If the current month or year value is not the same as the prior day’s month or year value, then we know we crossed the respective timeline boundary. We are using two arrays, snapShotNetProfit and yearProfit (to save time I use this array to store monthlty values as well, when that option is chosen) and a single array index ypIndex. If we have crossed the time boundary, the first thing we need to do is capture the EasyLanguage functionNetProfit’s value. NetProfit keeps track of the cumulative closed out trade profits and losses. Going forward in this description I am going to refer to a yearly reallocation. If it’s the first year, the ypIndex will be zero, and in turn the first year’s profit will be the same as netProfit. We store netProfit in the yearProfit array at the ypIndex location. Since we are in a new year, we take a snapshot of netProfit and store it in the snapShotNetProfit array at the same ypIndex location. You will notice I use the variable prevNetProfit in the code for netProfit. Here is where the devil is in the details. Since we are comparing today’s year value with yesterday’s year value and when they are different, we are already inside the new year, so we need to know yesterday’s netProfit. Before you say it, you can’t pass netProfit a variable for prior values; you know like netProfit(1) or netProfit[1] – this is a function that has no historic values, but you can record the prior day’s value by using our own prevNetProfit variable. Now we need to calculate the tradingCapitalAdjustment. The first thing we do is assign the variable the value held in yearProfit[ypIndex]. We then test the yearProfit[ypIndex] value to see if it is positive. If it is, then we multiply it by (1-removePerProfit). If you want to take 75% of the prior year’s profit off the table, then you would multiply the prior year’s profit by 25%. Let’s say you make $10,000 and you want to remove $7,500, then all you do is multiply $10,000 by 25%. If the prior year’s netProfit is a loss, then this value flows directly through the code to the position sizing calculation (auto deallocation on a losing year). If not, the adjusted profit portion of funds are deallocated in the position sizing equation.
The next time we encounter a new year, then we know this is the second year in our data stream, so we need to subtract last year’s snapshot of netProfit (or prevNetProfit) from the current netProfit. This will give us the change in the yearly net profit. We stuff this information into the yearProfit array. The snapShotNetProfit is stuffed with the current prevNetProfit. ypIndex is incremented every time we encounter a new year. Notice how I increment the ypIndex – it is incremented after all the calculations in the new year. The tradingCapitalAdjustment is then calculated with the latest information.
Here is a table of how the tradingCapital and profit retention adjustment are calculated. A yearly profit adjustment only takes place after a profitable year. A losing year passes without adjustment.
All tests were carried out on the trend friendly crude oil futures with no execution costs from 2006 thru 2/28/2204.
See how money is removed from the allocation model after winning years.
Here are some optimization tests with 75% profit retention on yearly and monthly intervals.
Yearly First-
Yearly retention with no stop loss or break-even levels.
Now Monthly-
Monthly retention with no stop loss or break-even levels.
What if we didn’t reallocate on any specific interval?
Huge drawdowns with very little change in total profit.
Add some trade management into the mix.
Here we optimize a protective stop and a break-even level to see if we can juice the results. All the trade management is on a positionbasis.
200K with No Reallocating with $14k and $8.5 stop/break-even levels [ranked by Profit Factor]No position sizing and no reallocation with $5K and $10K stop/break-even levels
200K and monthly reallocating with $7K and $5.5K stop/break-even levels [BEST PROFIT FACTOR]200K and monthly reallocating with $7K and $8.5K stop/break-even levels [2ND BEST PROFIT FACTOR]
Are we really using our capital in the most efficient manner?
If we retain profit, should we remove some of the loss form the position sizing engine as well. All the tests I performed retained profit from the position size calculations. I let the loss go full bore into the calculation. This is a very risk averse approach. Why don’t I retain 25% of losses and deduct that amount from the yearly loss and feed that into the position sizing engine. This will be less risk averse – let’s see what it does.
Not as good. But I could spend a week working different permutations and optimization sessions.
Are you wondering what Trend Following System I was using as the foundation of this strategy? I used EasyLanguage’s Linear Regression function to create buy and short levels. Here is the very simple code.
//Basically I am buying/shorting on the change of the linear regression slope //Also I have a volatility filter but it really isn't used If value2 >=0 and value2[1] < 0 and avgTrueRange(30)*bigPointValue < 10000 then buy positionSize contracts next bar at market; If value2 <=0 and value2[1] > 0 and avgTrueRange(30)*bigPointValue < 10000 then sellShort positionSize contracts next bar at market;
mp = marketPosition;
//I also have incorporated a 3XATR(30) disaster stop if mp = 1 and c <= entryPrice - 3 * avgTrueRange(30) then sell next bar at market; if mp = -1 and c >= entryPrice + 3 * avgTrueRange(30) then buyToCover next bar at market
If you want to see some more Trend Following models and their codes in EasyLanguage and TS-18 Python check out my TrendFollowing Guide and Kit.
Historical evidence suggests a potential seasonal pattern around the end of the month in the markets.
If you have been involved with the markets for even a short period of time, you have heard about this trade. Buy N days prior to the end of the month and then exit M days after the end of the month. This is a simple test to perform if you have a way to determine the N and the M in the algorithm. You could always buy on the 24th of the month, but the 24th of the month may not equal N days prior to the end of the month.
Simple approach that doesn’t always work – buy the 24th of the month and exit the 5th of the following month.
if dayOfMonth(d) = 24 then buy next bar at open;
if marketPosition = 1 and dayOfMonth(d) = 5 then sell next bar at open;
Before we get into a little better coding of this algorithm, let’s see the numbers. The first graph is trading one contract of the ES futures once a month – no execution fees were applied. The same goes for the US bond futures chart that follows. Before reading further please read this.
CFTC-required risk disclosure for hypothetical results:
Hypothetical performance results have many inherent limitations, some of which are described below. No representation is being made that any account will or is likely to achieve profits or losses similar to those shown. in fact, there are frequently sharp differences between hypothetical performance results and the actual results subsequently achieved by any particular trading program.
One of the limitations of hypothetical performance results is that they are generally prepared with the benefit of hindsight. In addition, hypothetical trading does not involve financial risk, and no hypothetical trading record can completely account for the impact of financial risk in actual trading. For example, the ability to withstand losses or to adhere to a particular trading program in spite of trading losses are material points which can also adversely affect actual trading results. There are numerous other factors related to the markets in general or to the implementation of any specific trading program which cannot be fully accounted for in the preparation of hypothetical performance results and all of which can adversely affect actual trading results.
Buying N days before EOM and Selling M days after EOMDitto!
No Pain No Gain
Looking into the maw of draw down and seeing the jagged and long teeth.
Draw down as a percentage of account value.Ditto2
The bonds had more frequent draw down but not so deep. These teeth can cause a lot of pain.
Well, George what is the N and M?
I should have done M days before and N days after to maintain alphabetic order, but here you go.
ES: N = 6 AND M = 6
US: N =10 AND M = 1
How did you do this?
Some testing platforms have built-in seasonality tools, but, and I could be wrong I didn’t find what I needed in the TradeStation function library. So, I built my own.
A TradingDaysLeftInMonth function had to be created. This function is a broad swipe at attempting to determining this value. It’s not very smart because it doesn’t take HOLIDAYS into consideration. But for a quick analysis it is fine. How does one design such a function? First off, what do we know to help provide information that might be useful? We know how many days are in each month (again this function isn’t smart enough to take into consideration leap years) and we know what day of the week each trading day belongs to. We have this function DayOfWeek(Date) already in EasyLanguage. And we know the DayOfMonth(Date) (built-in too!) With these three tidbits of information, we should be able to come up with a useful function. Not to mention a little programming knowledge. I was working on a Python project when I was thinking of this function, so I decided to prototype it there. No worries, the algorithm can be easily translated to EasyLanguage. And yes, I could have used my concept of a Sandbox to prototype in EasyLanguage as well. Remember a sandbox is a playground where you can quickly test a snippet of code. Using the ONCE keyword, you can quickly throw some generic EasyLanguage together sans trade directives and mate it to a chart and get to the nuts and bolts quickly. I personally like having an indicator and a strategy sandbox. Here is a generic snippet of code where we assume the day of month is the 16th and it is a Monday ( 2 – 1 for Sunday thru 7 for Saturday) and there are 31 days in whatever month.
currentDayOfWeek = 2; currentDayOfMonth = 16; loopDOW = currentDayOfWeek; daysInMonth = 31 #create the calender for the remaining month tdToEOM=0; #total days to EOM for j in range(currentDayOfMonth,daysInMonth+1): if loopDOW != 1 and loopDOW != 7: tdToEOM +=1; print(j," ",loopDOW," ",tdToEOM) loopDOW +=1; if loopDOW > 7: loopDOW = 1; #start back on Monday
Create a synthetic calendar from the current day of month
I just absolutely love the simplicity of Python. When I am prototyping for EasyLanguage, I put a semicolon at the end of each line. Python doesn’t care. Here is the output from this snippet of code.
On Monday 16th there were 12 Trading Days Left In Month Inclusive
Output of Python Snippet - use in EZLang.
I start out with the current day of the month, 16 and loop through the rest of the days of the month. Whenever I encounter a Sunday (1) or a Saturday (7) I do not increment tdToEOM, else I do increment.
Here is how the function works on a chart. Remember in TradeStation I am placing a market order for the NEXT BAR.
Counting the days until the EOM
This snippet of code is the heart of the function, but you must make in generic for any day of any month. Here it is in EasyLanguage – you will see the similarity between the Python snippet and its corresponding EasyLanguage.
{Python prototype tdToEOM=0; for j in range(currentDayOfMonth,daysInMonth+1): if loopDOW != 1 and loopDOW != 7: tdToEOM +=1; print(j," ",loopDOW," ",tdToEOM) loopDOW +=1; if loopDOW > 7: loopDOW = 1; }
loopDOW = curDayOfWeek+1; tdToEOM=0;
for j = curDayOfMonth to monthDays[month(d)] begin if loopDOW <> 1 and loopDOW <> 7 then tdToEOM +=1; // tdToEOM = tdToEOM + 1; loopDOW +=1; if loopDOW > 7 then loopDOW = 1; end; TradingDaysLeftInMonth = tdToEOM;
EasyLanguage Function : TradingDaysLeftInMonth
I used arrays to store the number of days in each month. You might find a better method. Once I get the day of the month and the day of the week I get to work. EasyLanguage uses a 0 for Sunday so to be compliant with the Python function I add a 1 to it. I then loop from the current day of month through monthDays[month(d)]. Remember month(d) returns the month number [1…12]. A perfect index into my array. That is all there is to it. The code is simple, but the concept requires a little thinking. Okay, now that we have the tools for data mining, let’s do some. I did this by creating the following strategy (the same strategy that create the original equity curves.)
if c >= average(c,movingAvgLen) and TDLM = numDaysBeforeEOM then begin buy("Buy B4 EOM") next bar at open; end;
if marketPosition = 1 and barsSinceEntry > 3 then begin if TDIM = numDaysAfterEOM then begin sell("Sell TDOM") next bar at open; end; end; setStopLoss(stopLossAmount); setProfitTarget(profitTargetAmount);
EasyLanguage function driver in form of Strategy
A complete strategy has trade management and an entry and an exit. In this case, I added an additional feature – a trend detector in the form of a longer-term moving average. Let’s see if we can improve the trading system. Thank goodness for Genetic Optimization. Here is the @ES market.
Get your Pick ready to mine!
Smoothed the equity curve – took the big draw down out.
Genetically MODIFIED – Data Mining at its best!
Here are the parameters:
Did not like the moving average. Wide stop and wide profit objective. Days to EOM and after EOM stayed the same.
Bond System:
Bond market results.
If you like this type of programming check out my books at Amazon.com. I have books on Python and of course EasyLanguage. I quickly assembled a YouTube video discussing this post here.
Conclusion – there is something here, no doubt. But it can be a risky proposition. It definitely could provide fodder for the basis of a more complete trading system.
Dollar Cost Averaging Algorithm – Buy X amount every other Monday!
I am not going to get into this controversial approach to trading. But in many cases, you have to do this because of the limitations of your retirement plan. Hey if you get free matching dough, then what can you say.
Check this out!
Buy $1000 worth of shares every other Monday. INTC
if dayOfWeek(d of tomorrow)< dayOfWeek(d) Then begin toggle = not(toggle); if toggle then buy dollarInvestment/close shares next bar at open; end;
Toggle every other Monday ON
Here I use a Boolean typed variable – toggle. Whenever it is the first day of the week, turn the toggle on or off. Its new state becomes the opposite if its old state. On-Off-On-Off – buy whenever the toggle is On or True. See how I determined if it was the first day of the week; whenever tomorrow’s day of the week is less than today’s day of the week, we must be in a new week. Monday = 1 and Friday = 5.
Allow partial liquidation on certain days of the year.
Here I use arrays to set up a few days to liquidate a fractional part of the entire holdings.
value1 = d + 19000000; if sellDates[cnt] = value1 then begin sell sellAmounts[cnt]/close shares total next bar at open; cnt = cnt + 1; end;
Notice the word TOTAL in the order directive.
You can use this as reference on how to declare an array and assign the elements an initial value. Initially, sellDates is an array that contains 20 zeros, and sellAmounts is an array that contains 20 zeros as well. Load these arrays with the dates and the dollar amounts that want to execute a partial liquidation. Be careful with using Easylanguage’s Date. It is in the form YYYMMDD – todays date December 28, 2023, would be represented by 1231228. All you need to do is add 19000000 to Date to get YYYYMMDD format. You could use a function to help out here, but why. When the d + 19000000 equals the first date in the sellDates[1] array, then a market sell order to sell sellAmounts[1]/close shares total is issued. The array index cnt is incremented. Notice the order directive.
sell X sharestotal next bar at market;
If you don’t use the keyword total, then all the shares will be liquidated.
To create a complete equity curve, you will want to liquidate all the shares at some date near the end of the chart. This is used as input as well as the amount of dollars to invest each time.
//Demonstation of Dollar Cost Averaging //Buy $1000 shares every two weeks //Then liquidate a specific $amount on certain days //of the year
value1 = d + 19000000; if sellDates[cnt] = value1 then begin sell sellAmounts[cnt]/close shares total next bar at open; cnt = cnt + 1; end;
if dayOfWeek(d of tomorrow)< dayOfWeek(d) Then begin toggle = not(toggle); if toggle then buy dollarInvestment/close shares next bar at open; end;
if d + 19000000 = settleDate Then sell next bar at open;
A cool looking chart.
A chart with all the entries and exits.
Allow Pyramiding
Turn Pyramding On. You will want to allow up to X entries in the same direction regardless of the order directive.
Working with Data2
I work with many charts that have a minute bar chart as Data1 and a daily bar as Data2. And always forget the difference between:
Close of Data2 and Close[1] of Data2
24 hour regular session used here 1231214 1705 first bar of day - close of data2 4774.00 close[1] of data2 4760.75 1231214 1710 second bar of day - close of data2 4774.00 close[1] of data2 4760.75 1231215 1555 next to last bar of day - close of data2 4774.00 close[1] of data2 4760.75 1231215 1600 last bar of day - close of data2 4768.00 close[1] of data2 4774.00
Up to the last bar of the current trading day the open, high, low, close of data2 will reflect the prior day’s values. On the last bar of the trading day – these values will be updated with today’s values.
Learn how to constrain trading between a Start and End Time – not so “easy-peasy”
Why waste time on this?
Is Time > StartTime and Time <= EndTime then… Right?
This is definitely valid when EndTime > StartTime. But what happens when EndTime < StartTime. Meaning that you start trading yesterday, prior to midnight, and end trading after midnight (today.) Many readers of my blog know I have addressed this issue before and created some simple equations to help facilitate trading around midnight. The lines of code I have presented work most of the time. Remember when the ES used to close at 4:15 and re-open at 4:30 eastern? As of late June 2021, this gap in time has been removed. The ES now trades between 4:15 and 4:30 continuously. I discovered a little bug in my code for this small gap when I was optimizing a “get out time.” I wanted to create a user function that uses the latest session start and end times and build a small database of valid times for the 24-hour markets. Close to 24 hours – most markets will close for an hour. With this small database you can test your time to see if it is a valid time. The construction of this database will require a little TIME math and require the use of arrays and loops. It is a good tutorial. However, it is not perfect. If you optimize time and you want to get out at 4:20 in 2020 on the ES, then you still run into the problem of this time not being valid. This requires a small workaround. Going forward with automated trading, this function might be useful. Most markets trade around the midnight hour – I think meats might be the exception.
Time Based Math
How many 5-minute bars are between 18:00 (prior day) and 17:00 (today)? We can do this in our heads 23 hours X (60 minutes / 5 minutes) or 23 X 12 = 276 bars. But we need to tell the computer how to do this and we also should allow users to use times that include minutes such as 18:55 to 14:25. Here’s the math – btw you may have a simpler approach.
Using startTime of 18:55 and endTime of 14:25.
Calculate the difference in hours and minutes from startTime to midnight and then in terms of minutes only.
timeDiffInHrsMins = 2360 – 1855 = 505 or 5 hours and 5 minutes. We use a little short cut hear. 23 hours and 60 minutes is the same as 2400 or midnight.
timeDiffInMinutes = intPortion(timeDiffInHrsMins/100) * 60 + mod(timeDiffInHrsMins,100). This looks much more complicated than it really is because we are using two helper functions – intPortion and mod:
) intPortion – returns the whole number from a fraction. If we divide 505/100 we get 5.05 and if we truncate the decimal we get 5 hours.
) mod – returns the modulus or remainder from a division operation. I use this function a lot. Mod(505/100) gives 5 minutes.
) Five hours * 60 minutes + Five minutes = 305 minutes.
Calculate the difference in hours and minutes from midnight to endTime and then in terms of minutes only.
timeDiffInHrsMins = endTime – 0 = 1425 or 14 hours and 25 minutes. We don’t need to use our little, short cut here since we are simply subtracting zero. I left the zero in the calculation to denote midnight.
timeDiffInMinutes = timeDiffInMinutes + intPortion(timeDiffInHrsMins/100) * 60 + mod(timeDiffInHrsMins,100). This is the same calculation as before, but we are adding the result to the number of minutes derived from the startTime to midnight.
) intPortion – returns the whole number from a fraction. If we divide 1425/100, we get 14.05 and if we truncate the decimal, we get 14.
) mod – returns the modulus or remainder from a division operation. I use this function a lot. Mod(1425/100) gives 25.
) 14* 60 + 25 = 865 minutes.
) Now add 305 minutes to 865. This gives us a total of 1165 minutes between the start and end times.
Now divide the timeDiffInMinutes by the barInterval. This gives 1165 minutes/5 minutes or 233 five-minute bars.
Build Database of all potential time stamps between start and end time
We now have all the ingredients to build are simple array-based database. Don’t let the word array scare you away. Follow the logic and you will see how easy it is to use them. First, we will create the database of all the time stamps between the regular session start and end times of the data on the chart. We will use the same time-based math (and a little more) to create this benchmark database. Check out the following code.
// You could use static arrays // reserve enough room for 24 hours of minute bars // 24 * 60 = 1440 // arrays: theoTimes[1440](0),validTimes[1440](0); // syntax - arrayName[size](0) - the zero sets all elements to zero // this seems like over kill because we don't know what // bar interval or time span the user will be using
// these arrays are dynamic // we dimension or reserve space for just what we need arrays: theoTimes[](0),validTimes[](0);
// Create a database of all times stamps that potentiall could // occur
// Now set the dimension of the array by using the following // function and the number of bars we calculated for the entire // regular session Array_setmaxindex(theoTimes,numBarsInCompleteSession); // Load the array from start time to end time // We know the start time and we know the number of X-min bars // loop from 1 to numBarsInCompleteSession and // use timeSum as the each and every time stamp // To get to the end of our journey we must use Time Based Math again. timeSum = startTime; for arrayIndex = 1 to numBarsInCompleteSession Begin timeSum = timeSum + barInterval; if mod(timeSum,100) = 60 Then timeSum = timeSum - 60 + 100; // 1860 - becomes 1900 if timeSum = 2400 Then timeSum = 0; // 2400 becomes 0000 theoTimes[arrayIndex] = timeSum;
print(d," theo time",arrayIndex," ",theoTimes[arrayIndex]); end;
Create a dynamic array with all possible time stamps
This is a simple looping mechanism that continually adds the barInterval to timeSum until numBarsInCompleteSession are exhausted. Reade about the difference between static and dynamic arrays in the code, please. Here’s how it works with a session start time of 1800:
theoTimes[01] = 1800 + 5 = 1805 theoTimes[02] = 1805 + 5 = 1810 theoTimes[04] = 1810 + 5 = 1815 theoTimes[05] = 1815 + 5 = 1820 theoTimes[06] = 1820 + 5 = 1830 ... //whoops - need more time based math 1860 is not valid theoTimes[12] = 1855 + 5 = 1860
Insert bar stamps into our theoTimes array
More time-based math
Our loop hit a snag when we came up with 1860 as a valid time. We all know that 1860 is really 1900. We need to intervene when this occurs. All we need to do is use our modulus function again to extract the minutes from our time.
If mod(timeSum,100) = 60 then timeSum = timeSum – 60 + 100. Her we remove the sixty minutes from the time and add an hour to it.
1860 – 60 + 100 = 1900 // a valid time stamp
That should fix everything right? What about this:
2400 is okay in Military Time but not in TradeStation
This is a simple fix with. All we need to do is check to see if timeSum = 2400 and if so, just simply reset to zero.
Build a database on our custom time frame.
Basically, do the same thing, but use the user’s choice of start and end times.
//calculate the number of barInterval bars in the //user defined session numBarsInSession = timeDiffInMinutes/barInterval;
Array_setmaxindex(validTimes,numBarsInSession);
startTimeStamp = calcTime(startTime,barInterval);
timeSum = startTime; for arrayIndex = 1 to numBarsInSession Begin timeSum = timeSum + barInterval; if mod(timeSum,100) = 60 Then timeSum = timeSum - 60 + 100; if timeSum = 2400 Then timeSum = 0; validTimes[arrayIndex] = timeSum; // print(d," valid times ",arrayIndex," ",validTimes[arrayIndex]," ",numBarsInSession); end;
Create another database using the time frame chose by the user
Don’t allow weird times!
Good programmers don’t allow extraneous values to bomb their functions. TRY and CATCH the erroneous input before proceeding. If we have a database of all possible time stamps, shouldn’t we use it to validate the user entry? Of course, we should.
//Are the users startTime and endTime valid //bar time stamps? Loop through all the times //and validate the times.
for arrayIndex = 1 to numBarsInCompleteSession begin if startTimeStamp = theoTimes[arrayIndex] then validStartTime = True; if endTime = theoTimes[arrayIndex] Then validEndTime = True; end;
Validate user's input.
Once we determine if both time inputs are valid, then we can determine if the any bar’s time stamp during a back-test is a valid time.
if validStartTime = false or validEndTime = false Then error = True;
//Okay to check for bar time stamps against our //database - only go through the loop until we //validate the time - break out when time is found //in database. CanTradeThisTime is the name of the function. //It returns either True or False
if error = False Then Begin for arrayIndex = 1 to numBarsInSession Begin if t = validTimes[arrayIndex] Then begin CanTradeThisTime = True; break; end; end; end;
This portion of the code is executed on every bar of the back-test.
Once and only Once!
The code that creates the theoretical and user defined time stamp database is only done on the very first bar of the chart. Also, the validation of the user’s input in only done once as well. This is accomplished by encasing this code inside a Once – begin – end.
Now this code will test any time stamp against the current regular session. If you run a test prior to June 2021, you will get a theoretical database that includes a 4:20, 4:25, and 4:30 on the ES futures. However, in actuality these bar stamps did not exist in the data. This might cause a problem when working with a start or end time prior to June 2021, that falls in this range.
Function Name: CanTradeThisTime
Complete code:
// Function to determine if time is in acceptable // set of times inputs: startTime(numericSimple),endTime(numericSimple);
if startTime <= endTime Then Begin timeDiffInHrsMins = (intPortion(endTime/100) - 1)*100 + mod(endTime,100) + 60 - startTime; timeDiffInMinutes = intPortion(timeDiffInHrsMins/100) * 60 + mod(timeDiffInHrsMins,100); end;
numBarsInSession = timeDiffInMinutes/barInterval;
Array_setmaxindex(validTimes,numBarsInSession);
startTimeStamp = calcTime(startTime,barInterval);
timeSum = startTime; for arrayIndex = 1 to numBarsInSession Begin timeSum = timeSum + barInterval; if mod(timeSum,100) = 60 Then timeSum = timeSum - 60 + 100; if timeSum = 2400 Then timeSum = 0; validTimes[arrayIndex] = timeSum; print(d," valid times ",arrayIndex," ",validTimes[arrayIndex]," ",numBarsInSession); end; for arrayIndex = 1 to numBarsInCompleteSession begin if startTimeStamp = theoTimes[arrayIndex] then validStartTime = True; if endTime = theoTimes[arrayIndex] Then validEndTime = True; end; end;
if validStartTime = False or validEndTime = false Then error = True;
if error = False Then Begin for arrayIndex = 1 to numBarsInSession Begin if t = validTimes[arrayIndex] Then begin CanTradeThisTime = True; break; end; end; end;
Complete CanTradeThisTime function code
Sandbox Strategy function driver
inputs: startTime(1800),endTime(1500);
if canTradeThisTime(startTime,endTime) Then if d = 1231206 or d = 1231207 then print(d," ",t," can trade this time");
I hope you find this useful. Remember to purchase by Easing into EasyLanguage books at amazon.com. The DayTrade edition is still on sale. Email me with any question or suggestions or bugs or anything else.
Easing Into EasyLanguage-DayTrade Edition [On SALE Now thru November]
Get Your Copy Now! On sale now for thru the end of November!
EZ-DT Pyramania is a strategy I introduced in the Day Trade Edition. The logic is rather simple – pyramid as the market moves through multiple levels during the trading day. – buy, buy, buy, dump or buy, dump, short, short, short, dump. The distance between the levels is constant. In the book, I showed an algorithm with a total of 6 levels with 7 edges.
Pyramania of @ES.D
Here the market opens @ 9:30 and the levels are instantly plotted and trades are executed as the market moves through the levels located above the open tick. Over the weekend, I had a reader ask how he could modify the code to plot the levels on the 24-hour @ES session. In the day session, I used the change in the date as the trigger for the calculation and plotting of the levels. Here is the day session version.
inputs:numSegments(6),numPlots(6);
arrays: segmentBounds[](0);
variables: j(0),loopCnt(0),segmentSize(0),avgRng(0); once Begin Array_SetMaxIndex(segmentBounds, numSegments); end;
if d <> d[1] Then // only works on the @ES.D or any .D session begin avgRng = average(range of data2,20); segmentSize = avgRng/numSegments; loopCnt = -1*numSegments/2; for j = 0 to numSegments begin segmentBounds[j] = openD(0) + loopCnt * segmentSize; loopCnt = loopCnt + 1; end; end;
//The following time constraint only works when all time stamps //are less than the end of day time stamp //This will not work when time = 1800 and endTime = 1700 if t < calcTime(sessionEndTime(0,1),-barInterval) Then begin if numPlots >= 1 then plot1(segmentBounds[0],"Level 0"); if numPlots >= 2 then plot2(segmentBounds[1],"Level 1"); if numPlots >= 3 then plot3(segmentBounds[2],"Level 2"); if numPlots >= 4 then plot4(segmentBounds[3],"Level 3"); if numPlots >= 5 then plot5(segmentBounds[4],"Level 4"); if numPlots >= 6 then plot6(segmentBounds[5],"Level 5"); if numPlots >= 7 then plot7(segmentBounds[6],"Level 6"); // plot8(segmentBounds[7],"Level 7"); // plot9(segmentBounds[8],"Level 8"); end;
Works great with @ES.D or any @**.D
I like this code because it exposes you to arrays, loops, and plotting multiple values. You can fix this by modifying and adding some code. I used the Trading Around Midnight blog post to get the code I needed to enable plotting around 0:00 hours. Here is the updated code:
once Begin Array_SetMaxIndex(segmentBounds, numSegments); end;
startTime = sessionStartTime(0,1); endTime = sessionEndTime(0,1); //let TS tell you when the market opens - remember the //first time stamp is the open time + bar interval if t = calcTime(sessionStartTime(0,1),barInterval) Then begin avgRng = average(range of data2,20); segmentSize = avgRng/numSegments; loopCnt = -1*numSegments/2; for j = 0 to numSegments begin segmentBounds[j] = open + loopCnt * segmentSize; loopCnt = loopCnt + 1; end; end;
// if startTime > endTime then you know you are dealing with // timees that more than likely bridges midnight // if time is greater then 1700 (end time) then you must // subtract an offset so it makes sense - endTimeOffset // play with the math and it will come to you if startTime > endTime then begin endTimeOffset = 0; if t >= startTime+barInterval and t<= 2359 then endTimeOffSet = 2400-endTime; end; if t-endTimeOffSet < endTime Then begin if numPlots >= 1 then plot1(segmentBounds[0],"Level 0"); if numPlots >= 2 then plot2(segmentBounds[1],"Level 1"); if numPlots >= 3 then plot3(segmentBounds[2],"Level 2"); if numPlots >= 4 then plot4(segmentBounds[3],"Level 3"); if numPlots >= 5 then plot5(segmentBounds[4],"Level 4"); if numPlots >= 6 then plot6(segmentBounds[5],"Level 5"); if numPlots >= 7 then plot7(segmentBounds[6],"Level 6"); // plot8(segmentBounds[7],"Level 7"); // plot9(segmentBounds[8],"Level 8"); end;
Modification to plot data around midnight
Here I let TS tell me with the market opens and then use some simple math to make sure I can plot with the time is greater than and less than the end of day time.
Plots from 18:05 through midnight until 17:00 the next day.
Email me if you have the book and want to companion code to the strategy – georgeppruitt@gmail.com
Have You Ever Wondered If You Just Reversed the Logic?
You have been there before. What you thought was a great trading idea turns out to be a big flop. We have all developed these types of algorithms. Then it hits you, just flip the logic and in turn the equity curve. Hold your horses! First off you have to make sure it’s not just the execution costs that is hammering the equity curve into oblivion. When testing a fresh trading idea, it is best to keep execution costs to zero. This way if your idea is a good one, but is simply backward, then you have a chance of creating something good out of something bad. I was playing around with a mean reversion day trading algorithm (on the @ES.D – day session of the mini S&P 500) that created the following equity curve. Remember to read the disclaimer concerning hypothetical performance before proceeding reading the rest of this blog. It is located under the DISCLAIMER – REAMDE! tab. By reading the rest of this blog post it implies that you understand the limitations of hypothetical back testing and simulated analysis.
The pandemic created a strong mean reversion environment. In the initial stage of this research, I did not set the executions costs – they defaulted to zero. My idea was to buy below the open after the market moved down from the high of the day a certain percentage of price. Since I was going to be buying as the market was moving down, I was willing to use a wide stop to see if I could hold on to the falling knife. Short entries were just the opposite -sell short above the open after the market rallied a certain percentage of price. I wanted to enter on a hiccup. Once the market moved down a certain range from the high of the day, I wanted to enter on a stop at the high of the prior bar. I figured if the price penetrated the high of the prior five-minute bar in a down move, then it would signal an eventual rotation in the market. Again, I was just throwing pasta against the wall to see what would stick. I even came up with a really neat name for the algorithm the Rubber Band system – stretch just far enough and the market is bound to slam back. Well, there wasn’t any pasta sticking. Or was there? If I flipped the equity curve 180 degrees, then I would have a darned good strategy. All it would take is to reverse the signals, sell short when I was buying and buy when I was selling short. Instead of a mean reversion scheme, this would turn into a momentum-based strategy.
if c < todaysOpen and todaysOpen-c = maxOpenMinusClose and (maxCloseMinusOpen + maxOpenMinusClose)/c >= stretchPercent Then canBuy = True; if c > todaysOpen and c- todaysOpen = maxCloseMinusOpen and (maxCloseMinusOpen + maxOpenMinusClose)/c >= stretchPercent Then canShort = True;
Guts of the complete failure.
Here I measure the maximum distance from the highest close above the open and the lowest close below the open. The distance between the two points is the range between the highest and lowest closing price of the current day. If the close is less than today’s open, and the range between the extremes of the highest close and lowest close of the trading day is greater than stretchPercent, then an order directive to buy the next bar at the current bar’s high is issued. The order is alive until it is filled, or the day expires. Selling short uses the same calculations but requires the close of the current bar to be above the open. The stretchPercent was set to 1 percent and the protective stop was set to a wide $2,000. As you can see from the equity curve, this plan did not work except for the time span of the pandemic. Could you optimize the strategy and make it a winning system. Definitely. But the 1 percent and $2000 stop seemed very logical to me. Since we are comparing the range of the data to a fixed price of the data, then we don’t need to worry about the continuous contract distortion. Maybe we would have to, if the market price was straddling zero. Anyways, here is a strategy using the same entry technique, but reversed, with some intelligent trade filtering. I figured a profit objective might be beneficial, because the stop was hit several times during the original test.
$2K was hit often!Using some trade filtering and stop loss and profit objective on the reversal of the original strategy.
If you like the following code, make sure you check out my books at Amazon.com. This type of code is used the Hi-Res and Day-Trading editions of the Easing_Into_Easylanguage series.
if c < todaysOpen and todaysOpen-c = maxOpenMinusClose and (maxCloseMinusOpen + maxOpenMinusClose)/c >= stretchPercent Then canShort = True; if c > todaysOpen and c- todaysOpen = maxCloseMinusOpen and (maxCloseMinusOpen + maxOpenMinusClose)/c >= stretchPercent Then canBuy = True;
if canTrade and t >= calcTime(dontTradeBefore,dontTradeBeforeOffset) and t < calcTime(dontTradeAfter,dontTradeAfterOffset) and t < sessionEndTime(0,1) Then begin if shortsToday = 0 and canShort = True Then sellshort next bar at l stop; if buysToday = 0 and canBuy = True Then buy next bar at h stop; end;
Trade filtering was obtained by limiting the duration during the trading day that a trade could take place. It’s usually wise to wait a few minutes after the open and a few minutes prior to the close to issue trade directives. Also, range compression of the prior day seems to help in many cases. Or at least not range expansion. I only allow one long entry or one short or both during the trading day – two entries only! Read the code and let me know if you have any questions. This is a good framework for other areas of research. Limiting entries using the mp variable is a neat technique that you can use elsewhere.
And as always let me know if you see any bugs in the code. Like Donnie Knuth says, “Beware of bugs in the above code; I have only proved it correct, not tried it!”
To Rollover or not to Rollover – that is the Question!
In actuality that is only a portion of what we need to ask ourselves when dealing with rollovers and data. As you probably already know, continuous futures contracts are created by adjusting historic data in an attempt to eliminate the gap between the new and old contract. If you don’t adjust your data, then the gap causes erroneous disruptions in your historic back testing. A trade that bridges a rollover does not accurately reflect what really happened. So, we must use adjusted data. Two problems arise from using back adjusted data:
The date of the rollover can have a large impact on algorithm performance.
In real time trading, rollovers must take place, and this increases execution costs. If you trade a longer-term strategy, you may need to execute 12 additional spreads (exit old – enter new) in crude oil annually. If you levy a $50 charge (slip and commission) for each turn, then that is $600 in additional costs annually.
Several options can be used to trigger when to rollout of the expiring contract and roll into the new one. Some traders use consecutive days of higher volume and open interest in the new contract in relation to the expiring one to begin the process. This mechanism is also popular in creating continuous contract data. Other traders, and I was one of them, chose a particular trading day of the month based on the expiring date of the contract. Many futures contracts expire after a certain number of days have passed in the expiring month or the prior month of expiration. This mechanism, n-days into the expiring month, can also be used when creating continuous contracts.
Using TradeStation’s Default @CL
Using Wilder’s Parabolic SAR Algorithm
$50 levied for execution costs
Using Pinnacle Data’s Adjusted Data for CL
$50 levied for execution costs
The two equity curves are similar. However, TradeStation’s data produced nearly $60K in profit, whereas Pinnacle only produced north of $20K. I discovered in the early part of TradeStation’s @CL data, the values were derived strictly from the electronic markets. Up until about 2008 or so, I think the pits were still more active. Take a look at this graphic.
Nov-Dec 2004 – see how the pit data looks more valid
Pinnacle Data knits the pit session data with the purely electronic data during the transition period. However, this only accounts for the early trades in the back test. The parabolic indicator uses its prior output to calculate the next value. Any disruption in the differences in the relationships of historic highs and lows of the data will perpetuate through the output of subsequent parabolic calculations. Things start to work themselves out later in the back test. Indicators like the Parabolic are more sensitive to data differences.
What if you wanted to trade the Parabolic SAR real time?
This can be done, but results may look different than your back test. In trading crude oil, you should trade all 12 contracts. And you will need to execute roll spreads once a month, if you are in a position. This doesn’t answer the question on how to calculate the Parabolic once a rollover takes place. You have two options, 1.) you can use what Futures Truth coined as overlap data to calculate the new indicator reading. Basically, we would load the new contract data going back as far as we could and derive the indicator values off of the new data. This would work just fine with crude data, but many futures do not have a sufficient amount of overlap. In other words, the new contract would have very little history. And this data wasn’t sufficient to calculate accurate indicator values. 2.) Roll into the new contract and adjust the historical data of the old contracts by the discount between the two contracts that were rolled out of and into. This would alleviate the problems with the futures contract with very little history.
I like the second option best, because it works for all markets and even in crude oil you may not have overlap data to do the calculation. Even if you have enough data, it might not be all that good. If you can go back 100 days in the new contract, the data may deteriorate before you get those 100 days.
Data starts deteriorating well before 100 days back.
Wouldn’t it be great if we could test with this type of data? You know rollover and then adjust all the historic data that we need for our indicators using good data from the old contracts, but adjusted in terms of our new data? Well, we are in luck, I spent several hours doing this with my Python TS-18 software. I used Pinnacle Data‘s actual contract data – you get this when you buy their CLC database and derived the discount between the contracts. Pinnacle rolls at the same time each month in crude – it’s not a calendar date but a number of days into the prior expiration month. Take a look at this trade-by-trade listing.
Trade-by-trade with rollovers.
This shows rolling out of the old contract and into the new contract, and then backing up 100 days of old contract data and adjusting the data by the discounts as they occur. This could involve more than three adjustments historically. This rollover inspired data was then fed into the parabolic indicator. The indicator was restarted on each rollover going back N days. In other words, when a rollover was observed, the indicator was reset with the new rollover adjusted historic data by using a loop. The rollover mechanism called the parabolic calculation N times going back N days into our dynamically adjusted data. Once the loop finished, the new contract data was then fed into the calculation on day at a time. This occurred until another rollover was observed.
I would show the results of this testing but…
I got very diverse results when I recalculated the parabolic on each rollover with different loop counts. On the rollover, if I looped back 40 days to bring the parabolic up to speed, I got very similar results to TradeStation’s results using Pinnacle data. What the heck I will show that to you.
TS-18 Results with rollovers
The results are similar but the draw down of late concerns me. However, if I only go back 30 days to synchronize the parabolic on each rollover, the results are much better. This tells me that the parabolic is very sensitive to the data that is used for its calculations.
This post may have raised more questions than it answered. But I will try to get down to the differences between the different look back lengths on rollover and report any findings back here. However, you must keep rollovers in mind when dealing with trading futures, period.
Daily Bar Daytrade to 5 Minute Bars — Not So Easy!
Like in the previous post, I designed a simple optimization framework trying to determine the best day of the week to apply a long onlyvolatility based open range break out algorithm on natural gas. You might first ask why natural gas? Its a good market that has seasonal tendencies that can actually be day-traded – there is enough volatility and volume. Anyway, by simply slapping setExitOnClose at the end of your code turns your algorithm into one that exits every day at the settlement (fictional value that is derived via formula) price. You can always use LIB (look inside bar) to help determine the magnitude of intraday swings and there respective chronological order. But you are still getting out at the settlement price – which isn’t accurate. However, if you are trying to get a set of parameters that gets you into the ballpark, then this is a very acceptable approach. But you will always want to turn you daily bar strategy with LIB intro an intraday version for automation and accuracy purposes.
Day of Week Analysis On Intraday Data
When you optimize the day of week on daily bars, the first bar of the week is usually Monday. When testing with intraday bars on futures, the trading actually starts on Sunday evening. If your daily bar analysis reveals the best day of week is Wedneday, then you must inform TradeStation to take the trade from the opening on Tuesday night through the close on Wenesday. If you are executing a market order, then you just need to tell TradeStation to execute at Tuesday night’s open (1800 eastern standard time). The TRICK to allow trading throughout the 24 hour session (1800 to 1700) on just a particular day of the week is to capture the day of the week on the first bar of the trading session. Here is some bookkeeping you can take care of on the first bar of the trading session.
if t = sessionstartTime(0,1) + barInterval Then Begin
todaysOpen = o; openDOW = dayOfWeek(d); canBuy = False; if openDOW = daysOfWeekToTrade then canBuy = True; if canBuy and d[1] <> date of data2 then begin canBuy = False; print(d," turning off canBuy"); end;
if mp = 1 then curTradeDays = curTradeDays + 1; if mp = 0 then curTradeDays = 0; barsToday = 1; end;
Bookkeeping on the first bar of trading session
Here, I check to see if the first bar’s time stamp is equal to the sessionStarttime + barInterval. Remember TradeStation shows the close time of each bar as the time stamp. If adding a barInterval to the opening time results in a non-time, then use the calcTime function ( 1800 + 60 = 1860 – a non-time.) I first store the open of the session in my variable todaysOpen. Why not just use openD(0). Well openD(0) is great for sessions that don’t span midnight. If they span midnight then the openD returns the open of the12:00 am bar. Here is the output from August 9th with the open on August 8th at 1800. The two other values are midnight on August 8th and August 7th. So the openD, highD, lowD, and closeD functions involve the 24 hour clock and not the session.
On the first bar of the trading session I also capture the day of the week. The days of the week, using intraday data on a 24 hour session, range from Sunday to Thursday, instead of Monday thru Friday. CanBuy is turned on if the day of week equals the input provided by the user. When using daily bars and you optimize the day of the week, remember if you chose 2 or Tuesday you really mean Wednesday. If today is Tuesday, then buy next bar at… This will generate trades only on Wednesdays. When testing on intraday data you do not need to make a modification for the day of week. If the first bar of the trading session is Tuesday, then you will actually trade what is considered the Wednesday session that starts on Tuesday evening at 18:00.
What About Holidays?
Here is a quick trick to not trade on Holidays. The daily bar does not include preempted sessions in its database, whereas intraday data does. So, if at 18:00 the date of the prior bar does not match the date of the daily bar located in the Data2 slot, then you know that the prior day was preempted and you should not trade today. In other words, Data2 is missing. Remember, you only want to trade on days that are included in the daily bar database in your attempt to replicate the daily bar strategy.
date close moving average
1230703.00 data2 2.57382 2.70300
1230704.00 missing data2 2.57382 2.70300 <-- no trade
1230705.00 data2 2.57485 2.65100 <-- matches
How to count the daily bars since entry.
Here again you need to do some bookkeeping. If you are long on the first bar of the day, then you have been in the trade for at least a day. If you are long on two first bars of the trading session, then you have been long for two days. I use the variable, curTradeDays, to count the number of days we have been in the trade. If on the first bar of the trading session we are flat, then I reset curTradeDays to zero. Otherwise, I increment the variable and this only occurs once a day. You can optimize the daysInTrade input, but for this post we are just interested in getting out at the close on the day of entry.
Controlling one entry per day.
We only want one long entry per day in an attempt to replicate the daily bar system. There are several ways to do this, but comparing the current bar’s marketPosition to the prior bar’s value will inform you if a position has been transitioned from flat to long. If the prior bar’s position if flat and the current bar is now long, then we can turn our canBuy off or to false. If we get stopped out later and the market rallies again to our break out level, we will not execute at that level, because the trade directive will not be issued. Why not use the entriesToday function? Again midnight cause this function to reset. Say you go long at 22:00 and you do not want to enter another position until the tomorrow at 18:00. EntriesToday will forget you entered the long position at 22:00.
Cannot execute on the first bar of the day – does it matter?
Since this is an open range break out, we need to know what the open is prior to our break out calculation. Could the break out level be penetrated on the first 5 minute bar? Sure, but I bet it is rare. I tested this and it only occurred 4 or 5 times. In a back-test, if this occurs and the subsequent bar’s open is still above the break out level, TradeStation will convert the stop order to a market order and you will buy the open of the 2nd five minute bar. In real time trading, you must tell TradeStation to wait until the end of the first 5 minute bar before issuing the trade directive to replicate your back testing. If this is an issue, you can test with 5 minute bars, but execute on 1 minute bars. In other words, have your testing chart and an additional execution chart. Skipping the first 5 minute bar may be advantageous. Many times you will get a surge at the open and then the market falls back. By the time the first 5 minute bar concludes the market may be trading back below your break out level. You might bypass the bulge. But your still in the game. Here is how you can test the significance of this event.
if canBuy and h > todaysOpen + volAmt *volMult then Begin print(d," first bar penetration ",value67," ",value68," ",(c - (todaysOpen + volAmt *volMult))*bigPointValue ); value67 = value67 + 1; value68 = value68 + (c - (todaysOpen + volAmt *volMult))*bigPointValue; end;
Testing for penetration on first 5 minute bar of the day
By the end of the first bar of the day we know the open, high, low and close of the day thus far. We can test to see if the high would have penetrated the break out level and also calculate the profit or loss of the trade. In a back-test you will not miss this trade if the next bar’s open is still above the break out level. If the next bar’s open is below the break out level, then you may have missed a fake out break out. Again this is a rare event.
if t = sessionstartTime(0,1) + barInterval Then Begin todaysOpen = o; openDOW = dayOfWeek(d); canBuy = False; if openDOW = daysOfWeekToTrade then canBuy = True; if canBuy and d[1] <> date of data2 then begin canBuy = False; end;
if mp = 1 then curTradeDays =curTradeDays + 1; if mp = 0 then curTradeDays = 0; barsToday = 1;
volAmt = average(range of data2,volLen); movAvg = average(c,mavLen) of data2; if canBuy and h > todaysOpen + volAmt *volMult then Begin print(d," first bar penetration ",value67," ",value68," ",(c - (todaysOpen + volAmt *volMult))*bigPointValue ); value67 = value67 + 1; value68 = value68 + (c - (todaysOpen + volAmt *volMult))*bigPointValue; end; end;
if mp[1] = 0 and mp = 1 and canBuy then canBuy = False;
if canBuy and t <> sessionEndTime(0,1) Then Begin if barsToday > 1 and close of data2 > movAvg and range of data2 > volAmt * volComp then buy("DMIntraBuy") next bar at todaysOpen + volAmt *volMult stop; end; barsToday = barsToday + 1;
if daysInTrade = 0 then setExitOnClose;
sell("LLxit") next bar at lowest(l of data2,llLookBack) stop; if mp = 1 and daysInTrade > 0 then begin if curTradeDays = daysInTrade then sell("DaysXit") next bar at open; end;
Intrday system that replicates a daily bar day trader
We are using the setStopLoss and setProfitTarget functions in this code. But remember, if your continuous contract goes negative, then these functions will not work properly.
This type of programming is tricky, because you must use tricks to get EasyLanguage to do what you want it to do. You must experiment, debug, and program ideas to test the limitations of EasyLanguage to hone your craft.
When Mining for Data, Make Sure You Have Accurate Data
Take a look at these results with no execution costs.
Results of Data Mining on Natural Gas
These results were derived by applying a simple algorithm to the natural gas futures for the past 15 or so years. I wanted to see if there was a day of the week that produced the best results with the following entry and exit techniques:
Long entries only
Open range break out using a fraction of the N-day average range.
Buy in the direction of the moving average.
Yesterdays close must be above the X-day moving average of closing prices.
Yesterday must have a wider range than a fractional multiple of the average range.
A fixed $stop and $profit used to exit trades.
Other exits
Exit at low of prior day.
Exit at the close of today – so a day trade.
Here is a list of the parameters that can be optimized:
daysOfWeekToTrade(1) : 1 – Monday, 2 – Tuesday…
mavLen(20): moving average calculation length.
volLen(10): moving average calculation length for range.
volMult(0.25): average range multiplier to determine break out/
volComp(.5): yesterday’s range must be greater than this percentage.
stopLoss(500): stop loss in $
profitTarget(1000): profit objective in $
daysInTrade(0): 0 get out at end of day.
llLookBack(2): pattern derived stop value – use the lowest low of N-days or the stop loss in $, whichever is closer.
if dayOfWeek(d) = daysOfWeekToTrade Then Begin if close > average(c,mavLen) and range > average(range,volLen)*volComp then buy("DMDailyBuy") next bar at open of tomorrow + average(range,volLen)*volMult stop; end;
if daysInTrade = 0 then setExitOnClose; sell("LLxit") next bar at lowest(l,llLookBack) stop; if marketPosition = 1 then begin if barsSinceEntry = daysInTrade then sell("DaysXit") next bar at open; end;
Why Is This Algorithm Wrong? It’s Not It’s the Data!
The algorithm, speaking in terms of logic, is accurate. The data is wrong. You cannot test the exit technology of this algorithm with just four data points per day – open, high, low, and close. If this simple algorithm cannot be tested, then what can you test on a daily bar basis accurately?
Long or short entry, but not both.
If the algorithm can enter both long and short, you need to know what occurred first. Doesn’t matter if you enter via stop or limit orders. Using a stop order for both, then you need to know if the high occurred first and got you long, and then later the low and got you short. Or just the opposite. You can test, with the benefit of hindsight, if only one of the orders would have been elected. If only one is elected, then you can proceed. If both then no-go. You must be able to use future leak to determine if only one of the orders were fillable. TS-18 allows future leak BTW. I say that like it’s a good thing!
L or S position from either a market or stop, and a profit objective.
If a long position is entered above the open, via a stop, and then the market moves even higher, you can get out on a sell limit for a profit.
Same goes for the short side, but you need to know the magnitude of the low price in relation to the open.
L or S position from either a market or limit and a protective stop.
If short position is entered above the open, via a limit order, and the market moves even higher, you can exit the short position at a loss via a buy stop order.
Same goes for the long side, but you need to know the magnitude of the high in relation to the open.
Long pyramid on multiple higher stop or lower limit orders, but not both.
The market opens and then sells off. You can go long on a limit order below the open, then you go long another unit below the original limit price and so on…
The market opens and rallies. You can go long on a stop order above the open, then you can go long another unit above the original stop price and so on…
Short pyramid on multiple lower stop or higher limit orders.
Same as long entries but in opposite direction.
Can’t you look at the relationships of the open to low and open to high and close to high and close to low and determine which occurred first, the high or the low of the day. You can, but there is no guarantee. And you can’t determine the magnitude of intraday day swings. Assume the market opens and immediately moves down and gets you short via a stop order. And from looking at a daily chart, it looks like the market never turned around, so you would assume you had a profit going into the close. But in fact, after the market opened and moved down, it rallied back to the open enough to trigger a protective stop you had working.
The entry technique of this strategy is perfectly fine. It is only buying in the direction of a breakout. The problems arise when you apply a profit objective and a stop loss and a pattern-based stop and a market on close order. After buying did the market pull back to get you stopped out before moving up to get you out at profit objective? Did the market move up and then down to the prior lowest low of N days and then back up to get you long? Or just the opposite? In futures, the settlement price is calculated with a formula. You are always exiting at the settlement price with an MOC or setExitOnClose directive (daily bar basis.)
No Biggie – I Will Just Test with LIB (Look Inside Bar. That should fix it, right?
It definitely will increase accuracy, because you can see the intraday movements that make up the daily bar. So, your entries and exits should be executed more accurately. But you are still getting out at the settlement price which does not exist. Here are the results from using LIB with 5-minute bar resolution.
You saw the beauty now you see the beast.
We Know the Weakness, But What Can We Do?
Test on a 5-minute bar as Data1 and daily as Data2. This concept is what my Hi-Res Edition of Easing Into EasyLanguage is all about. Here the results of using a higher resolution data and exiting on the last tick of the trading day – a known quantity.
As accurate as you can get – well with 5 minute bars that is.
These results validate the LIB results, right? Not 100%, but very close. Perhaps this run makes more money because the settlement price on the particular days that the system entered a trade was perhaps lower than the last tick. In other words, when exiting at the end of the day, the last tick was more often higher than the settlement price.
In my next post, I will go over the details of developing an intraday system to replicate (as close as possible) a simple daily bar-based day trading system. Like the one we have here.
Backtesting with [Trade Station,Python,AmiBroker, Excel]. Intended for informational and educational purposes only!
Get All Five Books in the Easing Into EasyLanguage Series - The Trend Following Edition is now Available!
Announcement – A Trend Following edition has been added to my Easing into EasyLanguage Series! This edition will be the fifth and final installment and will utilize concepts discussed in the Foundation editions. I will pay respect to the legends of Trend Following by replicating the essence of their algorithms. Learn about the most prominent form of algorithmic trading. But get geared up for it by reading the first four editions in the series now. Get your favorite QUANT the books they need!
The Foundation Edition. The first in the series.
This series includes five editions that covers the full spectrum of the EasyLanguage programming language. Fully compliant with TradeStation and mostly compliant with MultiCharts. Start out with the Foundation Edition. It is designed for the new user of EasyLanguage or for those you would like to have a refresher course. There are 13 tutorials ranging from creating Strategies to PaintBars. Learn how to create your own functions or apply stops and profit objectives. Ever wanted to know how to find an inside day that is also a Narrow Range 7 (NR7?) Now you can, and the best part is you get over 4 HOURS OF VIDEO INSTRUCTION – one for each tutorial.
Hi-Res Edition Cover
This book is ideal for those who have completed the Foundation Edition or have some experience with EasyLanguage, especially if you’re ready to take your programming skills to the next level. The Hi-Res Edition is designed for programmers who want to build intraday trading systems, incorporating trade management techniques like profit targets and stop losses. This edition bridges the gap between daily and intraday bar programming, making it easier to handle challenges like tracking the sequence of high and low prices within the trading day. Plus, enjoy 5 hours of video instruction to guide you through each tutorial.
Advanced Topics Cover
The Advanced Topics Edition delves into essential programming concepts within EasyLanguage, offering a focused approach to complex topics. This book covers arrays and fixed-length buffers, including methods for element management, extraction, and sorting. Explore finite state machines using the switch-case construct, text graphic manipulation to retrieve precise X and Y coordinates, and gain insights into seasonality with the Ruggiero/Barna Universal Seasonal and Sheldon Knight Seasonal methods. Additionally, learn to build EasyLanguage projects, integrate fundamental data like Commitment of Traders, and create multi-timeframe indicators for comprehensive analysis.
Get Day Trading Edition Today!
The Day Trading Edition complements the other books in the series, diving into the popular approach of day trading, where overnight risk is avoided (though daytime risk still applies!). Programming on high-resolution data, such as five- or one-minute bars, can be challenging, and this book provides guidance without claiming to be a “Holy Grail.” It’s not for ultra-high-frequency trading but rather for those interested in techniques like volatility-based breakouts, pyramiding, scaling out, and zone-based trading. Ideal for readers of the Foundation and Hi-Res editions or those with EasyLanguage experience, this book offers insights into algorithms that shaped the day trading industry.
Trend Following Cover.
For thirty-one years as the Director of Research at Futures Truth Magazine, I had the privilege of collaborating with renowned experts in technical analysis, including Fitschen, Stuckey, Ruggiero, Fox, and Waite. I gained invaluable insights as I watched their trend-following methods reach impressive peaks, face sharp declines, and ultimately rebound. From late 2014 to early 2020, I witnessed a dramatic downturn across the trend-following industry. Iconic systems like Aberration, CatScan, Andromeda, and Super Turtle—once thriving on robust trends of the 1990s through early 2010s—began to falter long before the pandemic. Since 2020 we have seen the familiar trends return. Get six hours of video instruction with this edition.
Pick up your copies today – e-Book or paperback format – at Amazon.com
