Tag Archives: EasyLanguage

A Quant’s ToolBox: Beautiful Soup, Python, Excel and EasyLanguage

Many Times It Takes Multiple Tools to Get the Job Done

Just like a mechanic, a Quant needs tools to accomplish many programming tasks.  In this post, I use a toolbox to construct an EasyLanguage function that will test a date and determine if it is considered a Holiday in the eyes of the NYSE.

Why a Holiday Function?

TradeStation will pump holiday data into a chart and then later go back and take it out of the database.  Many times the data will only be removed from the daily database, but still persist in the intraday database.  Many mechanical day traders don’t want to trade on a shortened holiday session or use the data for indicator/signal calculations.  Here is an example of a gold chart reflecting President’s Day data in the intra-day data and not in the daily.

Holiday Data Throws A Monkey Wrench Into the Works

This affects many stock index day traders.  Especially if automation is turned on.  At the end of this post I provide a link to my youTube channel for a complete tutorial on the use of these tools to accomplish this task.  It goes along with this post.

First Get The Data

I searched the web for a list of historical holiday dates and came across this:

Historic List of Holidays and Their Dates

You might be able to find this in a easier to use format, but this was perfect for this post.

Extract Data with Beautiful Soup

Here is where Python and the plethora of its libraries come in handy.  I used pip to install the requests and the bs4 libraries.  If this sounds like Latin to you drop me an email and I will shoot you some instructions on how to install these libraries.  If you have Python, then you have the download/install tool known as pip.

Here is the Python code.  Don’t worry it is quite short.

# Created:     24/02/2020
# Copyright: (c) George 2020
# Licence: <your licence>
#-------------------------------------------------------------------------------

import requests
from bs4 import BeautifulSoup

url = 'http://www.market-holidays.com/'
page = requests.get(url)
soup = BeautifulSoup(page.text,'html.parser')
print(soup.title.text)
all_tables = soup.findAll('table')
#print (all_tables)
print (len(all_tables))
#print (all_tables[0])
print("***")
a = list()
b = list()
c = list()
#print(all_tables[0].find_all('tr')[0].text)
for numTables in range(len(all_tables)-1):
for rows in all_tables[numTables].find_all('tr'):
a.append(rows.find_all('td')[0].text)
b.append(rows.find_all('td')[1].text)

for j in range(len(a)-1):
print(a[j],"-",b[j])
Using Beautiful Soup to Extract Table Data

As you can see this is very simple code.  First I set the variable url to the website where the holidays are located.  I Googled on how to do this – another cool thing about Python – tons of users.  I pulled the data from the website and stuffed it into the page object.  The page object has several attributes (properties) and one of them  is a text representation of the entire page.  I pass this text to the BeautifulSoup library and inform it to parse it with the html.parser.  In other words, prepare to extract certain values based on html tags.  All_tables contains all of the tables that were parsed from the text file using Soup.  Don’t worry how this works, as its not important, just use it as a tool.  In my younger days as a programmer I would have delved into how this works, but it wouldn’t be worth the time because I just need the data to carry out my objective; this is one of the reasons classically trained programmers never pick up the object concept.  Now that I have all the tables in a list I can loop through each row in each table.  It looked liker there were 9 rows and 2 columns in the different sections of the website, but I didn’t know for sure so I just let the library figure this out for me.  So I played around with the code and found out that the first two columns of the table contained the name of the holiday and the date of the holiday.  So, I simply stuffed the text values of these columns in two lists:  a and b.  Finally I print out the contents of the two lists, separated by a hyphen, into the Interpreter window.  At this point I could simply carry on with Python and create the EasyLanguage statements and fill in the data I need.  But I wanted to play around with Excel in case readers didn’t want to go the Python route.  I could have used a powerful editor such as NotePad++ to extract the data from the website in place of Python.  GREP could have done this.  GREP is an editor tool to find and replace expressions in a text file.

Use Excel to Create Actual EasyLanguage – Really!

I created a new spreadsheet.  I used Excel, but you could use any spreadsheet software.   I first created a prototype of the code I would need to encapsulate the data into array structures.  Here is what I want the code to look like:

Arrays: holidayName[300](""),holidayDate[300](0);

holidayName[1]="New Year's Day "; holidayDate[1]=19900101;
Code Prototype

This is just the first few lines of the function prototype.  But you can notice a repetitive pattern.  The array names stay the same – the only values that change are the array elements and the array indices.  Computers love repetitiveness.  I can use this information a build a spreadsheet – take a look.

Type EasyLanguage Into the Columns and Fill Down!

I haven’t copied the data that I got out of Python just yet.  That will be step 2.  Column A has the first array name holidayName (notice I put the left square [ bracket in the column as well).  Column B will contain the array index and this is a formula.  Column C contains ]=”.  Column D will contain the actual holiday name and Column E contains theThese columns will build the holidayName array.  Columns G throuh K will build the holidayDates array.    Notice column  H  equals column B.  So whatever we do to column B (Index) will be reflected in Column H (Index).  So we have basically put all the parts of the EasyLanguage into  Columns A thru K. 

Excel provides tools for manipulating strings and text.  I will use the Concat function to build my EasyLanguageBut before I can use Concat all the stuff I want to string together must be in a string or text format.  The only column in the first five that is not a string is Column B.  So the first thing I have to do is convert it to text.  First copy the column and paste special as values.  Then go to your Data Tab and select Text To Columns. 

Text To Columns

It will ask if fixed width or delimited – I don’t think it matters which you pick.  On step 3 select text.

Text To Columns – A Powerful Tool

The Text To Columns button will solve 90% of your formatting issues in Excel.    Once you do this you will notice the numbers will be left justified – this signifies a text format.  Now lets select another sheet in the workbook and past the holiday data.

Copy Holiday Data Into Another Spreadsheet

New Year's Day - January 1, 2021
Martin Luther King, Jr. Day - January 18, 2021
Washington's Birthday (Presidents' Day) - February 15, 2021
Good Friday - April 2, 2021
Memorial Day - May 31, 2021
Independence Day - July 5, 2021
Labor Day - September 6, 2021
Thanksgiving - November 25, 2021
Christmas - December 24, 2021
New Year's Day - January 1, 2020
Martin Luther King, Jr. Day - January 20, 2020
Washington's Birthday (Presidents' Day) - February 17, 2020
Good Friday - April 10, 2020
Memorial Day - May 25, 2020
Holiday Output

 

Data Is In Column A

Text To Columns to the rescue.  Here I will separate the data with the “-” as delimiter and tell Excel to import the second column in Date format as MDY.  

Text To Columns with “-” as the delimiter and MDY as Column B Format

Now once the data is split accordingly into two columns with the correct format – we need to convert the date column into a string.

Convert Date to a String

Now the last couple of steps are really easy.  Once you have converted the date to a string, copy Column A and past into Column D from the first spreadsheet.  Since this is text, you can simply copy and then paste.  Now go back to Sheet 2 and copy Column C and paste special [values] in Column J on Sheet 1.  All we need to do now is concatenate the strings in Columns A thru E for the EasyLanguage for the holidayName array.  Columns G thru K will be concatenated for the holidayDate array.  Take a look.

Concatenate all the strings to create the EasyLanguage

Now create a function in the EasyLanguage editor and name it IsHoliday and have it return a boolean value.  Then all you need to do is copy/paste Columns F and L and the data from the website will now be available for you use.   Here is a portion of the function code.  Notice I declare the holidayNameStr as a stringRef?  I did this so I could change the variable in the function and pass it back to the calling routine.

Inputs : testDate(numericSeries),holidayNameStr(stringRef);

Arrays: holidayName[300](""),holidayDate[300](0);

holidayNameStr = "";

holidayName[1]="New Year's Day "; holidayDate[1]=19900101;
holidayName[2]="Martin Luther King, Jr. Day "; holidayDate[2]=19900115;
holidayName[3]="Washington's Birthday (Presidents' Day) "; holidayDate[3]=19900219;
holidayName[4]="Good Friday "; holidayDate[4]=19900413;
holidayName[5]="Memorial Day "; holidayDate[5]=19900528;
holidayName[6]="Independence Day "; holidayDate[6]=19900704;
holidayName[7]="Labor Day "; holidayDate[7]=19900903;
holidayName[8]="Thanksgiving "; holidayDate[8]=19901122;
holidayName[9]="New Year's Day "; holidayDate[9]=19910101;
holidayName[10]="Martin Luther King, Jr. Day "; holidayDate[10]=19910121;
holidayName[11]="Washington's Birthday (Presidents' Day) "; holidayDate[11]=19910218;

// There are 287 holiays in the database.
// Here is the looping mechanism to compare the data that is passed
// to the database

vars: j(0);
IsHoliday = False;
For j=1 to 287
Begin
If testDate = holidayDate[j] - 19000000 then
Begin
holidayNameStr = holidayName[j] + " " + numToStr(holidayDate[j],0);
IsHoliday = True;
end;
end;
A Snippet Of The Function - Including Header and Looping Mechanism

This was a pretty long tutorial and might be difficult to follow along.  If you want to watch my video, then go to this link.

I created this post to demonstrate the need to have several tools at your disposal if you really want to become a Quant programmer.  How you use those tools is up to you.  Also you will be able to take bits and pieces out of this post and use in other ways to get the data you really need.  I could have skipped the entire Excel portion of the post and just did everything in Python.  But I know a lot of Quants that just love spreadsheets.  You have to continually hone your craft in this business.   And you can’t let one software application limit your creativity.  If you have a problem always be on the lookout for alternative platforms and/or languages to help you solve it.

 

 

How to Keep Track of BuysToday and SellsToday

The Useful MP

We all know how to use the reserved word/function MarketPosition – right?  Brief summary if not – use MarketPosition to see what your current position is: -1 for short, +1 for long and 0 for flat.  MarketPosition acts like a function because you can index it to see what you position was prior to the current position – all you need to do is pass a parameter for the number of positions ago.  If you pass it a one (MarketPosition(1)) then it will return the your prior position.  If you define a variable such as MP you can store each bars MarketPosition and this can come in handy.

mp = marketPosition;

If mp[1] <> 1 and mp = 1 then buysToday = buysToday + 1;
If mp[1] <> -1 and mp = -1 then sellsToday = sellsToday + 1;
Keeping Track of Buy and Sell Entries on Daily Basis

The code compares prior bar’s MP value with the current bar’s.   If there is a change in the value, then the current market position has changed.   Going from not 1 to 1 indicates a new long position.  Going from not -1 to -1 implies a new short.  If the criteria is met, then the buysToday or sellsToday counters are incremented.  If you want to keep the number of buys or sells to a certain level, let’s say once or twice,  you can incorporate this into your code.

If  time >= startTradeTime and t < endTradeTime and 
buysToday < 1 and
rsi(c,rsiLen) crosses above rsiBuyVal then buy this bar on close;
If time >= startTradeTime and t < endTradeTime and
sellsToday < 1 and
rsi(c,rsiLen) crosses below rsiShortVal then sellShort this bar on close;
Using MP to Keep Track of BuysToday and SellsToday

This logic will work most of the time, but it depends on the robustness of the builtin MarketPosition function Look how this logic fails in the following chart:

I didn't want entries in the same direction per day!
I only wanted 1 short entry per day!

MarketPosition Failure

Failure in the sense that the algorithm shorted twice in the same day.  Notice on the first trade how the profit objective was hit on the very next bar.  The problem with MarketPosition is that it only updates at the end of the bar one bar after the entry.  So MarketPosition stays 0 during the duration of this trade.  If MarketPosition doesn’t change then my counter won’t work.  TradeStation should update MarketPosition at the end of the entry bar.  Alas it doesn’t work this way.  I figured a way around it though.  I will push the code out and explain it later in more detail.

Input: rsiLen(14),rsiBuyVal(30),rsiShortVal(70),profitObj$(250),protStop$(300),startTradeTime(940),endTradeTime(1430);

Vars: mp(0),buysToday(0),sellsToday(0),startOfDayNetProfit(0);

If d <> d[1] then
Begin
buysToday = 0;
sellsToday = 0;
startOfDayNetProfit = netProfit;
end;

{mp = marketPosition;

If mp[1] <> 1 and mp = 1 then buysToday = buysToday + 1;
If mp[1] <> -1 and mp = -1 then sellsToday = sellsToday + 1;}

If entriesToday(date) > buysToday + sellsToday then
Begin
If marketPosition = 1 then buysToday = buysToday + 1;
If marketPosition =-1 then sellsToday = sellsToday + 1;
If marketPosition = 0 then
Begin
if netProfit > startOfDayNetProfit then
begin
if exitPrice(1) > entryPrice(1) then buysToday = buysToday + 1;
If exitPrice(1) < entryPrice(1) then sellsToday = sellsToday + 1;
end;;
if netProfit < startOfDayNetProfit then
Begin
if exitPrice(1) < entryPrice(1) then buysToday = buysToday + 1;
If exitPrice(1) > entryPrice(1) then sellsToday = sellsToday + 1;
end;
end;
print(d," ",t," ",buysToday," ",sellsToday);
end;

If time >= startTradeTime and t < endTradeTime and
buysToday < 1 and
rsi(c,rsiLen) crosses above rsiBuyVal then buy this bar on close;
If time >= startTradeTime and t < endTradeTime and
sellsToday < 1 and
rsi(c,rsiLen) crosses below rsiShortVal then sellShort this bar on close;

SetProfittarget(profitObj$);
SetStopLoss(protStop$);

SetExitOnClose;
A Better Buy and Short Entries Counter

TradeStation does update EntriesToday at the end of the bar so you can use this keyword/function to help keep count of the different type of entries.  If MP is 0 and EntriesToday increments then you know an entry and an exit has occurred (takes care of the MarketPosition snafu) – all you need to do is determine if the entry was a buy or a sell.  NetProfit is also updated when a trade is closed.   I establish the StartOfDayNetProfit on the first bar of the day (line 9 in the code) and then examine EntriesToday and if NetProfit increased or decreased.  EntryPrice and ExitPrice are also updated at the end of the bar so I can also use them to extract the information I need.   Since MarketPosition is 0  I have to pass 1 to the EntryPrice and ExitPrice functions – prior position’s prices.  From there I can determine if a Long/Short entry occurred.  This seems like a lot of work for what you get out of it, but if you are controlling risk by limiting the number of trades (exposure) then an accurate count is so very important.

An alternative is to test on a higher resolution of data – say 1 minute bars.  In doing this you give a buffer to the MarketPosition function – more bars to catch up.

 

Pyramiding and then Scaling Out at Different Price Levels – EasyLanguage

TOTAL, TOTAL, TOTAL – an important keyword

I just learned something new!  I guess I never programmed a strategy that pyramided at different price levels and scaled out at different price levels.

Initially I thought no problem.  But I couldn’t get it to work – I tried everything and then I came across the keyword Total and then I remembered.  If you don’t specify Total in you exit directives then the entire position is liquidated.  Unless you are putting all your positions on at one time – like I did in my last post.   So remember if you are scaling out of a pyramid position use Total in your logic.

vars: maxPosSize(2);

If currentContracts < maxPosSize - 1 and c > average(c,50) and c = lowest(c,3) then buy("L3Close") 1 contract this bar on close;
If currentContracts < maxPosSize and c > average(c,50) and c = lowest(c,4) then buy("L4Close") 1 contract this bar on close;


If currentContracts = 2 and c = highest(c,5) then sell 1 contract total this bar on close;
If currentContracts = 1 and c = highest(c,10) then sell 1 contract total this bar on close;
Scaling Out Of Pyramid

Why you have to use the Total I don’t know.  You specify the number of contracts in the directive and that is sufficient if you aren’t pyramiding.  The pyramiding throws a “monkey wrench” in to the works.

Multiple Ouput function in EasyLanguage

In the Pascal programming language you have Procedures and Functions.  Procedures are used when you want to modify multiple variables within a sub-program.  A function is a sub-program that returns a single value after it has been modified by say a formula.  EasyLanguage combines procedures and functions into one sub-program called a function.  Functions and procedures both have a formal parameter definition –  a list that describes the type of parameters that are being received by the calling program.  In Pascal procedures, you pass the address of the value that you want changed.  By modifying the contents of the address you can pass the value back and forth or in and out of the procedure.  In functions you pass by value.   Remember the parameter in a normal function call is used to instruct something within the body of the function and is not altered (e.g. the number 19 in value1 = average(c,19)).  This value doesn’t need to be modified it’s just used.  Look at the following code:

Here I am modifying mav1, mav2 and mav3 within the function and then passing the values back to the calling strategy/indicator/paintbar.  All functions must return a value so I simply assign the value 1 to the function name.  The key here is the keyword numericRef, once I change the values located in the addresses of mav1, mav2 and mav3 (address are provided by the keyword numericRef), they will be made available to the calling program.  This code allows the function to return more than just one value.

A Slightly More Eloquent Approach to Programming Our Pyramiding E-Mini DayTrading Algorithm.

Okay let’s see how I was able to add some eloquence to the brute force approach to this pyramiding algorithm.  The original code included multiple entry directives and a ton of hard coded numerical values.   So let me show you how I was able to refine the logic/code and in doing so make it much more flexible.  We might lose a little bit of the readability, but we can compensate by using extra commentary.

First off, let’s add flexibility by employing input variables.  In this case, we need to inform the algorithm the distance from the open to add additional positions and the max number of entries allowed for the day.

inputs : pyramidDistance(5),maxDailyEntries(3);

Now we need to set somethings up for the first bar of the day.  Comparing the date of today with the date of yesterday is a good way to do this.

if d<>d[1] then 
begin
canSell = true;
sellMult = 1;
sellStop = -999999;
entries = 0;
end;
First bar of the day housekeeping.

Here is a neat way to keep track of the number of entries as they occur throughout the trading day.  Remember the function EntriesToday(date) will not provide the information we need.

mp = marketPosition * currentShares;

if mp[1] <> mp and mp <> 0 then entries = entries + 1;
How to track the number of entries for today.

If the last bar’s mp[1] is not equal to the current bar’s mp then and mp is not equal to zero then we know we have added on another entry.  Okay now let’s think about eliminating the “brute force” approach.

Instead of placing multiple order entry directives I  only want to use one with a variable stop level.  This stop level will be guided by the variable SellMult.  We start the day with a wacky sell stop level and then calculate it based on the SellMult variable and PyramidDistance input.

if low <= sellStop  then
begin
sellMult = sellMult + 1;
end;

sellStop = openD(0) - sellMult * pyramidDistance;
Calculate and adapt sell stop level as we go along.

So on the first bar of the day the sellStop = openD(0) – sellMult * pyramidDistance or sellStop = openD(0) – 1 * 5.  Or 5 handles below the open.  Note you an change the pyramidDistance input and make it three to match the previous examples.

if entries = maxDailyEntries then canSell = false;
if time < sess1EndTime and canSell then sellShort 1 contract next bar at sellStop stop;
if mp <=-1 {and barsSinceEntry > 0} then buyToCover next bar at sellStop + 2* pyramidDistance stop;

setexitonclose;
That's it! Pretty simple isn't it?

Ok, we need to tell the computer to turn off the ability to place orders if one of two things happens:  1) we have reached the maxDailyEntries or 2) time >= sess1EndTime.    You could make the time to stop entering trades an input as well.  If neither criteria applies then place an order to sellShort at our sellStop level.   If price goes below our sell stop level then we know we have been filled and the new sellStop level needs to be recalculated.  See how we use a calculation to adapt the stop level with a single order placement directive?  This is where the eloquence comes into play.  QED.

Now you code the opposite side and then see if you can make money  (hypothetically speaking of course) with it.  If you think about it, why does this not work.  And the not so obvious reason is that it trades too much.  Other than trading too much it makes perfect sense – buy or sell by taking a nibbles at the market.  If the market takes off then take a big bite.  The execution costs of the nibbles are just way too great.  So we need to think of a filtering process to determine when it is either better to buy or sell or when to trade at all.  Good Luck with this ES [emini S&P ]day trading algorithm!

inputs : pyramidDistance(5),maxDailyEntries(3);
vars: mp(0),icnt(0),sellStop(0),sellMult(0),canSell(true),entries(0);

if d<>d[1] then
begin
canSell = true;
sellMult = 1;
sellStop = -999999;
entries = 0;
end;

mp = marketPosition * currentShares;

if mp[1] <> mp and mp <> 0 then entries = entries + 1;
if mp[1] = -1 and mp[0] = 0 then canSell = false;
if time > 1430 then canSell = false;

if low <= sellStop then
begin
sellMult = sellMult + 1;
end;

sellStop = openD(0) - sellMult * pyramidDistance;
if entries = maxDailyEntries then canSell = false;
if time < sess1EndTime and canSell then sellShort 1 contract next bar at sellStop stop;
if mp <=-1 {and barsSinceEntry > 0} then buyToCover next bar at sellStop + 2* pyramidDistance stop;

setexitonclose;
Much More Flexible Code

EasyLanguage Code for Pyramiding a Day-Trading System w/video [PART-2]

 

Check out the latest video on Pyramiding.

Here is the finalized tutorial on building the pyramiding ES-day-trade system that was presented in the last post.

I will admit this video should be half as long as the end result.  I get a bit long-winded.  However, I think there are some good pointers that should save you some time when programming a similar system.

EasyLanguage Source:

Here is the final code from the video:

vars: mp(0),lastTradePrice(0),canSell(true);

mp = marketPosition * currentContracts;

if date[0] <> date[1] then
begin
canSell = true; // canSell on every day
end;

if mp = -1 then canSell = false; // one trade on - no more
if time > 1430 then canSell = false; //no entries afte 230 central

if mp = 0 and canSell = true then sellShort next bar at OpenD(0) - 3 stop;

if mp = -1 then sellShort next bar at OpenD(0) - 6 stop; //add 1
if mp = -2 then sellShort next bar at OpenD(0) - 9 stop; //add 2

if mp = -1 then lastTradePrice = OpenD(0) - 3; //keep track of entryPrice
if mp = -2 then lastTradePrice = OpenD(0) - 6;
if mp = -3 then lastTradePrice = OpenD(0) - 9;


if mp <> 0 then buyToCover next bar at lastTradePrice + 3 stop; // 3 handle risk on last trade

// next line provides a threshold prior to engaging trailing stop
if mp = -3 and barsSinceEntry > 0 and lowD(0) < lastTradePrice - 3 then buyToCover next bar at lowD(0) + 3 stop;

setExitOnClose;
EasyLanguage for Pyramiding and Day-Trading ES

What we learned here:

  • can’t use entriesToday(date) to determine last entry price
  • must use logic to not issue an order to execute on the first bar of the next day
  • mp = marketPosition * currentContracts is powerful stuff!

In the next few days, I will publish the long side version of this code and also a more eloquent approach to the programming that will allow for future modifications and flexibility.

Let me know how it works out for you.

Take this code and add some filters to prevent trading every day or a filter to only allow long entries!

Hash Table in EasyLanguage [Part 1]

 

This concept may be considered advanced and only used by pure programmers, but that is not the case at all.  A Hash Table is simply a table that is indexed by a function.  The function acts like the post office – it sends the data to the correct slot in the table.  I utilized this data structure because  I wanted to know the closing prices for the past fifteen years for the “1stThuJan” (first Thursday of January.)  This, of course, would require some programming and I could simply store the values in an array.  However, what if I wanted to know the closing prices for the “3rdFriMar?”  I would have to spend more time and re-code, right?   What if I changed my mind again.  Instead, as we programmers often do, I wanted to be able to pull the data for any instance of “Week, Day Of Week, Month.”  This is where a table structure comes in handy.  With this table, I can query it and find out the average yearly closing prices for the “1stMonSep” or the “4thFriJuly” or the “3rdWedApr ” on a rolling year by year basis.  Why would you want this you might ask?  Would it be helpful to know the price  change from the “2ndMonMar” to the subsequent “2ndMonMar” on a rolling basis?  What if the average price change is 10%.  You could use this information to make sure you always buy on this particular day.  That is if you believe in this form of analysis.

Here’s how I created a table that stores the closing prices for the past 15 years for each entry in the table.  Remember each row value only comes up once a year.  You only have one “1stMonJan” in a calendar year.  So the first part of the problem was simple, create a table that can store the closing prices with all the different combinations like the “1stTueJan” for the past fifteen years.  The second part of creating the post office like function that places the correct closing price in the right row was a little more difficult, but not much.  Here’s how I did it.

As I said earlier, a Hash Table is a very simple concept and very useful as well!  For some of you out there, I just want to make sure that you know that I am not talking about a device to keep your cannabis off of the floor;-) All kidding aside, go ahead and take a look at the table below.  Notice how it stores the closing prices of all the possible occurrences of Week, Day Of Week, Month.   Column 1 is the key or Hash Index value.  You will need this key to unlock the data for that particular row.  Column 2 shows the number of years that the data was collected.  Column 3 and on are the closing prices for that particular day across the years.  Once you have the data collected you can do anything your heart desires with it.

Table Index Num. Years Close 1 Close 2 Close 3 Close 4 Close 5 Close 6
1stWedJan 6 603 496 450.25 589 612.75 684.5
1stThuJan 6 606.5 486.25 446 571.75 597.75 683
1stFriJan 6 607.25 491.75 451.5 564.75 597.75 674
1stMonJan 6 606.75 490.75 447 590.25 606.25 679.25
1stTueJan 6 619.25 507 447.25 578.25 612.75 682.5
2ndWedJan 6 617.75 446 412.5 600.75 605.75 688
2ndThuJan 6 615.5 444.75 409.5 612.25 565.75 692.5
2ndFriJan 6 635.5 490.25 400 618.5 553.75 702.5
2ndMonJan 6 652.5 460.25 451 576.75 574.25 717.75

Sounds cool – so let’s do it!

Step 1:  Calculate the size of the table.

Each month consists of 4.25 weeks (52/12).  Because of this, you can have up to five occurrences of any given day of the week inside of a month – five Mondays, Tuesdays, etc.,  So we must build the table big enough to handle five complete weeks for each month.  Since there are 5 days in a week and 5 weeks in a month (not really but plan on it)  and 12 months in a year, the table must contain at least 300 rows ( 5 X 5 X 12.)  Since we don’t know how many years of data that we might want to collect we could make the arrays dynamic, but I want to keep things simple so I will reserve space for 100 years.  Overkill?

Step 2:  Use measurements from Step 1 to construct the container and create an addressing function.

The container is easy just dimension a 2-d array.  A 2-d array is a table whereas a 1-d array is a list.  A spreadsheet is an example of a 2-d array.  Just make the table big enough to hold the data.  Remember the key component to the Hash Table is not what it can hold, but the ability to quickly reference the data.  Just like your home, we need to create a unique address for each of the three hundred rows so the right mail, er data can be delivered or stored.  This is really quite simple –  we know we need a distinctive address and we know we need 300 of them.  Like the table above we can create a unique address in the form of “1stMonJan.”  This is a nine character string.  This  string can easily represent the 300 different addresses.  We start with “1stMonJan” and end with “5thFriDec.”  These values most consist of only nine characters.  I could have done the same thing using an integer value to represent each address.  “1stMonJan” could also be represented with 10101.  The “3rdFriDec” would be 30512.  I liked the string approach because the addresses are instantly recognizable with little or no translation.  So we need to get to typing, right?  Always remember if you are doing something redundant a computer can do the chore and do it quicker.  Just a quick note here.  I  designed the table ahead of time with the values in column 1 already filled in.  I could have done it more dynamically, but creating a data structure and filling in as much information before can save time on the programming side.

Instead of typing each unique address into the table, let’s let the computer do it for us.  Remember, Easylanguage has some cool string manipulation tools and with a little bit of cleverness, you can create the 300 unique addresses in one fell swoop.  The following code creates an array (list) of all of the possible combinations of “Week, Day Of Week, Month.”  There are 100 lines of code here, don’t freak out!  It’s mostly redundant.  I used a Finite State Machine and Easylanguage’s Switch – Case programming structure.  So you are learning about Hash Tables, Hash Indices, Finite State Machines, and Switch-Case programming in one post.  And here, all you want is a winning trading system.  Well, they are hard to come by and you need as many tools at your disposal to unlock the Holy Grail.  This is just one way to come up with the address values.

{Developed and programmed by George Pruitt-copyright 2017 www.georgepruitt.com}
{Just provide credit if you reuse! Or buy my book ;-)}

Input: hashIndex[n](stringArrayRef);
Vars: done(false);
Vars: firstCount(0),secondCount(0),thirdCount(0),fourthCount(0),fifthCount(0);
Vars: state(1),arrCnt(0),tempStr(""),monthCnt(0),returnValString(""),iCnt(0),jCnt(0),numBucket(0);
array: dayString[5](""),monString[12]("");


dayString[1] = "Mon";
dayString[2] = "Tue";
dayString[3] = "Wed";
dayString[4] = "Thu";
dayString[5] = "Fri";

monString[1] = "Jan";
monString[2] = "Feb";
monString[3] = "Mar";
monString[4] = "Apr";
monString[5] = "May";
monString[6] = "Jun";
monString[7] = "Jul";
monString[8] = "Aug";
monString[9] = "Sep";
monString[10] = "Oct";
monString[11] = "Nov";
monString[12] = "Dec";


arrCnt = 0;
monthCnt = 1;
While not(done) and arrCnt<300
begin
if state < 6 then arrCnt = arrCnt + 1;
switch (state)
Begin
case 1:
firstCount = firstCount + 1;
tempStr = "1st";
tempStr = tempStr + dayString[firstCount] + monString[monthCnt];
hashIndex[arrCnt] = tempStr;
If firstCount = 5 then
begin
state = 2;
firstCount = 0;
end;
case 2:
secondCount = secondCount + 1;
tempStr = "2nd";
tempStr = tempStr + dayString[secondCount] + monString[monthCnt];
hashIndex[arrCnt] = tempStr;
If secondCount = 5 then
begin
state = 3;
secondCount = 0;
end;
case 3:
thirdCount = thirdCount + 1;
tempStr = "3rd";
tempStr = tempStr + dayString[thirdCount] + monString[monthCnt];
hashIndex[arrCnt] = tempStr;
If thirdCount = 5 then
begin
state = 4;
thirdCount = 0;
end;
case 4:
fourthCount = fourthCount + 1;
tempStr = "4th";
tempStr = tempStr + dayString[fourthCount] + monString[monthCnt];
hashIndex[arrCnt] = tempStr;
If fourthCount = 5 then
begin
state = 5;
fourthCount = 0;
end;
case 5:
fifthCount = fifthCount + 1;
tempStr = "5th";
tempStr = tempStr + dayString[fifthCount] + monString[monthCnt];
hashIndex[arrCnt] = tempStr;
If fifthCount = 5 then
begin
state = 6;
fifthCount = 0;
end;
case 6:
If monthCnt < 12 then
Begin
state = 1;
monthCnt = monthCnt + 1;
end
else
begin
done = true;
end;
end;
end;
HashIndexCreator = 1;
Hash Index Creator

Here is a brief overview of this code.  The switch statement requires matching case statements.  In this machine, there are 6 different states.  Based on whatever the current state happens to be, the computer executes that block of code.  If the state is 1, then the block of code encapsulated with case(1) is executed.  All other code is ignored.  I start building the array by executing all of the “1st”‘s in January – “1stMonJan, 1stTueJan, 1stWedJan, 1stThuJan, and 1stFriJan.”   The nine character strings are built using concatenation.  In Easylanguage and most other languages you can add strings together:  “Cat” + “Dog” = “CatDog.”  So I take the string “1st” + “Mon” +  “Jan” to form the string “1stMonJan.”  I store the three characters for the day of the week and the three characters for the month in simple arrays.  After the fifth “1st”, I transition to state 2 and start working on all the “2nd”‘s.  Eventually the machine switches into 6th gear, er uh I mean state.  If month count is less than twelve, we gear down all the way back down to state 1 and start the process again for the month of February.  The machine finally turns off after month counter exceeds 12.  The Hash Index is completed; we have a unique address for the 300 rows.  In Part 2 I will show how to map the Hash Index onto the Hash Table and how to store the necessary information.  Finally, we will create an indicator using the data pulled from the table.

Don’t Fool Yourself – Limitations of Back Testing with Daily Data [EasyLanguage]

Which equity curve do you like best? (created with EasyLanguage script) This one…

Or this one?

Obviously the first one.  Even though it had a substantial draw down late in the test.  What if I told you that the exact same system logic generated both curves?  Here is the EasyLanguage code for this simple system.

Buy next bar at open of next bar + .25 *avgTrueRange(10) stop;
Sellshort next bar at open of next bar - .25*avgTrueRange(10) stop;

setStopLoss(500);
setProfitTarget(1000);
Open Range Break Out with Profit and Loss Objective

This algorithm relies heavily on needing to know which occurred first: the high or the low of the day.   The second chart tells the true story because it looks inside the daily bar to see what really happened.  The first chart uses an algorithm to try to determine which happened first and applies this to the trades.  In some instances,  the market looks like it opens then has a slight pull back and then goes up all day.  As a result the system buys and holds the trade through the close and onto the next day, but in reality the market opens, goes up and triggers a long entry, then retraces and you get stopped out.  What was a nice winner turns into a bad loss.  Here is an example of what might have happened during a few trades:

Nice flow – sold, bought, sold, bought, sold again and finally a nice profit.  But this is what really happened:

Sold, bought, reversed short on same day and stopped out on same day.  Then sold and reversed long on same day and finally sold and took profit.   TradeStation’s Look Inside Bar feature helps out when your system needs to know the exact path the market made during the day.  In many cases, simply clicking this feature to on will take care of most of your testing needs.  However, this simple algorithm needs to place or replace orders based on what happens during the course of the day.  With daily bars you are sitting on the close of the prior day spouting off orders.  So once the new day starts all of your orders are set.  You can’t see this initially on the surface, because it seems the algorithm is so simple.   Here is another consequence of day bar testing when the intra-day market movement is paramount:

Here the computer is doing exactly what you told it!  Sell short and then take a profit and sell short 25% of the ATR below the open.  Well once the system exited the short it realized it was well below the sell entry point so it immediately goes short at the exact same price (remember TS doesn’t allow stop limit orders).  You told the computer that you wanted to be short if the market moves a certain amount below the open.  These were the orders that were place on yesterday’s close  This may not be exactly what you wanted, right?  You probably wanted to take the profit and then wait for the next day to enter a new trade.  Even if you did want to still be short after the profit level was obtained you wouldn’t want to exit and then reenter at the same price (practically impossible) and be levied a round-turn slip and commission.   You could fiddle around with the code and try to make it work, but I guarantee you that a system like this can only be tested properly on intra-day data.  Let’s drop down to a lower time frame, program the system and see what the real results look like:

Looks very similar to the daily bar chart with Look Inside Bar turned on.  However, it is different.  If you wan’t to gauge a systems potential with a quick program, then go ahead and test on daily bars with LIB turned on.  If it shows promise, then invest the time and program the intra-day version just to validate your results.  What do you mean spend the time?  Can’t you simply turn your chart from daily bars to five minute bars and be done with it.  Unfortunately no!  You have to switch paradigms and this requires quite a bit more programming.  Here is our simple system now in EasyLanguage:

Vars:stb(0),sts(0),atr(0),icnt(0);
Vars:buysToday(0),sellsToday(0),mp(0);

{Use highD() and XXXXD(0) functions to capture the highs, lows, and closes for the past 10 days.
I could have just used a daily bar as data2.
I am looking at five minute bars so we know how the market flows through the day.
}

{This loop kicks out a warning message, but seems to work
Just do this once at the beginning of the day - faster}

{remember true range is either the higher of todays high
Or yesterdays close minus the lower of todays low or
yesterdays close}

{ tradeStation time stamps at the close of the bar so
we capture the opening of the open time plus the bar interval -
in this case 5 minute - so at 1800 + 5 (1805) I capture the open
of the day}

if time = sess1StartTime + barInterval then
begin
Value1 = 0.0;
for icnt = 1 to 10
begin
Value1 = value1 + maxList(closeD(icnt-1),highD(icnt)) - minList(closeD(icnt-1),lowD(icnt));
end;
atr = value1/10.0;
stb = open + .25* atr;
sts = open - .25* atr;
buysToday = 0;
sellsToday = 0;
end;

mp = marketPosition; {The ole mp trick}

If mp = 1 and mp[1] <> 1 then buysToday = buysToday + 1;
If mp =-1 and mp[1] <> -1 then sellsToday = sellsToday + 1;

if buysToday = 0 and time < sess1EndTime and close <= stb then buy next bar at stb stop;
if sellsToday = 0 and time < sess1EndTime and close >= sts then sellshort next bar at sts stop;

setStopLoss(500);
setProfitTarget(1000);
Open Range Break Out Utilizing Five Minute Bars

Here is a validation that Look Inside Bar does work:

This is the trade from June 1st.  Scroll back up to the second chart where LIB is turned on.