Exporting Strategy Performance XML files that use 15 years of 5-minute bars can be cumbersome. Using Maestro can be cumbersome too. I know there are great programs out there that will quickly analyze the XML files and do what we are about to do plus a bunch more. But if you want to do it yourself, try working with 224,377 KB files.
If you want to do a quick merge of equity curves of different markets or systems use this simple method. That is if you have Python, MatPlotLib and Pandas installed on your computer. Installing these are really simple and everybody is doing it. I created this script with the help of a paid version of ChatGPT.
Using Python et al. to plot combined equity curve.
Steps:
1,) Create a folder on you C drive: C:\Data
2.) Put this code into your strategy code. Make sure you change the output file name for each market/system combo that the strategy is applied to.
if t = sessionEndtime(0,1) then begin print(file("C:\Data\Sysname-NQ.csv"),d+19000000:8:0,",",netProfit + Openpositionprofit); end;
3.) Copy and paste the following code into your favorite Python IDE or IDLE.
import tkinter as tk from tkinter import filedialog import pandas as pd import matplotlib.pyplot as plt import os
def extract_market_name(filename): """Extracts the market name from the filename formatted as 'system-market.csv'.""" return os.path.splitext(filename)[0].split('-')[-1]
def load_equity_data(files): """Loads and aligns equity data from multiple CSV files.""" data_dict = {}
def calculate_metrics(equity_curve): """Calculates total profit and maximum drawdown from the equity curve.""" total_profit = equity_curve.iloc[-1] - equity_curve.iloc[0] peak = equity_curve.cummax() drawdown = peak - equity_curve max_drawdown = drawdown.max() return total_profit, max_drawdown, drawdown
def calculate_correlation(df): """Calculates and prints the correlation matrix of equity curves.""" correlation_matrix = df.corr() print("\n--- Correlation Matrix ---") print(correlation_matrix.to_string()) print("\n--------------------------")
def plot_equity_and_drawdown(df, drawdown): """Plots the combined equity curve and drawdown as separate subplots.""" fig, ax = plt.subplots(2, 1, figsize=(12, 8), sharex=True)
4.) Run it and multi-select all the system/market .csv files
5.) Examine the results:
Combined equity, maximum draw down, and correlation matrix
How did I get ChatGPT to code this for me.
I pay $20 a month for Chat and it learns from my workflows. The more I work with it, the more it knows my end goals to the scripts I am asking it to create.
Here are the prompts I provided Chat:
I want the user to be able to use tkinter to select multiple files that contain two columns: 1-date in yyyymmdd format and 2 – the equity value for that date. The filename consists of the system name and the market name and follows the following patterm: system name”-“market name”.csv”. I need you to extract the market from the filename. If the filename is “Gatts-GC.csv”, the market name is “GC.” Once all the files are opened, I need you to align the dates and create a combined equity curve. We can do this in matplotlib if you like. I need you to calculate the total profit and the maximum drawdown among the combined equity curves.
[SCRIPT FAILED]
My data does not included headers. The first column is the date in yyyymmdd with no separators and the second column is a float value. I think your pandas are having a hard time interpreting this
[SCRIPT WORKED]
That is perfect. Can we had a correlation analysis to the equity curves and print that out too.
[SCRIPT WORKED]
Can we plot the draw down on the same canvas as the equity curves?
[SCRIPT WORKED]
Could we put the draw down in a subplot instead of in the profit plot?
[FINAL SCRIPT] Look above
Am I Worried About Being Replaced by Chat GPT?
With Python and its data visualization libraries and EasyLanguage you can overcome some of the limitations of TradeStation. And of course, ChatGPT. I have been programming in Python for nine years now and I am very good at base Python. However, Pandas are still a mystery to me and interfacing with MatPlotLib is not super simple. Does it frustrate me that I don’t know exactly what ChatGPT is coding? Yes, it did at first. But now I use it as a tool. I may not be coding the Python, but I am managing ChatGPT through my well thought out prompts and my feedback. Does my knowledge of Python help streamline this process? – I think it does. And I have trained ChatGPT during the process. I am becoming a manager-programmer and ChatGPT is my assistant.
Embracing AI: The Journey from Skepticism to Synergy with ChatGPT.
Using TradeStation XML output, Python and ChatGPT to create a commercial level Portfolio Optimizer.
As a young and curious child my parents would buy me RadioShack Science Fair Kits, chemistry sets, a microscope and rockets. I learned enough chemistry to make rotten egg gas. I grew protozoa to delve into the microscopic world. I scared my mom and wowed my cousins with the Estes Der Big Red rocket. But it wasn’t until one Christmas morning I opened the Digital Computer Kit. Or course you had to put it together before you could even use it – just like the model rockets. Hey, you got an education assembling this stuff. Here is a picture of a glorified circuit board.
My first computer.
This really wasn’t a computer, but more of an education in circuit design, but you could get it to solve simple problems “Crossing the River”, decimal to binary, calculating the cube root and 97 other small projects. These problems were solved by following the various wiring diagrams. I loved how the small panels would light up with the right answers, but grew frustrated because I couldn’t get beyond the preprogrammed wiring schema. I had all these problems I wanted to solve but could not figure out the wiring. Of course, there were real computers out there such as the HAL 9000. Just kidding. I would go to the local Radio Shack and stare at the all the computers. Hoping one day I would have one sitting on my desk. My Dad was an aircraft electrician (avionics) in the U.S. Navy with a specialty in Inertial Navigation Systems. He would always want to talk about switches, gyros, some dude named Georg Ohm and oscilloscopes. I had my mind stuck in space, you know “3-D Space Battle” – the Apple II game, to listen or to learn from his vast knowledge. A couple of years later and a paper route I had a proper 16K computer, the TI-99-4A. During this time, I dreamed of a supercomputer that could answer all my questions and solve all my problems. I thought the internet was the manifestation of this dream, but in fact it was the Large Language Models such as ChatGPT.
Friend or Foe
From a programmer’s perspective AI can be scary, because you might just find yourself out of a job. From this experiment, I think we are a few years away from this possibility. Quantum computing, whenever it arrives, might be a viable replacement, but for now I think we are okay.
The Tools You Will Need
You will need a full installation of Python along with Numpy and Pandas installed on your computer if you want ChatGPT to do some serious coding for you. Python and its associated libraries are simply awesome. And Chat loves to use these tools to solve a problem. I pay $20 a month for Chat so I don’t know if you could get the same code as I did if you have the free version. You should try before signing up.
The Project: A Portfolio Optimizer using an Exhaustive Search Engine
About ten years ago, I collaborated with Mike Chalek to develop a parser that analyzes the XML files TradeStation generates when saving strategy performance data. Trading a trend-following system often requires significant capital to manage a large portfolio effectively. However, many smaller traders operate with limited resources and opt to trade a subset of the full portfolio.
This approach introduces a critical challenge: determining which markets to include in the subset to produce the most efficient equity curve. For instance, suppose you have the capital to trade only four markets out of a possible portfolio of twenty. How do you decide which four to include? Do you choose the markets with the highest individual profits? Or do you select the ones that provide the best profit-to-drawdown ratio?
For smaller traders, the latter approach—prioritizing the profit-to-drawdown ratio—is typically the smarter choice. This metric accounts for both returns and risk, making it essential for those who need to manage capital conservatively. By focusing on risk-adjusted performance, you can achieve a more stable equity curve and better protect your account from significant drawdowns.
I enhanced Mike’s parser by integrating an exhaustive search engine capable of evaluating every combination of N markets taken n at a time. This approach allowed for a complete analysis of all possible subsets within a portfolio. However, as the size of the portfolio increased, the number of combinations grew exponentially, making the computations increasingly intensive. For example, in a portfolio of 20 markets, sampling 4 markets at a time results in 4,845 unique combinations to evaluate.
Calculating the number of combinations.
Using the formula above you get 4,845 combinations. If you estimate each combination to take once second, then you are talking about 21 minutes. N/2will produce the most combinations. Sampling 10 out of 20 will take 51.32 hours.
1 out of 20: 20 combinations
2 out of 20: 190 combinations
3 out of 20: 1,140 combinations
4 out of 20: 4,845 combinations
5 out of 20: 15,504 combinations
6 out of 20: 38,760 combinations
7 out of 20: 77,520 combinations
8 out of 20: 125,970 combinations
9 out of 20: 167,960 combinations
10 out of 20: 184,756 combinations
11 out of 20: 167,960 combinations
The exhaustive method is not the best way to go when trying to find the optimal portfolios across a broad search space. This is where a genetic optimizer comes in handy. I played around with that too. However, for this experiment I stuck with a portfolio of eleven markets. I used the Andromeda-like strategy that I published in my latest installment of my Easing Into EasyLanguage series.
Here is the output of the project when sampling four markets out of a total of eleven. All this was produced with Python and its libraries and ChatGPT.
Tabular Format
The best 4-Market portfolio out of 11 possibilities.
Graphic Format
ChatGPT, Python, Matplotlib, oh my!
Step 1 – Creating an XML Parser script
You can save your strategy performance report in RINA XML format. The ability to save in this format seems to come and go, but my latest version of TradeStation provides this capability. The XML files are ASCII files that contain every bar of data, market and strategy properties and trades. However, they are extremely large as each piece of data has prefix and suffix tag.
Imagine the size of the XML when working with one or even five-minute bars.
Getting ChatGPT to create an XML parser for the specific TradeStation output
The first thing I did was save the performance, in XML format, from a workspace in TradeStation that used an Andromeda-like strategy on eleven different daily bar charts.
I asked Chat to analyze the XML file that I attached to a new chat. I started a new chart for this project. I discovered the chat session is also known as a “workflow”. This term emphasizes:
Collaboration: We work as a team to tackle challenges.
Iteration: We revisit and improve upon earlier steps as needed.
Focus: Each session builds upon the previous ones to move closer to a defined goal.
Once it understood the mapping of the XML, I asked Chat to extract the bar data and output it to a csv file. And it did it without a hitch. When I say it did it, I mean it created a Python script that I loaded into my PyScripter IDE and executed.
I then asked for an output of the trade-by-trade report, and it did it without a hitch. Notice these tasks do not require much in the way of reasoning.
Getting ChatGPT to combine the bar and trade data to produce a daily equity stream.
This is where Python and its number crunching libraries came in handy. Chat pulled in the following libraries:
xml.etree
pandas
tkinter
datetime
I love Python, but the level of abstraction with its libraries can make you dizzy. It is not important to fully understand the panda’s data frame to utilize it. Heck, I didn’t really know how it mapped and extracted the data from the XML file. I prompted chat with the following:
[My prompts are bold and italicized.]
With the bar data and the trade data and the bigpointvlaue, can you create an equity curve that shows a combination of open trade equity and closed trade equity? Remember the top part of the xml file contains bar data and the lower part contains all the trade data.
It produced a script that hung up. I informed chat that the script hung up and that the script wasn’t working. It found the error and fixed it. Think about what Chat was doing. It was able to align the data so that open trades produced open trade equity and closed out trades produced closed trade equity. Well, initially it had a small problem. It knew what Buy and Sell meant, and the math involved with calculating the two forms of equity, open and closed. I didn’t inform Chat of any of this. But the equity data did not look exactly right. It looked like the closed trade equity was being calculated improperly.
Is the Script checking for LExit and SExit to calculate when a trade closes out?
Once it figured out that the equity stream must contain open and closed trade equity and learned the terms, LExit and SExit, a new script was created that nearly replicated the equity curve from the TradeStation report. When Chat starts creating lengthy scripts it will open a side bar window call the “Canvas” and put the script in there. This makes it easier to copy and paste the code. I eventually noticed that the equity curve did not include commission and slippage charges.
Please extract the slippage and commission values and deduct this amount from each trade.
At this point the workflow remembered the mapping of the XML file and was able incorporate these two values into the trade processing. I wanted the user to be able to select multiple XML files and have the script process these files and produce the three output files, bar_data, trade_data, and equity_data. I did have to explain that the execution costs must be applied to all the entries and exits.
I would like the user to select multiple xml files and create the data and trade and equity files incorporating the system name and market name and their naming scheme.
A new library was imported, TKinter and a file open dialog was used for the user to select all the XML files they wanted to process. These few chores required very little interaction between Chat and me. I thought, wow this is going to be a breeze. I moved onto the next phase of the project by asking Chat the following:
Can you create a script that will use Tkinter to open the equity files from the prior script and allow the user to choose N out of the total files selected to create all the combined equity curves using an exhaustive search method.
I knew this was a BIG ASK! But it swallowed this big pill without a hitch. Maybe us programmers will be replaced sooner than later. I wanted to fine tune the script.
Can you keep track of maximum draw down for each combination and then sort the combinations by the profit to draw down ratio. For the best combination can you create a csv file with the daily returns so i can plot them in Excel.
On a quick scan of the results, I did not initially notice that the maximum draw-down metric was not right. So, I pushed on the fine tuning.
This works great. In the output files can we delete the path name to the csv files. For the output I would just like to have the system name and the symbol for each combination.
Script created.
I had the following error message. NameError: name ‘all_combinations’ is not defined. I also asked Chat if it can you add “Date” to the header for the combination daily files?
Many times, Chat will just recreate the part of the code that needs to be modified. This makes copying and pasting difficult, because you have to make sure you find the right location in the code to replace. I always ask Chat to recreate the entire script with the latest changes.
Hey Chat, the following error message was created: system, symbol = filename.replace(“.csv”, “”).split(“_”) ValueError: too many values to unpack (expected 2)
The error was fixed, but I finally noticed the maximum draw down calculations were not right.
I don’t think the maximum draw down metric is being calculated properly. “Andromeda-Like_@SB, Andromeda-Like_@EC, Andromeda-Like_@S, Andromeda-Like_@JY”,
PL: 92141.8499999999,
DD: 92985.59999999992,
Ratio: 0.990926014350609,144.
The draw down metric is similar to the total profit metric.
Script created.
The draw down is still not calculated properly. I think there is an error in your reasoning. I calculate draw down by iterating through the equity curve and keeping track of peaks and subsequent troughs. The largest distance between the peaks and its associated trough is the max draw down.
Script created but looked a little primitive. So, I asked.
Is this the most efficient method?
Well, I should have kept my mouth shut. I felt like the Python libraries would be able to do this much quicker with list comprehension or with some other magic. And Chat agreed with me. Script created.
The draw down is still incorrect.
Script created.
It was still wrong and instead of going back and using the original draw down script I tried to nudge Chat into the right direction.
The equity curve will have multiple peaks and multiple troughs. You will need to keep track of all of the peaks and troughs and then find the maximum distance between the sets of peaks and troughs.
Script created and was still wrong. This is where you start going down a rabbit hole. Chat will produce debugging script to try and figure out where the error resides. But this is very time consuming, so I kept pushing Chat to do it right.
Each peak must be tied to its respective trough. Each distance between each respective peak and drawdown can then be measured for the largest value.
Script is wrong.
This still not working. I don’t think you are calculating the difference between peaks and subsequent troughs properly.
Script is wrong.
This is still not working. We should be able to do this in a single pass. Iterate through the daily values and track the highest level at that point in the series and then the subsequent deepest trough. Store the first difference. Continue iterating looking for a new peak and the subsequent deepest trough. Compare this value to the prior and take the largest of the two. Store this value and continue iterating.
Script from original request is generated. Finally got back to the starting point. What a waste of time. I will be more careful with my suggestions. However, the script is still wrong – arrgh! I decide to run the debugger on the draw down function and the code is right, but the data is wrong.
The problem lies in the equity series. It should contain the combined equity for the combinations. There should be a master date and each combination populates the master date. If there is a missing date in the combinations, then the master date should copy the prior combinations combined value.
Warning message: se obj.ffill() or obj.bfill() instead. df_aligned = df.reindex(master_date_index).fillna(method=”ffill”).fillna(0)
Chat created some deprecated code. This was an easy fix, I just had to replace one line of code. However, every iteration following this still had the same damn deprecated code.
Error or warning: FutureWarning: Series.getitem treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use ser.iloc[pos] peak = equity_series[0] # Initialize the first value as the peak
Script updated.
Can we drop the “_equity_curve” from the name of the system and symbol in the Perfomance metrics file. Will excel accept the combination names as a single column or will each symbol have its own column because of the comma. I would like for each combination in the Performance_Metrics file to occupy just one column.
Script created and this is what it should look like. Notice it was remembering the wrong maximum draw down values I fed it earlier. Don’t worry it was right in the script.
Combination
Total Profit
Max Drawdown
Profit-to-Drawdown Ratio
Combination Number
“Andromeda-Like_@SB, Andromeda-Like_@EC”
92141.85
92985.60
0.99
1
“Andromeda-Like_@S, Andromeda-Like_@JY”
82450.55
70530.45
1.17
2
Conclusion
I could have done the same thing ChatGPT did for me, but I wouldn’t have used the Numpy or Pandas libraries simply because I’m not familiar with them. These libraries make the exhaustive search manageable and incredibly efficient. They handle tasks much faster than pure Python alone.
To get ChatGPT to generate the code you need, being a programmer is essential. You’ll need to guide it through debugging, steer it in the right direction, and test the scripts it produces. It’s a back-and-forth process—running the script, identifying warnings or errors, and pointing out incorrect outputs. Sometimes, your programming insights and suggestions might inadvertently lead ChatGPT down a rabbit hole. Features that worked in earlier versions may stop working in subsequent iterations as modifications are applied to address earlier issues.
ChatGPT can also slow down at times, and its Canvas tool has a line limit, which can result in incomplete scripts. As a programmer, it’s easy to spot these issues—you’ll need to inform ChatGPT, and it will adjust by splitting the script into parts, some appearing in the Canvas and the rest in the chat window.
The collaboration between ChatGPT and me was powerful enough to replicate, in just one day, software that Mike Chalek and I spent weeks developing a decade ago. The original version had a cleaner GUI, but it was significantly slower compared to what we’ve achieved here.
If you’re a programmer, have Python installed with its libraries, and work with ChatGPT, the possibilities are endless. But there’s no magic—success requires thoughtful feedback and precise prompting.
Email me if you would like to have the Python scripts that accomplish the following tasks. If you are not familiar with pandas or xml processing, the code, even being Python savvy, will look a little foreign. No worries – it just works.
XML Parser – creates data, trades and equity files in .csv format.
TradeStationExhaustiveCombos – creates all the combos when sampling n out of N markets.
The simple Tkinter GUI and Matplotlib graphing tool to plot the combos.
There is a total of three scripts. Remember you will need to have Python, pandas, matplotlib already installed on your computer. If you have any questions on how to install these just let me know.
Thinking of using futures data from a vendor such as CSI, Pinnacle, Norgate…?
You would be surprised how many quants still use Excel or they own homegrown back-testing software. Determining trade execution is simple. On a stop order, all you need to do is see if today’s bar is higher than the prior bars calculated trade signal – right. And if it is then a buy stop order is executed and the calculated signal price. Store the entry price and when the exit occurs do the math to calculate the profit or loss. Before you can do this, you must know two very important contract specs on the futures data you are testing:
big point value – I will explain how to calculate this a little later
minimum tick move
That’s it. You don’t need to know anything beyond this to properly calculate trade signals and trade P and L. As long as you store this information for each futures/commodity market you are testing, then you will be good to go. Many testing platforms store more information such as contract size, trading hours, expiration date and several other contract specs. You don’t need to know this information if you all you are interested is proper signal placement and accurate trade accounting. Once you set the big point value and minimum tick move you are good to go, right? Yes, if you stick with the same data vendor. I kid you not. Data vendors often quote the same exact market with different decimal places. If you look up a sugar quote at the CME website, you will see it quoted like 0.1908. If you look it up on TradeStation it is quoted as 19.08. It’s the same size contract 112,000 pounds, but if you use the big point value at the CME, it will not produce accurate calculations on the TradeStation quote. You can’t use the contract size of 112,000 pounds to help with the math either. You have to delve into how each data vendor quotes their data and this will impact the big point move and minimum tick move. Most vendors are gracious enough to provide a futures dictionary with these specs. But it is wise to know how to do this by hand as well.
If you had to take a test, could you calculate the profit or loss from a trade in soybeans, in sugar or in euro currency?
Are you kidding me this is super simple. Just program the strategy in TradeStation or Amibroker or Multicharts or TradingSimula-18. I took the NFA Series 3 test more than two decades ago, and this type of question was on the test. The following information was given:
contract size – 5000 bushels
minimum tick or move – 1/4th of a cent or penny
long entry price 8.32’2 and sell exit price 8.48’6 (per bushel)
The entry and exit price may look a little funny. The entry price is eight dollars and 32 1/4 cents. Many still quote soybeans in 1/8ths. So, 2 X 1/8th = 1/4th. For fun let’s calculate the notional value of the entry and exit prices. The notional value of a soybean contract at the entry price is $8.3225 * 5000 = $41,612.50. As you know futures are highly leveraged and you can control this quantity of soybeans for a small percentage of the notional value (margin.) Now the exit price is $8.4875 (6*1/8th = 3/4th of a cent) and the notional value of the contract at exit is $8.4875 * 5000 = $42,437.5. The result of the trade was $42,437.50 – $41,612.50 = $825. Or you could simply multiple the difference between entry and exit by 5000. This would $0.165 * 5000. This makes perfect sense, but as a broker it was difficult to keep the contract size of a market and its minimum tick move in your memory. Well, if you did it long enough it wasn’t that difficult. You can reduce the math down to one easy to remember value and quickly do the math in your head. The concept of the big point move allows for this. If you download your data from Pinnacle or CSI the price of soybeans is usually quoted like this: 848.75. This would be eight hundred and 48.75 cents. The big point move is the amount of dollars required to lift (or drop) the first digit to the left of the decimal by one. The first digit to the left of the decimal in beans is a penny or one cent. A one cent move in beans is 5000 * $0.01 or $50. Going back to our trade example 848.75 – 832.25 = 16.5 – multiply this by $50 you get $825. You can also derive the minimum move dollar value too. If the minimum move is 1/4th of a cent, then you can multiply 5000 * $.0025 and this yields $12.5.
Why is this important to know?
You should know this if you are trading the market, period. Also, if you are a quant and are using some back testing software that requires you to set up the database outside the purview of the back-tester, then these values must be known, and must also be accurate. Since TradeStation integrates its own data, you don’t have to worry about this. But if you want to use a database from Pinnacle, BarChart, CSI or Norgate, then you have to take the time to set this up, right off the bat. I feel like the leading data vendors, provide accurate data. But there are differences from one vendor to another. Not all data vendors use the same number of decimal places in their market quotes, so you must be able to determine the big point value and minimum move from the data. You can do all the math to determine profit and loss from the big point value. Here is a snapshot of a few markets in the TS-18 dataMasterPinnacle and the TS-18 dataMasterCSI text files. Like I said, all vendors will provide this information for you, so you can set your database properly. Some back-testing platforms require more contract specs, but TS-18 just needs this information to carry out accurate calculations. Here TS-18 wants to know the symbol, big point value, minimum tick move, and market name.
Other than the symbol names the values for each symbol are very similar. Most data vendors, including CSI quote the Euro currency like this:
1.08735 and a move to 1.08740 = 0.00005 *$125,000 = $6.25
But Pinnacle data quotes it like this:
108.735 and a move to 108.740 = 0.005 * $1,250 = $6.25
The size of the contract isn’t different here. It is the quote that is different. The big point value is tied to the contract size and the format of the quote. This is why knowing the big point value is so important. If you don’t set up the contract specifications correctly, then you will receive inaccurate results.
One more test in sugar – from the CME website
contract size – 112,000 pounds
minimum tick or move – $0.0001
long entry price $0.1910 and sell exit price $0.1980 (per pound)
big point value = $112,000
$0.0070 * 112,000 = $784
From TradeStation
contract size – 112,000 pounds
minimum tick or move – $0.01
long entry price $19.10 and sell exit price $19.80 (per pound??)
big point value = $1,120
$0.70 * 1,120 = $784
Most fractional quotes are delivered in decimal format and the also very important minimum tick move.
The interest rate futures have minimum tick moves ranging from 1/256ths to 1/32nds. Most vendors will give you a quote like 120.03125 for the bonds. If you test in TradersStudio or Excel or any other platform where the data is not integrated and are using CSI (or any other vendor), then you must force your calculated prices to fall on an even tick. Assume you do a calculation to buy a bond future at 120.022365895. If you don’t take the minimum tick move into considerations, you might be filled at this theoretical price. In reality you should be filled on an even tick at 120.03125. This is worse but realistic fill. You could create what you think is a really awesome strategy where you are shaving off the price on every trade – getting a better fill on every trade.
Thinking of purchasing and using back-testing software or Excel and data from a data vendor?
Become your own quant and do this. You will learn a lot about algorithm testing and development. But first things first. Get your database set up according to your data vendor. Once this chore is complete, testing becomes smooth sailing. I provide the databases for Pinnacle and CSI with TS-18. Other software provides these as well, and a way to create your own databases.
In May of 2018 Morgan Stanley presented a report from their research department. The report was written by Tony Small and Matthew Hornbach. The information presented validated the idea of seasonality in the markets. The writers/researchers gathered all the data going back to 1984 on the ten-year futures and all the federal holidays going back over the same span of time. They then analyzed either going long or short M days prior to the holiday and exiting N days before the holidays. They also did the same thing after the holiday – entering M days after and exiting N days after. The boundary was set to five days before and after. This report was sent to me by one of my clients. The conclusion of the article was that there seemed to be a seasonality component to the days prior and after a major holiday. And that it seemed this component did indeed provide a technical advantage (on the surface.) Here is their conclusion.
“Given the risk of data mining, investors should accept this analysis with some degree of caution,” Small and Hornbach said. “There is no guarantee that historical seasonality and past price performance will continue in the future.”
How would one test this idea on their own?
Tools needed:
Python with NumPy installed.
An AI chatbot such as ChatGPT.
Access to TY futures data going back to 1984 (actual non-Panama adjusted data would be preferable.)
A testing platform that allows future leak such as George’s own TradingSimula-18. Excel would work too but would require a bunch of work.
Some programming knowledge.
Create output for all of the federal holidays going back to January 1984. This list must also include Good Fridays. This list should have two fields – the date and the name of the holiday.
TS-18 came in handy because I read the file that was created from step 1 and whenever I came across or spanned a holiday date, I created a loop that calculated the ROC for the prior five days and also for the five days that followed. I created a nomenclature to quickly recognize the ROC span.
B54 – ROC between 5 days before and 4 days before – 1 day
B53 – ROC between 5 days before and 3 days before – 2 days
B52 – ROC between 5 days before and 2 days before – 3 days
B51 – ROC between 5 days before and 1 days before – 4 days
B43 – ROC between 4 days before and 3 days before – 1 day
B42 – ROC between 4 days before and 2 days before – 2 days
B41 – ROC between 4 days before and 1 days before – 3 days
B32 – ROC between 3 days before and 2 days before – 1 day
B31 – ROC between 3 days before and 1 days before – 2 days
B21 – ROC between 2 days before and 1 days before – 1 day
A12 – ROC between 1 days after and 2 days after – 1 day
A13 – ROC between 1 days after and 3 days after – 2 days
A14 – ROC between 1 days after and 4 days after – 3 days
A15 – ROC between 1 days after and 5 days after – 4 days
A23 – ROC between 2 days after and 3 days after – 1 day
A24 – ROC between 2 days after and 4 days after – 2 days
A25 – ROC between 2 days after and 5 days after – 3 days
A34 – ROC between 3 days after and 4 days after – 1 day
A35 – ROC between 3 days after and 5 days after – 2 days
A45 – ROC between 4 days after and 5 days after – 1 day
This process created a file that looked like this:
Output of ROC with nested for loops
With this data I summed up each ROC calculation (B54 thru A45). From this you could pick an entry and exit day combination. The best results were wrapped around Memorial Day. Here was where I ran into a small but highly aggravating scenario. Most of the holidays had similar numbers, but there were a few that had some really strange values. I knew I couldn’t use regular adjusted futures data, because many times these values can go below zero and when doing division with these price adjustments, the values are simply not accurate. So, I used continuous non adjusted data, but ran into a problem with the rollover gap. Some of the holidays occurred really close to rollover dates. The calculations that bridged a rollover was not accurate. I then remembered Pinnacle data also includes Ratio Adjusted Data or RAD. I applied the looping algorithm to this data, and it SEEMED to fix the problem.
I was able to pick a B(n,m) or A(m,n) for each market. If the magnitude of ROC was sufficiently negative, you could use the dates to short the market.
ChatGPT and Python and Numpy to the rescue
I asked ChatGPT to provide python code to generate a list of all federal holidays from 1984.
from pandas.tseries.holiday import USFederalHolidayCalendar from datetime import datetime,timedelta import numpy as np # Create a calendar for U.S. federal holidays cal = USFederalHolidayCalendar()
# Get the holidays for the year 2014 holidays_2014 = cal.holidays(start='1984-01-01', end='2024-12-31').to_pydatetime()
federalHolidays = [] date_list = holidays_2014.astype(datetime).tolist() for n in range(len(date_list)): dateString = f'{date_list[n]:%Y%m%d}' federalHolidays.append(int(dateString))
Create a list of federal holidays
This worked great, but I had to convert the NumPy datetime construct to a string. I like to keep things simple. You will see this in the code. I started to get to work on the second part of the task when I discovered there were no dates for GOOD FRIDAYS – argh! ChatGPT to rescue once again.
def good_friday_dates(start_year, end_year): good_friday_dates = [] for year in range(start_year, end_year + 1): # Calculate the date of Easter Sunday for the given year easter_sunday = calculate_easter_date(year)
# Good Friday is two days before Easter Sunday good_friday = easter_sunday - timedelta(days=2)
# Add the Good Friday date to the list good_friday_dates.append(good_friday)
return good_friday_dates
def calculate_easter_date(year): # Algorithm to calculate the date of Easter Sunday a = year % 19 b = year // 100 c = year % 100 d = b // 4 e = b % 4 f = (b + 8) // 25 g = (b - f + 1) // 3 h = (19 * a + b - d - g + 15) % 30 i = c // 4 k = c % 4 l = (32 + 2 * e + 2 * i - h - k) % 7 m = (a + 11 * h + 22 * l) // 451 month = (h + l - 7 * m + 114) // 31 day = ((h + l - 7 * m + 114) % 31) + 1 easter_date = datetime(year, month, day) return easter_date
# Generate Good Friday dates from 1984 to 2024 good_friday_dates_1984_to_2024 = good_friday_dates(1984, 2024)
Hope this code is right for Good Fridays
Good Friday, as Easter, is all over the map. Tomorrow is Good Friday. The code looks pretty gross, so I just accepted it. It may not be 100% accurate, but if ChatGPT says so…. Once I had a list of all holidays, I was able to start cooking (MLK was added in 1986 and Juneteenth in 2021.) Since Juneteenth is relatively new it did not offer a statistically significant number of observations.
Future Leak comes in Handy.
Every quant needs some software that will look through data and do some calculations. I did this project with EasyLanguage, but it was a little difficult. I had to do a two-pass process (well I did the same with Python too.) I first created an indicator that imported the holiday dates and then outputted the close (or open of next bar) for close[5], close[4], close[3], close[2], close[1] and then waited 1, 2, 3, 4 and 5 bars after the date and printed out those respective closing prices. I did not do this step in EasyLanguage. I actually create EasyLanguage as my output. Here is what it looked like:
Future leak allows you to peak into the future and this comes in super handy when doing research like this. Here is some python code in TS-18 that does all these calculations in one sweep.
# The date of the holiday and the holiday name are stored in lists # Each Holiday has its own date and 20 ROC calculations # These calculations are stored in lists as well tempDate = int(extraData[extraCnt][0]) #first field in the csv file whichHoliday = extraData[extraCnt][1] #second field in the csv file if date[D] == tempDate or (date[D] > tempDate and date[D1] < tempDate): foundJ = 99 for j in range(12): if whichHoliday == listOfHolidayClass[j].holidayName: foundJ = j listOfHolidayClass[foundJ].holidayDates.append(tempDate); for j in range(-5,-1): #Before Holiday for k in range(j+1,0): denom = close[D+j] listOfHolidayClass[foundJ].rocList.append((close[D+k] - close[D+j])/denom) for j in range(0,4): #After Holiday - here is the future leak for k in range(j+1,5): denom = close[D+j] listOfHolidayClass[foundJ].rocList.append((close[D+k] - close[D+j])/denom) extraCnt = extraCnt + 1
Python code using future leak to calculate the 20 ROC from before and after holiday
I then printed this out to a csv file and imported into Excel (see above) and found out the best combination of days for each holiday. Now that I had the B(n,m) or the A(m,n) values, I need a process to tell TS-18 which day to buy/short and exit. Here is some code that, again uses future leak, to print out the five dates before the holiday and the five dates after the holiday.
tempDate = int(extraData[extraCnt][0]) whichHoliday = extraData[extraCnt][1] if date[D] == tempDate or (date[D] > tempDate and date[D1] < tempDate): for j in range(12): if whichHoliday == listOfHolidayClass[j].holidayName: foundJ = j print(date[D],end=',') print(whichHoliday,end=',') for j in range(-5,6): print(date[D+j],end=',') print("",end = '\n') extraCnt = extraCnt + 1
# Here is what the file looked like 19840103,NewYear,19831223,19831227,19831228,19831229,19831230,19840103,19840104,19840105,19840106,19840109,19840110, 19840221,PresidentsDay,19840213,19840214,19840215,19840216,19840217,19840221,19840222,19840223,19840224,19840227,19840228, 19840423,GoodFriday,19840413,19840416,19840417,19840418,19840419,19840423,19840424,19840425,19840426,19840427,19840430, 19840529,MemorialDay,19840521,19840522,19840523,19840524,19840525,19840529,19840530,19840531,19840601,19840604,19840605, 19840705,July4th,19840627,19840628,19840629,19840702,19840703,19840705,19840706,19840709,19840710,19840711,19840712, 19840904,LaborDay,19840827,19840828,19840829,19840830,19840831,19840904,19840905,19840906,19840907,19840910,19840911, 19841008,ColumbusDay,19841001,19841002,19841003,19841004,19841005,19841008,19841009,19841010,19841011,19841012,19841015, 19841112,VeteransDay,19841105,19841106,19841107,19841108,19841109,19841112,19841113,19841114,19841115,19841116,19841119, 19841123,Thanksgiving,19841115,19841116,19841119,19841120,19841121,19841123,19841126,19841127,19841128,19841129,19841130,
Create the dates that occur in the date B4 and after holiday
Now read the data in and pick how many days before to b/s and before to exit or after to b/s and to exit
#structure of holidayDatesData list - all strings # 0.) Holiday Date # 1.) Holiday Name # 2.) 5-Before # 3.) 4-Before # 4.) 3-Before # 5.) 2-Before # 6.) 1-Before # 7.) 0-Before/After # 8.) 1-After # 9.) 2-After # 10.) 3-After # 11.) 4-After # 12.) 5-After #Below I pick 1 day after to buy and 4 days after to exit [8,11] for j in range(numHolidayDates): if holidayDatesData[j][1] == "MemorialDay": holidayBuyDates.append(int(holidayDatesData[j][8])) holidayXitDates.append(int(holidayDatesData[j][11]))
# Okay Let's put in some logic to create a long position if not(long) and canTrade and date[D] == holidayBuyDates[gfCnt]: price = close[D] tradeName = "Hol-A1Buy" posSize = 1 tradeTicket.append([buy,tradeName,posSize,price,mkt]) # Long Exits if long and canTrade and date[D] == holidayXitDates[gfCnt]: price = longExit tradeName = "Hol-A4Xit" tradeTicket.append([exitLong,tradeName,curShares,price,mkt]) gfCnt = gfCnt + 1
Here are the results for the Morgan Stanley analysis in 2018
Best combos for entry and exit for each holiday.
My analysis covered the same period plus all of 2018, 2019, 2020, 2021, 2022, and 2023. My combos were very similar in most cases, but the last few years moved the needle a little bit. However, the results still showed a technical edge.
The concept of Scope when it comes to a programming language can get very confusing very quickly. If you are new to programming and you think Scope is a mouthwash, no worries here. I will give a full explanation of Scope in terms of programming with the help of ChatGPT. Its response is italicized.
In Python, the scope of a variable refers to the region of the program where that variable can be accessed or referenced. There are two types of scopes in Python: global scope and local scope.
Here’s an example that demonstrates the concept of scope in Python:
# global variable x = 10
def my_func(): # local variable y = 5 print("x inside my_func:", x) # x is accessed from the global scope print("y inside my_func:", y) # y is accessed from the local scope
my_func()
print("x outside my_func:", x) # x is accessed from the global scope print("y outside my_func:") # y is not defined in the global scope, so this will raise a NameError
Example of Scope as demonstrated by Chat-GPT
In this example, we define a global variable x with a value of 10.
We then define a function my_func that has a local variable y with a value of 5. Inside the function, we print the values of x and y. The variable x is accessed from the global scope, which means it can be referenced from anywhere in the program.
The variable y, on the other hand, is only accessible within the function, which is the local scope. Outside the function, we print the value of x again, which is still accessible from the global scope. However, when we try to print the value of y, we get a NameError because y is not defined in the global scope.
Okay that was pretty formal. In simpler terms, a variable declared inside a function can only be seen inside that function, unless it is declared globally. Another thing about Scope, is that the variables have amnesia. Once you step out of the function the variable forgets what it was, and you can’t refer to its prior value upon return to the function.
Unless you are an EasyLanguage function. Scope is still constrained by an EasyLanguage function, because it is a separate piece of code held within the library of Analysis Techniques. Most of the time you can’t see what’s in the function unless you open it with the ELEditor. However, the variables that are defined inside the function do not suffer from amnesia. If you need to refer to a prior value of a locally declared variable, you can. This type of function is what EasyLanguage calls a Serial function. The only downside to this function is it slows processing down quite a bit.
Okay. To make a long story short I wanted to show the magic of EasyLanguage function that I have been working with on a project. This project includes some of Ehlers’ cycle analysis functions. The one I am going to discuss today is the HighRoof function – don’t worry I am not going to go into detail of what this function does. If you want to know just GOOGLE it or ask ChatGPT. I developed a strategy that used the function on the last 25 days of closing price data. I then turned around and fed the output of the first pass of the HighRoof function right back into the HighRoof function. Something similar to embedding functions.
doubleSmooth = average(average(c,20),20);
Sort of like a double smoothed moving average. After I did this, I started thinking does the function remember the data from its respective call? The first pass used closing price data, so its variables and their history should be in terms of price data. The second pass used the cyclical movements data that was output by the initial call to the HighRoof function. Everything turned out fine, the function remembered the correct data. Or seemed like it did. This is how you learn about any programming language – pull out your SandBox and do some testing. First off, here is my conversion of Ehlers’ HighRoof function in EasyLanguage.
This function requires just two inputs – the data (with a history) and a simple length or cut period. The first input is of type numericSeries and the second input is of type numericSimple. You will see the following line of code
This code prints out the last three historic values of the HighPass variable for each function call. I am calling the function twice for each bar of data in the Crude Oil futures continuous contract.
Starting at the top of the output you will see that on 1230206 the function was called twice with two different sets of data. As you can see the output of the first two lines is of a different magnitude. The first line is approximately an order or magnitude of 10 of the second line. If you go to lines 3 and 4 you will see the highPass[1] of lines 1 and 2 moves to highPass[2] and then onto highPass[3]. I think what happens internally is for every call on per bar basis, the variables for each function call are pushed into a queue in memory. The queue continues to grow for whatever length is necessary and then either maintained or truncated at some later time.
Why Is This So Cool?
In many languages the encapsulation of data with the function requires additional programming. The EasyLanguage function could be seen as an “object” like in object-oriented programming. You just don’t know you are doing it. EasyLanguage takes care of a lot of the behind-the-scenes data management. To do the same thing in Python you would need to create a class of Ehlers Roof that maintain historic data in class members and the calculations would be accomplished by a class method. In the case of calling the function twice, you would instantiate two classes from the template and each class would act independent of each other.
One last nugget of information. If you are going to be working with trigonometric functions such as Cosine, Sine or Tangent, make sure your arguments are in degrees not radians. In Python, you must use radians.
Only Trade the Best Segments of the Equity Curve – Cut Out Drawdown and Take Advantage of Run Ups! Really?
Equity curve feedback has been around for many years and seems highly logical, but one can’t get an industry-wide agreement on its benefit. The main problem is to know when to turn trading off and then back on as you track the equity curve. The most popular approach is to use a moving average of the equity curve to signal system participation. When the equity curve moves below 30, 60, or 90 period-moving average of equity, then just turn it off and wait until the curve crosses back above the average. This approach will be investigated in Part 2 of this series. Another approach is to stop trading once the curve enters a drawdown that exceeds a certain level and then start back up once the equity curve recovers. In this post, this method will be investigated.
Programmers Perspective
How do you go about programming this tool to start with. There are probably multiple ways of accomplishing this task, but the two I have most often observed were the two pass process and the inline simultaneous tracking of the synthetic and actual equity curves. The two pass process generates an unadulterated equity curve and stores the equity and trades either in memory or in a file. The second part of the process monitors the external equity curve along with the external trades synchronously and while trading is turned on, the trades are executed as they occur chronologically. When trading is turned off, the synthetic equity curve and trades are processed along the way. The second method is to create, which I have coined (maybe others too!), a synthetic equity curve and synthetic trades. I have done this in my TradingSimula_18 software by creating a SynthTrade Class. This class contains all the properties of every trade and in turn can use this information to create a synthetic equity curve. The synthetic equity curve and trades are untouched by the real time trading.
Start Simple
The creation of an equity curve monitor and processor is best started using a very simple system. One market algorithm that enters and exits on different dates, where pyramiding and scaling in or out are not allowed. The first algorithm that I tested was a mean reversion system where you buy after two consecutive down closes followed by an up close and then waiting one day. Since I tested the ES over the past 10 years you can assume the trend is up. I must admit that the day delay was a mistake on my behalf. I was experimenting with a four bar pattern and somehow forgot to look at the prior day’s action. Since this is an experiment it is OK!
if marketPosition <> 1 and (c[2] < c[3] and c[3] < c[4] and c[1] > = c[2]) then buy next bar at open;
//The exit is just as simple - //get out after four days (includeing entry bar) on the next bars open - no stops or profit objectives.
If barsSinceEntry > 2 then sell next bar at open;
Simple Strategy to test Synthetic Trading Engine
Here is the unadulterated equity curve using $0 for execution costs.
Non adjusted equity curve of our simple mean reversion system. Wait for a pull back and then a rally before entering.
The Retrace and Recover Method
In this initial experiment, trading is suspended once you reach a draw down of 10% from the peak of the equity curve and then resumes trading once a rally of 15% of the subsequent valley. Here is an intriguing graphic.
Green means ON. Red means OFF. The lower curve is the resultant curve.
I did this analysis by hand with Excel and it is best case scenario. Meaning that when trading is turned back on any current synthetic position is immediately executed in the real world. This experiment resulted in nearly the same drawdown but a large drop in overall equity curve growth – $75K.
Put the Synthetic Equity Curve Engine to the Test
Now that I had the confirmed results of the experiment, I used them as the benchmark against my TS-18 Synthetic Trade Engine. But before I installed the Equity Curve algorithm, I needed to make sure my synthetic trades lined up exactly with the real equity curve. The synthetic curve should align 100% with the real equity curve. If it doesn’t, then there is a problem. This is another reason to start with a simple trading strategy.
Take a look here where I print out the Synthetic Equity curve on a daily basis and compare it with the end result of the analysis.
Synth. matches Reality
Now let’s see if it worked.
Testing with Synth. Equity Curve Trading Turned ON!
The equity curves are very similar. However, there is a difference and this is caused by how one re-enters after trading is turned back on. In this version I tested waiting for a new trade signal which might take a few days. You could re-enter in three different ways:
Automatically enter synthetic position on the next bar’s open
Wait for a new trade signal
Enter immediately if you can get in at a better price
Using the 10% Ret. and 15% Rec. algorithm didn’t help at all. What if we test 10% and 10%.
10% Ret. and 10% Rec.
Now that performed better – more profit and less draw down. Now that I have the synthetic engine working on simple algorithms we can do all sorts of equity curve analysis. In the next installment in this series I will make sure the TS-18 Synthetic Engine can handle more complicated entry and exit algorithms. I have already tested a simple longer term trend following strategy on a medium sized portfolio and the synthetic engine worked fine. The retracement/recovery algorithm at 10%/15% did not work and I will go into the “whys” in my next post.
Simple Donchian on a one contract basis. $100 Commission/slippage. Tested from 2000 thru December 31, 2021. Do you see why most trend followers failed after the 2008 monstrous year. Many funds caught the 2008 move and more funds were added soon thereafter. Promises of similar performance came to fruition in 2011. This kept much of the “new money” on the board. However, reality set in and weak handed funds left for greener pastures. Those that stuck it out were rewarded in 2014. The trend drought of 2014 -2019 eroded most of the confidence in managed futures. The rationalization that limited resources would eventually rise in price sounded good initially, but then fell on deaf ears after months of draw down. Well known CTAs and hedge funds shut their doors forever. The long awaited promise of 2008 came in the form of a pandemic – but it was too late. Maybe now the deluge that ended the drought will persevere (hopefully not in the form of a pandemic) into the future. Prices do not need to rise endlessly, but they need to move one direction or another without many hiccups.
Simple Donchian Caught Most of the Commodities Up Moves
Which Sectors Pushed this Curve through the Roof
These reports were generated by my Python based Trading Simula-18 using Pinnacle continuous data – rollover triggered by date. This is my new sector analysis report where I graph the last four years performance. The tabular data is for the entire 21 year history. The best sectors were energy, grains, financials and metals. Lumber was extraordinary
Here is the module for TS-18. There is a little extra code to keep track of sectors in case you want to limit sector exposure. However, this code takes every trade on a one contract basis. This code reflects my latest version of TS-18, which will be released shortly.
# Define Long, Short, ExitLong and ExitShort Levels - mind your indentations buyLevel = highest(myHigh,40,curBar,1) shortLevel = lowest(myLow,40,curBar,1) longExit = lowest(myLow,20,curBar,1) shortExit = highest(myHigh,20,curBar,1) ATR = sAverage(myTrueRange,30,curBar,1) stopAmt = 2000/myBPV
This system has been around for several years. Its based on the belief that fund managers start pouring money into the market near the end of the month and this creates momentum that lasts for just a few days. The original system states to enter the market on the close of the last bar of the day if the its above a certain moving average value. In the Jaekle and Tomasini book, the authors describe such a trading system. Its quite simple, enter on the close of the month if its greater than X-Day moving average and exit either 4 days later or if during the trade the closing price drops below the X-Day moving average.
EasyLanguage or Multi-Charts Version
Determining the end of the month should be quite easy -right? Well if you want to use EasyLanguage on TradeStation and I think on Multi-Charts you can’t sneak a peek at the next bar’s open to determine if the current bar is the last bar of the month. You can try, but you will receive an error message that you can’t mix this bar on close with next bar. In other words you can’t take action on today’s close if tomorrow’s bar is the first day of the month. This is designed, I think, to prevent from future leak or cheating. In TradeStation the shift from backtesting to trading is designed to be a no brainer, but this does provide some obstacles when you only want to do a backtest.
LDOM function – last day of month for past 15 years or so
So I had to create a LastDayOfMonth function. At first I thought if the day of the month is the 31st then it is definitely the last bar of the month. And this is the case no matter what. And if its the 30th then its the last day of the month too if the month is April, June, Sept, and November. But what happens if the last day of the month falls on a weekend. Then if its the 28th and its a Friday and the month is blah, blah, blah. What about February? To save time here is the code:
// 29th of the month and a Friday if theDayOfMonth = 29 and theDayOfWeek = 5 then endOfMonth = True; // 30th of the month and a Friday if theDayOfMonth = 30 and theDayOfWeek = 5 then endOfMonth = True; // 31st of the month if theDayOfMonth = 31 then endOfMonth = True; // 30th of the month and April, June, Sept, or Nov if theDayOfMonth = 30 and (theMonth=4 or theMonth=6 or theMonth=9 or theMonth=11) then endOfMonth = True; // 28th of the month and February and not leap year if theDayOfMonth = 28 and theMonth = 2 and not(isLeapYear) then endOfMonth = True; // 29th of the month and February and a leap year or 28th, 27th and a Friday if theMonth = 2 and isLeapYear then Begin If theDayOfMonth = 29 or ((theDayOfMonth = 28 or theDayOfMonth = 27) and theDayOfWeek = 5) then endOfMonth = True; end; // 28th of the month and Friday and April, June, Sept, or Nov if theDayOfMonth = 28 and (theMonth = 4 or theMonth = 6 or theMonth = 9 or theMonth =11) and theDayOfWeek = 5 then endOfMonth = True; // 27th, 28th of Feb and Friday if theMonth = 2 and theDayOfWeek = 5 and theDayOfMonth = 27 then endOfMonth = True; // 26th of Feb and Friday and not LeapYear if theMonth = 2 and theDayOfWeek = 5 and theDayOfMonth = 26 and not(isLeapYear) then endOfMonth = True; // Memorial day adjustment If theMonth = 5 and theDayOfWeek = 5 and theDayOfMonth = 28 then endOfMonth = True; //Easter 2013 adjustment If theMonth = 3 and year(d) = 113 and theDayOfMonth = 28 then endOfMonth = True; //Easter 2018 adjustment If theMonth = 3 and year(d) = 118 and theDayOfMonth = 29 then endOfMonth = True;
if endOfMonth and c > average(c,movAvgPeriods) then Buy("BuyDay") this bar on close;
If C <average(c,movAvgPeriods) then Sell("MovAvgExit") this bar on close; If BarsSinceEntry=4 then Sell("4days") this bar on close;
Last Day Of Month Function and Strategy
All the code is generic except for the hard code for days that are a consequence of Good Friday.
All this code because I couldn’t sneak a peek at the date of tomorrow. Here are the results of trading the ES futures sans execution costs for the past 15 years.
Last Day Of Month Buy If C > 50 Day Mavg
What if it did the easy way and executed the open of the first bar of the month.
If c > average(c,50) and month(d) <> month(d of tomorrow) then buy next bar at open;
If barsSinceEntry >=3 then sell next bar at open;
If marketPosition = 1 and c < average(c,50) then sell next bar at open;
Buy First Day Of Month
First Day of Month If C > 50 Day Mavg
The results aren’t as good but it sure was easier to program.
TradingSimula-18 Version
Since you can use daily bars we can test this with my TradingSimula-18 Python platform. And we will execute on the close of the month. Here is the snippet of code that you have to concern yourself with. Here I am using Sublime Text and utilizing their text collapsing tool to hide non-user code:
Small Snippet of TS-18 Code
This was easy to program in TS-18 because I do allow Future Leak – in other words I will let you sneak a peek at tomorrow’s values and make a decision today. Now many people might say this is a huge boo-boo, but with great power comes great responsibility. If you go in with eyes wide open, then you will only use the data to make things easier or even doable, but without cheating. Because you are only going to cheat yourself. Its in your best interest do follow the rules. Here is the line that let’s you leak into the future.
If isNewMonth(myDate[curBar+1])
The curBar is today and curBar+1 is tomorrow. So I am saying if tomorrow is the first day of the month then buy today’s close. Here you are leaking into the future but not taking advantage of it. We all know if today is the last day of the month, but try explaining that to a computer. You saw the EasyLanguage code. So things are made easier with future leak, but not taking advantage of .
Here is a quick video of running the TS-18 Module of 4 different markets.
Many Times It Takes Multiple Tools to Get the Job Done
Just like a mechanic, a Quant needs tools to accomplish many programming tasks. In this post, I use a toolbox to construct an EasyLanguage function that will test a date and determine if it is considered a Holiday in the eyes of the NYSE.
Why a Holiday Function?
TradeStation will pump holiday data into a chart and then later go back and take it out of the database. Many times the data will only be removed from the daily database, but still persist in the intraday database. Many mechanical day traders don’t want to trade on a shortened holiday session or use the data for indicator/signal calculations. Here is an example of a gold chart reflecting President’s Day data in the intra-day data and not in the daily.
Holiday Data Throws A Monkey Wrench Into the Works
This affects many stock index day traders. Especially if automation is turned on. At the end of this post I provide a link to my youTube channel for a complete tutorial on the use of these tools to accomplish this task. It goes along with this post.
First Get The Data
I searched the web for a list of historical holiday dates and came across this:
Historic List of Holidays and Their Dates
You might be able to find this in a easier to use format, but this was perfect for this post.
Extract Data with Beautiful Soup
Here is where Python and the plethora of its libraries come in handy. I used pip to install the requests and the bs4 libraries. If this sounds like Latin to you drop me an email and I will shoot you some instructions on how to install these libraries. If you have Python, then you have the download/install tool known as pip.
Here is the Python code. Don’t worry it is quite short.
url = 'http://www.market-holidays.com/' page = requests.get(url) soup = BeautifulSoup(page.text,'html.parser') print(soup.title.text) all_tables = soup.findAll('table') #print (all_tables) print (len(all_tables)) #print (all_tables[0]) print("***") a = list() b = list() c = list() #print(all_tables[0].find_all('tr')[0].text) for numTables in range(len(all_tables)-1): for rows in all_tables[numTables].find_all('tr'): a.append(rows.find_all('td')[0].text) b.append(rows.find_all('td')[1].text)
for j in range(len(a)-1): print(a[j],"-",b[j])
Using Beautiful Soup to Extract Table Data
As you can see this is very simple code. First I set the variable url to the website where the holidays are located. I Googled on how to do this – another cool thing about Python – tons of users. I pulled the data from the website and stuffed it into the page object. The page object has several attributes (properties) and one of them is a text representation of the entire page. I pass this text to the BeautifulSoup library and inform it to parse it with the html.parser. In other words, prepare to extract certain values based on html tags. All_tables contains all of the tables that were parsed from the text file using Soup. Don’t worry how this works, as its not important, just use it as a tool. In my younger days as a programmer I would have delved into how this works, but it wouldn’t be worth the time because I just need the data to carry out my objective; this is one of the reasons classically trained programmers never pick up the object concept. Now that I have all the tables in a list I can loop through each row in each table. It looked liker there were 9 rows and 2 columns in the different sections of the website, but I didn’t know for sure so I just let the library figure this out for me. So I played around with the code and found out that the first two columns of the table contained the name of the holiday and the date of the holiday. So, I simply stuffed the text values of these columns in two lists: a and b. Finally I print out the contents of the two lists, separated by a hyphen, into the Interpreter window. At this point I could simply carry on with Python and create the EasyLanguage statements and fill in the data I need. But I wanted to play around with Excel in case readers didn’t want to go the Python route. I could have used a powerful editor such as NotePad++ to extract the data from the website in place of Python. GREP could have done this. GREP is an editor tool to find and replace expressions in a text file.
Use Excel to Create Actual EasyLanguage – Really!
I created a new spreadsheet. I used Excel, but you could use any spreadsheet software. I first created a prototype of the code I would need to encapsulate the data into array structures. Here is what I want the code to look like:
Arrays: holidayName[300](""),holidayDate[300](0);
holidayName[1]="New Year's Day "; holidayDate[1]=19900101;
Code Prototype
This is just the first few lines of the function prototype. But you can notice a repetitive pattern. The array names stay the same – the only values that change are the array elements and the array indices. Computers love repetitiveness. I can use this information a build a spreadsheet – take a look.
Type EasyLanguage Into the Columns and Fill Down!
I haven’t copied the data that I got out of Python just yet. That will be step 2. Column A has the first array name holidayName (notice I put the left square [ bracket in the column as well). Column B will contain the array index and this is a formula. Column C contains ]=”. Column D will contain the actual holiday name and Column E contains the ; These columns will build the holidayName array. Columns G throuh K will build the holidayDates array. Notice column H equals column B. So whatever we do to column B (Index) will be reflected in Column H (Index). So we have basically put all the parts of the EasyLanguage into Columns A thru K.
Excel provides tools for manipulating strings and text. I will use the Concat function to build my EasyLanguage. But before I can use Concat all the stuff I want to string together must be in a string or text format. The only column in the first five that is not a string is Column B. So the first thing I have to do is convert it to text. First copy the column and paste special as values. Then go to your Data Tab and select Text To Columns.
Text To Columns
It will ask if fixed width or delimited – I don’t think it matters which you pick. On step 3 select text.
Text To Columns – A Powerful Tool
The Text To Columns button will solve 90% of your formatting issues in Excel. Once you do this you will notice the numbers will be left justified – this signifies a text format. Now lets select another sheet in the workbook and past the holiday data.
Copy Holiday Data Into Another Spreadsheet
New Year's Day - January 1, 2021 Martin Luther King, Jr. Day - January 18, 2021 Washington's Birthday (Presidents' Day) - February 15, 2021 Good Friday - April 2, 2021 Memorial Day - May 31, 2021 Independence Day - July 5, 2021 Labor Day - September 6, 2021 Thanksgiving - November 25, 2021 Christmas - December 24, 2021 New Year's Day - January 1, 2020 Martin Luther King, Jr. Day - January 20, 2020 Washington's Birthday (Presidents' Day) - February 17, 2020 Good Friday - April 10, 2020 Memorial Day - May 25, 2020
Holiday Output
Data Is In Column A
Text To Columns to the rescue. Here I will separate the data with the “-” as delimiter and tell Excel to import the second column in Date format as MDY.
Text To Columns with “-” as the delimiter and MDY as Column B Format
Now once the data is split accordingly into two columns with the correct format – we need to convert the date column into a string.
Convert Date to a String
Now the last couple of steps are really easy. Once you have converted the date to a string, copy Column A and past into Column D from the first spreadsheet. Since this is text, you can simply copy and then paste. Now go back to Sheet 2 and copy Column C and paste special [values] in Column J on Sheet 1. All we need to do now is concatenate the strings in Columns A thru E for the EasyLanguage for the holidayName array. Columns G thru K will be concatenated for the holidayDate array. Take a look.
Concatenate all the strings to create the EasyLanguage
Now create a function in the EasyLanguage editor and name it IsHoliday and have it return a boolean value. Then all you need to do is copy/paste Columns F and L and the data from the website will now be available for you use. Here is a portion of the function code. Notice I declare the holidayNameStr as a stringRef? I did this so I could change the variable in the function and pass it back to the calling routine.
holidayName[1]="New Year's Day "; holidayDate[1]=19900101; holidayName[2]="Martin Luther King, Jr. Day "; holidayDate[2]=19900115; holidayName[3]="Washington's Birthday (Presidents' Day) "; holidayDate[3]=19900219; holidayName[4]="Good Friday "; holidayDate[4]=19900413; holidayName[5]="Memorial Day "; holidayDate[5]=19900528; holidayName[6]="Independence Day "; holidayDate[6]=19900704; holidayName[7]="Labor Day "; holidayDate[7]=19900903; holidayName[8]="Thanksgiving "; holidayDate[8]=19901122; holidayName[9]="New Year's Day "; holidayDate[9]=19910101; holidayName[10]="Martin Luther King, Jr. Day "; holidayDate[10]=19910121; holidayName[11]="Washington's Birthday (Presidents' Day) "; holidayDate[11]=19910218;
// There are 287 holiays in the database. // Here is the looping mechanism to compare the data that is passed // to the database
vars: j(0); IsHoliday = False; For j=1 to 287 Begin If testDate = holidayDate[j] - 19000000 then Begin holidayNameStr = holidayName[j] + " " + numToStr(holidayDate[j],0); IsHoliday = True; end; end;
A Snippet Of The Function - Including Header and Looping Mechanism
This was a pretty long tutorial and might be difficult to follow along. If you want to watch my video, then go to this link.
I created this post to demonstrate the need to have several tools at your disposal if you really want to become a Quant programmer. How you use those tools is up to you. Also you will be able to take bits and pieces out of this post and use in other ways to get the data you really need. I could have skipped the entire Excel portion of the post and just did everything in Python. But I know a lot of Quants that just love spreadsheets. You have to continually hone your craft in this business. And you can’t let one software application limit your creativity. If you have a problem always be on the lookout for alternative platforms and/or languages to help you solve it.
Clenow’s algorithm is definitely an indicator for the current State of Trend Following (SOTF). However, the 3 X ATR trailing stop mechanism actually dampens the profit/draw down ratio. Take a look at this chart.
Battle of Titans
All Trend Following mechanisms have a very common thread in their entry mechanisms. The thing that separates them is the preemptive exit. Do you allow the the algorithm to exit on a purely market defined method or do you overlay trade management? Here the best approach was to let the Bollinger Band system run unfettered; even though it seems somewhat illogical. Many times trade management actually increases draw down. Is there a solution? What about this – keep risk down by trading a small, yet diverse portfolio of high volume markets and overlay it with a stock index mean reversion algo. Take a look.
Bollinger Marries ES Reversion
Should’ve, Would’ve , Could’ve.
This could be scaled up. The mean reversion helped lift the chart out of the flat and draw down periods of late. However, the smaller portfolio did OK during this time period too! Can four or five high volume markets replicate a much larger portfolio? All tests were carried out with TradingSimula18 – the software that comes with my latest book.
Backtesting with [Trade Station,Python,AmiBroker, Excel]. Intended for informational and educational purposes only!
Get All Five Books in the Easing Into EasyLanguage Series - The Trend Following Edition is now Available!
Announcement – A Trend Following edition has been added to my Easing into EasyLanguage Series! This edition will be the fifth and final installment and will utilize concepts discussed in the Foundation editions. I will pay respect to the legends of Trend Following by replicating the essence of their algorithms. Learn about the most prominent form of algorithmic trading. But get geared up for it by reading the first four editions in the series now. Get your favorite QUANT the books they need!
The Foundation Edition. The first in the series.
This series includes five editions that covers the full spectrum of the EasyLanguage programming language. Fully compliant with TradeStation and mostly compliant with MultiCharts. Start out with the Foundation Edition. It is designed for the new user of EasyLanguage or for those you would like to have a refresher course. There are 13 tutorials ranging from creating Strategies to PaintBars. Learn how to create your own functions or apply stops and profit objectives. Ever wanted to know how to find an inside day that is also a Narrow Range 7 (NR7?) Now you can, and the best part is you get over 4 HOURS OF VIDEO INSTRUCTION – one for each tutorial.
Hi-Res Edition Cover
This book is ideal for those who have completed the Foundation Edition or have some experience with EasyLanguage, especially if you’re ready to take your programming skills to the next level. The Hi-Res Edition is designed for programmers who want to build intraday trading systems, incorporating trade management techniques like profit targets and stop losses. This edition bridges the gap between daily and intraday bar programming, making it easier to handle challenges like tracking the sequence of high and low prices within the trading day. Plus, enjoy 5 hours of video instruction to guide you through each tutorial.
Advanced Topics Cover
The Advanced Topics Edition delves into essential programming concepts within EasyLanguage, offering a focused approach to complex topics. This book covers arrays and fixed-length buffers, including methods for element management, extraction, and sorting. Explore finite state machines using the switch-case construct, text graphic manipulation to retrieve precise X and Y coordinates, and gain insights into seasonality with the Ruggiero/Barna Universal Seasonal and Sheldon Knight Seasonal methods. Additionally, learn to build EasyLanguage projects, integrate fundamental data like Commitment of Traders, and create multi-timeframe indicators for comprehensive analysis.
Get Day Trading Edition Today!
The Day Trading Edition complements the other books in the series, diving into the popular approach of day trading, where overnight risk is avoided (though daytime risk still applies!). Programming on high-resolution data, such as five- or one-minute bars, can be challenging, and this book provides guidance without claiming to be a “Holy Grail.” It’s not for ultra-high-frequency trading but rather for those interested in techniques like volatility-based breakouts, pyramiding, scaling out, and zone-based trading. Ideal for readers of the Foundation and Hi-Res editions or those with EasyLanguage experience, this book offers insights into algorithms that shaped the day trading industry.
Trend Following Cover.
For thirty-one years as the Director of Research at Futures Truth Magazine, I had the privilege of collaborating with renowned experts in technical analysis, including Fitschen, Stuckey, Ruggiero, Fox, and Waite. I gained invaluable insights as I watched their trend-following methods reach impressive peaks, face sharp declines, and ultimately rebound. From late 2014 to early 2020, I witnessed a dramatic downturn across the trend-following industry. Iconic systems like Aberration, CatScan, Andromeda, and Super Turtle—once thriving on robust trends of the 1990s through early 2010s—began to falter long before the pandemic. Since 2020 we have seen the familiar trends return. Get six hours of video instruction with this edition.
Pick up your copies today – e-Book or paperback format – at Amazon.com
