TradeStation Portfolio Maestro

Backtesting Portfolios

There are three types of backtesting that can be performed in Portfolio Maestro: Standard, Optimization, and Walk-Forward (also known as an out-of-sample testing). By default, the backtest type is set to Standard.

• Standard - Uses the strategy inputs and ranking values as set in the strategy group.
• Optimization - Runs multiple tests by varying specified strategy input and ranking values over a defined range.  The Min and Max setting determines the range of values to test and the Step setting determines how much to increment the value for each test.

• Full Set - Tests every possible combination of strategy input and ranking values that are set to vary.  

• Genetic - Uses a genetic algorithm to minimize the total number of tests required to produce a statistically valid backtest.

• Walk-Forward - Also known as out-of-sample testing, which reduces the chance to fit the price curve when designing a strategy. This is done by having two period types: optimization and holding. Selected strategy inputs are optimized during the optimization period and are fixed during the holding period. This process is repeated again starting from the last date of the previous optimization period until the date range for testing ends.

Walk-Forward analysis allows you to say "what if I had optimized a certain parameter in the past at a certain frequency, using a certain look-back period (both customizable by your defined inputs). To do this, Portfolio Maestro generates a set of out-of-sample data, and trades the optimal past parameter forward as if the user had been optimizing in the past.

 Refer to # Input Combinations in the lower right corner of Backtest Portfolio window to review the number of tests are scheduled to be performed based on your current backtest settings.

Performing a Backtest

Select one of the following for a list of steps for a specific backtest type:

Backtest a Portfolio using Standard

Backtest a Portfolio using Optimization

Backtest a Portfolio using Walk Forward

Backtesting Speed Considerations

The following parameters are primary contributors to the time of the test and the size of the database required. These parameters can be adjusted in combination to achieve a valid study outcome without running into huge time required or database size limitations. As a rule of thumb, think in terms of total number of bars required. That will be based on:

(# of intervals per day) * (# days in history) * (# of symbols) * (# of input combinations)

Example

In the example above for the 10-min data # of intervals per day = 39 (390 minutes during a normal trading day), # days in history = 2500 (assuming 250 trading days per year), # of symbols = 100 and # of input combinations = 1000. Therefore, the total number of bars required is equal to 390*2500*100*1000 = 97,500,000,000.