(866)-482-2177

sales@alphatheory.com

REQUEST A DEMO

SYSTEM REQUIREMENTS


Please note the following System Requirements. Further, please limit the number of open applications (particularly price streaming applications) while logged in to Alpha Theory™.


Recommended System Specifications
Processor: Dual Core or Quad-Core 2.4GHz or faster
RAM: 4GB+
Browser: Google Chrome 30+
Screen Resolution: 1280 x 1024 or greater
Internet Access: Business Class High-Speed


Minimum System Requirements
Processor: Intel Pentium-M 2.0Ghz or equivalent
RAM: 2GB+
Browser: Google Chrome, Mozilla Firefox, Internet Explorer 9+ (without Compatibility View), Safari
Screen Resolution: 1024 x 768 or greater
Internet Access: High-Speed

Subscribe to Alpha Theory content

Alpha Theory Blog - News and Insights

13 posts categorized "Superforecasting"

November 02, 2018

Better Predictions Lead to Better Returns (Garbage In – Garbage Out)

 

Not every position is better off following the model position size (optimal) determined by Alpha Theory. However, the times when optimal outperforms are associated with higher forecast accuracy. If you put better forecasts into the model, the model does better. This is a straightforward demonstration of Garbage In-Garbage Out.

 

Correlation of Actual and Forecasted Returns for Positions that Under/Overperformed Optimal

 

Screen Shot 2018-11-02 at 12.09.17 PM 

Models are data dependent. When good data is input in the model, the model has higher predictive power. Bad data in and, well, it doesn’t have the same edge. The correlations hold if we expand into quartiles.

 

Correlation of Actual and Forecasted Returns for Positions that Under/Overperformed Optimal

 

Screen Shot 2018-11-02 at 12.10.57 PM

Picture0 

And largely holds for deciles:

 

Correlation of Actual and Forecasted Returns for Positions that Under/Overperformed Optimal

 

Screen Shot 2018-11-02 at 4.11.15 PM 

Picture1

 

What you’ll notice is that the correlation overall between actual and forecasted returns is fairly small with the highest decile showing an 18% correlation. Even though the signal is faint, it is strong enough to power a model that produces positive returns.

 

As the data shows, it is worth taking the time to measure your historical forecasting skill. If you have positive forecasting skill, then a simple model can dramatically improve results.

 

October 01, 2018

8 Mistakes Money Managers Make

 

The 8 Mistakes Money Managers Make was an article I released almost five years ago. As much as I would love to report that its publication has cured all ills, the mistakes are still prevalent today. I believe its time for a second circulation of “the mistakes.”

 

The process to fix the mistakes is easy. The human behavior change is hard. I hope that this article will both show some easy to implement processes and start conversations about how to change your own behavior and that of your firm.

 

We’re here to help. We have solutions and services designed to help managers who want to improve and outpace their competition stuck in a “pre-Moneyball” mindset.

 

DOWNLOAD NOW

 

Watch Alpha Theory Lunch and Learn Webinar on 8 Mistakes Money Managers Make:

 

WATCH RECORDING

 

August 17, 2018

Signs of Seasonality

 

One of the members of our Customer Success team was wondering about the difficulty of getting client attention at the end of August. We ran an analysis to try and answer the question, “how active are our clients by month?” We used price target updates, logins, and trades per month as a proxy for investor activity.

 

Signs of seasonality1

 

August was definitely the softest month, but clients weren’t as “checked out” as we expected. We hypothesized that the peak periods would be during earnings season and troughs will be after earnings. Here’s the rub, they’re in the same month. The end of second-quarter earnings season and the before school vacation season are in the same month.

 

To remedy this fact, we created periods starting on the 15th of each month (i.e. August 15th to September 15th). This allows us to catch each earnings season as its own isolated period. Here are the results:

 

Signs of seasonality2(final)

 

There is clear seasonality. The post Q2 earnings season is 2.5 standard deviations from the norm. I suspect that if we broke this down into two-week tranches, we would have seen even more pronounced deviation from August 15th to August 31st.

 

As expected, the Post Earnings Season cohort’s activity was light at 0.7 standard deviations below normal activity, while the During Earnings cohort was busy (+0.8).

 

One of my favorite parts of working at Alpha Theory is that we have a long series of robust, structured data that allows us to ask and answer interesting questions. If you would like to be able to do the same, the first step is collecting and maintaining well-structured data. Then you can ask interesting questions like “what season do we make our most money?”, “who is the best forecaster on my team?”, “how often do stocks go below our risk targets?”, etc.

 

If you would like to learn more about how we can help. Contact us at

 

(866)-482-2177  

sales@alphatheory.com  

 

July 05, 2018

More Evidence of Manager Skill – Concentration Manifesto Continued

In preparation for a webinar we hosted about the Concentration Manifesto on June 21st we had a client question using batting average (win percentage) as a way of measuring skill. Their contention was that high batting averages do not always result in great returns, because a low hit rate with high asymmetry (lots of upside with little downside) can be even more profitable than predictable low returners.

 

Screen Shot 2018-07-05 at 12.25.30 PM

 

To analyze that point, we looked at the Return on Invested Capital (ROIC) by the same buckets we analyzed batting average.

 

Chart2

 

You can see that there is a similar correlation. Assets that are sized the largest had the highest return on invested capital. Said another way, the Top 5 positions went up an average of 12.1% while the portfolio as a whole went up 8.4% (for shorts, went down 8.4%).  That’s 50% better!

Chart3

 

We then analyzed the distribution of returns by bucket.

 

Again, you can see a predictive quality in manager position sizing. Stocks that have smaller positions have a wider distribution of returns (and more downside). The smallest positions had the most upside, but what we see in the data is that managers can forecast more volatile positions and size accordingly.

 

To finish the point, I’ll pull up a chart from the original Concentration Manifesto where we use our clients’ forecasted returns (Expected Return) and created two portfolios. One with the 20 best forecasted returns and then the rest. In the graph below, you can see that managers can forecast which assets will have the best returns. This shows skill not associated just with positions sizing, but on forecasting price return.

Chart4

 

There is very little question that our clients demonstrate skill. There is also very little question that they have mitigated a substantial portion of their skill by having too many positions.

 

May 03, 2018

Positive Skew…Part 2 – Maybe It’s Not So Bad for Active Managers After All

In my last post, I discussed the negative impact of positive skew for active managers. Basically, that more than 50% of all stocks in a given market underperform the average because there are stocks that go up more than 100% but no stocks that go down more than 100%. This means that if you pick a random portfolio of stocks from the market, you have a greater than 50% chance of underperforming the market because most portfolios will not hold those few stocks that went up more than 100%.

 

Because of the popularity of the last post and TV appearance, we spent time digging further into the data to answer questions posed by readers and viewers. We noticed that there was a tendency for the returns between the average stock return and the index return to be different.

 

And that is the problem with using the average stock return as the hurdle for funds. Investors are not measured against the average stock return, they’re measured against the benchmark, typically the S&P 500. Most indexes are market cap weighted, meaning that the index return and the average stock return are generally different.

 

In the example below, we’ve taken the current S&P 500 constituents and calculated their return since the beginning of 2012 and compared that to an average return (Equal Weighted) and the actual return of the S&P 500. The S&P 500 over that period was up 136% vs 175% for the average stock (this isn’t a perfect analysis because the constituents in the portfolio changed over that time but it is an approximation).

 

Positive Skew-part2

 

The graph above shows the distribution of individual stock returns over that period. You can see the outliers that pull the average stock return (red line) up to a point where 63% of individual securities underperform the average of 176%. But the S&P 500 was up 136% (green line) over that period so only 51% of stocks underperformed the benchmark. Pretty much a coin flip.

 

We brought positive skew up with Andrew Wellington at Lyrical Asset Management. They have done some great analysis comparing the top 1000 stocks by market cap in the US to the S&P 500 each year going back to 1998.

 

Chart2

Source: FactSet and Lyrical Asset Management

 

As you can see in the chart above, the average stock beating the S&P 500 index is a coin flip. For the past 20 years, the likelihood of any individual stock beating the S&P 500 in any given year is 50.2%. If I build random portfolios using the Top 1000 stocks in the US, there is a high likelihood that the portfolio return will be close to the S&P 500 return.

 

Some years are clearly better than others. ’98 and ’99 were horrible stock picking years. If you didn’t own the few stocks that had meteoric rises, you had a high likelihood of underperforming the S&P 500. ’01 and ’02 were good stock picking years. Over 60% of stocks beat the index.

 

What this means, is that any given fund’s batting average should be compared to the batting average of the universe of stocks compared to the benchmark. A 54% batting average in ’98 is heroic, in ’03, 54% is just inline. Take a look at 2017. It was the 3rd hardest stock picking environment in the last 20 years using this metric.

 

But what about other indices? Thankfully, our friend Julien Messias from Quantology Capital Management has done the analysis (1999-2014) comparing the S&P 500 and Russell 2000. Below are thoughts from Julien on the topic:

 

The Russell 2000 components returns exhibit a much more leptokurtic distribution (fat-tailed) than S&P 500, meaning that you have a huge part of the index’s components suffering from huge loss (or even bankruptcies), with an average of more than 60% of the components underperforming the index performance and 2% of the components with huge performance (more than 500% per year). The performance of the index is therefore pulled up by those latter 2%.

Assuming a stock-picker operates at random to choose its investment within the index universe, this means that his performance should be closer to the median performance of the components, than to the index performance itself. Therefore, given that the median performance is almost always lower than the index performance (see chart below), an investor in Russell 2000 securities is very likely to underperform and very unlikely to outperform.

The S&P 500 distribution is much more mean-centered, with very shallow/thin tails, meaning that the average stock picker is much more likely to generate a performance close to the index performance (graph from Lyrical AM) and less likely to underperform.

 

Chart3

Source: Quantology CM

 

The Russell 2000 index more apparently displays the impacts of positive skew because it is less impacted by a contribution of a few very large companies. AAPL, MSFT, GOOG, AMZN make up 12.2% of the S&P 500 while the Russell 2000’s top 4 positions make up 1.7% of the index. The result is that the average of all stocks in the Russell 2000 is much closer to the Russell 2000 index return than the average of all stocks in the S&P 500 (recall the large difference from the 2012 to 2018 analysis that showed the S&P 500 return was 136% vs 175% average of all stocks).

 

This means that the index chosen as the benchmark for your fund has a profound impact on your ability to beat it. More specifically, the probability of beating the S&P 500 with a random portfolio is 50%, for the Russell 2000, it’s 42%.

 

There has been quite a bit of press regarding positive skew. It’s a great conversation but, for the average fund that is measured against the S&P 500, the impact is overblown. Almost every investor is compared against a benchmark. I recommend that you dig a layer into your benchmark and measure its positive skew, the likelihood of beating the average stock return, the likelihood of beating the index return, and compare your hit rate against the hit rate each year to know how difficult or easy it was for you on any given year.

 

Quantology Capital Management Russell 2000 and S&P 500 Analysis:

 ­ Screen Shot 2018-05-03 at 10.03.58 AM Screen Shot 2018-05-03 at 10.05.52 AM

Does not include management fees

Data is cleaned from index turnover, with updates every year

April 06, 2018

Positive Skew is Negative for Active Managers

 

Let’s play a game. In this game, there are 10 random poker chips in a bag. 9 of these chips will give you a return between -8% and +8% on the money that you bet. The 10th coin will give you a 100% return. The distribution of returns for this game has a positive skew.

 

Screen Shot 2018-04-06 at 9.29.11 AM
 

If offered to put money down on this proposition you would take it because you would expect a 10% return if you could play the game over and over.

 

Now let’s add a wrinkle. Your goal isn’t just to make a positive return, you have to beat the bag. The bag puts 10% of their money on each chip and pulls them all. Voila, a 10% return. One last wrinkle, you can only pick one chip at a time.

 

How many times out of 10 would you beat the bag? Only 1 in 10. 90% of the time you would lose to the bag. It doesn’t matter if we expand the number of chips as long as the bag maintains the same positive skew (we could increase the to 100 chips and you get to pick 10, 100 chips and you pick 1000, etc.)

 

By now, you’ve probably guessed that the bag is the market, the chips are stocks, and you are, well, you. This is the game we play when trying to beat an index. True, you can be better than the market at figuring out the good chips but given that initial conditions for a random game means you lose 9 out of 10 times, it’s really hard to beat the market. Add fees and the likelihood of beating the market goes down even further.

 

Positive Skewness has gotten a decent amount of press over the past year because of the championing of JB Heaton who wrote a paper1 researching the impacts of positive skew on manager underperformance. Heaton’s paper is similar to research from Dr. Richard Shockley in 19982. See below for an article written by Bloomberg News on the topic.

 

Picture1

Source: Bloomberg News (“Lopsided Stocks and the Math Explaining Active Manager Futility” by Oliver Renick)

 

Given that many of the conversations active managers have today revolve around active versus passive, “positive skew” should be top of mind. This is my push to increase awareness.

 

Given that active managers can’t change market skew, what should we do? We could measure skill in a different way. Let’s say I want to measure a manager skill. If I take all of the stocks of the markets they’re investing in and then randomly build 100,000 portfolios with the same number of securities as the manager. I can then plot where that manager falls on the distribution and give them a Z-Score for how far away from the norm they are. I could do the same thing for hedge funds by randomly buying and selling securities in the same universe as the investor.

 

I’m not saying that this excuses active managers from underperforming passive strategies, but it should at least be a more realistic assessment of their skill. My hope is that positive skew becomes just as common an explanation as fees when discussing active manager underperformance. Only by knowing the causes, will we be able to make changes that allow active managers to outperform.

 

 “Why Indexing Works” by JB Heaton, Nicholas Polson, and Jan Witte

2  “Why Active Managers Underperform the S&P 500: The Impact of Size and Skewness” published in the inaugural issue of the Journal of Private Portfolio Management. One of the original authors of the study is Richard Shockley.

Related paper: “Do stocks outperform treasury bills?” Hendrik Bessembinder of Arizona State University

 

March 12, 2018

Capital Allocators Podcast with Ted Seides: Moneyball for Managers

 

Learn how to enhance your investment results in this great podcast from Ted Seides and his guests, Clare Flynn Levy from Essentia Analytics and Cameron Hight from Alpha Theory.

This conversation covers the founding of these two respective businesses, the mistakes portfolio managers commonly make, the tools they employ to help managers improve, and the challenges they face in broader adoption of these modern tools. The good news is the clients of Essentia Analytics and Alpha Theory have demonstrated improvement in their results after employing these techniques. If you ask Clare and Cameron, you may develop a whole new appreciation about the potential for active management going forward.

 

LevyHight-FINAL

 

By creating a disciplined, real-time process based on a decision algorithm with roots in actuarial science, physics, and poker, Alpha Theory takes the guessing out of position sizing and allows managers to focus on what they do best – picking stocks.

In this podcast, you will learn how Alpha Theory allows Portfolio Managers convert their implicit assumptions into an explicit decision-making process. 

 

To learn how this method could be applicable to your decision-making process:

 

LISTEN NOW

 


 

 

March 02, 2018

Size-Based Batting - A Different Perspective on Stock Selection

 

How do you determine if an investor is a good stock picker? One commonly used measure is to count the number of positions that make money (winners) divided by the total number of positions. This metric is commonly called a Batting Average, analogizing stock picking with baseball hit-rates.

The problem with Batting Average is that several inconsequential positions that lose money can really bring down the total. We saw this with our clients. They have historically outperformed other funds (every year for the past six) but have a batting average, adjusted for the move in the bench, of only 51%.

We decided to take a different approach and measure the total exposure of positions that made money versus the total gross exposure of the fund. For instance, if 60% of a fund made money on an alpha-adjusted basis and the fund was 120% gross exposed, then the fund had a Sized-Based Batting Average of 50% (60/120).

Our clients had a Sized-Based Batting Average of 54% versus the non-sized based average of 51%. That means that our clients were good at selecting investments and at sizing them, but they were harming their overall returns with small losing investments.

Alpha-Adjusted Batting Average1

 

Screen Shot 2018-03-02 at 10.09.00 AM

 

In the table above, Size-Based Batting, while not perfectly consistent, is generally better from year-to-year for our clients (exceptions being 2012 and 2015).

We’ve performed other analyses that have proved this point, specifically that our clients’ positions under 1% dramatically underperform the rest of the portfolio, but Sized-Based Batting presents a compelling way to highlight the “small position” issue (see the “Concentration Manifesto” for other issues with small positions).

In our profession, it is incredibly difficult to detangle skill from luck and, as cathartic as it would just rely on returns, returns are actually negatively correlated with next year’s returns for most funds (i.e. funds that outperform in year N have a higher likelihood underperforming in year N+1 – there are multiple research sources that analyze mean reversion in funds, here is one).

Sized-Based Batting is a nice addition to the allocator’s tool bag for finding managers with stock picking skill. In much the same way, managers should use Sized-Based Batting as a way to highlight their strengths and compare it to traditional Batting Average as a way to potentially point out weaknesses.

 

1 S&P 500 for US securities and MSCI WEI for non-US securities

2 Why is “All Time” so low compared to each year? Reason #1: There are many more observations in the more recent years which skew the overall results to be more similar to the more recent years. Reason #2: There were many assets that were losers over “All Time” while being winners for multiple years (small win in 2015, a small win in 2016, big loss in 2017 = 2 winning period vs 1 losing but a loser in the All-Time bucket).

 

 

February 07, 2018

Alpha Theory Case Study: Top Performing Funds of 2017

Alpha Theory’s clients have historically outperformed (see 2017 Year in Review from last month), but 2017 was special as our most active client was also the 2nd best performing equity fund. We have worked with them since their launch, and their focus on discipline and process is a testament to how to build a fund. If you would like to learn more about the client, their challenges, their solution, and the data supporting their process, check out the Case Study.

 

DOWNLOAD NOW

 

 

December 15, 2017

Superforecasting for Investors: Part 2

Alpha Theory hosted a book club on December 6th with portfolio managers, analysts, and allocators coming together to discuss “Superforecasting” by Phil Tetlock. We were lucky enough to have a Superforecaster, Warren Hatch, moderate and perform forecasting exercises with the group. We spent 2 hours together and only scratched the surface on applying Superforecasting to investing.

 

Here are a few key takeaways:

1. COMMON ATTRIBUTES OF SUPERFORECASTERS:

INTELLIGENCE: Above average but genius isn’t required

QUANTITATIVE: Not only understand math but apply it to everyday life

FOXES, NOT HEDGEHOGS: Speak in terms of possibilities, not absolutes

INTELLECTUALLY HUMBLE: Understand the limits of their knowledge

SYSTEM 2 DRIVEN: Use the logic-driven instead of instinct-driven portion of their brain

DO NOT BELIEVE IN FATALISM: Life is not preordained

CONSTANTLY REFINE: Make frequent small updates to their forecast based on new information (but not afraid to make big changes when warranted)

COUNTERFACTUALS: Believe that history is one of many possible paths that could have occurred

OUTSIDE VIEW: Incorporate the internal and external views

GROWTH MINDSET: CONSTANTLY SEARCH FOR WAYS TO IMPROVE THEIR FORECASTING PROCESS

 

2. IDENTIFYING TALENT: There are identifiable attributes that can be used in hiring and have a profound impact on forecasting skill

 

Active Open Mindedness*

   image from alphatheory.typepad.com

Fluid Intelligence*

image from alphatheory.typepad.com

 

* At a prior book club, we measured participants and the results showed they had the attributes of Superforecasters with high Active Open-Mindedness (3.99 out of 5) and high Fluid Intelligence (8 out of 10 – this is the highest score that the Good Judgment  folks have seen).

Active Open Mindedness (i) and Fluid Intelligence (a) are two measurable traits that managers can use to select talent. In the chart below, the improvement impact of the definable attributes equates to about 40% of their forecasting skill over standard forecasts.

image from alphatheory.typepad.com

3. DEVIL’S ADVOCATE: Firms should appoint a Devil’s Advocate for each investment to expand critical thinking (someone to ask the question, “I see your downside is $40. How is that if the 52-Week Low is $22 and the trough multiple would put it at $25?”)

 

4. OUTSIDE VIEW: Firms should require an Outside View for every investment idea (“While everyone I’ve spoken to says this deal will close, only 20% of deals with one party under SEC investigation close.”)

 

5. REFINEMENT: New information should always be incorporated in forecast (think Bayesian).

 

6. TEAMS MAKE BETTER FORECASTS: Team dialog generally improves forecasting accuracy.

 

7. FORECAST CULTURE: Firms should embrace “forecast” as part of their vernacular and conversations should revolve around how information impacts the forecast.

 

8. MEASURE TO BE BETTER: We all forecast, but we rarely measure. That fact needs to change if we really want to improve.

 

9. CLUSTERING: Break complex topics into individual components that are better able to be forecast and use the combination of the smaller forecasts to forecast the more complex. (ie. Will AAPL break $200 is a complex forecast that can be broken down into Will iPhone X ship more than 400m units? / Will Samsung’s technology outpace Apple’s? / etc.)

 

10. INDEXING: Individual clustering questions can be weighted to come up with a forecast for the complex topic instead of using simple equal weighting.

 

11. DIVERSITY OF FORECASTS MATTER: Forecasts made from similar perspectives are less accurate than those made from multiple perspectives (see Boosting below).

 

12. BOOSTING: If you have three forecasters with different perspectives that all arrive at a 70% probability of an event occurring then the actual probability is greater than 70%.

 

13. GISTING: We didn’t get to spend much time here, but the idea is that complex subjects, reports, presentations, etc. can be distilled down into gists that the team votes on and refines into supergist. Full understanding is never just quantitative or qualitative. Superforecasting is quantitative. Supergisting attempts to provide the qualitative piece. 

 

14. HYBRID FORECASTING COMPETITION: IARPA, the defense agency that sponsored the forecasting tournament that launch the Superforecasters (Good Judgment) is sponsoring a new Man+Machine Forecasting Tournament. For those interested in Forecasting and Machine Learning, this is your spot: https://www.iarpa.gov/index.php/research-programs/hfc