(866)-482-2177

sales@alphatheory.com

REQUEST A DEMO

SYSTEM REQUIREMENTS


Please note the following System Requirements. Further, please limit the number of open applications (particularly price streaming applications) while logged in to Alpha Theory™.


Recommended System Specifications
Processor: Dual Core or Quad-Core 2.4GHz or faster
RAM: 4GB+
Browser: Google Chrome 30+
Screen Resolution: 1280 x 1024 or greater
Internet Access: Business Class High-Speed


Minimum System Requirements
Processor: Intel Pentium-M 2.0Ghz or equivalent
RAM: 2GB+
Browser: Google Chrome, Mozilla Firefox, Internet Explorer 9+ (without Compatibility View), Safari
Screen Resolution: 1024 x 768 or greater
Internet Access: High-Speed

Alpha Theory Blog - News and Insights

November 30, 2016

Using Analytics to Improve Investment Process

By Cameron Hight and Justin Harris

 

Data is taking a precedence in modern life as a necessity for those who want to improve. With specific relevance to asset management, finding insightful feedback in data is where managers can use unbiased facts to flesh out the inefficiencies in their investment process. As more and more managers rely on data to inform their investment process, the cost of not doing so increases. We, at Alpha Theory, have spent considerable time over the past year working with clients to help them hone their process, based on insights from data. Here are a few examples:

PROBABILITY AND FORECAST ANALYSIS

Alpha Theory uses analyst forecasts to optimize portfolio position sizing. Managers must therefore know what confidence they can put on analysts’ research, because optimizing on bad research could be deleterious to returns.  Said another way, managers need to know who to trust.

As a starting point, the average analyst in our system forecasts that they’ll make money on their investments 75% of the time compared to their actual success rate of 53%. Additionally, they forecast upside returns of 43% compared to their realized returns of 26%. Clearly there is a pervasive, systematic overconfidence. With that perspective, you can evaluate your own team.

Probability Analysis. One of the first areas to evaluate is the forecasted probability of making money compared to the actual percentage of investments that were profitable. In the graph below, we show a representative comparison of forecast versus actual success rates. As a manager, this is excellent information because in one glance, you see which analyst is most accurate and who has the most bias. The actionable information would be requesting that Jimmy and Matt cap their forecasted success percentage at 50% and reviewing specific names with Jim to see why he has such dramatic bias.

Screen Shot 2016-12-01 at 8.49.59 AM

Additional analyses could include long/short and sector breakout or alpha-based analysis. These additional insights may help clarify the analyst’s differences. The point is that you ask questions based on empirical data that help your analysts improve while, at the same time, giving the portfolio manager the real-data to back up hunches of analyst bias.

Performance Analysis. In the graph below, we show how price forecasts (bars) compare to actual outcomes (underlay). We see that, while Jim Braddock had a disappointingly low success rate (graph above), his price forecasts were excellent. In fact, because the asymmetry was positive, his ideas were net profitable even though only 25% of them made money. From a manager’s perspective, encouraging Jim to make more realistic probability forecasts, gives you data that could dramatically improve fund performance.

Screen Shot 2016-12-01 at 8.50.16 AM

Simple Graphs, Profound Insights. These two simple graphs provide many insights, as illustrated above. A few others to show the power of the picture:

  1. Matt Doherty’s names should be given lower confidence given that he only makes money in 50% of his names and he loses more on his losers than he wins on his winners.
  2. Ahtray Dahurt’s forecasts should receive higher confidence as his reward and risk price targets and probabilities are in line with actual results.
  3. Jim Braddock’s forecasts are net profitable, but his reward probabilities are over inflated. There is an opportunity to profit from his ability to call names with extreme upside returns, but, in the near term, scaling back confidence while working with Jim to improve his batting or at least decrease his forecast probabilities would improve the forecast process.

These are a few of the many insights that Alpha Theory provides to clients to help them become better investment managers. Data provides insights that lead to action that result in an improved process. This chain of improvement is the benefit of capturing and analyzing data. Until we “see the data”, the answer may not be intuitive. This is why it is so important to create a data driven approach, focused on process improvements, that allow you to keep pace with the rapidly evolving data-driven world.

October 07, 2016

LUCK VS. SKILL IN INVESTING (Alpha Theory Book Club with Michael Mauboussin)

On October 3rd, Alpha Theory hosted the “Success Equation” book club with the author, Michael Mauboussin, and 35 PMs, analysts, and allocators. Mr. Mauboussin led the discussion on an array of investing topics centered around the central theme of luck and skill in our profession.

Major takeaways:

    1. Investing is dominated by luck because investor skill level has risen to the point where the market is largely efficient

    2. Managers acknowledge the role of luck, but underestimate it

    3. Process improvements are the easiest way for investors to improve performance

The discussion began by exploring how to determine the influence of skill and luck on an endeavor. The measurements are far from precise, but there are some heuristics that give us strong clues.

In the continuum below, games that are dominated by luck, like blackjack and roulette, are on the left side, and games like chess, that are dominated by skill, are on the right side.

 

SE

 

Investing: More Skill or Luck?

We asked the attendees where investing fell on the continuum above. The average answer fell marginally closer to the skill end of the spectrum (near hockey). According to Mauboussin, investing is largely dominated by luck and is only slightly more skill-inclusive than gambling. Skill influences success, but it does not dominate. A monkey throwing darts can beat a sophisticated investor in any given year due to luck because the large number of skilled investors (high intellect, high work ethic, extensive training and experience) has resulted in markets that are largely efficient.

Skill vs Process Improvement

In the case of investing, skill has to be looked at in two dimensions, absolute and relative. Relative skill is key in the investment world, where there has been a dramatic narrowing in skill differences between investors. Because investing is dominated by luck, skill improvements make only small marginal differences in the probability of winning.  The saving grace for investors is that the average investor’s process is far from optimized and small improvements can have meaningful impacts on the probability of winning.

It is important to understand what makes something procedural and another skillful. In blackjack, no skill improvement will increase your chance of winning (assuming one considers card-counting “cheating” or not part of the “legal” rules of the game). On the other hand, process improvements (when to hit/stay/double down) can minimize your losses. You might ask, “why isn’t knowing when to hit/stay/double down a skill?” The answer is because it is formulaic (procedural): when the dealer is showing X and you are showing Y, you always do Z.

Said another way, no matter how good you get, you’re only going to win about 50% of the time. Compare this to chess on the skill side of the spectrum. A player with a 2600 ELO rating will beat a player with a 1600 rating 99.7% of the time. Improvements in skill (like deliberate practice memorizing optimal responses to your opponent’s opening) that improve a player’s ELO rating will increase his probability of winning.

In investing, building a model, making price forecasts, assessing business outlooks, grading the quality of management teams, and evaluating prospects of new products are all skills. Process in investing includes activities such as following a checklist of criteria that should be met for every investment, creating systems for measuring idea quality, tying idea quality to position size, adhering to portfolio rules (liquidity constraints, maximum sector exposures, max drawdown limits, etc.), and analyzing the efficacy of the process to refine it over time. The low-hanging fruit for investors comprise evolutions in process and, according to Mauboussin, are where they should be focusing their improvement efforts, given the heavy luck component at play.

Process enhancements should focus on those that are (1) analytical, (2) behavioral, and (3) organizational.  Alpha Theory speaks to the analytical improvement, where betting one’s edge intelligently is critical.  In terms of managing one’s organization, optimal collaboration is key.  This works best when (1) the size of team is larger, (2) cognitive diversity of the team is greater, and (3) management of the team offers [a.] dependability and [b.] “psychological safety” (fostering an environment where participants have no reason to fear sharing candid views).  Furthermore, the best leaders keep to an agenda, suppress their own points of view, and indeed successfully elicit the team-members’ perspectives – even those of the introverts.  (Alpha Theory can help here as well!)

IQ vs. RQ

Speaking of cognitive diversity and decision processes in investing, it is important to be aware of differences between IQ (intelligence quotient) and RQ (rationality quotient). Most people make the association between smart investors and high-IQ intellectual competency.  But in fact the best type of mental model that leads to appropriate investment decisions is RQ-oriented (really, the ability to make reasoned, judicious decisions efficiently and without equivocation in a fluid environment like the stock market).  Furthermore, one applied psychology study (see Bibliography below) found a surprisingly low correlation coefficient between IQ and RQ.  The investment industry may err on the side of hiring high-IQ analysts when it should be seeking higher RQ as a starting point – although there is not a ready test for RQ as of yet.

Ecology of Decision Rules

The stock market is a classic adaptive complex system – one where there can be ‘diversification breakdowns’ that result in the wisdom of crowds working until it does not work.  Diversity equates to different menus of decision rules each participant has, but when an asset price rises, many participants drop their own rules and conform to a single one, which breaks down diversity.  This tends to be a non-linear function with a ‘snap!’ phase transition, where reflexivity is defined.  But then diversity is restored when overcrowding corrects itself.

Ways to Improve Forecasting

Several process improvement steps come directly from “Success Equation” and are called suggestions to improve the “art of good guesswork”:

    1. Understand where you are on the luck-skill continuum

    2. Assess sample size, significance, and swans

    3. Always consider a null hypothesis

    4. Think carefully about feedback and rewards

    5. Make use of counterfactuals

    6. Develop aids to guide and improve your skills

    7. Have a plan for strategic interactions

    8. Make reversion to the mean work for you

    9. Develop useful statistics

    10. Know your limitations

Resources

SLIDES: Here is a link to a set of slides very similar to the one’s Mr. Mauboussin used and a video of him discussing “Success Equation”.

 

BASE RATE BOOK: A hot topic was the use of base rates to improve forecasting and decision making. Without a doubt, this is one of the best and easiest ways to improve your process. You can check out Mauboussin’s “The Base Rate Book” here and get a primer on how to implement it.

 

BIBLIOGRAPHY: One of the amazing things about Mr. Mauboussin is the catalog of referenceable articles, studies, and books in his head. Here is a list of all of those he referenced during the Book Club:

“Even God Would Get Fired As An Active Investor” by Wesley Gray

“On the Impossibility of Informationally Efficient Markets” by Sanford Grossman and Joseph Stiglitz

 “Agent Based Models” by Blake LeBaron

David Swensen quoted in “Asset Allocation or Alpha?” by Mimi Lord

“Vicarious Learning, Undersampling of Failure, and the Myths of Management” by Jerker Denrell

“The Three Rules” by Michael Raynor and Mumtaz Ahmed

“Luck versus Skill in the Cross-Section of Mutual Fund Returns” by Eugene Fama and Kenneth French

“Should Airplanes Be Flying Themselves” by Vanity Fair

“The Base Rate Book ” by Michael Mauboussin

Good Judgement Project  

Solomon Asch Experiments    

Greg Berns – Emory University

“What intelligence tests miss” by Keith Stanovich

 “Comprehensive Assessment of Rational Thinking” by Keith Stanovich

Cognitive Reflection Test (“Poor Man’s Test for RQ”) by Shane Frederick

Freestyle Chess

“What we miss when we judge a decision by the outcome?” by Francesca Gino

“Deep Survival” by Laurence Gonzolez

CFA Institute survey late 2008/09 – Quants vs. Fundamentals

“Use Cognitive Diversity to get the most of the Workplace” by Mark Miller

“Peak” by Anders Ericsson – Theory of 10,000 Hours book

“Robert’s Rules of Order” by Henry M. Robert (No one can speak 2x on a topic until everyone has had a chance to speak at least 1x)

“Forms Follows Functions” by Michael Mauboussin

"IQ vs. RQ" by Michael Mauboussin and Dan Callahan

 

Co Authored by: Cameron Hight & Dana Lambert