(866)-482-2177

sales@alphatheory.com

REQUEST A DEMO

SYSTEM REQUIREMENTS


Please note the following System Requirements. Further, please limit the number of open applications (particularly price streaming applications) while logged in to Alpha Theory™.


Recommended System Specifications
Processor: Dual Core or Quad-Core 2.4GHz or faster
RAM: 4GB+
Browser: Google Chrome 30+
Screen Resolution: 1280 x 1024 or greater
Internet Access: Business Class High-Speed


Minimum System Requirements
Processor: Intel Pentium-M 2.0Ghz or equivalent
RAM: 2GB+
Browser: Google Chrome, Mozilla Firefox, Internet Explorer 9+ (without Compatibility View), Safari
Screen Resolution: 1024 x 768 or greater
Internet Access: High-Speed

Subscribe to Alpha Theory content

Alpha Theory Blog - News and Insights

31 posts categorized "Risk Management"

January 31, 2017

ALPHA THEORY - 2016 YEAR IN REVIEW

Client Outperformance

For the fifth year in a row (as far back as we have data) our clients have outperformed the HFRI Equity Hedge Index. To date, our clients’ compound return is 20% greater than the index.

1

As good as our clients are, they would have been even better if they followed the optimal position sizes they built inside of Alpha Theory:

2

On average, Alpha Theory suggests a lower gross exposure. So, to compare on an apples-to-apples basis, we look at Return on Invested Capital. For 2016, the Optimal ROIC was 13.3% versus 6.5% actual. That’s a difference of 6.8%.

Let’s put that difference into perspective. Our clients manage over $100B using Alpha Theory. On an ROIC basis, 6.8% of additional return on $100B is $6.8B. Assuming 20% performance fees, our managers left almost $1.4B of income on the table.

In 2016, 84% of our clients would have performed better if they would have followed optimal position sizing.

Betting the Forecasting Edge

Lastly, 2016 was the best year on record for the correlation between our clients’ forecasts and actual returns. The correlation between expected and actual returns was 0.19 for 2016. While this may seem low, one would expect a correlation near zero if selected randomly. For every year since 2012, with the exception of 2015, the correlation between expected and actual returns has been positive.

We believe this is a strong indication of predictive power in analysts’ forecasts. If analysts’ forecasts were random, then optimal position size would not beat actual returns with such regularity.

There are many ways to try and improve but few are as easy as creating a discipline around position sizing. The evidence is clear, if a firm has any edge, then creating a repeatable process to bet that edge is the difference between good and great.

Additional Portfolio Metrics

Screen Shot 2017-01-31 at 6.56.58 PM

 

 

December 28, 2016

2017 NEW YEAR’S RESOLUTIONS

By Emma Vosburg and Cameron Hight

 

Every year, people make New Year’s resolutions and every year, people break them. Creating positive habits that reinforce resolutions is the difference between the people that keep their resolutions and those that break them.

Creating process that works for you is the key to forming habits that lead to accomplishing your goals. You must commit to the process. For investors, a lack of systematic investment process means it will be difficult to consistently outperform your competitors. Alpha Theory is process in a box!

Let us help you create a winning process and build the habits necessary to fulfill your New Year’s resolutions. Let’s look at a list of possible Alpha Theory New Year’s goals:

CREATE PROCESS ----> BUILD HABITS ----> ACHIEVE GOALS

Screen Shot 2016-12-28 at 9.27.05 AM

Why make an Alpha Theory New Year’s resolution? Alpha Theory’s research not only suggests that adoption of the application by itself leads to improved performance, but actual usage further enhances results. In the table below, we show that clients who are more process oriented (as measured by having price targets, frequency of review, and diligence at updating position size based on their forecasts) outperformed our clients who were less diligent.

2

A systematic approach to accomplishing goals is valuable in every aspect of life. In 2017, create process that builds habits and allows you to achieve your goals. No excuses!

 

November 30, 2016

Using Analytics to Improve Investment Process

By Cameron Hight and Justin Harris

 

Data is taking a precedence in modern life as a necessity for those who want to improve. With specific relevance to asset management, finding insightful feedback in data is where managers can use unbiased facts to flesh out the inefficiencies in their investment process. As more and more managers rely on data to inform their investment process, the cost of not doing so increases. We, at Alpha Theory, have spent considerable time over the past year working with clients to help them hone their process, based on insights from data. Here are a few examples:

PROBABILITY AND FORECAST ANALYSIS

Alpha Theory uses analyst forecasts to optimize portfolio position sizing. Managers must therefore know what confidence they can put on analysts’ research, because optimizing on bad research could be deleterious to returns.  Said another way, managers need to know who to trust.

As a starting point, the average analyst in our system forecasts that they’ll make money on their investments 75% of the time compared to their actual success rate of 53%. Additionally, they forecast upside returns of 43% compared to their realized returns of 26%. Clearly there is a pervasive, systematic overconfidence. With that perspective, you can evaluate your own team.

Probability Analysis. One of the first areas to evaluate is the forecasted probability of making money compared to the actual percentage of investments that were profitable. In the graph below, we show a representative comparison of forecast versus actual success rates. As a manager, this is excellent information because in one glance, you see which analyst is most accurate and who has the most bias. The actionable information would be requesting that Jimmy and Matt cap their forecasted success percentage at 50% and reviewing specific names with Jim to see why he has such dramatic bias.

Screen Shot 2016-12-01 at 8.49.59 AM

Additional analyses could include long/short and sector breakout or alpha-based analysis. These additional insights may help clarify the analyst’s differences. The point is that you ask questions based on empirical data that help your analysts improve while, at the same time, giving the portfolio manager the real-data to back up hunches of analyst bias.

Performance Analysis. In the graph below, we show how price forecasts (bars) compare to actual outcomes (underlay). We see that, while Jim Braddock had a disappointingly low success rate (graph above), his price forecasts were excellent. In fact, because the asymmetry was positive, his ideas were net profitable even though only 25% of them made money. From a manager’s perspective, encouraging Jim to make more realistic probability forecasts, gives you data that could dramatically improve fund performance.

Screen Shot 2016-12-01 at 8.50.16 AM

Simple Graphs, Profound Insights. These two simple graphs provide many insights, as illustrated above. A few others to show the power of the picture:

1. Matt Doherty’s names should be given lower confidence given that he only makes money in 50% of his names and he loses more on his losers than he wins on his winners.

2. Ahtray Dahurt’s forecasts should receive higher confidence as his reward and risk price targets and probabilities are in line with actual results.

3. Jim Braddock’s forecasts are net profitable, but his reward probabilities are over inflated. There is an opportunity to profit from his ability to call names with extreme upside returns, but, in the near term, scaling back confidence while working with Jim to improve his batting or at least decrease his forecast probabilities would improve the forecast process.

These are a few of the many insights that Alpha Theory provides to clients to help them become better investment managers. Data provides insights that lead to action that result in an improved process. This chain of improvement is the benefit of capturing and analyzing data. Until we “see the data”, the answer may not be intuitive. This is why it is so important to create a data driven approach, focused on process improvements, that allow you to keep pace with the rapidly evolving data-driven world.

 

October 07, 2016

LUCK VS. SKILL IN INVESTING (Alpha Theory Book Club with Michael Mauboussin)

On October 3rd, Alpha Theory hosted the “Success Equation” book club with the author, Michael Mauboussin, and 35 PMs, analysts, and allocators. Mr. Mauboussin led the discussion on an array of investing topics centered around the central theme of luck and skill in our profession.

Major takeaways:

    1. Investing is dominated by luck because investor skill level has risen to the point where the market is largely efficient

    2. Managers acknowledge the role of luck, but underestimate it

    3. Process improvements are the easiest way for investors to improve performance

The discussion began by exploring how to determine the influence of skill and luck on an endeavor. The measurements are far from precise, but there are some heuristics that give us strong clues.

In the continuum below, games that are dominated by luck, like blackjack and roulette, are on the left side, and games like chess, that are dominated by skill, are on the right side.

 

SE

 

Investing: More Skill or Luck?

We asked the attendees where investing fell on the continuum above. The average answer fell marginally closer to the skill end of the spectrum (near hockey). According to Mauboussin, investing is largely dominated by luck and is only slightly more skill-inclusive than gambling. Skill influences success, but it does not dominate. A monkey throwing darts can beat a sophisticated investor in any given year due to luck because the large number of skilled investors (high intellect, high work ethic, extensive training and experience) has resulted in markets that are largely efficient.

Skill vs Process Improvement

In the case of investing, skill has to be looked at in two dimensions, absolute and relative. Relative skill is key in the investment world, where there has been a dramatic narrowing in skill differences between investors. Because investing is dominated by luck, skill improvements make only small marginal differences in the probability of winning.  The saving grace for investors is that the average investor’s process is far from optimized and small improvements can have meaningful impacts on the probability of winning.

It is important to understand what makes something procedural and another skillful. In blackjack, no skill improvement will increase your chance of winning (assuming one considers card-counting “cheating” or not part of the “legal” rules of the game). On the other hand, process improvements (when to hit/stay/double down) can minimize your losses. You might ask, “why isn’t knowing when to hit/stay/double down a skill?” The answer is because it is formulaic (procedural): when the dealer is showing X and you are showing Y, you always do Z.

Said another way, no matter how good you get, you’re only going to win about 50% of the time. Compare this to chess on the skill side of the spectrum. A player with a 2600 ELO rating will beat a player with a 1600 rating 99.7% of the time. Improvements in skill (like deliberate practice memorizing optimal responses to your opponent’s opening) that improve a player’s ELO rating will increase his probability of winning.

In investing, building a model, making price forecasts, assessing business outlooks, grading the quality of management teams, and evaluating prospects of new products are all skills. Process in investing includes activities such as following a checklist of criteria that should be met for every investment, creating systems for measuring idea quality, tying idea quality to position size, adhering to portfolio rules (liquidity constraints, maximum sector exposures, max drawdown limits, etc.), and analyzing the efficacy of the process to refine it over time. The low-hanging fruit for investors comprise evolutions in process and, according to Mauboussin, are where they should be focusing their improvement efforts, given the heavy luck component at play.

Process enhancements should focus on those that are (1) analytical, (2) behavioral, and (3) organizational.  Alpha Theory speaks to the analytical improvement, where betting one’s edge intelligently is critical.  In terms of managing one’s organization, optimal collaboration is key.  This works best when (1) the size of team is larger, (2) cognitive diversity of the team is greater, and (3) management of the team offers [a.] dependability and [b.] “psychological safety” (fostering an environment where participants have no reason to fear sharing candid views).  Furthermore, the best leaders keep to an agenda, suppress their own points of view, and indeed successfully elicit the team-members’ perspectives – even those of the introverts.  (Alpha Theory can help here as well!)

IQ vs. RQ

Speaking of cognitive diversity and decision processes in investing, it is important to be aware of differences between IQ (intelligence quotient) and RQ (rationality quotient). Most people make the association between smart investors and high-IQ intellectual competency.  But in fact the best type of mental model that leads to appropriate investment decisions is RQ-oriented (really, the ability to make reasoned, judicious decisions efficiently and without equivocation in a fluid environment like the stock market).  Furthermore, one applied psychology study (see Bibliography below) found a surprisingly low correlation coefficient between IQ and RQ.  The investment industry may err on the side of hiring high-IQ analysts when it should be seeking higher RQ as a starting point – although there is not a ready test for RQ as of yet.

Ecology of Decision Rules

The stock market is a classic adaptive complex system – one where there can be ‘diversification breakdowns’ that result in the wisdom of crowds working until it does not work.  Diversity equates to different menus of decision rules each participant has, but when an asset price rises, many participants drop their own rules and conform to a single one, which breaks down diversity.  This tends to be a non-linear function with a ‘snap!’ phase transition, where reflexivity is defined.  But then diversity is restored when overcrowding corrects itself.

Ways to Improve Forecasting

Several process improvement steps come directly from “Success Equation” and are called suggestions to improve the “art of good guesswork”:

    1. Understand where you are on the luck-skill continuum

    2. Assess sample size, significance, and swans

    3. Always consider a null hypothesis

    4. Think carefully about feedback and rewards

    5. Make use of counterfactuals

    6. Develop aids to guide and improve your skills

    7. Have a plan for strategic interactions

    8. Make reversion to the mean work for you

    9. Develop useful statistics

    10. Know your limitations

Resources

SLIDES: Here is a link to a set of slides very similar to the one’s Mr. Mauboussin used and a video of him discussing “Success Equation”.

 

BASE RATE BOOK: A hot topic was the use of base rates to improve forecasting and decision making. Without a doubt, this is one of the best and easiest ways to improve your process. You can check out Mauboussin’s “The Base Rate Book” here and get a primer on how to implement it.

 

BIBLIOGRAPHY: One of the amazing things about Mr. Mauboussin is the catalog of referenceable articles, studies, and books in his head. Here is a list of all of those he referenced during the Book Club:

“Even God Would Get Fired As An Active Investor” by Wesley Gray

“On the Impossibility of Informationally Efficient Markets” by Sanford Grossman and Joseph Stiglitz

 “Agent Based Models” by Blake LeBaron

David Swensen quoted in “Asset Allocation or Alpha?” by Mimi Lord

“Vicarious Learning, Undersampling of Failure, and the Myths of Management” by Jerker Denrell

“The Three Rules” by Michael Raynor and Mumtaz Ahmed

“Luck versus Skill in the Cross-Section of Mutual Fund Returns” by Eugene Fama and Kenneth French

“Should Airplanes Be Flying Themselves” by Vanity Fair

“The Base Rate Book ” by Michael Mauboussin

Good Judgement Project  

Solomon Asch Experiments    

Greg Berns – Emory University

“What intelligence tests miss” by Keith Stanovich

 “Comprehensive Assessment of Rational Thinking” by Keith Stanovich

Cognitive Reflection Test (“Poor Man’s Test for RQ”) by Shane Frederick

Freestyle Chess

“What we miss when we judge a decision by the outcome?” by Francesca Gino

“Deep Survival” by Laurence Gonzolez

CFA Institute survey late 2008/09 – Quants vs. Fundamentals

“Use Cognitive Diversity to get the most of the Workplace” by Mark Miller

“Peak” by Anders Ericsson – Theory of 10,000 Hours book

“Robert’s Rules of Order” by Henry M. Robert (No one can speak 2x on a topic until everyone has had a chance to speak at least 1x)

“Forms Follows Functions” by Michael Mauboussin

"IQ vs. RQ" by Michael Mauboussin and Dan Callahan

 

Co Authored by: Cameron Hight & Dana Lambert

September 12, 2016

The Day the Music Died

For the past few years, I have had a significant edge in my fantasy football leagues because of my pre-draft preparation.  Each year, I would go online to do my preparation before the draft and pull data from multiple sources and look around for novel analytical approaches to player selection which I could combine with the data I was able to source.  Unfortunately, it looks like the years of my fantasy football spreadsheet’s domination has come to an end. I knew this day would come. Each year the data and analysis I was running sourcing seemed to get better and better. This year, I found 95% of my bespoke spreadsheet already prepared by someone else online at: http://apps.fantasyfootballanalytics.net/.  Undoubtedly, several of my competitors found it too.

Key attributes of my fantasy football sheet:

1. Improvement over Average (also called Value over Replacement) – this is by far the most important attribute. It basically states how much better any particular player is to the average player that will be drafted at the same position. Without IOA, the players with the highest potential points will always pop to the top, which is inaccurate because you can’t fill a team with just one position. For example, a QB may be forecast to score 350 points over the year and a WR is forecast to score 325. On that basis, the QB looks like the favorite. But the average drafted QB will score 280 and the average drafted WR will score 180 points. That means that the WR will add 145 points (325-180) against other teams vs the QB which will only add 70 points (350-280).

2. Multiple Sources – I pull projections from multiple sites to create a “wisdom of crowds” improvement.

3. Forecast Risk – measure standard deviation of projections from multiple sites to see how likely a player was to hit the average I was using in my model.

4. Average Draft Position – pulled Average Draft Position from multiple sources to compare my ranking to what was likely to happen.

5. Over/Undervalued to Average Draft Position – this allowed me to see if a player that my sheet really liked, could be selected in a later round because folks on average were drafting him later.

6. Dropoff – used to quickly see how big the difference is between the player and the next best player at the same position.

1

Now let’s compare to FFAnalytics.net:

2

The only thing my sheet has is a Strength of Schedule adjustment (#7) and I can just add that in after I download the data.

My reasons for writing this blog are two-fold. One, I want everyone to feel sorry for me that my edge is gone. Two, we all should recognize that with each passing year, data becomes easier to access and manipulate. If there are things in your business you consider your “edge” (competitive advantages), do a careful assessment of the landscape and forecast the probability that it can be undermined by data ubiquity, data analytics tools, or machine learning (artificial intelligence). Old businesses are being disintermediated in a matter of a few years by the likes of Google, Alibaba, Amazon, Uber, AirBNB, etc. Asset managers would be well served to take a hard look at what parts of their spreadsheets/processes and analytics are no longer novel and which are likely to persist and contribute as their true “edge”.    Without an edge in your processes or analytics, there is a limited likelihood that they will see an “edge” in their returns. 

September 01, 2016

THE CROWDING CONUNDRUM

Crowding has received quite a bit of attention lately due to the big drawdown in crowded names starting in the summer of last year. Because of that drawdown, we’ve seen clients increasingly include crowdedness as a negative risk factor when sizing positions. But how dangerous is crowding?

Proof #1 – Crowdedness (Alpha Theory)

At the behest of Benn Dunn, the head of Alpha Theory’s Investment Process Advisory Practice, we set out to quantify the impact of crowded names in client portfolios1. Here’s the shocking news: they helped. In fact, crowded longs grew 50% faster than the overall S&P 500 since 2011 and longs and shorts outperformed on an alpha basis.

Screen Shot 2016-09-01 at 2.21.31 PM

Our clients are among the best and brightest in the hedge fund space, so we were pleased to see there is a positive correlation between return and position size. In this case, crowding is the best form of “wisdom of crowds”. See below that Crowded Longs returned 154% since 2011 while the S&P 500 was up 85%.

Picture1

Proof #2 – Crowdedness (Novus)

Other sources show similar results. Below is a graph and table from “The Novus “4C” Indices” which shows a dramatic outperformance of Crowded names for funds2

Picture2

CROWDING – THE BULL CASE by Cameron Hight

Crowding outperformance can be interpreted as stock picking skill in its broadest form. If crowdedness is a measure of forecasted idea quality, then our clients did a great job of picking the winners. It suggests that active managers are good at picking a few names they truly believe in. Said another way, they show both selection and sizing skill.

We recognize that our clients are generally superior performers in both stock selection and position sizing. Alpha Theory clients are up 54% since 2012 versus 25% for the HFRI Equity Hedge Index. That being said, they have underperformed the S&P 500’s return of 85%. I believe that the best managers would be able to consistently outperform if they would concentrate on their best ideas.

The problem is that most managers retard returns by building large portfolios that dilute the skill of their best ideas with the luck of the rest. In fact, by distracting themselves with a portfolio of lower quality ideas, they not only drag down performance, they reduce the amount of mental capital they can allocate to their high quality ideas.

CROWDING – THE BEAR CASE by Benn Dunn

While “Hedge Fund Crowding” has often had negative connotations in the popular press, it has also historically been a significant source of alpha for hedge funds piling into the same names.  One explanation is pretty straightforward. When exceptional buyside analysts do deep qualitatively and quantitatively differentiated work, their views get disseminated to the broader hedge fund community through idea dinners, conferences, casual conversations, and quarterly letters. The idea spreads and those names begin to outperform as additional hedge funds crowd into the names and investment theses are borne out. Another reason for outperformance is hedge fund inflows. We saw the impact of inflows into equity hedge funds through the middle of last year.

The flip side of crowding is the narrowness of the exit when everyone heads for the door at the same time.  We saw the first hints of this in certain segments of the high beta growth universe in March/April of 2014 and a bit of replay when the Peoples Bank of China took their first steps towards a currency float last August.  The first quarter of 2016 makes these more recent episodes look trivial by comparison. 

It is now widely acknowledged through 13F filings that significant equity gross exposure came out of hedge funds in Q1 with the most widely held names suffering disproportionate drawdowns relative to what traditional downside risk measures would predict.  It would seem that the tailwind of positive inflows into the equity hedge fund universe is at best abating if not reversing altogether which would indicate that crowding will be a less significant source of alpha in the near future.  Going forward, one of the most important questions an analyst or PM needs to ask when adding a position to the portfolio is “who is the marginal buyer of this stock?”  If the answer is “another hedge fund” then the hurdle rate for getting that name in to the portfolio needs to be much higher.

Conclusion

Is it the great idea alone or the crowding into it that causes outperformance? It is almost certainly a bit of both. In fact, without each other to positively reinforce the other, the outperformance would typically not materialize. However it happens, the positive impacts of crowding and conviction are real and hopefully this article provides some positive counterpoints for a factor that has been largely maligned.

1 The methodology consisted of constructing three mock portfolios, Long (top 10 most crowded longs), Short (top 10 most crowded shorts), Total (top 10 longs and top 10 shorts). The positions were equal weighted. The portfolio was rebalanced once a month at the beginning of the month and represents the time period 02/01/2011 to 08/01/2016. Each portfolio was simulated as a 100% gross exposure portfolio.

2 Crowdedness Index is comprised of the most crowded stocks in hedge funds. A crowding score is assigned to each security in our database and serves as the ranking factor for the Index. The score is based on the number of managers invested in the stock and the liquidity of the hedge funds in each security. The two factors are components in the crowded score. In other words, these stocks are a blend of “illiquid” and “popular." Since they are rather illiquid, when managers try to sell they can drive the price down considerably in a short period of time. But overall, the Index is not a great short candidate since it outperforms the market by a wide margin over the long term.

August 11, 2016

The Power of Process — How Fundamental Investors Benefit from Quantitative Thinking

 

A recap of speech given on August 3rd, 2016 at Evercore ISI Quantitative Symposium

Why do Fundamental Investors need to think more Quantitatively? 90%+ of fundamental managers we’ve interviewed do not have their 5 best ideas as their 5 largest positions. The primary reason for this is:

    1. QUALITY MEASUREMENT: Fundamental investors generally do not have a repeatable process for measuring idea quality

    2. POSITION SIZING: Fundamental investors generally do not have a repeatable process for sizing positions

Quantitative investors “score” or measure the quality of an idea and use that score, in concert with portfolio constraints, to size positions. Most fundamental investors try to do this heuristically and fail. The failure has been overlooked for years because:

    1. CLOSE ENOUGH: Fundamental investors are smart and they can get pretty close in their heads. Yes, there will be big mistakes at times when emotions get in the way, but that’s more the exception than the rule.

    2. WIDE MARGINS: Since the publishing of “Security Analysis” after the Great Depression, fundamental investors have been able to take advantage of “Mr. Market” by holding true to fundamental investing axioms.

With every passing decade, fundamental margins shrink and are quickly approaching a point where “close enough” is no longer sufficient to generate positive returns. Many fundamental investors and allocators are recognizing this trend and seeking out ways to be more precise and maximize their fundamental advantage. Process is the key.

PROOF OF THE NEED FOR CHANGE

MONEYBALL

If you want to see how an industry can be transformed by process, look to the recent revolution in sports. Sports managers are just like great fundamental investors. They try to add great players (investments) to their team (portfolio) to maximize their chance of winning (generating positive alpha). Moneyball created a process around each step by making assumptions explicit and measuring their impact on the desired outcome. It’s not any more complicated than that. This simple concept revolutionized all of sports in a matter of 20 years. Investing is in the early years of Moneyball adoption and, if sports is an apt proxy, it will change rapidly.

EXPERT STUDIES

Over the past 60 years (dating back to Paul Meehl’s “Clinical versus Statistical Prediction” paper), scientists have studied the judgement of experts. There are hundreds of published studies that have a similar theme. Give an expert any and all available data that they want and ask them to make a judgement germane to their field of expertise (ex. Oncologist – how long will a patient live, Parole Board – who is most likely to recidivate, Wine Expert – price of wine at auction, etc.) The one request, is that they tell the scientist which variables are most important in their decision.

The scientist goes off and builds an improper (equal weights all factors) or proper (regressed) model and compares their model to the forecasts of the “experts.” Over the hundreds of expert studies for 60 years, the expert beats the simple model a paltry 6% of the time. And when the expert does forecast more accurately, it is usually by a very small margin.

We are not as good as we think we are at making complicated decisions. But we’re very good at determining the variables that matter. The logical conclusion from those facts is that we need to follow Bob Jones of System Two’s advice (spoke after my presentation at the ISI event with a similar message):

    1. Decompose a complex decision into its critical components

    2. Evaluate each individually

    3. Combine algorithmically1

(1) Weights used are not critical in most cases

EMPIRICAL EVIDENCE

The importance of process is evident in our analysis of client data. Below, we show positions where explicit price targets and probabilities were forecast outperformed those positions without forecasts.

Picture1

In the graph below, we show that clients who are more process oriented (as measured by having price targets, frequency of review, and diligence at updating position size based on their forecasts) outperformed our clients who were less diligent.

Picture2

We also show below, that had our clients followed their own model their performance would have been 13% vs 7%. Our clients know the variables that matter and following their own process would improve outcomes. Sounds just like the conclusions from the Expert Studies mentioned earlier.

Picture3

SO WHAT DO WE PROPOSE?

“Objectivity is gained by making assumptions explicit so that they may be examined and challenged, not by vain efforts to eliminate them from analysis.” – Richards Heuer, Psychology of Intelligence Analysis

I believe the changes required to be more process oriented are completely intuitive to investors but require repetition and time before habits are formed. The catalytic change that ignites the whole process is the simple switch from implicit to explicit assumptions.

Picture4

Step 1: EXPLICIT FORECASTS: Some investors chafe at price targets because they smack of “false precision.”  These investors are missing the point because the key to price targets is not their absolute validity, but their explicit nature which allows for objective conversation about the assumptions that went into them.  Said another way, price targets improve the investment process because they foster great questions and force the team to be able to defend the methodology behind their calculations.

Step 2: EXPECTED RETURN: Apply probabilities to forecasts and convert them into expected returns. In Moneyball, the statistic that proved best at predicting win percentage was On-Base Percentage. In investing, it is Expected Return. It is the underpinning of good decision making in many scientific fields: actuaries, odds makers, poker players, physicists, etc. and is a requirement for making good decisions as an investor.

Step 3: TURN QUALITATIVE INTO QUANTITATIVE: Just like in the Expert Studies example, define the variables that are important in your decisions, analyze them independently, and combine them algorithmically.

Step 4: DEFINE RULES: Every investor has rules that guide their decision making. Make those rules explicit and construct a framework to measure your adherence to them.

Step 5: MAXIMIZE TRANSFER COEFFICIENT: Make sure all of your rules and assumptions are being transferred into the portfolio. To do that, create a model that expresses all of your rules and assumptions as a position size. This allows you to compare your a priori self with your actual decisions. Said another way, it creates an unemotional version of you. Said yet another way, it creates a system that looks at every position brand new every day and asks the question, “if I were investing in this asset for the first time today, what position size would I take?”

Step 6: ANALYZE RESULTS: Once you’ve done the first five steps, you can measure your explicit assumptions and model for correctness.

Step 7: REFINE PROCESS: Take the results from Step 6 and draw conclusions for ways to improve forecasts, inputs, and the model.

Step 8: REPEAT WITH IMPROVED PROCESS

These steps are straightforward. The adoption of this process is critical to success in the future where the edge for fundamental investors has dramatically shrunk. The difference between two equally skilled analytical minds will be the process applied to maximize that analytical prowess. The future of fundamental investing is clear if Moneyball is a true harbinger of things to come. Embrace the benefits of process today and be at the vanguard of investing. Ignore the benefits of process and slowly lose to competitors more adaptable to change. 

 

 

July 18, 2016

Superforecasting – Alpha Theory Book Club

Alpha Theory hosted its first ever book club on July 12th with over 40 portfolio managers, analysts, and allocators coming together to discuss “Superforecasting” by Phil Tetlock. We were lucky enough to have two Superforecasters, Warren Hatch and Steve Roth, moderate and perform forecasting exercises with the group. We spent 2 ½ hours together and only scratched the surface on applying Superforecasting to investing.

Here are a few key takeaways:

1. RAW TALENT: On average, our group had the attributes of Superforecasters with high Active Open-Mindedness (3.99 out of 5) and high Fluid Intelligence (8 out of 10 – this is the highest score that the Good Judgement folks have seen).

Active Open Mindedness

 

 1

Fluid Intelligence

2

2. IDENTIFYING TALENT: There are identifiable attributes that can be used in hiring and have a profound impact on forecasting skill (40% - see chart below).

Screen Shot 2016-07-18 at 4.02.10 PM

3. DEVIL’S ADVOCATE: Firms should appoint a Devil’s Advocate for each investment to expand critical thinking (someone to ask the question, “I see your downside is $40. How is that if the 52-Week Low is $22 and the trough multiple would put it at $25?”)

4. OUTSIDE VIEW: Firms should require an Outside View for every investment idea (“While everyone I’ve spoken to says this deal will close, only 20% of deals with one party under SEC investigation close.”)

5. REFINEMENT: New information should always be incorporated in forecast (think Bayesian).

6. POSTMORTEM: An Accuracy Score should be calculated for every investment and should frame the conversation of “what did we do well?” and “what did we do poorly?”.

7. TEAMS MAKE BETTER FORECASTS: Team dialog generally improves forecasting accuracy.

8. FORECAST CULTURE: Firms should embrace “forecast” as part of their vernacular and conversations should revolve around how information impacts the forecast.

9. MEASURE TO BE BETTER: We all forecast, but we rarely measure. That fact needs to change if we really want to improve.

10. BOOK CLUBS ARE COOL!!!

The October Alpha Theory Book Club topic will be “Success Equation: Untangling Skill and Luck” by Michael Mauboussin. Mr. Mauboussin will moderate and highlight the book’s application to investing. Contact your Alpha Theory representative if interested in attending.

 

June 21, 2016

How Good Are My Analysts? Building a Better Hedge Fund Through Moneyball & Superforecasting

Traditionally, measuring hedge fund analyst skill has been an opaque process mired in ambiguity and subjectivity.  It is often misconstrued and tainted by portfolio manager influence in the form of sizing decisions, liquidity constraints and other non-analyst determinants.  But, in the same way Moneyball revolutionized evaluating baseball player value by prioritizing on-base percentage over batting average, Alpha Theory has distilled the key indicator for predictive aptitude. Alpha Theory invented the Alpha Theory Accuracy Score to introduce radical transparency into the rating of forecasting skill for hedge fund analysts.

P&L is Yesterday’s Batting Average

Using the Moneyball analogy, quantitative disruption of baseball player evaluation changed the way players are paid by isolating the player skill that contributes most to team wins. Using that data, managers now pay athletes in proportion to the amount of that winning skill they individually possess.  As such, the key metric for baseball player value evolved from batting average, to the more predictive on-base percentage, or OBP. 

Specifically, OBP has a 92 percent correlation with runs scored compared to batting’s 81 percent, making it more predictive.  Also, OBP’s 44 percent correlation year-to-year is more persistent than the 32 percent correlation of batting.  The predictive reliability and performance consistency make OBP a superior metric to forecast wins for baseball teams.  OBP’s disruption of batting average is an apt metaphor for the way Alpha Theory’s Accuracy Score will transform analyst ranking and assessment today.      

In 2016, analysts are still primarily rated by the profits and losses their investments generate for the fund, or P&L.  But making money on an investment is a misleading measure of analyst skill.  Beyond its tendency to be distorted by portfolio manager discretion, P&L performance, both good and bad, often masks the integrity and quality of investment processes.  Thus, P&L often misleads portfolio managers into thinking lucky analysts are actually skilled and vice versa.

For example, take these two analysts:

How good is my

Looking at the table above and using P&L to measure skill, Analyst #1 would be exceptional and Analyst #2 would be sub-par.  But Analyst #1 and #2 had the same forecasts, so their forecasting skill is actually identical.  P&L does not translate into forecast skill because analysts do not have ultimate control over position sizing; the portfolio manager does!

More Science, Less Art                                                                                                                         

Inspired by the ideas presented in the groundbreaking book, Superforecasting: The Art and Science of Prediction, Alpha Theory’s Accuracy Score delivers quantitative insight into a qualitative blind spot for portfolio managers.  Authored by Wharton Professor Phillip Tetlock and Dan Gardner in 2015, Superforecasting applies a Brier Score-inspired approach to quantifying predictive skill.  The Brier Score was created by meteorological statistician, Glenn Brier, in 1950 and measures the accuracy of probabilistic outcomes.  Superforecasting applies Brier’s methodology to only binary, or yes/no, outcomes.  

The New Standard

Alpha Theory’s Accuracy Score is an algorithmic solution that measures analysts’ predictive skill over a 0 - 100 percent range, where 100 is the best.  Scores are calculated on a per-forecast basis and then averaged per analyst.  The Accuracy Score algorithm transforms point estimate price targets and probability forecasts into an implied probability distribution, enabling each forecast to be independently scored.  By distributing multi-faceted outcomes across a range of probabilities, the Accuracy Score can measure forecasting skill for any price along the distribution.

The distribution of scores across our Alpha Theory clients is shown below.  The results follow a normal distribution, which further validates the Accuracy Score’s efficacy in rating analysts’ ability to forecast future price movements.

Screen Shot 2016-06-21 at 9.41.12 AM

Good forecasts are the most essential component of fund success and critical when portfolio managers are sizing positions.  Using a data-driven approach to determine which analysts make the best forecasts allows managers to apply those forecasts with greater confidence, leading to better position sizing and superior performance.

The Good Judgement Project

In 2011, the Intelligence Advanced Research Projects Activity, a U.S. government research organization, sponsored a geopolitical forecasting tournament that would span 4 years. The IARPA tournament enlisted tens of thousands of forecasters and solicited more than 1 million forecasts across nearly 500 questions related to U.S. national security.

A group called the Good Judgement Project entered the competition, engaged tens of thousands of ordinary people to make predictions, and the won the tournament. The GJP’s forecast accuracy was so persistent that IARPA closed the tournament early to focus exclusively on them. In fact, GJP was able to find a select group of “Superforecasters” that generated forecasts that were "30 percent better than intelligence officers with access to actual classified information.” 

Ways to Improve Forecasting Skill

The main findings of the GJP and the book that followed are especially relevant to investors. The research in Superforecasting indicates that predictive accuracy doesn’t require sophisticated algorithms or artificial intelligence.  Instead, forecast reliability is the result of process-oriented discipline.  

This process entails collecting evidence from a wide variety of sources, thinking probabilistically, working collaboratively, keeping score and being flexible in the face of error. According to the book, the 10 traits that most Superforecasters possess are: 

    1.  Intelligence - above average, but genius isn’t required

    2.  Quantitative - not only understand math but apply it to everyday life

    3.  Foxes, not hedgehogs - speak in terms of possibilities, not absolutes

    4.  Humility - understand the limits of their knowledge

    5.  System 2 Driven - use the logic-driven instead of instinct-driven portion of their brain

    6.  Refute fatalism - life is not preordained

    7.  Make frequent and small updates to their forecast based on new information

    8.  Believe that history is one of many possible paths that could have occurred

    9.  Incorporate internal and external views

    10. CONSTANTLY SEARCH FOR WAYS TO IMPROVE THEIR FORECASTING PROCESS

Accountability = Profitability

Organizations cannot improve without systematic and data-driven assessments of their personnel.  Take Bridgewater Associates, for example.  One of the primary factors driving the persistent outperformance of Ray Dalio’s storied fund has been the institutional commitment to radical transparency and accountability.  Similarly, Alpha Theory’s Accuracy Score illuminates blind spots and holds analysts accountable through the precise measurement of predictive skill. For funds that lack the time, inclination or internal resources to create their own probabilistic forecast-grading models, Alpha Theory’s Accuracy Score fills the void.

To this end, Alpha Theory is exploring areas of collaboration with the leadership of Good Judgment Inc. (a spin-off from the Good Judgement Project in “Superforecasting”).  As the competitive landscape for investment capital tightens, discretionary managers must leverage probabilistic data to survive.  Alpha Theory’s Accuracy Score is a mission-critical asset that can help funds compete in the current investment landscape, improving operating inefficiencies and better aligning analyst pay with their intrinsic value to the firm.

May 31, 2016

All in a Day’s Work – Mental Capital Allocation

A Portfolio Manager acts as the final arbiter of what is a good idea and what is not. The Portfolio Manager’s acumen is what investors are ultimately paying for. They need to know each position almost as well as the analyst to be able to properly position it against the rest of the positions in the portfolio. The problem is that most PMs don’t have the time to carry that responsibility because they’re invested in too many positions.

We can prove that PMs don’t know their positions well enough by breaking down a PM’s performance over the course of a year. Let’s assume that the average portfolio manager has 2,500 work hours a year to dedicate to stock analysis (50 weeks times 50 hours). The amount of time dedicated to stock analysis is determined by calculating the percentage of time dedicated to raising capital, communicating with existing investors, running the business, and staring at the P&L.

If the typical portfolio consists of 100 positions and has 50% turnover, then the PM had to understand 150 positions over the course of the year. Now, let’s assume that a PM evaluates two ideas for each position they put in the portfolio. That doubles the number of positions analyzed to 300.

All in a day

The result is 6 hours (basically a day) per year for each position in the portfolio. A PM is ultimately in charge of answering a simple question, “Do I believe my analyst’s assumptions?” 6 hours is hardly enough time for due diligence of that sort.

Don’t take my word for it. Do the math for yourself:

    1. HOURS: Measure your yearly hours using the steps above

    2. ACTUAL POSITIONS: Measure historical average positions per year

    3. ANALYSIS TIME: Measure the time it takes to properly analyze a company (take an example of work you felt really good about)

    4. THEORETICAL MAX NUMBER OF POSITIONS: Divide #1 by #3

    5. RESULT: If Actual Positions is greater than Theoretical Max Number of Positions, then you have too many.

Self-reflection is a key to success because it shows us the proper steps to improve. Reflect on how you allocate your Mental Capital and take the time to figure out if you have the time.