(866)-482-2177

sales@alphatheory.com

REQUEST A DEMO

SYSTEM REQUIREMENTS


Please note the following System Requirements. Further, please limit the number of open applications (particularly price streaming applications) while logged in to Alpha Theory™.


Recommended System Specifications
Processor: Dual Core or Quad-Core 2.4GHz or faster
RAM: 4GB+
Browser: Google Chrome 30+
Screen Resolution: 1280 x 1024 or greater
Internet Access: Business Class High-Speed


Minimum System Requirements
Processor: Intel Pentium-M 2.0Ghz or equivalent
RAM: 2GB+
Browser: Google Chrome, Mozilla Firefox, Internet Explorer 9+ (without Compatibility View), Safari
Screen Resolution: 1024 x 768 or greater
Internet Access: High-Speed

Subscribe to Alpha Theory content

Alpha Theory Blog - News and Insights

40 posts categorized "External Articles"

August 18, 2017

Man Versus Model of Man: Lewis Goldberg

I recently read an article by Jason Zweig and saw a reference to Lewis Goldberg’s, “Man Versus Model of Man” paper on Expert Studies in the 1970 Psychological Bulletin. There are hundreds of published studies that have a similar theme. Give an expert any and all available data that they want and ask them to make a judgement germane to their field of expertise (examples include Oncologist – how long will a patient live, Parole Board – who is most likely to recidivate, Wine Expert – price of wine at auction, etc.)

The experts tell the scientist which variables are most important in their decision and the scientist goes off and builds a model and compares the model’s results to the forecasts of the “experts.” Over the past 60 years, hundreds of expert studies have been performed and show that the model beats or ties the expert 94% of the time (1).

There was one of Goldberg’s quote about the use of models versus clinical decision making made me laugh:

Such an enterprise, originally viewed with considerable disdain by clinical psychologists, has recently weathered a period of intense controversy (Gough, 1962; Meehl, 1954; Sawyer, 1966), and may soon become a reasonably well accepted procedure in psychology—if not in medicine, stock forecasting, and other professional endeavors.

Consequently, it now seems safe to assert rather dogmatically that when acceptable criterion information is available, the proper role of the human in the decision-making process is that of a scientist: (a) discovering or identifying new cues which will improve predictive accuracy, and (b) constructing new sorts of systematic procedures for combining predictors in increasingly more optimal ways.

This quote was written 46 years ago yet clinical judgement still dominates psychology, medicine, and stock forecasting. Given the evidence, it is hard to argue against model-based decision making or man + model, but expert judgement still dominates.

The experts that will dominate the future (and are already beginning to do so) are the ones that embrace models as an extension of their own expertise. Models do not replacement human judgement. The parameters models are built upon are determined by experts. Experts also are required to intuit when exceptions to the model are necessary.

My belief is that Lewis Goldberg’s prediction will come true in the next decade as computing power, statistical techniques, software, and zeitgeist have grown to a point where Man + Machine will become the rule instead of the exception.

Here’s a few other great quotes from Lewis Goldberg’s article:

- Mathematical representations of such clinical judges can often be constructed to capture critical aspects of their judgmental strategies.

- The results of these analyses indicate that for this diagnostic task models of the men are generally more valid than the men themselves. Moreover, the finding occurred even when the models were constructed on a small set of cases, and then man and model competed on a completely new set.

- Ten years of research on the clinical judgment process have demonstrated that for many types of common clinical decisions and for many sorts of clinical judges, a simple linear regression equation can be constructed which will predict the responses of a judge at approximately the level of his own reliability. For documentation of this assertion and for details of the methodology, see Hoffman (1960), Hammond, Hursch, and Todd (1964), Naylor and Wherry (1965), and Goldberg (1968). While such regression models have 424 LEWIS R. GOLDBERG been utilized (probably somewhat inappropriately) to explain the manner in which clinicians combine cues in making their diagnostic and prognostic decisions (see Green, 1968; Hoffman, 1968), there is little controversy about their power as predictors of the clinical judgments

 

(1) “Comparative Efficiency of Informal and Formal Prediction Procedures” – William Grove and Paul Meehl, published in Psychology, Public Policy, and Law (1996)

July 28, 2017

Crist on Value

There is a paper, famous in value investing circles, called Crist on Value. It is a chapter from a book written by horse handicapper Steven Crist who opines on the short-comings of the average horse bettor. Its popularity amongst value investors is due to its sage advice that can readily be applied to investing. The article is of moderate length and a must read for any fundamental investor. I’ve taken the liberty of highlighting a few of quotes pertinent to our profession:

- “How often have you or a fellow track-goer opined that you're a pretty good handicapper but you really need to work on your betting strategies or your so-called money management? The problem with this line of thinking is that it suggests betting is some small component of the game, which is like pretending that putting is a minor part of championship golf.” INVESTOR CORROLARY: Investors that believe they are good stock pickers but just don’t get the position size right.

- “Even a horse with a very high likelihood of winning can be either a very good or a very bad bet, and the difference between the two is determined by only one thing: the odds. A horseplayer cannot remind himself of this simple truth too often, and it can be reduced to the following equation: Value = Probability x Price." INVESTOR CORROLARY: Investments decision process requires three components: profit from win, cost from loss, and probabilities of each.

- “Now ask yourself honestly: Do you really think this way when you're handicapping (in probability-weighted returns)? Or do you find horses you "like" and hope for the best on price? Most honest players will admit they follow the latter path. This is the way we all have been conditioned to think: Find the winner, then bet. Know your horses and the money will take care of itself.” INVESTOR CORROLARY: Every investment requires story and value. With both, you don’t have an investment.

- “Sticking to your guns is easier said than done, but it is the only way to win in the long run. The horseplayer who wants to show a profit must adopt a cold-blooded and unsentimental approach to the game that is at variance with both the "sporting" impulse to be loyal to your favorite horses and the egotistical impulse to stick with your initial selection at any price. This approach requires the confidence and Zen-like temperament to endure watching victories at unacceptably low prices by such horses.” INVESTOR CORROLARY: If you’re human, you’re subject to bias and emotion. Define rules and procedures in advance that highlight discrepancies between your actions and your rules.

- “I cannot argue in good conscience that Two Item Limit had precisely a 60 percent chance of victory as opposed to 57 or 63 percent, and I doubt that such calibration is in fact achievable. It is, however, possible through experience to get close enough that if you demand sufficient value to cover the margin of error, you should outperform the competition-your fellow horseplayers.” INVESTOR CORROLARY: Coming up with probabilities for stock price outcomes are even more subjective, but that doesn’t mean you can skip the exercise. Play around with a range of outcomes and figure out what is “too conservative” and “too aggressive” to give you comfort for where the probability should be.

 - “If every horseplayer but you were a certifiable idiot, betting at random on names and colors, you would win every day. Conversely, if the only people betting into the pool were the small number of professionals who make a living this way, your chances for long-term victory would be slim.” INVESTOR CORROLARY: How does your competition in the stock market stack up today?

- “Your opportunity for profit at the racetrack consists entirely of mistakes that your competition makes in assessing each horse's probability of winning.” INVESTOR CORROLARY: While we look at the move to passive as a negative; more passive money should increase the number of opportunities for active investors.

- “There is no shame in passing a race because you just don't see any value in it. Nor should you force yourself to play a race in which you have no confidence in your own odds line.” INVESTOR CORROLARY: Good ideas are hard to find, but worth the wait.

- “Recognize the difference between picking horses and making wagers in which you have an edge. The only path to consistent profit is to exploit the discrepancy between the true likelihood of an outcome and the odds being offered.” INVESTOR CORROLARY: Probability-weighted return is the arbiter of all decisions.

October 07, 2016

LUCK VS. SKILL IN INVESTING (Alpha Theory Book Club with Michael Mauboussin)

On October 3rd, Alpha Theory hosted the “Success Equation” book club with the author, Michael Mauboussin, and 35 PMs, analysts, and allocators. Mr. Mauboussin led the discussion on an array of investing topics centered around the central theme of luck and skill in our profession.

Major takeaways:

    1. Investing is dominated by luck because investor skill level has risen to the point where the market is largely efficient

    2. Managers acknowledge the role of luck, but underestimate it

    3. Process improvements are the easiest way for investors to improve performance

The discussion began by exploring how to determine the influence of skill and luck on an endeavor. The measurements are far from precise, but there are some heuristics that give us strong clues.

In the continuum below, games that are dominated by luck, like blackjack and roulette, are on the left side, and games like chess, that are dominated by skill, are on the right side.

 

SE

 

Investing: More Skill or Luck?

We asked the attendees where investing fell on the continuum above. The average answer fell marginally closer to the skill end of the spectrum (near hockey). According to Mauboussin, investing is largely dominated by luck and is only slightly more skill-inclusive than gambling. Skill influences success, but it does not dominate. A monkey throwing darts can beat a sophisticated investor in any given year due to luck because the large number of skilled investors (high intellect, high work ethic, extensive training and experience) has resulted in markets that are largely efficient.

Skill vs Process Improvement

In the case of investing, skill has to be looked at in two dimensions, absolute and relative. Relative skill is key in the investment world, where there has been a dramatic narrowing in skill differences between investors. Because investing is dominated by luck, skill improvements make only small marginal differences in the probability of winning.  The saving grace for investors is that the average investor’s process is far from optimized and small improvements can have meaningful impacts on the probability of winning.

It is important to understand what makes something procedural and another skillful. In blackjack, no skill improvement will increase your chance of winning (assuming one considers card-counting “cheating” or not part of the “legal” rules of the game). On the other hand, process improvements (when to hit/stay/double down) can minimize your losses. You might ask, “why isn’t knowing when to hit/stay/double down a skill?” The answer is because it is formulaic (procedural): when the dealer is showing X and you are showing Y, you always do Z.

Said another way, no matter how good you get, you’re only going to win about 50% of the time. Compare this to chess on the skill side of the spectrum. A player with a 2600 ELO rating will beat a player with a 1600 rating 99.7% of the time. Improvements in skill (like deliberate practice memorizing optimal responses to your opponent’s opening) that improve a player’s ELO rating will increase his probability of winning.

In investing, building a model, making price forecasts, assessing business outlooks, grading the quality of management teams, and evaluating prospects of new products are all skills. Process in investing includes activities such as following a checklist of criteria that should be met for every investment, creating systems for measuring idea quality, tying idea quality to position size, adhering to portfolio rules (liquidity constraints, maximum sector exposures, max drawdown limits, etc.), and analyzing the efficacy of the process to refine it over time. The low-hanging fruit for investors comprise evolutions in process and, according to Mauboussin, are where they should be focusing their improvement efforts, given the heavy luck component at play.

Process enhancements should focus on those that are (1) analytical, (2) behavioral, and (3) organizational.  Alpha Theory speaks to the analytical improvement, where betting one’s edge intelligently is critical.  In terms of managing one’s organization, optimal collaboration is key.  This works best when (1) the size of team is larger, (2) cognitive diversity of the team is greater, and (3) management of the team offers [a.] dependability and [b.] “psychological safety” (fostering an environment where participants have no reason to fear sharing candid views).  Furthermore, the best leaders keep to an agenda, suppress their own points of view, and indeed successfully elicit the team-members’ perspectives – even those of the introverts.  (Alpha Theory can help here as well!)

IQ vs. RQ

Speaking of cognitive diversity and decision processes in investing, it is important to be aware of differences between IQ (intelligence quotient) and RQ (rationality quotient). Most people make the association between smart investors and high-IQ intellectual competency.  But in fact the best type of mental model that leads to appropriate investment decisions is RQ-oriented (really, the ability to make reasoned, judicious decisions efficiently and without equivocation in a fluid environment like the stock market).  Furthermore, one applied psychology study (see Bibliography below) found a surprisingly low correlation coefficient between IQ and RQ.  The investment industry may err on the side of hiring high-IQ analysts when it should be seeking higher RQ as a starting point – although there is not a ready test for RQ as of yet.

Ecology of Decision Rules

The stock market is a classic adaptive complex system – one where there can be ‘diversification breakdowns’ that result in the wisdom of crowds working until it does not work.  Diversity equates to different menus of decision rules each participant has, but when an asset price rises, many participants drop their own rules and conform to a single one, which breaks down diversity.  This tends to be a non-linear function with a ‘snap!’ phase transition, where reflexivity is defined.  But then diversity is restored when overcrowding corrects itself.

Ways to Improve Forecasting

Several process improvement steps come directly from “Success Equation” and are called suggestions to improve the “art of good guesswork”:

    1. Understand where you are on the luck-skill continuum

    2. Assess sample size, significance, and swans

    3. Always consider a null hypothesis

    4. Think carefully about feedback and rewards

    5. Make use of counterfactuals

    6. Develop aids to guide and improve your skills

    7. Have a plan for strategic interactions

    8. Make reversion to the mean work for you

    9. Develop useful statistics

    10. Know your limitations

Resources

SLIDES: Here is a link to a set of slides very similar to the one’s Mr. Mauboussin used and a video of him discussing “Success Equation”.

 

BASE RATE BOOK: A hot topic was the use of base rates to improve forecasting and decision making. Without a doubt, this is one of the best and easiest ways to improve your process. You can check out Mauboussin’s “The Base Rate Book” here and get a primer on how to implement it.

 

BIBLIOGRAPHY: One of the amazing things about Mr. Mauboussin is the catalog of referenceable articles, studies, and books in his head. Here is a list of all of those he referenced during the Book Club:

“Even God Would Get Fired As An Active Investor” by Wesley Gray

“On the Impossibility of Informationally Efficient Markets” by Sanford Grossman and Joseph Stiglitz

 “Agent Based Models” by Blake LeBaron

David Swensen quoted in “Asset Allocation or Alpha?” by Mimi Lord

“Vicarious Learning, Undersampling of Failure, and the Myths of Management” by Jerker Denrell

“The Three Rules” by Michael Raynor and Mumtaz Ahmed

“Luck versus Skill in the Cross-Section of Mutual Fund Returns” by Eugene Fama and Kenneth French

“Should Airplanes Be Flying Themselves” by Vanity Fair

“The Base Rate Book ” by Michael Mauboussin

Good Judgement Project  

Solomon Asch Experiments    

Greg Berns – Emory University

“What intelligence tests miss” by Keith Stanovich

 “Comprehensive Assessment of Rational Thinking” by Keith Stanovich

Cognitive Reflection Test (“Poor Man’s Test for RQ”) by Shane Frederick

Freestyle Chess

“What we miss when we judge a decision by the outcome?” by Francesca Gino

“Deep Survival” by Laurence Gonzolez

CFA Institute survey late 2008/09 – Quants vs. Fundamentals

“Use Cognitive Diversity to get the most of the Workplace” by Mark Miller

“Peak” by Anders Ericsson – Theory of 10,000 Hours book

“Robert’s Rules of Order” by Henry M. Robert (No one can speak 2x on a topic until everyone has had a chance to speak at least 1x)

“Forms Follows Functions” by Michael Mauboussin

"IQ vs. RQ" by Michael Mauboussin and Dan Callahan

 

Co Authored by: Cameron Hight & Dana Lambert

February 22, 2016

How Do Hedge Funds Become Better Forecasters? - A collaborative study between Novus and Alpha Theory.

We believe that one of the few untapped frontiers in Alpha Generation is measuring and putting process around forecasting.  Alpha Theory co-authored “How Do Hedge Funds Become Better Forecasters?” with our friends at Novus to explore a few ways investors can improve their process and forecasting acumen.

 

CLICK HERE TO DOWNLOAD THE ARTICLE

 

Selected Quotes from the Article:

“Many investors chafe at price targets because they smack of “false precision". Those investors are missing the point because the key to price targets is not their absolute validity but their explicit nature which allows for objective conversation about the assumptions that went into them.”

“Unlike real life, investors can track every investment choice they have ever made. Being able to analyze statistically significant trends on a complex and numerate datasets is a huge advantage and is a crucial tool in avoiding the confirmation biases that anecdotal thinkers lean on when rationalizing decisions.”

“Developing a process orientation isn’t about stifling fluidity or gut feel. It is about recognizing that intuition is actually an informal process. By being able to document and empirically study past behaviors, all investors can understand flaws in their internal process.”

January 30, 2015

The Worst Year Ever for Hedge Funds (Novus Article)

Our friends at Novus put together an interesting article called “The Worst Year Ever for Hedge Funds” that used statistics to analyze which market environments are easiest for fundamental investors to generate alpha. The conclusion was that 2014 was the hardest year ever for hedge funds to generate alpha because the dispersion between winner returns and loser returns were the tightest in hedge fund history (read the article for a thorough explanation). I will not attempt to articulate any deeper, for doing so would basically require me to read you the article. What I will attempt is a few basic observations:

1)      I really hope the premise of this article is correct and 2014 marks a nadir for hedge fund outperformance. As my partner, Benn Dunn, keeps saying, “what hedge funds really need in 2015 is a good down 5% for the S&P 500.”

2)      For the hypothesis of this article to hold in practice, there is an assumption of luck. Basically, if we pick random portfolios, 2014 was the toughest year to produce outperformance. If alpha generation were pure skill, the evidence for 2014 wouldn’t stand-up because the skillful manager would buy the very high performers and short the very low performers…even though there were a limited number. The truth lies somewhere in between.

3)      If there is a prospective element (these variables are forecastable) then a manager could use them to determine leverage (i.e. when it is easy, lever up, when it is like 2014, lever down). However, I’m finding it difficult to find variables in the analysis that are forecastable or are even stable. If they were at least stable, then you can use a tight current spread to suggest lower leverage, but the spreads can change rapidly.

Overall, this is a great read and made me think critically about disaggregating alpha. If you haven’t read the author Joe Peta’s book “Trading Bases”, you should. Finally, I’ll leave you with my favorite quote from the article, an excerpt I chose for clearly selfish reasons given Alpha Theory’s focus on position sizing:

    “At Novus we’ve examined the performance of many hundreds of hedge funds and we’ve found that, by far, the most important skill to possess is the ability to size positions effectively. That’s because unlike Exposure Management, and to a lesser degree Security Selection, the ability to size positions efficiently is the most persistent and consistent alpha-generating skill that a portfolio manager can possess.”

March 21, 2014

Dynamic factor modeling reveals hidden risks

GUEST POST FROM BENN DUNN, President of Alpha Theory Advisors:

Damian Handzy, CEO of Investor Analytics, and I developed the concept of dynamic factor modeling in this latest article on Risk.net.  We argue that traditional 3rd party vendor models do not accurately reflect many firm’s investment processes and leave measurable risk hidden.  Using beta as a common language between risk and portfolio managers, we recommend leveraging the literally thousands of listed instruments and funds to ease the process of risk measurement and hedging.

Click Here to full the article on Risk.net.

November 05, 2013

Less Correlation Gives Stock Pickers Opportunity

We’d like to welcome our first blog from Benn Dunn who runs our Risk Consulting practice. I’m a little biased, but I believe that Benn is one the smartest risk minds in investing today. Check out this article on Risk.net where he is quoted on the topic of correlation in portfolio management.

While lower correlations across asset classes and within markets are generally thought of as positive for security selection, the path to lower correlations can often be confusing.  Traditional risk models deliver confusing and difficult to interpret results during these regime shifts.  Fortunately, Alpha Theory is not dependent on trailing correlations when making portfolio construction recommendations. 

Click here to read the full article on Risk.net

 

May 01, 2013

The Checklist: Evolution of an Effective Decision Process Part 2

As a continuation to The Checklist: Evolution of an Effective Decision Process Part 1 we now turn our attention to money managers and their checklists. 

Fund managers fall into similar traps of overconfidence about their ability to manage the complex system of investing. Dr. Gawande discusses investing (Chapter 8) by analyzing three public equity managers that employee an explicit checklist and a study by Geoff Smart, which analyzed 51 venture capital managers. The evidence, although not a large sample size, is compelling. The public managers attribute the checklist to being a primary component of their success and a distinct competitive advantage.

“The checklist doesn’t tell the manager what to do. It is not a formula. But the checklist helps him be as smart as possible every step of the way, ensuring that he’s got the critical information he needs when he needs it, that he’s systematic about decision making, that he’s talked to everyone he should. With a good checklist in hand, he was convinced he and his partners could make decisions as well as human beings are able. And as a result, he was also convinced they could reliably beat the market.” “They (Checklists) improve their outcomes with no increase in skill. That’s what we are doing when we use the checklist.” “When he first introduced the checklist, he assumed it would slow his team down, increasing the time and work required for their investment decisions. He was prepared to pay that price. The benefits of making fewer mistakes seemed obvious. And in fact, using the checklist did increase the up-front work time. But to his surprise, he found they were able to evaluate many more investments in far less time overall.” 1

And even though their competitors have noticed their success and asked them for their secret, when told that the key is a checklist, they turn up their nose.

“In the money business, everyone looks for an edge. If someone is doing well, people pounce like starved hyenas to find out how. Almost every idea for making even slightly more money— investing in Internet companies, buying tranches of sliced-up mortgages, whatever— gets sucked up by the giant maw almost instantly. Every idea, that is, except one: checklists. I asked one of the equity managers how much interest others have had in what he has been doing these past two years. Zero, he said— or actually that’s not quite true. People have been intensely interested in what he’s been buying and how, but the minute the word checklist comes out of his mouth, they disappear. Even in his own firm, he’s found it a hard sell.” “I find it amazing other investors have not even bothered to try,” he said. “Some have asked. None have done it.” 1

The evidence from the VC community is even more compelling because the results are more statistically significant (51 managers) and are empirical. Mr. Gross categorized managers into five different categories by how they made decisions. One of the classifications was “Airline Captains” which meant they used checklists extensively to make decision.

“Smart next tracked the venture capitalists’ success over time. There was no question which style was most effective— and by now you should be able to guess which one. It was the Airline Captain, hands down. Those taking the checklist-driven approach had a 10 percent likelihood of later having to fire senior management for incompetence or concluding that their original evaluation was inaccurate. The others had at least a 50 percent likelihood. The results showed up in their bottom lines, too. The Airline Captains had a median 80 percent return on the investments studied, the others 35 percent or less.” 1

Smart’s study was performed over ten years ago and his findings are known by the VC community. You would think these fact-based results would change behavior, but as Gawande found when asking about Smart about behavior changes for VC managers:

“But when I asked him (Geoff Smart), now that the knowledge is out, whether the proportion of major investors taking the more orderly, checklist-driven approach has increased substantially, he could only report, “No. It’s the same.” We don’t like checklists. They can be painstaking. They’re not much fun. But I don’t think the issue here is mere laziness. There’s something deeper, more visceral going on when people walk away not only from saving lives but from making money. It somehow feels beneath us to use a checklist, an embarrassment. It runs counter to deeply held beliefs about how the truly great among us— those we aspire to be— handle situations of high stakes and complexity. The truly great are daring. They improvise. They do not have protocols and checklists.” 1

The Checklist Manifesto is well worth the day or so it takes to read. The real question, are you willing to adopt a Checklist mentality? Will you take the time to check the boxes? Are you disciplined enough to stop yourself from making decisions unless those boxes have been checked? I know that if I were back on the buyside, I would take out my mini-checklist and expand on it, formalize, and refine as we improve our process. I’ve seen firsthand the importance of making decisions explicit. But I think Dr. Gawande says it best:

“Instead they (pilots) chose to accept their fallibilities. They recognized the simplicity and power of using a checklist. And so can we. Indeed, against the complexity of the world, we must. There is no other choice. When we look closely, we recognize the same balls being dropped over and over, even by those of great ability and determination. We know the patterns. We see the costs. It’s time to try something else. TRY A CHECKLIST.” 1

 1Gawande, Atul (2009-12-15). The Checklist Manifesto: How to Get Things Right. Picador.

April 30, 2013

The Checklist: Evolution of an Effective Decision Process Part 1

Back before I started Alpha Theory, I was an analyst at a hedge fund. I spent my days trying to find good ideas in which we could invest. Along the way, I built a mini-checklist to speed up the process of evaluating potential ideas. The checklist included mostly quantitative metrics that I could build into a fancy spreadsheet like current PE compared to 10 Year Average PE / High / Low, same for EV to EBITDA and Price to Sales, leverage ratios, accounting ratios, insider holdings, short interest, etc. I could pull this all together pretty quickly thanks to a combination of links from Bloomberg, Capital IQ, and Factset and a few hours of manual due diligence.

With our quick checklist, we could plow through dozens of names and eliminate those that didn’t qualify with speed. In a world where positive expected return bets are few and far between, a streamlined evaluation process increases your odds of finding more good bets. Of course, we ended up doing customized analysis after it passed the initial test, but answering some basic universal questions was step one.

I’ve spoken to hundreds of managers and I believe that most funds have an implicit checklist (one that is understood but not formalized). An analyst generally knows what questions the portfolio manager will ask and experience leads them to ask many of the other important questions. However, after reading the book, The Checklist Manifesto, by Atul Gawande, I think it would be wise to make the fund’s implicit investment checklists explicit. Alpha Theory is in the business of providing systems to help managers make their decision process more objective by making it explicit. But The Checklist Manifesto raises explicit to a whole new level. The book begins by explaining the challenges of decision making in complex systems. Complex systems are simply too grand, too multivariate, for a human to comprehend their full scale. Because of this complexity, we use heuristics (rules-of-thumb) to simplify the pieces into understandable parts. These heuristics, while handy, can leave us with blind spots. Gawande, who is a practicing surgeon, was tasked with trying to improve surgical success for the World Health Organization. His answer, a checklist for the nursing, surgical, and anesthetist teams to run through before and after surgery. After a few iterations, the success was astonishing.

“The final results showed that the rate of major complications for surgical patients in all eight hospitals fell by 36 percent after introduction of the checklist. Deaths fell 47 percent. Overall, in this group of nearly 4,000 patients, 435 would have been expected to develop serious complications based on our earlier observation data. But instead just 277 did. Using the checklist had spared more than 150 people from harm— and 27 of them from death.” “More than 250 staff members— surgeons, anesthesiologists, nurses, and others— filled out an anonymous survey after three months of using the checklist. In the beginning, most had been skeptical. But by the end, 80 percent reported that the checklist was easy to use, did not take a long time to complete, and had improved the safety of care. And 78 percent actually observed the checklist to have prevented an error in the operating room. Nonetheless, some skepticism persisted. After all, 20 percent did not find it easy to use, thought it took too long, and felt it had not improved the safety of care. Then we asked the staff one more question. “If you were having an operation,” we asked, “would you want the checklist to be used?” A full 93 percent said yes.” 1

These results are astonishing. If a drug or medical device reduced death rates by 50%, it would be a blockbuster that every doctor demanded. But even after these astounding results were published in the New England Journal of Medicine, many doctors and hospitals were reluctant to implement the checklist. Why? Perhaps inertia, dogma, ossification? Whatever the reason, it seems to be common in most professions. The mentality suggests “I’m sure it is important for others, but I don’t need it”.

Check in tomorrow for Part Two of “The Checklist: Evolution of an Effective Decision Process.” We’ll find out what Dr. Gawande discovered about money managers that use checklists.

1Gawande, Atul (2009-12-15). The Checklist Manifesto: How to Get Things Right. Picador.

 

February 01, 2013

Kelly Criterion in Practice Part 2

As a continuation to Kelly Criterion in Practice Part 1 we dive into more tests and conclusion of the evaluation of the Kelly Criterion for portfolio management.

Trial #2. The performance improvement of the portfolio made me wonder how a group of investments with similar Expected Returns (all 20%) but different payoff structures would look (see below).

Here we see that neither the Uncorrelated or Correlated portfolio includes Investment #3 because it is severely impacted by its 90% chance of complete loss even though its expected return is 20% like the others. Investment #1 is favored by a wide margin because it has the lowest probability adjusted chance for loss of 15% (50% loss * 30% probability) versus 32% for Investment #2 (40% loss * 80% probability) versus 90% for Investment #3 (100% loss * 90% probability). In this case, the Kelly bet did a very good job of constructing the portfolio. This makes sense because the problem with the Kelly Formula for portfolio management is that it looks at each bet individually. We allowed the Kelly Formula to work individually by making all of the investments have the same Expected Return of 20%.

One of the most interesting outcomes of the second portfolio trial is that the CAGR of the Correlated Portfolio is 8.0%. That is lower than the 9% attained we could get from just betting the Kelly Bet of 80% in Investment #1 and holding the other 20% in cash. In this case, the best result is not a portfolio. I believe this is an important caveat in portfolio construction and should make cash an option for any portfolio. So instead of 3 Investments, you would always have 4 because cash may be a better option in certain circumstances.

Lessons Learned. So what have we learned from this analysis:

-Kelly bet size alone is not sufficient to determine position size

-Expected Return alone is not sufficient to determine position size

-Investment correlation has a large bearing on position size

-Probability Adjusted Chance for Loss has a large bearing on position size (Probability * Potential Loss)

-Cash is a legitimate option even if it has a 0% expected return

Ultimate Solution. The ultimate solution requires incorporation of research's scenarios analysis, correlation amongst assets, and picking the array of position sizes that maximizes long-term geometric expected return. The first step is to forecast thousands of arrays of returns for each investment based on its probability weighted scenario analysis resulting from your analyst's research. Next, choose the one set of arrays from the thousands generated which most closely matches the correlation statistics for each asset and the portfolio. Finally, use the best set of arrays to interpolate the bet size for each investment that maximizes portfolio return (if someone knows of a closed-form way of producing the array of position sizes please let me know).

Problems with the Ultimate Solution.

-Forecasting payout streams that have inter-correlation is very difficult. They could be calculated with historical correlations but that would suggest that future correlations will be the same as the past. The firm could forecast correlations but that is complicated and may be too much to ask from an operational perspective.

-The arrays are constructed at random which causes them to change with each iteration. This could cause unnerving degrees of variance in optimal position sizes depending on the randomized set of arrays. For instance, you could see that the optimal position size for IBM is 4.3% one minute, have nothing change, and the optimal position size move to 4.6% because the random array being selected has changed. My guess is the arrays would have to be stabilized until the addition of a new asset or research is updated. There is some work being performed by Sam Savage to standardize arrays (http://probabilitymanagement.org/index.htm).

-An optimization function is required to incorporate time horizon, liquidity, gross and net exposure, sector exposure, analysis confidence, etc. Certainly not an insurmountable hurdle, but could pose pitfalls.

Conclusion. Alpha Theory has tackled the problem in a straightforward way that accomplishes much of the task of position sizing described above. As we showed in Trial #1, Expected Return is a good predictor of position size in situations where there is variance in Expected Return. The predictive power is greatly enhanced by adjusting for the probability-weighted chance for loss (loss * probability of loss) which Alpha Theory does. The result is position sizing that is close to optimal position sizing (non-correlated). Lastly, because Alpha Theory is a linear function, it gives stable results and can scale position size to control for minimum and maximum position size, return parameters, liquidity, market correlation, time horizon, sector exposure, portfolio exposure, etc. Alpha Theory represents the best solution by providing most of the benefits of the complex ideal while still maintaining practicality for everyday use. At a bare minimum, it is light-years ahead of what most money managers are doing today!