Manager evaluation post selection
Allocators of capital should strive to understand performance evaluation before and during their assessment of the value add from passive and active asset managers. It is not a simple task. The CFA Institute in its Certificate in Investment Performance Measurement (CIPM) Program identifies some of the key aspects you should consider when evaluating performance as Performance Measurement, Performance Attribution and Performance Appraisal.
This helps when assessing historical performance but our responsibility as capital allocators stretches beyond this. Our job is to identify skilful asset managers that might or might not have performed well historically but are likely to outperform their benchmarks and peers in future. Some people would argue this is trying to predict the future, an impossible task!
That’s not entirely true. We use our comprehensive manager research process to analyse, among other things, a manager’s philosophy, process, people, and portfolio construction and risk management process to decide on the likelihood of positive future performance. Sonal Bhagwan takes a manager research perspective in her article and talks about how we qualitatively assess asset managers.
Our manager research work culminates in a list of a select few asset managers that we may invest with, although not all of them would make it into our funds. In this article, we explore what we consider when selecting managers, and how we assess their success or failure after the “hiring” decision. The article touches on how our individual biases can potentially affect our decisions.
Understanding the landscape
First we need to highlight that prior to evaluating managers, we need to understand the market drivers of the various asset classes. For example, to construct an equity fund in the early 2000s, you could classify the equity market in South Africa into two investment styles – value and growth. Similarly, you could group SA equity asset managers into those two buckets. Value managers focused on buying cheap companies based on metrics such as low price to earnings ratios, while growth managers focused more on high return on earnings and other metrics.
Since then, the market environment, and the asset management industry has evolved, and today there are more nuances to consider than those two simple dimensions. Why is this important? Today we need to understand that most asset managers move between various investment styles depending on where they see opportunities. This makes both quantitative and qualitative assessment of an asset manager important. Fortunately, return and holdings data to perform quantitative analysis is widely disseminated in the market. Historic returns data allows you to do performance measurement and risk analysis.
Focus on the manager first
Using returns data we can calculate a manager’s historic active returns against appropriate benchmarks, against peers and outperformance targets. During this analysis, we pay special attention to what was happening in the market when a manager out or underperformed and whether there is a trend in a manager’s active return in certain market environments. We assess a manager’s skill by using measures such as information ratios which quantifies a manager’s active return per unit of active risk taken. Moreover, we look at their active return volatility through measures such as tracking error.
We do this analysis for several managers and it gives us a sense of which managers have historically fared better. This information helps but is not enough to say Manager A should be hired instead of Manager B.
So we continue with our analysis and run other measures such as cluster analysis and correlation metrics to understand which managers have similar active return profiles. We need this information to build diversified portfolios.
We then incorporate holdings data into the analysis. Using holdings data we can further our risk analysis by evaluating metrics such as contribution to tracking error, active positions and beta, a measure of sensitivity to the market. We also analyse the manager’s holdings signature to identify which parts of the market they prefer.
We analyse the quantitative information, incorporate our understanding of the manager gained from the qualitative assessment process to try to understand the repetitive nature of the manager’s performance. When incorporating qualitative information, it is important to note defining moments in a manager’s life. These are moments where structural changes have happened such as a senior portfolio manager leaving the company, significant team and culture changes or a change in philosophy.
We then decide on which managers we want to invest in and the weightings of each. Once included in one of our funds, we continue to monitor and review the manager’s role in the fund, in the context of why they were included (the portfolio construction framework). This is a challenging task, especially when a manager underperforms in an environment you thought would suit them. We also ask questions when a manager outperforms in an unfavourable environment, but most asset allocators are guilty of not doing so (surely great performance should never be questioned, or so they think).
In this review, all performance measures and risk metrics calculated before we hired the manager are again assessed. At this stage, we have the advantage of having sight of the manager’s daily holdings to perform an attribution analysis. The key to attribution is to always analyse decisions that an asset manager actually made, not the circumstantial ones.
For example in a multi-asset class fund, you cannot give credit or criticise a portfolio manager for security selection if all they make is the asset allocation decision and gain exposure to asset classes by investing in building blocks managed by other portfolio managers. Attribution helps us to identify and analyse the drivers of performance, a piece of information that is important when engaging with the manager regarding performance.
Shine the light on ourselves
The challenging part in all of this is when we have to shift our focus from the asset manager to ourselves and really start asking difficult questions. This might include questions about whether we really understood the manager before we hired them. Behavioural finance teaches us that as humans we have several biases, and as allocators of capital this can shape which asset managers we choose for our funds.
The resultant effect of this is investing only with managers that resonate with us and not those that our analysis suggests we should hire. It is important to be aware of our individual biases and create a team culture that encourages colleagues to question each other’s logic. We need to consistently revisit the investment case that was compiled at the time of hiring the manager and ask ourselves if, given what we know about the manager today, we would still hire them.
If the answer is yes, we keep our investment as even good managers underperform. However, if the answer is no, we then need to engage with the asset manager on how performance can be improved, and where necessary part ways. Team work once again becomes a necessity as colleagues can sometimes pick up something you might have missed or give a different perspective.
The fiduciary duty carried by allocators of capital is of paramount importance to clients’ wealth. It is imperative to always exhibit care and diligence when performing it. Performance evaluation plays an important role in quantifying how well we have delivered on our client’s objectives. Prior to hiring asset managers, we should conduct a comprehensive analysis focusing on quantitative and qualitative aspects.
Historical returns and holdings data is widely available and we need to use it to measure a manager’s performance and the risks they have taken to produce that performance. We should use attribution analysis to identify the drivers of their performance and engage with them on those drivers to ascertain the likelihood of their persistence going forward. Equally important is to ask difficult questions when performance is different to our expectations and not shy away from taking decisive actions. Investors need to appreciate that performance evaluation does not stop after a manager has been hired but continues until they are fired.
We also need to be aware of our own biases and not hold back on rectifying our mistakes when new information suggests we were wrong in our initial analysis. Finally, our clients entrust us with their hard earned money because they believe we have their best interests in everything we do. We need to prove that we are worthy of that trust every single day.