Manager evaluation post selection

Manager evaluation post selection

“Investors should always remember their client fiduciary duties and own biases when deciding which asset managers to invest with. “

Allocators of capital should strive to understand performance evaluation before and during their assessment of the value add from passive and active asset managers. It is not a simple task. The CFA Institute in its Certificate in Investment Performance Measurement (CIPM) Program identifies some of the key aspects you should consider when evaluating performance as Performance Measurement, Performance Attribution and Performance Appraisal.

This helps when assessing historical performance but our responsibility as capital allocators stretches beyond this. Our job is to identify skilful asset managers that might or might not have performed well historically but are likely to outperform their benchmarks and peers in future. Some people would argue this is trying to predict the future, an impossible task!

That’s not entirely true. We use our comprehensive manager research process to analyse, among other things, a manager’s philosophy, process, people, and portfolio construction and risk management process to decide on the likelihood of positive future performance. Sonal Bhagwan takes a manager research perspective in her article and talks about how we qualitatively assess asset managers.

Our manager research work culminates in a list of a select few asset managers that we may invest with, although not all of them would make it into our funds. In this article, we explore what we consider when selecting managers, and how we assess their success or failure after the “hiring” decision. The article touches on how our individual biases can potentially affect our decisions.

Understanding the landscape

First we need to highlight that prior to evaluating managers, we need to understand the market drivers of the various asset classes. For example, to construct an equity fund in the early 2000s, you could classify the equity market in South Africa into two investment styles – value and growth. Similarly, you could group SA equity asset managers into those two buckets. Value managers focused on buying cheap companies based on metrics such as low price to earnings ratios, while growth managers focused more on high return on earnings and other metrics.

Since then, the market environment, and the asset management industry has evolved, and today there are more nuances to consider than those two simple dimensions. Why is this important? Today we need to understand that most asset managers move between various investment styles depending on where they see opportunities. This makes both quantitative and qualitative assessment of an asset manager important. Fortunately, return and holdings data to perform quantitative analysis is widely disseminated in the market. Historic returns data allows you to do performance measurement and risk analysis.

Focus on the manager first

Using returns data we can calculate a manager’s historic active returns against appropriate benchmarks, against peers and outperformance targets. During this analysis, we pay special attention to what was happening in the market when a manager out or underperformed and whether there is a trend in a manager’s active return in certain market environments. We assess a manager’s skill by using measures such as information ratios which quantifies a manager’s active return per unit of active risk taken. Moreover, we look at their active return volatility through measures such as tracking error.

We do this analysis for several managers and it gives us a sense of which managers have historically fared better. This information helps but is not enough to say Manager A should be hired instead of Manager B.

So we continue with our analysis and run other measures such as cluster analysis and correlation metrics to understand which managers have similar active return profiles. We need this information to build diversified portfolios.

We then incorporate holdings data into the analysis. Using holdings data we can further our risk analysis by evaluating metrics such as contribution to tracking error, active positions and beta, a measure of sensitivity to the market. We also analyse the manager’s holdings signature to identify which parts of the market they prefer.

We analyse the quantitative information, incorporate our understanding of the manager gained from the qualitative assessment process to try to understand the repetitive nature of the manager’s performance. When incorporating qualitative information, it is important to note defining moments in a manager’s life. These are moments where structural changes have happened such as a senior portfolio manager leaving the company, significant team and culture changes or a change in philosophy.

We then decide on which managers we want to invest in and the weightings of each. Once included in one of our funds, we continue to monitor and review the manager’s role in the fund, in the context of why they were included (the portfolio construction framework). This is a challenging task, especially when a manager underperforms in an environment you thought would suit them. We also ask questions when a manager outperforms in an unfavourable environment, but most asset allocators are guilty of not doing so (surely great performance should never be questioned, or so they think).

In this review, all performance measures and risk metrics calculated before we hired the manager are again assessed. At this stage, we have the advantage of having sight of the manager’s daily holdings to perform an attribution analysis. The key to attribution is to always analyse decisions that an asset manager actually made, not the circumstantial ones.

For example in a multi-asset class fund, you cannot give credit or criticise a portfolio manager for security selection if all they make is the asset allocation decision and gain exposure to asset classes by investing in building blocks managed by other portfolio managers. Attribution helps us to identify and analyse the drivers of performance, a piece of information that is important when engaging with the manager regarding performance.

Shine the light on ourselves

The challenging part in all of this is when we have to shift our focus from the asset manager to ourselves and really start asking difficult questions. This might include questions about whether we really understood the manager before we hired them. Behavioural finance teaches us that as humans we have several biases, and as allocators of capital this can shape which asset managers we choose for our funds.

The resultant effect of this is investing only with managers that resonate with us and not those that our analysis suggests we should hire. It is important to be aware of our individual biases and create a team culture that encourages colleagues to question each other’s logic. We need to consistently revisit the investment case that was compiled at the time of hiring the manager and ask ourselves if, given what we know about the manager today, we would still hire them.

If the answer is yes, we keep our investment as even good managers underperform. However, if the answer is no, we then need to engage with the asset manager on how performance can be improved, and where necessary part ways. Team work once again becomes a necessity as colleagues can sometimes pick up something you might have missed or give a different perspective.


The fiduciary duty carried by allocators of capital is of paramount importance to clients’ wealth. It is imperative to always exhibit care and diligence when performing it. Performance evaluation plays an important role in quantifying how well we have delivered on our client’s objectives. Prior to hiring asset managers, we should conduct a comprehensive analysis focusing on quantitative and qualitative aspects.

Historical returns and holdings data is widely available and we need to use it to measure a manager’s performance and the risks they have taken to produce that performance. We should use attribution analysis to identify the drivers of their performance and engage with them on those drivers to ascertain the likelihood of their persistence going forward. Equally important is to ask difficult questions when performance is different to our expectations and not shy away from taking decisive actions. Investors need to appreciate that performance evaluation does not stop after a manager has been hired but continues until they are fired.

We also need to be aware of our own biases and not hold back on rectifying our mistakes when new information suggests we were wrong in our initial analysis. Finally, our clients entrust us with their hard earned money because they believe we have their best interests in everything we do. We need to prove that we are worthy of that trust every single day.

By Lubabalo Khenyane,

Portfolio Manager,
STANLIB Multi-Manager

Managing Risk and Expectations

Managing Risk and Expectations

The topic of risk management is very broad, and a myriad of discussion points come to mind when one ponders on it.

Investing money in both traditional and alternative assets comes with uncertainty regarding the range of outcomes. Equity investors often worry about whether the companies they are invested in will deliver earnings and live up to their assessment of intrinsic value, while bond investors concern themselves with the borrower’s ability to service the coupon payment and pay back the initial investment at the end of the bond’s tenure – this is just one way of looking at risk. At STANLIB Multi-Manager, one of the ways we view risk is from our clients’ perspective, and we concern ourselves with delivering outcomes that are either in line with or exceed our clients’ expectations. Our clients come to us looking for solutions that give them exposure to various asset classes (whether its equities, property or fixed income) or target a specific outcome or goal, and each of these has inherent risks associated with it. In the next two sections we will explore how we look at risks in the portfolios we construct for our clients and the measures we take to mitigate them.

Risk from a client’s perspective

Our starting point is to try and understand the client’s return objective, and his/her tolerance and ability to take risk. For most clients, the objective is often to outperform a certain benchmark or market index – this could be the JSE All Bond Index (ALBI) in the case of South African (SA) bonds or the FTSE/JSE Shareholder Weighted Index (SWIX) in the case of South African equities. By definition, to outperform, one has to deviate from the benchmark and this deviation gives rise to both absolute and active risks to a client’s portfolio.

Active risks can be measured by active share or tracking error (the former being an ex-ante measure and the latter being an ex-post measure). If you can identify and measure the risks, then they become possible to manage. In this article we will highlight a few of the absolute and active risk metrics we look at, but these are by no means all that we consider.

Absolute risks are the risks inherent in specific asset classes and can be proxied with metrics like volatility – a measure of the uncertainty or dispersion of a portfolio’s return; or maximum drawdown – which measures the maximum loss from peak to trough during the life of an investment. All proxies of risk have inherent weaknesses, and the two listed above are no different. They both share one weakness, which is that they are backwards looking, relying on historical returns, which may or may not have a bearing on future risk or uncertainty of future returns. They do however provide insights into the reality of the past, and we can decide how this could be extrapolated into future expectations. We design portfolios aiming to have lower absolute risks than the asset classes they invest in.

However, what’s more important to us is active risk, and the metrics that measure it, as this will represent the risk of not meeting our clients objectives. As stated previously, in trying to outperform client selected benchmarks, one has to take some active risk, and both the portfolio manager and the client need to understand this. The amount of active risk taken, depends on the outperformance sought, but one needs to be very careful when interpreting this as higher tracking errors (a measure of active risk) do 

not necessarily imply better prospective returns i.e. active risk is a necessary but not a sufficient condition for outperformance. Very often, high tracking errors result in underperformance, sometimes substantial underperformance.

We therefore contemplate a client’s active risk tolerance in addition to their outperformance objective before designing an appropriate portfolio, to ensure we deliver excellent risk-adjusted returns. The information ratio – a risk-adjusted return measure – is one of the metrics that provides an indication of how well this has been achieved.

How do we mitigate these risks while continuing to keep the client promise?

As a business that invests with various asset managers, it is absolutely imperative for us that we develop an in-depth understanding of the various South African (and global) investment houses. Our team spends countless hours researching asset managers and assessing them both qualitatively and quantitatively on their various capabilities. This assessment gives us a good understanding of the manager’s investment philosophy, process and people, and how this all works together in delivering past performance. It also helps us in understanding how managers have fared over different market environments, or other dimensions that distinguish manager performance i.e. how they and the portfolios they manage behave when different things are happening in local and global capital markets. All of this information and understanding assists us in forming our views on how these managers are likely to perform in future, and what role they may play in our portfolios.

Kamini Moodley, our Head of Manager Research, explores more of this in her article on how we assess asset managers from a risk point of view. Equally important to us is understanding various asset classes, their macro-economic drivers and alpha opportunities. Our knowledge of asset managers and asset classes allows us to take advantage of what is often referred to as the only free lunch in investments, diversification.

In the first quarter 2016 Mindset we covered the importance of diversification across investment houses and investment styles at length so I will not dwell on this here. Another important risk mitigating lever is to have access to relevant technology.

At STANLIB Multi-Manager we have the privilege of having access to both off the shelf systems like Bloomberg, I-NET, Morningstar and IMaps, as well as our own proprietary systems. These systems allow us to identify, measure, mitigate and monitor various risks that our client portfolios are exposed to, both on an ex-ante basis (using up to date portfolio holdings/positions), and on an ex-post basis (to understand and explain past performance and risk taken).

The views and measures we have highlighted so far give us a platform to construct portfolios that can deliver on our client’s expectations, but our efforts do not end there. Post this phase, ongoing portfolio management and portfolio review becomes crucial. We continually review our portfolios relative to their guidelines to ensure that we continue to deliver outcomes that are consistent with our client’s return and risk expectations. Our portfolio review process forces us to constantly review our assumptions and conclusions.

Finally, the vast experience of our team, who have seen various market cycles and know the managers and their people really well, help us position our portfolios to take advantage of return opportunities without taking undesirable risks. It also helps ensure that investment ideas are exhaustively debated before they find their way into our portfolios. These efforts help us mitigate most of the unwanted risks that our client portfolios would otherwise be exposed to.

In conclusion

We understand that there are many ways of looking at risks and taking the client’s perspective is only one of them. However, as individuals who take huge pride and responsibility in managing money on behalf of a large portion of the South African investing community, we feel that it is critical to focus on risk from our clients’ perspective. In conducting portfolio risk management, we think that it is imperative to consider both ex-ante and ex-post risk metrics as proxies for the actual risks faced by portfolios in attempting to outperform their benchmarks.

In this article, we highlighted some of the risk metrics we consider – however, we did not cover all of them and by no means do we claim that the ones we have not mentioned here are of any less importance. Notwithstanding the effectiveness of some of these metrics, we caution that portfolio managers and clients need to understand the limitations that accompany any risk proxies or metrics.

Finally, in trying to mitigate many of the risks our portfolios face, we rely heavily on our insight into asset managers and how they think about capital markets and their portfolios, as they are an important defence in the risk management framework. We also rely on our market knowledge, team experience and debate, and smart use of technology to help us carry out this task with due care and diligence.

As individuals who take huge pride and responsibility in managing money on behalf of a large portion of the South African investing community, we feel that it is critical to focus on risk from our clients’ perspective.
Lubabalo Khenyane

By Lubabalo Khenyane,

Portfolio Manager,
STANLIB Multi-Manager