More Of The Same? Reinforcing People's Success By Risking Statistics

Any article with "statistics" in the title risks losing its readership well before we reach the proverbial audience-shedding equations to come (just kidding!). Yet I’d like to reveal important developments among many of our financial services clients and how they are increasingly using statistics about their people’s performance to improve business performance. Our clients are rapidly becoming more scientific about managing people. Take these four recent examples where we have been helping:

• a global investment bank is now using statistical systems to predict which staff are likely to leave the firm over the next year based on a variety of variables such as length of service, latest review scores, bonus payouts, etc. The firm then targets those it wishes to keep and works hard to increase retention and reduce churn;
• a large commodities firm is correlating performance on a number of operational risk variables with training, and finding that initial staff selection is far more important than almost any permutation of post-hire training;
• a multi-national sales organisation found that, by using records of its successful sales people and the training courses they took, the firm was able to predict which courses would most help new arrivals, rather than subject the new sales people to the full, expensive battery of courses;
• a financial trading firm expected to see increased performance based on length-of-service at a trading desk, but instead discovered that most staff performed as well as they were likely to after their first six months.

What all of these examples have in common is that financial services firms are becoming more scientific in the way that they evaluate staff. These firms construct hypotheses such as "staff are more likely to leave our firm after a large bonus" or "training makes our staff more consistent" or "with more trading experience staff become better". But rather than deploy these hypotheses directly, more and more firms are testing them, often with surprising results. There is nothing surprising about the idea of "scientific management". Frederick Winslow Taylor (1856-1915) famously promulgated five principles:

• scientifically study each task;
• select the right people for the job;
• train, teach and develop workers;
• provide appropriate financial incentives;
• have managers plan work methods and have workers execute the methods.

If you believe the hype, scientific management is bound to rocket skywards, like 90 years ago - ahem. If you’re a skeptic, then business as usual will prevail. So, what might be different? Well, first, far too many ‘soft’ approaches to managing people haven’t worked. It’s probably important to point out that many of them haven’t failed either; it’s just difficult to prove that many fads have made a difference. Second, while many "scientific management" or McGregor’s "Theory X Theory Y" projects were ideologically driven, today’s firms are really looking for things that work rather than being concerned with imposing an ideology. We’ve ‘gone post-modern’ on management theory. Third, information technology and information about people finally provide cost-effective ways of learning about people’s performance and what might improve it. For instance, we now have good databases about people who work in our firms and how they’ve performed. While there is a tremendous amount that can be achieved using just basic Excel functions, even high-end predictive statistical systems (see Z/Yen’s PropheZy for instance) are now affordable by most firms. If you have the data, you can rapidly build predictive scorecards on just about any topic – how long people will stay with the firm, what personalities correlate with new sales, what personalities correlate with relationship management, who is really an operational risk, what personalities generate the most profit.

A good example of how quickly lessons can be discerned from recent information is found in Powerchex’s short report, "Can Past Performance Assessment Assist HR Departments In Developing Recruitment?", August 2006, that provided some quick guidance on the characteristics of staff who stay with their firm. Of course, many of Z/Yen’s larger clients are beginning to realise that statistical methods must be used internally too. Leaving aside some quibbles about independent sample sizes, it doesn’t really take that many internal instances to justify using predictive statistics. In fact, poor-but-adequate predictive statistics based on your firm’s experience may be far more important than excellent-but-irrelevant statistics based on the entire market. If you care about profitability, knowing that two key variables indicate a person is likely to leave your firm within a year should cause you to think about those two variables in your job specification, your advertising, your briefing of recruitment firms, your screening, etc. Would you rather have a completely proven theory (95% confidence level) about staff retention based on you and all your competitors, or learn some interesting lessons about the people you’ve actually hired?

So what can you do today? Well the first thing is to be methodical about assembling information about your staff, their backgrounds, their training and their performance. Without data, there can be no statistics. Second, regularly run some brainstorming sessions where you set out some theories you have about staff and design some tests to prove or disprove the theories. Perhaps at a six-monthly human resources meeting you propose – "we do best with bright graduates we can induct into our way of thinking". How can you test this? Do you have a way of assessing ‘bright’? Do you genuinely track how much experience graduates have outside your firm, or do you use age as a surrogate? Do you have any objective measures of ‘best’? Third, work hard at one or two such proposals to prove or disprove them. Either way you will learn.

We frequently hear much about ‘learning organisations’, yet too infrequently do we see genuine examples of scientific learning within our own firms. It would be nice to hear more statements such as "we thought bright graduates did well, but upon investigation we found that that was only true with graduates working directly on our trading desks, not with our middle or back office staff. We’ve learned some very interesting lessons." My own firm used to ask applicants to dwell at length on at least one "significant failure" and what was learned. While it makes a great interview question, we found little correlation between the question and great performance.

However, beware of the limitations of trying to analyse an entire firm. There are few, if any, authoritative tools to aggregate the views of thousands of people in an organisation and see how the people fit the organisational risk/reward profile. For instance, one might suggest psychometric testing of individuals and assume that the aggregate or averaged profile is the profile of the organisation. However, this approach assumes much – that the psychometric tests are valid, that they have been calibrated for culture (e.g. we’re not using mid-Western USA college student responses to evaluate our global petrochemical managers), that the underlying questions ‘test’ the risk/reward space (a stretching assumption when faced with "which would you prefer" or word affinity questions), or that the questions are summable or averageable. Is one Belbin "monitor-evaluator" worth two "resource-investigators" in an investment bank; in a government organisation? We face a paradox, particularly as financial, systems or analytical people – you can’t make rational decisions without planning around people – yet that requires placing an unprovable value on non-financial inputs and outcomes.

A big area for future intra-firm research is "can we develop a means of sampling individuals’ risk/reward profiles and use that sample to infer concordance with organisational risk/reward profiles?" We are at the early stages of understanding how individual risk/reward propensities, ‘personalities’ if you will, affect organisational ‘personalities’. However, as people are the biggest risk, financial regulation and compliance needs to develop further theories and further tests in order to turn from a rule-based, imposition approach towards helping organisations establish dynamic, evolving risk/reward cultures.

Professor Michael Mainelli, PhD FCCA FCMC MBCS CITP MSI, originally undertook aerospace and computing research, followed by seven years as a partner in a large international accountancy practice before a spell as Corporate Development Director of Europe’s largest R&D organisation, the UK’s Defence Evaluation and Research Agency, and becoming a director of Z/Yen (Michael_Mainelli@zyen.com). Michael was the British Computer Society’s Director of the Year for 2004/2005. Michael is Mercers’ School Memorial Professor of Commerce at Gresham College (www.gresham.ac.uk). Michael’s humorous risk/reward management novel, "Clean Business Cuisine: Now and Z/Yen", written with Ian Harris, was published in 2000; it was a Sunday Times Book of the Week; Accountancy Age described it as "surprisingly funny considering it is written by a couple of accountants".

Z/Yen Limited is a risk/reward management firm helping organisations make better choices. Z/Yen operates as a think-tank that implements strategy, finance, systems, marketing and intelligence projects in a wide variety of fields (www.zyen.com), such as developing an award-winning risk/reward prediction engine, helping a global charity win a good governance award or benchmarking transaction costs across global investment banks.

Z/Yen Limited, 5-7 St Helen’s Place, London EC3A 6AU, United Kingdom; tel: +44 (0) 207-562-9562.

[An edited version of this article first appeared as "More Of The Same? Reinforcing People's Success By Risking Statistics", Powerchex, September 2006).]