In this chapter we shall:
Regardless of the size, not-for-profit organisations should follow a proper process for selecting computer software applications. Even if the cost of the software seems relatively low, the “all-in cost” of a solution, including training and the implementation effort is usually very significant to the organisation. The cost of failure is usually even higher, when you count the psychological cost and benefits not achieved. Based on empirical evidence, IT systems projects appears to be an inherently risky area of activity. A 1996 study by The University of Sheffield Institute of Work Psychology (Clegg et al., 1996) indicates that more than 80% of IT systems projects are late to a material extent. According to a Standish Group CHAOS report (see English, 1995), over 31% of IT systems projects are cancelled before they are completed and 53% of projects cost at least 189% of their original budgets. Only 9% come in on time and on budget. A more recent study by Andrew Taylor (2001) provides little comfort, boasting only 12.7% of projects 'successful', using Standish criteria for success.
People in not-for-profit organisations often express surprise at the above statistics. "It's not just us charities who botch up IT projects," they say, "it's a general syndrome". Yes it is, but that should not provide comfort. Quite the opposite. If commercial organisations expect to overspend and run late, at least they probably have the excess resources to pay for it. Hopefully the reader is now convinced that there is need for improvement. Naturally, using a proper process for choosing the right software is not the only requirement for a successful IT project (other factors are covered throughout this book, not least chapter "Governance: Valuing IT" and "The Four C's"). Nevertheless, we believe that using a form of the following process minimises the risks and maximises the potential rewards from an IT project. Naturally, the process for smaller solutions is likely to be quicker and less detailed than the process for larger solutions. The process and tips set out below can be scaled to suit the size and complexity of your organisation and its needs.
The main stages of the process to choose computer software applications should be as follows:
The following sections of this chapter cover each of these in turn. The figure below illustrates the decision making processes you should be making processes you should be following as you move through the stages.
The feasibility stage is sometimes referred to as the “business case stage”. The purposes are:
It is astonishing how many IT systems projects are attempted without any feasibility stage at all – this is akin to setting off on a journey without knowing where you are going or why you are going there. Even for smaller projects, we suggest that as a minimum you should define the objectives of the project and attempt to quantify the benefits, or rewards sought. It is important at this stage to ensure that appropriate people are consulted, that the project is properly defined and that people understand what is and is not to be included within the scope of the project (see also chapter 16).
To start with, ask the simple question "can we achieve our strategic objectives with our current systems?" (You might still choose to change systems to improve economy and efficiency, or you might tweak your existing system to achieve the objectives). Outline the project objectives in terms of the strategic objectives you are setting out to meet. Your analysis should also include your estimated quantification of potential rewards, estimated costs and a risk-based assessment for each objective and for the project overall.
Don't forget to take risk factors into account, e.g. have you discounted the cash flows of this project with an appropriate risk-adjusted discount factor - i.e. a discount rate that takes into account the amount of risk inherent in the proposed project? How sure are you about your cost estimates and how sure are you that you will achieve the efficiency gains? What is the likelihood that the project fails completely and how much might it have cost you by the time you bail out?
Many projects which easily appear feasible using a simple cost/benefit analysis do not make it past the feasibility hurdle once a reasonable analysis of risk is included. Many systems projects proceed with little or no definition of project objectives and without clearly defined benefits. Lack of definition makes the specification and selection phases quite difficult, to say the least. It really is worth the little bit of time and effort into the feasibility stage. Template 14.1 might help.
Desired timescale and reasons
Other relevant points
Suitability of existing systems (with enhancements?)
Is there available commercial software?
What are the (rough) estimated costs?
Is there a low cost option that might achieve most but not all the benefits sought?
What are the essential service level requirements?
What are the main risks and how can these be reduced?
Use of IT infrastructure required?
The purposes of the specification of functional requirements stage are as follows:
The specification process should define the scope of the project in more detail and set out the user requirements in the areas to be covered. This stage is essential in beginning to involve users, to help them to understand their requirements and to begin the process of changing working practices. The specification document should form the basis of the invitation to tender and eventually form part of the contract with the selected supplier(s). Staff engaged on specification work ideally should have both business analysis and IT skills.
There was a time when the specification stage was governed by “Thumpers Law” E = mc2, where ‘m’ is the mass of the specification document (kilograms weight or centimetres thickness), ‘c’ is the clunk the document makes when it lands on the desk (decibels) and E is the enormous fee extracted by professional advisors (£ tens of thousands). In more modern approaches, Thumpers Law no longer applies – the specification document itself should be the minimum required to explain the system needs for users and to potential suppliers. As a minimum, set out core functions, features, workflows and information needs. If you overspecify, you'll frequently find that you have defined a requirement that only a bespoke development can solve.
The specification document should focus on the trickier and unusual elements of your requirements, where you will need to specify details in order to communicate your needs clearly indicate where items are essential, highly desirable or simply “nice to have”. The table below provides an outline of a basic specification of functional requirements. Figures 14.2 and 14.3 give examples of workflows.
|Section Title||Section Contents||Comments|
|Executive Summary||Precis of document, summarising core functions, features, workflows and information needs||
if required - smaller organisations might not need
avoid using an executive summary when you want executives to review the whole document!!
|Scope and objectives||The scope of the system sought and the objectives for seeking the system||normally should come from feasibility documentation - see template 9.1|
|Current situation and future needs||Outline existing systems, current staffing around those systems, current technical environment (hardware, operating systems, database systems) and estimated volumes, outline of anticipated needs changes, future technical environment and estimated volumes, reasons for seeking systems change||much of this information should come from the feasibility documentation - see template 9.1|
|Main processes||Workflow diagram(s), explanations of workflow diagram(s)||see sample figure 9.2|
|Outline user requirements||User needs and wants based on the processes set out in the workflow diagram(s). Make sure that reporting requirements and links with other systems are spelled out here.||indicate which requirements are 'needs' (i.e. mandatory) and which are 'wants' (i.e. desirable|
|Technical requirements||Include needs and wants relating to user interface (the look and feel of the system), documentation, facilities (e.g. response times, integration mechanisms, links to other software), security, continuity features, support & maintenance,||indicate which requirements are 'needs' (i.e. mandatory) and which are 'wants' (i.e. desirable|
|Data entities and fields||A diagram of the main data entities and a summary of the information you wish to hold for each||
see sample figure 9.3
for larger and/or non-standard systems (basic financial software specifications should not need this level of detail)
|Classification codes||Set out tables of classification codes||
for larger and/or non-standard systems (basic financial software specifications should not need this level of detail)
avoid setting out this information if you expect to change these classification significantly as part of your system change
|Example sets||Forms, reports and listings from the current system||
again, non-standard stuff only need apply here
again, avoid setting out elements that you expect to change substantially and therefore might prove misleading in their current form
Preparing the invitation to tender entails finding a “long list” of potentially suitable suppliers and “topping and tailing” the specification of requirements to describe the tendering and selection process.
Supplement your specification document (see Table 14.1 above) by setting out the selection process, including the envisaged selection and implementation timescales. Ask questions about the supplier themselves at this stage (e.g. their financial position, the prevalence of the solution(s) they propose, references). Make sure you include tendering contractual matters, such as fees (the usual practice is for you to pay no fee during the tendering process and this should be clearly stated in your invitation to tender document) and state matters of principle for the eventual contracts. The invitation to tender document should include the following:
|Section Title||Section Contents||Comments|
|Introduction||Explain what the invitation to tender is, your confidentiality rules, procedures for tendering (including the fact that you do not intend to pay for proposals) and your selection criteria including attitude to costs (e.g. your budget)||try and keep this part to two or three pages maximum|
|Background to the tendering||Brief background on your organisation, summary of the requirements, main reasons and objectives of tendering, timetable||try and keep this part to three or four pages maximum|
|Specification of requirements||see Table 9.1 above||exclude your internal management summary and any information you included that you would not want a third party to see (e.g. staff names and salaries)|
|Supplier questionnaire||Ask for company details, company performance, approach to quality, approach to equal opportunities and other ethical issues, staff experience, company experience with similar organisations and similar projects, user base for similar requirements, do they have a user group?, user references, summary of costs and benefits, reservations about the proposal, modifications they would need to make to standard software, information on other products and services, extent to which they work with relevant third parties, options on pricing and "working in partnership" with your organisation||this section is key to ensure that you get the answers to your main questions in a standard form. Having said that, even the best supplier questionnaires are mysteriously ignored by some suppliers who insist on providing you with their standard proposals, sometimes not even cross-referenced to your questions!|
|Terms and conditions of response||Your terms of business for the tendering, including the matter of costs for the tender (usually none to be born by you), confidentiality, and key contractual terms that you will insist upon in your contract (e.g. acceptance procedures, correction of faults and errors, reliance on suppliers representations)||you might as well find out now if some of your "showstopper" contractual terms make you a non-starter for some of these suppliers.|
Think broadly when choosing a long list of potential suppliers. For example, if you suspect that a bespoke solution might be required, include package providers and bespoke development houses on the long list so you may compare the two approaches. Be prepared to use your imagination somewhat when considering possible packages. For example, a good commercial sales and contact management system might meet your fundraising needs. The Cancer Research Campaign, for example, chose a legal probate system for its legacies administration rather than a conventional legacies administration system.
Try to avoid having too long a long list – remember that you will need to review all the tenders you receive and that some suppliers (sometimes the better ones) will be reluctant to quote if they know that they are on a very long list. 6 to 12 suppliers is a sensible rule of thumb.
Software directories, consultancy advice, word of mouth networking, magazine articles and common sense are all useful techniques in coming up with an appropriate, well-balanced tender list – you should probably use them all to maximise your chances of success.
The purpose of this stage is to:
An ideal short list should comprise two or three suppliers, all of whom appear capable of meeting the requirements. Remember that up until this point you have only tested the product fit (does the software seem to meet the requirements). At the skills level, so far, you have only tested the suppliers’ ability to write proposals and produce marketing literature. You have not tested the skills you actually want to buy, i.e. you have not yet tested the suppliers' skills at providing the services you will need to get your required benefits from the software.
The modern approach to this stage of the process is “hands-on”, with the purchaser looking carefully at the track record and business acumen of suppliers, as well as the skills and manner of the suppliers’ people. Unless you are keen on slick, standard presentations, it is usually a good idea to discourage suppliers in advance from spending much time on “talking up” their companies. Uncontrolled software demonstrations spend much of the time on glossy generalities and fail to allow enough time for the detailed discussions that might usefully cover key issues and options (i.e. the stuff that is really useful to you). Software salespeople are often highly skilled at "selling the sizzle and not the sausage". Software demonstrations should focus on demonstrating the suppliers’ ability to meet your key and most tricky requirements.
You will often need to take a lead to ensure that the demonstration achieves your objectives. Don't be afraid to ask tough questions, questions on subjects where you might feel a bit green and especially pertinent questions to which you think you could guess the answer. For more complex, non-standard needs, you should encourage suppliers to prototype a non-standard element of your requirement to demonstrate their ability to understand and meet your needs – this is sometimes referred to as a “competitive model office” test.
A competent software salesmen will be very nice to you and compliment you on the quality of your invitation to tender, your choice of fine consultants, or alternatively your wisdom in neglecting to produce an invitation to tender and/or your wisdom in not using consultants. Some will serve you decent coffee and biscuits. You might even hit the jackpot and get lunch. These aspects test very little. When you are potentially buying services as well as product, try to insist on meeting at least one of the operational staff who will be involved in your project, not just sales (or business development) staff.
At the end of this stage, you should ideally find yourself with a preferred supplier with whom you hope to agree a contract, plus a good reserve in the wings just in case the due diligence and contracts work puts you off your preferred supplier.
With large or complex applications, most not-for-profit organisations can benefit from independent help by appointing consultants. Even with smaller applications, good consultants should be prepared to help you on a modest scale, e.g. by providing some quality assurance on specification and tender documents produced by your staff, helping you to find suitable suppliers for the long list and quality assuring key decision points (e.g. shortlisting, and choosing a preferred supplier). You might well also need specialist help with due diligence and contract negotiation (see below).
The use of hardware or software suppliers as consultants may result in a conflict of interests when choosing solutions, so should normally be avoided. Good consultancy help should ensure that benefits are clearly defined, the objectives and scope of the project are clear, a proper methodology is followed and contracts with solution suppliers are robust enough to control the project risks.
The purpose of due diligence is to understand the risks inherent in proceeding with your preferred supplier. The purpose is not to put some ticks in boxes on a checklist and congratulate yourself that you have completed a checklist in record time. You should be asking hard questions at this stage and you are unlikely to get satisfactory answers to all of your hard questions. The key point is that you know what you are letting yourselves in for and are able to plan around the risks and weaknesses you uncover. Above all, be prepared to walk away from a supplier if that supplier is really falling short of your required standards.
Under normal circumstances, we suggest that you only undertake the due diligence process on your preferred supplier. Occasionally, where you have a difficult choice between two suppliers, it makes sense to undertake some or all of the due diligence on both suppliers to help you with your final choice.
The following checklists set out the main aspects you should consider and show an example of the questions you might ask when following up references.
|Due diligence task||Done?||Notes and comments|
|Several telephone references with suitable clients of supplier|
|Site visits to one or two especially suitable clients of supplier|
|Visit to supplier's site(s) for support and maintenance|
|Financial stability checks|
|Quality systems checks|
|Technical compatibility with your current and planned IT infrastructure|
|Key technical competencies assessment|
|Supplier's project management skills and methods|
|Outstanding points on software functionality|
|Reasonableness of supplier profits in deal|
|Full and reasonable disclosure of both parties abilities|
|Agreed policy on publicity (e.g. future references, articles etc.|
|Appropriate sharing of risks (e.g. liability, risk transfer) in deal|
|Adequate warranties from both parties to form part of deal|
|Questions||Notes and comments|
|How long have you been working with the supplier?|
|What work did the supplier do for you ?|
|Why did choose the supplier ?|
|What was the competition like?|
How did the implementation go?:
How do you use the software?
Do you do much reporting?
Have you build your own reports?
|How much searching and querying do you do?|
Did the software meet your expectations?
Do you use the software for purposes other than those originally anticipated - if so, what purposes?
|How do you rate the toolset?|
Were you disappointed in:
Any element of the software?
The project managenet of the implementation?
Other elements of the supplier's work?
Support and maintenance?
Do you import/export data ?
|Describe your relationship with the supplier's staff|
Is there any advice you can offer us in our dealings with:
The supplier as a company?
The supplier's people for implementation?
The supplier's people for support and maintenance?
The software itself?
Documentation and ancillary material?
|Other questions specific to your selection|
As a minimum, you should seek telephone references from several customers and visit one or two customer sites of similar type and/or complexity level to yourselves. However much you might be relying on trusted external consultants to help you, it is vital that key charity staff are also involved at this stage of the process – often the final decision between two equally good technical solutions comes down mainly to personalities and a greater feeling of comfort (“simpatico”) with one supplier.
Surprisingly little effort is required to take a proper look at shortlisted suppliers (e.g. a few days in more complex cases for an experienced buyer to scrutinise two or three suppliers). Remember, very few suppliers look exemplary under the microscope, but you will understand the strengths and weaknesses of the supplier before deciding to buy. Further, you can plan around the expected shortcomings of suppliers and avoid unpleasant surprises.
Supplier weaknesses uncovered at this stage often include poor project management and inability to deliver projects on time and on budget to the original project plan. Don't just shrug your shoulders and say "oh well" or "shoot the messenger - they probably just didn't work well with the supplier". Think carefully about whether you want to proceed in these circumstances. If you do, then you should consider applying resource to make up for the supplier shortfall in project management skills (e.g. through your own strong resource or through trusted external advisors). Ask the supplier to explain shortcomings in the references. Also, make sure that you pay a great deal of attention to making the supplier think through the project plan and all the tasks required to deliver. Sometimes you'll find that their estimated costs and timescales increase. So be it, at least you have flushed out the truth before signing a contract. And you can always walk away from the deal if it no longer seems right for you (see 'Contract negotiation and agreement' below).
In completing your due diligence, fill in any gaps that were not adequately covered in the supplier's response to your invitation to tender. Perhaps you still need to assess the suppliers' commitment to quality (e.g. does the supplier have a quality system such as ISO9000/TickIT or a total quality management programme?). Perhaps you still need to assess the stability of suppliers (is the supplier's financial position secure, is it an imminent merger/takeover target). Perhaps you still need to assess the supplier's appropriateness to your organisation longer term (e.g. does it have a significant, appropriate client base or share of the market?).
There are two conflicting extremes or pitfalls which you should try to avoid at this stage of your selection process:
It really is not possible to provide generic guidance on the above pitfalls, simply to say that you should look out for them and try to keep a clear mind throughout.
The purpose of this stage is to:
Even if you have a strong preference for one supplier, it nearly always makes sense to make your decision “subject to contract” and to keep an alternative supplier in play (if there is a suitable alternative) until the contract is signed. Remember, that if you do fall back to your reserve supplier that you should complete your due diligence on that supplier if you have not already done so.
Increasingly, procurement is about imaginative contractual relationships (e.g. testing suppliers' ability to provide services to support products by making levels of support service contractual, job swapping between supplier and charity, performance related pricing). These are techniques that can help you to share the risks and rewards of this relationship fairly, co-operatively and sensibly.
While concluding your evaluations, remember that you still have the (surprisingly rarely used) option to reject all suppliers because none offer sufficient rewards to compensate you for the costs and risks inherent in the project. Don’t let fancy features and fast talk make you lose sight of your objectives. It is always worth stopping and thinking before signing on the dotted line. Is the proposed solution really in line with the objectives, scope and benefits sought? Is there still a business case for proceeding?
Wherever possible, you should ensure that your requirements specification forms part of your contract with the supplier, unless your requirements are entirely standard (e,g, basic financial ledgers).
The contract should normally include a process for installation, testing, acceptance and training, clearly defining both parties' responsibilities.
When buying standard software (or even when commissioning minor modifications to standard packages), the contract will normally comprise a licence to use – you would not normally expect to own the intellectual property. If, despite all warnings to avoid this route, you are set on commissioning a bespoke solution, you should normally insist on owning the resulting intellectual property. Note that in the absence of a stated intellectual property clause the software house is deemed to own the intellectual property, even though you have paid for it.
You will almost certainly require a contract for support and maintenance of the software – this sometimes forms part of a single contract (e.g. a licence to use which includes support and upgrades) and sometimes comes separately.
In particular, you should be prepared to walk away from a supplier if you find them unwilling or unreasonable in seeing through promises or written undertakings to the level of contractual commitment. If trust between the parties has become soured at the contract negotiation stage, it is unlikely to improve as a result of your capitulation over contracts. Indeed, you are probably letting yourselves get set up for a failure.
Before actually signing the contract, remember your initial feasibility and your objectives from the whole process. Are you still going to meet enough of your original objectives? Do the benefits outweigh the costs? Are the rewards you expect from the project commensurate with the risks you are taking? Do you believe you are sufficiently geared up to manage the risks to ensure that the project succeeds? Have you completed your legal review? If the answer to any of the questions is no, either you have not really completed the selection job or there is now an inherent flaw in the original proposition. If the answer to these questions is yes, you should be ready to sign and you should have set yourselves up for a successful implementation. Good luck.