Evaluation and Selection

In our last post we looked at requirements definition. It was discussed how you need to identify and document requirements and quantify those requirements to create your essential criteria. It was also mentioned the importance of scaling down your POTENTIAL supplier list based upon your essential criteria. Today, we are doing to discuss Evaluation and Selection. (Please bear in mind that the following is geared towards an enterprise computerised system solution - but you can scale these steps as guidance for your needs). 

Evaluation:

At this stage, you probably have quite a long list of potential suppliers whose marketing will seek to convince you that they are what you need. Your role is now to remove the marketing and see exactly what the potential supplier can and cannot provide you. Do not be sold by the glitz and glamour of sales presentation. An internet based presentation will allow a much more focused look at the software and its potential fit for the organisation’s needs.

• Perform a GxP Assessment against the requirement. Identify, whether the solution provider will need be checked, if their quality approach is robust enough to provide a quality solution, (and perhaps supplement any potential validation needs).
• Research software solution against each requirement.
• Rank each solution against each requirement.
• If necessary, conduct interviews with a selected list of suppliers using a predefined questionnaire, based on the requirements. Keep the conversation focused on a list of prepared questions. This will provide enough information to begin comparing supplier against supplier and will probably allow for the immediate elimination of a few suppliers.
• Reduce the list of suppliers down as a response.
• Request a web based demonstration of the top ranking solutions and grade the outcome of the demonstration. Demonstrations can last from between, 1 to 3 hours, hence the need to reduce the list of potential suppliers.
• Develop a demonstration script and provide this to the selected suppliers.
• Reduce the list of suppliers down as a result of analysing the responses.
• Create and send a request for proposal to the top 3 suppliers.
• Reduce the list of supplier down as a response.
• Create and send a request for proposal of the top 3 suppliers.
• Request an onsite demonstration using defined use cases that YOU have prepared. This is, without a doubt, one of the most significant steps in any software selection process. Unfortunately, it is typically one of the most poorly managed steps in this process. Ensure to capture the teams reaction to each demonstration immediately after, when the information is still fresh in their minds.

It is very important to maintain control of the onsite demonstration, so that the users witness the actual value of the solution and not the value the supplier wants to portray. If the supplier representative spends the majority of the demonstration reviewing presentations and little time in the actual solution, then the supplier may be attempting to prompt the users to perceive a higher level of quality that is not necessarily in the solution. A supplier who invests little in the onsite demonstration may offer little investment in after-sales support. This is a useful marker for the selection process.

Selection:

• Evaluate the RFP responses. For example, is the response well thought out and written? Was all the known requirements quoted for? Is the quote within budget? Do they have a QMS?
• Evaluate the on-site demonstrations. For example, was the demonstration script followed? Did the system look user friendly? Were all questions answered confidently and satisfactorily?
• Reduce the short list to the top 2 suppliers.
• If necessary, perform and audit of the potential suppliers premises, QMS, SDLC, security and so on. Ensure that quality is built into the solution and whether the supplier’s documentation can be leveraged if the solution impacts GxP. Ensure that you are happy with their backup process, business continuity and particularly their support and maintenance approach.
• Select the supplier of choice and notify the suppliers of your decision.
• Negotiate the contact.

Contact us today to hear more about our robust COTS approach.


Next time we'll take a look at Implementation.