Testing the Waters with a Proof of Concept – Part Two
Part Two – Proof of Concept in Vendor Selection
As discussed in a previous post, a Proof of Concept (POC) is equally valid in vendor selection and as a implementation milestone. In this post, we look at the POC as part of the vendor evaluation. Conducting a POC in this scenario will allow the customer to:
- Evaluate key functional and usability requirements of a prospective system (with the understanding that some functionality might not be supported yet)
- Make a go/no go decision. If Go, then move to implementation, if NO GO, conduct a POC with an alternative vendor
- Evaluate the “fit” with the vendor, their responsiveness and understanding of the customer’s business prior to committing to purchase and implementation
What it is (and what it isn’t)
- It is a basis of vendor selection
- It is not part of the implementation
- It is conducted over a short period of time on key scenarios, typically the ones that have been highlighted as being of most concern for the business
- It should not require a full-blown configured system
- Should be a paid trial for the vendor with fees being deducted from the final price should the customer decide to go with the vendor
Prerequisites for Conducting a POC as Part of a Vendor Selection:
- There must be a clear understanding and communication between the customer and vendor that the POC is part of vendor selection rather than an implementation milestone.
- It is a good idea to have clearly documented business processes to ensure that they system supports current, or in the case of a transformation project, the to-be processes.
- The functional requirements must be documented and prioritized so that they can be used for evaluation testing.
- Identification of key business users to evaluate the system functionality, usability and business process support. It is fine for the solution to be technically elegant, but if it is not suited for the purpose of the business it will not be used.
- Identification of other potential vendors. If there are no other appropriate vendors, then conduct the POC as an implementation milestone.
- The POC should last no more than 2-4 weeks.
Creating the POC Environment and Running the POC
- Document key scenarios / Test Scripts with pass/fail criteria that will be used by the key business users in their evaluation testing.
- Develop an evaluation scorecard and scoring parameters that the key business users can use to score how well the vendor’s system satisfies the functional, technical and usability requirements. If the system provides the functionality and technical specifications required, but does not facilitate the customer’s business processes and workflows because it is not usable or requires numerous workarounds, this needs to be captured and scored.
- Conduct user training on the system as well as on the use of the scenarios/test scripts and the evaluation scorecard with the key business users. This should be a joint effort between the vendor and the customer
- The vendor and/or customer need to prepare a POC environment where the key business users can test the system away from the demands of their jobs and the system is set up with a test database (and, if appropriate test data), required interfaces, etc.
- Carefully select the test data to use as part of the POC so that it is meaningful and relevant to the test users. For example, it might be appropriate to use a combination of last month’s data pre-loaded on the environment, and in addition ask the users to enter current data during the POC.
- The key business users use the scenarios/test scripts to evaluate the system and record their results using the evaluation scorecard.
Outputs of the POC
After the testing and evaluation has been completed, the customer will decide whether to select the vendor or to engage in another POC with another vendor. Because of the effort required by both the vendor and customer, a POC for vendor selection should be entered into with only one or two vendors, with the preferred vendor going first.