The target dashboard: a new vendor evaluation strategy introduces a new way to think of business metrics

Don's note:  Infrics.com is usually about emerging technologies and strategies around the big trends shaping the future.  Although this article addresses a tactical concern--evaluation of vendor performance--the bigger picture about metrics in the enterprise and the value of subjective input makes this a worthwhile topic.  

In all my years of working with tech research, I don't believe I've ever seen this particular target display anywhere else, and I think it's an idea we can work with across many areas in which you want to know, "how am I doing?"  This article also describes steps to add a social component to vendor evaluation; in the era of you, more and more business work will involve, and benefit from, the same kinds of online social interactions we've come to expect in our lives beyond work. 

This report presents a vendor dashboard, designed to allow executives and vendor managers to create a snapshot view of vendor performance.  Using a simple graphic device--a target--you can read vendor performance and localize vendor performance issues into one or more of 8 KPIs that will be valid across your entire vendor landscape.


This is what it looks like :

This is a sample for a single vendor.  At a glance, you can see this vendor is on-target with their financial performance.  Their positioning report indicates they are both highly strategic and highly critical.  Because of that, the red dot, indicating issues with SLA performance, is an especially serious problem, even though they’ve done well delivering what you initially negotiated.  Although the relationship indicates they are highly flexible and transparent in their dealings with you, caution flags are up for the sales and customer service side.  At a glance, you have a problem alert with this vendor, and a pointer to a likely contributor to the problem.

Four areas touch the evaluation of vendor performance, each is represented by a quadrant in the target display:

1. Financial: Are they competitive on cost, are they performing to expectations with A/P and A/R? What is your assessment of the vendor’s financial risk?
2. Contractual: Are they delivering to the contracted specs? Are they in conformance with negotiated Service Level Agreements?
3. Relationship: How do they measure on interactions with the sales team before the sale, and with their customer service after the sale? Are their processes and deliverables transparent to your organization, and are they responsive to your needs?
4. Positioning:  Is the vendor’s role strategic or tactical? Is delivery of their products and/or services mission-critical, or is it more ancillary in nature? A strategic vendor, such as a consulting company, is not automatically mission-critical, whereas a cloud-based e-commerce storefront is both strategic and mission-critical.

How do you get there?  To start, think of the way you manage vendors for your own home and life: utilities, contractors, stores where you buy food.  Using a very sophisticated--but intuitive--algorithm in your own head, you measure and weigh such things as value-for-price, reliability, customer service, and delivery-as-promised.  You don’t use a formal tool, but when you choose one grocery over another, for instance, it is a response to the that internal dashboard.  

By introducing that analogy, we can start a process that allows your vendor management office to work through this tool in a morning, resulting in actionable information about your vendor landscape health by that afternoon.  At this highest level, it will largely be subjective, but the framework is designed with two ideas in mind: a) you will be quantifying subjective input in such a way that it can be tracked as a measurable metric, and b) you can also roll up vendor-by-vendor structured data metrics as they become available, include them with the subjective evaluations, and use them to make the tool more specific for each vendor while retaining useful comparisons between all vendors you deal with.  As long as your choice of metrics informs the four broadest evaluations and the eight major KPIs, you can abstract complex evaluations down to very useful insights.  Get started right away, add specificity and confidence with iteration.  The dashboard structure will not change, only the data points feeding it.

This tool is designed for use with existing vendor relationships; it assumes that the vendor has been chosen, and that prices, contractual terms, and SLAs are all in place.  

To begin using the dashboard, you use a simple spreadsheet program and give 0-10 values to a range of questions across the performance measurement areas, which generates an X-Y chart in real time.  

Each vendor target snapshot is a grouping of the 4 charts, with each chart’s axis orientation changed  to place better performance at the center of a grouping of all four.  Alternatively, charts for the same quadrant for a number of vendors can be compared to see benchmarks--for instance, as a report to your finance office.  You can view the spreadsheet with sample data here, or download a blank one as a template here.  

Best practices and resources in vendor performance evaluation

Now that you are equipped with tools to begin seeing useful measurement, let’s also examine the underpinnings of vendor performance within an enterprise, an IT operation, and a Vendor Management office.

What do I measure? Where does the data come from?

Common across experts in vendor performance management is one theme: “don’t get buried in too many metric data points, measure that which informs your knowledge of a vendor’s contribution to business value.”  Restated for the purposes of the dashboard presented here, it says, “add specific metrics that contribute to your confidence of the accuracy of the eight KPIs you’re tracking.”  

Some are intuitive: financial performance data from your A/P & A/R systems. Delivery-as-scheduled information from ERP. Uptime and latency reports from the data center and Operations. Cloud vendors like Salesforce.com expose internal performance dashboards. Others require the trick of stating subjective values from end users as a quantified value, like “on a scale of 0-10, how satisfied are you with Vendor X’s customer service?”  The vendor performance spreadsheet included in this report is easily edited to add values to measure; always keep in mind the operating concern, “how few things can I measure and still believe my KPI answers reflect the actual experience with our vendor?”

Because you must include strategic value and criticality as part of your dashboard process, ownership of vendor evaluation is most effective within the business unit that best understands those interests. Typically for CIOs this would be a vendor management office, a program management office, or within a high level division devoted to operational excellence and strategy.  Procurement and Finance are important contributors to the dashboard, not logical owners of the process.

For more on this topic, see “Supplier Metrics that Matter,” by Jonathan Hughes, from the CPO Agenda site. Hughes also confirms the value of the four broad measurement areas in this report as choices to organize your view of the vendors in your portfolio.


How do I compare my vendor’s performance with other clients, or against similar vendors?

Become social about your vendors.  With the speed of social information sharing, you need to know news about your vendors in real time, and know what others are saying about them.

This is an area where emerging use of social tools in business offers a lot of promise, and several opportunities to share vendor experiences in the same way Facebook friends rely on each other’s opinions about shopping choices.  

Supplierevaluations.com uses the Angie’s List/Yelp model to socially gather and share customer’s supplier opinions.  Because of the prevalence of the term “supplier” in procurement and supply chain, you will frequently find it used instead of vendor in discussions relating to vendor management.  They are not strictly interchangeable, but they are closely-enough related that any discussion of vendor measurement will be richer for the inclusion of “supplier” as well.

Use Twitter.  Search hashtags (the keywords commonly used to filter Twitter topics) such as #vendor, also try #ensw (enterprise software.) Use conventional search as well; “vendor management” returns a useful list of discussions.  Whereas Twitter is unmoderated, comments on vendor pages on Facebook and/or Google Plus can be edited by the companies, and may not be accurate representations of user feedback.  Follow each of your vendors on Twitter to see user discussions as well as news from the companies themselves.  If you do not already use a social feed management tool like TweetDeck or HootSuite, they can do a lot to help you organize streams of information into feeds that make sense.

Add monitoring tools like socialmention.com, which will track keywords like vendor names across all social media, and can send e-mail alerts.  If you use an internal social tool like salesforce.com’s Chatter, you can get real-time feedback from your own users by following vendors as keywords.


What companies offer vendor performance management software?

These are major vendors. As you see, the field is currently experiencing a lot of acquisitions:

SAP and Oracle, as part of Supplier Lifecycle Management (SRM) modules

Emptoris, offering supplier lifecycle management tools including vendor dashboards and reports. IBM acquired Emptoris on Feb. 1, 2012.

Ariba, who offers a supplier management tool as part of a web-based service called the Ariba
Commerce Cloud

BravoSolution, formerly VerticalNet, offers Supplier Performance Management software, as well as a suite of other products branded Supply Management Excellence

Open Ratings, acquired by Dun & Bradstreet, offers some supplier performance management through their Supplier Risk Manager.

Biznet Solutions, a web-based supplier performance management tool

Other resources:

For members of the CIO Executive Council, there are member-contributed vendor scorecard templates at http://council.cio.com/content.html?content_id=24.910.6eaaaa58.

Sherry R. Gordon’s book, “Supplier Evaluation and Performance Excellence,” cited in this report.

Conclusion

By its nature, a dashboard is a summary, an “at a glance” insight into the health or performance of a system.  The use of Supplier Lifecycle Management tools or vendor supplied dashboard data is still worthwhile, and can contribute to your dashboard.  Neither excludes the other.

As you work with the dashboard and the KPIs, they can inform evaluations and SLA development for future vendor engagements, driving expected (and contracted) vendor performance across the eight measurement areas.  Use the tool and stay focused on answers to fundamental questions represented by Relationship, Financials, Performance, and Positioning.   Share findings with those who use the vendors products, and with the vendors themselves, involve both in the process of knowing and improving vendor performance and value delivery.