Why Spend Analysis Frustrates Those Who Need It Most
|Why Spend Analysis Frustrates Those Who Need It Most
by Eric Strovink
January 16, 2007
It's easy to construct a data warehouse to answer questions that don't deviate from its pre-defined structure. But what about all the other questions?
Within most organizations there are three groups of consumers for spend data:
- Group I: those interested primarily in reports;
- Group II: those who drill around a spend dataset, occasionally, to explore specific areas of interest, or to track down individual payments;
- Group III: power users who are using the spend data to locate, drive, and monitor the next level of savings initiatives -- initiatives that are aimed past low-hanging fruit that has already been harvested.
Satisfying all three of these groups is beyond the reach of most of the current generation of spend analysis products. The issue is related to the essential difference between a data warehouse and a data analytics product. It's very difficult to be both, and putting the word "analysis" in the product name doesn't make it so. A data analytics product has to be able to change data, dimensions, and rules, instantly and easily, on the fly, to support whatever ad hoc analysis is required. Contrariwise, a data warehouse cannot change frequently -- by its very nature, it is shared by dozens, even hundreds of users. It has to remain static for weeks, if not months. Data warehouse-based analyses therefore have to be done with whatever data structure was in place when the warehouse was published. Much of the time, that structure is either useless (a committee decision, for example), inappropriate for the analysis at hand, or obsolete. That's why the concept of using a data warehouse to do ad hoc data analytics is flawed. Imagine a spreadsheet whose structure cannot be changed -- it may be useful for a specific purpose, but it is useless for any other.
Which is why the target user for spend analysis vendors has usually been the Group II user. It's relatively easy to construct a data warehouse that can be used to answer questions that don't deviate from its pre-defined structure. But what about the other consumers of spend data, and all the other questions? Attempts have been made to build suites of static reports to satisfy the Group I users, but static reports are flawed, simply because they are static. Everyone is acquainted with a report that doesn't do "quite" what one wants, or a report that rolls up the wrong way, or provides totals that have to be imported into another tool (perhaps manually transcribed, even) in order to derive meaningful numbers. And, unfortunately, it's not possible to point a standard reporting package at an OLAP database and expect anything but hours-long delays, because relational reporting systems simply don't work on voluminous datasets. A reporting facility has to be adaptable to dataset changes and requirements changes, and it also must leverage OLAP power to generate its results -- without requiring its users to be IT experts. Custom reports from the vendor, or reports built by hand, by in-house experts, are therefore typically the only "solution" -- an expensive and frustrating one, to be sure.
Group III power users are usually the first to realize that a spend analysis system isn't providing value. After an initial burst of enthusiasm of the "I didn't know that" variety, it becomes apparent that fixed hierarchies and fixed dimensions are of little use for ad hoc analysis. The power users' response, because they are more proficient with data manipulation than other consumers, is to work around the tool by downloading raw transactions from the warehouse directly onto their desktops. Then they hack at those transactions manually with Access and Excel to produce reports and analyses in the same way that they've always done. The value of the spend analysis package to them is minimal -- they think of it merely as a cleansed transaction store.
At the end of the day, then, what is the look-back on a typical spend analysis project? Often an enormous expenditure has been undertaken; a spend dataset has been made generally available, with great effort; but after a brief period of cherry-picking and high value, the static reports generated from the system are only marginally useful. In some cases, usage of the system actually drops to zero, other than a brief spike of activity after every refresh period, during which the Group III users download a fresh set of raw transactions to feed to their hand-built offline reporting tools.One can hardly expect a frustrated organization that has gone through the above process to expend even more time and money on the creation of dozens of new commodity-specific spend datasets -- yet this can be key to the next level of value in the spend management process (see " Spend Analysis: MacroMap and MicroMap).