Business intelligence survey reveals mixed satisfaction in its results

Rethink IT, Oct, 2004

The software boom of the late 1990s saw business intelligence taking a prominent position among a raft of popular and often overhyped technologies claiming to make enterprises more effective and well run. With roots in database query tools and executive information systems, the new breed centered on data warehousing, data mining and analytics. Like most software, these technologies went into a decline in the IT recession and the sector has seen extensive consolidation in the past year or so. Yet many CIOs swear that business intelligence tools have helped their companies survive recession and become more competitive and agile. In this survey, we questioned 100 European CIOs in large organizations to see how far there is new growth in the sector and what IT users think about their investment so far in data warehouses and related products.

A year ago, we saw the start of a major shake-out in the business intelligence sector, with a hat trick of acquisitions within two weeks of each another--Hyperion's decision to buy Brio following only days after the news that Business Objects would take over Crystal Decisions, and shortly after that, Lawson announcing it would purchase Numbercraft. This continued a wave of consolidation that had been kicked off back in December with Cognos' $160m purchase of business performance management firm Adaytum, followed by Geac's purchase of Comshare for $52m in June.

Business intelligence has traditionally been controlled by medium-sized specialists and, as in ERP, companies of this size are racing to fend off acquisition by the big names, by increasing their own size through quick friendly mergers.

Business Objects was tipped as a potential takeover target for Oracle, and may be seeking to make itself less easy prey, as well as dramatically extending the range of capabilities it offers in a world where comprehensive function rather than niche excellence is currently in favor. It also needs larger size to fend off Microsoft, which has become a serious player in this market in the past couple of years.

Along with privately held SAS Institute, Cognos is the stalwart of the sector. It is playing up the disruption that its rivals will suffer as they try to integrate two product lines and cultures--something Hyperion struggled with when it previously bought Arbor. It seems likely that Cognos itself will be casting around for acquisitions of its own soon.

The wave of acquisitions last year indicated that growth in the market was slowing. IDC forecasts that the BI market will grow at an annual compound rate of only 4.1% between now and 2007, a far slower rate than in previous years, particularly the peak of 1999-2000. But there is still room for pure play vendors in this sector, unlike many software markets where specialists have long since been absorbed into larger suppliers' suites.

So, at the other end of the spectrum from Cognos or Oracle is a host of small analytics specialists such as Applix. Some of these are also likely to fall prey to larger companies seeking to add depth to their ranges. There will always be a customer base that requires very sophisticated tools at a level that cannot be delivered by the general purpose software houses.

This month's survey is about Business Intelligence software, where live transactional data is copied from a transactional system for deeper analysis. This may be through explosion into summarized form or indexed form for easy interrogation, (data marts or data warehouses).

Some software tools--often separate from the data warehouse--are used to query or examine this data more effectively or to generate regular reports for business managers.

Sometimes when handling very large amounts of data, special tools that look for patterns or trends that might not be immediately apparent with simple scrutiny, can be used, and this process is called data mining. The survey also asked a few questions about these tools.

These types of tools are collectively called Business Intelligence tools and are sometimes referred to as analytics, and are used by many large businesses to gain a greater understanding of just what their customers are buying, where they are buying them from, and for what reasons. Clear decisions, taken once this type of understanding has been achieved, can often be shown to result in improved sales and improved customer satisfaction.

THE RESULTS

Well over half the respondents had some form of business intelligence tools in place for operating data warehouses or data marts--58% had these tools. Almost 59% of these sites also had ETL (extract, transform and load) tools for loading data from transactional systems into warehouses or data marts. Predictably, no sites had ETL without data warehouses or data marts.

Of the sites with data warehouses, 57% also had data mining tools in use. Mining was in use by 36% of the total sample, 4% less than ETL. Three sites had data mining running without any data warehouse, data mart or ETL.

Of the total sample, 26% ran all three technologies--data warehousing, data mining and ETL. 17% had a data warehouse or data mart with no other related tools.