(WEB HOST INDUSTRY REVIEW) — In an effort to make more informed business decisions, organizations are adopting sophisticated Business Intelligence solutions to capture, analyze, warehouse and mine customer information. But it’s not just about technology; It requires people, processes, and technology working together to so that the company can respond more quickly to change.
This is the basic idea behind HP’s “connected intelligence” approach to BI. Connected intelligence connects the dots between data from across the organization, probing places like BI apps used by marketing, sales reporting systems and profitability tracking applications in financial divisions. And not only does it create a data warehouse, but also a trusted data environment that encourages business users to standardize data across business teams and develop an enthusiastic community.
Vickie Farrell, marketing strategy manager for HP’s Business Intelligence solutions, marketing and alliances, frequently writes about the opportunities and challenges BI poses for businesses on the HP Business Intelligence blog.
In an email interview, Farrell explains how connected intelligence can enable businesses to compete in an information-centric world.
WHIR: Some business people I have spoken to have said that what they’ve seen over the past 18 months or so has had them question a lot of their conventional wisdom on how they operate their organizations. Did you see the rise in business intelligence use corresponding to the declining economic conditions during the financial crisis?
Vickie Farrell: With the recession have come emerging regulatory and transparency requirements, the need to better manage risk, control fraud, and not just increase the ability to cross-sell and up-sell, but to improve the whole customer experience and provide better service as a means to retaining customers and increasing their long-term value.
Prior to the financial crisis, companies were focused on generating a consistent set of accurate financial reports to help them understand and manage the business. The goal was to achieve a single version of the truth of (historical) data in the data warehouse for analysis. But analysis based on history means you’re using yesterday’s results to determine tomorrow’s performance. And many companies were beginning to plan for making their data warehouses and BI systems more “operational.”
Some companies kept data mart proliferation in check, but many did not. Where there was data mart consolidation, the approach was to move the existing data structure (problems and all) to a new platform. In most cases, data quality, data governance, MDM were left to be dealt with “later.”
With the onset of the financial crisis and economic uncertainty, priority shifted. IT went into cost-cutting mode, becoming more scrutinizing about acquisition costs and payback periods, and replacing traditional data marts and warehouses with new low-cost appliances. Dollars per terabyte moved to the forefront in evaluating data warehouse platforms. Low-cost and open source BI tools took hold in the market. The need to improve staff productivity put emphasis on increasing efficiency of costly data integration processes. We have seen investments in the past couple of years in data quality, data governance and MDM products and services.
As we begin to see a leveling off of the economic decline, we’re seeing another shift. The need to cut and control costs continues. There is a sentiment that this is not a temporary recession, but a new economic reality. On the other hand, competition is keener, driving the need for better customer management. There is recognition that if you can’t respond to conditions in the moment, you can’t influence their outcome. It is this dilemma that is driving a change in companies’ use of business intelligence. There is a strong belief that BI can be tapped to help control business and IT costs, but at the same time, it is imperative that they do a much better job of exploiting data as a valuable asset, to do predictive analysis, make use of data generated by social media and other unstructured content, optimize operational decisions, etc.
IDC notes that industries will “come out of the recession with a transformation agenda and look to IT as an increasingly important lever for these initiatives.”
Responding to this dilemma calls for a different BI infrastructure than what companies have been using. Here are just a few examples.
- More collaboration with partners, suppliers, trade associations, consortia, industry information exchanges. This means better quality data and a common data framework for sharing and reuse.
- More analytic support for operational decisions and processes. This means lower data latency, real-time analysis added to the current BI load, and high availability systems.
- Improved decisions. This involves integrating data from many different sources in real time, including seemingly insignificant and unrelated events.
In summary, we see both a rise in use of BI as well as a different use, driven by not so much the recession itself as the widespread belief of a new economic reality as we move forward.
WHIR: Increases in technology have only recently opened up the possibilities of collecting and analyzing massive amounts of data. Can you tell me a little about the infrastructure needed to process this data and present it in usable ways?
VF: There are numerous new and existing technologies aimed at increasing the ability to collect and analyze large volumes of data by improving performance: data compression, 64-bit computing and in-memory processing, in-database analytics, MapReduce, column-oriented database storage, massively parallel processing and flash memory, to name a few. But there is more to this challenge than just the volume of the data.
We are entering a new generation of DW/BI with an expanded set of business needs that pushes the data requirements in several ways; in several directions. Just a few examples:
- Analyzing unstructured and semi-structured content along with structured data
- Integrating data from many sources, including federation of data not in the data warehouse
- Using a Complex Event Processing engine to analyze incoming real-time data streams, joining event data with data persisted in the DW
- Lower data latency – loading data into a DW for analysis on a continuous basis (vs. periodic batch load, including frequent mini-batches)
- High availability systems that support data analysis for mission-critical applications
- Automating data classification and business actions
And organization’s data volumes as well as the number and types of users plus requirements in any or all of these areas will dictate the needed infrastructure, which will be different in each case. But for sure, an infrastructure that is built today will need to be able to process the data volumes, as well as address the above requirements, either now or in the near future because they represent the direction in which BI is going.
WHIR: Unlike some other business services, BI is not one-size-fits-all when it comes to implementation. When a business comes to HP asking for BI, how involved do businesses have to be in order to ensure very pervasive BI?
VF: The goal is not necessarily for BI to be pervasive, although wise organizations would like to leverage and maximize their investment in DW/BI. The initial goal is to address a business problem or opportunity. The more business people can be involved in the design, development and implementation of the BI system, the more likely it is that it will meet their needs. In addition to defining the needs, they can act as advocates to the rest of the organization in encouraging use of the DW and communicating success.
One of the things that HP helps clients with is overcoming the cultural and political issues in sharing and integrating data. Many of our clients consider us a trusted advisor in helping them manage these issues. HP guides clients in the establishment of not just a data warehouse, but a trusted data environment that will encourage business users to standardize data usage across business teams and develop a thriving data user community.
The use of BI has not become pervasive in organizations. BI usage is only about 24 percent. I just published a blog post where I discussed many of the obstacles that must be overcome before it can be more pervasive:
WHIR: One of the success story examples of an organization using BI was Blue Cross Blue Shield Kansas City, which, among other things, used its newly found insight into its data to streamline internal processes, and even expose previously unavailable data to physicians could use that knowledge to better treat patients. It seems to me that some of the benefits of BI in this case were, perhaps, not altogether anticipated. What are some of the potentially unforeseen or unexpected benefits of BI for organizations?
VF: This is a fascinating topic. People pay money to hear companies tell these stories. I’ll give some examples that I’ve seen over the years.
- I think it was a surprise, even to Wal-Mart, when several years ago, they were able to track the spread of flu across the US more precisely and accurately than the CDC, based on purchases in their stores of TheraFlu and similar products.
- I have seen a few occasions where BI analysis done in the data warehouse had large discrepancies with the operational system reports. Further investigation uncovered errors in the operational systems which had an impact on the company’s business.
- Continental Airlines’ BI system allowed them to understand and serve their OnePass customers so well that when Jet Blue was launched in the early ‘00s, Continental was able to forgo the route their competitors took when they introduced low-cost lines like Ted and Song. Continental’s analytics enabled them to differentiate themselves on their service and attract and retain the customers willing to pay a premium for better service, a move that helped them maintain a strong competitive market position in the face of downward price pressure.
- Technically, data mining is different from BI, but if you’re using the broad definition of BI which encompasses everything including the data warehouse, then we can talk about data mining here. Discovering the unknown and unanticipated is what data mining is all about, as illustrated in the proverbial beer and diapers example. As the urban legend goes, analysis of Osco store receipts indicated a correlation between purchases of beer and diapers, and moving them near each other resulted in a spike in sales.
WHIR: Is there anything else you would like to say about BI?
VF: There is indeed. And much of it can be found on the new HPBI blog where I am a primary blogger and have published several posts:
No related posts.
