BI Scorecard Blog

BI Adoption Remains Flat

By Cindi Howson, BI Scorecard

(Please note new blog address:  http://www.biscorecard.com/blog/.  This blog will no longer be updated.)

The world of business intelligence has been revitalized by the buzz of big data, the promise of personalization, and the agility of visual data discovery. Yet many companies struggle with the basics of accessing data, particularly new data sources. BI has long been considered a tool for power users and for management to run the business, but easier-to-use tools, mobile BI, and visual data discovery have allowed new classes of users to exploit the value of data.

BI Scorecard's annual survey of BI users, administrators, and directors shows success rates have remained flat since the survey was first conducted in 2006 (based on 513 respondents). Success is defined by BI's ability to improve access to data, decision-maker's perception of value, and impact on revenues, customer service, and operating efficiency. However, the portion of customers saying BI has delivered significant business impact declined in 2013 by six percentage points to only 28% -- the lowest since the survey began. While there are some very successful and impactful BI deployments, the majority are stuck in the middle, with respondents reporting only slight to moderate success and business impact. On the bright side, few respondents (3%) reported outright failure.

BI adoption as a percentage of employees remains flat at 22%, but companies that have successfully deployed mobile BI show the highest adoption at 42% of employees. (see Figure)

Survey respondents say that if they had appropriate budget, clean data, and organizational policies that support BI, 50% of employees on average should be using BI. The gap between deployed BI and potential BI users shows that companies are not tapping BI's full potential. While much of the industry focuses on software and technical innovations, the main barriers are organizational and cultural, according to 84% of survey respondents.

Executives can address organizational and cultural challenges, making leadership the most important organizational criterion for success. Aligning BI efforts to business goals is the second most important success factor. Alignment requires a strong business/IT partnership and ongoing dialogue, something many companies lack. BI teams that report directly to the CEO or to a line of business show the most significant business impact. However, strained BI resources make it increasingly difficult to expand the reach of BI or for companies to take advantage of new innovations. As demand for new data sources, more users, and analytic complexity increase, BI teams have to work smarter, and business users have to embrace self-service BI.

Most companies (66%) have standardized on a BI platform across the enterprise or per line of business. However, the companies that deploy tools from specialty vendors report higher success rates and business impact than companies that use one BI platform exclusively.

Big data stole most the headlines in 2013, but dashboards, new data sources, and self-service BI are the main priority for investments in 2014, followed by mobile BI, visual data discovery, and upgrading to the latest BI releases

I have long advocated that customers should evaluate the quality of customer account management and technical support. The greater the partnership between a BI vendor and a customer, the greater the likelihood that both sides will manage—and meet— expectations.

Overall, customers of specialty BI vendors such as Tableau and Qlik had higher satisfaction rates, while those of larger BI vendors Oracle and SAP were less satisfied. However, the size of the BI vendor alone does not equate with lower satisfaction, as Microsoft, for example, scored well on quality of technical support. The quality of technical support has a huge influence on success and business impact. We are talking software, and sometimes disparate components, so it's almost a certainty that BI teams will encounter problems.

The difference among BI vendors is in how quickly and adequately those problems are resolved. And yet, the quality of technical support for some BI vendors has only gotten worse as they've built broader portfolios and experienced high employee turnover. How much does it matter? Customers who rate the technical support as very good to excellent, also have the highest degree of business impact, with 45% rating BI's impact as significant, compared to the survey average of 28% rating the impact as significant.

The BI Scorecard Successful BI survey is not sponsored by vendors. The full report can be purchased here. For a free copy of the 30-page highlights, take next year's survey. Sign up here to be notified of when the survey opens.

Big Trends in BI for 2014

By Cindi Howson, BI Scorecard

Note: this blog is being migrated to a new site. If you have this on an RSS feed, please visit new location for continued updates: http://blog.biscorecard.com/

In some respects, 2014 echoes some of the same themes as 2013, but faster and louder when it comes to visual data discovery, cloud, and mobile. Big data will also continue to garner attention, but with a more pragmatic view. Simplicity, meanwhile, experiences a second coming. Here are the top five trends for 2014.

1. Visual Data Discovery and Self-Service BI

Visual data discovery took center stage in 2013, with specialty vendors (Tableau, QlikView, TIBCO Spotfire) growing rapidly and mega vendors (SAP, IBM, Microsoft, Oracle, MicroStrategy, SAS) stagnating. BI heavyweights have taken notice that more agile and visual solutions have eaten into their bread-and-butter, query-and-reporting market share. All have responded with new interfaces and solutions. The intial releases of visual data discovery solutions from mega BI vendors tended to have limited capabilities, but the gap is slowly closing. Both the specialty vendors and the heavy weights -- are trying to find the balance between agile yet trusted data -- user driven but IT controlled. It's not an easy line to straddle.

The entire BI tools market was largely founded on the concept of self-service BI: let users create their own queries without having to know SQL. The difference is in how much IT is involved upfront and how much users are able and willing to do themselves. Visual data discovery tools have brought a greater degree of ease, appeal, and flexibility than other BI modules such as production reporting, business query, and dashboards.

Tip: If you haven't already made visual data discovery part of your BI portfolio, your company is missing an opportunity to empower business users. Make it a priority to adopt in 2014.

Here's what you can expect from vendors in the year ahead:

• Tableau , TIBCO Spotfire, and MicroStrategy released major new versions in 2013 that they will be pushing customers to adopt in 2014. Actuate also released a new version of its platform, Actuate Analytics, with integrated, advanced capabilities for customer segmentation.
• QlikView Next, currently in beta, brings a next-generation architecture to market and is expected to become generally available this year.
• LogiAnalytics released a new LogiVision interface in early January aimed at the business user and competing with visual discovery solutions.
• The concept of story telling is increasingly appearing in BI products in which information and analyses are presented to support a decision-making process -- a bit like PowerPoint. Look for this in the latest SAP Lumira, Tableau, and QlikView.Next products due in 2014.
• As visual data discovery deployments expand, roles and responsibilities for BI teams must change to ensure business agility doesn't degenerate into chaos. BI teams that have taken a hands-off approach to visual data discovery tools need to support data provisioning and best practices; conversely, users who have relied on IT for every last sort, filter, and customization will need to learn to fish for themselves.

2. Cloud

Cloud has been hot in almost every software segment except BI. Salesforce.com and NetSuite have put a dent in sales of business application market leaders SAP and Oracle. But the same cannot be said in BI, where cloud BI vendors such as Birst, GoodData, and Indicee have remained niche players.

I think 2014 will be the year that cloud finally takes hold in BI. Part of the change will be due to market maturity, but also, the degree of pain and opportunity cost for BI teams to maintain their own infrastructure has reached a tipping point. BI resources are limited. Would your company rather have that available talent spend time accessing and exploring data, testing innovative ideas, and bringing in more data; or do you want them bogged down maintaining infrastructure and upgrading software?

What are the signs that 2014 will be a pivotal year in cloud BI?

• Birst and Gooddata gained additional VC funding in the second half of 2013
• JasperSoft's Cloud Analytics launched in 2013 as a pay-as-you-go option on Amazon Web Services and it's showing strong momentum.
• Look for MicroStrategy to unveil a similar initiative to make their software available through a broad set of public cloud providers, such as Amazon Web Services (AWS), with users subscribing by the month, day or even hour.
• Microsoft Power BI, a SaaS solution for self-service business intelligence, currently in technology preview, is expected to be available in 2014
• Oracle intends to launch a SaaS-based BI offering in 2014
• IDC predicts cloud spending overall will grow 25% in 2014.

With more choices and scalable databases offered in cloud environments, concerns about data scalability seem less valid than in the past. Some BI vendors let you keep your data on premises while using the cloud for software delivery, giving customers the best of both worlds.

Tip: To get started in cloud, consider a proof-of-concept project for a line of business looking for particular functionality not available through your on-premises BI platform.

3. Mobile

For years we've been hearing "mobile first" from a number of BI vendors, so you'd expect a degree of maturity and stability in this segment. However, the market is still debating how to deliver mobile, what capabilities to offer, and which devices to support. For many BI teams, mobile BI is still something you have to push users to embrace, despite the industry hype.

There's significant room for improvement in mobile BI, with ig big differences in functionality remaining in four areas:

• HTML5 versus native apps: To date, native apps have provided the best user experience, but might 2014 be the year that HTML5 takes over? Developers would like to develop once and deploy anywhere. Users, meanwhile, want the best user experience, with full touch-screen functionality, rich graphics, and great performance.
• Security: Mobile Device Management systems allow customers to grant authentication at the device level in addition to supporting user names and passwords. They also support remote wipe and lock out in the event a device is lost or stolen. Some BI vendors provide this functionality within their own platforms, and others rely on third-party MDM vendors.
• Interact or author: I've surveyed many users and found that for now, they primarily want interactive consumption of BI in the form of view, refresh, sort, filter, and drill. Authoring reports and capturing new data may be cool features, but most BI users are still waiting for the basics to be covered.
• Offline: Airplane mode aside, there are also many locations -- remote buildings, warehouses and so on -- with limited cell service. For these situations, offline capabilities are important for mobile BI, yet few vendors provide this functionality. In particular, consider whether the offline support that is available is viewing of static page images or an interactive app that can explore cached data. When offline use is supported, security clearly needs to be robust.

Tip: Just because users aren't asking for mobile BI doesn't mean there is no business value. Identify mobile workers and pilot the capabilities so users understand the potential.

4. Big Data and Little Data

Big data may be mainstream news, but it's nowhere near mainstream deployment. According to a recent TDWI survey, only 10 percent of companies have deployed big data technologies, and a similar finding from a Tech Target survey puts the number at 14 percent.

As I discuss in my book, Successful Business Intelligence: Unlock the Value of BI & Big Data, there is still confusion about what "big data" means. Many consider it synonymous with Hadoop and NoSQL technologies and, in the early days, predicted the demise of the data warehouse. Fortunately, even providers of Hadoop solutions, such as Cloudera, are increasingly recognizing that Hadoop is only one part of a big data ecosystem. Analytic appliances such as IBM PureData System for Analytics(formerly branded Netezza) , Pivotal Greenplum, and HP Vertica (to name a few) are also part of the big data market. In-memory computing with the likes of SAP Hana, Oracle Exalytics, and Microsoft Hekaton (due in 2014) also supports big data analysis.

The earliest adopters of these technologies may have large data volumes, but others have small data volumes. These organizations are looking to mine insights from unstructured content such as sensor data, tweets, Facebook posts, and email messages that arenot so easy to explore with traditional relational databases. They, too, need new technologies even if it's not about "big data."

In the year ahead:

• Expect tighter integration between Hadoop technologies and traditional data warehouse storage with a greater emphasis on the right use cases.
• Look for better support from BI vendors for a wider range of big data sources.
• In the Hadoop-only world, specialty solutions from startups such as Datameer, Platfora, as well as new entrants will mature.
• SAP, Oracle, and Microsoft continue the quest to run transactional applications all in-memory, threatening traditional disk-based storage.
• Vendors will be innovating, but BI teams are largely still getting educated.

5. Simplicity

My dad was a disc jockey when I was growing up, and our stereo system, with so many knobs and dials, was so complicated that I didn't dare go near it. In much the same way, many business people tend to shy away from BI tools. There have been attempts to achieve simplicity in BI, but with lackluster success.

Expect 2014 to be the year that the new face of BI will be simple -- without sacrificing smarts. These products will be driven in part by visual data discovery, but also by search, improved dashboard interfaces, and embedded BI.

In the area of search, the year ahead will bring:

• Oracle Endeca 3.0 (released in late 2013), with cleaner visualizations and faceted navigation of both structured and unstructured content.
• Microsoft Q&A, an interface for the upcoming PowerBI release that will let users enter simple search terms to find existing content or to create data visualizations.
• IBM Project Neo, a SaaS-based offering that combines search, smart visualization, and advanced analytics
• Startups NeutrinoBI and DataRPM are using search and natural language processing to bring simplicity to a broader base of users.

Other trends

In writing this article, I started first with what was top of mind and tried to hold myself to five – a tough constraint. I also had a look at what I thought would be big in 2013 and whether or not those trends proved to have staying power. Last year, I wrote about next-generation dashboards, and vendors certainly delivered in that space. In-memory as a computing approach was more of a headline last year that has become integral to big data, cloud, visual data discovery, and mobile. I've held my list to the top five trends for 2014, but a top-10 list would have to include advanced analytics, collaboration, social data, the talent shortage, and privacy. We saw a lot of activity in 2013 in advanced analytics with SAP acquiring KXEN in 2013 and broader support for R. Companies still seem to be struggling with basic data access though, and limited expertise and talent shortage is limiting broader adoption of advanced analytics. Vendors are trying to address this by embedding the smarts directly in the software. At the same time, I'm encouraged by the number of universities beefing up their BI and analytics curriculum.

I would think that with all the success of Facebook and most recently a Twitter IPO, collaboration in the BI world would now be showing strong momentum. It made my top trend list last year, and yet, that social influence has not yet gained traction in BI. It's continuing to emerge, but business users seem disinterested. I attribute the slow adoption to a combination of product immaturity and requisite change in workflow and culture. While collaborating around data is lackluster, mining social data has become part of the savvy marketer's toolkit. I've seen good solutions from BI vendors, particularly Information Builders and IBM, but also from Attivio that integrates with a number of visual data discovery vendors. Privacy though can through a big wrench in all this big data and social data. From Snowden and the NSA phone tapping to the Target data breach, individuals have gotten a big education on just how much data we are creating, how much it's being mined, and how easy it is to steal. The greater good for all this data needs to be weighed against the need for privacy. Expect some more breaches and backlash.

Next Week at TDWI: BI Platform Bake-Off

Next week at TDWI in San Diego, I’ll be facilitating the 2011 BI Bake Off with IBM Cognos, Microsoft, and SAP Business Objects. With all three vendors recently releasing major new versions of their BI platforms, the bake off will be the usual exciting, nerve wracking, and enlightening session.

I conducted the first BI bake off at TDWI back in 2005, at a time when few companies had BI standards, and the market was highly fragmented.

Today, the BI platform market is more mature. The market has evolved into mega vendors that have BI platforms, numerous smaller vendors that focus predominantly on BI platforms, and dozens of specialty vendors that focus on a single segment within the BI space.  With greater market segmentation, it might seem that BI buying has gotten easier, but it hasn’t. Smart companies make BI investments wisely, and yet business users often demand the newest innovations, regardless of who is touting them.

Most large companies now have a BI platform standard. Even though customers have made an initial investment, these companies continue to evaluate solutions, because:

• Vendors constantly release new versions, begging the question if it is better to upgrade or to make a switch.
• Customers are looking for opportunities to lower licensing costs.
• Some are not satisfied with support from the incumbent vendor.
• They were acquired or merged with another company, with a different BI standard.
• They’ve outgrown the capabilities of the product initially purchased.
• Budgets tightened in 2009 and 2010 have loosened. There’s pent up demand to expand BI capabilities.

For small and mid-sized companies, BI may be a new endeavor. They’ve outgrown their fixed, operational reporting solution and need something more robust. For many of these companies, BI platform breadth may be less important than ease of deployment and time to value.

When I conducted the first BI Bake Off, I kicked the session off with dry-ice emitting smoke, a bow to the smoke-and-mirrors in many vendor demonstrations. I promised attendees that we would try to cut through that smoke, with a candid look at side-by-side differences.Sadly, I have had to abandon the smoke as it violated the hotel’s fire code (killjoy, right?).

 In the first few bake-offs, I also thought it would be sufficient to politely stand when a vendor’s time was up (I told them in the agenda they would have only six minutes, surely they would stop talking, wouldn't they?). Well, I learned that I needed something more akin to a gong, or at least a timer, as vendors will take as much air time as possible. Six years later, and it still feels rude to beep someone mid sentence, but it certainly makes for a fairer session. The more seasoned vendor participants now know never to run over time.

In the bake off, I give vendors a carefully scripted demo of the most critical features and differentiators that customers often look for.  Attendees then vote on which product best met their requirements for that capability. Everyone knows that it is indeed just a demo and that a real-world test may yield different results. Still, what is clear is that the TDWI attendees are a sophisticated audience, equally tolerant and unforgiving. Attendees give high marks to vendors who candidly say “we don’t do that out of the box, but let me show a work around.” Conversely, one vendor who started with a marketing video was unilaterally panned, thank goodness with lack of votes, and not out-and-out booing.

Surprizing to me is when a vendor sells their product short. I’ve tested all of them, so I know their strengths and weaknesses.  As the goal is strictly education – but some fun in the process – I do point out when a demo just didn’t go well or when I disagree with something a vendor said. I have sometimes chalked up these incidents to a vendor’s not being prepared (another thing attendees are unforgiving about), but frankly, I also think it’s that all the product lines have gotten more complex. While the marketing folks may be able to articulate product differences well, it is often the technical consultant who knows the product nuances. Hence, many vendors now bring two participants to the bake off.

For companies large and small alike, an ongoing challenge in managing your BI tool portfolio is in knowing when to standardize and when to mix and match. Standardization has the lure of lower cost of ownership, better support for business users, and integration across modules. That’s the lure; reality varies vendor to vendor. And yet, there are dozens of emerging vendors bringing often-times superior functionality than what may be available from a BI platform vendor. This is true of dashboards, visual discovery, mobile, search, and SaaS offerings, to name a few.

To help companies better evaluate these specialty solutions, in the last year, we have added detailed scores on BI Scorecard for a number of specialty products and vendors . At TDWI, we introduced the Dashboard Bake Off in February 2011. This class is a deeper dive on an increasingly important module; BI platform vendors and specialty vendors compete side-by-side on dashboard capabilities.  Dashboards are also part of the BI platform bake off in August. We also give vendors a chance to showcase something cool and innovative that most differentiates them. It’s the one topic the vendor chooses; the rest of the scripted demos are topics I and the attendees choose.

With Cognos previewing their new visualization product in the labs and Microsoft whispering about project Crescent, I suspect visualization may be what these two vendors showcase during the “cool topic.” With SAP being one of the few vendors to natively supporting the BlackBerry Playbook, perhaps they will choose Mobile BI for their cool topic … but there has been a fair bit of noise about collaboration as well, so we’ll see.

While the BI Bake Off provides side-by-side product insights for three vendors, the other goal of the course is to help companies improve their own selection and bake off process.

To make the most of your software evaluation:

• Define your requirements and prioritize them, differentiating between near and long-term needs
• Understand the trade-offs between a solution from a single-vendor and pursuing a predominant standard
• Get educated about BI pricing and packaging early on.
• Ask vendors to follow scripted, not canned, demos
• Gather feedback from a broad constituent of BI stake holders
• Recognize that requirements change over time as users and software capabilities evolve. Never assume your BI tool strategy will stand still.

 Making smart software investments and managing your BI tool portfolio is important to BI success. But as I often say, ultimately the question of which tool is less important than how you use your BI tools.

Regards,

Cindi Howson

 

 

 

 

 

 

 

 

 

Managing the Mayhem of Mobile BI & QlikTech's New Approach

In-memory and dashboard vendor QlikTech did an about face on its mobile BI strategy last month: one of the first BI vendors to natively support iPad and Android devices, QlikTech is now taking a device- agnostic approach, relying instead on the browser.

QlikTech isn’t the only vendor avoiding native mobile apps, with Information Builders and LogiXML sticking to their more browser-oriented approach. There are, however, some notable differences, and indeed, QlikTech is one of the few to have changed its mobile strategy.

Then there are other BI vendors, such as MicroStrategy and specialty vendor RoamBI who have native apps for Apple and BlackBerry devices. SAP BusinessObjects takes a similar approach but with their dashboards relying on Flash (which Apple does not support), the question with SAP becomes not only which device and which approach, but also, what content. IBM Cognos uses an app approach for BlackBerry’s and a browser approach for Apple, for now. Actuate also uses an app approach. Tibco Spotfire offers a native iPad app as well.

Why the difference and what should a customer do?

The tantalizing vision for browser-based Mobile BI is that it works on all devices. That’s great for vendors who don’t want to develop device-specific apps when tablet and smartphone vendors are still battling for market leadership. As Howard Dresner, author of a study on Mobile BI said, “It’s in the interest of the vendors to write the code once.” It’s also a help to customers who lack smartphone and tablet standards. The other appealing aspect is that nobody wants to develop reports and dashboards for every device.

That’s the vision. Reality, though, is quite different.

What’s good for vendors is not always best for customers. Currently device-specific apps promise a richer user experience, with faster performance via device-based caching. In theory, an HTML5-based solution could take advantage of smartphone features like location awareness and device-based caching, but they don’t necessarily do this. QlikView 10 mobile, for example, does not. Instead, it has optimized its current AJAX viewer to be touch-aware. This certainly means that an iPad user gets the full QlikView experience of searching, filtering, changing the chart display, and so on. I could not, however, use the iPad gestures to enlarge a chart, and the chart quality was not as rich as BI products that use native apps.  Performance was at the mercy of my network.  In part, this has as much to do with QlikView’s architecture as with its browser-based approach.

Information Builders mobile, for example, leverages the vendor’s Active Report technology along with an optimized WebApp based on HTML5. Active Reports provides users with interactivity while caching data locally for offline use and better performance.

As for the whole theory of not having to redesign reports for particular devices, i think it's currently a false hope. Tablets and smartphones have such drastically different screen sizes that report authors have to take that into consideration. Ideally, mobile BI apps (or optimized webApps) will sense the device and re-render the same report for that device, but again, that direction is still a work in progress for most BI products.

The arguments for and against browser versus device-specific apps will continue to rage (see last year’s blog). Greater adoption and use of HTML5, which supports device-based caching and location awareness will add intensity to the debate.  It’s too soon to tell, though, how well mobile BI WebApps that use HTML5 will compare to native apps. Independent analyst Mike Ferguson who  will be speaking on Mobile BI at TDWI Munich, says, “HTML 5 is not as effective as a native app for exploiting the device cache,” due to the smaller size of the cache and how its managed.

With the technology in a state of flux, here are my recommendations for managing Mobile BI:

• Agree on which devices are more important in your company. I won’t even say “set a standard” because the tablet market in particular is changing too rapidly
• Know what approach your vendor uses today, for which devices, and what their strategy is for the future
• Evaluate functional differences between a general browser client, optimized WebApp client, and native app. In all cases, focus on the information consumer role, not the author.
• Be aware of development and redesign efforts for rendering content on specific devices
• Be prepared for change.
• Make Mobile BI a priority. Just because there is change and risks with Mobile BI, it’s not a reason to delay or ignore it.

Have you taken this year’s Successful BI Survey yet? Tell us how important Mobile BI is to your BI success here.

For QlikView customers, to take advantage of this new mobile approach, you need to upgrade to QlikView 10. QlikView has stated it will continue to support the current device-specific apps for those customers who have deployed them.  For more information on QlikView 10, purchase our in depth product review here.  

Sincerely,

Cindi Howson, BI Scorecard

The 5 Myths of Self-Service BI

It’s pretty rare for me to do a vendor-sponsored white paper. I simply am too swamped with product reviews, blogs, articles, courses, and customer consulting.

So when Endeca first approached me about writing about self-service BI, I politely declined, suspecting they really wanted a funded product review. I had looked at their new product at a high level and liked what I saw. Their new product Endeca Latitude combines faceted search (if you’ve ever shopped at ebags, for example, you’ve experienced Endeca) with a dashboard interface. Earlier this month, they participated in my TDWI Dashboard bake off  along with IBM Cognos and Tableau (look for a new vendor line up at the Chicago conference).

Endeca assured me I didn’t need to mention their product at all. The paper could be what I wanted, a candid discussion of what I saw in companies struggling with self-service BI.  So you have my five myths of self-service BI:

1: Business users will create their own queries.

2. BI is so easy to use, even casual users will embrace BI.

3. Self-Service BI is only for internal users.

4. “I will have access to all my data.”

5. Once we have self-service BI, the business won’t need IT (and IT will be jobless).

The vision for self-service BI is an enticing one: provide business users with direct access to all the data they need to make critical business decisions. In theory, self-service BI closes the gap between the decision-maker and the person with access to the data.

And yet so much seems to go wrong in efforts to fulfill this vision, generating frustration for both BI teams and business users. I think a lot of the failures have more to do with unrealistic and mismanaged expectations rather than an unachievable vision. Let’s take that first myth, for example. The goal of self-service BI should not be to get mainstream business users to create their own queries.

At this point, you might be thinking “what?! Why not? Isn’t that why we just invested millions of dollars in our business /ad hoc query tool?”

Sadly, you might have been sold on that promise by someone – either your vendor painting an unrealistic picture or your CIO/IT Manager declaring IT has to get out of the reporting writing business. But it’s not realistic. At first blush, business query tools should let power users and business analysts create their own queries and reports, not the majority of prospective BI users. The empowerment for these mainstream users is in interacting with what a business-savvy power user has created as a starting point, and then if necessary, tweaking that content. Even for these power users, when you first move to a self-service BI environment, you don’t want to start them with a blank screen, forcing them to choose from a cryptic list of 1000s of data elements (read my TDWI Top 10 Mistakes from 2004!!).

So what to do when expectations are different from reality? For each of the myths, I’ve presented a reality checklist. You can access the white paper here, and I thank Endeca for sponsoring it.

Have you taken the 2011 Successful BI Survey yet? Take the survey here to tell us your reasons for BI success or failure. You could win $100!

Regards,

Cindi Howson, BI Scorecard

 

 

Has BI Finally Become Pervasive?

After years, decades even, of BI being adopted by only about 25% of employees and mainly power users, I’m predicting that one of the findings of this year’s Successful BI Survey is that BI adoption has increased. At last! It may not be mainstream yet, but it’s well on its way and gaining momentum.

I first ran the Successful BI Survey in 2007 as research for my ground-breaking book Successful Business Intelligence: Secrets to Making BI a Killer App. It confirmed some of what we already knew: you have to have executive level sponsorship, for example. The survey also held some surprizes: CIO’s often sponsor BI projects but bring less impactful BI than any other executive. The difference is in how well the CIO understands the business and is a member of the business team versus being simply the data gatekeeper.

In the 2009 survey, we learned that agile development and the role of the hybrid BI leader who understands both business and IT were critical to success (you can access free highlights here). While the world was reeling from a recession, the silver lining from a BI perspective is that the importance of business intelligence increased. Companies realized that it was not enough to have clean data and good BI tools; efforts had to be aligned to the goals of the business. The last few years continues to be evidence that it is the survival of the smartest. In our 2009 survey, we added some perspectives on which BI tools were used and how.  There was little surprise that IBM Cognos and SAP BusinessObjects were the predominant BI standards, but that SAS and MicroStrategy had the biggest contribution to business performance was an interesting finding.

So what will the 2011 survey reveal? Share your successes and reasons for failure here. I am predicting that BI adoption will have finally increased. The reasons are multi-faceted. From a macro level, all the mega vendor acquisitions in 2007 and 2008 certainly raised awareness about the importance of BI. The recession forced people to think more analytically and operate more efficiently, supported by fact-based decision making. BI vendors have innovated to make BI more appealing and engaging to the masses. The simplicity of search makes BI accessible to novices. Mobile BI makes business intelligence more relevant for front-line workers. And those eye-catching dashboards have helped managers and executives rise above the noise and data deluge to focus on what matters most in the business.

That’s what I anticipate anyway. But maybe you’ll tell me that IT and the business hate each other more than ever or that lack of common business definitions means nobody trusts the data. Is impactful BI still more pipedream than reality? Have commercial tensions with your BI vendor sabotaged your best laid plans?

Look for a blog in a couple months and I’ll share the high level findings from this year’s survey.

Regards,

Cindi Howson, BI Scorecard 

 

 

The Ongoing Battle Between Excel and BI

I would have thought that by now business intelligence tools and Microsoft Excel would be happily coexisting.  In some cases they are, but  a larger number of Excel users, managers, and BI professionals simply seem battle weary.

Last week at TDWI in Washington, DC, I taught an updated course on "BI & Excel: Friends or Foes?” I last taught this course four years ago. I learned my first lesson about BI and spreadsheets the hard way back in the early 1990s. At the time I was the project manager for a reporting system based on a custom transaction system. Typical of many IT projects, I gathered business-user requirements, went away for a couple months (at least it wasn’t a year!), and we developed some parameterized reports on the mainframe. The final solution was flexible, interactive, and exactly what the business users asked for. We launched the new reporting app in a training class I had personally developed and was thrilled to be teaching.

We were only half an hour into the class, when the power user in the group, Frank, declared, “I don’t want any of this. I just want all my data in Excel.” Frank was the statistician in the group -- the trusted analyst. Normally, he and I were on the same side. We would swap notes on how to tweak Lotus macros or how to extract data from mainframe sources. But he basically just told me I had wasted months of effort and that what my team had built was crap.

The business managers in the room liked the reports, but they relied on Frank for all things data. Frank had planted a seed of doubt in their minds that our whole approach was wrong. Frank had become my foe. I tried not to get teary-eyed (hey, I was in my 20s!) and decided to cancel the rest of the class.

Frank felt fixed reports constrained him. He needed flexibility. For the managers in the group, who were not yet proficient in spreadsheets, the mainframe-based reporting system might have been fine. But Frank could make any data looking prettier in Excel, with better formatting, colors, charts, and so on.

Even though that lesson about the role of spreadsheets in BI was learned 20 years ago, BI leaders, managers, and Excel gurus continue to grapple with BI and Excel’s coexistence.

At the start of last week's TDWI class, about a third of the attendees agreed with the suggestion that Excel and BI are friends; Excel helps improve decision-making and fulfills the vision for business intelligence.  “I'm on the friend side because  I know if I treat Excel as a foe, I will lose," said one attendee.  "It's easier to embrace your enemy and gain trust and creditability.”

The remaining two thirds felt that BI and Excel are foes: too many spreadmarts (some in the hundreds of megabytes) and multiple versions of the truth undermine the BI team’s efforts to provide a single version of the truth.  One attendee who started the day thinking spreadsheets were the enemy later said  the class had changed her view.

To fully appreciate Excel’s role in BI, start by understanding why users love their spreadsheets. Sometimes it may be personal and job security. Knowledge is power, and being the designated source of data is an enticing role.  Beyond job security, there are a number of valid reasons that Excel plays a powerful role in business intelligence, such as:

• Familiar user interface: Attendees acknowledged that Excel is the preferred interface for power users, but managers and front line workers prefer dashboards and easier- to- use tools (they were wowed by SAS’s Email integration).
• Ability to tweak a report, whether to sort, filter, pivot, or remove a column
• Extensive formula library
• Access to multiple data sources: Excel’s ability to combine data from multiple data sources is a must-have requirement for many types of analyses
• Ability to “massage” the data: few attendees said they use Excel for data cleansing, but several people spoke of changing group roll ups.
• Ideal prototyping environment where users themselves can build applications.

For any given requirement, assess whether Excel is the best solution or if the BI tool would be better. For example, newer version of BI software allow users to filter, sort, and interact with a report via a browser. Habits are sometimes hard to change, and if users don’t know about these capabilities, they will fall back to what they have done for years (click that Export button). 

The multiple-data-source issue is probably the biggest challenge. BI teams don’t like personal or departmental data to be accessed from a central business view and certainly not from within the data warehouse.  Excel is ideally suited for joining data from multiple data sources for a one-off analysis. The scalability improvements with the PowerPivot feature in Excel 2010 make it even more suitable. However, for recurring analyses that require joining data from disparate systems, multi-source becomes a problem for the entire  organization. The BI team has to recognize this or they risk being less relevant and  the cause of spreadsheet chaos. Don’t back business users into a corner.

Most BI platforms now offer an Excel add-in that supports data integrity while giving users their data in the familiar Excel interface. It surprized me, though, how few attendees were either aware of or using these add-ins.

Why aren't add-ins being used? In some cases, it’s performance and licensing problems. In other cases it seemed to be lack of awareness. There also seemed to be a degree of fear: BI teams shy away from Excel because they’ve been burned in the past.

My recommendation is to recognize Excel as part of the BI environment. Manage the spreadsheets, whether via SharePoint, a content management system, or the BI portal. Consider carefully when it’s appropriate to use Excel versus the BI tool; and in all cases, make available add-ins part of your BI tool portfolio.

By the end of the morning, attendees were more positive, with two thirds agreeing that Excel and BI can be friends when managed. The remaining third are still wary. Where do you stand? Have you successfully deployed these Add-Ins? I'd like to hear from you.

And if you followed my last blog about a few Washington, DC, mishaps, for this trip, the cherry blossoms were in full bloom. The view of the Jefferson Memorial, in 80 degree sunshine, was nothing short of spectacular ... and I didn't get kicked out of a single cab!

Regards,

Cindi Howson

 

 

 

 

 

BI Licensing & Lawsuits: A Sure Sign of Failure

Software licensing is never an easy topic, for either the buyer or the seller. For buyers, it's so much easier to discuss features and capabilities and whether or not the product is a good fit. For the seller, it's so much easier to highlight why their company and product is the best.

So you do your proof-of-concept analyses, confirm requirements, and capabilities, and at the eleventh hour, bring in your purchasing department to negotiate the best deal possible. Relationship doesn't matter here. Discounts do.

Buying BI software is particularly complex. Rarely can you buy the "BI suite" or "everything demonstrated"; instead, it's a plethora of unclear choices between roles and products, server-based or named-user licensing, optional modules that the unsuspecting buyer would have thought was standard, maintenance based on list versus discounted prices, and so on.

There is a big disconnect in BI buying: buyers want as much as they can get at the lowest price. They certainly don't want to be forced into the embarrassing (career-ending?) position of having to go back to the executive committee to garner more money for a BI module they overlooked. Vendors, on the other hand, want to extract as much value from the customer as their product warrants. Both sides want what's fair.

So if a customer upgrades hardware and does not increase the number of users, should the customer have to pay a fee to the BI vendor? That doesn't seem fair to me, but that is a consequence of server-based licensing that considers CPU-clock speed and power ratings. What about a report consumer who normally refreshes a report, but now wants to drill down into the details? The capability to drill may involve a higher-level, role-based license. I don't know many organizations that can track to that fine a level of detail exactly what users want to do, or that can accurately anticipate how user requirements and capabilities will evolve overtime.

I cringe at the stories I hear. One manufacturing company has a full-time equivalent tracking BI licensing compliance. This doesn't seem to be value enhancing. One health care company is in a legal dispute with its BI vendor because it didn't realize virtualization wasn't explicitly allowed. The firm also didn't immediately disable logins for employees as they left the company, so it  temporarily exceeded a named-user license count. One influential BI consultant will never again recommend a particular product to a customer because of a miscount in the way the software tracked its license usage for one of his customers. A legal dispute followed. Yet another customer is abandoning a project because of the sticker shock it faced when it explored rolling out a BI application  to more users.

I know there are some disreputable customers out there who use grey market software. But most of the situations I've encountered are honest mistakes and the fault of complicated licensing models that are out of sync with dynamic infrastructures and workforces. Once a customer and vendor entera legal dispute, any notion of a partnership in BI success is destroyed.

My recommendation to customers remains: buyer beware (see this article or BI Scorecard subscribers should view our webinar on BI packaging). Read and understand the fine print early in your evaluation process. Involve procurement early in the buying cycle. For vendors: simplify, simplify, simplify. Nickle and diming customers for a short-term profit is a recipe for longer-term failure. Beyond simplifying licensing policies, make it easier for customers to track licenses and compliance. There should be no surprizes for either side.

I could speculate on why this problem seems to be growing: a difficult economy, mega vendors who have greater account control, vendors who know BI switching costs are high. But you tell me: is this problem declining or increasing?

How clear are you on your vendor's licensing policies? Post a comment here, or if you are worried about a back lash, e-mail me confidentially at [email protected].

Regards,

Cindi Howson, BI Scorecard

Big Themes in BI for 2011

There is nothing like a fresh snowfall and a new year to reflect on the hot topics in BI for 2011. Here is my list of the five most important trends for the year ahead.

1.     Advanced visualization and dashboards become mainstream

Last year, QlikTech went public, Tableau Software revamped their product, and IBM Cognos released their second generation dashboard solution, dubbed “Business Insight.” Advanced visualization and discovery tools continues to garner significant interest because of their ease of use, visual appeal, and ability to speed the time to insight amid vast amounts of data. The challenge for companies is in understanding when technologies overlap and the degree that they are converging. Should you look for these capabilities from your BI platform vendor or from a specialty vendor?

For dashboards, the answer depends on who your BI platform vendor is. Vendors such as Oracle and MicroStrategy have the most robust, appealing, and integrated dashboard solutions (feel free to disagree with me here, but as always, my conclusions are from hands-on testing of hundreds of criteria). This is not to say that other vendors haven’t made strides in their latest releases, but these two vendors are furthest along.

If, however, you are looking for an advanced visualization product with dashboard capabilities, the better products remain with specialty vendors. A few of the leading BI platform vendors have some exciting products in development in this area.

Beyond these tools, businesses will continue to struggle to decide who owns them and who should develop related content: business or IT? First dashboard attempts will look more like glitzy pin ball machines, but as people gain experience and expertise, these types of tools become the most useful component of BI.

2.     Mobile BI gets recharged

Mobile BI didn’t even make my top trends last year, so one could say its appearance this year is quite a leap. Also in my cool BI class I teach at TDWI, this innovation rarely gets top nod. However, few could have foreseen the wild adoption of Apple’s iPad. The wider screen real estate and sheer beauty this tablet brings to BI is reason to take stock of your mobile BI strategy. What devices will you support, and what is your vendor’s strategy? Vendor capabilities here remind me of my living room New Year’s Eve morning: unclear, fragmented, and all over the place. Support for devices, content, consumer ability, and native versus browser support vary widely. MicroStrategy’s mobile BI solution is the most broad, robust, and includes support for iPads as well as Blackberrys. QlikTech and Actuate also are further ahead than some of the larger platform vendors. Niche vendor RoamBI who supports content from multiple BI vendors is worth investigating.  For companies who fail to consider mobile BI’s potential and the limitations, their mobile BI strategy will be forced into reactionary mode, risking security breaches, mixed devices, and inconsistent support.

3.     Facebook gives BI more than a facelift

Facebook hit 500 million users mid 2010, a community now larger than the population of the U.S. and Canada combined. It’s changing the way a generation interacts with each other, lobbies, advertises, complains, and thinks. It’s impossible for it not to impact BI. Social networking brings new data sources for analysis. It also brings new demands for how users respond to insights from data, and that is never in isolation.

To date, a report may be viewed as part of a task or decision-making process. Any discussion around the report is typically off-line. Key decisions from the insight are made verbally or shared via email or in documents, sometimes held in a document management system.

Envision a Facebook influence on BI: decision-makers bring together the right people virtually; users themselves control the flow and content instead of a central IT group who secures the data and subsequent analyses. Tasks, comments, opinions, and even new sets of data and analyses are brought together seamlessly. Sounds too futuristic? Initially some vendors offered annotations to BI content, such as Information Builders Performance Management Framework and Dundas Dashboards. Moving far beyond just a portal technology, consider the new collaboration capabilities in Microsoft SharePoint 2010 integrated with their BI products. Beyond SharePoint, Outlook 2007 has collaboration features that SAS has been quick to leverage. Demonstrating further social networking’s influence, last year SAP launched StreamWork (but without BI content), Oracle BI EE 11g included support for Web Center (someone should rename this product, seriously), and IBM Cognos 10 shipped with Lotus Connection. While the BI heavy weights are making inroads here, niche vendor Lyzasoft seems to have gotten the “user-directed” aspect most right.

So we’ve seen some innovation here, but the convergence with BI is still very young.

4.     The economy rebounds … stretching BI teams further

The economy shows signs of recovering. Companies who weren’t using BI to work smarter are no longer with us. BI budgets are once again expanding. The challenge is to continue to spend wisely, but also, to keep up with insatiable user demand. Central BI can’t handle it all. Bringing BI to the masses also means balancing what to handle centrally and enterprise-side, and when to let users do their own thing. It’s not always a matter of departmental BI being a throw-away or stand-alone application. Sometimes, it’s a matter of intelligently separating responsibilities. SAP BusinessObjects 4.0 has an interesting approach to its meta data development that allows for this. eThority also seems to nicely blend enterprise class with departmental control.

If a shrinking economy helped spur open source adoption, what will a recovering economy do? I would argue that open source’s growth is not primarily because it’s free; it’s because it’s open, allowing for easier embedability. Such aspects are more important to ISVs and others who build industry-focused applications which in itself is a sizable market. For open source to compete more squarely with closed-source commercial BI, their products must continue to improve; some are strong in one or two core areas, but not across the BI spectrum. Beyond the U.S. market, open source is also attractive for governments and foreign companies who don’t want such critical assets to be powered entirely by U.S. software makers.

5. Will new releases be upgrade fever or flu?

Major new software releases mean powerful new capabilities, as long as the upgrade process is seamless. Painful migrations can leave IT resources strained with testing, bug resolution, and redesign. The top four BI vendors have all had major new product releases in 2010. Customers who have been burned with painful migrations in the past will not rush to adopt the latest version. In some cases, it will be a time to assess the cost of upgrading with the cost of switching preferred vendors.

It’s been interesting to see how both IBM Cognos and SAP BusinessObjects have a “co-existence” strategy in their latest releases: leave old content as is, running on the new platform. That approach is fine as a transition strategy but the proof point will be in how easy and quick a full transition is.

In-memory and more

So what about all the other things I wrote about last year: in-memory, cloud, BI for SMBs, culture, and predictive analytics? All these areas continue to be critical, and I continue to be both impressed and overwhelmed by the degree of innovation and success from smaller vendors in the BI space.  My five picks above are those things that I think will garner the most attention, generate the most activity, and cause the most disruption in 2011. If I added a sixth item, it would be continued growth of in-memory analytics. In-memory proved to be one of the biggest BI technology themes in 2010 (think Microsoft’s PowerPivot and SAP’s HANA) and will be a capability that many companies will implement this year, whether as part of visual discovery and dashboards (item 1), insatiable user demand (item 4), or instantaneous response time that mobile requires(item 2). It seems, though, that while in-memory is a cornerstone to some vendors’ BI strategy, others are more reticent. Will they continue to have a wait- and-see approach, or will they advocate greater focus on other technologies for fast, big data BI such as appliances, indexing, various caching, and columnar databases?

As I look at last year’s post at this time, my BI predictions seemed fairly accurate. If only, now, it carried over to football! Anyone want to place bets on the Packers this weekend? They are one game further than last year, and that’s got to be a good omen, right?

Happy New Year!

Cindi Howson, BI Scorecard

 

 

Don't Let BI Standardization Be an Excuse for Stagnation

BI’s gotten bigger, with everyone from executives to casual users having an opinion on the company’s BI strategy.  Is that a good thing?

I find myself having a number of conversations lately around how the new BI landscape shapes BI tool investments, both at the enterprise and department level. Maybe your company uses SAP for its ERP system, but Cognos is your BI standard. Should you switch your BI standard to SAP? Microsoft just released PowerPivot which has all the familiarity of Excel, so perhaps that should be the company ‘s new BI standard. Or maybe you are implementing Oracle’s PeopleSoft and wondering if the prebuilt analytic applications that require OBI EE would meet your BI needs. Meanwhile, individual departments are clamoring for the central IT group to support easier to use and faster to deploy tools such as QlikTech QlikView and Tableau.

Vendor consolidation was supposed to make everyone’s life easier, with fewer players to choose from and more clear market leaders, but that hasn’t been the case. Stakes are higher with millions of dollars in BI investments and even more in missed decision opportunites; switching costs are high, and IT careers for investing in the wrong vendor or skill set are on the line.  Why all the debates and what should you do?

All 4 mega vendors have released major new version of their product, forcing assessments of whether to upgrade or whether to switch. What is most beneficial and most cost effective really depends on the particular vendor and product. Some upgrades are more painful than others. The degree that  you should re-assess also depends on how successful your current deployment is. I never recommend changing vendors for change’s sake or for minor feature improvements. Vendors will continue to leap frog each other in particular areas but their vision, execution, and philosophies are profoundly different. The most successful deployments keep a close pulse on their BI tool standards and reassess their portfolio every couple of years.

According to BI Scorecard’s 2009 Successful BI Survey, 57% of organizations now have a predominant BI standard. The key word here is predominant. I have never bought into an idea of an exclusive BI standard, yet it seems CIOs in particular want this simplistic, one-size-fits-all approach. If you consider the 6 or 7 modules that make up a BI suite, no single vendor is excellent in all modules. While smaller companies with less complex analytic requirements may be able to manage with a single vendor solution, the needs of larger enterprises are different. Beyond differences in functionality among vendors, there are also trade-offs in deployment costs as well. For example, open source vendors  and smaller vendors such as Information Builders and LogiXML continue to be wise tools to assess for extranet deployments with thousands of report consumers.  If mobility is key to your BI strategy, then MicroStrategy has been the most aggressive in building out those capabilities. Dashboards continue to be a battle ground for innovation and significant differences in capabilities.

Meanwhile, lines of business and departments care little about the strategic view of BI standards. They have business problems that need solving faster than central IT can address. They continue to grapple with the ongoing challenge of trying to make sense of their data, in some cases, in simply accessing their data without IT as the gate keeper. Enter visual discovery and in-memory tools such as QlikView QlikTech, Tableau Software, or TIBCO Spotfire. Should they replace the BI standard or supplement them? Or should IT squash departmental efforts to dabble with what may be nothing more than the latest fad. What about BI search tools like Endeca, or specialty mobile vendors like RoamBI and PushBI? Many of these smaller vendors have also had major new releases, becoming more robust, flexible, and enterprise-grade. All these changes give rise to departments lobbying IT to support them.

In trying to answer this last question, I find myself drawing analogies to cooking and cutlery. A Swiss Army knife, like a BI platform standard, may be less expensive than a full Henkel’s set and has a smaller footprint. But when I’m chopping celery for stuffing, I want the dedicated French chopping knife. If I’m frosting a cake, I want a non serrated spreader.  The problem with BI and analytics is that we don’t quite understand the nuances of the different ways that BI tools work and the distinct ways that they empower users. “Seeing is believing” or “making sense of your data” may be good marketing lines, but they fall flat in a board room when you are asking for funding and even flatter when you are asking for official IT support. The bottom line is that larger companies need an arsenal of BI capabilities. BI standardization is good when you are trying to rationalize overlapping, redundant capabilities. It’s a bad strategy when companies assume a one-size fits all approach and fail to consider how technology and user requirements change. In the extreme, colored pencil and paper were once standard “BI tools.”

As the pace and competitiveness of business has accelerated, users need new capabilities that yield faster insights and time to value. That means periodically reassessing and expanding your BI tool portfolio.

Let me know what's happening at your company: Is your CIO advocating a one-size fits all approach or a selective mix and match?

Sincerely,

Cindi Howson, BI Scorecard