BI Scorecard Blog

TIBCO Spotfire Announces In-Database Visual Data Discovery

Last week at TIBCO’s annual user conference in Las Vegas, TIBCO announced Spotfire 5 with in-database processing and a new predictive analytics engine for open source R models.

TIBCO Spotfire 5 is expected to become generally available before year end, bringing in-database processing, a revamped in-memory engine, and a new predictive analytics run-time engine for R. Like many visual data discovery tools, TIBCO Spotfire has long had its own in-memory engine to provide speed-of-thought exploration and analysis. In contrast to chief competitors QlikView and Tableau, Spotfire has had an on-demand option to interactively load subsets of data into the engine as users get to a certain exploration point. With version 5, more of the processing can be pushed to the database, currently for Teradata, Oracle 11g and Exadata, and Microsoft SQL Server and Analysis Services, preserving the investments that customers have made in those platforms to bring better data scalability. I was impressed by how easily a Teradata star schema was used to build an initial Spotfire application, requiring less modeling time than previous releases. In addition to the in-database processing, TIBCO Spotfire has revamped into established in-memory engine, claiming 5 to 10x faster over version 4, maintaining sub second response time at over 100 million rows of data. (TIBCO did not provide test configuration information).

A differentiator for Spotfire has been its integration of predictive models based on the S+ engine. TIBCO Spotfire 5 introduces a run-time engine for R so that existing R models can be run and integrated with the Spotfire applications. The models can either be run from the desktop Spotfire Professional client or from the server.

While much of the BI industry is banging the BI and collaboration drum, TIBCO seemed to treat these technologies as distinct, even though they have some powerful capabilities. First released two years ago, Tibbr is like a Facebook for the enterprise, competing with the likes of Yammer (acquired by Microsoft June 2012) and SalesForce Chatter. Spotfire integrated with Tibbr’s collaboration capabilities last year in Spotfire 4. The next release of Tibbr brings additional capabilities such as an employee’s social profile, influence, and recent activity. The InterPort Police, responsible for controlling international ports, spoke of how Tibbr made it easier to share information with other agencies, where real-time data and mobility are critical to national security.

After TIBCO acquired Spotfire in 2007, Spotfire’s relevance in the visual data discovery market seemed to wane. They appeared on fewer customer short lists and sales did not keep pace with the rest of the BI market. According to IDC estimates, 2011 revenues once again showed solid growth. With TIBCO giving greater attention to analytics and Spotfire 5’s differentiators of in- database and predictive, the product is well positioned to take on both the pure play competitors and the major BI platform vendors.

Regards, Cindi Howson, BI Scorecard

SAP Commits To Cloud, Beefs Up Mobile And In-Memory Options

At SAP’s seventh annual influencer summit in Boston last week, Co-CEO Jim Hagemann-Snabe highlighted four areas of SAP innovation: core, mobile, cloud, and in-memory. Interesting to me is that SAP now considers the core to include business intelligence (or to use SAP’s umbrella term, business analytics) as well as its well-established business applications.

Mobile, cloud, and in-memory also cut across both BI and SAP’s business applications.  In mobile, SAP currently has a number of first-mover and competitive advantages. Sybase Unwired gives them a development platform as well as device management. SAP now has 30+ mobile applications. However, in the BI space, its solutions are mixed (check out BI Scorecard's  latest summary scorecard here to see how they stack up), depending on the device and the BI content. For example, mobile interfaces vary depending on whether they're from the SAP BusinessObjects 4.0 WebIntelligence, Crystal, Explorer, Dashboards.  Look for this to improve in 2012 (with more detail in this article about what's ahead in SAP BusinessObjects Feature Pack 3).

For cloud, SAP BusinessObjects was one of the first BI platform vendors to have an on-demand strategy. Day two of this summit was largely focused on cloud computing, with SAP emphasizing existing and new cloud applications, including  Sales OnDemand, Sourcing OnDemand , Travel OnDemand and so on.  The  pending acquisition of SuccessFactors will position SAP as the numbertwo SaaS vendor in revenue after SalesForce.com. SuccessFactors provides employee performance management capabilities in the cloud, with 15 million users and a 70% growth rate.

All of this shows SAP is committed to the cloud, but here, BI was like a forgotten step-child. One exec said, “except for Business ByDesign, we didn’t have cloud solutions to sell until this year.” Maybe he misspoke, but it reinforced my perception that BI OnDemand has been more of a check-box than a fully supported BI cloud strategy. In yet another show of stepped-up commitment to cloud, Google announced it will integrate Google Apps with SAP Business ByDesign, the company’s cloud-based business applications suite.

Meanwhile, there is little doubt about SAP’s commitment to in-memory with its Hana technology expected to power not just BI but also business applications. SAP’s vision for in-memory and Hana is to redefine real time. Snabe declared in-memory as the next-generation architecture. He took a subtle dig at Oracle’s (not explicitly naming the vendor) use of "hybrid" technology (relational plus in-memory) in Exalytics, saying hybrids don’t last. 

The benefits of in-memory in terms of real-time, analytic complexity, and performance were backed up by a number of customer testimonials. However, we have to remember that the number of live deployments are probably in the low hundreds. I suspect this is both because of the newness of the product (GA’d in June this year), but also the steep cost for customer to acquire. SAP claims lower long-term ownership costs compared to traditional BW and relational deployments but upfront expenses can be an obstacle. SAP needs to figure out how to bring in-memory to customers with lower budgets. This may be where Hana in the cloud has a role to play or competitors such as QlikTech will continue to expand. Hana will also be the database powering BI On Demand.

One of the most interesting announcements in this area was planned support by third-party vendors including  Tableau and Tibco Spotfire. Tableau has long had a philosophy and architecture of not requiring customers to replicate data into their in-memory engine (an option added in 2010 through Tableau Version 6). So support for SAP’s HANA makes sense. It’s a notable contrast to Oracle, which discontinued its OEM relationship with Tableau earlier this year.  You can preview a demo of Tableau running on  Hana here. Spotfire, in contrast, has typically had an extract-and-load-into-memory approach (with  a drill on demand option), so I will be curious to see how this is implemented. Neither solution is, as of yet, generally available.

Customer FreshDirect provided an interesting view of their journey to self-service BI. They are using SAP's newly released Event Insight to track on-time delivery during the day. Unexpected events like Obama coming to town can certainly disrupt their deliveries throughout New York City. Event Insight alerts them to issues and enables them to quickly add reserve trucks to routes when delays pop up.

FreshDirect routinely monitors BI usage and if someone is not frequently using their license, it gets taken away and allocated to another user. Users author their own reports on everything from delivery performance to number of parking tickets. “If you don’t have access to analytics, you’re going to get squashed by your competition, ” says Brandon Arbiter, FreshDirect's manager of business intelligence.

It was a big picture kind of summit, so I had to reflect on where SAP has been in BI and where it’s going. Executive VP of Business Analytics Sanjay Poonen committed to being best of breed in BI. I couldn’t help recall SAP’s stance prior to the BusinessObjects acquisition, striving only to be "good enough." It’s better for customers and the industry that they are now pursuing excellence.

Sincerely,

Cindi Howson, BI Scorecard

 

Advanced Visualization Moves to Top of Mind at TDWI

I’m just back from sunny San Diego from the TDWI conference. That everyone stays in the classes all day, rather than exploring the tempting water front, always impresses (and surprizes) me!

I briefly visited the Executive Summit to hear about what trends CIOs and VPs of Business Intelligence think will have the biggest impact over the next three years. Predictive analytics once again topped the list, but now in the second highest spot was advanced visualization and discovery.  This is good news for vendors/products such as Tableau, Spotfire, SAS JMP, and Advizor Solutions. This week we published an in-depth evaluation of TIBCO Spotfire that highlights the product’s strengths, challenges, and competitive differentiators.

Surprizing to me, though, was no mention of in-memory, but I am chalking this up to an oversight in the list of voting items rather than any real indication of its importance.  Mobile BI ranked near the bottom again, but as one retailer commented, that result will look very different in a year from now given how fast iPads are selling. Juicing up the competitiveness in this market too is new support for Flash on Droid smartphones.

The BI Bake Off was once again exciting and nerve wracking (for me as the facilitator!), with Information Builders, MicroStrategy, and Oracle demoing side-by-side. It was the first look for many Oracle prospects to see OBI EE 11g which officially went production last week. I know-- everyone wants to know who won the bake off and where the vendors were on par with one another.  But I have to save that result for attendees only!

 Regards,

Cindi Howson, BI Scorecard

Mainstream BI or Mainstream Predictive Analytics?

We talk about “mainstream BI” that continues to be more vision than reality, but how about mainstream analytics?  This week visualization and analytics vendor TIBCO Spotfire is one step closer to making that a reality with its 3.1 release.

Statistical analysis and predictive analytics requires expertise in both the data and statistical methods. Consuming the results of such analyses, however, does not. So imagine an expert who creates a model that optimizes which customer is most likely to respond to a marketing campaign. The process of determining which factors (such as age, gender, income, and customer tenure) most predict likelihood to respond is something the expert normally does. So the expert runs the model, refines the model for maximum response rates, and ultimately gives a list to the marketing department of which customers are their best targets.

In Spotfire 3.1, there are two new aspects that make predictive analytics more accessible to the final decision-makers (in this example, the marketing department). First, are parameters that allow a business user to select the variables that go into the model. So maybe, age really doesn’t provide the best likelihood to respond. Maybe, instead, it’s education level. Via a simple parameter drop down, a business user can choose which factors they’d like to use as input to the model.

Second, the business user can see the results of the model via an intuitive dashboard from within Spotifire. The R engine is still processing the model behind the scenes, but the results don’t stay off to the side as is often the case today.

A few vendors have tried at least to facilitate the consumption of the results of models, with varying ease and effectiveness. SAS in the way it can surface stored processes in Web Report Studio would be one example. MicroStrategy’s optional Data Mining Services would be another.  And yet, neither is particularly widely adopted, partly for complexity and partly for licensing reasons. A few weeks ago, SAS previewed a new product that would be a big enhancement to the concept of “mainstream predictive analytics,” but details are under NDA.

So while “mainstream BI” may still be more of a vision, this release of Spotfire brings “mainstream predictive analytics” one step closer to reality.

Sincerely,

Cindi Howson, BI Scorecard

Can BI Help Battle Swine Flu?

If you have school-bound children as I do, no doubt you are reading the headlines of swine flu wondering how you, your children,  your company will battle this year’s flu season now that school is back in session.  It’s the highly contagious, seemingly unavoidable aspect of swine flu that worries me. When you own your own company, there is no such thing as “paid sick days.”

Business intelligence is helping monitor swine flu in a number of ways. Emergency Medical Associates (EMA) operates hospital emergency rooms in northern New Jersey and New York metro area. They have a relatively unique ability to collect patient data across multiple hospitals, on a daily basis, and then analyze it with SAP BusinessObjects (see graph).  (The red line refers to 3 times standard deviation, yellow is 2 times.) I have previously written about EMA in my book Successful Business Intelligence, because of the unique ways they use BI to to improve patient care and manage costs. You can catch a glimpse of how they are now using BI to identify swine flu outbreaks in this video.

At the CDC, hospitals and other centers report data on a weekly basis so there is a lag in knowing when an outbreak is occurring. That New Jersey, my home, has one of the highest pediatric death rates is alarming. But that’s all I know, and yet, I have no doubt the data could tell us so much more.

Knowing an outbreak is occurring is helpful. However, identifying the patterns in who contracted it, where, when, and the severity, would be even more helpful. This is where advanced visualization tools fit in. While much of BI lets us answer questions that we know to ask (who, where, how many), advanced visualization tools allow for more exploratory type questions such as:

·         What characteristics do these people have in common?

·         Are they football players?

·         More boys than girls showing symptoms?

·         Does Mrs. Smith’s class have more unusual absences than Mrs. Kane’s?

I wish our town, our school, or even parents like myself could ask and answer these questions. In many cases, the data is there, but without the tools in place to explore the data, it’s not actionable.

For an evaluation framework of advanced visualization tools, you can download a new BI Scorecard template here (visitor login required).  Also, I highly recommend visualization expert Stephen Few’s latest book and this article by Seth Grimes.

So while much of the country awaits the release of the new flu vaccine, I’d like to see our schools and hospitals add BI to its arsenal of tools for battling an outbreak.

Regards,

Cindi Howson,

Founder, BI Scorecard

Fun BI Holiday Shopping App

It’s true – I love these viral marketing apps from BI vendors that are not only fun to test drive, but also, practical! So to celebrate the holiday season, visualization and analytic vendor TIBCO Spotfire has created this fun holiday gift finder (warning, unless you want the continuous jingle tune to draw the ire of co-workers, set your volume to low).

According to Spotfire, the data in the gift finder is based on data from Amazon and includes prices, popularity rankings, and ratings for a subset of product categories. As I wrote in the recent Intelligent Enterprise feature article, users can see the value of in-memory analytics in action here as you interact with data visually.

On the “Shop” tab, you can view a bar chart of products grouped by price and readily filter them by product rating and category. What I found interesting was the “Analyze tab” in which you can see what is most popular by price. So while there are some stereotypes about ladies being big spenders, I thought it was revealing that the most popular high-end gifts are the typical “male toys” – TVs! (Tip: lasso around the dots to see a list of products in the bottom pane that then link back to Amazon.).  And when I filter by rating, it’s not the Sony I got my husband a few years ago, it’s the Samsungs!

But as we are all tightening the budget this year, I was curious about the most popular jewelry items with a high rating but in the lower price end.  I stumbled across these nice earrings for $155 to discover they are now on sale for $59. Maybe this gift finder is more than a fun demo after all.

Merry Christmas!

Cindi

Founder, BIScorecard, a resource for in-depth BI product reviews

Author, Successful Business Intelligence: Secrets to Making BI a Killer App