« SAP Releases Visual Intelligence, Innovates in BI | Main | Information Builders Hits Its Stride, Launches WebFOCUS 8 »

June 13, 2012

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83517f23953ef0176156bbe4f970c

Listed below are links to weblogs that reference QlikTech Acquires Expressor, Aims to Improve Data Load Process:

Comments

Jmichel_franco

I was starving for your food for thought, Cindy. Great taste !

It seem that self service BI is reaching a tipping point. It this still good enough anymore to let power user access predefined data marts? If it is not, then QlikTech may have touched a sweet spot with this acquisition (some other are going in this directions, e.g. SAP with SAP Hana Information Composer ). In such scenarios, some power users want to build new models out of schema-neutral/schemaless sources, reuse sanctioned data in new context, compose data mash-ups... As you mention, this is a different discipline that EIM, which is still needed for the upstream heavy lifting of data and to deliver and secure access to enterprise "sanctionned" data, now for a wider scope that only traditional BI by the way (eg MDM).
Another great point you touch is the challenge of knowing when to leave data on disk or in a data warehouse versus loading into in-memory. This is another hot spot in Information management nowadays. Should we stream or rather sync data ? Should we opt for physical or logical Data marts and data warehouse ? Should we keep all data in memory or only "hot" one ? And how to make all those choice transparent to the one who consumes them. This is indeed a challenge for QlikTech, but also for our industry as a whole.

Wayne Eckerson

After writing my own analysis of the acquisition and then reading your interpretation, I came up with a big juicy question:

"Is it better for a BI tool to have a BI semantic layer or a data integration tool?" And as follow-ons: "Do you need both and what's really the difference between the two?"

Technically, a BI semantic layer normalizes and joins data using dynamically generated SQL. A data integration tool moves data from point A to B and transforms it along the way. Both tools transform data but ETL tools have built-in transformations and can deal with much greater volumes since the processing is done in batch.

Data integration tools ultimately are for creating data warehouses and data marts, while BI semantic layers are for creating role-based views of those data sets. So in this sense you need both. But expressor really gives QlikView both worlds. However, it's semantic layer is only accessible to IT administrators not business users (i believe.)

So does QV need an end-user semantic layer? I don't think so.

Cindi Howson

Thanks for your feedback Jean Michel! You raise many good points as always - users do need to get to their own data sources, sometimes. I don't think we are at that tipping point though across the board. The flexibility within pre defined subsets of in-memory data is really powerful in QlikView. However, indeed there are always situations when users need more data and then the issue is how they either bring it into the app or if they export and analyze in Excel. User-defined merged data sets has long been a strength of BusinessObjects, key to Power Pivot, and strong in Tableau 6.
So the needs are there, it's just how soon and which classes of users get to.
Also agree that in-memory vs physical disk is an industry wide debate and too early in large scale to say what is right. But for sure, I want clean, de-dupped data in the data warehouse. Where I analyze that data is another story.
Regards,
Cindi

Cindi Howson

Hi, Wayne, thanks for the comments. I think you are right - you need both, because they serve different purposes. The semantic doesn't normalize the data - it serves two main purposes. One is to hide the complexities of the physical schema from an end user. The second is to provide re-usability and consistency for calculations, transformations, filter logic and the like. Related to this of course is tooltips and buisness definitions but for the most part BI tools have been weak to expose such things to information consumers. So does QlikView need a semantic layer to hide physical complexities - to some extent this is less important because it's a subset of the data, somewhat flattened, loaded in-memory so users don't need to define joins themselves. BUT all the important things about that data - like that inventory should not be aggregated over time and how it should be calcuated, as well as the re-usability are still required. Bottom line:they are on their way to fixing one part, I'd like them to address the other.

Regards,
Cindi

Jmichel_franco

Thanks Cindy, very interesting debates around there (with Wayne too)
Regarding the ability of users to get their own data sources source, you note : "the issue is how they either bring it into the app or if they export and analyze in Excel"
This is perfectly correct, but my feeling is that QlikView/Expressor may provide
a very differentiating option if it makes it easier to do it in the app. This way, some users may not only mashup data for their own sake, but also prototype new applications for their lines of business or community. Based on those prototypes, IT can then take the leads to "productize" the protypes by retro-engineering the new data feeds and turn them into sanctioned, accurate, quality proofed data.
This is, in my opinion, a good way to balance the benefits of departmental and enterprise BI, and to channel "shadow IT" activity into collaboration between business and IT.
And this is an area where QlikView has already some strong arguments argainst some of their competitors because it has only one data access layer, not one for personal BI, one for "entreprise" self-service, etc.

Michael Tarallo

You all bring up valid and accurate points. Thank you for your insight and comments.

Traversing "the BI and DI trenches" myself, I have witnessed firsthand that if you build it - they won't always come. Whether you are talking about a DW or a semantic layer. It is nearly impossible to anticipate all the questions that are going to be asked and have the data support those questions. So what do I do when I cannot get the data I need to support my business questions?

I or IT access the disparate data using “extraction” methods that then allow me to import it into “other” tools for personal analysis and further manipulation. Now I have introduced my own version of the truth, with my own interpretation of metrics and dimensions that I cannot easily share with anyone.

Taking this one step further let’s introduce the issue of data quality. Even though DI tools may claim to have addressed this already prior to reaching the DW, I have witnessed DW data still needing additional de-duplication, enrichment, standardization and conformance. This prompts the business users to seek those aforementioned alternatives to get what they are after. This continues to enable the problems that the BI and DI tools were originally selected to solve.

With the QlikView Expressor solution we are defining metadata the QlikView away. Both IT and QlikView developers can easily define reusable QlikView Expressor metadata models with conformed attributes and business rules. Models can be defined directly from the DW or from individual sources of data. Even non-technical users can simply assemble QlikView Expressor metadata from a library of predefined templates and model components. Making it easier for them to create and share their own views of data while allowing them to collaborate with BI Administrators / IT and be part of the process. The result is a reusable and consistent approach when developing QlikView applications. It is also great for those one off requests as well as prototyping new answers to new questions. In addition to all of this, data lineage is captured and exposed through a Metadata Intelligence solution allowing Data Governance within all QlikView deployments.

Jean Michel said it best when using the word “balance” – You must have a proper balance of control and flexibility that includes both the business user and IT. I believe the QlikView Expressor solution offers that and much more.

Regards,

Michael Tarallo
Sr. Product Marketing Manager
QlikView Expressor

Performance Coaching

I think this is great content and I enjoyed it very much. This is clearly very well researched information on your part. I totally agree.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment