Drill to Detail - Drill to Detail Ep.15 ‘Modern BI and the Gartner BI Magic Quadrant 2016’ With Special Guest Cindi Howson

Episode Date: January 17, 2017

Mark Rittman is joined in this episode by Cindi Howson to talk about Modern BI and last year's Gartner BI & Analytics Magic Quadrant, the rise of self-service BI and IT's new role as enabler rather than owner, CIO priorities around BI & Analytics ... and, is the BICC dead?

Hello and welcome to the first Drill to Detail episode of the new year and I'm your host Mark Whitman. So Drill to Detail is a podcast about the strategy and business of big data analytics and business intelligence where we try and talk about the why and for what reason something new in our industry has come about. And I'm joined each episode by a special guest who's either making the products or setting the agenda within our industry. And in this episode, I'm really pleased to be joined by somebody who certainly set the agenda in the business analytics industry last year, someone who's writing an industry knowledge I've had for many years, Cindy Housen from Gartner. So Cindy, welcome on the show and maybe introduce yourself to the listeners. Yes, thank you, Mark. It's a pleasure to finally connect. I likewise have been following

your work for a number of years and I have been in the BI space for 20 years, originally as a user and customer at Dow Chemical, later as a consultant for Deloitte & Touche, and then my own business, BI Scorecard. Hopefully some of you have read my books, Successful Business Intelligence, and joined Gartner two years ago. Excellent. Well, it's great to have you on, Cindy. And really what I wanted to talk about was something actually led on from a blog post I saw that you wrote a while ago, which was about, I guess, kind of modern BI platforms. What is a modern BI platform today? What do sort of CIOs and companies do? I guess following on from the magic quadrant from last year.

And I guess really your thoughts on the industry and where things are going. So I thought first of all, what would be good would be just to kind of get you to recap on what was the kind of main thoughts behind Gartner's modern BI platform we talked about last year and the way I suppose in which you see the market changing in that way? Yes, so the last year's magic quadrant was both exciting and painful depending on which side you were on. But we really needed to reflect what had been shifting in the marketplace for a number of years. So first off, the magic quadrant is supposed to serve net new buyers. It's not supposed to just be a reporting of the market. It is to help buyers figure out who they should continue to invest in or who they should have on their radar.

And there had been a multi-year shift from what we call Mode 1, traditional BI systems, which are largely report-centric, to Mode 2, more agile, business user-led but IT-enabled, and more visual data discovery platforms. So if you look at the 2015 Magic Quadrant, we had both traditional vendors in there. So let's say we were comparing Crystal Reports and Microsoft Reporting Services with the likes of Tableau and Click. And in reality, customers are not buying those Mode 1 traditional BI platforms at the same pace that they're buying the modern. And we have data that actually came out after we published our magic quadrant that split the market share.

So we see that the traditional BI space, the license revenues there is flat to declining depending on the vendor we're talking about. So people are maintaining those, but they're not doing much net new buying. In contrast, the modern BI platforms showed a 64% growth rate in constant currency in 2015. There is a bit of a slowdown in 2016 but um we really wanted to make the magic quadrant best serve those prospective buyers yeah does that answer the question mark it does yeah i mean i think it's i think it's the important distinction there i mean what you're talking about is what's actually happening and you know that is where the spend is happening people are going out there and they're

buying the tableaus the clicks that i suppose the sas apps and so on And that reflects where the action is happening, really. But I think it also, it does reflect kind of where I suppose the innovation is happening. And certainly looking at the list of five areas that you talked about for the modern BI platform, that was certainly very kind of thought provoking, really. And I think it would be interesting kind of to talk through with you some of those things and just get your take really on, I suppose, in a way where that came from. And I suppose what's the impact and implication for listeners really. And the first one actually, there were five areas you talked about in the Modern BI platform. And the first one was kind of about data sources and upfront modeling.

And probably the thing that had the most impact, really, on people I know in the industry was this kind of feeling that it was talking about no more need for data modeling. Or I suppose kind of data modeling should be, in a way, kind of optional and all to magic. I mean, what was you thinking on that? And what are you seeing in the market? Right. So let me clarify something, too. We're not saying that there's no need for data modeling and no need for IT or governance. There is absolutely a need for this, but there are smarter and more agile

techniques available today. So if you think 10, 20 years ago, a centralized monolithic data warehouse was really best in class, the single version of the truth. But if you look at the number of new data sources we have, we didn't have sensor data before, or maybe not as much. All the external data sources, whether it's economic data, mobile activation, social data. So the need to be able to rapidly ingest and model as you go before you get to any kind of insight, that is really part of what makes up a BI platform. So if you have a data warehouse already made,

and that's very useful for some data, it might be predictable internal transactional data, financial data, orders data, great. Leverage that. We want the modern BI platform to leverage that. But if you have a new data source, as of yet an unknown value, variable quality, then you really want to make it easier for the user with the domain knowledge about that data to model in an agile way. So a little more agility, a little more ease of use, and we are starting to see even the software help with that, telling you where you have data quality issues, advising options of how maybe to mash or join data together. So we want data modeling.

We just want it more agile, easier, smarter. I think I agree. I agree. I think it's about knowing the right time and the right place, really, isn't it? I mean, certainly the very kind of, I suppose, kind of strict way in which we used to bring in sort of data into systems was, I suppose, kind of a function of the tools we used. But also it was a function. I mean, I suppose in a way we were being too perfect, really, and we were being too prescriptive about how things were done. And certainly I think the optionality is interesting I mean you mentioned about sort of tools that help I suppose discover data domains and so on have you seen anything interesting in the market there around that when what's your thoughts I suppose really on using say kind of machine learning to help with this yeah so and this is why some people say oh you're so boring how can you work in the same space for 20 years? And yet, as you know, there is constant innovation and just still so much going on.

So this is where there were a number of new vendors to the Magic Quadrant last year. I'll mention two, for example, ClearStory Data. So they are a cloud platform. They leverage Spark and machine learning to profile the data and help you smartly prepare it, but then also recommend other data sets to blend it and mash it with. So they were new to the 2016 Magic Quadrant. Another vendor that was new to last year's Magic Quadrant, BeyondCore, who Salesforce recently acquired. And what's interesting about BeyondCore is they also help you find the insights

and patterns in your data that you didn't even know were there, or you didn't know the questions to ask to interrogate. And we actually, as disruptive as the 2016 Magic Quadrant was for everyone, we really think this moved to modern. We think there's another wave of disruption on the horizon, smart data discovery, that the likes of BeyondCore, IBM Watson Analytics, SAP has a solution, their cloud, their business objects cloud has capabilities in it. So we do think this smart data discovery is yet another wave of disruption on the horizon. Yeah, interesting. And a lot of that innovation, I guess, is coming from outside

the BI space. I mean, for example, you mentioned BeyondCom, and that's a classic kind of BI tool, but there's Pexata, for example. And there's also, for example, what Amazon are doing with Amazon Glue. I mean, I don't know if you've sort of seen that and got a view on what they're trying to do with Glue and the kind of automatic discovery of things they're trying to do there. making available but this is where for me it's always what's what's really a turnkey solution or what is a difference between an idea and a vision that the vendor has so for sure amazon has a lot of capabilities and ip i want to see it um packaged in a single product. Yes so I mean moving on from that point you talked generally about IT being an enabler rather than a producer and I think that was a a very good point really I mean the idea that IT you know isn't a blocker of things but is is there to kind of enable the business users in the business to get the most out of the data I

mean what was you thinking around that really? Yes Yes. So this is, and actually, we do have, of course, we have the Gartner Summits coming up in February and March in a number of cities, the US and London. And I will be talking about, more about this concept of what should the roles be going forward. And this is something that CIOs at the symposiums were very worried about. Do they even have the right skills in their organizations? So I think in the past, it was fine for IT to be a little more reactive and let's say order takers from the business. So building to a specific set of requirements.

Now we have advocated in BI and analytics for a number of years the benefits of agile development, collaborative development. But I think this has become even more so where IT now may just provision the data and the business is actually producing the dashboards and the visualizations. Now for some content, if you have a very predictable, what we would call a mode one report where you require the governance, you have the predictability. So maybe this is a financial statement or a regulatory report.

You still want IT to own that. If you have more an agile question like why did sales decline this month or hey, if I offer this coupon to my customers, will they respond? That's more the mode two discovery. So let the business produce that. If it gets widely shared, maybe you would pass it back to IT to say, help me make this more scalable, more performant. Help me enforce the governance and security policies to prevent any problems or discrepancies.

But it is a shift in roles. And this means that some of the skills that IT needs to work on are more the listening skills, the collaborative design, the facilitation, and the inquisitiveness. And that can be very uncomfortable for those IT people who really like to work in a much more structured, documented environment. Yeah, exactly. And we'll come on to that a bit more actually in a bit. There's quite a topic around that I'd like to talk through with you. But that's interesting.

And also, I think you talked in the various papers at the time about the ideal thing would be for vendors and products to be able to bridge this gap between mode one and mode two. So that, for example, things that the users discover and things that become useful become part of the kind of formal curated data model and so on. Are you seeing that happening in the market? Are you seeing vendors that are kind of doing that? Or is it still more of an idea? Yes, this is where the devil is in the details, Mark, because this is what every customer would like. They would really like a single solution that provides both mode 1 and Mode 2. And I would say some vendors are further along in fulfilling that vision in a single product.

So we do have two resources. We have the Magic Quadrant, which scores vendors on capabilities, but also on their vision, their roadmap, customer support. We have a companion note, and this is why I joined Gartner at this point in time. It's very much like what the BI scorecard used to be. We call it the critical capabilities, and it is a side-by-side scoring of the vendors on how they do on this. So we do score them on the Mode 1 capabilities,

which are things like having a consistent semantic model, having strong security governance, having the ability to distribute a report maybe as a scheduled PDF, for example. And then we also look at all the Mode 2 capabilities. Can you create something in an agile way and promote it back to that systems of record reporting? So on this point, there's room for improvement. There are some vendors I think that do well in providing both those capabilities, but oftentimes it's across two separate product lines. And that is why we see a lot of augmentation, mixing and matching of vendors in the customer base yeah exactly exactly so so you also touched on analysis and insight delivery in that list of five um sort of features i suppose and there's obviously a big

drive towards kind of data discovery and freeform analysis and collaborative storytelling data storytelling and so on is that something you see as being kind of like i guess i guess more popular more being adopted and what's the value i guess what's the value in that type of analysis? And is it the only type of analysis, really? So it's not the only type of analysis, but it is a very important part of the analysis because when we're literally drowning in data, we have to use what we know from brain research

and visual perception to speed the time to insight. A dense page of numbers is not easy to interpret. So using things like charts or geographic mapping, smart coloring, and smart coloring. So none of this should look like the pinball wizard pinball game. It really is about using color, and highlighting, and even animation to reveal the patterns and data. And also, as we just published some predict notes, using things like natural language generation

to then explain the chart to you in textual form. And there's some interesting analysis actually out of Sheffield University that shows that even students in data analytics really don't do well at interpreting graphs. So we want that narrative in addition. So all of that is part of what we say is sharing the findings and part of that data storytelling. And these really will become standard features as part of a modern BI platform. Yeah, interesting. I mean, you mentioned natural language processing. I mean, do you see voice as being part of the future of this? I mean,

yeah, absolutely. Yes. So and we've seen some interesting work from some vendors on their use of voice. So Sisense, for example, also on the Magic Quadrant, they have some interesting capabilities where Alexa will use, you can ask Alexa, show me my sales for this period. Am I positive or am I declining? And Alexa will answer you back. Of course, for Microsoft, we're seeing the integration with the cortana digital assistant so yes voice uh definitely part of this interesting so so for the part of the audience who read your papers then who weren't kind of worried about bi when they saw the classic data

warehouse people that saw the paper that was the role of the data warehouse and semantic models that was kind of interesting as well because it talked about you know no intervening semantic model required data warehouse not a prerequisite I think we kind of generally accept that but there was also talk about self-service data prep by business users I mean that again is an area that's been interesting and I think Tableau for example have launched something recently with that in there most vendors do what's your view on self-service data prep, in particular in the context of BI tools? Yes, and again, just to reinforce, Mark,

so when we say it's not required, we don't want it to be a prerequisite. If you have it, we want the modern BI platform vendor to leverage it. So it's not dead. We're not saying it's dead. It doesn't go away by any means. But yeah, so self-service data preparation,

it is about this rapid ingestion of new data sources or unmodeled data sets that might be coming from a data lake. And I mean, we have a market guide on self-service data prep vendors. There's at least 20 vendors in there. Some of them are coming from the BI and analytics front-end space. So this is where, yes, Tableau announced Project Maestro.

And in the Tableau space, of course, this is where we see other partners like Alteryx, Paxata, Trifacta being used in conjunction with Tableau. IBM, for example, also signed a partnership with DataWatch to help with self-service data prep for both the Cognos customers as well as the Watson Analytics customers. So we see movement from the BI and analytic platform vendors, but also the traditional ETL vendors like Informatica has their self-service data prep tool as well.

And then, of course, in the data science space, there's solutions there as well. So I think all of this reflects the need for the people that know the data, know the domain, to be able to model it, not just have it in the hands of a few of those ETL developers. So again, ETL doesn't go away. There was a really good case study, a customer I was speaking to, where they actually use these self-service data prep tools

and it feeds back into the requirements for the industrial strength ETL process. Yeah, exactly, exactly. So there was a blog post, I think, someone from Datamir wrote recently, I think it mentioned you. And it was talking about data governance, and basically talking about the fact that data governance is an area that I think big data has got away with a bit in the past. Again, data governance and a whole organization around that. Is that an area that you think is still in focus? Have you seen some innovation in that space? And what's your thoughts on data governance? Well, it's definitely in focus because there is fear that in the self-service world that things will just degenerate into chaos. And we don't want that.

We don't want that. We want empowerment and agility, but without the chaos. And a webinar that my colleague Rita and I did last year, we did a poll on this and asked customers where they were on their journey. And fortunately, I think it was a small percentage, but like 10% feel like they are already in the chaos world. So we'd rather none. But everyone is very worried about this. So it is something that you need to keep in mind.

But I think as an industry, we all need to remember that it has always been for the business to decide what is the acceptable level of risk, what data needs to be tightly governed, what data can we be a little more lax about, and it's really IT's role to enforce those policies. I think in the past it was almost, well, IT both enforced the policy by the sake of the technology being so difficult to use and remaining in the hands of a few. So almost that governance was built into it by sake of technology. Now we have to step back and say, okay, where do we build the governance into the process? And when do we just let users have free reign? So what's your view on BICCs at the moment?

I mean, I think you can probably find a statement from Gartner everywhere saying something is dead. But certainly, BICCs, are they still valuable? Are they still being used? What's your thoughts on that? Absolutely. Well, so this is my colleague, Frank Boyden-Dyke, who in the keynote said he hated everything about the BICC. First, it has C for centralized, which he didn't like. in the keynote said he hated everything about the BICC. First it has C for centralized, which he didn't like.

So they're not dead. And I do want to say, call it what you want. Some organizations still call it BSS, decision support, which we might say is a little bit of a dated term. We do use the term analytic center of excellence increasingly really to signal the shift that we don't want it only centralized. So you do want some centralized experts for economies of scale, for the commonality, let's say, and perhaps for the Mode 1 requirements.

Where you want some decentralization is where you want that deep domain expertise, so for a particular business unit or functional area. But the important thing is that it's not in silos as maybe it was done in the early 90s. It really is done in coordination with the centralized team. So the centralized team, the analytic center of excellence becomes the facilitator, the evangelist, the connector and identifying the commonalities. So they enable the decentralized teams. And all of this, the goal is to get to those higher levels of maturity where we're not just doing reporting.

We're really doing that diagnostic and predictive analytics as well. Okay. Okay. So the last thing I want to talk to you about, just last topic really, is you talked about, I think you've often talked about how to select a BI tool. And I think certainly in a blog you wrote recently, you talked about the process someone to think through when you're selecting a BI tool and a BI kind of platform, really. And you talked about, for example, you know, stop trying to focus on a single vendor and so on. Just outline what your thoughts were there on that and what are the golden rules you have, Cindy, on making BI tool selections? So I think, again, depending on where you come from, I think there was a view, can we just have a single vendor do everything and that would lower our cost of ownership and simplify the integration. And I think that's a good vision, but the reality is that most organizations

do have mixed portfolios, and best capabilities are often spread across multiple vendors. So I don't want companies to be afraid of mixing and matching. What you want to look for is to minimize the redundant, the overlap. So you don't want two tools that really do the same thing and there's no clear use case for when to use one or the other. So minimize the overlap, maximize when they provide business value. That should be ultimately the way you measure if you bring a vendor into your portfolio is that they will help you achieve more value, ideally at the lowest cost of ownership,

but sometimes that's difficult. And then in terms of a process, we do have a selection process. A lot of it starts with what really the market has changed to free trials and land and expand. Try it on a small proof of concept, understand the value, prove out the value, and then go back and reevaluate and say, okay, who are we going to use to provide these capabilities? Yeah, exactly. And I think, again, just I think something else you just mentioned at some point is make sure you've got a business goal and a kind of way of measuring that as well. Because so many projects we see, so many initiatives don't actually have a kind of a point to it really and no kind of particular goal or objective. I mean, do you think, do you agree with that really? Absolutely. We, in fact, we did a study last year that unfortunately 85%

of organizations do not measure or have clear measures of their business outcomes from BI and analytics. So it's not to say that they're not achieving them. They're just not documenting and tying them into those hard business benefits. So we think that's really important, both for the buy-in, but also to ensure ongoing engagement. Yeah, exactly, exactly. So, well, look, I'm conscious of your time, Cindy. So just obviously busy time coming out for you now you've got the kind of next magic project coming out and so on i mean roughly when

when should we expect that is it gonna be the next couple of weeks few weeks or what's your time to discuss now well on our calendar uh on our public facing calendar we always aim for february some things may happen to derail that but that is the plan that I hope will happen. Okay. Excellent. Excellent. Well, hopefully it'll be as big a bombshell this year as it was last year. I mean, certainly it was interesting. I mean, certainly in my part of the industry, it was certainly, I think I was talking, but with all these things, there's always an element of truth in these. And I think that reflecting where money's being spent, where the interest is, is fantastic. And also any kind of, any signs that the business is taking BI seriously, getting involved in it is all good, really. So as we said, you know, IT should be an enabler.

It's great there's interest out there. And let's make sure, work as a team and, you know, and so on, really. I mean, it's very, very interesting sort of, you know, article you wrote and paper and so on, really. Thank you. And Mark, thank you for all you wrote and paper and so on really thank you and mark thank you for all you have done and continue to do for the bi and analytics community yeah thank you thank you cindy thank you very much and take care and hopefully speak soon thanks bye Thank you.