Drill to Detail - Drill to Detail Ep.21 'The Fall, and Rise, of Microsoft BI' With Special Guest Chris Webb

Episode Date: March 14, 2017

Mark Rittman is joined by Independent Consultant Chris Webb to talk about MDX & DAX, MSAS and SQL Server and the fall ... and rise, of Microsoft BI

So my guest in this week's episode of Drill to Detail is someone I've often described as me in a parallel universe where I got to do Microsoft technology rather than Oracle. So Chris Webb, welcome to the show. And why don't you introduce yourself to our listeners and tell us a bit more about what you do? Hi there. Well, I'm a consultant and trainer specializing in certain bits of Microsoft BI. Microsoft BI is too big for any one person to really know it well. So I kind of specialize on analysis services. And for those of you that don't know, there are two flavors of analysis services, multidimensional and tabular. I do both, but I'm probably better known for multidimensional. And then on top of that, related products like PowerPivot, PowerQuery, the Excel-y bits of that, and Power BI now as well.

Fantastic. Well, the reason I wanted to get you on the show, Chris, obviously I know you from before and so on, but there's a few interesting areas you've been working in. You mentioned, obviously, analysis services. And I want to talk about, I suppose, the history you've had with that. And I suppose the journey that's been through, because originally there was, originally there was a kind of the multidimensional side. You mentioned Tabula and so on. Be good to kind of talk through that and your history with that and where that technology has gone and so on there.

I suppose in a way, my kind of career history in IT has been very closely linked to Microsoft BI. And, you know, I've kind of been there at the very beginning. I've seen the highs, the lows, and then the highs all over again. So, well, I'm 41 now, and I guess I started working with Microsoft BI right at the start of Microsoft actually having a BI strategy, which would have been, I'm not even sure, probably late 1998, somewhere around then, right when SQL Server 7 was about to be released and a company I was working with got advised to take a look at the brand new OLAP tool

that was shipping with it, which was called OLAP Services then, and which subsequently became known as Analysis Services. So I started working with it then. I was lucky to be on a project where I was doing a lot of really interesting stuff with it and learned it really well. After that, I went for a couple of years to actually work with Microsoft in Microsoft consulting. It's not really Microsoft proper. Microsoft consulting is kind of the bottom of the food chain as far as Microsoft goes. Though hopefully there aren't any Microsoft consultants listening to that. But I'm not going to pretend that I was anything senior in Microsoft at all, which gave me even more experience of it.

And then I did some more PERMI work. And then I kind of made the decision about 10 years ago to kind of set up on my own. And it was that kind of fateful decision that I guess you've made and other people have made to say, well, you know, let's give up the day job and let's become a consultant. You know, I've done consultancy before, so it wasn't that much of a big step. And I always thought, well, you know, if it all goes wrong, I can fall back on being a contractor or something. But, you know, I'm always very keen to point out that I'm not a contractor. I'm a genuine consultant and I work with lots of different customers.

And I don't do kind of long term projects or six months or so. So, yeah, about about 10 years ago, I made that decision. And I haven't really looked back since as well. Fantastic. So you've had a blog for a long time as well. I mean, your blog has been running for probably as long as I've been blogging. Tell us about kind of that. How did you get into kind of blogging as well? Well, again, I suppose this is probably 11 or 12 years ago. And it was about that time when blogging suddenly became fashionable. And I was doing a lot of work answering questions on news groups. And, you know, I kind of felt like I was answering the same

questions over and over again. And I thought, well, you know, I of felt like I was answering the same questions over and over again and I thought well you know I should I should write up some of this stuff as a blog post so that next time somebody asks me the same question I can just post a link to my blog rather than having to keep on repeating myself and I don't know. I always think like writing a blog is kind of like going on a diet. The vast majority of people who start a blog, like the people who kind of start on a diet, give up after a couple of months because it feels like effort. But there are always going to be some people who, I don't know, probably like you, I guess, there's some kind of inherent need to write stuff down. There's probably a certain amount of narcissism there. There's a certain amount of, you know, kind of just enjoying the attention.

But also, I think there are a lot of other good reasons why I like to blog. Partly, it's kind of part of my own personal learning in that if I if I see something if I learn something and I don't write it down then I'll have forgotten about it and you know all the time when I'm googling for stuff for work purposes I find old blog posts that I've written and I've completely forgotten that I even knew all this stuff so I'm kind of learning from myself and yeah I suppose when I went into business as a consultant, it turned out to be very useful for marketing purposes as well. But I can always remember the guy I was sitting next to at my last job. When I told him what I was going to do and I was going

to give up and be a consultant, he turned around and said to me, of course, you realize that you've got to stop blogging now. And I said, why? And he said, well, if you write down everything you know in your blog, you're giving your knowledge away for free. Why would anybody hire you when they can just read everything you know on your blog? And actually, I didn't have a good answer for him I you know it's a good question I had no idea but I just went and became a consultant anyway and I suppose it just turned out that most people will read a blog they'll find the answers to the blog and it acts as it acts as good marketing that way. But something that I certainly do is that I know there's a blog post there. I'll read it.

I won't always take the time to understand it. But in the back of my mind, there is this feeling that whoever wrote that blog post obviously understands the problem. And, you know, if I had that if I had that problem in real life, I might actually rather than try and understand what he'd written, he or she had written, rather, I might just actually go and hire him as a consultant. And I suspect that's what happens with me quite a lot. Exactly, exactly. And I've said exactly the same words myself, actually, that it's one thing to read something, but the actual ability or the time to go and do it yourself is another thing, really. And it's always a great advert. um so so i mean just let's go back in time really to to what i would call almost like peak peak analysis services time so i was probably parallel to you i was starting consulting you know about 10 15 years ago and uh i remember at the time that

tools like oracle express and oracle olapp were out but then analysis services came along and and it was it just took the market by storm and it was suddenly in terms of OLAP it dominated the market. Why do you think analysis services then became so popular and why did it do so well at that time? Well I guess a couple of things. I mean first of all I think that the technology was great. You know analysis services multi-dimensional to be honest has not had that much love from Microsoft in the last couple of years, but is still incredibly popular. And I still do a lot of work with it. And that's that's a testament to exactly how good the technology is. It was built with love by people who kind of dedicated a couple of years of their life to it.

And they did a really good job. And, you know, good software always endures. So there was that definitely. And at the same time, Microsoft was very clever with its pricing and its bundling with SQL Server, and the integration with Excel as well as a client tool. And so there were there was this kind of, you know, a couple of good, a couple of factors, a couple of clever decisions that Microsoft made, made it all come together. And suddenly, by the time, you know, the classic Microsoft version three, by the time we had SQL Server 2005 and Analysis Services 2005 came out with it, it was just an irresistible force in the world of OLAP, I think. Yeah, absolutely.

And at one point, that was the de facto OLAP server that was being used. I mean, things like Oracle and Cognos and other ones there were being used, but Analysis Service was this kind of de facto kind of thing there. But I guess there was a lot of success with the OLAP server kind of the OLAP server but the I suppose the reporting side analysis side at the time was always a little bit kind of fragmented there was the prior clarity stuff and so on yeah what kind of happened tell a story around that and and what were the what was what was the kind of reporting tool side of it at the time well I mean the initial strategy that Microsoft had back then I think was to to keep away from the front-end market

because analysis services was part of the SQL Server product group and that was, you know, that server-side technology. And to a certain extent, their idea was that if they were front-end agnostic, there would be a lot of diversity out there and it would help people choose a tool that was right for their needs. And I think that was true to a certain extent. You know, at the same time, they were building up Excel a little bit. But back in those days, I'm not sure the, you know, the Excel team were all that serious about BI. So there was support there, but it wasn't all that great. But it got to the stage really where a lot of customers wanted analysis services, but they also wanted to deal with

Microsoft to have a complete BI solution. And Microsoft never had that. They would say, well, you know, here's Proclarity or here's Panorama or here's one of, you know, several other small companies out there that were active, that were there as client tools for analysis services. And that that kind of fragmented approach really meant that Microsoft lost a couple of, you know, lost a couple of customers who might have wanted to have a complete Microsoft BI solution rather than a Microsoft server and then deal with a smaller company for the front end. And then and then after the kind of peak analysis services, we probably come on to the beginning of the bad years for Microsoft BI or the less successful years anyway. And that that kind of starts off with the Proclarity acquisition. acquisition um so microsoft had pro clarity is its its closest partner as a front end for analysis services and um it then or part of microsoft and remember you can't think of microsoft as this kind

of monolithic whole uh part of microsoft said well okay we're going to buy Proclarity because this will be a great addition to the whole kind of office suite of products. And they bought Proclarity and then they did the most unimaginably dumb thing possible. They bought it and then said, oh, well, we'll take the developers and then we'll kill the product and then we'll build something else equally good and of course they did that and they built something different that wasn't as good that was turned out to be something called performance point which never really took off and in a in a stroke they they killed off their closest partner and killed off all of the other you know a lot of the rest of the partner ecosystem because microsoft then these guys then thought, well, you know, Microsoft have bought ProClarity, that will be the kind of classic Microsoft, Microsoft now has got a full stack, who's going to look for a third party tool to work on top of Microsoft BI. And that immediately decimated the third-party tool market. There are still a few out there that have got some quite strong tools that are in certain niches.

But that killed off the partner ecosystem, and then Microsoft killed off Proclarity, and then they didn't really have anything. That was an interesting time, wasn't it? Yeah. I mean, it was the most ridiculously stupid decision. And, you know, to a certain extent, Microsoft benefited from some of the guys they got. Slowly, I think, from Excel 2007 onwards, Excel became a much better front-end tool for analysis services. And certainly by Excel 2010 and 2013, and now with 2016, actually Excel, I think it's a pretty good front end for analysis services. And certainly

the tool of choice for most people who are working with analysis services and need a front end. So it did have some kind of long-term benefits, but it did lead to a lot of bad feeling in the Microsoft part of the community. And a lot of customers who were just scratching their heads thinking, what on earth is Microsoft doing? Because it was a crazy decision. It's interesting, isn't it? I think big companies do have this habit occasionally of taking a product that works well and then doing something just ridiculously crazy with it like that. And you mentioned there about Microsoft buying a kind of a front-end sort of tool vendor. that it works well and then doing something just ridiculously crazy with it like that and you you mentioned there about microsoft buying a a kind of a front-end sort of uh you know tool vendor um

whereas previously the kind of strategy was to get analysis services under in working with as many products as possible and you see that with with google for example with android you know one minute they they put android onto as many phones as they can then they go out and launch pixel and and it confuses kind of like you know i, I suppose the ecosystem, what do they do? You know, are you competing with Google in that case? Are you kind of, you know, are you partnering and that sort of thing? I mean, as a consultant, it's always hard when that sort of thing happens, isn't it? Because that's your business.

But you've then got a crazy situation to deal with. Yeah, yeah. deal with yeah yeah and i'm i'm lucky that um the microsoft bi ecosystem was already so so big and so strong that it was able to withstand that kind of shock yeah yeah but yeah it was a difficult time um and then shortly after that it was kind of the beginning of perhaps again some some interesting decisions on the part of Microsoft about where its BI strategy was going. Actually some of these decisions have actually paid off in the long run and you know we're talking about seven or eight years ago now and actually they're beginning to pay off big time now but you But Microsoft then made two key decisions a couple of years after that. They first of all decided that after Analysis Services 2008 and R2, around that time,

they thought, okay, Analysis Services, the kind of classic Analysis Services, it needed a bit of re-architecting because this was a product that had been originally, development had originally started before Microsoft had bought it in the mid 1990s. And, you know, the BI technology game had changed and Microsoft obviously thought, well, OK, we do need to make some fundamental changes because clearly the way the industry has gone has been towards in-memory column store databases and analysis services, classic analysis services, analysis services, multidimensional as it's now known, is disk based and certainly not column store either, row store based. So they thought, well, OK, we need to rip out the engine and replace it.

And in order to be able to do that, they had to do some pretty serious re-architecting work. And at the same time, there was this feeling, which was probably quite rightly, probably quite right, that analysis services had done massively well, but it hit a certain glass ceiling in that the concepts involved with dimensions and cubes and things like that were a bit too difficult for people to learn. And Microsoft wanted to go to the next stage of BI. So as well as kind of re-architecting analysis services, they decided to change a lot of the concepts as well as kind of re-architecting analysis services, they decided to change a lot of the concepts as well. And that's how we got what became Analysis Services Tabular, where it does a lot of the same things as Analysis Services Multidimensional, but it's probably easier to develop and easier to use.

And it's got this kind of fast in memory column store database behind it and then they also took this decision to say well okay if we want to really get this out there then we need to work more closely in excel because this was the stage where we were coming to self-service bi everybody was talking about self-service bi it was when tableau and click were really beginning to take off and you know at the same time, Microsoft has got this ongoing problem with Excel, which is, okay, everybody uses Excel, but people only tend to use about 10 or 20% of the functionality of Excel. And, you know, if you come to somebody, you know, if a Microsoft rep would go to somebody, you know, a big company and say, hey, listen, it's time to upgrade to the

next version of Office, most people would say, well, we don't want to because we're just using all the features of Excel and Word and things like that that we've always done. So we don't want to pay for any more Office. We don't have to pay for the latest and greatest version. You know, we're happy with what we've got. So what Microsoft wanted to do, I think, was to say, well, okay, let's, we know that people do a lot of reporting and analysis in Excel.

We know there are a lot of really obvious problems with doing BI in Excel, and the likes of Tableau and Clip do a lot of marketing to try and convert people who are doing BI and Excel away to their products. So they thought, well, OK, actually, let's make Excel into a better BI tool. So they took this kind of new analysis services tabular engine that was still available on a server as analysis services, and they put it inside Excel as PowerPivot. And that was actually a really, really good idea. So it first of all turned up as an Excel add-in. And then as of Excel 2013, it's a native part of all Windows desktop versions of Excel.

And at the same time, they thought, well, this is really good. We also need to have some other self-service BI tools. So they grabbed some other bits of technology from around Microsoft. They grabbed what became Power Query, which is a kind of self-service ETL tool, really. And that is literally, that was integrated, first of all, as an Excel add-in. And now with Excel 2016, it's a native part of Excel 2016. And now, in fact, as of this month and the very latest release of Excel for Office 365,

it's the native way of loading data into Excel. There were some other things that went on, which are kind of less successful, like Power Map, which is kind of 3D maps inside Excel. But, you know, there was this whole focus on Excel as the center of your BI world and, you know, making everything work with Excel, which was good in parts. But there were a couple of inherent problems, I think, with this strategy in that it assumed that all BI was self-service BI. And certainly lots of BI is self-service BI, but not all BI is. And the existing Microsoft BI partner ecosystem was used to doing kind of old style server based build a data warehouse, build cubes on top of it, build reports like bi so it didn't really work well with that and then there was still the problem of people of companies actually being on much older versions of excel so they could use um the the brand new excel toys that microsoft had given them uh and then there were ridiculous

problems with the licensing where you really needed to spend several days studying the licensing to understand which SKUs and which editions of Excel had all of the functionality in. And it always turned out that the SKU that your company had didn't support all of the bits you needed. And then, of course, in any big company, you'd say, well, I need the latest and greatest version of Excel to do all my BI stuff. And you go and find the person that's in charge of Office inside your company. And they would say, yeah, we're going to upgrade to that version of Office in five years time. And that was, in a lot of cases, the end of the story about Excel-based BI with a lot of Microsoft customers.

So it was a good idea. It went down well with the analysts. But actually, it was going to be far too long before it paid off for Microsoft to bet their entire BI strategy on it. And that really went on too long. You know, the whole kind of all BI's Excel strategy, it just felt a bit like a bit of a dead end for Microsoft BI. And what was worse is they didn't spend as much time and love on the kind of traditional SQL Server BI components as they should have done and so it kind of ended up that you know there was a feeling of stagnation for a couple of years I think that's when I saw you I mean you spoke

at an event that we ran my company ran and interestingly I mean we we thought you guys we thought you'd come in and you'd be basically saying everything was fantastic but certainly there was a kind of transition point, wasn't there? And at certain points, I guess listening to you, Chris, it does sound, though, that Microsoft made a bunch of bets and they paid off, really. And if you look at, I mean, to do things like, I mean, winding it back there, there's a ton of information you've given us there. Winding it back to MDX.

MDX was the de facto language for querying uh olaq cubes and even now i mean ironically oracle now supporting it in the database with analytic views in 12.2 um to basically to move off of that and to move off of what is the market leading kind of i suppose data multi-dimensional server and language that must have been at the time a big gamble for them really and and for you i mean your whole career you are mr mdx aren't you yes um you know that that was a that was cheese moving on a grand scale i think um but you know i it felt very painful at the time um i think with hindsight i can i can understand why mic why Microsoft did it. There were some good reasons. I think MDX was pretty difficult to learn. I'm not sure its replacement in the Microsoft world, which is DAX, quite difficult to learn. And at the same time, when Microsoft was making this big

play about everything being related to Excel, if they were going to put all their BI in Excel, then the calculation language for that BI had to be something that was easy to learn for Excel users and had to be consistent with Excel formula language. And that's how we ended up with DAX, which is this new language. I mean, MDX is still around. Excel pivot tables still generate MDX. You know, analysis services and Power BI can all be queried in MDX still. So it's not really gone away.

But, you know, it was one of those points, I think, when people have, that probably everybody has in their IT career, when you come onto the fact that you've spent ages learning a bit of technology, and that technology has come to the end of its natural lifetime. So do you think, I mean, you mentioned self-service there, and I think, again, a gamble that Microsoft took, and I remember you talking about it at the time, the SQL Server Analysis Services Tabula I mean you know and you go to go from multi-dimensional to anything Tabula you know just send shivers down anyone's anyone's back that is is an OLAP developer

but looking at what went on with say sort of self-service and with kind of tools like Tableau with their kind of cache and click view it looks quite kind of forward-looking really I mean I mean I guess that was kind of one question I suppose to you is you know do you i mean a lot of work used to go into the design of an overlap cubes and and that sort of thing do you still think there's a kind of place for uh you know carefully designed server-side models really for for the kind of analysis that you do or has that day passed really what's your thoughts on that well i i'm biased but i i do think even though the this whole idea of kind of formally defined server-side models is very unfashionable at the moment um there are still a lot of people out there still doing them um a lot

of people in the microsoft world and i guess a lot of people in the oracle world as well and i think their times their time will come. And maybe something we can talk about a bit later is actually Microsoft's current strategy is very clever, and that it allows for you to make the choice between that. But, you know, it's one of those things where, on one hand, yes, the whole kind of self-service, build your own model option gives you a lot of flexibility but you know as you and i probably know well um where there's a free-for-all and everybody's building their own models there's a lot of opportunity for people to redesign the wheel

multiple times for you know this explosion in the number of models for people to make mistakes for people to um you know redefine calculations a hundred times for for maintenance to become an absolute nightmare uh and these are all stories that i i heard from here from people who are working with self-service bi tools not just in the microsoft world but but in other spaces as well. So the idea of a kind of central model built by someone like me, you know, a kind of an IT department person, I still think that there's a big place for it. And in fact, a lot of the time when I talk to people who say, yeah, we want self-service BI, it actually turns out that what they mean by self-service BI is really more the ability to build your own reports

and do a bit of modification rather than build anything from scratch. And the idea that somebody will do all the difficult work and build a data model for them on the server side, something like a kind of traditional OLAP cube, is actually very appealing for them. Yeah, exactly. So, I mean, we'll move on to Power BI in a second, but yeah, what did you think when you saw that MDX was being supported in the Oracle database with the new analytic views feature? I mean, it must have raised an eyebrow from you at the time. Yeah, it's always nice to see mdx still carrying on i've got a couple of

couple more years before retirement now uh and i'd still like to think that i do make some money from mdx till you know till the day i retire but um i suppose the thing about mdx is that so long as excel still generates mdx from pivot tables and cube formulas and things there is always going to be there are always going to be third parties apart from Microsoft who want to support MDX in their own lab tools so I don't think pivot tables in Excel are going to move away from MDX anytime soon so MDX is always going to have that that that life in it okay so let's let's wind forward to kind of now really so so power bi has been the the breakaway success really isn't it for microsoft

and and it's gone from being a tool i mean i think i remember reading some of your blog posts back in about 2013 or 2014 and and there was there was quite a lot of kind of you know i suppose discussion and angst and and and kind of pivots and so on around sort of power b tell us the story of power bi and and how it went from being something that was kind of you know maybe not so good to being just the kind of i suppose the de facto you know the the tool everyone uses now for for kind of personal bi really so i mean power bi the original power bi was a kind of brand name for this whole Excel based BI strategy. So covering Excel plus add ins like Power Pivot and Power Query and Power Map and blah, blah, blah. And so this this went on for a couple of years and it became clear that nobody was really happy with it.

People were not buying it. People were not buying it people were not using it um the basic strategy was right but there were just too many problems with um you know licensing and additions and things and things were progressing too slowly so it came to a head and microsoft somebody inside microsoft said okay we need to we need to completely reboot the strategy so this is when we had the new power bi turn up and it was quite closely linked with a guy james phillips from who came from outside microsoft and was kind of recruited to be the the head of the new power bi and i think some of the you know some of the kind of good decisions that were made at that point are probably attributable to him. But basically, they said, Microsoft sat down and said, well, we've got good technology. We've got some good ideas.

We just need to look and see why Power BI is going wrong. And so what they did was say, OK, the whole Excel strategy, good in theory, but there are a lot of problems. We'll carry on doing it. And so the Excel strategy is carrying on today and it's beginning to pay off long term. But they took the technology behind Power Query and Power Pivot and some of the visualization strategy that had been hanging around in tools called Power View that had, again, not really taken off. They put them all together in a single desktop tool that is free to download called Power BI Desktop. They created a separate cloud-based service where you could publish your

models and your reports up to, which I'll call powerbi.com. And then they made it available through a very simple, straightforward licensing scheme where there's a free tier and a very inexpensive professional tier. And they put Microsoft's full marketing force behind it. And it was one of those overnight successes that had been 10 years in the making, because it wouldn't have worked if it hadn't had a long history of technology development behind it. But when they actually finally put all of these bits of technology that have been hanging around forever together in a way that worked, then suddenly it all came right. And, you know, this is more or less where we are today. And I think it was probably also the point where Satya Nadella took over at Microsoft and there was a big kind of culture change inside Microsoft. Microsoft felt like it had its mojo back at the kind of highest level.

It seemed like Microsoft was making good decisions again. And, you know, probably one of these good decisions was to pour a lot of money into bi and power bi in particular so you know we've now got you know microsoft you know pouring all of its money into something that's actually going to make it money with its product base rather than doing stuff like you know buying nokia and you know trying to come up with some music players which is yeah particularly ridiculous yeah i mean microsoft's almost dare i say it cool now isn't it really i think certainly with with satya there is the kind of the the leader and so on um but i think for me when i realized suddenly realized that power bi was good was when i was finding it was being used in my my old company an oracle kind of consultancy

we were using it internally there and i was thinking this is interesting so um and it's actually being used instead of kind of tableau i mean again what's your take up in Tableau was the other kind of breakout success around this sort of time I mean have you had an experience with Tableau what's your kind of view on on that compared to Power BI and how they're different similar and that sort of thing yeah I mean I think Tableau is interesting I'm a big fan of Tableau I mean I think anybody that does BI looks at Tableau and can see what they do really, really well. And, you know, in the in the data visualization space, they are still the clear leader. But again, it's one of those I think it's one of those situations where the the software has done well, but perhaps is not adapted to some of the kind of new realities

of IT. And so when Power BI came along, they were native cloud. In fact, they're cloud only. There's still a lot of talk about a kind of on-premises version of Power BI, which will be turning up later this year. But to be honest, the cloud-first, cloud-only strategy really paid off, I think, for Microsoft. And at the same time, Microsoft, the way it licensed Power BI meant that there were going to be a lot of cases where people compared products. They compared Power BI with something like Tableau. They could see that in terms of data visualization, Tableau is a lot better in terms of some aspects of maturity, Tableau is a lot better. But on a technical front, Power BI actually is pretty good and in some respects a lot better than Tableau. And then you would compare the price and then it would be, well,

clearly, you know, Tableau is a very, very expensive product. Power BI is very, very cheap. We're going to go with Power BI. So, you know, and Power BI has had a lot of money spent on it to improve it. You know, when you see the new features that come out every month, you know, some of them, like all features, some of them are a bit rubbish. Some of them are really good. But, you know, in general, the amount of new features and the amount of really good new features and the speed that it's developed, you know, I've never seen any product change that quickly. And it's kept up pace know, I've never seen any product change that quickly. And it's kept up pace for over two years now. So, you know, it's clear to everybody, you know, clear to the customers in particular, this is something that Microsoft is very serious about. And, you know,

in one of those, it's one of those situations where you'll look at it one month and say, yeah, it's great, but it doesn't have this one particular feature that I really need. And then next month, that one particular feature turns up. So you kind of have that faith that even if it can't do what you need at the moment, it's going to get there sooner or later. One question I want to ask you is, it sounds like you've had exposure in your business model, you know, as a consultant, as a trainer, as a kind of an entrepreneur in this sort of space, you've been exposed to this world of self-service and cloud probably a lot longer than, say, people from my world of Oracle and so on, where that's newer. I mean, how have you managed to keep a consulting business going? And when you've had customers who are doing things self-service,

who are used to doing things in the cloud, where there's not so much kind of installation work and so on, how have you managed to stay in business and still stay relevant and doing well over all this time? I think probably by doing more training. It's a really good question in a world of self-service BI. And even though you might be cynical about self-service BI, I think it is true that there is a lot more of the work that you or i used to do being done by people inside the business so that there's there's less consultancy work out there so as a result you know probably the only way to to carry on making money is is really to to do

bring more training into the mix i mean i do i do formalbased training, but I find also a lot of the consultancy work I do has got a very strong training component to it. And so in terms of the mix of consultancy and training I do, there's a lot more training that's crept in over the last four or five years. But do people still go on classroom-based courses then? I mean, you'd have thought, again, if you believe the dream of self-service, there's no training required. It's just kind of straight. Do people still go on formal training courses then these days? Oh yeah, absolutely. I think that's not going to go away. The thing that I always laugh about with self-service BI is that if self-service BI is really a thing, how come Tableau and Click consultants get paid so much? Yes, exactly.

There's a lot of hype about self-service. But in actual fact, there is that trend that more and more people in the business are doing this. And also, I think what's really true about self-service, and this is where it pays off, especially for Microsoft, is that it's not unskilled analysts that are doing the BI work. It's the people who've got good IT skills, people who maybe, you know, worked a bit with relational databases, who've done a bit of development, who are then told by their boss, well, you've got to go and do some BI now. And they've what Power BI and all of the other self-service BI tools have done is that it's made it easier for. IT people who don't necessarily have that strong specialization with BI to go away and do BI and reporting work.

And it's democratized it within the IT department. And certainly what I'm seeing with Power BI now, especially, is that it's massively popular with very small and, you know, anything up to kind of small and medium-sized businesses, up to, you know, a couple of hundred people. Companies that have got like, you know, four or five people in their IT departments, companies that would have never considered building a data warehouse in the past or using any of the previous BI tools, companies that probably wouldn't have been able to afford any of these other BI tools and certainly wouldn't afford Tableau or buying analysis services on a server. But that's where Power BI does well

with the IT departments of those kinds of companies. I mean, it does well at the enterprise level as well, but that's where I think the big growth on Microsoft is coming from. It's for companies that have not done BI before or have been doing BI with Excel and Access. Interesting. So you touched early on on i suppose interesting

architecture is interesting um capabilities of the tool now and i also i think i noticed on your blog you did a kind of a guest post on the pyramid a pyramid sort of a uh blog as well it was about kind of i think i think about being a run maybe power bi on on premise as well i mean what where are things now with the technology what's interesting about what what microsoft are doing now with power bi and and the stack at the moment so i think what we're at the point now is where all of the bets that microsoft have placed in the in the last couple of years have really begun to pay off because um you know the the the long excel strategy which was painful to watch for so long, has suddenly got to the point where actually most people, you know, when you go to a customer, most people have got Excel 2013.

And also now that people are buying their Office in a different way, so, you know, buying it on a subscription through Office 365 365 which then usually means that you get the the version of office on your desktop updated for you automatically which means you then don't have to wait for that guy somewhere to to upgrade you to office every the next the next version of office every five years that means that the the excel based strategy is paid off. It's all, you know, PowerPivot and PowerQuery are all nicely integrated into Excel, and the Excel world is coming to know it. And then you can take all of the skills you've learned there, and you can use them in Power BI and use that in a self-service scenario. And then you can take exactly the same skills and use them with analysis services tabular on a server. And then you can take exactly the same skills. And now we've got analysis services running in Azure.

You can take exactly the same skills and take them up to Azure. And, you know, in literally 30 seconds, so long as you've got an Azure subscription and a credit card you can spin up an instance of Azure analysis services and it works just like on-premises analysis services and Power BI works really well with it and suddenly just a self-service story and a corporate BI story you've got this this story where the lines have been blurred. You can do self-service BI with Power BI or Excel. You can do corporate BI with analysis services on-premises or on the cloud. And then you can move seamlessly in between them because the same skills, the same technology underpins everything. And I think this is where Microsoft's story at the moment is so strong. And this is where microsoft's story at the moment is so strong and this is where

you know people in the microsoft world are you know really really pleased that um all of these strategies have started have really come together yeah i mean it's it's it's it's it makes you sick doesn't it really it's um certainly uh i think it's uh fantastic i mean i guess um maybe it's a kind of like a an outside threat though i mean a lot of companies I go to now are using Google Docs as their replacement for Office. I mean, do you find that the amount of companies and customers that are using Office itself is going down and therefore is a threat? Or is that more just, you know, is that minor, really? Well, I mean, I see it to a certain extent, but as with the Oracle world, as with the Microsoft world, as with the SAP world, we do tend to exist in our own little bubbles of the Microsoft ecosystem. So I guess I probably work with people, with companies who have bought into the Office 365 world, which then plays nicely with the rest of the Microsoft BI stack.

I'm not sure Microsoft is that worried about the whole Google Docs thing. It is taking a bit of market share, but I don't think it seems as much of a threat as it used to be, you know, again, four or five years ago now, because Microsoft have really got their act together with Office 365. You know, there are always going to be people who hate Microsoft. But I think Microsoft has managed to entrench its position quite well and is in a good position

to kind of ward off the threat of Google Docs. Yeah, exactly. So, you know, we've talked about Microsoft all the time in this, and that was a point of the call. But, I mean, do you ever play – I mean, you must have – you mentioned you read my blog in the past, and I obviously follow yours. I mean, you've seen that I've been playing around with things like kind of Hadoop and a lot of open source technologies.

Have you had a look at those at all? And what's the crisp view on that kind of world, really? I mean, what's your thoughts on that? I mean mean it's something that i'm interested in definitely um and certainly in the microsoft world um there are a lot of these technologies being brought in i mean microsoft is is very big in promoting hadoop and spark and other open source technologies um i suppose it's one of those situations where I don't feel like I've got enough hours in the day to really to get a handle on them. But they are certainly, you know, encroaching on my world and interfacing with what I do.

I suppose with Hadoop, we're kind of coming to the end of a hype cycle where people are feeling a little bit more despondent about it. And having been at the bottom of that trough a couple of years ago, I can I can relate to people that are going through that now. But I suppose the one thing that really sums up my experience with the business intelligence world is that the market is by no means saturated. You know, the market is still growing massively and it still seems like you can go to a customer and they've still got no BI at all and you can show them what's going on. So there's enough room in the market and there are enough different use cases and enough different areas where you can use BI. It's not like, you know, companies use BI for everything they could

possibly use BI for. But the whole kind of analytics and analysis and, you know, data market is just growing and doesn't show any sign of stopping. So, you know, the whole open source Hadoop thing, it's going to be really good for certain types of problems um you know I guess I've kind of defined myself as a as a kind of you know medium-sized data person uh small to medium self-service to kind of medium-sized corporate bi you know Spark and Hadoop and things like that are great for larger data volumes. Maybe those are not going to be the customers that I work with. Yeah, I think certainly, I think, again, looking from a distance,

I think what you've done well is you do one thing well. I mean, you are the kind of the most, to my knowledge, the most well-known and most kind of experienced and so on. A person working with Microsoft OLAP in the past, MDX and so on, I think that particular market will be there for a long time, really. And I think it's, in a way, looking at business models, I think it shows sometimes, you know, do one thing, do it well, get a name for yourself, don't get sidetracked with other things, really.

And you're still there now. And I think you've also been very lucky or fortuitous in backing a kind of company that just it's all gone right for them really in this area and it's kind of well done really for that yeah absolutely it's always it's all luck really in that respect yeah exactly exactly so so just take the crystal ball looking forward i mean where do you think i mean microsoft i suppose from what you're saying in this conversation is microsoft are not scared to take big bets, really. Where do you think this is going to go in the future? What do you think, you know, the future that Microsoft BI would look like in a few years time?

What's coming down the line that you think is interesting, really? Well, obviously, there are a lot of things that I can't talk about. But I think I would say Microsoft is going to consolidate and link up. So we are beginning to see the situation where Power BI is linking up with cloud-based analysis services and the kind of traditional server-side Microsoft BI stack. But Microsoft has got a lot of other BI and data tools out there. There's things like Azure Machine Learning. There is Azure Data Factory, Azure Data Lake, tons and tons of Azure-based BI tools. And they don't quite, you know, they've not all of them reached a certain level of maturity yet. And they don't all work you know, they've not all of them reached a certain level of maturity yet.

And they don't all work with each other as well as they probably should do yet. It's still early days. You wouldn't expect them. But as those things grow, as those things link up, I would say Power BI will be part of the glue that sticks all of these things together. And we end up with this ever growing Microsoft BI ecosystem. And, you know, it's again, it's again, a bubble is again, a certain amount of closed of a closed system that people don't always look outside but that's it's that kind of consolidation and and growing of a larger wider stack that that i think is where microsoft is going in the future so so and i noticed also i mean you didn't used to write books in the past just pick up conferences do you still do that sort of thing as well oh yeah absolutely i i mean i still i still do a lot of speaking at user groups and conferences and that's still a big thing um i'm the question of writing books is is a really

difficult one i'm being hassled by an editor right now um hopefully he's not listening again i feel a bit guilty about this about um writing a kind of second edition of the last book I wrote on Power Query. But what I'm finding with writing books is that, especially with a product like Power BI, writing a book about that is absolutely impossible because it changes so much every month. You know, everything you write is out of date almost immediately. The last book I wrote, which is getting on for three years ago, I had to go through and do the screenshots three times because, you know, do the screenshots as you're writing. And then I realized about halfway through that everything had changed. I had to redo the screenshots. And then I got to the end and realized that the

screenshots had all changed. And in the one month gap between the book being finished and being sent to the printers there was another release of what i was writing about and that put a whole load of stuff out of date and you know certainly now two and a half years later that book is it's almost a museum piece it doesn't really bear much relation to the product now and that that's that's almost a problem with these fast-moving products you know software as a services that you know products that we're we're dealing with so much now in that you can't write a long form piece on them because it goes out of date so quickly. And that means the economics of doing this are completely gone. Because I think anybody that's written a tech book knows that

unless you're writing a tech book on something like Excel, where there is a mass market, there's no money in writing books. You'll earn a certain amount of money, but nowhere near as much as if you'd spent all of those months actually doing consultancy and working with customers. You know, the reason you write books is to it sounds a bit cynical to say this, but the reason you write books is to kind of make a name for yourself technically. And then you make the money from selling consultancy and training and or maybe in a new job as a result but that all depends on people kind of picking up your book reading it and saying yeah this is a good book he knows what he's talking about and you know in the world we're living in now people will buy a book maybe three months after the release they'll try to follow what you've written compared to what's in front of them it'll all be completely different and they'll say well you know this this

book's rubbish it's completely out of date why did i buy it and you don't get that warm fuzzy feeling that then leads you to think ah chris knows what he's talking about i'm going to spend some money with him so it does make you wonder whether the world of tech books is doomed almost well i think it's a definitely good calling card isn't it i think it's something where i mean i've built one i wrote two books in the end but one the one the main one i wrote i think it took me about two years in the end and it was yeah out of date the minute it was published but i think it it's like it's like the ultimate calling card really isn't it and uh i think it's a very good way of kind of getting

your name out there but in terms of a reference piece yeah as you say it's out of date immediately I mean must be the same I mean you do a lot of courses I mean keeping the courses up to date with changes in I suppose the difference between then and now is this kind of like software as a service releases every quarter and so on back in the days we started there's a release every two or three years and now it's every two or three months isn't it yeah I mean keeping the course materials up to date is a bit of a problem and every time i do a power bi course i've got to spend a couple of hours beforehand going through my slides and saying oh that's changed that's changed um but that's probably not quite as difficult as getting a keeping a book up to date because there's so much extra layout and editing involved with doing

that um and it's a shame, because I think there is still inherent value in books and long form writing and value in books that you don't get from blog posts as well. You know, having somebody really sit down and gather their thoughts about how you should use a product and what works and what doesn't. That's not something you get anywhere else. But at the same time, it's almost like newspapers. You know, I still believe that newspapers are a good thing, but the economics of the internet are kind of forcing them to die, whether we like it or not.

And I think the same is probably going to be true of tech books. Yeah, excellent. Well, I guess the lucky thing for you is that you picked a technology and a product in an area of the kind of market that has done well, really. And I think, you know, certainly the fact you're still blogging, the fact you're still kind of consulting and doing, and your name is still there as well, is a testament to sort of what you've done really. So, I mean,

thank you very much for coming on the, on the, on the show. It's been really good to speak to you actually. And I'm conscious that you're away in Europe at the moment on, on a consulting gig. So thanks for sort of, for doing this for us. So yeah, it'd be great to have you on the show and Chris, thank you very much and you know, have a have you on the show. And Chris, thank you very much. And, you know, have a safe rest of the week. And thanks a lot.

Thank you. It's been a pleasure. Thank you. Cheers.