CppCast - Modules

Episode Date: July 21, 2016

Rob and Jason are joined by Gabriel Dos Reis, Principal Software Engineer at Microsoft to discuss C++ Modules. Gabriel Dos Reis is a Principal Software Development Engineer at Microsoft. He is also a researcher and a longtime member of the C++ community. His research interests include programming tools for dependable software. Prior to joining Microsoft, he was Assistant Professor at Texas A&M University. Dr. Dos Reis was a recipient of the 2012 National Science Foundation CAREER award for his research in compilers for dependable computational mathematics and educational activities. News Dan Saks Keynote and more program previews Debugging Tips and Tricks for C++ in Visual Studio C++ Edit and Continue in VS 2015 Update 3 Developer Assistant now supports C++ Red Hat at the ISO C++ Standards Meeting: Parallelism and Concurrency Gabriel Dos Reis Gabriel Dos Reis Links Module TS Draft Modules in VC++ Consuming headers as module interfaces Compiler-neutral Internal Program Representation for C++ Sponsor Incredibuild

This episode of CppCast is sponsored by Incredibuild. You won't believe how fast your build can run until you download your free developer version at incredibuild.com slash cppoffer or just click the link on our website. CppCast is also sponsored by CppCon, the annual week-long face-to-face gathering for the entire C++ community.

Episode 64 of CppCast with guest Gabriel Dos Reis recorded July 20th, 2016. In this episode, we discuss new tools in Visual Studio Update 3. Then we talk to Gabriel Dos Reis from Microsoft. Gabriel tells us about the history and the only podcast for C++ developers by C++ developers. I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? All right, Rob, how about you?

I'm doing pretty good. No real news for me this week. You? Nothing at the moment, no. Okay. Well, at the top of your episode then, I'd like to read a piece of feedback. This one's pretty straightforward. It comes from Matt on Twitter, and he said, great CppCast episode on IncludeOS.

So, yeah, I thought that was a pretty good episode. Right, Jason? Yeah, that was a fun episode. We got pretty deep in the weeds with how IncludeOS works, but hopefully listeners got something out of it, and it seems like it's a pretty cool platform for developing microservices. Oh, and many of our listeners ask for us to go deep. Yes, that is very true.

Well, we'd love to hear your thoughts about the show as well. You can always reach out to us on Facebook, Twitter, or email us at feedback at cppcast.com. And don't forget to leave us reviews on iTunes as well. Joining us today is Gabriel Dos Reis. Gabriel is a principal software development engineer at Microsoft. He is also a researcher and a longtime member of the C++ community. His research interests include programming tools for dependable software. Prior to joining Microsoft, he was assistant professor at Texas A&M University.

Dr. Dos Reyes was a recipient of the 2012 National Science Foundation Career Award for his research in compilers for dependable computational mathematics and educational activities. Gabriel, welcome to the show. Good morning. I'm happy to be here. Thanks for getting me here. You know, that's pretty cool, a National Science Foundation Award. How does that work?

What does that involve? That was past life, but yeah. You know, the National Science Foundation has a program to encourage junior faculty who show certain talents in research and education. And it's a very competitive award. And once you get it, it basically gets you on your path, career path to greatness. Well, it seems to have worked for you then. Yeah, I was fortunate.

Then that was in 2012. Then after that, certain things happened in life, and then I decided to change career again. When did you make the change in careers to go to Microsoft? Oh, that was in 2013, so a year after I got the career award. Life is wonderful with turns and unexpected opportunities. Cool. Well, we have a couple news items to discuss. and unexpected opportunities. Cool.

Well, we have a couple news items to discuss. Gabriel, just feel free to jump in on any of these, and then we'll start talking to you about modules and some of your other achievements in C++. That sounds good. Yeah. So this first one, another announcement from CppCon. They still haven't released the full schedule, I don't believe. No.

I thought that was supposed to be last Sunday, though. Did I get that date wrong? The last tweet or message I saw said, I believe, next week or the end of this week. Well, this one mentions that Dan Sachs is going to be doing one of the keynotes. He's pretty well known in

the embedded C and C++ development community. And his keynote is going to be called Extern C, Talking to C Programmers About C++. That sounds like a pretty interesting keynote. Yes. Yeah, and then they went over a bunch of other new or various talks that are going to be given, some related to embedded programming.

There's also two of your colleagues here mentioned, Gabriel, James McNellis, and Gore Nishinov, both talking about core routines. Yep. As you know, Gore has been taking the lead on giving C++ developers good tools for synchrony. We call this thing

GoRoutines, the core routines being developed by GoR. It's a very efficient mechanism to do synchrony, and I've been quite impressed by what they did. I'm looking forward to

inclusion soon in C++ after 17, I think. Yeah, so right now they're currently at TS, so we're hoping that they'll make it into C++ 20 most likely. Yeah, that's also my hope, because it's really wonderful. I've seen it

in use by developers. It's just amazing what you can do with it. It's kind of funny. It's a concept that's been around for an extremely long time at this point. Oh, yes. Since the 60s or probably before that, you'll have to ask Go for the exact history. But yeah, it has been long for a long period of time. Next, we have a couple of articles from the Visual C++ blog. Visual Studio 2015 Update 3 just came out about a week or two, I believe.

So, yeah, a couple new articles about some of the new features there. This first one is debugging tips and tricks for C++ in Visual Studio. And this one actually goes with a video that you can watch from Andrew Hall going over, I guess, mostly new debugging features for C++ developers in Visual Studio. So if that's your world, it's definitely a good video to watch to learn about some of the new debugging features.

It looked like a bunch of interesting things in that. Yeah, unfortunately, whenever I'm debugging C++ these days, I'm still stuck on 2013, so I can't use some of this. But I need to get switched over soon. And this next one is specifically about edit and continue in Visual Studio 2015 update 3 and some of the things that were fixed there, which is a very powerful feature. So, you know, a funny thing about, to me, edit and continue is I've never used it because my brain fundamentally cannot accept that that would work. Like, I'm like, well, there's this edit and continue thing.

No, clearly that can't actually do what I want it to do. So I'm just going to ignore this feature. That's, I guess, close-minded of me but uh give it a try and let us know okay okay and this last one is uh about developer assistant which basically lets you search stack overflow and other things you would get from doing a Bing search on your code, which is pretty awesome because I know lots of developers spend lots of time looking at Stack Overflow. So being able to do that right from the IDE is pretty cool.

Yeah, this strikes me. Go ahead. Okay, yeah. So there are just so many distinct errors you can make, right? So the odds that you run into something obscure that someone has already hit is quite high. So I think it's a very nice feature that you can combine search, machine learning with other traditional tools. So this is quite interesting. I'm looking forward to more features

as they integrate more capabilities into their product. Yeah, and the other thing to mention is, I just mentioned doing a Stack Overflow search, but you can also look at sample projects by searching on GitHub right from within Visual Studio IDE as well. So you can look at different DirectX sample

apps and stuff like that for code to get started. It's pretty nice. It just makes me wonder how far we are from, you know, you start typing then you let IntelliSense autocomplete the function call, then you let Stack Overflow autocomplete the function call. Then you let Stack Overflow auto-complete the function arguments, and you're just done.

Auto-complete your project? Yes, just an auto-complete project. There you go. I really hope that we're far away from auto-complete everything. I don't want to lose my job to AI. That would not be good. You just have to get more involved in research. Stay one step ahead of the AI. That would not be good. You just have to get more involved in research. Stay one step ahead of the AI.

There you go. Yeah, the AI developers will be the only ones who still have work to do. Okay, and then the last one is another blog post from Red Hat at the ISO C++ meeting, Ulu, giving some more info with their trip report. They specifically talked about coroutines a little bit here as well and some of their suggestions on how to improve coroutines.

Is that right? Yeah. So this is from Talbot. And Talbot has been a proponent of unifying the coroutine design that Goal was proposing and the one that was coming from Chris Holoff. And I think this was the second iteration of his proposal and was discussed in Oro.

I'm hopeful that we can get to a place where we all agree on what co- coroutine facilities should look like in C++, and we can start using very soon. So I'm looking forward to Talbot's third iteration and action from the standards committee to move forward with something that's realistic, proven, practical, and performant. Now, I'm kind of curious, and I know it's not necessarily your specialty, but do you know what kinds of things are changing

or would like to see changed from what Gore has proposed and what Visual Studio currently supports to what maybe will end up in the standard? So essentially, yeah, you're right. This is not my usual arena, but my understanding is you have essentially two sorts of coroutines. The one they call stackless coroutines essentially use a very tiny portion of the calling function

and then stackful coroutines, which tend to be closer to what people call fiber or something much heavier. The problem is people don't want to have very similar, but distinct, no, sorry, very similar feature, but with completely distant syntax and distinct semantics.

They would like to have as uniform and as close and well-integrated specificity as possible. So the real problem is just to find a good way to retain the efficiency of GOR's coroutine and also the capability of Stackful coroutine. So that's where the real myth is. Okay. Okay.

So it's essentially between you have something like just regular function and something that's slightly heavier than a visual function close

to thread or system thread. It's not completely system thread, but it's close enough. So, no, the resource in terms of space that is stacked for coroutine consumes much, much higher than just stackless coroutine. That does sound pretty heavy. Yeah, it's the kind of details that we worry about at the standards committee level.

And one challenge is to ensure that most C++ programmers don't have to know all the gory details because that's not a very useful use of their time. Right. Okay. Well, Gabriel, let's start talking a little bit more about your history. I guess we went over this a little bit in your bio, but do you want to tell us a little bit more about how you went from being we went over this a little bit in your bio, but do you want to tell us a little bit more about how you went from being heavily involved with the

GCC community to coming to Microsoft? Is that a trick question? So, I have a passion for giving developers great tools so they can build awesome applications, infrastructures, and so forth.

I was involved in GCC for more than 15 or 16 years. I was a lead student C++ contributor and maintainer. I think my first contribution to GCC was the implementation of the Valary components, and then I did the rest of the numerics. Then I took interest in the front end, so I became the maintainer of the GCC diagnostic infrastructure. And again, there the idea is to just give developers great tools, and the same was true for the work I did on the G++ front end.

And then later on, I took on trying to bring the GCC development community move from GNU KNRC. When I started working on GCC, think about 96, 97, it was GNU K&R. Wow. That shows my age, right? And then moved the community to using C++ as an implementation language. That took about 10 years, let's put it that way.

First time when I was told, are you insane? To we actually using C++ by 2012, 2013 as implementation language. In academia, I also studied fundamental problems of tools, building

software, that kind of stuff. I have real passion for building tools for developers and joining Microsoft C++ just fits that broader perspective. If you think that the Visual

C++, its customer base is more than half of the entire C++ community. You have the opportunity to reach a lot of people. So it just seemed natural. More than half is more than I would have expected. I don't know the exact numbers, but in terms of proportion,

I can tell you with confidence, and you can check this independently, that if you take the entire C++ community, more than half of the C++ community uses Visual C++. Well, so you mentioned that you've always had a passion for bringing good tools to developers. So as a professor, I'm curious what your field of research was. Oh, so I'm a mathematician by training, but I spent too much time in the computer room and by the time I finished my PhD in mathematics, I was already representing France on the SQL standards committee. So when I moved to Texas A&M

and joined the ANUS team, we were basically three, you know, with Yacoub Yiravi, who you probably know, also working on programming language tools and so forth. We did work on generic programming together. My general area of interest was essentially trustworthy computing. Can you trust your compiler, for example? Can you trust your tool chain? When we see things like, oh, use RAII type of resource management. Why should you follow that? Is it just because BN is saying it?

Or is there a mathematical, fundamental reason that you can trust in addition to, you know, believing in BN? So those were the kind of things I was involved in. And, of course, generate programming, symbolic computation. Like I say, I'm a mathematician. So you probably hear of Maple or Mathematica. Yeah, so they do perform computations without having values for entities. They are manipulating.

And they can do pretty amazing stuff. So that was the type of thing I was doing. And naturally, that meant that I was working on C++ because symbolic computation is one of the origins of generic programming. And if you look at the elements of programming, you know that book by Stepanov and Mac Jones? Yeah. Yeah.

It is quite amazing and remarkable that almost all of the concepts that Alex and Paul talk about in that book have specifications and implementations in the Axiom system or the Open Axiom system I worked on. So Axiom or Open Axiom is just

an algebraic and symbolic computation system. It is free software. If you type openaxiom.org you should reach some page with, you know, I haven't changed for a while since I joined Microsoft. I didn't have a lot of time. But you can download and try it, right?

So the other thing that I did consider was how do you make generate programming succeed? What I mean here is not just that people use templates or some other small functions, object-oriented programming, almost everybody knows that now, right? It's mainstream. That's what the thing that B&A did was to take

object-oriented programming from labs and put it into the streets, into the hands of ordinary programmers. And for generic programming, Alex Stepanov showed us with the STL what was possible. It's really an amazing programming technology. And the idea is how can we improve the language, the C++ language, so that we can make generic programming also mainstream

and put it in the hands of ordinary programmers. That pretty much led me to add stuff to programming, sorry, language features to C++. And in particular, think about concepts. Concepts are so much related to generic programming And in particular, think about concepts. Concepts is so much related to general programming that sometimes we forget to work with template aliases to allow you to parameterize your types, right? Variable templates.

And of course, concepts. So I've been working on concepts for about 15 years. So sometimes I forget to mention it. So yeah, everybody knows about concepts C++ or X and then the

new one we're doing, which is in ETS right now with Andrew Southern. In fact, I cannot think about generic programming, which is actually programming at scale without thinking about modules. They're interrelated.

We'll talk more about that later, but I just wanted to mention that modules and generic programming, they're just different aspects of the same problem, which is how do you program at scale? The C++ community is huge. And when I joined Microsoft, I got the opportunity to get insight into billions of lines of code. Sometimes I'm wondering, how can we manage this system? How can we make it easy for ordinary programmers to thrive through that massive amount of code?

That's what we call programming on scale. It cannot be cleverness, right? It can only be that clever. And if you write your code and you require people or yourself to be as clever as you are while you're writing the code, then you can't scale. It's not because we don't have a lot of clever people, but after a certain amount of cleverness, you can't really go further. So you have to make your programming much simpler, much easier for people to follow.

What else can I say? Oh, yeah. My work on symbolic computation actually opened my eyes to automated deduction. And I realized something quite interesting about the existing template system and the STL that became successful. When you look at templates,

most of the time, you don't have to write an explicit instantiation of a function before you can call a function, right? When you call a copy, a std copy, or a std fill, if an instantiation is needed, the compiler will just take care of it. It's an automated form of deducing when something is needed. I believe that is very fundamental to the success of any real support for generic programming. And that is why the concept system we have today, the one that Andrew Sutton has been working on as a TS,

doesn't have any notion of, oh, I'm going to require you to tell me exactly how this type models that concept before we can use the concept. If you design a functionality for C++ and you make that a central piece of the design, then I suspect you have missed the whole point about programming at scale. You know, again, just look at the STL.

It doesn't require you to go through some kind of elaborated negotiation with central authority to decide who gets right, what, and when. So that's an important point to retain. And lastly, about my research. So, again, as a mathematician, I really feel obligation to provide some kind of credible evidence when we tell people, oh, go and use REI-type techniques to manage resources in C++. Why should you believe that, you know, apart from we're nice? With Xavier Leroy, whom you may probably know as the author of CompCert. CompCert is a C compiler that has been written in a proof assistant, basically meaning that the algorithms are written in the proof assistant and all the properties that the algorithm actually generates the right code have been mechanically

verified, meaning that Xavier wrote the proof and a machine checked that the proof was correct. So you can actually believe that the code that's being generated is conforming to the semantics of the language and the input source code. As long as you believe that the proof assistant itself is correct, doesn't have a bug. Well, yeah, you have that thing. You have to believe something. But if you believe that the proof assistant's bug or it doesn't have a bug

or when it has bugs doesn't affect that part of the software, then you can have 100% confidence in the output of the compiler. That's an amazing feat. For a long period of time, researchers have been trying to accomplish that, and Xavier Le Roy made a breakthrough. So with him and one of his students, Tahina Hermanandro, we looked

at the core C++ semantics, right? You know, constructors, destructors and we built a formal semantics for how constructors-destructors work and the essence of C++.

And we proved that, well, first, the common vendor ABI, you know, the ABI that's used by GCC, Clang, and a bunch of other compilers is actually a good, you know, ABI for C++ semantics. The other thing we proved was that the constructor-destructor meta-protocol that when an object comes to life through constructor and gets destructed after its lifetime and objects are constructed in certain order, destroyed in order,

and you can't destroy, you can't construct a sub-object twice and you can't destroy it twice. We proved that it's actually sound. And because it is sound, the REI technique is also sound. You know, so of course, Bjarne knew that more than 30 years ago, but now you have a mathematical proof

that he was right. So that was the other aspect of my research, right? It's how do you trust your compiler? You have to go through some kind of formalization, mathematics to get there. This is very important when you consider that these days, the airplanes that you fly, they are controlled by a lot of software and a bunch of things are controlled by software.

So you want to do testing to ensure that everything is correct, but you also want to use more mathematical models to ensure that you reduce costs of ensuring, bringing trust to your software. It's an important piece of work. I'm kind of curious, in your efforts to prove these things if you found any aspect of the standard that was incomplete or or buggy compilers or something like that that you were oh yeah so so yeah yeah let's see good question yeah we did find uh some incompleteness uh in the specification in 11 specification, and reported that to

the committee in time. It was fixed in time in SCLESSUS 11 standard. That had to do with the specification of what happens to dynamic type, what we call top ID or co-creature functions in the destructors. The standard said something, but that wasn't actually clear. And if you squint either way, it was not coherent. So it was fixed. The other thing that we found with the idea was that we talk a lot about empty objects.

What happens if you have a array of empty objects? Should that be size zero, one, and how can you lay them out? And we found out you can actually have various optimized, tightly optimized layouts where essentially the array itself, the suboptic array itself doesn't have size most of the

time, and that would be okay. And when it ought to have size because of distant addresses then you can lay it out with some other sub-objects with distant time so it was other possibilities of ABI and we were quite pleased with the result because that was the first time we know anybody did something of that magnitude and while we doing that, we were able to identify other optimized layouts and finding bugs in the standard too.

Cool. Okay. I wanted to interrupt this discussion for just a moment to bring you a word from our sponsors. IncrediBuild dramatically reduces compilation and development times for both small and big companies like EA, Microsoft Game Studios, and NVIDIA. Incredibuild's unique process virtualization technology

will transform your computer network into a virtual supercomputer and let each workstation use hundreds of idle cores across the network. Use Incredibuild to accelerate more than just C++ compilations, speed up your unit tests, run more development cycles, and scale your development to the cloud to unleash Unreal speeds. Join more than 100,000 users to save hundreds of monthly developer hours using existing hardware. Download your free developer version at incredibuild.com slash cppoffer, or just click on the link in our link section.

Maybe we should move over towards modules. Before we talk about the current state of modules, could you maybe go over the history? Because the idea has been around for quite a while, right? You're right. I think people have been talking about C++ modules for as long as C++ existed. And I suspect when Bjarne considered first the topic of module, I must have been in kindergarten or something like that. So it's an old topic.

Yeah, when I was at Texas A&M with Bjarne, we looked at programmable scale because that was the practical research theme that we were looking at. And I mentioned generic programming earlier that it's an aspect of programming at scale but modules were also the obvious one. We were funded by the National Science Foundation to work on

generic programming at scale. So both generic programming and scale. So that's concept, the work that Andrew Sutton did was directly funded by that grant we got from National Science Foundation. And also as Andrew was taking ownership of the concept tiers, my focus has shifted to modules. And when I joined Microsoft, that was just some kind of coincidence that I was shifting and I was focused to modules and then the topic became hot again.

And what I did was to write a design paper for modules and really turning the discussion that was mostly about implementation into a programming language design issue. And coincidentally, it turns out that Bianna and I also work on something completely unrelated at the time, about 10 years ago, what we call the IPR. It is a data structure to represent all C++ programs,

the entire C++, compactly and efficiently. And it turns out that the IPR became that foundational model for the module systems that was put forward. Now, if you look at only what happened in WG21, the WG21 history of modules, of course you have the earlier exploration done by David Van der Voort from EDG in the 2000s. He wrote a series of papers, revisions of one paper, exploring the design space for possible modules. Then Doug Greger from Apple had implementation of Clang module maps,

mostly for Objective-C with syntax and then C and C++ with dot syntax. So we have to use something on the side called module map to describe mostly for Objective-C with syntax and then C and C++ with dot syntax. So you have to use something on the side called module map to describe modular relationships between your source code. After that, Google's involvement came in through Richard Smith and his colleague to make sure that the module map, the Clang module map, also worked with Google's own internal build system. So if you look at Clang, you probably have this thing called implicit modules and explicit modules.

An implicit module was what was working, what Debriger had working for Apple, and explicit module was what came later through Richard Smith because they have different build system requirements. So this is actually an opportunity to highlight something. When you look at modules as just a compiler implementation issue, then it really has to do with build systems, how you organize the build to build machinery around your software. If you look at it from

language design perspective, then the question is, how do you want to affect how people write code? How do you want people affect how people write code, right? How do you want people to write code? And how do you want them to express the architecture of their code? And I think that is the better approach because if you focus only on the compiler aspect, you know, its build system aspect,

you can quickly end up with mess. We have a lot of build systems out there in wide. I'm not talking about internal Microsoft. I'm talking outside. We have a Ninja based system. Google has Bazel. My suspicions is that they probably have more than that. And internal

to Microsoft, I cannot even count how many build systems I have. I have MSBuild, I have NMake, I have Build.exe. It's a lot of stuff going on. It becomes a mess. The way you solve a messy situation

is first trying to find structure. There must be some structure in that chaos. You look at the language aspect, what it is that people should be writing. Once you've done that, you'll identify the right path to the solution. And that is what I did with the design that I put forward to the company the first time. That was in summer 2014.

So that was two years ago at the Rappersfield meeting. And we had almost everybody involved in modules there. Doug Greger was there. The Clank folks, Richard Smith were there. David van der Voort. And we had that whole discussion about where we wanted to go. And the entire committee and community decided, well, we should try the approach I was suggesting, and that's how we started with this whole new wave of modules. It is game-changing, right?

We talk a lot about build improvement, but that's only one aspect. From my perspective, the most fundamental is componentization. The idea that you can tell what are the boundaries of the components in your system. You can't gain out of magnitude in build if you don't have structure to your software. What do I need to compile now again and again? And what does it mean to be compiled? You can't find that without structure. So you need componentization.

Build improvement, virtual improvement is a great thing, right? And if you ask almost every C++ programmer, they will tell you that's top of my list. But it is also a one-time thing, right? Once you've scaled your build, it's done, right? You get that technology done. The thing that will come over and over, and that is opening new possibilities,

is that when you compile modules, for example, you would produce some metadata, which is a description of declarations that are exported and that kind of stuff. Then you start seeing a path toward, let's say, wrong time reflection where you can query what were the classes in my program when I was in an know, all the space disk compiled. How can I call into a foreign library, right? You start seeing possibilities

that are beyond just scaling one time you build. And that is, I want the community not to forget that. It is really fundamental. It is what's going to take C++ from where it is now to places where we haven't seen before. People, if you take other languages like C Sharp

or even Swift, they have these capabilities of reflection, right? We don't have a good way of calling into another component. We don't have a good way of doing DLLs, for example. So modules are not DLLs or

shared library, that kind of stuff. But once you have the right description, runtime description of the entities in your program, you're only one step away from doing things that dynamic languages like Python do.

So, you know, it's an enabling technology. And it's a real opportunity that I don't want the C++ community to forget. So that's why I keep insisting that the module problem is a language design issue. It is not just a compiler development issue. It shouldn't be, oh, let's hack this thing and then document it as module. No. You have to first sit down and say what it is that we want people to write. What is possible and design toward that, making sure that whatever you end up with will also deliver on the build throughputs and other

important problems. So Visual Studio has had some support for modules for your design for a while now, right? That's right, yeah. One of the things that I liked working at Microsoft is once you come up with your design, you're responsible for making it real. So, I was given time to prototype the module design

and that is what we released in Visual Studio Update 1 and also what I demoed at TPPCon last year. And I got support from my management and I got more devs helping me,

in particular John Caves and a new hire working with me on the implementation side. And we also collaborated with folks from the Windows division, teams from Windows division, because you don't just want to build this thing and throw it over the fence and say, hey, try it out and tell me how it works. No, I'm very practically oriented. I want continuous feedback with developers I want to help. And so it has been quite interesting.

Recently, I saw a kind of benchmark. Someone inside the Windows division is developing a new library, and he's generating C++. You know, more than C++, meaning heavy template, that kind of stuff. And I think it is probably what people call header-only library. It's humongous. The generator code is humongous. And he saw at least enough magnitude improvement in compile-time usage

when you use the component through module, you get at least 10x, not 10%, 10x, okay, that magnitude in build throughput. And so compared to traditional solutions like PCH, he also saw that the metadata that we generate when we compile his module is 10 times smaller than producing a

PCH. So those are the kind of early experiments that get me very, very excited. He's been trying it internally and I'm hopeful that if we do it right, we can bring that kind of improvement, 10x improvement to the larger C++ community.

And that's what I'm aiming for. That would be huge. So this kind of goes into the next question I had anyway. To what extent exactly is the Windows team using modules at Microsoft? I mean, I'm using Windows 10 on my machine right now. Are there modules in the operating system? Not the one you're using right now. I suppose you're not using the latest experimental version.

So, no, it's not the entire Windows division, right? Windows is a big, big division, really. Sure. So there is the team that is in charge of the, you know about the Edge, Microsoft Edge browser? Sure. Yeah, okay. So the team in charge of that product has been collaborating with us and is using modules in its fundamental libraries. Okay.

And the next thing is, you know, the next step is, of course, to get the entire Windows division the ability to use modules. And, you know, it's an ongoing collaboration. You need modules, but you also need to take advantage of it through the build system. And so I've been interacting on a daily basis

with architects on the Windows division, both on the module, purely module aspect, but also on the build system aspect. And like I said, i said a lot of build systems in inside microsoft and especially in the windows division so we have been collaborating and making sure that you know things we see actually happen i i i at this point i can't give you prediction prediction of when your next version of Windows will have modules, but I can tell you that there are already elements of modules in some of the components,

and they will come in gradually. It's a huge good base. Well, so then on that topic, what does adding modules to your existing library involve? Or making your existing library module enabled? I don't even know the best way to phrase that. Oh, yeah. So the thing about modules is that sometimes you can consume modules without knowing. It all depends on your compiler, library, and provider. The one thing that I try very hard to do

to accomplish the module design is that you can incrementally consume it, right? You can just add a statement like imports to vector and that if your compiler support it, it will just automatically bring that to you without you having to change anything else. So for example, an include file, an include file that your library vendor gives you

can just contain that statement, right? So if you only have hash include vector, internally, that vector header can just contain import std vector, which will search an internal cache for metadata for vector. So you don't necessarily need to modify your source code as a consumer to be able to consume modules. That's the interesting thing about modules. Now, if you want to provide a module-enabled library, then you need to do more work, right?

Getting the structure of your software surfaced in the software is not just a matter of switching, adding a switch to the compiler. The compiler is just compiling whatever is there. It is about you surfacing the structure, expressing that. So you will have to do certain work. Pavel Curtis, who is the architect on the Windows side, who is one of the guys actually consuming modules in the, let's say, in the Microsoft Edge product. And we wrote a paper about realistic transition path, which is if you're writing new code, you probably want to write code with modules. You start with modules, my modules,

dot something, and then you export the functions that you want to export, the classes, the tabs they want to export. That's for new code. For existing code, you can benefit from the build improvements by using a couple of tricks that I showed in one video on Channel 9. I'll give you the link. You should have the link. It is very simple, you just take existing header and then generate metadata out of it and you can also generate accompanying header file that says, oh here are the macros that were essential for consuming that header file and that's it, right? You adjust your input path and you're done.

The transitional scenario is very simple. I've heard also a lot of insistence on macros, whether you export them from modules or not, what that entails. And that probably is the main point of discussion at this point with the Google folks working on Client. We will have more discussions at some point. I guess we'll reach some form of compromise to allow this kind of scenario to work. But the blanket export of macro and then export network

is not going to give you a good module system. You just get all the problems that you have with existing preprocessor without getting the benefits. Remember, when you export macro, someone is going to get them. So the question is, does that someone get it by default? If yes, what's the point? If that someone has to do something, then we're thinking about changing the source code,

and that's a different kind of game and complexity. On that note, has the Clang or GCC community different kind of game and complexity. On that note, has the Clang or GCC community been working on implementing modules themselves, or is it currently just Visual Studio? So from what I understand, the GCC community, the developer community, and the Google folks working on Clang, we all agree that we want to pursue this module design.

We believe it is important for C++20. So, you know, therefore, there is expectations that at some point they will provide the implementation, the support of modules. I just can't tell you what the schedule is because first, when I joined Microsoft, I was no longer a release manager for GCC. And every vendor has its own priorities.

And I didn't know that modules are very important for Google, for Facebook. They have Express so at committee meetings. And so they will put effort into it. I'm pretty sure of that. I just don't know how that is prioritized with respect to some other important issues they have to deal with. Okay. Okay.

I think that might be all the questions I have. Jason, do you have anything else? No, that's everything I had also. Okay. Well, Gabriel, it's been so great having you on the show and going in deep about modules. Where can people find more information about you online, and where can they maybe get the latest information about updates with modules?

Okay. So if you go to isocpp.org, it will give you the list of all the papers. So the, you know, one thing I forgot to say earlier was that we, after the Jacksonville meeting, we successfully opened a new item for the module TS.

So now we actually have a document, a formal document, you can find on issbp.org. Its title is C++ Extensions for Modules. And now we can provide feedback. That's now a formal design that the entire committee is working on. And if you believe that something needs to change here you have a base document to use to ask you know to make it diff to propose a patch okay yeah so in terms of implementations you

still have the VC blog. That's the place you want to go and get the latest. I'm very hopeful that soon we'll see the

implementations of the TS in Clang, GCC, and other C++ compilers.

And hopefully that will happen before next year. Okay. Well, it's been great having you on again. Thanks so much for your time today, Gabriel. Thank you very much for having me. It's been a pleasure. Thanks for joining us. Okay. Thanks so much for listening as we chat about

C++. I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in, or if you have a suggestion for a topic, I'd love to hear that Thank you.