CppCast - Making Nu Languages

Episode Date: April 9, 2020

Rob and Jason are joined by Sophia Turner. They first discuss updates to {fmt} and SourceTrail. Then they talk to Sophia about some of the languages she's worked on; including Chaiscript with Jason, Typescript at Microsoft and Rust at Mozilla. They then talk about her current project: NuShell. News Stopping A Laser Beam in Mid-Air Stopping a laser beam in mid-air with Tom Scott (BTS) {fmt} 6.2 released Sourcetrail 2020.1 When is *x also &x Links Nushell Typescript Rust Growing TypeScript and Rust Sponsors PVS-Studio. Write #cppcast in the message field on the download page and get one month license Read the article "Zero, one, two, Freddy's coming for you" about a typical pattern of typos related to the usage of numbers 0, 1, 2

Episode 242 of CppCast with guest Jonathan Turner recorded April 8th, 2020. Sponsor of this episode of CppCast is the PVS Studio team. The team promotes regular usage of static code analysis and the PVS Studio static analysis tool. In this episode, we discuss updates to source, trail, and format. Then we talk to Jonathan Turner. Jonathan talks to us about some of the programming languages he's worked on and his new project, New Shell. Welcome to episode 242 of CppCast, the first podcast for C++ developers by C++ developers. I'm your host, Rob Irving. Joining me is my co-host, Jason Turner.

Jason, how are you doing today? I'm all right, Rob. How are you doing? I'm doing okay. Should we briefly talk about how your stream went last week? Sure, why not? You mentioned doing the Doom, converting it from C++. Really, the goal is really just to get it to compile with a C++ compiler.

And I was streaming for 13 hours total. I took about 2 hours and 45 minutes and breaks during that. And, uh, then I came back and at 11 PM, my time, I just said, I'm done. I, and I felt like Forrest Gump after he ran across the country 14 times and he goes, I'm going home now. That's just what I was like. I was just done. So I shut it down and then i came back to it on monday morning and i only had 20 minutes left of work to finish the linking to get it running wow were you able to to play it after all that yeah yep it played it was mostly just type conversions just convincing the compiler that, uh, sure. Retreat that void pointer as a whatever, you know?

Right. Cause C, C is like, whatever, do whatever you want to do. I don't care. They're all bits to me. C plus plus is not like that.

Right. Right. That's very cool. Okay. Well, at the top of our episode, I'd like to read a piece of feedback. Uh,

this week we got a message from Claire, uh, and she's suggesting a guest and actually she suggested our guest from last week danielle engert so thank you for that suggestion claire but this one is uh she actually put a link to a different youtube video and i don't know if you watch this jason but the video is from tom scott and it's stopping a laser beam in midair and i watched this it's pretty cool uh and basically midair. And I watched this. It's pretty cool. And basically, Tom Scott is the one doing all the talking, but the person who is controlling the laser wrote some software in C++

that's controlling the laser and making it so that with the way video cameras work, digital video cameras, he described the technique. I can't remember what it's called but uh by turning the camera on its side and controlling the laser like quickly starting and stopping it you could make different effects happen like having a laser beam appear to be stopped midair and he does like the kylo ren force stopping a laser like he did in the uh force awakens movie i think you know i was just gonna say i don't see the point in the tricks i could do that with my mind

well it's pretty cool and yeah so there's a second video which is just from the guy uh seb lee who wrote the software to control the laser and uh she's suggesting we have him on the show since that library is all in C++. So maybe we'll reach out to him. Yeah, it could be cool to talk about lasers. Okay, joining us today is Jonathan Turner. Jonathan is a passionate open-source advocate and programming language nerd. He joined Microsoft in 2012 to work on a secret project

that will become TypeScript. He stayed on at Microsoft for the next three and a half years, helping grow TypeScript and helping Microsoft Shift to become more open source friendly. Later, he joined Mozilla, helping to grow projects like Rust, Servo, and WebAssembly. He also worked on LVM, Clang, and Apple, Chapel at Cray, and numerous side projects, including ChaiScript with Jason. In his free time, he's currently working on a new project called New Shell. Jonathan, welcome to the show. Hey, awesome to be here. I think in full disclosure, this is where we. Hey, awesome to be here. I think in

full disclosure, this is where we mentioned that yes, we are in fact related. We don't just share. We are related. It's true. It's true. Yeah. I've known Jason literally my whole life. No, that's not true. I've known you literally my whole life. I'm six months older. Yeah, yeah. But we did spend quite a bit of time when we were young together. Yes, lots of programming and gaming when we had the chances to. So what was the first thing the two of you guys did together growing up involving programming? Well, I can remember two things. One of them, we had a game that we were working on.

So Flashback, you know, put the waves in, right? Flashback, back then, Ultima was like the hottest role-playing game. And everyone that I knew in high school was trying to make an Ultima clone. So, of course, I was trying to make an Ultima clone as a budding programmer. And Talk Jason into helping me with it. We actually dug up a compiled copy of this. I don't know if we want to put it online though i do have it yeah we both have a copy yeah i think that is like one of the first projects that we collaborated on but we've i think it must be yeah we wrote an operating system in high school together that we put in the

science fair and then got like first place which is pretty awesome that's pretty cool it's very simple but yeah yeah that was fun yeah and then flash forward we did chai script um in 2008 nine i don't know jason help me out i think nine might have been when we tagged yeah i think i think 2009 was the first commits, yeah. Which were on Google Code. Yeah, whatever that thing was back in the day. One of the canceled Google projects. Right, right. Very cool. Okay, well, Jonathan, we've got a couple news articles to discuss.

Feel free to comment on any of these, and then we'll start talking more about some of the other programming languages we just mentioned that you've worked on, okay? Sure. Okay, so this first one is uh format 6.2 was released uh what are some of the highlights here jason and i i gotta ask like what is format still have enough in it that it's worth using net compared to what's going to be in c++ 20 um i think once your vendor supports it in c++ 20 you should probably just use what's officially there. But I don't know. It's going to depend a lot on quality of implementation

or whether or not they grab his implementation and such. But the top change here, you get better compile time errors instead of 17 pages of template errors. If you try to output something that doesn't already have enough formatter that exists for it is huge yeah definitely no i am when i saw this i pinged jason i said you're trying to set me up right it's like what do you mean so there's a there's a very similar thing in rust called fmt or format and it looks exactly the same and uh but we learned very quickly that they're both inspired from python

which is kind of a fun fact and that one is also in the std namespace that looked like in rust or something yeah it's it's part of the standard library in rust so they'll end up like looking yeah very similar i guess that's good cross-pollination is a good thing yeah so yeah if you didn't catch that and i know you don't necessarily follow every bit of the c++ standard is this has been approved uh for c++ 20 but not the full suite of utilities mostly the format to a string portion of it has been approved but not any of it's like direct printf replacement kind of options. Okay, next article we have is SourceTrail 2020.1 new version. And the new thing, our main new thing here is keyboard based navigation. So if you are not a heavy mouse user, if you try to do most things with the keyboard, you can now navigate

around the app, which I'm sure some of our listeners will find handy. I think to me, the more significant thing is that it's the first release since the project became open source. Very true. Very true. And I should we should mention again that they are looking for patrons to help support the project now that it's open source. They're certainly doing better than they were on the Patreon since uh we first mentioned that uh you know maybe a month or two ago whenever that was yeah they only had like ten dollars in support then yeah they're still not at their goal though they're like halfway they're at a 230 of 550 bucks is their goal so

definitely try out source trail and if you like it you should definitely consider becoming a patron they also have a few dozen committers now oh nice that's very good i'm kind of curious if john has any comment on this not source trail but since you've been involved in a lot of projects that or tried to push open sourcing of projects and that kind of thing just i don't know any general thoughts from what you've seen here um i don't know in particular about source trail but something i was just thinking about is um kind of flash forward a bit to when we're talking about typescript when typescript first was open source it was on the codeplex which was microsoft's uh proprietary open source thing it felt like every company had one of those at one point.

I think CodePlex actually still exists, though, doesn't it? I think it does. I was told at one point it was like a machine under someone's desk. I don't think that's true, but that's kind of what it sounded like. And when we moved TypeScript off of CodePlex onto GitHub, I think overnight I had 10 times the number of viewers of the project and then it just kind of skyrocketed from there.

So just getting into Vue and getting into Vue in a really open source way, it makes a humongous difference. Yeah, cool. Okay, and then this last thing we have is a post from Arthur O'Dwyer's blog and it's when is pointer x also reference x and i thought this was a pretty good post uh kind of explaining some of the differences and his way

of remembering the differences although i did want to get your take on his one suggestion here jason where he says to teach pointers first. No, I don't teach pointers first, but I also try really hard to not teach intro to programming classes. I know that that's not my forte. So, uh, I've, I've, I've been surprised a couple of times when I was teaching introductory classes that I didn't expect to be introductory classes. And, um, you know, at the first break i go okay i need to rearrange what material i'm teaching now it's um it's not a surprise that i prefer to have um yeah i think the sad thing for me though with this article is i saw the headline and i said with function pointers. And then I said, I've spent too much time with this language. Yeah.

Anything else you want to comment on with this one? It's fun. Yeah. There's some good lightning talks that are kind of similar about some of these things you can do with function pointers too. I can't remember the titles now. Okay. All right. Well, John, so we talked a little bit about in your bio just all the different languages you've worked on do you want to tell us maybe what you're working on currently to get

started sure um my current project and if you get me talking about it it will be the whole podcast so make sure we're time limited um yeah i'm really passionate about this project it doesn't sound like it would be something that would hook you as deeply as it's hooked me. It's a project called New Shell. Well, that's the current name. We may come up with a better name, but let's go with New Shell for now. And New Shell is a... Okay, what is the elevator pitch? So we've got PowerShell, which is like an object-oriented way of thinking about shell. So it's not just text in text out, like standard in standard out is only text, you actually have this ability to work in objects. Flash forward, you know, I was talking to some friends last year, and we were thinking about what happens if you have

structured data passed through this pipeline, not objects, not state, not mute, mutability, but actual like structure that you can destructure it on the other side, you can work with it as if it never lost the structure that was in the original file. And so that is basically was like the initial idea. And then from there, it's kind of grown out to being how can we think about programming in a bit of a broader way and something that's a little bit more beginner friendly, but has all the capability of being a shell has all the capability of working with data. So yeah, I'm sure we can kind of drill into that more. But that is something that I would I'm really loving working on. And that's open source.

That's open source. Yeah, you can find that on GitHub, New Shell slash New Shell or something. Okay. Well, maybe instead of going too deep on that right now, we should start with what first got you interested in programming languages because you worked on so many. Yeah. So this was kind of a surprise to me, too. So back in 2005, 2006, I was working with a mutual friend of me and jason that we went to college with a guy named monosporny and um we had a contract we were working on together

and my coding in that contract i will just fess in this podcast it was not the best i kind of cut a few corners. And it kind of came back and bit me. And I realized like, there was so much more about programming that I wanted to learn, so that I was always ready. Like I always could go deep and have like really strong solutions to problems. And that's really was the impetus to act to study programming languages and to study how do I work in these languages in a way that is actually really smart. Jason, to his credit, was by that point, I think, was really starting to dig deep into C++

and was starting to pick up how you do really strong C++. So he was a good tailwind, I would say, to push me deeper into this pursuit of how do you think about programming? How do you do really good programming so that the code that you read is very readable, it's very crisp and clean. And then I started thinking about, all right, if you could remove all constraints, as an engineer, and just build a good programming language. Well, of course, you know, that's a bit of the audacity of that is pretty obvious, right? But it's a good way to think about, think about your problem in a different way. So I started making these little programming languages. I made one that looked like English called Pineapple, the pseudo natural programming

language or something. It was a terrible, absolutely terrible, but it was a good start to have something where you're working on a parser and then it does a thing. And then you're like, oh my gosh, look, look, it does a thing. And then you write a little bit more and then it falls over and like, huh, okay, I need to think a little bit deeper about how this is built um so yeah i worked on pineapple i worked on a couple other languages uh i think a couple it was like a year or two later jason had this idea for um working on an engine it's like we can make so i think at the time you're working on, what was that language binding layer into C++? DispatchKit? Maybe.

No. Oh, Swig, working with Swig. Swig, yeah, yeah, yeah. There you go. Oh, yeah, and I hated that. That's right. That's right. So he's working with Swig.

Maybe I'll have Jason tell that part. But basically, he saw Swig and that it could bind to basically any language in. But what would it be like to have a language that sat naturally inside of C++ that could use all the types for C++, use the way the dispatch works and everything? And so he started working on an engine. And at that point, I was two years in to designing programming languages. I was like, hey, I'll write the scripting side of that for you. He's like, done, go for it, write the parser. So that was the first early versions of ChaiScript

were born from those early experiments. Yeah, without someone who knew how to write a parser, that never would have happened. It would have just been an interesting library. Yeah, cool. So how did you go from kind of building your own language open source project to going to Microsoft to work on TypeScript? Yeah, so I did. So I kept doing these programming languages. I worked on ChaiScript with Jason. And at that point, I also started working on one called Minnow, which I was trying to remember the name of it just now.

I'm like, it was a fish. What the heck was it? For anyone watching the video, that was the look in my face. I was trying to remember what the heck that was. So this is a predecessor to Go, but a similar idea. That everything was an active object or an actor. And you could send messages back and forth to these things. And they're all sat on lightweight threads.

I was going to say, you should probably describe what active object is for the sake of our listeners. Oh, yeah, yeah, yeah. Let me do that. Let me do that. So let me back up. So an active object or an actor is a... So you've got a normal object.

And the only way to interact with it is to call a method directly on it, right? So you call the method directly. That means you've got a pointer to it or reference, and then you can, you can get to it. What an active object would be, would be imagine an object that's sitting on its own thread somewhere. And the way that you communicate with it, you've got a channel that you can send a message to it. So it's sitting there in its own thread, waiting for messages on that channel, it gets a message, it does something, it probably like mutates itself in some way. And then the only way it interacts back with you is through that channel or through a separate channel. So instead of everything being asynchronous by

default, now all of a sudden, everything's asynchronous by default. You have kind of a mesh of these messages being sent back and forth between these actors. So the model is interesting in that you can model, as we found with Go, you can model services that way. As these messages come in, you can kind of field them and spawn off some workers and they can do something interesting. So, so yeah, that was, that was when I worked on for a while. I got it really fast for one benchmark and I was really proud of myself for that one benchmark of, of speed. You drove yourself nuts with a lockless cues at that point, right? Yes, yes I did. Um, yeah, if anyone is listening and knows what the ABA problem is, that's what I hit, which is like, I don't know if we really want to dig into it right now, but maybe, okay, here's the TLDR of ABA problems.

So you have like a message come in, let's say that's A, and then you delete that. And then that goes back into a memory pool that you can reclaim later. And then so something else comes in is B. But before your lock, your lockless compare and swap happens, you've deleted that and replaced it with the first one again. So your compare and swap succeeds when it should not have because you're on recycled memory. So that is a gotcha that you have to watch out for in lock-free data structures. And I had no idea what that was until grad school.

And I was like, oh, that's why it's hard. I got it. Tony Van Eerd has been doing a series of talks at C++ Now called a lock-free queue part whatever, one of N. And he does another one every year at C++ Now. I don't know, he's on like part four or one of N. And he does another one every year at C++ Now. I don't know, he's on like part four or five of N. And the conclusion is always don't do this. Don't do it.

Yeah, don't do it. I switched the lock-free thing out. I don't know if I told you that, Jason. Before I was applying to grad school, I did switch it to a locked version, and it was just as fast and it was, you know, correct, which is a good thing to have fast and correct.

Um, yeah. So I worked on minnow and I worked, I loved it so much that I actually quit my job so I could work on it full time. And after about a year and a half of that, I said, ah,

I should go to grad school. Like this is, this is, this passion will not leave me alone. And if I go to grad school, I at least have a piece of paper to show for it. So that's what happened. So I went to grad school in Boulder, Colorado. So I was living right beside Jason, which was awesome. We got to hang out a lot and work on on projects and i did that for a few years and worked at apple on lvm got to work with chris latner which is amazing and doug gregor who's

also totally amazing and we worked on c++ kind of a module system for c++ what they didn't tell me that uh when i interviewed at apple was that in this interview, they're like, what would you say you would do to make your perfect programming language? And I described this thing that had a trait system or like concepts back then. So it has a concept system. It has better modularity. So the modules are nice and clean. And they had to have been laughing on the other side of that phone when they were interviewing me, because this was before

Swift now. They were working on it in secret. And I just described what Swift was to them over the phone. So yeah, I got the internship and then went and worked with them, which was an amazing experience. They're just brilliant off the charts. I think related to that and maybe worth the aside is, do you want to talk about what your professor that you were working under was doing? Yeah, yeah. So my professor then was Jeremy Seek. He was my PhD advisor, though I did not get the PhD. No spoilers, but yeah, I did not get the PhD. No spoilers, but yeah, I did not get the PhD. But

he had been working on a few different projects. One of them was concepts in C++. So he and Doug Rager worked together on the original concepts that did not make it in, but I think kind of laid some of the early thinking and groundwork that would be the revised, clean, simplified version later on. So they worked on that really hard. Jeremy, one of his main things in grad school was around concepts and how do you deal with the complexity, the subtle complexity. It's just, it's off the charts. I remember reading some of his papers and just like, wow, this is amazing. So yeah, I worked with Jeremy. Jeremy's other project, his papers and just like, wow, this is amazing. So yeah, I worked with Jeremy. Jeremy's other project, which kind of leads us into Microsoft, was he was looking into gradual typing system. And at the time, it was like gradual typing Python. So

normally Python is dynamically typed. It has no types in the code. And what happens if you actually sprinkle some type information where, you know, you want a little bit of checks, but not a whole bunch of checks, you can just sprinkle a little bit in. So he was experimenting with that. And lo and behold, Microsoft also was experimenting with that. So my the second internship I got, or the I guess, the third in grad school, after Apple was Microsoft. And that's when I started working on the TypeScript project. So I guess we just skipped right over Chapel at that point. Yeah, sorry.

That's fine. I just realized I don't really know anything at all about that language. It's really, really cool in that it's meant to be a systems language, but it's also very parallel by default. So it has a, I guess maybe it's claimed a fame instead of having an index. So, all right, here you go. So you index into a vector and C++ is just a number and you go right, you calculate right to that place in memory. Okay. That's cool. But what if your memory space is stretched across many, many, many machines? Do you still

use just a number? Well, the trick that they figured out is if you have just a way to calculate the index from like a simple function, you could kind of project that code onto different machines data sets by just like rewriting the index that it's looking in. So, you know, still imagine that your, your memory addresses are contiguous, but the addresses run across many, many, many machines. Now you can just lay the data across all these machines and the indexer would say, um, uh, they have this sense called this idea called like a distribution so the distribution would lay across all these machines and the way that you would index into them it would say it would kind of be like um i can't remember the exact algebra but it'd be like machine number plus offset in that

machine would be the actual the actual index right so you could just have a function that would create these for you and now you've got indices that will jump you into whole blocks of data across these many machines. So I may not have done the best explanation of that because it's a bit rusty. But maybe you can get a sense for working at really large scale distributed systems. And the code that you could write across this just treats the whole thing like one machine, right? So you've got X number of processors and X number of, you know, X amount of memory. And all of that is handled by the compiler. This makes me think I, you know, C++ Twitter, I'll have every now and then see comments from people that are like, Oh, do we

really care that the size of a vector is 64-bit? No one actually has more than 4 billion objects in a vector, right? Yes. Yes, people do put more than 4 billion objects in a vector. Exactly. Go model the weather. Right. Yeah.

So that was kind of cool. Yeah, Chapel, most of the stuff that i worked on in chapel i did implement concepts in chapel and i helped them with the module system and i also i had an internship at cray and there i worked on like lambdas and passing around first um functions first class first class Passing functions to whatever. Whatever it is. Passing functions to first class functions. Is that right?

Passing class functions. You could do that. Sure, why not? Sure. But being able to pass functions into functions into functions and whatever, so that it could be arbitrary. Right.

We kind of glossed over your modules that you mentioned when you were interning at Apple. Do you know if that ended up becoming Clang's implementation of modules? I was like racking my brain to remember the design that we had at Apple. I remember some of the stuff they were trying to tackle was around, so C++ has carried along with it c's macro

system right and the macros have this really nasty name polluting uh side effect it's not pretty and so if you think about modules as having to be separate compilation units having the macro system muck with that really breaks that breaks that so we had some rules that we built around like macros that were allowed to exit and then as they exited what they were allowed to do to the namespace as they exited but beyond that i cannot i can't remember the details yeah it all sounds like stuff that they continued to argue with for the next what was that probably eight nine years ago you were working there yeah 2011 yeah okay that they continued to argue about for the next eight years until we got a standard for c++ yeah my um i i designed this other thing at apple too that um we that we'll talk about Rust here shortly.

And it was kind of a stroke of luck that I ended up designing something very similar. So I had thought, you know, when you compile C++, you compile, you can get some object file, but then you have to lug around this header at the same time. And the header allows you to stamp out your templates and actually get specialized functions for you know good optimization i was like you need to be able to bundle this together maybe we call that whole thing a module and uh sure enough that's how rust does it so that you can you can still specialize and get all your generic functions get specialized on the type and get,

you know, highly performant. But in order to do that, they have to lug around the metadata and the kind of the pre-processed source code. Yeah. That's our guest last week, basically described C++ modules as containing a form of pre-compiled header for that kind of use case as well.

So we're getting there. Yeah. Yeah. Once you have it there. Yeah. Yeah. Once you have it, it's, it's like, it's great. We love it in rest.

I want to interrupt the discussion for just a moment to bring you a word from our sponsor. This episode of CPP cast is sponsored by the PVS studio company. The company produces the same name PVS studio static code analyzer. That's proved to be great at the search for errors and especially typos. PbS Studio company. The company produces the same name PbS Studio static code analyzer that has proved to be great at the search for errors and especially typos. The tool supports the analysis of C, C++, C sharp, and Java code. The article 012 Freddy's coming for you, which has been recently posted, clearly demonstrates the analyzer's outstanding abilities of searching typos.

The article shows how easy it is to make a mistake even in simple code and that no one is immune from making it. You can find a link to the article in the podcast description. There's also a link to the PVS Studio download page. When requesting a license, write the hashtag CppCast and you'll receive a trial license for a full month instead of one week. I guess before we talk too much more about Rust, could you maybe go back to Microsoft and TypeScript a bit? Because I think you kind of hinted at sprinkling types onto a language. I'm not sure how many of our listeners are familiar with TypeScript, though. Sure. So I guess maybe the elevator pitch for TypeScript

is imagine you've got JavaScript, and this is great for making what they say, like the monkey dance on the screen. You put a couple lines together. It animates. Everyone's happy. And that was kind of JavaScript's original use case. But you flash 20 years into the future, and now you're having to write full applications.

Real programs. The only way to write it. Yeah, you've got like tens of thousands of lines, hundreds of thousands of lines of JavaScript, which, yeah, that is a recipe for disaster if you don't have some kind of type system, some kind of static analysis to help you out. So that's really where TypeScript, you know, kind of came into being.

We are working with, so this is developer division at Microsoft who is working with people like Office is developer division at microsoft was working with people like office or making these gigantic web apps and they needed something that um that they could use that was more performant and whatever and the most performant thing that you could use is just javascript so that kind of kicked off this idea of you take javascript and then lay on top of it an optional layer of types and these types just peel away and you don't get like really gross javascript as a result you should get javascript that's like you would write by hand so that's really where it kind of this this idea of

types just for the sake of tooling and for kind of static analysis, not for necessarily optimization, not for, you know, some of the other stuff that you would use types for in a compiler. Basically only just the tooling side. So we did, they had an early implementation of it before I started. So it wasn't even public. I kind of came in and helped them with some of the last, you know, bits and bobs that they needed to fix in the design and then helped them take it public. And that was it's an experience that, you know, I think I will always remember trying to talk to JavaScript developers about, hey, here's this type thing. And they just kind of stare at you blankly. Why would you do that?

So then I open my editor and then I start getting auto-completion. And they're like, hold on, what was that? Go back, what was that? And then I get squiggles when I mistype something in the JavaScript standard library. You get a red squiggle and it's like, because I mistyped it or I passed it the wrong type.

And so little by little, we started getting a small following. And at the time, we had no idea if the project really had legs. We knew we had a couple of teams interested, but that was it. So we kind of took it public. And then I spent the next three, three and a half years, you may have heard of some of these in your your work so these are like really complex machines right and uh i wanted to see if we could get some of them using typescript because if you did now anyone using angular or react would have a bunch of type information.

They don't have to use it, but it's optionally there that they could pull it in, then their editor could light up, and you could get all these auto-completion and errors and whatnot. So that was one of the last things that I did on the TypeScript team, was I got the Angular project moved on to TypeScript, and then started the Ember project moving as well. And so, yeah, that was really awesome because after I left, I kind of said, all right, here, replacement, good job. If you have any questions, you can always ask me.

And then I just went away and then just got to watch TypeScript just take off, which was an amazing, amazing experience. I was definitely waving my pom-poms from the sidelines. I was going to say, it definitely has taken off over the last few years. Isn't Google using TypeScript now? I think anyone who's doing a serious JavaScript project does, honestly. I'm curious what that first editor was that you were showing it off in. Was that Visual Studio, VS Code? Do you remember?

I think the very, very first one was Visual Studio. Because VS Code hadn't even been created yet i couldn't remember yeah it was a project called monaco which was an online editor that used the same logic incidentally also written in typescript they they're written in the early version and had to keep rewriting because we kept breaking them. Props to them for continuing with it because it turned out really good for them later on. So yeah, so when VS Code actually came out, then yeah, we kind of added support. And then that's basically these days,

that's what you would use would be VS Code. You know, I know Matt still listens to the podcast and he might correct me here, but I'm pretty sure that Compiler Explorer's code editor is Monaco. That sounds right. I think that's right. Yeah, that's awesome. You can right click and get familiar looking menus. Okay. Should we talk about Rust and your involvement in that?

Sure. So after three and a half years of working on typescript i kind of hit a limit where i'm like i'm i'm not going to work on programming languages anymore i'm done and to rust yeah so i took a vacation i could clear the cobwebs out i i live in new zealand now and part of the reason is because i took a vacation and came through new zealand on that vacation um so you know you're supposed to get a visa you're not allowed to just stay right just stay i'm never leaving i'm gonna be a hobbit too so just to be clear he is living there legally i'm just giving him a hard time now immigration is going to come looking for me, Jason. Right.

Yeah, so I took this vacation. And at the same time, so I mentioned Ember earlier. Someone that I met, oh, while I was working on TypeScript, I was also working on the JavaScript design committee called ECMAScript. So this is like the standard JavaScript body. And then once that's published, the browsers take it and turn it into JavaScript. Okay. One of the people I met on there is this guy named Yehuda Katz.

He worked on like Rails, Ruby on Rails 3. He's like a, in that area, he's like a really well-known developer. But I had only just met him through the committee. And he and I kind of stayed friends. So we just kind of hit it off. And after I left TypeScript, he kept banging on about this Rust project. Now, I had seen it before, and it looked, I'll admit, it did not look very good. I was like, it has some good ideas, y'all, but I see a bunch of ampersands.

I see a bunch of sigils. This is just, this is not pretty. I'm just, I'll stick with C++ or whatever if I'm doing systems levels. And he just like kept saying, you know, you should try it, you should try it, you should try it. So I finally said, okay, fine. So I did two projects with Rust.

I worked on a ChaiScript-like thing called Ray. R-H-A-I. I was wondering how you pronounced it. Yeah, right. You take the C and turn it into an R because it's not C++. It's Rust. Right. So I did that to try to understand how parsing would work and the type system

works in Rust. And then the other thing I did, which Jason loves to chat about sometimes, is I like to write nintendo emulators that's how i learned programming languages hey now to be fair this is awesome go ahead since you brought it up go ahead and list all the languages that you have written in an nes emulator in c c++ c sharp java maybe java javascript typescript rust wow okay it's a way to C, C++, C Sharp, Java, maybe Java, JavaScript, TypeScript, Rust. Wow. Okay. That's a way to learn.

I mean, if you use the same docs, you start figuring out the patterns, and then you just have to figure out the language after that. But the nice thing about having a project like an emulator is that for me, there's a lot of nostalgia. The Nintendo was my first console that I really loved as a kid. So you have a lot of nostalgia the nintendo is my first console that i really loved as a kid so you have a lot of a lot of joy just getting oh super mario works holy crap that's amazing like that joy doesn't go away at the 10th time that it happens you're still like yes super mario and that's the easiest one to get started with right no no it's actually really hard oh i

thought super mario brothers one was uh okay never mind no yeah it's not the one you should start with if you're listening to this and want to do a nintendo emulator pick a different game pick a different one to start with do you have to give a recommendation now you can't just say that i know i know um any i'll think of one we'll put it in the show notes but the reason i'll just tell you why a super mario is a little bit hard you have to oh gosh this is not a nintendo emulator podcast fine go for it it's 20 seconds yeah it's fine so when you're looking at the play screen there's that little coin at the top where it says how many coins you have. Underneath that coin, there's a little half sprite.

And that is how it knows how to switch from the score display to the play display. And so all your timing has to be correct. And then it trips across the bottom of this coin. And that has to be correct. And then the play screen will draw correctly. So you have to have a fair amount of the Nintendo emulating well before that works. Yeah.

Definitely try not to do that one first, but I, I, yeah, I, I still, yeah, I like the challenge.

So yeah, I did the Nintendo emulator. I did this other Ray project and then just fortuitously, I don't remember if it was yuhuda if i was following the rust twitter account they said oh we're looking for a new developer on the rust team at mozilla if you're interested you know come hit us up and i said well you know i've got all this i worked at microsoft and typescript i'm doing all this rust stuff right now. Maybe, I don't know, maybe we can work something out. And so that led to a bunch of interviews and I ended up at Mozilla working on the Rust project. And that was awesome. I mean, it's just a passionate group of folks.

I had met some of them a couple of years earlier when I was at Microsoftrosoft i put on this thing called ling.next l-a-n-g.next okay and it's um it's like a like how many language developers can we get in one room like we had bjorn astrustrup there we had uh um what's the guy that did the c++ template metaprogramming book um uh andre alexandrescu yeah we had we had Alexandrescu talking about D we had a bunch of people which was an amazing group to have incidentally, I'll tell this one quick Brianna story

so we had him at the speaker dinner and I was sitting beside Nico who was the lead of the ROS compiler team and Brianna was sitting across the table and we're just like chatting him up and once he realizes that you know we actually like c++ we're not just here to like dog on c++ we started having like this really awesome conversation about what it was like to

design c++ over the years and so much so he said he said this um this little nugget that to him was probably just this throwaway set of sentences. But he said, you know, you realize that when you first design something, at first people resist it because they think it's too subtle and they make you make a bunch of syntax just to make it obvious what's happening. And then, you know, they start acclimating to it. So there's kind of like a middle period where they acclimate. And then by the end, they're complaining that there's so much ceremony and syntax why wasn't it simpler and so this this kind of became strewstrop's rule and this is something that in the rust design team they still use this to this day this idea that you know you have to be

careful how much ceremony you're putting on new designs because people have to acclimate and you have to give people time to acclimate to a design so yeah that was cool um so yeah i had met some of the rest folks but then actually getting to sit and work with them day in day out i mean it's just a bright team of like really good-hearted people which is it's awesome it was an awesome experience um I kind of helped them at that point. I don't want to take too much credit, but it was at that time, it was a research language becoming a full blown industrial language. And so I kind of came in with my two cents and like helped give them a push in that direction. One of the things that I worked on was the usability of the language.

So specifically around the first thing I did was error messages. We worked on a kind of a cleaner way to print out an error so that you can actually see your code. Not a lot of error messages, just most of your code. And then we would just underline under like hotspots, interesting areas in your code that, okay, this is where the error is. And you give a little label beside that. Okay. This is why it happened. And then also some secondary, okay, here's a secondary line. And what, all right, here's a definition that came from here. So you look at the code and in like two seconds, you see the big red squiggle, the message, the line number, and also maybe some secondary information. And when people tried this out, it was surprisingly positive, which to me, for such a big change, was really great feedback.

It's like, yes, yes, we're going the right direction. And then, yeah, now it's kind of a thing that Rust is known for is having an error message that just jumps you straight into the code that you wrote and the order that you wrote it, and labeling so that you can jump right back to your code and know how to fix it as quickly as can be expected in a compiler. It's not zero seconds, but there's definitely a speedup from where it was before, which is really great. My memory of these events is that just a few months after that But there's definitely a speed up from where it was before, which is really great. it anyhow is starting to get to the kinds of errors that you're talking about with we were

discussing even gcc 10 last week right rob yeah with like they're putting in hyperlinks if your terminal supports it hyperlink so you can click on the error and it'll take you to the documentation i love it i love it yeah i'm i um i knew it was kind of like one step in the right direction and then once people took that they would just kind of keep building and evolving it. So I love it. I think it's great. The other thing that I worked on as part of this usability effort

was to build their first language server. So people may not know that when you're working at IDE, you have basically a live compiler in the background all the time. That's how I can do the error messages, the squiggles. That's how I can do autocomplete and whatnot. And the way to do this is actually completely different than you would write a normal compiler. So in a normal compiler, you know, you might hit build and now you've got to wait seconds, minutes until it's

complete, right? You're not going to wait seconds. If I hit dot, I want you to give me the answer. Like, I'm not going to wait 30 seconds to get the answer when I'm in an editing situation. create the data structures in memory that you can update live and would hold pretty much everything in the memory state in the in the state of the program but then as you're editing would just update the little bits and pieces that need to be updated recompute that and then give you the type information or the autocomplete so i worked on stuff like that for rust, it was kind of a, it wasn't the greatest piece of engineering, I will admit. I just got it enough to work. And then you could get completions, you could get error messages. But the hope was to say, you know, here's error messages, here's IDE experiences,

we can have nice things. And I think that set a nice precedence that now in once you kind of set that higher bar, you know, the command line tools or the error messages or whatever it is, if they don't meet a bar, it feels like, oh, that's not quite that's not quite the Rust way, it should be a little bit nicer. And that's a that's a great thing to kind of instill into a small group of people. And then from there, it grows out into more of the community. Yeah. So then after I worked on Rust for, I guess, a couple of years at Mozilla, and I saw Servo, which was a, it's like an experimental web engine went in Rust. And they had started porting out part of Servo and putting it into Firefox. So Firefox users may remember Firefox Quantum, and that is the first time that they actually took out something from servo

and put it into Firefox. It's the first reasonable-sized chunk of Rust in the Firefox code base, which, as I recall, caused no end of consternation to Linux distros. So a little bit of apology to those Linux distro people that had to weather adding a whole Rust compiler and tool chain to your distros just to compile Firefox. So yeah, we worked on that. I worked on the servo team for a while. And I guess the last thing I worked on was I put together the WebAssembly team. I wouldn't say I put it together. The loose coupling of WebAssembly engineers, I gave them a weekly meeting that they could

talk to each other from different parts of the organization. And that helped shape the early days for WebAssembly. It had just become ratified. And then this gave us a chance to talk about how do we tool WebAssembly? How do we get maps into the original source code? You know, how did we build a set of additional tooling? So I did that. I worked on that for a while. And then that was kind of the end of my Mozilla tenure. But it feels like a good chunk of work. Yeah. Yeah. I think you just outlined what, six, 15 years of being involved in programming languages for the most part?

Does that make you feel old? No, but now that you mention it. Do you have any advice for anyone who's thinking, hey, I've been wanting to make my own toy programming language to do this thing? Just do it. Just do it. I mean, I think your ambitions will outrun your skill and that's how you should start and then try it out and then learn how to do a parser,

learn how to do a type checker. My skill level was at like a certain line. And when I was working in grad school, Jeremy, my advisor, Jeremy Seek said, you know, you need to read this book and understand it. And it was a book called Types and Programming Languages. Please do not start with this. I feel like the warning podcast, do not start with Super Mario. Do not read Types and Programming Languages as your first book. Read it as like your second or third one it's incredible because it will like really level you up in the way that you think about how a programming language is put together so like what is a function what is a structure what is a record what is and then you start thinking about what is a trade or a concept and it just kind of keeps raising and raising the level of abstraction. And while he

does this, between each chapter, there is a little programming language that is basically like the simplest programming language you can think of, plus this new feature. And so you can see it in isolation. So to really learn my way up into the skills that I needed in grad school, I went through that book and at every one of those chapters with the mini language, I just went and implemented that and then played with it. And then I went to the next chapter and then implemented that. And I just went through the whole book and implemented all the languages. As you begin to get faster and faster and faster, by the end, now you've written a parser, you've written all type checking for all kinds of forms. And it's a, yeah, it's a great way to, to, to work up, to be able to do a full

blown programming language if that's what you want to do. So what's your next programming language? After NewShell? Let's just do NewShell first. That'll be good. Oh, okay. Sorry. I was just going to say, are you done with programming languages now and focusing on NewShell? You want to talk more about that uh sure sure i think i kind of gave a brief introduction at the beginning but um yeah one of the things that i like about new shell that i think is apropos for the podcast is that you can extend new shell with plugins and the plugins all they do is talk with json json not json it happens all the time it's fine

they just use json to talk back and forth with the new shell kernel and so you can write a plugin in any language in fact jason wrote one in c++ i did to prove it was possible yeah yeah is it possible jason it was possible, yeah. Yeah. Is it possible, Jason? It is possible, in fact. It didn't take that much effort. I think the hardest part was picking a JSON parser that I wanted to use for C++, basically. Yeah.

So I think that's one of the nice things about it, is not only does it have this structured way that it views data, but you can participate in it, and you can kind of participate in your own language. And you just use the API, the JSON-based API. All you have to do is read in, standard in, and standard out, and that's the whole protocol, basically.

Yeah, yeah. I just did a bunch of work last weekend on it, and I'm looking forward to doing it again. This is one of those projects that, you know i don't mind working nights and weekends because it actually feels like uh it feels like it kind of fills me up you know scratches that design it and scratches the the right hobby edge what's uh like what's a good example of the type of thing you can do with new shell that might not be possible with either regular command line or PowerShell? Sure. So you can do this a little bit in PowerShell, but maybe the canonical example for

new is you type ls and you hit enter. And what you get back is a table. And it's not just a pretty ASCII drawn table. It's actually a structured set of rows and columns. So you can query this, you can pull out a single row, or you can pull out columns, you could say, ls pipe, where size greater than 10 kilobytes, and it knows how to understand what each of those mean. And then as ls output each row, the where command, the second one in the pipeline, then does the actual filtering. So it's much more composable. And because all of the commands talk in the same data description language, if you want to call it that way, the data type language, they all connect together. So if you understand how to filter the

LS table, the same thing works. If I type PS and get a bunch of processes, I can filter that table and work with that as well. So that's kind of, you know, from 10,000 foot view that everything, all the commands in new or this structured format allows you to compose really long lines of, you know, useful things that would be kind of hard to do if you didn't know your bash really well. You would have to remember all the command line arguments, and then you'd have to remember how to compose all these together. But because new is functional and kind of compositional style, you can just write the pipeline up to the point that you can figure

out up to that. And then it outputs the table as it stands. Oh, that looks good. But you know what I should do is reverse that. And then you say pipe slash pipe reverse. And then it reverses it. And then you just experiment. And it's interactive. Yeah, I know exactly one person who would go,

well, I could just do that in awk in one line. There's always that person. There's always that person. There's always that person. This is an aside, and this is not to knock C++ at all. But there's always that one C++ person that can say, hey, I can do that in C++. Why am I listening to you? I had one of those in one of my chapel talks.

I'm like, I could just do like crazy template metaprogramming. Like you can, it's true. If you know how to do it, go for it. That does, I guess we're getting about to the end of our time here, but you have a plethora of talks online as well yourself if people want to go find these. Yeah, there's, if you're interested in TypeScript,

you can search for like TypeScript, Jonathan Turner. The Rust stuff, I was telling Rob before we got on that I used to do a Rust podcast called the Rusty Spike. If you want to hear some old Rust news, there's that out there. And I have the blog that I write every once in a while. I write a blog post. And so there's blogs on New Shell and Rust of various topics in Rust, if you're at all interested in that.

And I think if someone were to get particularly bored, I think they can still dig up Minnow. They can. Yeah, I don't know. I'm pretty sure they can. If you can find it, good for you. Yeah, it's out there somewhere. Okay.

Well, it's been great having you on the show today, John. I'm sure we could go on for a lot longer, but it's been great talking to you. Oh, it's been great having you on the show today, John. I'm sure we could have go on for a lot longer, but it's been great talking to you. Oh, it's been awesome. Thanks for coming on. Thanks so much for listening in as we chat about C++. We'd love to hear what you think of the podcast. Please let us know if we're discussing the stuff you're interested in, or if you have a suggestion for a topic, we'd love to hear about that too.

You can email all your thoughts to feedback at cppcast.com. We'd also appreciate if you can like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving and Jason at Lefticus on Twitter. We'd also like to thank all our patrons who help support the show through Patreon. If you'd like to support us on Patreon, you can do so at patreon.com slash cppcast. And of course, you can find all that info and the show notes on the podcast website