CppCast - Networking TS

Episode Date: December 3, 2020

Rob and Jason are joined by Robert Leahy. They first talk about an open source flappy bird clone and the C++ framework it was built with. Then they talk to Robert Leahy about the Networking TS that will hopefully be a major feature of C++23. Transcript CppCast 276: Networking TS Transcript from PVS Studio News A Small Open Source Game in C++ The C++20 initialization flowchart Error codes are far slower than exceptions Beman Dawes has passed away Links The Networking TS from Scratch: I/O Objects - Robert Leahy - CppCon 2020 N3747 - A Universal Model for Asynchronous Operations P2161 - Remove Default Candidate Executor Sponsors PVS-Studio. Write #cppcast in the message field on the download page and get one month license PVS-Studio: analyzing pull requests in Azure DevOps using self-hosted agents Why it is important to apply static analysis for open libraries that you add to your project Use code JetBrainsForCppCast during checkout at JetBrains.com for a 25% discount

Thank you. of smart IDEs and tools like IntelliJ, PyCharm, and ReSharper. To help you become a C++ guru, they've got CLion, an intelligent IDE, and ReSharper C++, a smart extension for Visual Studio. Exclusively for CppCast, JetBrains is offering a 25% discount on yearly individual licenses on both of these C++ tools, which applies to new purchases and renewals alike. Use the coupon code JETBRAINS for CppCast

during checkout at JetBrains.com to take advantage of this deal. In this episode, we talk about an open-source Flappy Bird game written in C++. Then we talked to Robert Leahy. Robert talks to us about the networking TS. Welcome to episode 276 of CppCast, the first podcast for C++ developers by C++ developers.

I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing today? Doing all right, Rob. How are you doing? Doing okay. Getting into the final stretch of this year. It's been a rough year, but hopefully we have a lot to look forward to next year. Hopefully the final month is not so bad. Yeah, that sounds great. Let's have a great 2021. Yeah. Yeah. Here's hoping. Okay. Well at the top of every episode, I'd like to read a

piece of feedback. Uh, this week we got a tweet, uh, speaking of it being the end of the year, this is from, uh, Nicholas Lind on Twitter. And he said, thanks for for the year and he tweeted at us and some of his other favorite podcasts including Go Time and one of his other ones the ROS Developer Podcast, Compagnette and The Boring Startup but we were his number one podcast

for the year so that's nice. Awesome. Yeah. We're number one. Sorry. We'd love to hear your thoughts about the show. You can as always reach out Awesome. Yeah. We're number one. Sorry. Yeah. Well, we'd love to hear your thoughts about the show. You can as always reach out to us on Facebook, Twitter, or email us at feedback at cpcast.com. And don't forget to leave us a review on iTunes or subscribe on YouTube.

Joining us today is Robert Leahy. Robert is a graduate of the University of Victoria where he specialized in graphics, gaming, and digital geometry processing. After four and a half years in full-stack web development, he pivoted to financial technology in early 2017. He's since become involved in the ISO C++ Committee and strives to deliver software which is high quality and process-driven, while also meeting the rigorous performance standards for which finance is so well-known. Robert, welcome to the show. Thank you for having me on.

Can you tell me what kind of things you do with digital geometry processing? What that means? Well, when I was in school and actually dabbling in that sort of thing, on. Can you tell me what kind of things you do with digital geometry processing? What that means? Well, when I was in school and actually dabbling in that sort of thing, one of the big things that we did was capture and registration of geometry. So dealing with the problem of given a real world object, how can you use technology to accurately and quickly represent it in sort of like a mesh? So as a pre-graduate project, we implemented the Connect Fusion paper, which is a registration technology and technique that uses like the Connect camera from one of the Xboxes. I never had the Xbox. I only ever dealt with the camera. So we implemented that.

When I was kind of like tantalized by the idea of going back for graduate school, there was a lot more registration problems that were like a lot more high fidelity and a lot more cutting edge but i never wound up doing that so okay interesting so it's kind of like photogrammetry but maybe with other and more diverse kinds of sensors involved yeah mostly what you use is time of flight sensors that give you just like instead of a color reading you get a distance reading um at that pixel right and so you're you know that's for one camera and so it's the question becomes like do you integrate over many cameras do you integrate over one moving camera and that's just registration like the the field also includes processing like you know um subdivision um like how do we represent high fidelity meshes using the least amount of space possible while still making it so we can recover the general shape?

Things like that. My, I guess the person who would have been my advisor who supervised my like pregraduate work, he was very interested in the problem of modeling the human hand, right? Cause fingers are like a tricky thing to model because he used to like to do this with his finger and like point to the joints and be like,

you see how it like compresses here. That's a big problem. I've written like three papers about this. Oh, that's cool. Yeah. Okay, well, Robert, we got a couple news articles to discuss. Feel free to comment on any of these, and then we'll start talking more about your work

on the networking proposal, okay? Sure. All right, so this first one we have is a small open- game in c++ and this is on preshing.com and it's uh you know talking a little bit about the game but it's more advertising plywood which is not a game engine but a c++ framework that the author built the game with and uh the demo game is a flappy hero or flappy bird clone called flap hero which kind of brought me back to the early uh 2010s when this was a real you know phenomena for a little while yeah the guy who made flappy bird like got so

frustrated with all of the attention he like dropped out of the world and took the game off the App Store. I read up on him again after seeing this article, and he actually claims he did it just because he was upset that people were getting so obsessed with the game because he meant for it to just be something you play for a few minutes when you have some downtime, but people being you know very very obsessed with the game yeah i kind of remember that game as being like the the genesis of the mobile gaming phenomenon of like now you know i take the subway and it seems every second person is playing some kind

of game on their phone whereas before flappy bird or whatever you know they would be reading a book or just sitting there listening to music or something like that yeah yeah very addictive but you know easy game to play and just lose yourself and yeah i took a look at the the plywood framework and i think one of the things that was the most interesting is that it looks like you build the modules and declare them like in c++ itself like you write a little function that takes in a parameter and then you like declaratively build your module i've never even considered that as something that you could do i think other languages languages do that with some of their frameworks where, you know, you build up your packages in the language itself, but I've never seen someone try and do that with C++.

So that really stood out. And I spent a little bit of time like peering at that and being like, wow, that's interesting. And these modules, are they related at all to C++20 modules? He actually has, I actually have the webpage open and there's a little information bullet here that says specifically plywood modules should not be confused with C++20 modules. So they thought of that one. Okay, very cool.

Well, next article we have is this post on the C++ subreddit. And this is the C++20 initialization flowchart. And it is a giant, giant flowchart, which is just kind of scary to look at and think that initialization is still this complex. Still? It's more complex if you release C++.

It gets more complex. That's true. That's true. Now, one of the themes in the committee is that if anyone ever puts a paper out there that actually makes the language simpler in any way, people are like, this is amazing. How could you possibly think to do something like that? So yeah, it seems like initialization, especially just every single release and every single meeting, there needs to be another special case. Like, oh, this thing doesn't quite work. I guess

on this flowchart, let's add another couple like nodes with a couple more decision points to make this work the way we think it should. The PNG is actually so big that whenever I switch back to the tab, it takes a few seconds to render it. Yeah, I saw that too. And I feel like some of these corner cases

and stuff that end up in the flowchart were things that are actually, and I feel like perhaps apocryphal stories but uh discovered by compiler implementers when they're actually trying to write compilers and like oh wait a minute we need another corner case here yeah it's interesting how often that happens in like software engineering in general where you like try and think of everything and then you get to implementing you're like no this this totally doesn't work um so i can't decide whether it's heartening or disheartening that it happens with the with the committee too like you put the best minds and c plus plus in a room and they still can't be a substitute for one guy trying to implement something in the compiler

i think it's funny though the author of this says it honestly took a lot less time than i was expecting so i wasn't when i i read that before I clicked on it and I was expecting a relatively small flow chart. And then when I realized that like you have to zoom in on it and scroll around unless you've got like a 48 inch monitor or something, I actually hold on. I have a 49 inch monitor and see if I can, it still doesn't really fit.

I can confirm that it does not. It kind of fits, but you can't read it. You can't read it still, okay. Yeah, you can't read it. You still have to zoom in. You need to upgrade to an 8K 49-inch monitor, and then you would be able to read it. Yeah, I have to lean in really close to the monitor, but maybe I can manage, yeah. Yeah. Oh, that's funny okay the next thing we have is this blog post on

lordsoftech.com and this is error codes are for far slower than exceptions and uh this post they're basically saying that you know with modern uh 64-bit architectures you know performance concerns should not be a reason to avoid exceptions and they did a couple uh benchmark tests with an xml parser one implemented with error codes one implemented with exceptions and found that the error code uh using implementation was uh you know about six percent slower than the exceptions. I want to start by, Oh, sorry. Sorry. Go on. Uh, I just wanted to say, I agree, but in nested cases, particularly that error codes are going to be slower than exceptions. And if exceptions are truly exceptional, cause I've done my own study on that too, but I don't fully agree with the examples here because the author switched from an API that returns a

value to an API that takes the value by reference. And if he had instead done something like expected where it was the value plus the error code as the return value, then I think we would have gotten slightly different numbers. That's all. Well, maybe that could be, you know, a third implementation to try out, you know, do error codes, exceptions, and then expected. Yeah. Yeah.

Kind of in that same vein, like with expected, I mean, I think that expected is pretty much in like standards committee purgatory, but Niall Douglas's outcome is like you can download and use that. But I'm pretty sure when he was going through a boost peer review,

he ended up removing the monadic interfaces, but like kind of halfway, like two thirds, the way down the article, there's this example of like how the code gets simpler when you use exceptions and the fallback is this propagate error macro. But I remember working with a version of expected that had the monadic interface like then. Yeah. And that was like really, really way to work with with error codes like there was almost there was no boilerplate like you just did like um i think they were calling it bind at the time like just bind bind bind and then you got out the final value or any of the errors along the way um with this

implementation though like with the xml parser throwing exceptions on badly formed xml that gets into a question that i've actually heard raised several times in the networking study group, which is like, what do you mean by an error? Like who's error, right? Like is end of file an error? Is bad XML exceptional, right? Like, are you reading random files and trying to figure out if they have XML in them? Or are you getting input that you think is XML?

So the question becomes like, what level of extraction are we working at here? And what kind of error actually is exceptional? Because the answer to that question isn't clear. You have to establish a taxonomy of errors and then that itself becomes complicated. Yeah. And so I think that what the conclusion I've come to is in the general case, the only thing that you can really, really, really assume in the general case is truly exceptional is like bad Alec. But then you have people who are like, truly believe that bad Alec should never happen. And if it does, you're on Linux, and it's going to kill you anyway. And

if it doesn't, you can't recover. And so you should just abort. And every allocator should be no throat. Well, yeah, on Linux, a new or Malik, I mean, is never going to give you back an all pointer, right? It's only going to cause you a problem once you try to actually access that memory. Yeah, that's, I mean, I don't, I don't know how I feel about that personally, but I know it's the case and this is the world that we live in. And so, you know, it's, it's interesting to think about the fact that unless you know the domain you're in, you can't ever make the conclusion that something is truly exceptional because someone could be trying to use your XML library just to figure out like, is this random string XML? In which case,

like it's not XML isn't exceptional. That's half their use case. Interesting point. Whenever students ask me about error codes versus exceptions or whatever, I'll make my arguments and then I'll say, but the key is that you're consistent in your code base. I hate it when I go to a code base and I have no idea what to expect.

Is true an error? Is false an error? Is, you know, are you using exceptions? Whatever, just be consistent. when I go to a code base and I have no idea what to expect is true and error is false and error is, you know, are you using exceptions? Whatever. Just be consistent. Yeah.

The true false one. I think that typically on POSIX, I remember correctly, you get back falsy values on success. Yes. And, and then in windows it's the opposite. And so if you end up doing any cross platform operating system programming, you can almost guarantee you're going to get the check wrong at least once in your code base somewhere.

Yeah. Okay. Last thing we have to mention is some sad news that Beeman Dawes has passed away recently. This was the author of Boost and File System. He was on the ISO committee and dedicated a lot of time and brought a lot of good things to the C++ language. So it is sad to hear him go.

Consolences to his family. I think we did try to reach out to him and get him on the show about two years ago at the time that he was retiring. Unfortunately, we weren't able to get him on the show about two years ago at the time that he was uh retiring it's unfortunate we're able to get him on but uh yeah sorry to see him go yeah it's always sad to get that kind of news i someone im'd me about it like 30 minutes for the show and i was like that's yeah you never want to hear something like that yeah okay well let's start with uh you know you're you're here today robert to to tell us a little bit about the networking TS, which I don't think we've spent too much time talking about in much detail on the show. Could you maybe start off by giving us a little status of it? What's it looking like for C++23?

So for, as far as I've heard, like within the mailings and stuff like that, and the committee and conversations, we still want to ship executors and networking in C++23. Clearly, the whole COVID thing has thrown a wrench into the best laid plans. Right now, study group four is kind of like there's a lot of long running tasks, like investigations and longer running things that some people are doing to try and get a head start on the integration with executors. Because if you look at the way that the TS is currently written, and then you look at the way that ASIO is right now, right? Because ASIO is kind of the reference implementation, the de facto reference implementation. ASIO has actually incorporated what we think or hope executors is going to look like, whereas

the TS is quite old at this point. And so there's a lot of work and investigation that has to go into meshing them together. And the calls that have been happening because SG4 has had work to do have been focused around that. So there still is some modicum of progress being made on networking, but for the time being, the work is quite slow

because until Executors is set in stone, all the work is kind of tentative, right? Like you can't, you can be like, we think that it's probably going to look this way, but like there could be some last minute, like, you know, live grenade thrown into a room where everyone changes everything and then all your work goes out the door,

which is something that's been brought up in most of the executors discussions. A lot of times it kind of turns into this very pragmatic and legitimate argument that like, sometimes you end up talking about things that are easy to change later. Like, do we want to ship this thing or not it's like well just don't ship it now put it somewhere else because we've got a lot of work that's blocked on executors that we want to get into 23 and until we can stamp it and say this is what it's going to look like a lot of

that work can't proceed and so that's really the the meat of the the networking ts progress right now i mean um one small paper did get forwarded to the library wording group, a paper by me to like tweak one small thing in networking. But in the interim, it's a lot of SG4 investigation and planning and discussion about how we think we're going to integrate with executors when they've been given the stamp of approval. They aren't just sort of like this semi-crystallized thing

that could hypothetically change, but actually hasn't really changed shape in a while now. So you brought up a question that I had, which was, I knew that the networking TS started from Boost ASIO, but I didn't know if they had then diverged in that point. But you're saying Boost ASIO is still basically the reference implementation of... I believe that, and I could be totally wrong on this, but in my personal experience, and I actually do a decent amount of work with Azio professionally, I think that most code that would

be valid to write against the couple of reference implementations of the TS floating around is still, thanks to semi-heroic effort by Chris, valid to write against ASIO, even though ASIO has progressed. And so I consider them to be somewhat of a substitute for each other, at least in terms of the common subset they support. I would consider ASIO. ASIO is a big superset of the networking TS. It has a lot of really useful stuff

that probably will never be in the TS. Okay. Right, like stuff like it has classes for asynchronously reading and writing to and from serial ports. And like, that's to like i would i want to say four people but it could be five you know um so i don't that's really useful to those people and i like the fact that it's there in asio but i can't ever see that getting standardized and so it's uh i would consider it to be a superset um because it also has the executors integration right like it has real

for the last i think just the last release it has a full bonafide implementation of P0443, which is the executor's paper. So anything in 443, you can play with in the latest version of ASIO and it integrates with the components that are going to go into the TS in the way that I guess Chris and SG4 up till now, like the work we're doing now, we think it's going to look like. You know, having a library for asynchronous serial reading and writing would have been incredibly useful to me in 2004. I don't think ASIO came out until like 2005 or 2006. Yeah. Although I know saying this, I knew the moment I said it, we're going to get probably several

comments from people saying, wait, I use serial ports every day in my work, which there are still a fair number of people that do, I use serial ports every day in my work, which there are still a fair number of people that do, I am sure. Yeah, I used a serial port once in college, I think, and that was it. But I mean, in finance, you know, that's not really where I would expect to find people using serial ports. So it's not like a really good authoritative statement. You know, I don't use serial ports, but I don't work in a field that would use serial ports and therefore no one uses serial ports. Isn't really a legitimate argument. I used a integrated serial port and an ARM chip that was talking to an onboard PIC microcontroller. So people are still totally doing that kind of thing today, I would assume. Either serial or I squared C or something to those devices.

Yeah, I was going to say I squared C or serial I would expect to find in those domains. I just have never really been in those domains except like the obligatory class in college, you know? Right. So how exactly do executors and the networking proposal relate? Like, you know, you're saying the boost ASIO reference implementation has executors built in. Is executor something you're going to be very aware of when you're writing networking code? Or is it something that's more under the covers? I think that depends on the level at which you're actually writing the networking code. Like if you're talking about the very low level stuff, like when you're bootstrapping your networking code from nothing, the executor is basically a way for you to get that background

context to execute work. Because that's one of the things that's the most interesting fundamental problems with anything asynchronous is that when you're writing synchronous code, you know, you have a context of execution. It's, you know, the fact that you descended from main or the fact that you descended from the start of some thread that someone spawned somewhere. So you have the CPU and you're executing and you can go into a function and stay in there and then come back and you still have somewhere that your code can execute. When you're trying to execute asynchronous code, there's all this stuff going on in the background. And that raises this question of like, well, where, like where does that context to actually run the code come from? And so the analog between

executor and allocator, which is, it kind of falls apart a bit. I actually, the paper that I recently got through in the, the committee talks a bit about why they're not a perfect comparison in the context of the fact that like std allocator is a good default and there's no good default for executor. But that's the that's why you're aware of it. Like when you when you actually put some work to be done asynchronously, the executor determines how and where and when that work is actually going to run in the background. So if I try and read from a socket on a on an asynchronous socket, that work completes. And then where the the next piece, the completion handler runs, is determined by the executor.

So if you're implementing something from the ground up, you need to be very aware of that. Otherwise, like when you get to the higher levels of writing asynchronous code, the executor is something you kind of choose at the beginning. Like in the code that I write professionally, what typically happens is that you just inherit the executor that some socket or something is using that you're given. You're given a socket. You're like, okay, I guess I'm inherit the executor that some socket or something is using

that you're given you're given a socket you're like okay i guess i'm using its executor now and you don't think about that and then in like main you think about that because you're like okay i need like four different threads um and so i need these four different you know contexts that have their executors then you pass them around and that determines where the work runs and what kind of like threading and synchronization guarantees you want to give. I heard it articulated in the discussion about what exactly an executor means that an executor is not just a place to run work but also a policy about how that work can run which is another good way to think about it like what kind of concurrency is allowed is also something that an executor encapsulates right like is this whole thing single threaded and not thread

safe or is it thread safe and that's a decision that you make when you choose the executor. Does this have any relation at all to the execution policy that parallel algorithms take? I am not 100% sure of the answer to that. I want to say yes, because I have heard executors discussed in the context of parallel algorithms, but I've not really delved into that. But I think there may be some relation, but executors are obviously much more flexible. Like you can just write one from nothing. rhythms, but I've not really delved into that. But I think there may be some relation, but executors are obviously much more flexible. Like you can just write one from nothing. You can write one with your, with anything that you want, right? Like you can write an executor from scratch. It's not choosing from some bounded set of policies.

Okay. So what does the interface to an executor look like since you just said we can write one from scratch? Well, right now there's actually a paper that I had pulled up. There's a paper that wants to change that a little bit, P2254 in the latest mailing. But right now, the interface to an executor is super simple, which is one of the places that a lot of the controversy around executors comes from because some people really want the simple interface and some people need a lot more than the simple interface.

So the simple interface to an executor is literally one customization point object, which is called execute. And you pass it an executor and something that can be invoked with no arguments. And it invokes the thing with no arguments inside of the execution context that executor is a handle to.

Okay. And how do people want to complicate that? What do they want to add? Well, I mean, one of the perfectly legitimate concerns that gets brought up a lot is, okay, I want to execute this work in the background, so I pass it to execute. Okay, great. Let's say that the submission of the work was completely successful. The executor or the execution context, the executor is a handle to, because executors are handles to execution contexts in the same way that allocators are handles to heaps. So this

execution context now has the work and it's going to run it sometime, somewhere. But then imagine that like sometime in the near future, that becomes impossible. How does your work find out about that? Right? Some people don't care, right? A lot of the code that I write, that's like pure networking code. The only time that really ever happens is a massively catastrophic error, in which case my application is going to shut down and I guess my work never gets invoked. Well, that's fine. Or I want my application to shut down and the work never gets invoked, but that was intentional because the application is shutting down. But there are people who care very deeply about that sort of thing. And there's no standard generic way for the executors as currently envisioned to communicate that. And that's actually the

point of P2254, the paper I mentioned. And it's also the point of the senders and receivers interface, which is like this parallel to executors. It's also in P0443, the executors paper. So senders and receivers have an interface. Well, the receiver has an interface that's actually a triple. So there's three channels rather than one. Whereas an executor takes a function that can just be invoked or not invoked. So invocation is the single channel. A receiver has three channels, set value, set error, and set done, right?

And so it also bifurcates error from done, right? Like it kind of, there was this paper that was going around, which is, what was the title of it was cancellation is serendipitous success. Okay. Right. Like, so basically an operation that's canceled succeeded without ever having to actually do anything. Right. And so modeling it as an error isn't exactly correct, but modeling it as success also isn't correct because it depends on how you're thinking about the operation, but which kind of comes back to what I was saying earlier about the

taxonomy of errors, right? Like is cancellation an error? Well, that really depends. Right. And so giving it its own channel called done makes sense to do certain set of people, whereas other people don't care. So then as the counter argument for why you can keep the interface simple, the fact that you're passing it any invocable thing and you just make that invocable thing have its own communication channels back or something bound to it. Well, yeah, the problem really is, is there's no way for the execution context to tell your invocable when something goes wrong. Right. Right. Yeah.

And so if you can obviously take an invocable interface and give the customization point object you use to submit work to an executor, not just take a nullary invocable, but also be able to take a receiver. So if it wants to communicate, like I can't invoke you because of some error, then it has a set error channel it can actually call so it can communicate those without having to rely on execution context specific machinery. Because that's what executors fall back on, right? If there's an error, it'll be handled according to whatever the execution context is, which doesn't help you when you're writing generic code. But it does help you if you're writing concrete code, right?

Like if all you care about is the success conditions and you actually do want to separate out the error path, then executors make sense. Like when I write networking code, those catastrophic errors, I want to handle them way back in main. Like I don't want them handled inside my operations. I want to handle them in main and be like, okay, I'm logging something and ending the program. I want to wrap the discussion for just a moment to bring a word from our sponsor, PVS Studio. The company develops the PVS Studio Static Code Analyzer, designed to detect errors in code of programs written in C, C++, C Sharp,

and Java. The tool is a paid B2B solution, but there are various options for its free licensing for developers of open projects, Microsoft MVPs, students, and others. The analyzer is actively developing. New diagnostics appear regularly, along with expanding integration opportunities. As an example, PVS Studio has recently posted an article on their site covering the analysis of pull requests in Azure DevOps using self-hosted agents. Check out the link to the article in the podcast description. So you're talking about Boost ASIO earlier and how that's the reference implementation. If listeners are not familiar with Boost ASIO, could you maybe tell us a little bit more about what it's going to look like when we have the networking TS in practice? I mean, if you're, you know, kind of being introduced to this networking library and you want to write some, you know, code to interact with the web, what's it going to look like?

To interact with the web. What's it going to look like? To interact with the web like HTTP? Yeah. So the thing with the networking facilities that we're trying to standardize and that are in ASIO right now is their very basic low-level kind of functionality, right? So you get like a TCP socket or a UDP socket or a raw socket even. And then the idea is that like so many things in C++, we want to make sure that you have the capability to build whatever you want on top of that. And that's also been a topic of discussion in the networking study group, because a lot of people want to do things like give people TLS by default, which is an interesting argument that makes sense at a lot of levels of analysis, right? Like if you're writing a phone app, like it makes sense that it should kind of be impossible to make an unencrypted connection because if you make it possible, right, you can make the argument that most people

are not gonna end up doing that and it's gonna really complicate things and it's gonna be insecure by default and you don't want that. But at the same time, that's not really a low-level building block, which is what the TS and ASIO set out to do. So for communicating over HTTP specifically, there's actually another

library in Boost that builds on top of ASIO and does that. And that's Vinny Falco's Beast library, which I haven't used it directly professionally, but one of my co-workers, I believe, has. And it's a great library. I mean, one of the projects that I'm working on right now, we're actually debating whether or not the cost benefit is there to replace a third-party library that uses libcurl and is all like kind of C-like. We're debating whether it's worth it to replace that with using Beast and ASIO so we can get the full advantage

of the operating system. Because the shape of the interface that curl gives you really like limits the kind of optimizations you can take advantage of. Like you can't do zero copy networking, for example, which is something that one of my coworkers

has complained bitterly about because it leaves performance on the table, especially when you're transferring really big files. So that's what I would, yeah, that's what I would say about that. If you want to like pick something up and you want to start, you know, using the networking TS the way

something built on top of it might look to interact with HTTP, you're going to be wanting to use BoostBeast. Whereas the networking TS itself leaves open the possibility of basically going down to the very bottom level and implementing stuff from the ground up. So you mentioned the discussion about whether or not you would by default get a TLS socket.

Does that mean, so it does support encryption, the networking proposal, I guess. The proposal itself doesn't have any facilities for encryption, but ASIO does, which is another thing that has been discussed a bit in the networking subgroup. Like, this overhead of standardizing encryption is non-trivial, right? Like, which encryption are we standardizing? Is it implementation-defined? If it's implementation-defined, how is it actually useful? Like, how do I know what values I can actually use?

Right. So there's been a lot of discussion around that. And personally, I think that standardizing some form of encryption would be nice. But the argument that I've always brought up when that comes up is that, you know, right now we want to ship the networking TS in 23 and we can add something to do encryption. Whereas if we mandate that we ship encryption and we miss 23, we can't unmiss 23, right? Whereas in ASIO, for example, the TLS support is just a couple classes, right? It doesn't require any fundamental changes. But the counter argument to that is that there are some operating systems that just aren't

happy with that, right? Like I believe that some of the mobile operating systems, like you can't really create raw TCP connections unless you have a certain level of permissions that most applications don't. I think that iOS is like that, although I could be totally wrong about that one. So you basically get TCP, UDP, IPv4, IPv6. Yep. And I think there's, I'm not sure if the raw sockets are actually in the TS. There's raw sockets in ASIO, but you basically get like kind of a, an asynchronous analog of Berkeley sockets, but like a lot of the, a lot of the verbiage in the TS, like the standard ease, like kind of hand waves and says this acts kind of like this thing in POSIX to avoid like, you know, actually talking about things that are beyond the scope of, of this, the standard. Cause then you have to pull in all these definitions,

then you'd be re-standardizing things so basically just you know calls out deposits and says this kind of acts like connect okay so no lower level than that like i'm not going to write a ping app or something icmp packet custom things you know what i don't again i don't know if this is in the ts because there's so many things that um that are in ASIO, but not in the TS, but I'm just, I'm pulling it up right here. I'm pretty sure there's ICMP in ASIO. In ASIO, okay. Yeah, yeah.

There's IP ICMP. Okay. Yeah, I have no idea if that's actually in the TS or not. I'd have to actually take a look. But, you know, again, right? Like one of the things that I like to say about ASIO and the TS specifically is that I think the most important thing that the TS gives you isn't the canned stuff that you can do.

It's not the sockets because you can implement those yourself, right? Like I've actually, I did a little toy project. I think it was a couple of years ago now where I wanted to fool around with IOU ring, right? Because IOU ring is the new cool thing to talk about in Linux high performance IO. I was like, well, let's see if I can take this and make it look kind of like the networking TS. And I ended up having to implement my own execution context and implement my own sockets and handles and stuff like that.

And that was totally possible. So I think the most important thing that we're going to get from the TS and we're going to get in conjunction with executors is this model for making asynchronous I.O. look and behave a certain way so that we can build all these libraries that just take a template parameter that's like an async stream or something so that we can build all these libraries that

just take a template parameter that's like an async stream or something like that. And they all work together regardless of whether you chose to use like Uring or completion ports or whether you're writing to a file or a socket or something like that. You do get the kind of canned stuff you need to do the most basic socket stuff, but I'm not quite sure the extent of the things that you get and the extent of the things that are non-standard in ASIO, but you definitely get IP, TCP, um, UDP, and then like you get both families, V4 and V6. That's definitely in the TS. Cause I've used, um, strict TS implementations and I've used those things. Okay. So you said the main thing now that you're waiting on is executors. And, uh,

for the most part, the actual networking proposal has been pretty stable. It has been, it has been, it's, it's been stable. And the papers that we kind of have pending and we're talking about seem like they're fairly stable. I think the last time we had a call where we, we made progress, we were mostly trying to update some older idioms. Like there were some older idioms in the paper that was trying to bring the TS into alignment with the executor's TS. And it was using some older patterns that were kind of C++ 11isms, something like that. I just pulled up the TS itself and searched it, and I don't see any hits on ICMP to answer that question. It was bothering me. So I'm just thinking like uh it would seem to me that co-routines and executors and the networking proposal all have things in common and i was just wondering like

if the networking proposal needs to be or had been updated recently to be able to take advantage of co-routines or if that's even relevant at all or if that comes with executors once that gets implemented. And I have to admit, I really don't fully understand my own question, but no. I kind of, I mean, you know,

I mean, I guess if you don't understand your question, I can answer what I think your question was. That sounds great. It sounded good in my head, but once it came out as, yeah. Well, it actually hits to something important that happened recently. I think we actually voted this paper forward or we had a decent vote of confidence on it in Belfast. That would have been, man, that would have been over a year ago now.

That's wild, the whole thing. Every day is Thursday and it's been the same Thursday for nine months. Anyway, in Belfast, we really like this idea because the TS's framework includes this mechanism called completion tokens. So when you pass the final argument, when you're beginning an operation, the final argument isn't a completion handler, right? It's not necessarily a function that gets invoked. It's a token, which means that it's a way of finding out what function should be used to indicate completion. And the difference there is that you can fully customize the completion reporting mechanism, right? So you can magically take any operation in the networking TS or that's written in the style of the networking TS,

and you can pass it a token called use future. And all of a sudden, instead of calling calling a function the operation returns the future and uses a promise internally like totally seamlessly um and so this mechanism you know to your question jason the mechanism seems like okay well can i make it so these are like these are co-routines and the answer to that before we decided we really like this paper was yes with a big but and the reason it was a big but is because before this paper i forgot the number it was one of chris's papers before this paper the operations in the ts always began eagerly right so as soon as you call what we call the

initiating function which is the function that actually gets it started the operation was pretty much already already going like it returned to you and off in the background somewhere this operation was already trying to make forward progress and the problem with that is that when you were trying to transform your initiating function into something that used co-routines the fact that it was off running in the background potentially and could complete meant there was a race condition between the first suspension of a co-routine and the possible resumption right and so basically what ended up happening is when you tried to make any of these operations into something that used coroutines,

you had to introduce a mutex, which is like, that kind of belies the whole point of C++ and zero-cost abstractions. And the reason for this was simple. It was that the networking TS was always eager, but eagerness wasn't necessarily the best approach for every single way of notifying of completion. And so the change that was made

was that it's actually possible now within the completion token machinery. This is something I actually talked about in my talk this year at CppCon. It's possible for the completion token mechanism and implementing itself, right? Like in customizing the initiating function, it can just capture an operate like a function that encapsulates what the operation would do to start itself and a whole bunch of arguments. And it can just put those away somewhere and then wait and start the operation again later. So you can seamlessly transform one of these into something that uses coroutines. It will just capture what's called the initiation, the thing the operation wants to do to get started. It

captures all the arguments, it puts them somewhere, and then it waits for the first suspend of the coroutine. And then it actually starts the operation. So it doesn't have to lock. Chris actually laid this out as kind of the motivation for why he made this change. Cause he was trying to implement kind of co-routine support and was like, now I have to put a mutex everywhere and that's not very performant. And so I need to change something. And this is the change that got made. And it actually ends up a lot of the machinery that he implemented kind of to stand beside this actually makes implementing the operations themselves a lot simpler. There's like one helper function you call. You don't have to worry about how you actually go about deducing the final completion handler and all the stuff

you had to before. You just call this one function and you pass it your initiation lambda, and it just takes care of all of it for you. And magically you get to write an operation that supports coroutines, promises, futures, and like anything anyone can dream up and implement. I think it's really elegant. One of the reasons I got so interested in the networking TS and asynchronous IO in general was I read Chris's paper from ages ago where he lays out completion tokens and what he calls the universal model for asynchronous operations. If you Google universal model for asynchronous operations, that's the first hit that comes

up is a paper by Chris laying out an early version of completion tokens before it supported this, but the principle is still the same. Now that does sound much cleaner. Yes. And I've experienced that problem of the work completing before you were ready to accept the results and having that race condition, but I would not have considered that in this context, which is what makes asynchronous programming so much fun, right? Yeah, it's wild how quickly it goes from being simple to complicated. Like if you put the pieces together correctly, your higher level code can be really simple, but then you get down to the lower level code and you're just tearing your hair out. And one of the ways that I found that you can get around that, not easily, right? But one of the ways that you cheat is that you basically make everything single threaded and then you shard things out along

an alternate axis, right? So instead of having, let's say your application has eight threads, it's a server and has eight threads. One of the temptations that you want to give into is, okay, well, I want to distribute all my work across these eight threads. The problem with that is now you have like one connection, let's say, and two parts of that connections handling logic are on two different threads. So now you need to figure out locking and ordering and all this great and wonderful stuff and you're pulling your hair out. But the alternative to that is that instead of having eight threads and you just kind of smear all the work, you have eight threads and you kind of round robin strands of work, right? So, okay, I just got an incoming connection. You're going to be assigned to thread two and

everything to do with you is handled on thread two. So whenever you write some logic that just handles something to do with one connection, you're like, I'm just going to assume that's single threaded. And since I know it's single threaded, I don't need locks. I don't need to worry about ordering. No one else is executing at the same time I am and modifying my state. And you get the benefits of concurrency by just round robining over this thread pool. Like, so you go to thread one, you go to thread two, thread three, and then you loop back around. And the only time that really becomes an issue is if you have super intense, super long running work. And through bad luck, it all ends up on thread three, for example.

Right. You mentioned that you gave a CppCon talk about networking TS. Should we talk a little bit more about that and about some of the subjects you brought up in that talk? Yeah, actually one of the topics that I just went over was brought up

in the talk that I gave in 2019, which is this way of handling how you can leverage multiple cores while not boxing yourself into writing a bunch of locks and tearing your hair out about threading. And the approach that I recommended was exactly the one I just described. And in fact, I use that at work, that approach where I'm using an application or a family of applications that I'm working on right now. And it seems to work out, work so very well, right? The code itself is easy to write, but we do get to take advantage of the fact that the machine has many cores because it's a server. And so it accepts many connections and it just assigns each connection

to a thread. And as long as we're, we don't have them bunching up, then, then everything is, is just peachy. So that's like, sorry, go on. How often does that happen that, that you do get things bunching up? Like what kind of utilization do you manage to average across the system? The utilization managed across the system in my measurements of this particular system seem to be mostly IO bound because it's a very heavy IO. The server does a lot of, it's sitting in front of a custom in-house database. And so it does a lot of file IO that it's waiting on. And then when it gets the,

when it gets the data and it actually reduces it to JSON and sends it out. And because JSON is so outrageously bloated, it spends most of its time waiting on IO. And so it's perfect, right? Like I've never seen anything bunch up and people, people kind of rightfully so, right?

When something, some library comes out and it wants to be basically the end all to be all you're right to be like, okay, can I get the performance that I really need out of it? And so that comes up a lot when you're talking about the TS and ASIO. And I, I can saturate a gigabit link pretty easily with just one core, like just sending the buffers off and the, the connection is capped out. The CPU, when I'm measuring it, the CPU isn't capped out the cpu is kind of not sitting there idling it's still got to like pull data in and turn it into json but in terms of sending and throughput it's completely saturated the link and i've scaled that one up like to man i did a

scalability test with a co-worker who's writing the the application that interacts with the server and uh we got some pretty ridiculous throughput when we ran the two of them on the same box they didn't have to worry about the gigabit link between them. We were pumping some pretty serious data across. I think it was like 32 or 64 connections with eight worker threads, I believe, between the two. Using just on both ends. He was using ASIO. I was using ASIO.

And then I believe the front end to his application, which is HTTP, was implemented with Beast, which I mentioned earlier. And yeah, there was no bottleneck there. Um, the biggest bottleneck that we sorted out was a buffer management. Um, he was, something happened in his application was, was allocating a buffer every time it went to receive. And so we changed that. Um, you know, earlier we talked about my graphics and gaming background. I was like, well, you know, when you're, when you're rendering a scene, you have like a front buffer and a back buffer, right? And then you then you just swap them like what if we just do that right like because you're trying to just move things between two pipes like okay you're writing

into this one from from this socket and then you're writing out from the other one and then you just swap them and then you never have to to allocate that's what i do on the back end and then just everything worked out um so it's amazing how much like one allocation done frequently can totally hamstring your performance. So now this is a little bit of an aside, but early on you made, well, I believe you attempted to make an analogy, but said that maybe it doesn't quite hold up between allocators and memory resources with heaps and the executors and the execution context. And now I'm curious with the kind of throughput you're talking about, if you were using custom allocation strategies at all, or PMR taking advantage of those more recent things in the C++ standard. So we were, we are not, because maybe it's just the finance background in me talking.

I just try and not allocate at steady state. And that's what this double buffering strategy basically allows. Like there's this idea that, especially when you're in a throughput task, right? Like if you're in a latency task, all of a sudden the calculus flips, right? Like any allocation ever on a latency sensitive task is suspicious, right? Because the latency, like the incremental latency matters. Whereas in throughput, the incremental latency doesn't really matter, right? Like if you're only caring about throughput, then the fact that this piece of data became available two microseconds earlier, like that doesn't matter as long as

you're keeping that link saturated. Right. So this is very much a throughput application. And so I don't care that allocations happen or even that when they happen, they might take a while because they're using the default system allocator. What I care about is that at some point in handling a very large high throughput connection, I stop allocating. And so the way that I do this is that I set a high watermark. I'm saying like, okay, you know, we're trying to build a buffer then to send over TCP. When that buffer gets to be over 10 kilobytes, like you need to send it, swap the buffers and then fill the next one. And so what ends up happening is like, let's say that these are the memory buffer from the format library is what it's actually filling because it's doing JSON. And so those are kind

of vector-like, right? So what will happen is that they'll start out being very small. And then as you put JSON in them, they're going to allocate and resize and resize and resize. But at some point, you're going to perform the largest allocation you ever need. And then after that, that's never going to fill up anymore and your connection can pump out. Literally, I've done queries against this database

where it pumps out tens or hundreds of gigabytes of data and it just doesn't allocate anymore because it's putting the JSON directly into this buffer using the format library. The buffers reach the largest size it's ever going to reach.

So there's no need to allocate at all anymore. And then it just sends it and swaps to the other buffer and fills that buffer. So while it's waiting on the operating system to send, it's still generating data from the backend database. And it never has to allocate either of these buffers because it just keeps swapping them back and forth. So we didn't actually see a use case for custom allocation. That's something that my company does when we write latency-sensitive applications, ones that are actually processing live market data which this isn't this is a database application we're processing live market data we use all sorts of allocation strategies that that go so far as to like allocate linux huge pages and

and do a slab allocation in there so that you know you don't get as many tlb misses and stuff like that because that's where every microsecond of incremental latency actually matters right you're not trying to maximize throughput. You're trying to say, okay, what's my time from ingesting this on the NIC until I can actually make it available for someone to make a trading decision? Okay, cool. Thanks. I do have one other question. I'm not involved in standard library implementation at all, that some of the larger standard library proposals, Boost.regx, FileSystem, Ranges, the parallel algorithms, have a lag in their actual implementation inside our standard libraries. Like libc++, Clang standard library implementation,

doesn't have the parallel algorithms yet for example and i'm curious if you foresee or anticipate any concern with this for once networking is approved and executors is approved that these sound to me like big proposals do you think that we'll see the same kind of uh lag or not in our standard library implementations? That's a really good and interesting question. And I think the answer to that is it really depends. I know that I was looking through some old... I forget why I was looking through them, but I was looking through some old posts on r slash C++,

and I saw that there was one of the standard libraries actually had a reference networking TS implementation. Now, that would be an older version of the TS, which would need to be upgraded. But like, I think that means that at least one vendor, probably Microsoft, they tend to, yeah, some kind of a version of it available. So I would anticipate that if that's the case, the lag might be minimal. And the other thing that, you know, might happen and it might not, right? Like I'm just, I'm not a standard library implementer either. Right. So one of the things that could end up happening is basically using ASIO as a really, really good reference implementation. I think that could hurry things along because Chris puts a lot of work into making sure that ASIO is backwards, forwards

compatible, works, is tested, builds across a lot of different platforms, and also simultaneously most closely models what the current understanding of the TS is. And so I think that, you know, we're going to be lucky in that we have that, but whether or not that actually hurries up the implementation by the vendors, really, that depends on the vendors. I mean, I think that, you know, what I just said has a counter example in that, you know, if there is a lag in providing ranges, right, ranges has a reference implementation and has for years, right? And, but, you know, it's, it's complicated. And if someone wants to implement their own from, from scratch, then that's going to, that's going to take a while. And the TS is, is also quite large and there's a lot of stuff to implement the same with executors.

So I'm not really sure what the answer to that is. The thing that I say in all my talks at the very beginning is that, you know I know that it's kind of, you know, not, not the coolest thing in the C++ world because of how weird C++ package management is. But like, if you want to use the networking TS right now, like just get used to typing boost ASIO and use ASIO. And you get even more than what the TS offers you. You get like asynchronous handling of Unix signals, for example, which is really, really useful.

And I use a lot, but that's not even considered to be in the TS. But then you have to deal with, you know, and actually you don't have to deal with downloading Boost if you don't want to, because a lot of people really don't like Boost. Because Chris also releases ASIO as a standalone. So you just get ASIO and that's it. And so you can use that. And that doesn't even go out and use like the Boost error codes or anything like that. It uses std error codes.

So you can just grab that and use that. And in one of the projects at work, we're actually using that version. We do use Boost pretty heavily in some of our projects, but in this project, we haven't pulled it in. And so we're using standalone ASIO and it works great. So I would advise people who aren't terrified of package management and dependencies to just be using ASIO until such time that the TS is mature. And I think it's a really it's a really luxurious place to be in because it's not, we're not depending on some language feature, right? That's the problem with language features, right? Like you can't roll it as a library. Whereas, you know, we've, we've had ASIO for over a decade. I don't know the exact length of time, but it's been around a while and it's, it's fairly mature. And to some degree, I think

that might almost be a counter argument against standardization. Like we have this thing, why do we need to go through all this stuff for to standardize it? It's like, well, because you really, really want the vendors to. Like we have this thing. Why do we need to go through all this stuff to standardize it? It's like, well, because you really, really want the vendors to be able to optimize this thing. Like you want them to be able to implement it. You want it to ship. And you want a common baseline for building everything from the ground up. Because networking is really fundamental to what we do these days. Like it wasn't in the 90s when C++ really became a big thing.

But nowadays, like if your program doesn't actually communicate with the network in some way like what does it really do that's a good point do you i know you know we're getting ahead of ourselves because we don't have networking yes but networking yet but do you see you know other parts of asio or possibly other parts of beast being standardized in the future on top of what will be delivered in the networking TS? Well, I think that like earlier, I was mentioning the discussion around the TLS component in ASIO. And that's something that I would really like to see standardized. I don't know about a lot of the other stuff

that's really, really handy in ASIO. I'm just kind of scrolling through the reference now to take a look. It's kind of platform specific, and that's always a real struggle to get standardized because you have to figure out how to describe what it means without actually saying what it means, right? Because you can't say, oh, this just uses this syscall because, you know, you could be implementing the standard library on Windows, but syscall doesn't exist. Or you could be implementing it somewhere where like the idea of that doesn't exist, right? Like a lot of embedded

systems don't have things that we take for granted when we're on our pcs right like you know i've heard people say like yeah the implementation of file system on my little microcontroller just always fails because there's no file system right um yeah which is something you don't really think of so like things that that are in asio that that could be standardized or things like um like ssl context and and stream which i mentioned. And it probably should be renamed because, right, and that kind of pointers something I said earlier, right?

Like when you standardize encryption, right, then you have to keep up with encryption, which is its own complete, mature field of computer science, right? Like, so we don't have SSL anymore. It's TLS, right? But the class in ASIO is still named SSL because we can't change the name of anything ever.

And so then you have other stuff in here that's really, really useful, but it's platform specific. Like you have like a bunch of POSIX specific stuff and Windows specific stuff. Like you have, for example, you have a class that's boost ASIO Windows object handle, which obviously only has meaning on Windows.

And if you're doing WinAPI programming, that's super useful, but that's not something that the committee would even consider standardizing. It's way too vendor specific. Okay. Well, Robert, it's been great having you on the show today. Uh, where can people find you online? Uh, the only really place that I'm at online is just reachable via email. I don't, uh, I don't indulge. I, I, I read Twitter occasionally. Um, but I never really feel like I have anything that I can fit in 140 characters that I just want to throw out there.

It's 280 now. Oh, it's 280? Jump in. Maybe I need to jump in now. All the things I wanted to say, actually, were 256 characters. So now the world has opened right up for me. Okay. Well, it's been great having you on.

Yeah, thank you for having me. Thanks for coming. Have a great day. Thanks so much for listening in as we chat about C++. We'd love to hear what you think of the podcast. Please let us know if we're discussing the stuff you're interested in, or if you have a suggestion for a topic, we'd love

to hear about that too. You can email all your thoughts to feedback at cppcast.com. We'd also appreciate if you can like CppCast on Facebook and follow CppCast on Twitter. You can also follow me at Rob W. Irving and Jason at Lefticus Thank you.