Two's Complement - Swift (with Doug Gregor)

Episode Date: October 12, 2022

Ben and Matt chat about the Swift programming language with special guest (and Swift creator) Doug Gregor. Doug teaches us a thing or two about Swift's design, and how it could possibly be a C++ successor. Matt rambles; Ben asks intelligent questions.

I'm Matt Godbolt and I'm Ben Rady and this is Two's Compliment, a programming podcast. Hey Ben. Hey Matt. So you know the other day we were chatting about languages and all the various things that are interesting to us about languages. And one of the languages we mentioned along the way, in fact, many of the languages we mentioned along the way, we don't really have any idea about. We kind of fussed around, said some things, made some words up um in particular when we were talking about um carbon there were some sort of magical properties i uh ascribed to other languages uh like witness tables i think was the magic words i use which i don't understand but um somebody contacted us and said you don't know really uh that much about the languages you're talking about do you and i'm like well no and um in particular swift we mentioned it very much in passing and uh and doug gregor who is uh well who's on the call with us and we'll speak to in a second said why don't i come and tell you

a little bit about swift because clearly you know nothing which is absolutely true yeah so we kind of called out to twitter saying, you know, hey, correct us if we're wrong. And someone called us on it. And here we are. So hi, Doug. Hello. Glad to be here. Very pleased you could come along and, yeah, try and divest us of some of our, like, thoughts about languages that we don't understand. So, yeah, we experienced them very much from afar. And now we've got somebody who can tell us a little bit about it. So you are, in fact, one of the original designers of the Swift language. So you're best placed to talk about this kind of thing, right?

Yes, I was one of the first people that worked on Swift back 10 years ago. And I'm on the language working group that guides the direction of the Swift programming language. Got it. So I didn't realize it had been around that long. Maybe that's just how old I now feel. But 10 years. How did it come about that you started working on Swift? What kind of led to Swift and your background feeding into it? My pre-Swift background is actually a lot of C++. I spent a good 10 years on the C++ committee. Worked on things in the library like std function, some language features like variadic templates, which I hope you've used. I have.

I'm a bit of an old school C++ programmer, but I have used variadic templates. Excellent. And then I was part of the group that implemented C++ support in Clang. Back when Clang was new and unknown to the world, we were implementing templates and all the major overloading, all the other major features that you need for a full C++ compiler. Gosh, gosh, those were indeed heady days. I remember that when GCC was the only pony in town, really. and so we were kind of stuck with it unless you were on a Windows platform or anything like that. And then suddenly

Clang came along and we realized, oh gosh, error messages could be so much better amongst many other things. They could. There's a lot of benefit you get from being able to start fresh, knowing what the best out there is, and try to improve on things. Right. And of course, Clang was written in C++, whereas GCC at the time, at least, was all C all the time. And so you had an opportunity to build something in the language you were writing for, which must give you some interesting thought processes about how things fit together. It certainly helps. You're using what you build every day, all day. Right. And you know where the weak points are, so you can improve on them.

The ultimate dogfooding, really. Indeed. So that was C++. And obviously, I know you from Boost libraries and things, and as you say, your committee work and whatever, but how did that lead into Swift? Right. So at Apple, which is where Clang started, we use primarily C and Objective-C as the language in which we write our frameworks and provide APIs for developers to write applications that run on the Mac and iOS and so on. And Objective-C is essentially like an object-oriented layer on top of C. It's very much inspired by Smalltalk. And it's an interesting language, but what we realized is we need something better. Right. And because there are things about Objective-C

that we really liked, and there were problems there. So being on top of C, it was not memory safe. Got it. And there was no way to fix that. Right. And there's many other languages. Many languages have tried to fix C's, you know, any number of Fortify macros and sanitizers, but ultimately it's a fundamental limitation. Yes, fundamentally it's a limitation and you really have to start over to build memory safety in from the beginning. And we also wanted modern conveniences and make it an easy language for people to start learning and write good code in. And so that was the genesis of the Swift project, to build a better programming language in which

to build apps and other programs that was safe, easy to use, but still had the sort of advanced type features that we needed to really build great libraries. Got it. So is it fair to say that there was a kind of domain specific need for the language, like, or rather the domain, sorry, the language was made to fit a particular purpose to start with? You mentioned applications specifically, which is sort of has a different flavor and feel to it than, say, just writing

backend servers or mobile webpages, something, something. Was there something specific about the language that was tailored that way? Not the language per se. So the language was always designed as a general purpose programming language where you could build high-level applications, but also go off and write servers and write more low-level things. It was intended to be a scalable language from the start. I would say the first community we went out to to say, you should really use this because it's so much better for what you're doing, was the application development community. Right. And of course, in the position that you were in, you had a set of folks who were already using Objective-C to do this because that was the the the easy on road to making apps in iOS and the like and so this was

kind of a drop-in so it was I know we talked about successor languages and you know it's Carbon being a successor to C++ and and you know in the email exchange we had before this you were like well what is what does it mean to be a successor but was is Swift the successor to Objective-C by your definition? And what is a successor language? Right. So you had me thinking from your last episode about what it means to be a successor language. And Swift is the successor to Objective-C, but that's not really a statement about the language.

That's more a statement about the language, that's more a statement about the platform. Okay. Objective C is a language in which Apple was writing its frameworks and building the SDK that you program against. This is what the UI frameworks are running, for example. And the platform in this context is iOS or all of Apple's products? So it can be anything. So it's all of Apple's products? So it can be anything. Yeah. So it's all of Apple's products. That's the Apple-centric view. But we use platforms all the time.

And they're written in a programming language. Now, most of those are based on C. So pretty much any operating system you use has got a POSIX layer. And the way you interact with your operating system is through those POSIX C headers. So C is kind of the lingua franca of all APIs that are tied into a platform. It's also unsafe and inflexible. And so one way you can view Swift as the successor to Objective-C is we want to use this to build our platform APIsis so it's a better way to interact with the system

and we can provide these these rich apis that are also easy to use and memory safe right so the way that's what i'm hearing that is that like it's excessive a successor to the uh the whole objective c would leave if you took it out right you're like we need something to plug in the gap here and in the case where you were this was the layer directly above the operating system um but it's not like it's not like i can take objective c and just change the extension from dot objc or whatever the extension is to dot swift and run it so it's not a success in as much as i can reuse code or is it is that how does interoperability work right so. So you certainly cannot just take Objective-C code and compile it in the Swift compiler and it works.

Instead, Swift is a very different language. So its emphasis is very different. We can talk about that, but it's much more in the realm of generics and value semantics and so on. But it does provide interoperability with Objective-C that goes fairly deep. So at the moment Swift became available, you could take any Objective-C API, any framework, and the compiler would automatically translate everything it had in it, every bit of API, and you can call it directly from Swift. So from day one, you could use everything that was there. And you could do so at a like file by file granularity within your project.

Okay. Okay. Got it. And the idea here is when you have this really tight interoperability, so I can write code in Swift that uses Objective-C and vice versa. So directly. And vice versa. Okay. Bidirectional as well. Okay. Yes. So you can also write at obc on some of your types in Swift and then they become available to your Objective-C code. Got it. This is a great path forward because it means that you can write your new code in Swift, but leave your old code in whatever language you have,

in this case Objective-C, and sort of incrementally work towards the safer, more ergonomic language as you rewrite things or as you pick up new features and do new work. So just from a point of view of what one might consider a success, that does seem like an important characteristic, is that you can, in order to succeed in language, you have to be able to use something that was there from before. And so it

sounds like you've got a story for for it there and the interoperability with objective c is pretty strong by the sounds of it again as you say you can do it bi-directionally that is i mean i guess that's what when we were talking about carbon and success of a c++ certainly when i was sort of internally and spitballing the idea of like what does it mean to be a successor having very strong interoperability so that you can you know so i can go to my manager and say hey i know we have this you know 250,000 line program written in c++ we'd like to try out something new where we haven't got these restrictions can i rewrite the whole thing and of course the answer will be no and then even when you start a greenfield project you know hey i'm about to start this thing i'm probably going to

spend two years on it. Do you mind if I try some random flash in the pan language I've just found on Hacker News? You know, those things are difficult sales, you know, for good reason. But in order to be able to sort of leg into it and say, well, OK, we're going to make the parser of the new parser for the blah part of my project. I'm going to write that in a new language. And then you have a way to incrementally improve upon it is a successor but that's sort of like a um i guess that's a sort of logical successor in terms of like we replaced the the niche that was there before

there's not like a spiritual successor aspect to this then i mean and maybe there isn't in carbon either i mean it's not we don't know enough about carbon to be able to answer the those questions you know some of the things are maybe a little more spiritual. But it sounds like in order to fix some of the things that you had in Objective-C, you mentioned memory safety. Presumably, you have to change the way you think about things. Yes, you do. And I mean, in many ways, Swift is very different from Objective-C. Objective-C is object-oriented through and through.

You create a class for anything that you want to do. Swift goes the opposite direction, in a sense. We like to use value semantics throughout. Value semantics are something that are very familiar to C++ programmers. We love our vectors and values. We miss them whenever we have to use any other languages, and we forget that everything else considers everything's a damn reference, and then we're like, why is it changing under my under my feet yeah but value semantics are very powerful right

and so they're very very they're core to much of what swift does so it we made it very very easy to define new structs and new enums and compose those together values max are very nicely composable uh we skew toward immutable or at least make immutable things as easy as possible. So the way you decode a variable usually is let. It's immutable. If it's value semantic, it can't change. So it's Lisp-like let. It's Lisp-like. This is defining this.

Yeah. Now, if you want mutability, you can use var, and then you have local mutability to just change that particular variable. So it's value semantics through and through. And that also goes for the standard library. So all the basic types, like arrays and dictionaries and so on, and also string. These are value semantic types backed by copy on write. It's copy on write. I was going to say.

So like, yeah, dictionaries, for example, you take a dictionary, you just get a new dictionary back by saying, you know, get dictionary and add this key and value to it and you get a new one but you know that's certainly when if people have waxed lyrical about immutability and languages and stuff that's the way they explain it to me and i say that's all great and wonderful and everything that's sort of the logical view but under the hood somewhere someone's doing something like copy or write for you and that had to be written using mutability but you're presumably hiding that away so So we are hiding that away. So the idea with Swift is we do have mutability. If you have a variable of dictionary type, you can go add and remove keys from it. And that's perfectly fine. Okay. Under the hood, it's doing copy on write as an optimization. So if you uniquely hold on to the thing that you're referencing, we'll just modify it in place and there's no problem.

Nice. But if you've made copies elsewhere, well, then you have to go and make a copy for yourself to mutate locally, and then that's yours to keep and you can mutate it efficiently locally. So, I mean, there's lots of benefits to immunability and programming for sure. Are there specific benefits for thread safety that you get for that? And I ask this because I've written a fair amount of Clojure in my life. And one of the wonderful things about Clojure is it's actually kind of hard to write thread unsafe code. Like you have to try. And that's really fantastic, right? Like you sort of like, you know, wind up in these situations where it's like, oh, I have this map over this function. Well, I'm just going to change

that to PMAP. And now it's just going to run in threads, and my life is nice now. So do you see that in Swift because of this sort of default to immutability? Yes, we do. So immutability is very good for concurrency there. Value semantics are also good. As long as you're making copies, you're perfectly fine, because your mutations stay local, where you have a local variable, and you are just copying across the wire. So we did recently introduce a concurrency model into the Swift language.

So there's things like async await. We pulled in the notion of actors, and we have current tasks running. And when you are working in this realm, we're using primarily value types or these actors, which is a kind of synchronized reference type, you don't have to worry about the synchronization problems because you're always working in different values.

This is actually something we're working towards in the Swift language. Over time, we're going to start enforcing in the compiler that the types that you share across your concurrency boundaries are actually these value type-like things that are safe to share. Yeah, that was gonna be my next question is, can the compiler check that for you? Be like, oh, you're passing in a variable here and that means that this won't be thread safe,

which means this other thing won't be thread safe, so no. Yes, and that is what we're working toward. So we call it data race freedom. It's one of the big pushes of Swift 6 in the future that we're working toward now. You can enable some of this checking already in the future that we're working toward now. You can enable some of this checking already in the compiler, where we will check when you send something from one concurrency domain, like a concurrent task,

to another one, is this something that is safe to send? We call it sendable. And if it's a value type composed of other sendable things, yeah, of course it's safe to send. It's just a copy. And if it's a reference type like a class that doesn't have any synchronization, the answer is no, you need to go find another way

to do this safely because you probably have a data race. That's pretty cool. I mean, so how does the, I mean, I'm always have a sort of performance focused view of the world through far too many years of staring at assembly code for obvious reasons. How does the performance of uh swift marry up or compare against um some of its you know contemporaries i

guess c++ you know if we're like maybe building an argument that swift might be a viable uh um successor language to c++ then one of the niches that you have for c++ is like an embedded machine with no resources or a huge cloud running millions of copies and all that you have for C++ is like an embedded machine with no resources or a huge cloud running millions of copies and all that kind of stuff. So how does it compare? Right. So Swift is a fully compiled language. And so it is generating machine code. There's a pretty powerful optimizer that understands the copy and write semantics. It understands the reference counting mechanism behind the scene. And it does a fairly good job in most cases. Now, one of the things you'll notice with Swift, if you're coming from

C++, is you have a little less control because of the memory safety, because we're doing reference counting to keep that memory safety. The compiler is doing some of this work behind the scenes, doing a lot of work to optimize it away but you can fall off a cliff sometimes and right sometimes you'll do something and it's like now i can't prove this i have to count it at runtime or whatever and that might be a surprise to you just like when yeah some optimizations you otherwise rely on c++ like in aggressive inlining you hit some threshold and suddenly what should be just return three is a huge block of code. So this exists in other languages, these kinds of engines. Yes, this exists in almost any language that has powerful abstractions,

where at some point you've abstracted so far. I was promised free abstractions. I want my money back. So in Swift, we are working toward an ownership model that will surface in the language that can help here. So if you see a case where the optimizer hasn't done exactly what you want and you want to take more control and say, no, no, don't make a copy here.

I want you to move this data down and make sure that this data has moved so there's no more copies, no more allocations, then you can do that. And so we view it as most of the time the optimizer does a good enough job, you don't have to think about it. And it's easy to work in this model. Sometimes you hit a hotspot and we'll give you the power tools to go ahead and make that

better without losing any of the safety guarantees. So that's the thing I was going to ask, because this sounds a little bit like, oh my gosh, I'm going to have to roll my sleeves up in rust or whatever and type unsafe. And then suddenly it's the Wild West again. So yeah, there are unsafe facilities in Swift. Any safe language always has unsafe facilities somewhere. And you can use them in Swift for this. What we're working on is a stronger ownership model

so that you have something that's safe but still gives you a lot of control for these cases. Interesting. So could I write a device driver in Swift? That's like usually the edge case where, you know, the even venerable C programmers finally justify the volatile keyboard as a keyword, sorry, as being an actual, no, this is what it's meant for. You could certainly write a device driver in Swift. You'll have to play around a little bit

to get the memory mapped IO kinds of interactions. And you will certainly be in the unsafe code territory. But yes, you can write it by Striver and Swift. You'll probably want to limit your usage of features like generics that are separately compiled in a different shared library. I mean, you would do if you're writing in C++, most of the time you will have like the bit that's actually touching the memory mapped IO. And then you very quickly you you add your own abstraction so it's not necessarily a deal breaker there but so you mentioned reference counting and i meant to i'm sorry i've gone jumping all over the shop here um anything that has reference counting uh i think you know python python has reference counts and they're great right up until you get a cycle and then you're like oh dear now

we need some other deus ex machina thing to come in and save me, aka garbage collector. Does Swift have a garbage collector? How does it avoid having one if it doesn't? Right. So Swift does not have a garbage collector. It is reference counting. And if you have a cycle, you've created a cycle in your data structure somewhere, and you should break that cycle in your data structure. So there are tools to visualize them when this happens to see where the links are. And then you would go in and replace a strong reference to something, which is the default, with a weak reference.

Right. And so you have a weak reference here that doesn't keep the object alive. However, it knows when the object goes away. Right. person and that will refer to the person if it's still alive and it will revert to the nil state which means there's nothing here when the object is gone and so it's a safe way to break up your cycles got it so like my classic example for thinking about this is like a doubly linked circular list that you might have with a magic head pointer or whatever and so that classically there you'll have the next pointer is the strong one and the pre pointer is like the weak one or something like this to sort of say well okay this is a cycle but what kind of overhead

uh is this because you know there are many ways to to do that sort of tracking how might it work you know i'm used to seeing like well there'll be 16 bytes for everything there's an eight bytes one way eight bytes pointing back the other way but clearly if there's something else going on there may be a reference count somewhere and there may be some other data structures that are allowing you to determine when the last reference goes and maybe nulling out a load of stuff? Or how does that work? Just humanly. Sure. So reference types are the things that are actually reference counted. And so reference types already have an object header that is a common layout amongst all the types. This has things like type information.

So you can dynamically ask the type of an object, are you this is used for things like dynamic casting c++ has the same notion right right in that object header is a reference count with information about you know what the current reference count is whether there are weak references to it and it's all so there's sort of a shared pointer like thing going on but it's actually in the object header. It doesn't have to be a separate block again, although most of the time you can combine them in C++. But because you know that every object has this header, you can put the information there.

Exactly. And it's always in the same place for all objects. So we can reason about objects abstractly. And because it's one set of operations that are quite critical for performance, we've optimized it with atomic operations throughout to make this as cheap as possible to do reference counting. Got it. Okay, that makes a lot of sense to me.

I mean, recently I've been spending a lot of time in the Python world, and I've been watching and reading how their counting and tracking and garbage collection stuff works. And so there's been some interesting thoughts there. Sorry, apropos of nothing, this is just more me go like i found this the other day it seemed really cool um there's a thing coming for python soon where they're actually going to have um only use a small number of bits to count the references and then it saturates and once it hits the maximum they're like okay there are just too many to track now it has to be garbage

collected later on which i thought was an interesting trade-off right if you think most of your objects have one two maybe three references to them you don't want to store a whole 32 bits or 64 bits so like that's what the reason i bring that up is because like what is the minimum size of an object obviously you need to have a weak count and a strong count which means at some point you've made a call of like what's the maximum number of incoming counts gonna be is it 32 bit is 24? Are you packing these things? So I don't remember the exact bit counts. I'm very sorry.

Sure. However, when we saturate, we will do a site allocation to track very, very large reference counts. So we can deal with that. Oh, I see. That's cunning. Because we can't just stop. We can't stop and fall back to a garbage collector

because there is none. And we can't just say this, you've hit some arbitrary limit. Now your object's never going to go away that would be utterly unreasonable for the programmers immortal immortal at uh 65 000 references or whatever yes that that does not work these things we should probably just stop tracking them well we can't because part of having our fencing system without a garbage collector is you get deterministic destruction ah there which is an important property for understanding how your program is going to work right that is a very interesting point there now obviously um having reference counts means that you say

deterministic it's deterministic when the last reference goes away but you know in that in the sort of standard way that when you have reference counting it's like when everybody owns it nobody really owns it. Are there ways of dealing with this? Things like files. Canonically in C++, you'd have like a file object that opens a file, then you know the destructor will close the file. And it's like, well, if I lend out references to other people by file ref,

that's on me. And when the object goes away, that's tough. But in things like Python, which have both, you kind of have to use these with blocks to sort of magically say, well, actually there's a scope that's not strictly tied with the lifetime of this object. Is there a Swift equivalent of that? The Swift language doesn't have a specific equivalent to that.

So this will often be handled by reference counting. Sometimes you'll just write a defer block if you really want to just close it at the end, no matter what, you can write a defer block to do that. I do think this will change in the future. So one of the pieces of the ownership work that we are doing in the Swift language involves the introduction of move-only types. And move-only type, you're controlling the lifetime. You know where it is created. You know where it's going to go away. And move only type. Okay. You would actually, you're controlling the lifetime. You know where it is created. You know where it's going to go away. And it's explicit if you're handing it off to someone else for them to actually close it.

Is it fair to say a move only type is effectively something which can only have a reference count of one while it's alive and then it's dead after that? So it's just got a reference count of one. And then you can make these assumptions because if you have one, then you're the one who's going to be taking the reference down to zero when when the time comes yes you can think of it that way but having a rough count of one implies that there's an object header that's storing that reference count which it doesn't this is all handled by the compiler it's statically known where this this uh move only value is this value of the moving type, is going to be destroyed. So if you'll forgive me for bringing C++ back into this again from this thing.

So every object currently is sort of a compiler-assisted shared pointer. So every reference, every reference type, a referenceable type, is a sort of compiler-enhanced shared pointer where you know some things about the semantics and you can do some optimizations and stuff. And then these movable types would be more like a unique point of where

you're like well i can make a reference to this and i can hand it around to people it doesn't live on the stack because it can't because i made my stack's going to go away but i can pass it on to somebody else as long as the compiler can prove that i can't see it anymore after i've moved it is that a fair thing or am i am i i think that's a fair analogy uh although except for the part about the stack there's no reason okay you can't put a move only value on the stack because you know when its ownership is being transported around and it could be returned and that's perfectly fine so i think yeah uh what this is is you know with unique pointer it has to be on the heap because it's allocated with new yeah maybe there's some clang passes nowadays that do some very, very clever analysis of like

memory elision and all that kind of stuff. But most of the time, you're just guaranteeing that you've put something on the stack. Right. So Swift has a deeper understanding of what's going on with the types declared on the stack. And so it will define it on the stack if it's needed, and it can move it back to the caller if it's needed, and it can move it back to the caller if it needs to move the value out. It's perfectly fine. The address does not need to be stable. And with the unique pointer, you're expecting that the address of the value is going to always be stable. And well, then of course it has to be on the heap. With a value in the Swift world, the address is not necessarily a stable thing. You can't reason about it from within the

language, except in very narrow circumstances where you've told the language, I need the address is not necessarily a stable thing. You can't reason about it from within the language, except in very narrow circumstances where you've told the language, I need the address of this thing, and you're not allowed to escape that address out to someplace else. Got it. Okay, that makes a lot of sense. So it sounds like it is as performant

as you would expect from a language that's compiled. There isn't any overheads other than the reference counting that if you wanted the semantics that the reference countable objects give you, you would have to have some way or another either through garbage collection or through your own reference counting like a shared pointer would give you but you get some acceleration from the compiler in this case that's so how does memory safety fit into this is is that because of these reference counts so the memory safety guarantees come about and and

the fact that objects can't die i mean how about things like arrays of things and not going off the end of arrays that kind of aspect of memory right there are certainly other aspects of memory safety and so yeah in addition to you know reference counting being the memory management strategy and not escaping pointers out or not even showing pointers into the user model all arrays and accesses are bounds checked. So this also includes things like if you have an optional value, you need to be careful about trying to get the value inside there

and deal with the case that there might not be a value there. Right. There's no kind of minus arrow. Trust me, I know it's there. Just interpret the bits there that an optional has. Everything is a dot at of the thing

or a dot get or a dot whatever. The thing that throws, I forget which one it is now, but like, it's like, no, we're done here. And then what happens actually? What is the, yeah, what happens if you do access an optional that isn't there? For sure. So the language makes it harder for you. So there's a lot of syntactic sugar around,

I have a value of optional type and I want to be able to do something on it, but only if there's a value in there. So there's some nice, there's this nice if let syntax that says, hey, if there's a value inside this optional, get me the internal value. And that's what's in scope inside the body of the if statement. And outside of it, you can't reason about it otherwise.

Outside, you have to reason about the optionality. A bit like the two-phase if statements that I would use in optionals in C++. It's just mandated. You know, where I would normally say, if the reason about the optionality bit like that so like the two phase if statements that i would use in options in c++ it said it's mandated you know where i would normally say if the thing and the thing is not empty open you know if auto the thing inside of it and the thing is not empty then i've got it got the actual object now in fact no you can't do that in c++ yeah forget i said that but yeah i i see what you're getting at c++ doesn't actually let you do this you can check is there a value in here? But then you always still have to write a star beforehand to say, I know there's a value in here.

Exactly, which is what I was thinking there. But of course, yeah, in this instance, you're saying, no, I want you to do two things. Either it's not there, in which case don't do anything, or it is definitely there, and give me the thing that it's referencing in one go. Got it. Yeah, exactly.

And there's some other nice sugar around it. So say you have a value of optional type and you want to call a method on it, but only if there's something there. So you can do this directly with saying, you know, X question mark dot and then call the function. And what they're saying is, well, if there's something there, go ahead and call the function.

If there's nothing there, you can't call the function. And then the whole result of that call is then wrapped up in optional. It tells you, did the function call happen? And if so, you can't call the function. And then the whole result of that call is then wrapped up in optional. It tells you, did the function call happen? And if so, you get the value in there. And if the function call didn't happen, then you just get nil. It makes it very easy to work with optionals in a way that's correct. Now you can force it.

There is a force expression. It's spelled with a postfix exclamation point. Big exclamation point. Be careful here. And that will do a runtime check to make sure that there is a value in there. And if there's not a value in there, it'll give you a nice error message that says, this is the mistake you made.

Here's the point in your code at which you did it. The debugger will jump there, and so on. Many programmers that use Swift just ban these things outright. They say, don't use the post exclamation point. My view is a little bit softer on that. I think if your program invariants are such that there must be a value here and there's no other reasonable way to write the code, it's fine to use it. And it's documentation that this isn't an invariant. Right, right right right yeah you're not expecting it to be not there but for whatever reason you took an optional as a parameter or whatever it's an optional field in a class or whatever but you

said like i've just checked the enum value that says this is of some particular type and i know that this must have a price or this must have a quantity or it must have a whatever and so as you say it's documentary you're like no it's actually a program this is an assertion here it's an assertion saying that something else is wrong so talking of errors um that's another sort of like place where often languages can have quite strong opinions uh c++ sort of has uh a split brain about whether exceptions are great or at least the community has a split brain about whether exceptions as they are written are uh are uh the way forward how does swift handle uh unusual i mean i won't even say exceptional situations because sometimes they're not but how does it deal with the equivalent of exceptions errors and the

like right so uh errors are they're part of the language and you can mark a particular function as being throwing so if you say it throws, then it can produce errors. So in that sense, it's like C++ in that you have functions that can throw and functions that can't throw, but the default is flipped. So most code in Swift cannot produce errors. It just does its job and produces a result value. Are you perhaps intimating that C++ has a wrong default? That seems very unlikely. I would never say such a thing but you can infer okay all right i don't want to put words in your mouth um so you can opt in to my function can throw when you do so of course the caller must handle the thrown error and so you can do this with a

do catch block you can do this with also making your function throws. There's a couple of other options you have there. But one of the things that Swift did that's a little bit interesting is we mark all of the places in your code at which you do something that could throw. So if you're in a function and you call some, you know, open a file function that could throw an error because it wasn't available in the file system, where you have that open call, you mark it with the try keyword. And it says, here is a point at which there might be an error thrown. And it needs to be handled somewhere else. But it marks in your program logic the point at which you have to think, wait a minute, this is a point where I'm going to get some possibly interesting control flow that jumps out of my function or jumps out to a do catch block. So that sounds an awful lot like a Java checked exception. Am I misunderstanding that?

I think you are misunderstanding that. So with a Java checked exception, I don't think you, you don't mark it in the source code of where it actually happens so you say in the function you say here are the here are the exceptions that i might throw and then the compiler does a check that a function that you call inside there doesn't produce an exception that isn't covered by that set of exceptions but you don't write anything in the source to say this is the call that might throw a particular error fair Fair. So I think what you're saying, Doug, is something like you'd say let f equals try open file blah. So you have to put this try keyword actually in the expression that could cause an exception to be thrown

every time that you call a function or at any point where an exception could be thrown, you have to kind of tag it in this way, which was like a huge criticism of C++ is that like effectively it's an interview question where are the points where an exception could be thrown in this innocuous looking piece of code and it's like well everywhere a hundred times right and so is that the is that what what you're doing in that instance as well as marking at the function level i am a function that could throw and if i'm calling a function that could throw,

I have to mark my function upwards as like, well, I also could throw unless I'm handling it. In this instance, even when you call something within that function that could cause an exception to happen, you have to sort of tag it with a like, I know this part of the expression could throw. Right. Am I right there? That's correct. And the reason is you can think about this like a return, right? What does return tell you the reason is you can think about this like a return. What does return tell you? It tells you at this point, my function is going to stop and execution is going to continue for my caller. Try is saying at this point, instead of going on to the next line of code, it's going to jump out of this code block and maybe it'll end up in a do catch, in the catch block somewhere. Maybe it's a throws function and the error is going to propagate out. But this control flow, especially in the error case,

which is the hardest one to think about when you're writing code, it's a reminder that at this point, you've got some control flow out. So you should think if you've tried to start setting up an invariant, but it's not in a good place, if this fails, well, then you better put

a do catch block around it or rework your invariant but it's not in a good place if this fails well then you better put a do catch block around it or rework your invariant in some way that makes sure that you're okay along this along this path it might jump out of the function or this bit of code that may i mean from my point of view i think i've having spent so much time realizing that the exceptions can be thrown in weird places and that that matters in c++ i can understand why putting that in is there but maybe uh you know in the garbage collected languages like Java, it doesn't matter as much because some of these invariants that are

handled by the garbage collector later on, it's like, well, I don't care if I made a new object, it's going to go away in the end of the time. Yeah, I think there's probably one other thing going on here, which is that thrown errors are fairly rare in Swift. So if you think of C++,

bad alloc can happen to you anywhere at any time, and it makes it impossible to say which calls are going to possibly throw that exception. In Java, you have the null pointer exception. These notions don't exist in Swift, and so there aren't systemic problems that can happen to basically any line of code anywhere. It's someone deliberately made this thing such that it can throw an error

because there's an underlying reason for it. And so the cases where you actually need to mark something as try aren't all that common, which keeps them meaningful because they're rare. You know, it's something you need to think about. Right. It's not like every third line of your code has to be trying something, which it would be if it was like, say, the equivalent in C++ or even D, other languages like Python that use exceptions as control flow in some cases. Right. So that makes sense. So is there a sort of more standard way of file opening is a classic one

because sometimes opening a file is exceptional. It's like, well, I know know the file's there so it's an exception if it's not there but other times it's like i'm opening the file because i'm not sure if it's there or not and i don't want to do a round trip and have the race condition of what if it's not there afterwards right so it means different things to different people how how does swift sort of communicate the non-exceptional failure type is that optional is that something else so often it's optional so I'm not sure I would do it for opening a file Because you probably want to provide no why it wasn't there. I want more information about what happened But often it's got it. You know either I can do this or there's nothing there and optional is used for that So it's fairly common to have an optional

I'm a function return an optional if it can't do something if there's no value there already and the fact that optional is sort of syntactically sugared throughout the language makes that very easy to work with a convenient thing to do right got it that makes sense well as i keep asking c++ centric questions i feel poor ben is left sort of out so i'm gonna throw throw him a very characteristic bone and he's gonna hate me for this but like how do you have like uh is there like a uh an appropriate uh sorry a very swift specific test testing sort of framework a lot of languages have uh the newer languages like rust whatever will have like a way that you do testing inside the language like it's bound up there's like cargo test is a thing you could just

do you write code in a particular way with a magical thing, and this is the test for that bit of code, and that's great. How does testing fit into Swift? Sure. So most testing goes through the XCTest library. So this is something that's been around for a while. It's actually inherited from Objective-C, but I think it's sort of this spiritual lineage goes back to CPP unit and JUnit for the style of testing it does.

But yeah, essentially, you write separate test targets, and you just write a bunch of test functions they have set up and tear down and can provide assertions that values match. There's a fairly rich API there for writing tests, and it tends to be integrated in the IDE fairly well. So it's easy to go and rerun one given test or run your whole test suite and see what happened there. So one of the unfortunate things that happens whenever you're building a platform for somebody else to use, whether it's a standard library for a programming language

or just some other library within a greater ecosystem, is that you can wind up with these kinds of platforms and libraries that are not designed to be tested. So as a person writing application code, it's very difficult for me to write my test because of the way that you designed your library, and not for any particular essential reason, just because it didn't happen that way. So is that something that you're thinking about as you design some of the libraries around Swift and the standard libraries? Think about how would a person who's using this test their own code in a way that doesn't require them to spin up a database in order to make a function call? Oh, that's

interesting. So I tend to stay on the lower level standard library and compiler side of things, where the functionality tends to be very narrow inputs and outputs and is very functional in nature. So this concern doesn't come up so much. But for folks that are working in the application space, they often have been using Swift's abstraction features around protocols to allow for things like mocking. So instead of using a specific structure class somewhere, they will write a protocol that describes its essential characteristics. And then in production, they put in the real implementation with the server backend or the database. And then in their testing, they can drop in a different implementation of that protocol that works on their internal local database or some testing data that's easily predicted and understood. Yeah. I mean, this can absolutely happen with

very low level APIs. I just want to write to a file, but I don't actually want to write to a file because if I actually write to a file, then my test runner is going to clutter my machine with a bunch of temporary files that I don't need. So I want to pass in a fake file, mock file, perhaps, or some other thing that I do. But if the API doesn't let me do that, it's like, no, no, this has to be an absolutely real file. Otherwise, you can't do it. Then I'm stuck writing to the file system and then writing a bunch of other code that's going to clean up the temporary file that I wrote. And then what happens if something in that fails? And then I get something, this weird state. So I mean, is that something that you have considered at all with some of this stuff? Yeah, I mean, generally,

when I work with something like that, I would try to have the virtual, the file system virtualized, so that we can work on it. Generally, it's a good idea anyway, with something like a file system, because you will be dealing with different file systems at some point in your life. It's not all going to be just whatever the POSIX APIs give you. And so in general, I would say that the way you would write that is you would write it abstracted over a file system. Here is the implementation of my file opening and writing some data and closing it. Generally in Swift, you would do this with probably a form of generic programming. You can write a lot of things in terms of protocols and generic algorithms that operate on the protocols for things that can be higher level. And then your particular implementations will have to target something like a file system, an actual file system.

And maybe you build a sandbox version for your testing. Okay. One other question I had not related to testing, and this is maybe looping back to the thing that sort of spawned this originally is like, you know, we've been talking for a while now about like all these really fantastic features in Swift. And if I was an Objective-C programmer, which I am absolutely not, but if I was, I can imagine that I would really be interested in taking my giant legacy Objective-C code base and starting to introduce Swift. And you sort of had like a great story for that in terms of like being able to,

you know, mix and match the source from both of them and operate cleanly. Do you have a similar story for C++ either today or like as you look forward into the future? Like if I have a C++ code base and I wanted to introduce Swift into it, what does that look like today and what do you want it to look like? Right. So today what it looks like is you would probably wrap up your C++ interface in a C interface and we can access that directly. However, we are actively working on C++ interoperability for Swift. So there is a work group that's active on the Swift forums. It's part of our general open evolution process that is working on Swift C++ interoperability. And they're working through, what does it mean to map a C++ class or method or template over into...

Or template, right. Template's always exciting. Over into Swift and use that from the Swift language. And is it ergonomic to do so? Do many C++ APIs come over well enough that you can go ahead and use them directly from Swift? Or at the very least, write some clean little wrappers that make it easy to work with those APIs? And again, with the other direction. So have your Swift library and export from it a C++ interface that you can go and use in the C++ side of your code. And the vision here is to do basically the same thing for C++ that we ended up doing for Objective-C,

where you have this tight integration that makes it easy to do sort of file-by-file adoption of Swift within an existing C++ code base. That's a lofty but impressive goal. I mean, that's kind of what was the USP for me with Carbon, amongst other things, was this intention, stated intention. Obviously, Carbon is a lot of things. It's not a lot of things at the moment. It's a lot of ideas. But to have that very strong interrupt, the ability to instantiate a C++ template inside another language and be able to code gen and do all the hairy things that need to go with that is kind of an appeal

when you're trying to move a code base from one to the other. But if you're planning for Swift as well, that opens up the door and definitely means that Swift is at least as good a successor to C++ as the mostly vaporware carbon at the moment. So that's an impressive thought. We hope to make it nice and ergonomic. It's been really interesting. We've focused a lot on containers.

You have a container like a vector or your own container. We want to bring that into the collection model and let you use Swift's for each loop on it and use all of Swift's collection algorithms on it. And it sounds like a small thing, but the C++ iterator model doesn't exactly match the way that we iterate through things in Swift. And so the C++ Interoperating Workgroup is tackling all these problems, figuring out how to map things, and then testing it by writing new code to see how it works. How does one even think about doing

ownership in those worlds where, you know, you've already described to me, if you're going to have a reference to something in Swift, then you've got this object header that's going to be there, which clearly isn't there on a vector of ints or whatever. So that sounds like a really difficult and challenging problem to solve. It is. So there's a couple of solutions. So C++ is interesting because it has some value types and it has some more reference-y types. And the value types, you know, map over to Swift value types, and those are fairly easy.

You're dealing with values and so on. With the reference types, it's interesting because we have to know which things they are, and you don't have a normal reference counting scheme. So we have to do something that maintains the model of it's a reference type, but ensures that either we can handle it safely. So if they do have a reference counting scheme, we can tie into it and use it. We've done that in some narrow cases. If they don't have a reference counting scheme, that's okay. But maybe you have to refer to this through code that is unsafe

when you're dealing with this thing. We can't control the lifetime. We don't know the lifetime because the pointer has escaped and it's a C++ type without any rough count. I don't know if you've ever looked at how PyBind 11 works, which is this template library for binding bidirectionally Python and C++. And it kind of does a lot of stuff behind the scenes where it's like, well, Python, obviously, everything's reference counted. And it can expose a C++ reference counted object as, well, OK, this is in the same thing. If the reference goes up and down, I can add ref and remove ref, whatever. But sometimes it has to put these interposed objects that are like, well, this facade onto the real thing and it's kind of an option of that and if i think the other thing's gone away in c++ land then i'm afraid it's not here anymore although you can

still count how many references this object has it's just unfortunate that it doesn't point at anything anymore um these are difficult problems i mean it's uh i'm really interested in how this is going to pan out, but it does genuinely sound like C++ has another potential contender in the arena for successor. Well, if you're really, really interested, the C++ Interoperability Workgroup for Swift, it's open. We are on forums.swift.org where they are doing their discussions. And I think they have a roadmap document to lay out. These are all of the problems that we're looking at tackling in the rough order in which we're doing it to make this C++ and Swift interoperability layer work.

And we're using it now. So we actually use it in the compiler so that we can write some of our compiler in Swift and some of it in C++ and really try out these ideas and see how well they work. Oh, neat. Yes. So you are dogfooding it.

So I think I was saw, I happened to notice that one of the more recent changes to Swift was to have a parser for Swift written in Swift. And so you're starting to migrate and share C++ and Swift even in the Swift compiler itself. Yes, and we have some optimizer passes that are also written in Swift and use this C++ interoperability mechanism.

It's kind of our playground. So we can go improve C++ interoperability, see how much nicer the code gets, and then keep dogfooding and iterating to make it better. Nice. Well, we're running a bit low on time, but I do want to ask one thing because I did throw a statement

that I had just copied off of somebody on Carbon in being in conversations with them, which was what is a witness table? And can you explain it to me in two sentences? That is tough. So a witness table is the runtime representation that says how a type you have, like an array of integers,

satisfies the requirements of a protocol like collection. Witness tables are used for separately compiled generics in Swift. Right, right. Yep. And they are also used for what we usually call type erasure in C++, where you know that you have something that is a collection of integers, but you don't know what the type is because it could change at runtime. Right. Witness tables also have a big impact there. So in my mind's eye, when we were talking about it in the carbon world or whatever,

that was kind of what I was thinking. It's this sort of like little bundle of like, well, this is the layer that allows me to adapt something to fit a, like essentially a virtual function table but not explicitly defined in the class but separate so that some other piece of code can patch them together such as when i'm writing a type erasure by myself i have to unfortunately go through and manually write all of the protocol myself and then adapt it to a template type or whatever to say hey this is one of those and the virtual table analogy is a very good one because it's the same idea, but separated from the object. And again, we also use them for our generic system

because we have separately compiled generics. And so you have to pass the witness tables through to say, does this type actually conform to the protocol? Oh, you see, this is making me think more about other cool questions that I want to ask you, but we honestly don't have time, which is really unfortunate. But it does sound like we've got a viable C++ successor in Swift. Certainly when the interrupt story improves with C++, it sounds like it's a great choice for almost everything

that you would use for C++. In fact, not almost anything you would write in C++ Swift. Is there like a single phrase, unique selling point for Swift that you would like to sort of proselytize or give us, leave us with? Hard to summarize that. So that's the toughest question yet. Oh, I put you on the spot there. I realized I should have asked this before. It is. what is the two-seasons elevator pitch for Swift so I think Swift is there to make programming fun and easy and help you build really good code from the start and support you in doing that cool sounds good to me fantastic well thank you so much for your time and for taking up us on our challenge of like telling us where we're wrong and we were very wrong about Swift, it turns out. Yes.

I didn't even know enough to be wrong. So thank you once again, Doug, for coming in and talking to us today. Yes, it was great. Thank you for having me. It was great talking with you. Thanks a lot. You've been listening to Two's Compliment, a programming podcast by Ben rady and matt godbolt

find the show transcript and notes at twos compliment.org contact us on twitter at two cp that's at t w o s c p Theme music by Inverse Phase.