Algorithms + Data Structures = Programs - Episode 159: The Roc Programming Language with Richard Feldman (Part 3)

Episode Date: December 8, 2023

In this episode, Conor and Bryce conclude their interview with Richard Feldman about the Roc programming language!Link to Episode 159 on WebsiteDiscuss this episode, leave a comment, or ask a question (on GitHub)TwitterADSP: The PodcastConor HoekstraBryce Adelstein LelbachAbout the Guest:Richard Feldman is the creator of the Roc programming language, the host of the Software Unscripted podcast, and the author of Elm in Action from Manning Publications. He teaches online courses on Frontend Masters: Introduction to Rust, Introduction to Elm, and Advanced Elm. Outside of programming, he’s a fan of strategy games, heavy metal, powerlifting, and puns!Show NotesDate Recorded: 2023-11-13Date Released: 2023-12-08Software Unscripted PodcastThe Roc LanguageDenoRust YewThe Roc Language TutorialIntro Song InfoMiss You by Sarah Jansen https://soundcloud.com/sarahjansenmusicCreative Commons — Attribution 3.0 Unported — CC BY 3.0Free Download / Stream: http://bit.ly/l-miss-youMusic promoted by Audio Library https://youtu.be/iYYxnasvfx8

Not too simple for somebody who has a lot of experience programming, but I think also if I was even just learning a programming language or if this is my first programming language, I think the tutorial's well laid out for a newcomer. Wow, thank you so much. I spent a bunch of time on it, so it's great to hear. welcome to ADSP the podcast episode 159 recorded on November 13th 2023 my name is Connor and today with my co-host Bryce we conclude our conversation with Richard Feldman about the rock programming language interesting I don't actually know if that downside does apply in rock so well let me describe the other thing which is the big idea of rock and we'll see if it does or does not well so first of all i actually i'm not sure to what extent people would want to use rock for gpus anyway but setting that aside um so the big idea of the language is what we call platforms and applications. And the basic idea is that every rock program has exactly one platform and exactly one application.

So if I'm going to build an application, like what we traditionally think of as a program, um, could be a library also, I suppose. Uh, but basically I'm going to pick a platform to build that application on. So let's say, for example, I'm building to pick a platform to build that application on. So let's say for example I'm building a command line application. I would probably choose a platform like we have one called basic CLI which is just a platform for building command line applications or let's say I want to build a web server. That's my application. I would pick a platform that's designed for building web servers like we have one called basic web server. Let's say that I want to build a native GUI desktop application. Again, pick a platform that's sort of good at that. So in some sense,

a platform from an application author's perspective feels like picking kind of like a framework. It's like just a starting point. It's like this is something domain specific. It's like this is going to give me the tools that I need to build whatever thing I'm going to build. What's different between Rock platforms and your traditional framework is what is in scope for them. So traditionally, in most languages, C++ included, in your standard library, you have a bunch of IO operations, as well as data structures and math and stuff like that. And then if you're using a framework for something, it's kind of like as a nice to have on top of standard library stuff. It gives you sort of like more that's like maybe domain specific. And also your memory management stuff and like C++ is all going to be kind of

separate. You can do that however you want. Now in Rock, what's in scope for a platform is, yeah, it's all the domain specific stuff. So like a web server is going to have a way to do like HTTP, it's going to have a way to do, you know, handle requests and turn them into responses. But also the platform is responsible for providing all of your IO primitives. So the rock standard library doesn't have any IO in it at all doesn't even know how to do it. It's just like, it's just 100% data structures. So all that comes from your platform. Separately, the platform is also responsible for memory management so if i were to write a platform like for i'll give the easy example of web server the basic web server platform so rock does automatic reference counting that's kind of the

default memory management system but the platform is actually comprised of two pieces one is the sort of public facing Rock API. But the other part is something implemented in a lower level language than Rock that sits beneath that, that is responsible for actually implementing the like IO primitives and basically all the like low level stuff that you would normally associate with like a language runtime, including memory management. So the way that that works is essentially, if I'm writing my rock application, every time that application needs to allocate some memory, the rock compile

is going to generate a call to a function that's implemented in the platform somewhere called rock underscore alloc. So it's basically like malloc. And whenever the reference count gets to zero, it's going to call rock dealloc, which also is implemented in the platform. So here's an example of what's cool about this. Let's say I'm building on the basic web server platform. I happen to know that basic web server just does kind of your traditional malloc and free. So it basically just feels like a normal automatic reference counting system. Fine.

There's a work in progress web server platform that is planned to be API compatible with that basic web server one. So if you've built something on top of the basic web server platform, you could just try switching to this and see how you like it. But its main selling point is that instead of using malloc and free, it bump allocates on a per request basis. So if you have a request handler, your request comes in, that entire request is going to be doing bump allocation into an arena that's specific to that request for the entire duration of the request. It's not going to free anything. And then once you get to the end of the request and you've sent the response back, it's just like, cool, we'll just reset the arena

and then move on. And you also set, you say like, here's how much memory I want to allocate to each request. Each request gets this big of an arena. And if any individual request exceeds that, then you get a 500 error and nothing else happens to the rest of the in-flight requests. So this has like a number of cool properties. One of which is obviously it's, we assume, going to be faster than like managing a free list and stuff like that. Like bump allocation and deallocation is like about as fast as it gets in terms of memory management strategies for heap allocations. But secondly, it also means that you get this really nice granular control over memory limits on a per request basis, rather than, for example, saying like, you know, this entire server has this

memory limit. And if we go over that, then, you know, maybe like something bad happens to the other requests that are in flight, etc. It's sort of very narrowly scoped and sort of isolates. If something is misbehaving or using way more resources than we anticipated, it's really limited to just that one request handler and nobody else's experience is affected. So that's something that I don't know of any other automatically memory managed language where you can do something like that. When, when, um, uh, can you write a rock application where like, do you write code that is platform agnostic? Um, or do you write code specifically to apply? Well, if I'm writing an application, it's yeah, it's going to be for that particular platform. Um, but I guess the better answer is you can do both. So certainly we expect libraries in the ecosystem to be almost all platform agnostic.

We have a plan, and this is something that's not implemented yet, but a design, I should say, for how to make it so that you can write platform agnostic I.O. packages, like things that depend on I.O. but don't depend on any particular platform's implementation of that I.O. They just say, like, I depend on IO, but don't depend on any particular platform's implementation of that IO. They just say like, I depend on HTTP. It, you know, we have a common understanding of what HTTP IO means across multiple platforms. And as long as you're using a platform that provides that, cool, you can use this package for, you know, error reporting or something like that. So we have a plan for all that, but that's not, that doesn't exist yet. That's still the designed but not implemented stage. So the cool part about this to for all that, but that's not, that doesn't exist yet. That's still the designed but not implemented stage.

So the cool part about this to me is that it means that as an application author, I get a user experience that's quite familiar. I'm like, oh yeah, okay, I choose this. It's not called a framework, but it's, you know, something similar to that, that like it's domain specific and I'm going to use as my starting point to build on. Maybe it feels a little bit weird that my IO primitives don't come from the standard library, but rather they, you know, come from there, but like, whatever, that's not very different.

It's just like, where am I importing them from? But then you just sort of automatically get these cool benefits in terms of like, you know, memory management. And another really interesting one, which is a whole long tangent is security. So one of the things you can do with this is somebody can implement a platform called, let's call it like SafeScript. And basically the

idea here is it's like, this is a command line application where the feature is anytime the rock program that you're running on this platform tries to do any kind of IO, it will prompt you, like say, hey, this rock program wants to read from slash Etsy slash password. Do you want to allow that? Yes or no. Um, this wants to write to this folder and, you know, on your, on your hard drive. Do you want to allow that? This wants to do this network request to this URL. Do you want to allow that? Um, and the purpose of this would be, I'm sure you guys have had this experience, but like, you know, you download some program from the internet and it's like, Hey, this is going to do a convenient thing for you. And I'm like,

cool. Or it's going to destroy my, you know, computer or turn it into a bot or like, you know, install something that I don't want installed. Um, there's, there's really no nice way to verify that like something that you download from the internet is not doing something malicious. Like kind of the state of the art is like either like run it through some sort of third party checker that hopefully will tell you whether it's legit or not or inspect all the source code and the source code of all of its dependencies yourself, which is like nobody does.

So the cool part about this is that if you have this, a platform like that, you can verify something you downloaded from the internet just by looking at like the one line of like what platform is it using. And if I know that that platform is the one that does the prompting, then I'm like, cool, I don't have to trust this thing because it's going to be like I'm running it in the browser where like anytime it tries to do anything that could possibly harm my system, it's going to prompt me. And because the platforms are in charge of 100% of IO and there's no workaround for that there's no like escape hatch that lets you get around that in the language I know with 100% certainty that it's like yeah if it tries to do anything malicious like I'm gonna

get a prompt and I'm gonna find out about it I'm gonna have to approve that before it can do the thing um that's also something which I don't know I think the closest I have heard of any language doing something like that is Deno and uh you you know, again, without going on a big tangent, like they don't really do it in that granular of a way. Interesting. Yeah, it's interesting. I've heard this on your podcast because you've talked about people that are more familiar with security. And, I mean, it sounds like it's offering, like Rock is going to be offering more than like any other language in this space in terms of... Was that your motivation for the platform's concept?

Like what was your original motivation for the platform concept? That's a great question. So it actually wasn't security at all. I just kind of realized that partway through. The original motivation was actually just that, so I'm a big fan of the Elm programming language. Before I got into Rock, I spent quite a few years doing Elm and really loving it. I wrote a book about it called Elm in Action from Manning. I taught courses on it.

I spent more than 100 hours personally standing in front of classrooms teaching beginners elm um and uh the reason that that like i felt motivated to start rock was basically wanting a language that had an elm-like feel to it but which could be used in sort of the long tail of domains like elm is a very domain specific language it's like really focused on like building browser-based uis and if and when it expands to other domains evan's approach is to like, look at that domain and try to answer the question, like, how can we do a really, really great job, like, you know, supporting this domain in this language, and that could potentially mean making language changes to do that. And so there's naturally this sort of long tail of use cases that Elm is never

going to address, like Elm itself. Examples that come to mind are things like editor extensions. Like I was using Vim for a long time and I was always like, I would like to write my own extension, but I don't want to do Vim script. Like Vim script does not have a good reputation in terms of like user friendliness. I would really love to use Elm or an Elm-like language for that, or like database extensions or, you know, robotics, making a clock. Like there's just all these things where it's like elm's never gonna get into that domain and so i wanted a language that could try to recreate the elm like experience but for all these domains one of the things that

i think is key to the the niceness of elm is that for a given domain elm is like look here's all the primitives that make sense in this domain. Like you don't have, like in the browser, for example, you don't have like file system stuff like Elm just doesn't have any file APIs because you don't have files in the browser. In contrast, like I know that you can use Rust in the browser, like there's a package called u-y-e-w that is for like building user interfaces on WebAssembly and Rust in the browser. But of course, like in the Rust standard library, there's a whole module on file system stuff. Like if I'm depending on a third party library that does a bunch of file system

stuff, I don't even know what's going to happen if I use that in the browser. I also don't even know which of my dependencies like are going to do that or are going to rely on that functionality being there if i try to use them as dependencies in my browser-based project um so the whole ecosystem in uh in in most languages is sort of let's assume that like everything is available even though it might not be i have similar questions about like database extensions like if i'm writing a database extension i don't want that thing to be doing arbitrary network requests or like HTTP in the middle of my database. That's that's like not good for such a performance sensitive environment.

So and likewise, you know, if I'm running like a little, you know, servo thing on a Raspberry Pi, maybe it's not connected to the Internet. Like what if I'm using a dependency that relies on doing a network connection. So basically, I really wanted to try to recreate that sort of domain-specific focus while still addressing the long tail of domains. And this was the solution I came up with, was just to say each individual domain, somebody can create their own little Elm-like experience in that domain using a platform.

They can say, here are the primitives that are available that I think make sense in this domain, in this use case. And, you know, here's the memory management strategy that I think makes sense for this use case. And, you know, they can sort of be in charge of that. And as a user, I don't necessarily need to think about like, oh, what can't I use here? It's just like, oh, well, just I picked my platform

and that's all the stuff I can use is there and all the stuff that, you know, doesn't make sense in this domain just isn't available. And so there's not really anything to think about. So, but like, so you can only use one platform at a time. Yes. Everybody asks that. Okay. But what if I want to use two platforms? So, I mean, the short answer is that it's actually fascinating to me that this is like a ubiquitous question. Whenever I describe this, everybody's like...

So actually, a common way that people phrase it is like, can I compose platforms? So let me describe how platforms actually work under the hood. And then I think you can maybe see why the question doesn't doesn't have like a reasonable answer so literally what's happening is that rock itself actually compiles down to basically the equivalent of a c library like just an object file um it just exposes a bunch of functions that's it rock is not in charge of like main or you know start in the like in the binary um rock is just like here's a bunch of functions the platform is actually in charge of main the platform is like i have you know my c program or my rust or whatever you're using to

implement the low level stuff um and you have main and then at some point you call the rock function and the rock that that sort of kicks everything off from the rock perspective so another use of rock by the way because of this is like you can embed it in like a very large other project. So you could use it as like a Lua alternative and like a game engine or something like that if you wanted to. But because of that, it's like, well, how would you have multiple platforms? That's like saying like, how would you have multiple mains in an executable? It's like, well, you can't, there's like, there's only one starting point. That's the whole point. So because the platform is in charge of all that stuff um it just doesn't really make sense to have more than one you know competing executable inside your one executable that makes sense but like what if i so like

like you mentioned uh a command line platform in a like a web server platform right like what if i wanted to have some sort of command line tool that has a web server inside of it? Would I need a separate platform for that? That's a good question. I mean, so not necessarily. It just kind of depends on how you want to have things set up. So for example, in our command line tool,

we do have TCP socket support. So just using that, you can just build a web server on top of that in pure rock if you want to there's nothing stopping you from doing that what you won't get is you won't get the like bump allocated per request experience that you get if you get like a um uh yeah but now another thing to note is that depending on what you're using it for like so actually i actually have run into that at work before um that that exact use case and it was specifically for doing um like an oauth authentication flow where you needed to like stand up a web server locally and then the thing you're authenticating with would go hit that um but i think there's actually a pretty

good reason that you wouldn't just want to always just use this you know command line tool that um like in addition to missing out on the potential um you know bump allocation benefits on the request handlers um which is that like i kind of don't want my web server typically if it's if it's mainly going to be a web server to have the option to like read from standard in and like block on that that's not good like that's not that's not going to do anything they're not not going to do anything good um that's something that totally makes sense for a command line app but like for a web server it's like if you're ever reading from standard in something

has probably gone wrong because there's not going to be a keyboard attached to that thing most likely yeah okay that makes sense to me you mentioned this before but it's worth highlighting again that you did say that there's going to be a facility for writing libraries.

I don't know if you want to call them like platform generic libraries. Platform agnostic, yeah. If you're trying to write something generic enough that it should be able to work across a certain number of libraries, there's going to be a way or a number of platforms. There's going to be a way to do that in the future. You're not going to need n different libraries for n different platforms, which is, I think, the important part. Because a lot of the times applications, that's less when you care about like generosity versus when you're writing a library. You don't want to have to rewrite that library every single time you want a new application on a different platform. Yeah, totally.

And like we have a really, that part of the design i'm super excited about it's just like it's not it's not the top priority to implement it right now because right now we're more in the like hey let's make sure that people can have a good experience like writing on the one platform there is no real ecosystem yet to to like to make you know platform agnostic um but but yeah like i i am very excited for once we do implement that feature because I think the design turned out really, really well. We could go into that, but I think we're already probably over on time again. I was going to say, I mean, we talked a lot about platforms and the continuation monad, a.k.a. task chaining IO stuff. Is there any, like, in the last couple minutes, though, if you just want to rapid-fire ecosystem or language stuff for people to be excited about if it exists or

will exist, is there going to be a package manager? Yep, that's the plan. So you already can do packages using a URL thing. This is something I think, instead of rapid-firing, I'll just mention this one thing because I think it's pretty cool. So the way that you do packages today, we don't have a package index yet, like a searchable way to like discover them but you can

already just like automatically download packages and we have as far as i know this is we're the only language to do it this way i think this design is awesome um and more languages should steal it um so basically the way that works is let's say i want to depend on luke's json parsing library the way that i do that is in my dependencies, I say, I'm depending on this JSON parsing library, and here is the URL for it. Now, the thing that we do that nobody else does is that when I specify that URL, the URL has to be in a particular format, which is that the end of the URL has to be a Blake3 hash of the contents of what's going to be downloaded from that URL.

It has to be in the URL itself, which means that a couple of things. One is once I download it, first of all, we can cash it forever now if we want to, um, because we know that it's never going to change. If it does change, well, it's just going to fail the, the, the checks on when we download it because the hash is different. This also means that if that domain expires or gets compromised or hijacked or whatever, somebody like publishes a new version, even if I have never downloaded this thing before, like I'm just reading about this off a tutorial, I copy paste the code in, I've never used rock in my life. I have no, there's no infrastructure for this. There's no centralized anything. Like I just

use that URL. Somebody took it over. I still don't get a bad package. The worst possible thing that can happen is I get an error. Um, from a security perspective, this is, I think unprecedented. I think every, every other language that uses URL based, um, uh, like packages, like lets you download a package direct from a URL as a dependency, either they're like not caching it, or even if they are, there's potentially some security vulnerability in there somewhere, or else there's like a, well, I should say and or, potentially there is also additionally a security vulnerability, unless there is some like centralized system that is like pro like proxy and caching all those

which i think is what go does but most languages i don't think do that like they don't have like a oh the last time anyone downloaded this we checked it out and it had this hash but the hash has changed now so beware you know it's it's potentially malicious but what i love about it is that it's just baked right into the ur URL so if anyone does want to they do successfully hijack the domain or whatever are otherwise able to publish that URL like what are they going to do like if they want to publish something that's different

they have to change the hash at which point the URL becomes different and that's not what people are going to be downloading so although I gotta say it makes for unattractive looking URLs. My first thought when I was looking at the tutorial, my first thought on, on the first example where it's got a package line was to tell you like, Hmm, maybe you could come up with a shorter URL for this.

So, you know, I, that was my fear with this system. And I thought like, okay, well, we'll start with this and then we'll see like how many complaints we've gotten. Um, and then like, you know, about the URLs being really long. And then like maybe we can find, you know, some way to centralize and like shorten them. I'm not even joking. This just now is the first time I've ever heard anyone complain about that. And I'll tell you the reason that I think you'll never hear anybody complain because it it it looks bad in the tutorial

um but nobody's ever typing it out people are just copying and pasting it exactly that's that's that's what i figured yeah and that's what i do too like i i also never like write it out by hand um but at the end of the day like i think that's fine like it has a lot of really nice technical properties and like if everyone's just going to copy paste it anyway and then never really look at it like i don't know seems fine i think it's fine i mean i guess i guess the thing that um uh you know eventually once you have a package manager then you can shorten a large part of the the url before the hash you know you can just have it be rot dot rock dot package or whatever yeah potentially i mean so there's that's a whole like long tangent discussion but um

but yeah it is definitely something we potentially can do this has been a whirlwind of uh talking about everything from strange loop to rock and the beautiful thing about rock is unlike some of the languages we talk about you can actually go and try it out now that's true is it rock.org it's uh i wish we had that it's a rock-lang.org and i and i do want to say i think it's got a really good tutorial um you know and i when you were talking earlier about how it's a language that's meant to be easy to use, that was one of the first things that I got out of the tutorial, because the tutorial is a tutorial that I think is designed, approachable, like not too simple for somebody who has a lot of experience programming,

but I think also if i was even like just learning a programming language or if it's my first programming language i think um the tutorials well laid out for for a newcomer wow thank you so much i spent a bunch of time on it so it's great to hear is the tutorial a preview of the new style for the upcoming rock lang uh website because the rocklang.org just takes you to the what kind of looks like you have a markdown file and it's being rendered but the rock tutorial has got lots of colors lots of fonts um you got that great font that uh what is it permanent marker marker yeah um i'll give you a preview after we sign off of the of the new one um but uh

yeah by the time people hear this they'll just be like what are you talking about this is just what the whole website looks like yeah yeah i imagine there's going to be some uh orange site posts and buzz when the when the new website is up uh or maybe not maybe that's not in the the plan but if you don't i I'm sure someone will. Yeah. I mean, you can never tell these things, but I wouldn't be surprised. All right. Well, thank you for taking so much time out of your day to chat with us. I know the listeners.

I mean, they love when we chat about cutting edge, bleeding edge programming language stuff. I mean, we love chatting about it, but our listeners, I think, also enjoy. And this is in the, I mean, like it's not a C++ successor initiative, but it's kind of in the space. If you're trying to be fast, you know, that's C++ adjacent, you know, even if it's a completely different paradigm.

So interesting, you know, we might have some people on the Twitters or whatever the websites are called these days. Oh, yeah, there's a great way to end it. Where can people find you? some people on the Twitters or whatever the websites are called these days. Oh, yeah, there's a great way to end it. Where can people find you? Are you on the Twitters? Are you on the Mastodons?

Or have you given up on social media sites? Or how can people reach out to you if they want to either get involved or ask questions? Yeah, I don't post a lot, but I am on Twitter, RT Feldman, also on GitHub. But if you want to get a hold of me, Twitter's a good place to do that. My DMs are open, so that's kind of the main thing that I use it for. That's what we like to hear. It's what we like to hear, because we hope the site doesn't go up in flames, but every week that passes, I have less and less faith in Bryce's conviction

that the site's going nowhere. I'm telling you, look, I'm just a stubborn man. We're going to outlast Elon. Okay. I mean, but is, is the site going to outlast Elon? That's the real question. We sure, we sure we will outlast him, but. I'm telling you any day now, man.

Any day now. All right. Well, thank you once again, Richard. Yeah. Expect, you know, some amount of uptick and interest whether that's one person or you know several um i'm sure there's going to be a few people checking out rock you can cut this out if you want kind of everybody can make a rock repo and get i mean we have several thousand listeners so uh if even a percentage of you did that, it would have to be even like if 5% of you went and did it, it would push it over the edge.

I think it's kind of a silly GitHub thing. They don't just like allow you to like, like they should have a model where you can self-serve and do the work on your end to add the syntax. Oh, I think I think you do have to do it anyway. It's just they won't accept your PR unless... I think the reason they do that is probably to prevent extension squatting or something like that. Yeah, that is exactly what I was thinking. Because I think it's probably like there's 10,000 languages

if you count everyone's pet language. I think even not counting the pet ones, it's in the thousands. And if you count the pet ones, then it's in the tens of thousands. And I guarantee you there'd be so many extension clashes. And a lot of languages, they choose more than one, right? Yeah. Okay. Fair enough.

All right. Well, with that, signing off. And stay tuned, listener. I don't know what episode this is. We're probably on 156 to 159 inclusive, which means you might be hearing from Sean in episode 160, unless if that's right around Christmas time, in which case we might be bringing Zach back on and having our holiday special

where we chat about random stuff, which is basically what we just do anyways. So anyways, thanks for coming on again, Richard, and good luck with Rock going forward. And hopefully we expect to see it, you know, being used by the whole entire world in a year in the future well thanks for having me i appreciate this is fun be sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we

mentioned in today's episode as well as a link to the github discussion where you can leave comments thoughts and questions thanks for listening we hope you enjoyed and have a great day. Low quality, high quantity. That is the tagline of our podcast. It's not the tagline. Our tagline is chaos with sprinkles of information.