The Changelog: Software Development, Open Source - Why Neovim? (Interview)

Episode Date: August 31, 2021

This week Neovim core maintainer TJ DeVries joins Jerod and guest co-host Nick Nisi (from JS Party) to follow-up on our Vim episode with a conversation dedicated to Neovim. TJ tells us why Neovim was ...created in the first place, how it differs from Vim, why Lua is awesome for configuration and plugins, what LSPs are all about, the cool tech inside tree-sitter, and how he's writing his own fuzzy file finder for Neovim called Telescope.

Transcript
Discussion (0)
Starting point is 00:00:00 What's up? Welcome back. I'm Adam Stachowiak, and you are listening to The Change Log. On this show, Jared and I talk with the hackers, leaders, and innovators from all areas of the software world. We face our imposter syndrome, so you don't have to. Today on The Change Log, Jared is joined by his special co-host, Nick Nisi from JS Party, for a follow-up on our Vim episode, talking to TJ DeVries about NeoVim. TJ is a core maintainer of NeoVim, and he tells us why it was created in the first place, how it differs from Vim, why Lua is awesome for configuration and plugins,
Starting point is 00:00:30 what LSPs are all about, the cool tech inside TreeSitter, and how he's writing his own fuzzy file finder for NeoVim called Telescope. Of course, big thanks to our partners, Linode, Fastly, and LaunchDarkly. We love Linode. They keep it fast and simple.
Starting point is 00:00:43 Get $100 in credit at linode.com slash changelog. Our bandwidth is provided by Fastly and LaunchDarkly. We love Leno. They keep it fast and simple. Get $100 in credit at leno.com slash changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags Powerful LaunchDarkly. Get a demo at launchdarkly.com This episode is brought to you by Gitpod. Gitpod lets you spin up
Starting point is 00:01:02 fresh, ephemeral, automated dev environments in the cloud in seconds. I'm here with Johannes Landgraf, co-founder by Gitpod. Gitpod lets you spin up fresh, ephemeral, automated dev environments in the cloud in seconds. And I'm here with Johannes Landgraf, co-founder of Gitpod. Johannes, GitHub made a big announcement recently with Codespaces, validating that it is now time for dev teams to consider what automated dev environments can do for them. What do you have to say to that? I'd say, welcome to the party, GitHub and Microsoft. No, honestly, we were very excited because it validated to the developer community what
Starting point is 00:01:25 we have been pioneering over the last years. That developer environments need to be automated and ephemeral. We are now at the right place and the right time to move software development to the cloud for everybody, not just for developers working for the Googles, Facebooks or Shopifys who left local development already for several years. Gitpod is open source and provisions for every development team on GitHub, GitLab, and Bitbucket cloud-powered dev environments. You can access your developer environments via upstream VS Code running on your desktop or in the browser and soon also all JetBrains IDEs. Very cool. If this gets you excited, learn more and get started for free
Starting point is 00:01:59 at gitpod.io. Gitpod is free for individual developers for 50 hours a month, can be self-hosted, and is available for every developer today. Again, Getpod.io. we are back for a follow-up episode to episode 450 on vim first of all i want to say that i appreciate all of the positive feedback around our vim episode we put a lot of work into that and it sounds like everybody enjoyed it we're're happy to hear that. And one of the things that we heard over and over and over and again was when NeoVim, so similar to the crypto kitties say when moon, the NeoVim kitties were saying, hey, you've done Vim. Now give us the NeoVim treatment. So by popular demand, we have TJ DeVries here to talk NeoVim with us. TJ,
Starting point is 00:03:07 thanks for coming on the show. Yeah, it's my pleasure. Really excited to chat and just explore some fun topics together. We are excited too. And I have with me not Adam Stack, it is Nick Nisi, my co-patriot at JS Party. Nick, welcome to the changelog. Hoi hoi. I'm happy to be here, excited to talk about NeoVim. And just for the record, it wasn't just me hounding Jared to get a NeoVim episode. That wasn't your bot just telling me over and over again. I can't confirm or deny that, but it wasn't me specifically. Well, if you know JS Party, you know Nick is often trying to turn it into Vim Party.
Starting point is 00:03:44 And he brings up Vim whenever he has opportunity. And he has been, for a little while, a NeoVim user. How long have you been on NeoVim, Nick? It's been a couple of years. But I'm a recent convert to the NeoVim 0.5 slash Lua lifestyle that I'm excited to talk about today. I will also confess that after our Vim episode and our Modern Unix tooling episode, I've been reinvigorated and I installed NeoVim and I threw away my Vim config and I'm doing survivor style, which configs make it into the new config game, line by line, and I'm having lots of fun.
Starting point is 00:04:20 I feel like a kid again, just tweaking and changing and kind of relearning how to use Vim because I've been using it for a very long time in a dual capacity with a full-time GUI. So I was on Sublime Text for years. I switched to VS Code, and I always had Vim open in a shell, and I would use it on servers, and I would use it kind of casually. As you know from that episode, I always use Vim mode whenever available, but I was not using it full-time as my programming editor until a few weeks ago. So I'm back and I'm giving NeoVim a try and I'm having a lot of fun.
Starting point is 00:04:51 But TJ, give us some of the backstory here because NeoVim has been around for a while and we've never done a show on it despite multiple cries to do so. Who created NeoVim? When did they start it? Why? Why did this have to exist, et cetera? Give us the backstory. Yeah, so I think the main person who first started it was Tiago Taruta. When did they start it? Why? Why did this have to exist, et cetera? Give us the backstory.
Starting point is 00:05:10 Yeah, so I think the main person who first started it was Tiago Taruta on GitHub. And it was probably about 10 years ago. I don't remember the exact date. The main thing that happened was Taruta was really interested in getting sort of some of the ability to run like external jobs asynchronously from within Vim right and that was the first main push and Teruda submitted some patches to Vim dev which is the mailing list for Vim but unfortunately at the time Teruda the patches didn't work for Windows and there were some other problems that would have broken compatibility with like a lot of the places that vim expects to build and so due to that it was sort of like this struggle of i don't know struggles the right word but just sort of a difficulty trying to figure out how these features could work within vim and all the places that vim
Starting point is 00:06:00 needs to build and the size requirements and all these other things, right? Which are like super valid concerns to have if you're on all the places that Vim can build, right? Like, do you want to still be able to do that? That's one of the selling points is sort of the universalism of where Vim is. He didn't have an old Solaris machine to test it on? Right. He did not make any patches that worked there. So basically at that point, Teruda was kind of like,
Starting point is 00:06:26 well, I don't want to necessarily have to support those. Like I want to do different things that can work differently. And I want to like push forward in some of these areas that might make us end up in a place that's not the same place that Vim is, right? And so Teruda basically started NeoVim. And then from there, Teruda also had several other sort of like large ideas in combination with people like Justin MK, who's still part of the
Starting point is 00:06:51 project, as well as Zix, who's one of the largest early contributors to NeoVim. And they added other things like remote plugin support and started doing Lua scripting and things like that. But that was sort of the main branching point that started to allow NeoVim to go down a separate path from Vim. Exactly how different is it? Yeah, so this is, I think, a pretty common misconception that Vim and NeoVim are like very, very, very different. Vim and NeoVim still share much of the same code base. NeoVim still ports patches from Vim like all the time. We have some contributors, especially like Jan Lazo, who in the last 0.5 release helped to get us to over a thousand Vim patches ported from Vim into NeoVim. So there's still sort of like an insane amount of similarity between the
Starting point is 00:07:47 two right and if you open them up like you won't really see in effect like any differences from uh like lightly configured and it.vim or vim rc whatever you you want you know that's an important sort of like backwards compat thing that we're interested in trying to work hard on. There are some things that are removed from NeoVim that are in Vim. Like, for example, NeoVim doesn't ship with a GUI by default like GVim. NeoVim sort of removed all of the code that tightly coupled it to a GUI application and instead implements GUIs over RPC. So that's like an example of if you were expecting to be able to just open up GVim, but with NeoVim, that doesn't happen because those
Starting point is 00:08:30 two things have been sort of decoupled as well as various other things. Like if you installed NeoVim, it wouldn't install like view the command view, which is like opening them in read-only mode because you can instead do like NVim dash dash r or something i don't remember what it is off the top of my head sorry everybody but you could just alias that to be the same and then it would uh retain the same behavior so of course there are some you know similarities that are obvious at the start and some things that are a bit different when you're first encountering them but in general there's a incredible amount of overlap still between the two projects, I would say. That would be my experience so far is that I haven't noticed a difference in just like
Starting point is 00:09:11 regular text editing. Like I could just alias it and never notice. Now, once I started configuring it and trying to do some more advanced things, you start to see where the seams are, the differences are. Go ahead, Nick. What were you going to say? Oh, I was going to say something similar in that when I started using it, I actually started using it, I think for the same reason that it might've been created now that I'm hearing that. And that's because I heard about this async
Starting point is 00:09:31 job support and I had just gotten into like, you know, configuring linting and things like that. But every time it would lint, it would like completely freeze Vim. And that was just super annoying. So then I heard about NeoVim and how it could do that asynchronously. And so I set it up, but I was like, this is just temporary. I'm sure Vim's going to catch up and I'm just going to switch back to it. So I installed it and I did that alias, alias Vim equals nvim. And I still have that. And that's still how I invoke it. But that alias is like five years old at this point or five or six years old. And now I, yeah, it's just interesting that I really can't tell the difference or couldn't tell the difference in the beginning, aside from like where your config lies and that. But
Starting point is 00:10:11 otherwise, when you start up, it said, instead of Vim, it said nvim. That was pretty much the big difference. Yeah. And like, just to mention as well, like Vim does have those capabilities now, right? Like it does have asynchronous jobs and other things. I just figure i want to throw that out there in case people are interested in adding some of those things to their own vim config but it does exist in both now yeah that's one of the cool things that i've noticed i wonder if neo vim is directly to blame or to a credit for this but it seems like vim's development picked up with regard to performance, asynchronization, and certain things that maybe weren't being advanced. And maybe there was a little bit of, you know, competition spurs innovation thing going on. That was just my
Starting point is 00:10:54 casual observation. Is that your thoughts as well? Or if we asked Bram about it, would he say, yeah, I saw NeoVim coming and I was like, holy cow, I need to start doing more? Or what's the story there? I actually think that this is probably like a common misconception i think bram likes implementing things that he thinks people are going to use and like it wasn't really clear like from the community that this was an important thing for them to like have previous to this my personal opinion obviously I'm not inside Bram's head and we've only spoken just casually a few times, you know, on GitHub issues or email is that Bram just was like, oh, there's people requesting this now in Vim dev. And they're like
Starting point is 00:11:36 thinking about it a lot, like, oh, we can implement that. And so then he implements it and it works well. And then he's happy with like the solution i'm not a hundred percent confident that like the neo vim like spurred him on to do that i don't necessarily think that that's exactly the case i think it's much more that bram saw that people in vim dev were like this is cool we should have this and he was like okay i'll make one that works well for inside of him and then that's what he did. Yeah. And to that point, both of them have like async and other similar features, but have they arrived there in different ways? Like, is it different APIs or ways to like invoke async, for example? Yes. So to like start external jobs,
Starting point is 00:12:17 the jobs APIs aren't compatible. There are like plugins that sort of wrap them to make them very similar. So you could like have one that's shared and you call my shared lib dot start job or whatever, right. And you could be doing that and sort of experiencing working them in both, but their implementations are different. Like NeoVim is really focused a lot on building around like lib UV, which is right. The stuff that like, I think node, I remember correctly is built on top of probably know about that in js party land right so like
Starting point is 00:12:50 neo vim's event loop and stuff is built around like that and so we use lib uv bindings to do like the async jobs and many of our other event loop type things that we're interested in doing and i haven't looked in depth at what then how it actually implements it. If I recall correctly, they're not using lib UV. So like the underlying implementation is not the same. Right. But their APIs are like pretty similar. And it's possible to write code that works in both. It's sometimes a bit annoying, but that's okay. And with the help of those like community plugins, is the burden on like plugin developers kind of lessened or almost non-existent by that? Or does that kind of cause some issues?
Starting point is 00:13:30 If all you want to do is run a job asynchronously, then it is not like incredibly difficult. There are other things that are much more like difficult to maintain shims between or some things that are very specific to Vim or very specific to NeoVim that either haven't been like unified or never will be. For example, right now in Vim, there's APIs, they're all prefixed with pop up underscores, there's like pop up underscore create or pop up underscore menu or things like this, right? And they allow you to make little floaty windows. NeoVim currently doesn't have those APIs and we have not a shared floating window implementation underneath the hood. So it's not necessarily simple to just like port the patches over and get the results. But I'm working on a project and other people are contributing as well to create
Starting point is 00:14:21 like Lua API that we can create windows with. And then we could expose a thin wrapper over that back in Vim L that's the same or looks the same to the user so that you could write plugins in Vim script and they would work in Vim or NeoVim, even though the underlying implementations aren't the same. So it's not that we've completely abandoned any hope of like still making some things the same. But like, right now, if you wanted to have floating windows in one versus the other, you'll have to write very different code. But if it's literally just like, I just want to spin up a job and have it run externally, you could probably have that working for your plugin in like five or 10 minutes for simple cases, I would say.
Starting point is 00:15:07 So it sounds like the longer it goes, the less underpinnings they share. You mentioned that there was a thousand patches or so in the 0.5 that were like direct ports from Vim into NeoVim. Do you expect over the course of NeoVim's life that that number would decline over time? Or do you think there's enough foundation that's shared that there's always going to be massive amounts of shared porting the solutions that come with over there onto the NeoVim side?
Starting point is 00:15:36 In general, I try not to predict the future too much because it's a difficult game to play. But what trend are you watching? Give us the trend. Yeah. In certain areas of the code, I would see us still porting patches for a long time. And port is, in some cases, very loose.
Starting point is 00:15:52 It might be literally just applying style differences and putting them in, and then porting tests written in Viml into Lua because we write most of our tests in Lua. So some of those patches are very simple, and they are almost direct applications. So there are some areas that I would expect that to like happen for a long, long time, like a bunch of things and like eval.c, which is where you know, Vim script things get done, like it's like, okay, it was simple, we just added this new argument to
Starting point is 00:16:20 a Vim script function. Okay, cool, we can try and port that over and then we can have that the same in both but in other areas there already exists no way to port patches for example like for floating windows code we have two separate implementations so it makes no sense to port any of those patches they get marked as not applicable for neovin But it may be that we, in effect, port some of those patches over in attempts to keep the community as close as we can, right? Like with the example I gave of the pop-up APIs, where Neovin would still like to implement those so that people don't have to worry about which one they're writing for, and they can continue serving both communities easily. More and more startups are using Retool to focus their time on their core product. And that's exactly why they launched Retool for Startups.
Starting point is 00:17:27 This is a program that gives early stage founders free access to a lot of the software needed for great internal tooling. And Retools worked with thousands of startups. And the trend line they noticed was technical founders spending tons of time building internal tools. That means at this critical stage, these founders were distracted from their core product. The goal is simple, make it 10 times faster to build the admin panels, CRUD apps, and the dashboards most early stage teams need. And Retool has bundled together a year of free access to Retool with over $160,000 in partner discounts to save you money
Starting point is 00:18:00 while building Retool apps with common integrations like AWS, MongoDB, Brex, and Segment. There is so much you can do with Retool. You can use these free credits to build tools that join product and billing data into a single customer view, tools that convert manual workflows into fully featured apps for your team, or tools that help non-technical teammates get access to your database to read and write data, analyze, and query. These are just a few examples. Learn more, apply and join lightning demos at retool.com slash startups.
Starting point is 00:18:29 Again, retool.com slash startups. So let's talk Lua. Yeah, that sounds good. On our Vim episode, I asked everybody what frustrations they have with the editor, and Gary Bernhardt said, Vim script. I'm pretty good at complaining, so I'll limit myself to just one thing, Vim script. To understand Vim's relationship to its scripting language, let me contrast with Emacs. Emacs is almost like an operating system
Starting point is 00:19:07 that happens to ship with an editor. And long-term Emacs users will effectively end up rewriting parts of that editor sometimes, but it's still Emacs because the layers underneath are still Emacs. So Emacs Lisp is a whole programming language. It was designed to be a programming language. Vim is not like this at all. Vim was designed to be a text editor. Itim is not like this at all. Vim was designed to be a text editor.
Starting point is 00:19:26 It had a configuration language. That configuration language grew over time to acquire normal programming constructs, and that is what we call VimScript. So it's a hodgepodge, and it wasn't sort of designed all at once, and I don't think it's controversial to say it's kind of a mess. So that is the most frustrating part of Vim for me, and it also is one of the reasons that I avoid configuration when I can.
Starting point is 00:19:46 I have a very sort of mostly stock Vim configuration despite 15 years, and VimScript is why. So this is something NeoVim's taking head on, isn't it? Yeah, I think NeoVim's choice of choosing Lua as what I would consider sort of like the de facto future of configuring NeoVim. Not that it won't be possible to write VimScript or that we will like remove the engine or anything like that.
Starting point is 00:20:11 That is not happening. We will continue to support that and allow that to happen. I think Lua has invigorated a lot of people to be interested in both contributing and sort of being able to explore making NeoVim more their own. Not only is like Lua like a programming language that has lots of external libraries that you could install with like Lua rocks and things like that. And there's a lot of other external resources for learning Lua, but there's also really cool things like LuaJIT, which allows for crazy performance that's just like sort of insane.
Starting point is 00:20:47 We'll talk probably a little bit about Telescope later. But like Telescope can find and fuzzy sort literally like tens of thousands of items as you're typing in one main thread attached to the UI. We currently have some stuff in progress to do a few things in a little bit smarter way so it doesn't block. But regardless, it's doing an incredible amount of work like as you're typing and that sort of like performance is just not possible with old Vim script. It's true that Bram's working on Vim 9 script and there will be performance improvements there. There's a lot of people I know that i've talked to personally that are excited to use lua because there's applications outside of neo vim right they're like oh i used this for scripting before in a video game or i've
Starting point is 00:21:34 used this in you know whatever c application or oh i've embedded this here so they have sort of like previous experience or future experience that they'd like to have with the language that maybe makes it more accessible and exciting for them, as well as being fast. Given NeoVim's architecture on top of libUV, was JavaScript considered as a potential language? It was, I think, considered but discarded. I mean, JavaScript's ability to be embedded easily inside of a C application is not, I think, considered one of its strong suits, as well as sort of its minimalism, right? I mean, Lua can be run and is run inside of some Linux kernels, so the portability and the size and all of these other things
Starting point is 00:22:19 are really great features, as well as NeoVim, I guess, pinned in a way to lua 5.1 which is uh the language is complete there's no changes there's no new things being added to the language there's no things being removed which is really nice for writing a plugin now and having it work in 15 years as opposed to not throwing shade at anything in j, but I feel like it is a common problem that I see people talk about on the internet that yesterday my NPM build worked and tomorrow it does not, not to mention, you know, changes within the language itself. So there's a lot of things about Lua that provide sort of an incredible match with NeoVim. Like I said, embeddability,
Starting point is 00:23:03 size, portability, and the fact that it's sort of static are really great things for us in terms of shipping a small binary that can work in a lot of places. Yeah, it almost seems like Lua was purpose-built for use cases like this one. Yes, that is exactly what the design principles of Lua are. One of the primary design principles is embeddability. So yes, this is exactly the case where you would want to use Lua are. One of the primary design principles is embeddability. So yes, this is exactly the case where you would want to use Lua. You do not want to use Lua to write your 10 million line monolith. That is not the purpose of the language. The purpose of the language is to be embedded inside of other languages to provide relatively safe and easy ways to script and extend something, right?
Starting point is 00:23:47 Like you're not going to have a memory like leak in the same way or a memory problem with like writing C to script your editor or something like that. That's not exactly the same kind of fun experience, at least for me. Right. Now, Nick, I know that you've been porting your Vim config over to Lua in the last couple of weeks, because as TJ mentioned, NeoVim, you can configure it in Vim script and you can also configure it in Lua. You can have it in a.vim or is it a.lua? Is that the same file name? Yeah, you can have either one. You should not have both. But Nick, you've
Starting point is 00:24:22 had both here for a while or I don't know how you've been managing that. What's been in your experience? Because I don't think you've used Lua previous to this. Is that right? That's true. I have not used Lua before this. And I initially was hesitant to even start because I was just like, I don't want to learn Lua. I don't have any practical application for it. But then I was like, wait a minute, I'm learning VimScript. I don't have any practical application for that outside of this right the thing that really pushed me into it as i mentioned i've been kind of a recent convert to that i i know that a lot of people have been using like nightlies of neo vim like i guess 0.5 maybe i'd call it the lua support version for a while but i only recently jumped on that and started using it and it was because i saw some cool plugins that I wanted to
Starting point is 00:25:05 use. And so it initially just started out with me installing the nightly of NeoVim and then just having my init.vim and putting in like a Lua require to open up the configuration for like a specific plugin and be able to use that. And I kind of didn't like that as I started adding more and more plugins, I didn't like that. I just kept calling out to Lua for that. And I kind of didn't like that as I started adding more and more plugins, I didn't like that. I just kept calling out to Lua for that. And so I started like, I just went in and as an experiment to see like what it would take to translate my, like at the time, 600 something line of init.vim into init.lua. At the end of it, it came out as like 240 lines of Lua, which was pretty surprising. But that's because I like split out plugins into its own plugins.lua that set up all of
Starting point is 00:25:50 that. But that was still only like 50 lines. I lost 200 lines in there somewhere and I feel like it's more powerful now. And that's losing, not like losing your car keys. That's like losing weight. You know, that's the kind of losing you want to do. Exactly. Yeah.
Starting point is 00:26:03 But I guess from your perspective, TJ, would you say that there's a big benefit to converting like your configuration to lua or is this was the primary intention to be more like for plugin authors to be able to write more robust plugins yeah so there's like a couple i guess sort of aspects to chat about like in that area the first one is i don't think that we have yet implemented everything in NeoVim to make it like super elegant and easy to write your entire configuration in Lua. Like that's going to take time. Like I have a work in progress PR for auto commands, for example, that would allow you to directly pass Lua refs inside and like NeoVim in the C code will hold a reference to that Lua ref until it's ready to release it and then release it. But that requires like a lot of changes, but that's a story for a different day if we want. But the point being,
Starting point is 00:26:58 you still basically have to write your auto commands in Vim script, even if you're like inside of Lua. I have a lot of InVim execs in there for that stuff. Yeah, exactly. So like that doesn't really give you any necessarily gain. So there's some parts of my config that are still in VimScript and probably will be for a long time. I mean, the other part too, is like, if it's working really great for you, I don't see any reason to change it over. Like it's going to work well, like to do a lot of the set operations or like simple things like that like vim script is a dsl for doing those it will work probably nicer than we can sometimes do inside of lua although some of them have sort of been mitigated by things like
Starting point is 00:27:38 vim.opt which allows you to sort of use lua and metatables to set Vim options more like ergonomically, I guess. So I would say like there are things that are nice about setting up your config in Lua. Like, you know, you can use closures really easily and there's a lot of ergonomics about passing around functions or doing validation or whatever that is. But I don't see like a whole lot of gain of like just switching over to init.lua right now. I think a lot more of the benefits are about extending like particular parts of your, you know, configuration.
Starting point is 00:28:16 Maybe you have a complicated function that, you know, shelled out to some commands and parts some strings and then would do something within a certain buffer of a certain file type. Well, that might be a lot nicer to write in Lua than it is in Vim script, right? But like changing set number to vim.op.number equals true, I don't know. That doesn't really strike me as like, wow, that's so mind blowing.
Starting point is 00:28:42 Pulling me out right now. So maybe someday I would say like without hesitation you should just port it over but i think the primary focuses are much more about scripting the editor in a more sort of deep and customizable way than just like setting options if that makes sense for sure and yeah i think that from that translation a big thing that i really liked like yeah auto commands and things like that i I'm just like, basically it's just a string of Lua in my config that's being run. But other things like, um, I use a plugin I've used for a plugin for years called Vim startify that lets me kind of configure what the Vim start screen looks like. And, you know, it shows the recently used files and things
Starting point is 00:29:19 like that, but it has a big like object that you configure and tell it exactly what you want it to show and do and all of that. And doing that in Vim script, like I've always just hated that syntax because you have to like, as you put things on a new line, you have to have the forward or the backslash on it everywhere. And it's, it was just really annoying. But things like that, being able to use like a Lua, do you call that a table? Yeah. Yeah, a Lua table to hold all of that information and then just be able to access it from Vim. And I can do things like access other objects like that through the, I think it's underscore capital G for the global. And then being able to access from Lua Vim functions through Vim.fn.
Starting point is 00:30:01 That stuff is really cool, being able to do all of that and i think that i'll probably keep my config in lua going forward and just kind of adopt the new features as they come in like being able to set auto groups and things or like auto commands and things like that but yeah right now that's just glorified vim in strings being called like if you know someone is starting today and they made their config in lua, I wouldn't be like, switch it back to VimScript, it's better or something. And I wouldn't really do the same, like, vice versa either. I think, you know,
Starting point is 00:30:32 if it's working for you and you're, like, happy with the end result, there's no reason to, like, spend a bunch of time switching the language to be something else. I think they're both, like, really good options for, like, configuration at this time. Yeah. I made the decision that I was going to start it in Lua because I'm like, I'm starting
Starting point is 00:30:49 fresh. I'm starting in Lua. And then Nick showed me his and I'm like, nah, I'm going to go back to NIT.vim because I can just copy paste those right in when I want them without thinking. So totally. The other kind of little bit challenging thing is depending on like if you you know spend your free time perusing dot files on github if you find something cool in lua you either have to translate it to vim if you're not using lua or vice versa and understanding how to do that translation is going to be fun for everybody not the most transferable of skills you know you're not going to put that on your resume. I can translate script configurations between Vim script and Lua. Oh, wow. Speak for yourself. decent amount of stuff for which is trying to basically transpile vim9 script the new language that bram is making into lua and then keeping as much of the semantics as we can between the two
Starting point is 00:31:55 one of my goals is like i would hope that we can keep them in neovim like as close and friendly as possible like no one on the neovim team has any animosity with people on the vim team i don't really think vice versa either so like it's a long-term project that i can't decide if it's real or not or how much of my life i will devote to it i have like some preliminary things and i can generate like valid lua from vim 9 script so there's like still even in places where people might at first glance think that it's going to be impossible for Vim and NeoVim to like live together and be friendly. I still have goals to make them be friends. That's interesting because that's been one of my kind of lines of questioning that I'm thinking about here is that I'm starting to see most recently we'll get into TreeSitter and stuff like that. But yeah that I'm starting to see most recently, we'll get into tree sitter and stuff like that.
Starting point is 00:32:45 But yeah, most recently starting to see some plugins. There's one called limelight, which was written in Lua for Neo Vim using tree sitter for better dimming. And it was the first time, cause you know, I watch a lot of these things, what people are working on,
Starting point is 00:32:58 et cetera, for change log news. And a lot of times it's like, this is a Vim slash Neo Vim thing. And that was like for years, it was either Vim only or Vim slash Neo Vim. And I'm starting to see like, this is a Vim slash NeoVim thing. And that was like for years, it was either Vim only or Vim slash NeoVim. And I'm starting to see like, this is a NeoVim thing. And there's no Vim equivalent.
Starting point is 00:33:11 There's no, they didn't take the time to whatever you have to do to maybe write it twice or whatever it is. And so I started wondering like, will that start to pull people away from Vim on a NeoVim? And it's interesting your perspective of like, you don't necessarily want that to happen. Yeah. I mean, some of them are just impossible, at least today, like there's no way to get TreeSitter, at least that I know of, like inside of Vim, right? So no matter what you do, if you want to build something on top of TreeSitter, which is a really powerful and interesting technology, like that's going to most likely end up being NeoVim only like plugin. And I mean, even for me, like, you know, I will probably talk about Telescope a little
Starting point is 00:33:49 bit, but like Telescope's NeoVim only because it uses a ton of NeoVim specific APIs that are only available in NeoVim and like uses the window things in complicated and interesting ways. And like, it relies heavily on the fact that you have LuaJIT installed and uses native C modules that we can link against C that optionally you can use to do sorting, to make sorting even faster. So I think like there will be plugins that exist that are only for NeoVim and there will be plugins that exist only for Vim and that's fine.
Starting point is 00:34:26 My goal and like the NeoVim team goal, I think that I was trying to say before is basically just like, we're not actively trying to make the community not work for both, right? Yeah. Well, I really appreciate that perspective because so many times when there are projects
Starting point is 00:34:41 and efforts like these, so often it's like a hostile takeover kind of a thing or a long, you know, it's a long con. Like, so often it's like a hostile takeover kind of a thing or a lot, you know, it's a long con like, well, we're going to slowly take over. And I think that,
Starting point is 00:34:49 I mean, at the end of the day, it's Vim, you know, it's a different Vim. It's a newer Vim. It's got things Vim doesn't have, and it takes things out,
Starting point is 00:34:57 but still the same editor that we all know and love has been around for all these years. Like, why would you want to crush that? Why would you want to kill it? Yeah, and I mean, like, I respect Bram a lot. Like, he's made an incredible piece of software, and it's, like, pretty bonkers if you just think about, like,
Starting point is 00:35:15 where it started to where it is now and all the things that he's done for it. And, like, plus, Neovim is literally a fork, and so it's like we share a bunch of code that we never wrote. Like, I don't know, it feels kind of not very grateful and like not very thankful to just be like, and now we're our own thing. And I don't like, you know, it's like an angry teenager, like, I don't even like you, dad. You know, it's not exactly like the pinnacle of maturity, I think, necessarily all the time to like have that kind of situation. And so it frustrates me when I see it online between people as well. It's just like, if they
Starting point is 00:35:50 want to use Vim, that's cool. Why would you be mad about that? You know what I mean? That just seems cool to let, to have them do that. That's great. Yeah. I mean, if we're going to hate on something, hate on the Emacs people, come on, what's wrong with you? Oh, I mean, Emacs is now closer to Vim and NeoVim in spirit than many other places, right? So not to name any other names, obviously. Yeah. They need to join forces to take on the onslaught
Starting point is 00:36:14 of the graphical editors. Oh, yes. The evil graphical editors from faraway lands. In my browser now. Yeah. This episode is brought to you by our friends at Square. Square is the platform that sellers trust. There is a massive opportunity for developers to support Square sellers by building apps for today's business needs. And I'm here with Shannon Skipper, head of developer relations at Square. Shannon, can you share some details about the opportunity for developers on the Square platform?
Starting point is 00:37:08 Absolutely. So we have millions of sellers who have unique needs. And Square has apps like our point of sale app, like our restaurants app. But there are so many different sellers, tuxedo shops, florists who need specific solutions for their domain. And so we have a node SDK written in TypeScript that allows you to access all of the backend APIs and SDKs that we use to power the billions of transactions that we do annually. And so there's this massive market of sellers who need help from developers. They either need a bespoke solution built for themselves on their own node stack, where they are working with Square Dashboard, working with Square Hardware or with the e-com,
Starting point is 00:37:47 you know, what you see is what you get builder. And they need one more thing. They need an additional build. And then finally, we have the app marketplace where you can make a node app and then distribute it so it can get in front of millions of sellers and be an option for them to adopt.
Starting point is 00:38:00 Very cool. All right. If you want to learn more, head to developer.squareup.com to dive into the docs, APIs, SDKs, and to create your Square Developer account. Start developing on the platform sellers trust. Again, that's developer.squareup.com. so nick you mentioned that you were waiting for the lewis stuff to land or to be official 0.5 came out this summer july 2nd i believe a huge release 4 000 commits as i mentioned before
Starting point is 00:38:40 a thousand of those reports from vim, but lots of new stuff. And the community rejoiced. I mean, this must have been a long time coming, TJ. How much work went into this release? Well, yeah, there was a lot of work. There was probably, if I had to take a guess, probably, I don't know, three or four weeks of time of me just answering when will NeoVim 0.5 come out. You know, just replying to messages like that online. We started a meme on my Twitch channel
Starting point is 00:39:10 when people would ask that. We did exclamation, the rule, and then that took a bot command back and it explained that every time someone asks about NeoVim 0.5 release, the release date has to be pushed back one day. Yeah, but in all seriousness, I mean, it's been a long time coming. And it, you know, was a culmination of a lot of sort of really long term project and vision
Starting point is 00:39:32 that the team had, I can't take credit for all of it, or even probably, you know, I can't take credit for most of it or anything. But, you know, seeing things like the LSP come to fruition, there's some fun history there where you can go dig up some of the original issues and my first PR probably in 2016 or 2017, where I proposed a lot of the first ideas about how we would put LSP inside of NeoVim and have it be in such a way that it still is the spirit of them. You know, that it's not, we're trying to smash everything in here and re-implement every wheel that we can.
Starting point is 00:40:12 How can we do it in that way? So for me, it's been very fun to see that finally released. You know, it's been in my head for a long time and other people who helped along the way to finally implement it. And then as well, you know, putting in TreeSitter, which has also been sort of a long-term vision about making NeoVim as an editor understand the text at a much higher and more, I'd say, interesting or semantic level
Starting point is 00:40:38 than just like strings of characters, right, which is one of the goals of TreeSitter. Let's focus in on LSP, the language server protocol. So this part didn't make the Vim show, but I did ask each person what they FOMOed from other editors, and we wanted to keep it tight, so we dropped that section out. But I'll tell you, Drew Neal mentioned LSP specifically in VS Code. Yeah, well, I come back to language servers again i mean uh about a year or two ago i started
Starting point is 00:41:08 learning typescript and i know that typescript has a very good language server i mean pretty much um language servers and vs code and typescript were all kind of invented under the same roof and so they're very very well integrated. And I knew that I wanted to understand what a good TypeScript development experience is. And so I made a decision not to use Vim while I was learning TypeScript. I thought, okay, VS Code is an editor that's very well positioned for beginners.
Starting point is 00:41:39 And while I'm being a beginner in TypeScript, I'm gonna be a beginner in VS Code as well, and just learn the way that good language server integration would feel like so that I can sort of take that learning and bring it back to Vim so that when I make that step of, okay, I'm going to now switch all my TypeScript development back over to Vim, I know what I'm looking for. For those who don't even know what LSP is,
Starting point is 00:42:02 can you just explain what it is, where it came from, and then we'll talk about it in context in the OVM. Yeah, sure. So LSP is a protocol designed by Microsoft. It's actually what Microsoft's VS Code uses to communicate and basically get the language smarts that it has. Its big goal is, if you think about, you have M editors, right, and N languages. And when you want to implement support for the next language, you have to implement that M editors, right, in N languages. And when you want to implement support for the next language, you have to implement that M times, right? And so I think as developers, we're very familiar with M times N problems being very difficult and feeling intractable and also not
Starting point is 00:42:38 fun to solve, right? Because even if you solve it once, you feel you're going to have to solve it N times again later. So LSP is basically a protocol designed to talk from editors to some sort of server that's giving you information about what you're editing, right? So if you have your cursor on somewhere in a document, you can say, hey, what's the definition of this? And so there's a protocol, text document slash definition. You send that over the wire to something that's running. It can be anything you want. It can be on your machine, another machine, doesn't matter. Just a protocol that you send over the wire and ask, hey, what's the definition of this text document at this position? And it will return you back a list of possible definitions or one, you know, whatever the result
Starting point is 00:43:24 is. And you'll be able to go there in the editor, right? So the editor will receive that response and know what to do. So this is really powerful because in theory, it's not always exactly like this. It'll turn the problem from M times N to M plus N, which is a much better problem to solve. And it's probably more like M plus N plus, you N plus Z, right? There's some extra work still. Yeah, it still has to happen. Some language servers have specific commands that only make sense for their language or things like that. But that's okay. That's still a much more fun problem to solve. And so that's sort of the basics of what happened. And like in the context of NeoVim, what we saw was that this is sort of, in a lot of ways, just like an interface to your editor. And that was one of my big pushes for how we should think about LSP inside of NeoVim, is this is really just defining
Starting point is 00:44:16 a standard way that is shareable between other pieces of technology in how you can talk to your editor, right? So I recently implemented something for my work at Sourcegraph, where I implemented like a go to definition over LSP. But what it does, instead of running something locally, it asks this external program where the definition of this thing is, right. And so that was basically like a way for me to do that. And I didn't, I mean, obviously, I knew how to move to the file and all these other things because I am familiar with NeoBIM, but it doesn't have to be so. Right. So it allows this interface to do. And then if I can do another plug for Lua in here, right, the way that we did this with Lua is that it's very easy because everything is just a function, you can replace a function with a basically set of parameters and an expected thing to happen and replace it with some new behavior that's more suited to your purpose. I'll give an example for that, which is in Golang, we have a bunch of stub or test implementations
Starting point is 00:45:22 for some interfaces that we have at my work. I don't want to jump to those when I hit go to implementation 99% of the time. So what I did was I can ask the LSP, hey, where are the implementations of this? I wrote some Lua code that walks through the results and says, if this file ends with underscore test or like underscore mock, discard it and throw that away from my results. Generally at work, we just have one result that remains and then I just jump straight to there
Starting point is 00:45:55 if there's only one left. Otherwise I open quick fix list. So that's like really cool because now I only have to press the button to jump to that implementation. And it does exactly what I hope only in Go code and everywhere else it does the standard definition. Right. So there's this concept and idea of like the way that we want to design it was this is just an interface for both language tool makers and like editor tool makers to be able to interact with in a way that you can customize
Starting point is 00:46:27 to be just the way that you want. Good explanation. It's just amazing. I'm trying to think of ways that I might implement this. Nick, sometime you're welcome to come on my stream and we can work on implementing it for you or something. It would be fun for some other case. That'd be a lot of fun.
Starting point is 00:46:42 That does sound fun. So tell me about the client server situation is this like a network server or is this like a an embedded binary that you call how does it actually talk and explain that yeah so lsp can the primary modes that people use to communicate are either just standard in standard out or you can do it over t. Generally, if you're doing it over TCP, people just use some pipe and then pretend that it's TCP or something like that instead of actually doing TCP. But so NeoVim doesn't care what's on the other side.
Starting point is 00:47:18 We take a list of commands, we spin up that thing and start it, and then we have standard in standard out between the two. And we just send basically JSON over the wire for request. And then they'll send JSON back over the wire to get to us. And it's just like a pipe, generally speaking, it's possible to do to do other things. So you could have anything on the other side. So for example, GoPlease, the Go language server, is written in Go. And so it's just a Go binary on your machine that you say, GoPlease, start up. You send basically a configuration request so it can understand what directory am I in, what's part of this project, all those kinds of things, right?
Starting point is 00:48:02 That's part of the protocol to understand. And then it will just respond to requests as you send it. There's also things like Microsoft's PyLance, which you can only run in VS Code due to licensing concerns, but that's written in TypeScript. So it's not written in Python. It just does the analysis in TypeScript and then sends the responses, but you're just editing Python files. You don't care what it's written. And it's just a protocol that talks between the two. So let's say I wanted to teach NeoVim a new language. Yep. Let's say I had a language called Inc. And I wanted it to understand Inc.
Starting point is 00:48:35 Would I then provide my own LSP or would I plug something into the LSP that NeoVim ships with? How would I get it in there? Ah, so that maybe I should take a step back. So NeoVim's LSP client isVim ships with, how would I get it in there? Ah, so maybe I should take a step back. So NeoVim's LSP client is just Lua code that's inside of NeoVim. It's an LSP client. It does not have its own language server in there. Right, that's correct. So it doesn't know how to answer any of the questions.
Starting point is 00:48:58 It only knows how to ask. I thought maybe it had like a set number of languages that it already had all of the stuff ready to go. Yeah, no, it does not. We have nvim lsp config, which is just like a repo that helps you get some of those running. But you need to figure out how to install those and make them executable on your machine. Being a package manager is not what we're good at. We're good at being a text editor.
Starting point is 00:49:22 Someone else can figure that out and do that. And there are various plugins to help you install. But NeoVim just has basically a bunch of Lua code that knows how to handle the responses and ask the right questions. You need to install separate executables to do that. And to that end, I'm using something called LSP install that handles installing those and then using LSP config to configure them for NeoVim. Yeah, it'll just install it for you and stuff. So we leave that to other people, you can handle that. I'm sure there will be one for specific distros or specifically for Nix,
Starting point is 00:49:55 or there's one for Docker containers. There's a lot of different ways that you can do that. That's not our strong suit. We try and make text editors good and fast. Gotcha. Well, let's move on the tree sitter because yeah another cool technology it seems like there's they seem like they're similar right i mean you're talking about the syntax tree of a language help us understand tree sitter and how it's different than lsp yeah so this is a really common question so it's a good one for sure so a really high level way that you should start thinking about both technologies is that TreeSitter only deals with one particular file,
Starting point is 00:50:32 and it only deals with the text that is in that file. It has no additional information. And LSP operates much more on like a project-wide level. It's going to open up and read all your files. It's going to index all of them. It's going to open up and read all your files. It's going to index all of them. It's going to put a bunch of things in memory. And then it's going to be able to have some specific way that you can ask certain questions and it will give certain responses. So that's sort of like the first aspect. The second aspect is TreeSitter is built in a lot of ways specifically to run like inside of your text editor as opposed to like an LSP which is some external process.
Starting point is 00:51:09 So TreeSitter at a high level for those who don't know is effectively a library for writing error recovering incremental parsers and it's very good at doing that which is awesome for text editors because most of the time when you're writing code, the code is broken, right? If you type a line of code until you get to the semicolon in C, right? It is broken. So you want to write parsers that can recover and not like drop highlighting for the rest of the file or do things like that, right? You want to actually do error recovery and smart error recovery. And that's like quite difficult to do, but TreeSitter just sort of gives you that for free if you write a grammar in the correct way. So that's the first part.
Starting point is 00:51:56 And then the second bit that's very important is that it's incremental. So it does not reparse the whole tree, which is awesome for when you're typing keystrokes in a very large file because you're going to smash 80 keystrokes. You don't want to reparse the entire file 80 times, right? Instead, it actually just incrementally parses the things that it needs to and generates a new tree. Does that make sense so far?
Starting point is 00:52:19 Any questions there? I'm with you. Okay. So once again, TreeSitter only focuses on exactly that file. So this is cool because this lets you do things like you can request highlights for the file by asking the tree questions and basically getting back named nodes. You can write queries in this scheme-like language. It will return to you named nodes.
Starting point is 00:52:42 And then in those nodes, they have ranges. And so you can color them in your editor if you want. It will return to you named nodes. And then in those nodes, they have ranges. And so you can color them in your editor if you want. This is sort of like the thing that everyone recognizes about TreeSitter, but I find somewhat like the least exciting of all of the TreeSitter technologies, but it is the most obvious. And it also makes sense from like a performance perspective that you would like to only change the highlights that need to be changed. So this is like a really great improvement and allows you to write much better and powerful syntax highlighting than you could with like just regexes, which is like the built in Vim syntax engine. And it prevents you from getting these situations where like, I don't know
Starting point is 00:53:20 if you've ever had this, but in certain complicated file types, if you like scroll down the page, sometimes it like thinks everything in the rest of your file is like a string. And you're like, what is happening? Why is this all string? That can't happen when you have a tree, right? That's not going to be an option. So that's sort of like the first level of TreeSitter.
Starting point is 00:53:43 And it doesn't communicate over the wire. This TreeSitter is running inside of NeoVim and it's like embedded inside of NeoVim. Now the parsers and the queries and things like that, those are external to NeoVim, can be configured by users. I have my own custom grammar for Lua to do some special things, etc. So those are all sort of configurable, but the engine itself is built into NeoVim. And is the engine something that was built by the NeoVim team or you're actually embedding somebody else?
Starting point is 00:54:13 Is TreeSitter its own project? Yes, TreeSitter is its own project. It was originally made for Atom, but several other editors now have used it because it is designed to be a library to be embedded inside of other editors. And so it's very, very fast and performance and they spend a lot of time making it really awesome. And the tree sitter team is super cool. To get it started with it, you kind of similar to LSP config, you have to install a separate plugin. And is that just to help with the configuration then? I think that's where I like had confusion was because I was like,
Starting point is 00:54:45 I thought that it was built in, but I have to install this plugin. Yeah, so to be like clear about it, you actually don't have to install plugins for either. Like you can start up the server by yourself for LSP. It just is a little bit more boilerplate. So that's why we have LSP config, but it's like certainly possible to do.
Starting point is 00:55:02 LSP config literally only has configurations but for tree sitter you also don't necessarily have to install the plugin it'll just be more complicated like it would involve you downloading a grammar file running tree sitter to generate the bindings because it's separate executable to generate these bindings and then creating your.so shared executable and then putting that in the right place. So you could do that. And then the engine that's built inside of NeoVim will see that shared executable loaded up and appropriately attach it, I guess, to the buffer. But that is not very fun to do. So that's where Envim-tree-sitter comes in. And over time, we hope to, you know, upstream more of like nvim-tree-sitter into NeoVim core, but there's a certain level of
Starting point is 00:55:54 backwards compatibility and polish that we want to have in the core repository. And we felt that that would really hold back a lot of experimentation and interesting things happening. We tried to squish that all inside of NeoVim core, which is where Envim TreeSitter sort of came out as a separate plugin. So the engine, all of the interior things to do, like get the tree and have it parse incrementally as you type and all of that kind of stuff, that's inside of NeoVim all the time. Someday, someday, my goal would be we would at least ship TreeSitter grammars inside of NeoVim for C, Lua, and Vim, VimScript, because those are the languages that NeoVim deals with. And I think it would be
Starting point is 00:56:40 cool to have a really awesome experience out of the box for those languages for people, but we're just kind of far away. Not maybe maybe i don't know how far away we are but we are not there yet to be able to do that so it sounds like tree sitter is driving a lot of the innovation around these cool new plugins that i keep seeing that are neo vim only not vim as well because of the tree sitter support yeah i can give you a really cool example of a few things that people have been doing with tree sitter that are like much further beyond just highlighting please do yeah so a relatively like straightforward and easy to understand one is a plugin called nvim ts comment string i think off the top of my head i don't remember exactly the
Starting point is 00:57:22 name but what it does is tree sitter tells you what language you are currently inside of. And those languages can be embedded inside of the same file. So for example, in JavaScript land, you can have, you know, like react elements. And so then inside of there, you're kind of like in reactor. I don't really know how any of that works because I write backend languages, but all of a sudden I'm like, am I reading HTML? I thought there was JavaScript. I don't know what's going on. But one thing that's kind of annoying is if you use like a commenting plugin and you try and comment something in the like JavaScript part versus like the HTML style part, you'll comment them incorrectly, right? So yes. And thents comment string, all it does is it will just update the comment string option, which is a built in option, depending on which language tree sitter currently
Starting point is 00:58:14 detects. So if it detects that you're inside of JavaScript land, it'll set it to slash slash space percent s. If detects that you're inside of HTML. It sets it to the long HTML thingy. That's annoying to type. Right. That's cool. That one just honestly rocked my world because I've had that problem in my life for ever. Yeah. And so this is even cooler because it doesn't just work for those languages. In Lua, you can write FFI code, which is actually C, right? And it is somewhat deterministic how you start and end those C def blocks. So you can actually write a tree sitter query that tells NeoVim, hey, inside of these, this string, it's actually C code. And so you get C highlighting in this like random string in your file
Starting point is 00:59:02 that you weren't expecting. And then if you're using TS comment string and you comment out a line in there, it comments out that line like it's C code. So it's very, very powerful to be able to do something like that. It's very exciting. That's awesome stuff. That's one example. I'll give you another one that I think is, for me, it's very fun. So I use a snippets plugin called Lua Snips and it's kind of interesting.
Starting point is 00:59:26 I've just been exploring it lately, but it allows you to sort of run Lua code as you're expanding your snippets to sort of generate what the next text should be. So if you're familiar with Golang at all, you know that there's like, you write if error not equals nil, like 10,000 times a day.
Starting point is 00:59:44 Maybe that's an understatement. I haven't profiled myself yet, but I think it's somewhere around there. And what you do inside of those error not equal nil sections is you have to write return and then the type signature basically with the default values, right? So if you return like an int and error,
Starting point is 01:00:02 then you have to do like return zero comma air, right? So like you write this thing a ton of times. So I actually wrote a Lua snippet that will ask TreeSitter, basically, it has a custom query that I wrote inside of Golang that will ask TreeSitter, hey, what is the return type of this function? So it can query the tree. It gets back the current node that you're in. It asks for, hey, what is the nearest function scope that I'm inside of? So it's not a regex, right? It works for like inline functions inside of another function, all these kinds of things.
Starting point is 01:00:37 It asks for that. It gets what the return signature looks like. And then it generates the snippet in the corresponding things with the default values and then error so it's very cool because i can just type like ie for if error expand my snippet and correspondingly the right signature is generated as a return value these are the kinds of things that make me very very very excited about TreeSitter. And why I say like, it's cool to get better highlighting. And like, we look at our highlighting all day. And that's very fun. But for both LSP and TreeSitter, the things that I'm most excited about, like,
Starting point is 01:01:15 especially on NeoVim core team, is we're trying to design interfaces that empower people to have much stronger sort of tools and understanding of the code that lets them extend their editors in sort of very unique and interesting ways which is like i think the snippets one is a fun example of being able to do that goodness gracious that is really neat we're getting tight here but we've teased telescope a few times but let's yeah let's hit the nail on the head tell everybody about your peer into it oh there you go yeah that's why we pay you the big bucks so so telescope is a fuzzy finder that uh i started and i've had a lot of contributors to over the
Starting point is 01:01:56 past year i i cannot thank enough the contributors to the project in just over the last year, it's gotten over 3,000 stars and nearly 130 contributors. And I think that by itself is somewhat of a testament to how accessible Lua is, how interested people are to extend the editor and to like continue working on it and their ability to sort of pick up little pieces and do it. But in terms of like what telescope is and why like I made it, I partially made it just to explore writing some stuff in Lua as a way to push our API and boundaries to find out what we should be including in NeoVim core. But as I was working on it, some of the things that I've struggled with doing in the past with fuzzy finders, I just wanted to try and solve. For example, like I love FZF. It's a very
Starting point is 01:02:45 awesome tool. I use it all the time on the CLI, but it's very difficult to do something like pass a function through FZF because you have to encode it in some way and then decode it again later because FZF processes standard in and standard out, right? So Telescope is all Lua from the top of the stack down, or FFI, but that still counts. And so you can pass a function reference, which might be what you want to do if this item is selected, at the beginning of your code, and then you can pass that all the way through.
Starting point is 01:03:21 And so when you select it, you can do something with that item. So my main goal for making it was just to basically create like the most extensible fuzzy finder that I could imagine. So like every part of telescope is configurable. So like the sorting algorithm is just an interface of a function. So you can have FZY sorting, FZF, strict matching, Ngram matching. You can write whatever you want to do and you can just plug that in and that will just work with everything else. The previewers are all just Lua functions. So some of them, like if you're previewing buffers,
Starting point is 01:03:57 it just literally opens the preview in a NeoVim buffer. So you get exactly the same highlighting as you would if you open the file because it it is literally you opening the file, the like ability to sort and how you're sorting those. And when you filter them, those are all configurable, the display and UI and different themes, they're all like, individually configurable. And what like, my goal was, was to make the most configurable like fuzzy finder that I could do with an easy api to basically plug in whatever you can imagine that you wanted to sort on and then use it so all my to-do list was to yank nick's fzf neobim setup and use it for myself should i just bypass that is telescope ready for me should i just use it right away or is it still baking? Yeah, so there's a couple things
Starting point is 01:04:45 still baking for sure in Telescope. I mean, FCF is just like a rock solid piece of software that's super cool. If you're doing things that might be searching millions of files, for example, you should stick with FCF. Go is going to be a better solution for you than what I can do in Lua, at least today. We'll see. Maybe someday it'll be different. We are nearing completion of merging a very important PR that will greatly improve performance for really, really large searches, like 500,000 items will still feel fast and not really block the editor. So it's up to you. I would say if your primary concern is speed, then you should probably stick with FZF at least for a while. Maybe you can talk to me again in a year and maybe I feel
Starting point is 01:05:31 I'm close enough to FZF speed to say that we're there. We're working on some projects like in C to compile as optional dependencies that you can throw in here and work super fast and cool. But we're not there yet. And that's cool. Like I said, I like FZF, but I use it every day. I no longer use FZF inside of NeoVim. I have lots of friends who no longer use it inside of NeoVim or other fuzzy finders. They just use a telescope. So I would say it's ready for daily use, but you know, if you're going to be searching 10 million things or something like that, then running something inside of the program that you're doing is probably not a good call. You should use an external executable that's going to manage that memory maybe a little bit tighter. I have an answer for you.
Starting point is 01:06:15 Why not both? Why not both? Yeah, I have recommended that for some use cases for people, like if they're trying to grep every line in their very large monorepo, there's no reason not to keep FCF around, at least for a while. For me, like I have a hard time if the UI like isn't as consistent. It just sometimes like, oh, like notice that or like, oh, it's not the same as what I'm used to. So for some people that really bothers them. And I understand that's totally cool. And they say they're going to stick with FCF. And I say, awesome. That's great. That is one thing that I think telescope really has going for it is it just has a really beautiful UI. Thanks. That does help, doesn't it?
Starting point is 01:06:53 Yeah. And I mean, it is very interesting when you actually start using it and you notice that you get exactly the same colors as what the rest of your editor is. I don't have that with FZF. Yes, it is not possible as far as I understand, because like, for example, when you open up a buffer inside of NeoVim and if you have TreeSitter, it will literally TreeSitter highlight the same way with the same exact colors because it is just another buffer. It would be like if you did colon edit this file, it will show it like that in the preview. Very cool. So one thing I want to touch on as a closer is I mentioned the enthusiasm around the 0.5 release. I mentioned how many people said win NeoVim on the changelog.
Starting point is 01:07:35 There's a lot of people very excited about this project. They love the ability to write plugins in Lua. They like the new capabilities that TreeSitter is making available to them. Maybe speak to the community a little bit. I know you lead some live streams. There's a lot of fun being had. There were celebrations around the big release. And then maybe give waypoints for people
Starting point is 01:07:58 who would love to get involved with NeoVim, the community, like where does they gather, etc. Yeah, so we did a 0.5 release stream on Twitch on my live stream channel. And it was like, the excitement and enthusiasm was really mind blowing for me. You know, I've been working on NeoVim for a long time. And sometimes you're sort of like, just working and you're pushing stuff to GitHub and like, OK, cool. Like somebody's using it, I'm pretty sure. But we had like 500 people there live to do the like release drop on my stream. And it was like really crazy for me.
Starting point is 01:08:35 And people were showing a lot of support. And so it's very exciting. Even, you know, just getting messages from people saying thanks. Just as a general note, if you're using open source software and it's especially if it's maintained by people who aren't getting fang but like salary to do the maintenance, it means a lot when you get a thank you
Starting point is 01:08:54 or like, wow, I really like this. Do that. And I mean, I was really pleased to see Neovim in the top spot for most loved editor in Stack Overflow 2021 survey. A little shout out there. But yeah, I think the community is great and it's really exciting to see how much people
Starting point is 01:09:11 are excited. And I've really appreciated it. And it's been fun sort of being on the receiving end of that. In terms of like, where can people get started and how can they help? First thing is we're on Element or Matrix, I guess. There's a NeoVim chat room there. I believe it's linked from the read me. If it's not, then it should be. And so you can go there or it's on Gitter or it's on IRC. They're all sort of like bridge to be the same place. And you can hang out
Starting point is 01:09:38 there and chat or ask questions. There's a lot of issues that if you commented on and said, I want to work on this or like, how can I fix this? People are super glad to help. I actually, that's how I got involved. I did a PR in maybe during my senior year of college, adding a new thing to status line. And I just had like a ton of really nice people be there to help and to encourage me and give me advice. And they helped me grow a lot as software developer, even on just that one PR and taught me a lot of really great things.
Starting point is 01:10:13 I hope and I think it's a goal of the other maintainers as well that we basically make a welcoming and happy place for people to learn a lot more. So don't be afraid to ask questions or to comment and say, I'd like to work on this. Where should I start? That's how I got started. And so I think that's a really great way to go. There are some labels like good first issue or mentored project or things like that
Starting point is 01:10:36 that you could look into, but don't be afraid to leave a comment or say hello. We're super happy to see you. Or just say, thanks for working on it. That always makes me feel better and gives me energy to keep on you know improving the oven and working on it awesome we definitely appreciate all of the work that you're doing i'll just echo your sentiment if you have a piece of software that you use and love find the people that make that software and let them know especially if it's open source but proprietary as well there's people behind the scenes of proprietary software you know putting the work in so that's definitely appreciated nick i want to
Starting point is 01:11:10 thank you for being my changelog co-pilot hopefully none of the things you generated today were gpl code but uh happy to have you here with me i think i broke the show with that one tj we really appreciate you coming on the changelog to To everybody who requested this episode, we thank you as well. We do take requests. Head to our website changelog.com slash request. Fill out the form there. Let us know guests you'd like to hear, topics you'd like to hear. If you want Nick to come back, if you'd like us to banish him into eternity, let us know. We'd like to hear from our audience. Any final words, TJ or Nick? Just thanks for having me. It's cool to see NeoVim be requested on the show, and I'm always happy to talk about it. If you want to see more of me or hear my voice, I stream a lot on twitch.tv slash teej,
Starting point is 01:11:58 that's T-E-E-J underscore D-V. You could come hang out there. We're live pretty often, and I do a lot of NeoVim- related work there. Otherwise, just glad to be here and glad to be participating in such a fun community. Yeah. And thanks for having me on as well. And TJ would gladly take you up on that offer to dig into some custom LSP stuff. That's really cool. Cool. We are tentatively planning some related live streams. We have nothing locked in, but definitely follow TJ on Twitch and maybe on Twitter as
Starting point is 01:12:27 well for announcements. Of course, follow Changelog on Twitter. You can also follow Nick Nisi on Twitter, but I don't know. You decide if you want to do that. Thanks, Jared. All right. That's our show for this week. We'll talk to everybody next time. Sorry, Nick.
Starting point is 01:12:44 In true JS Party fashion, I had to roast you a little bit just at the end. Absolutely. Gotta love roasting Nick. That's it for this episode of The Change Law. Thank you so much for tuning in. Got a special show coming up soon with Corey Wilkerson on GitHub's transition to Codespaces. Adam Jacob on open source business models.
Starting point is 01:13:00 FaunaDB with Evan Weaver. If you're not subscribed, now is a good time. Subscribe at changelog.com and everywhere you listen to podcasts. We'll see you next time. Game on.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.