The Changelog: Software Development, Open Source - Rails as a day job, Diesel on the side (Interview)
Episode Date: November 4, 2017Sean Griffin joins the show to talk about doing Rails full-time, his love of Rust. and his project Diesel - a safe, extensible ORM and query builder for Rust. We discuss Sean’s path to working full-...time on Rails, what he works on specifically, why Rust, why Diesel, and how much of Diesel’s design and featureset is a product of his experience with ActiveRecord and Rails.
Transcript
Discussion (0)
Bandwidth for Changelog is provided by Fastly.
Learn more at fastly.com.
And we're hosted on Linode servers.
Head to linode.com slash changelog.
This episode is brought to you by Linode, our cloud server of choice.
Everything we do here at Changelog is hosted on Linode servers.
Pick a plan, pick a distro, and pick a location.
And in seconds, deploy your virtual server,
drill-worthy hardware, SSD cloud storage, 40 gigabit network, Intel E5 processors, simple,
easy control panel, nine data centers, three regions, anywhere in the world they've got
you covered.
Head to lyndo.com slash changelog and get $20 in hosting credit.
Welcome back, everyone. This is the Changelog, a podcast featuring the hackers,
leaders, and innovators of open source. I'm Adam Stachowiak, editor-in-chief of Changelog.
On today's show, we're talking with Sean Griffin about doing Rails full-time,
his love of Rust, and his project, Diesel,
a safe, extensible ORM and query builder for Rust.
We talked about Sean's path of working on Rails full-time,
what he works on specifically, why Rust, why Diesel,
and how much of Diesel's design and feature set is a product of his experience with ActiveRecord and Rails.
All right, I'll jump right into it.
Added helper types for inner join and left outer join.
Diesel debug query has been added as a replacement for debug SQL.
Is this your changelog?
Added support.
Yeah, is this not where I come on and dramatically read Diesel's changelog?
That was pretty
good. That was good, yeah. Keep going.
I'm way unprepared if that's not what this is
about. I was entertained
for a split second. Literally
the changelog. Love it.
Sorry, I had to get one
dad joke in early on.
I came up with that one like two days ago
and I've been waiting for it.
I like it. We should leave that in. I don't know why. This is up with that one like two days ago, and I've been waiting for it. I like it.
We should leave that in it.
I don't know why.
This is tape, man.
Let's go.
All right, let's go.
Well, now that we know what kind of guy you are, Sean,
you'll fit right in here on the show.
But we have two subject lines.
I believe they're intertwined, as most things are,
and that's the subject of you full-time on Rails and your work there on ActiveRecord.
And then secondly, what you're up to with Diesel, which is an ORM and query builder for Rust and the interrelation between those two things.
But let's start off learning a little bit more about you and your path to what we often call living the dream, which is working on open source full-time.
And in a way that seems like a pretty awesome way.
So tell us about your work with Rails, how you got involved,
and how you ended up full-time at Shopify.
Sure.
So let's see.
So I got started with Rails, gosh, like four years ago now, I think.
I had done a commit here or there, but nothing particularly major.
But I'd been on some projects for a while
that all sort of had these weird, similar
needs in terms of modifying
attribute accessors
on ActiveRecord objects.
And then at, I think it was RubyConf
like four or five years ago,
Ernie Miller gave a talk
where he was talking about how Rails is missing
a lot of APIs that are kind of one level down in abstraction.
And so all this culminated in me wanting to build out
what is now today known as the Attributes API.
And it just so happened that Aaron Patterson was,
I lived in Denver at the time,
and he was in town for a local meetup to give a talk.
So I went out with him for drinks afterwards and was like, Hey, I have this idea for this API.
I got really, it sounds polite, but I was actually really drunk and kind of shouting
at him about this API. Whatever works. But anyway, so he was like, yeah, sure. Whatever.
Do it. I think, I think he was partially just trying to get me to stop asking him about this idea that I had,
since the majority of people who do that are never going to follow through.
Anyway, that resulted in a pull request that was way too large to be reviewed,
and then a series of smaller pull requests over time.
And ultimately, I vastly underestimated the amount of work that
implementing that was going to require.
And when it was all said and done,
I had rewritten a significant chunk of active record for like non
association and non ARL related things.
Um,
at which I was like,
well,
crap,
I guess I'm maintaining this code for the rest of my life.
Um, so that I merged then. Oh yeah. well, crap. I guess I'm maintaining this code for the rest of my life.
So I assume that got merged then.
Oh yeah, yeah, definitely.
What was that like to feel like you're maintaining it for the rest of your life?
Are you being facetious or are you being serious?
I mean, both.
I didn't really go, well, I'm maintaining this for the rest of my life. But yeah, it ultimately was.
Part of why I ended up sticking around was just because
I felt a
responsibility to be around to fix
the bugs in this code that I had submitted
and
kind of make sure
that it continued to evolve in the way I had envisioned.
So just real quick for clarity
for those who are not in the Ruby on Rails scene,
ActiveRecord is the ORM or the database library that ships with Ruby on Rails
and is a significant portion of the Rails code base
and probably a significant portion of the complexity and a lot of the bugs.
And so rewriting any small subset of ActiveRecord itself is not a small undertaking.
And probably, Sean, you know better than anybody
at this point, as maintainer of ActiveRecord,
while thousands, if not tens or hundreds of thousands
of people use ActiveRecord,
there's probably a group of people
that you could count on one or two hands
that are intimately familiar with the way that it works.
Is that fair?
Yeah, I think so.
And to give some actual numbers,
I don't know what it is in the code base today,
but around Rails 4.2,
so not including action cable
and not including active storage.
Active Record accounted for about 80% of our code base.
And back then, I would say it was about 80% of our code base. And back then, I would say it was
about 80% of our issues as well.
Certainly, I would guess the percentage
of the code base has gone down
just because we've added two additional libraries.
But probably still similar,
probably like 70%, I would guess.
Although interestingly, I've been noticing
that it is starting to become a smaller percentage of our opened issues, which is good.
Yeah, you can probably see that as a win since you've been dedicating a lot of your time there.
So you open up this massive PR, this idea that you were challenging Aaron Patterson,
who was then very involved with Rails. I'm not sure of his current status. In fact, he was full-time on Rails,
I believe working for AT&T Interactive
or somebody was paying him full-time
to work on Rails back then.
He now works at GitHub
and I'm assuming is still very much involved.
And so he said, go ahead and do it.
You went out and accomplished this goal
and now a lot of the ActiveRecord code base is your own
and so you began maintaining
it um that doesn't get us to full-time at shopify so tell us like the personal slash business end
of that same timeline and how you ended up being able to do this as your job uh yeah so i mean
i guess the first thing is what is when i decided that this is what i wanted to do
uh because it's funny you call it living the dream and to a certain extent i kind of call I guess the first thing is when I decided that this is what I wanted to do.
Because it's funny, you call it living the dream, and to a certain extent, I kind of call it living the nightmare.
That's the great irony, isn't it?
Yeah.
But basically, I'm bad at work-life balance.
And I started to realize that if I didn't do this as my full-time job, I had two full-time jobs. So I set out to switch doing it full-time
simply because I'm bad at convincing myself
to not work.
And if I didn't do it during work hours,
I was just going to be doing the same amount of work
on my own time.
So it was purely a move for my own sanity.
I sort of really started to get a taste
for actually spending a significant amount of my time doing it.
I was working at Thoughtbot as a consultant at the time,
and I had some significant downtime between projects,
and so that was when I really started to get to
try to start spending more of my work hours doing it.
And I'd been looking for ways to convince management that it was a worthwhile thing
for me to be able to continue doing uh ultimately just the interest didn't didn't align there um
so i ended up leaving and i ended up at shopify because uh basically i went to conferences and
begged a lot of people to give me money and Shopify ultimately was a company
that saw the value proposition.
When you say full-time open source,
it's a convenient way of summing up a job
with the emphasis on
the focus of your job is working on open source.
It by no means means,
it does not mean that I
spend 100% of my time working on open source.
Open source
is more than just code.
It's a lot more than that.
It's like documentation, community,
even just research and development, I'm sure,
is not just the things you do.
Yeah.
How long was this time frame of you petitioning the community,
so to speak, with what you wanted to do
and how you wanted to do full-time.
How long was that span of time?
About six months.
Not too bad.
And you were full-time at ThoughtBot during that time?
Yeah.
Let me just do something unprecedented here on the changelog.
I'm going to read, Sean, with your permission,
I'm going to read Sean's LinkedIn bio.
Can we do that?
He was reading a changelog to us.
Oh man, I haven't touched this in a long time.
Go ahead.
I'm assuming you haven't changed it for a while
because it sounds like it's still in your petition phase.
Oh no, I had that there
just to get recruiters
to stop spamming me.
Oh boy.
I enjoyed this.
I knew that Sean was at Shopify
but I wanted to see where he was previously
Thoughtbot makes a lot of sense
because that's probably where you started doing the bike shit
and so I saw this on LinkedIn
and I thought it was funny
he said I'm currently only interested in opportunities
that will allow me to focus primarily
on open source contributions
the best way I can contribute to the community
is with my work on ActiveRecord
that includes your company.
If you agree, drop me a line.
My dream stack is the one where recruiters leave me the hell alone.
Please read the previous paragraph.
If you think that, quote, focusing on open source contributions and, quote, working at
a company that uses open source are the same, then go away.
It's interesting there because you're very clearly stating at that time, and probably to this day,
that you were interested and you were set
on opportunities that allow you to do
open source contributions full-time,
or primarily is what you said,
which is probably more fair than full-time.
Then of course the call to get recruiters to leave you alone
is hilarious because I've been on LinkedIn for many years,
and the only people who've ever contacted me on LinkedIn
is recruiters.
I haven't had a legitimate request ever.
So that was kind of funny, and perhaps written back
when you were ready to go from a consultant at ThoughtBot
to maybe on the path that you're down today.
So tell us, at Shopify, you got the job.
Can you tell us what your official position is?
And you said it's more than just doing open source stuff.
What all is involved in your role at Shopify?
Well, my official position is 10x Hacker Ninja Guru.
Oh, good.
And if you're wondering what that means,
it means that I once thought it'd be funny to see what happened
if I put that on a government form related to my work permit.
And it turns out what happens is that I cannot legally work in Canada
with any job title other than 10x Hacker Ninja Guru.
Oh my gosh.
That is so true.
Oh, wow.
So, I mean, my job...
So one of the things that...
Right now I'm the only person who focuses on Rails
as their full-time job.
And one of the biggest challenges that we face as a project
is just issues and pull requests come in at a greater volume
than we can generally process them.
So I try to keep at the very least
one of the baseline things I do as part of my job
is day-to-day, or I guess really week-to-week,
open issue count doesn't go up
and open pull request count does not go up.
So probably the majority of my day
is just doing issue triage more than anything else
and reviewing pull requests and merging or closing them. the majority of my day is just doing issue triage more than anything else.
And reviewing pull requests and merging or closing them.
Does it feel like you're making significant
when you say that the majority of your time is issue
triage and
when I think about different
ways that people work on open source and
the ways that I think companies should
invest in open source,
I think the way that Shopify is going about it is necessary in terms of pushing
the thing forward because certain projects,
certain tasks and needs are so intellectually or not stimulating,
but stressful or require,
they require so much thought that you couldn't possibly one week,
one day a week or one week a month or two hours at
the end of your day make significant process, like for instance, on a rewrite of ActiveRecord
or a portion of it. And I think we need people who are thinking about these things all the time,
because that's how you move software forward is you have to have the entire system in your head.
And I like to think, hey, at least Sean Griffin has ActiveRecord in his head and is moving it forward. And then when I hear you say,
most of my time I'm triaging issues and hoping they don't, I'm not working, it's not that I'm
working them down. It's that I just don't want them to get any bigger. And I wonder, wow, is,
are we moving forward or are we just maintaining the status quo?
I mean, a little bit of both.
I think that moving
forward also has a different meaning for ActiveRecord
today than it used to.
If you guys
go to a RubyConf or RailsConf,
unless the tone is very different
than it was a few years ago, there'll be a lot
of people like, oh my gosh, is Rails
dying? Is Ruby dying?
I think the notion of that
in general is just sort of silly,
but I think a lot of where that comes from
is a misinterpretation of the signals
that come from just we as a community
are no longer at the point where we're going to be
hip and breaking new ground.
Ultimately, Rails has shifted
to a mature, stable framework.
So to me,
it's actually,
it's interesting
because I've been finding it
harder and harder
as time goes on
to convince people
why they should
upgrade to new Rails versions
because we just don't have
as many killer features
as we used to.
But there is a lot of work
that goes into each release
and I think a lot of the things
that people are missing...
At Shopify,
I end up pairing with some people quite a bit
on trickier
test failures
that they run into as part of upgrading
Rails versions. And
one of the ones that I've seen a lot recently
was I changed how
dirty behaves in after-save
callbacks in ActiveRecord. So basically, if you ask if an attribute has changed inside behaves in after-save callbacks in ActiveRecord.
So basically, if you ask if an attribute has changed
inside of an after-save callback in 5.2,
that's going to behave the same as if you had that code,
rather than in a callback,
if you actually just put that code directly
after the call to save,
so it behaves as if it was after save.
In 5.1 and earlier,
the way it works is we do the persistence,
we run the after save callbacks,
and then we clear the dirty flags.
And so in 5.2, the way that's going to work
is we do the persistence,
then we clear the dirty flags,
and then we run the callbacks.
So there's a bunch of new methods
that are also more clear on what question
you're trying to ask.
So the new methods are all named,
do you want to know if there is a change
that has yet to be saved,
or do you want to know if there is a change
that was just persisted?
That's nice.
And all of the places that I've run into in Shopify
that I've inadvertently broken things
are places where there's a bunch of code
that is in an after save callback
mutating something
and then expecting changed attributes
to include both the things that were just saved
and the thing that has yet to be saved.
And that actually just doesn't work in 5.1.
It'll only, that change would only be reflected
in the do we have an unsaved change method.
Anyway, so every time that I've gone in with somebody to look at how to fix this test failure,
ultimately, we've realized that the code was really, really funky to work around this quirkiness.
And once we fixed the test failure, we actually were able to just delete a bunch of this hacky
code that was working around a Quark inactive record.
And I think that's a thing that a lot of people don't realize
going from version to version is, for me,
a successful Rails version is when you can
upgrade and delete a bunch of your hacks.
That's definitely progress.
Especially when you talk about moving from a state
of franticness
or from lack of clarity or
conciseness to a state of like a good thought through API.
And when you have that many people using different versions, you know, just
moving a thing from step one to step 1.5 perhaps requires a lot of work.
Such as you say, clarifying those specific,
even just the method names in 5.1 versus previously.
And so a lot of work goes into it.
On the outside, when you're looking at the new feature,
it has less of the wow factor,
less of the big wins that we were getting,
perhaps in the first 10 years years when there was obviously things missing
and huge gains to be made.
Certainly, I think probably active storage is the thing
that we've added since Rails 5 is the first time we've had something
I think everybody would agree was just a thing that almost every application needs
and was obviously missing from Rails.
Tell us about Active Storage.
This is the cloud storage adapter type stuff?
Yeah, it serves a similar purpose to refile
or why am I spacing on the name of the other one?
That's really popular.
I'm still using Paperclip.
Does that make me old school?
It does, yeah.
Carrier Wave is the other one I was thinking of.
Carrier wave, yep.
Paperclip is a thing that definitely,
like, it used to be carrier wave and paperclip,
but I don't think anybody's really using paperclip anymore.
Well, you found him.
You know, on that note, though,
to pull, maybe this is a wrench
to some degree in the conversation,
but you said, and maybe it's even something you can't speak about, but you mentioned how you left ThoughtBot because management didn't align, which I'm not going to assume exactly what you said.
But it was something about your desire to do this full time.
But then like Jared, you use Paperclip.
That's a ThoughtBot project, right?
Right. full-time but then like jared you use paperclip that's a thoughtbot project right right and
there's so much open source out there around rails that's almost default for many rails developers
that's from thoughtbot so you would assume almost that thoughtbot and management of thoughtbot would
see eye to eye with your future sean like what you were trying to do and that's in hindsight
seeing this and like this conversation just reminded me
how prolific ThoughtBot is to Rails and open source.
And I don't know.
I kind of have a question mark on that.
I mean, I don't want to undersell
their dedication to open source at all.
Right, of course.
But there's a big difference between a developer
who is billing versus a developer
and works on open source one day a week because Thought billing versus a developer and works on
open source one day a week because
Thoughtbot just does not bill on Fridays.
So there's a big difference between that
and somebody who is not
billing at all.
And you need to be somebody that was doing this
consistently day-to-day, full-time.
Yeah. And that just wasn't possible.
Well, yeah, exactly.
I think, so it's a big question of just, I don't want to be working somewhere and that just wasn't possible. Yeah, exactly.
It's a big question of just,
I don't want to be working somewhere because it's charity.
I think that for the right company,
I very much do earn my paycheck and bring sufficient value to the company
to justify my job's existence.
So I say our goals didn't align.
I mean that it just was not a situation
there where I was going to be able to
justify my paycheck there.
I shouldn't.
Sorry, I should have asked that because I was like,
hearing paperclip here in this conversation
reminded me of that.
They do such great work and that's why I just had that
question mark.
Yeah.
It was probably a couple years ago now, we were on the giant robots podcast and that and back when
ben was hosting it and i even said to him that day that like my career has very much been just
following the thought bot and what they do and using their open source tools and even today like
our new stuff is not on ruby on rails Our CMS is on Elixir and Phoenix,
and yet we're still using some ThoughtPot libraries
that happen to be Elixir-based.
So couldn't speak more highly of the open-source stuff
that they've done through the years.
Absolutely.
Yeah. This episode is brought to you by GoCD.
GoCD is an open source continuous delivery server built by ThoughtWorks. It provides continuous delivery out of the box with its built-in pipelines,
advanced traceability, and value stream visualization.
With GoCD, you can easily model, orchestrate, and visualize complex workflows from end to end.
It supports modern infrastructure with Elastic On-Demand Agents and Cloud deployments.
And their plugin ecosystem ensures GoCD will work well in your unique environment. To learn more about GoCD, visit gocd.org slash changelog. It's open source and
free to use, and there's also professional support and enterprise add-ons available from ThoughtWorks.
Once again, gocd.org slash changelog. back to active storage now because i've've only read it in brief, but give us, I think this is probably a big deal in terms of like, this is a flagship type of feature.
Tell us a little bit more about it, when it's coming, that kind of stuff.
So it's coming to Rails 5.2, which as far as I know, we haven't actually announced a date yet.
But it's currently on Rails Master.
And it's like I was saying, it serves a similar purpose to Carrier Wave and Refile.
And I'm honestly not intimately familiar enough
with either of those libraries or Active Storage
to tell you specifically how it differs,
other than generally, as with most things
when they come into Rails,
API that kind of better suits what we see as the Rails way.
I think the one thing that it does much more up front
than any other solutions thus far
is it has a big focus on making direct upload
for a service like S3 much easier and more integrated,
which is a big deal if you are expecting large file uploads
and you're hosting on something like Heroku
where if your user takes more than 30 seconds
to upload that file, you're screwed.
Or actually, if the combined time for them
to upload to your server and also for you to upload
to S3 or otherwise transmit to your database
or wherever else you're going to store it
because you can't store it on the file system of the server.
The second part of that transaction is usually pretty fast
because Heroku's on AWS infrastructure. So as long as you're on AWS also,
it's fast, but yeah, point taken.
Anyway, direct upload is a thing that you end up needing.
I used a gem for it in the past, actually, with Paperclip, I think.
That worked fine, but now it's built in, out of the box.
Yeah, just looking at it from the maintenance point of view,
it has an interesting structure just for all the files themselves
actually do live on a single table, which is, I don't know if,
I've never looked at refile.
I know CarrierWave, as far as I know, doesn't structure things that way.
I know Paperclip did it as columns on your table of the model it was attaching itself to.
So that's kind of interesting in that
you can always just add attachments
to a model without needing a database
migration.
Like I said, I really can't speak to
specifically how it differs from the other libraries
out there too much.
One last topic on the
Rails and full-time aspect
before I move on to talking about your new shiny, which is diesel.
I was recently on your guys' show on The Bike Shed.
If you all don't listen to Sean and Derek Pryor on The Bike Shed podcast, check that out.
We'll link up that episode in the show notes where we talk about funding, sustainability, so on and so forth, platforms.
If you're not sick of us talking about that, go listen to that episode. If you are sick of it,
well, bear with us a little bit because we've got to get this figured out.
One of the things I said on that episode was that
I would love for there to be more Sean Griffins out there and more Shopify specifically.
Shopify is a company that I think because of
its leadership and because of its roots in programming and Rails, I think gets it better than most in terms of the value of having a full-time staffer working on the infrastructure that they rely upon.
But first of all, do you think there's a future for more people in roles like yours in corporate?
I'm going to say corporate America,
but corporate world?
And then secondly, if so, how do we go about,
like, I don't know, getting more John Griffins out there?
Oh boy, that's a loaded question.
Yes, I think there's room for more people with similar roles.
I think that for it to be successful, it does need to be a job that is worth having
regardless of the PR aspect of it.
So it needs to be structured in such a way
that it brings value to the company.
There's a couple of things that are always going to exist
that bring value for having a full-timer.
Number one is
I sort of try to
avoid specifically doing something
in Rails because Shopify needs it.
That said, I tend
to fix the problems that I'm most exposed to
and I'm exposed to Shopify's problems
more than other people's problems, so
things tend to get priority that way.
And certainly when I do
just know something that, regardless of whether of Shopify or not. And certainly when I do just notice something that regardless of
whether of Shopify or not that just when I'm
pairing with somebody like oh that's bad we should
fix that there's a certain
there's a certain benefit to
just when you notice something during
that sort of work to be able to just go and commit it.
And then the other one is
just having the
resource available of a person who can answer questions
that not many other people can
and can generally act as sort of a multiplier
on the rest of the team.
So you act as a liaison essentially
to answering questions in and around Rails
to Shopify and the developers there.
Yeah, and sometimes it's more beginner questions
that other people could have answered, and that's fine.
There's also benefits to just having a flexible enough schedule
to be able to spend as much time as people need answering questions.
But every now and again, there will be a time
where I'll be helping somebody debug something,
and we'll spend two or three hours on it, and we'll figure it out.
And it'll be the sort of thing where they,
it would have taken them a day or two otherwise.
And those are the days that I feel really good about,
about,
about my job.
So that's the part that feels good about giving value back to the company
that's,
you know,
obviously employing you,
not just the success you want to have as being full-time open source on
rails.
Those are kind of two different things to some degree?
Yeah, I mean...
Because it sounds like you wanted to be employed somewhere
to be full-time doing what you want to do,
but also give the value back.
So it seems like it's two-headed in some capacity.
Right. Like I said, full-time open source
does not mean 100% of my time is spent on open source.
It's just a nice way to bundle up
and the focus of my job is open source.
I think one of the benefits that just comes from it is
if somebody does just need somebody to pair with them on a hard problem,
whether it's Rails-related or not,
I tend to be less bound to deadlines
because I don't work on the product.
So I'm much more available than most people are
for whatever impromptu, for answering questions,
for pairing with somebody on whatever,
doing code review.
Shopify could probably use a few dozen people like that
with the number of developers we have,
but there's certainly a lot of value
in just having people who are available,
senior developers who are available
just to help in general,
who the answer's never going to be,
I would love to, but I really can't
because I have to finish this feature.
I've been trying to think of a metaphor that works well
when discussing maybe the roles of developers
inside of a company.
One that gets to a scale where it's beyond three people
or a small team, where there's multiple teams of developers doing different things.
And what makes sense in terms of business language that people can use with regards
to roles like yours? And I think one metaphor I've been thinking about is in terms of
kind of back office, front office type of employees.
I'm not sure how business-centered y'all are, but with regard to the people who are out in sales,
marketing, HR, HRB, back office,
certain customer-facing jobs
versus accounting, controllers,
those types of roles
where the people up front are the ones, quote
unquote, making the money, right?
And the people in the back are the ones, quote unquote, saving the money or managing the
money.
And with developers, I think at a certain point, when so much of your infrastructure
is outside of your control, in the case of software as a service, and you get to a certain
size, and you have hundreds of thousands, if not case of software as a service, and you get to a certain size,
and you have hundreds of thousands, if not millions of lines of code that are maintained
by people that don't work for you. I think having kind of your front office developers,
these are the people who are working on new features, new products, so on and so forth.
And then having kind of a set of infrastructure people, which is really nice. Like you said,
when somebody does need help or needs a pair,
you're generally more available
because you don't have that deadline.
I wonder if that's a metaphor that resonates with you guys
or if I'm just barking up the wrong tree.
No, I think it makes a lot of sense.
To a certain extent,
the reason I've tried to structure this position the way
that it is for me is just because
at the end of the day, if I'm working on the product,
I'm not going to be that much
more effective at building a Rails app
than any other developer is.
The majority of building
a Rails app is boring,
which is a good thing.
I don't mean boring like I don't want to be doing it,
but boring in that
just about any developer
is going to be able to do an equally good job
and take about the same amount of time.
And I do think that there is
a unique knowledge set
that comes with just maintaining the framework
that I want to be able to apply
as much as I can.
If you had to, what would be a way to template the kind of role you have to others?
Like if we can copy and paste what you do there, like is there a job description?
How do you outline the type of role?
I mean, obviously you covered the things you just did here now.
How do you take that to LinkedIn, GitHub? Obviously GitHub does probably a lot, but
Stripe does a lot. How do you take that to the non-deep tech companies and say,
here's how you embed somebody? Like Jared said, you have so much leaning on lines of code you
don't even control that aren't employed by you. how do you copy and paste what you do and apply it elsewhere?
So that's the thing is I don't know
that that's the way to grow it.
I think that for the people who want to do it full, full time,
it is going to be a very unique job
dependent on both the person doing it
and what they want out of the role
and also what makes sense for the company
that's hiring them.
I think really the way that we grow more people
in open source is we do just have more people
who aren't necessarily full-time on it
but are spending some portion of their time.
So there are
issue triage, for example, is the sort
of thing that can generally much more
easily be done on a day-a-week scale.
Smaller features
are something that can be done on a day-a-week scale. Smaller features are something that can be done
on a week-a-month scale.
Something that Shopify has been doing recently,
which I think is really cool,
is we have people within the company
submit proposals to the Rails team
of stuff that they would like to work on,
and then we pick some people,
and they come join the Rails team for a quarter
and just do open source full-time for one quarter.
It's not everybody in the company does it that way.
It's just whoever gets picked.
But ultimately, that is the sort of thing people talk about,
20% time, and I would wager 25% time is not a huge jump after that.
I think if more companies offered 20-25% time for open source and also gave
people the flexibility to
spend that time in
whatever chunks make sense for the thing
that they're trying to accomplish.
If they're just wanting to get their feet wet in open source,
Fridays makes a lot of sense.
If they're maintaining a project a week, a month might
make more sense. If they really want to dig into
building a big feature,
there are things that you just can't do
that take more than a week that you
just cannot effectively do unless you're able
to really go heads down,
spend your time thinking, spend
full time thinking about it.
And so allowing people even to go
a quarter a year.
I think if more companies offered that sort of thing,
that would be the better way to grow it
than more positions like mine.
Okay, one last question on this and then we'll cut ourselves off. Cause honestly, I think Adam and I could prod you about this particular thing all afternoon. Yeah. But the last thing
I'll say is, you know, I joke, I said that you were quote unquote living the dream. You jokingly
said, you know, kind of more like living the nightmare, but in all honesty, are you happy
with what you've achieved in terms of your
role and the ability to do it is it does it play out uh overall good or overall bad
another loaded question come on no that's not loaded that's fair no i mean uh i would i i i'm
very satisfied with how things have turned out and i I think I'm happier doing what I am right now
than I would be doing something else.
Well, that's the safest way to answer that.
That's my very political way of saying no,
but I'm going to be more miserable doing anything else.
Right.
No, I mean, so that was the other half
of kind of why I went into doing this full time.
I found myself getting less and less able to motivate,
get motivated by business problems,
which would be a whole rabbit hole.
So we don't have to go into that,
but that was discussed recently on go time.
We mentioned that in the pre-call jokingly,
Hey,
it's go time,
but we have a show called go time and it was discussed on their recent show.
I,
I,
uh,
I think it might've been chase Adams where,
or somebody else
i can't recall the person but if we do i'll link it up in the show notes that they were just saying
that they they uh they enjoyed building building developer tools more serving the developer the
developer was their customer rather than the product that the company was delivering you know
so like they felt comfortable in that arena much more than they did, as you had said,
which was dealing with business problems.
Yeah.
Well, then that's why open source as a full-time job can kind of suck
because how many developers give every single one of their customers
a direct hotline to them?
So Sean, you write a Ruby ORM as your day job
and for some reason you decided to build a Rust ORM
as what I'll just call your side gig.
Tell us about Diesel and why you decided to dive into the world of ORMs
once again, this time in Rust.
So just to preface one thing,
because I've been noticing a lot of people
tune out Diesel immediately with that label.
So I'd like to free face it with,
Diesel is very much more of a query builder first
and an ORM second.
That's funny because in our notes
I actually have a specific question.
Is it really an ORM and do we care to bike shed
the acronym for a while?
Because we can.
And I'm sure people do.
It is an ORM in that it does provide
some functionality to map rows in your database table
to objects in your system.
Okay. But more about
the Query Builder. Yes, but it's more
about the Query Builder than anything
else, because I think that's the more interesting part.
Why do people
tune out when they hear ORM,
do you think?
I think that when people hear ORM,
they think they have PTSD flashbacks
to the most painful parts of ActiveRecord
or Hibernate or Django ORM.
And they think about callbacks
and over coupling business logic to persistence logic
and all that good stuff.
Fair enough.
Anyway, so you had asked what made me want to jump back into it.
So Rust was a language that, I want to jump into it because Rust, basically.
And Rust was a language that I was sort of, before I ever actually tried writing a line
of it, was aware of and interested in, just because I was very into
various functional programming languages,
mostly Scala and Haskell.
And I found Rust interesting
because everybody's trying to solve
the problem of shared mutable state,
and most functional languages
are going about that by removing mutability,
and I found Rust, if nothing else, novel
because it went about fixing it by removing mutability and I found Rust, if nothing else, novel because it went about fixing it by
removing sharing.
And so it
piqued my interest if for no other
reason than
because it was doing something
novel.
So I
had inadvertently, towards the end of my
time at ThoughtBot, I had inadvertently
become the 3D rendering engine guy.
So I was on a client project.
Basically, I wrote a 3D rendering engine,
and it turns out when you do one of those,
there's a lot of 3D rendering engine contracts
that come out of the weeds.
And why I ended up having to write one
is a story that'll take me 10 minutes to tell.
But basically, out of necessity, I ended up having to write one is a story that will take me 10 minutes to tell. But basically, out of necessity, I ended up having to write one for a different project
at sort of the beginnings of WebGL being supported by browsers.
And then the projects kept coming.
Anyway, so we run this project that was C++.
And it was for mobile.
And because of the polycount requirements, we couldn't take the overhead of a framework like Unity.
So it was kind of raw OpenGL engine.
And Rust had just gone 1.0 at this point in time,
and so for fun on nights and weekends,
I sort of ported it to Rust just to get a feel for the language.
And got sold on it, if nothing else, as a replacement for C++
because I had a segfault that I was having a bear of a time tracking down.
And this was like me just one-to-one, as close as I could,
porting the C++ code over.
And this was not modern C++.
This was a guy with a decent understanding of C trying to write C++.
So, you know, not using templates in any meaningful way. This was a guy with a decent understanding of C trying to write C++.
So, you know, not using templates in any meaningful way.
Certainly not using smart pointers the way I should have been.
So it was generally pretty easy to kind of directly port to Rust,
but then I finally figured out where the segfault was coming from because it wouldn't compile.
And so I never shipped the Rust port,
but just the act of porting it
fixed a bug in my code.
So I sold on it for a replacement for C++
immediately there, but
I don't generally write C++
because I'm very bad at it.
So that wasn't terribly interesting.
But I did realize
how amazing its
type system was as part of that project.
And so Rust type system is basically, to a certain extent,
it is what if you took Haskell's type system,
removed higher-kinded types,
but then in exchange gave certain other types of genericism
in type class instances,
and you end up with a Rust type system.
And so that was really cool.
And so then I started wondering, hmm, I wonder if Rust could work as a high-level language.
So Diesel was me trying to answer that question originally.
That's an interesting concept.
Well, first of all, maybe you should write a book, because two ways that you described
Rust to me just now were a bit more tangible.
And we've had shows on Rust.
Steve Klabnick's taught us about Rust.
You did Cats and it never sticks very well with me.
That was the episode that actually originally got me interested in Rust.
Oh, very cool.
So it's the circle of life.
Nothing we love here more than that.
That's awesome.
That's right.
So random aside on Haskell. So RustConf was this weekend. Nothing we love here more than that. That's awesome. That's right.
Random aside on Haskell,
RustConf was this weekend.
I was there giving a talk.
The talk was about Rust's type system.
It was called Type System Tricks for the Real World.
The actual goal of the talk was to explain the concept of monomorphization
in a way that was accessible
to people who were new to both programming and new
to Rust. I'm not going to
recap the whole talk.
I don't know how long editing is, so it may not
even be online when this goes up. If it's up, there'll be a link
in the show notes, but if not
the video of it will be up in a few weeks.
there's this case of
infinitely sized types. That's kind of an error that you really
only run into if you're trying to implement a singly linked list in Rust, where the naive way
you'd write it is you have a struct or an enum of some type where one of the fields is the same type.
And in a language like Haskell, that's sort of implicitly behind a pointer. Everything's
heap allocated, so the size is known and that's fine.
And that's how you solve the problem in Rust as well.
But in Rust, if you kind of naively just write a struct
where one of the fields is the same type as the struct itself,
then that has an infinite size because the size of a struct
is the sum of the size of all of its fields.
And so the example that everybody uses for this is singly linked lists
because that's really the only time you ever practically run into this. But in every example I've ever seen,
you always have it be like, it's a list of bytes specifically, because you don't want to deal with
generics explaining this problem. And I've always kind of hated that because number one, not
everybody knows or cares what a singly linked list is and number two nobody's ever going to be implementing a list of bytes and i had this
revelation i was working on this talk if i make if i make it uh specifically a list of type car
i can call it list string and number one it becomes more tangible to people who don't know
what singly linked lists are and number number two, I get to make fun of
Haskell a lot during this talk.
And there were like
three people in the audience
who did Haskell as their full-time job,
and they thought it was really funny.
That's when you're really talking to a niche.
First, you're at a
RustConf, and at RustConf, you get three
people in an audience that
understand your joke.
Well, I mean, no.
So, funny other story, which I'll tell in just a second.
But no, I mean, I think a lot of people got the joke because, like, just representing string as a singly linked list of characters is kind of a bad idea in general.
And most people seem to just, at the very least, laugh.
But there were the three people, just because I think everybody who does Haskell will agree that that choice has caused
so much pain to the Haskell community
and so they laughed much more because of that
so it wasn't the three people got it
the three people were hit by it basically
my very first conference talk I ever gave
my first slide was a joke that was really only funny
if you knew
Haskell. And it was at a Ruby
conference, and I way overestimated
the number of people
in Ruby who know Haskell.
Jessica Kerr found it very
funny. I know that
because she was the only person in the entire
room who laughed. You should put that
on a t-shirt or something. Jessica Kerr thinks
I'm funny, you know?
She thought that one joke was funny,
if nothing else. This episode is brought to you by TopTow.
TopTow is the best place to work as a freelancer
or hire the top 3% of freelance talent out there for developers, designers, and finance experts.
In this segment, I talk with Josh Chapman, a freelance finance consultant at TopTal, about the work he does and how TopTal helps him legitimize being a freelancer.
Take a listen. Yeah, in my arena within TopTal, I specialize in everything from market research to business
plan creation to pitch decks to financial modeling, valuation.
And then that leads very naturally into fundraising strategy, capital raising strategy, investor
outreach, closing a deal, deal negotiation, how to value the company, how to negotiate
that.
And all those skill sets that I have continued to hone
over on the TopTal side are ones that I actually deploy
every single day in my own company.
Freelancing can sometimes be seen as not legitimate
or subpar work.
Now, I would argue that when you work with a company
like TopTal, they put so much vetting
into not only the companies that you work with,
but also the talent that you work with, but also the talent
that you work with, which I'm on the talent side, that it adds a level of legitimacy that isn't
seen across other platforms. And that for me, as the talent side, is incredibly fruitful and awesome
to be a part of, right? I enjoy the clients. I enjoy the other talent that I get to talk to.
I enjoy the TopTal team. And that creates an overall
positive experience, not only for TopTal, but for me as the talent and for the client as the
company on the other side. And that is really not seen or is the experience across other platforms
in the freelance market. So if you're looking to freelance or you're looking to gain access to a
network of top industry experts in development, design, or finance,
head to toptal.com, that's T-O-P-T-A-L.com, and tell them Adam from The Change Law sent you.
For those wanting a more personal introduction, email me, adam at changelog.com. you had mentioned that diesel was an attempt to uh i'm paraphrasing your words now was basically
present rust as a higher level language or not It was to figure out if it could even
be that. Yes.
And I think
that and I think maybe some other efforts
around web frameworks and such things are starting
to cast Rust more in a general
purpose light. Whereas when
it was first presented to the community
and it's been around for a little while now, but
we've always said this is
a systems language.
Sure.
And I remember you saying on the bike chat at some point
that you think that has perhaps done a disservice to the language.
I was just curious if you could expand on that.
Yeah, so I think the problem with just referring to it
as a systems language is the things that systems languages can do
and the things that Ruby can do.
This isn't like a Venn diagram where, you know, or even a, it's not like two disjoint
sets, right?
So a systems language, being a systems language does not prevent the language from doing anything
that a high-level language does.
It's just that historically languages that were able to do the sort of things that systems
languages are able to do, which is generally just be able to control memory allocation, have been painful to use for higher level tasks.
C++ developers may disagree with me here, but that's fine.
Like Go, for example, was originally presented to the world as a systems programming language.
Yes. And they eventually reframed it as a general purpose programming language
because ultimately, even though technically general purpose
is less general, like if you're going to classify things,
general purpose is less general purpose than systems
because the term general purpose programming,
Ruby's a general purpose programming language.
Nobody is going to claim that you can write an operating system in Ruby.
I mean, you can, it just won't be a very
good one or a very
fast one at least.
That's one of the
reasons I appreciated
how Swift was
presented to the
world because it was
very clearly from the
start it's supposed to
be everything from a
single line script
that you execute just
in time to build an
operating system with
it.
And so it had this
huge ambition,
but they clearly stated it from the very beginning,
at least from the release date.
Whereas with Go and Rust,
they've kind of been trying to not really figure out
what it is, but realize that maybe perhaps
it's been cast in a light that people tend to put it
into a corner and say, oh, you're just for that.
And so they don't think of it as a tool they can grab.
I think that the Rust team
overestimated Rust's appeal to C++ developers
and underestimated its appeal to the broader audience.
It's interesting because you're coming to it
as a Rubyist who had C++, some skills,
but you liked it because of it.
I do not have C++ skills, let's be clear.
Well, I said some skills, because you were building a thing with C++,
so there's some skills right there.
I think that's telling, right?
That the only reason I decided to actually take a look at it
was as a replacement for C++.
When I was aware of it before that,
I never realized what it could do outside of,
oh yeah, you would use it for everything you would use C for.
Right. So you set out to see if it was good at such things as being an ORM,
and so you started building Diesel, and that was a while ago.
Tell us where you got with that, where Diesel's at in its life,
and just open up
that a little bit for us yeah so uh i shipped diesel thanksgiving day 2015 because i was trying
to avoid family um family's listening oh boy i i hope not uh I doubt it.
So, Diesel's coming up on its second birthday.
Right now, I guess where it's at in its lifespan is we are looking to ship 1.0,
and right now our target date is November 23rd,
which will be Diesel's second birthday.
That by no means is going to be Diesel's feature complete,
but it is a commitment to stability of the API. And basically, all of the features that I expect to be able to implement in the near
future, and by the near future, I would call that within two to three years, that I also
expect to require breaking changes are done.
So the features that have been on the 1.0 milestone are not the sort of things that are super pressing
compared to some of the other things,
but are the things
I expect to require breaking
changes.
One of those is
refactoring
most of the stuff that's actually gone now,
but refactoring our error
types to be a slightly better structure.
It's one of those things that's actually
been on the issue tracker since June.
It's a low-priority thing.
It's one of those, yeah, there's a few tweaks
we want to make here, but there's no pressing need for it.
It's not like our current error handling API is super bad,
but it's one of those that is going to be a breaking change,
so we need to do it before 1.0 or commit to not doing it for a while.
Ironically, as I've been going through this milestone,
actually I've been finding more and more things.
I'm like, you know what?
Actually, I think I can do this backwards compatibly.
So I'm going to take this off the milestone.
You mentioned that you started this Thanksgiving 2015.
That's when I released 0.1.
Okay. I started on I released 0.1. Okay.
I started on about six months before that.
And you mentioned that you got into this as a desire to learn more about Rust, right?
So was this your learning thing for Rust, or is that not where you started?
No, no.
Certainly my learning thing for Rust, at least for the very basics of it, was that 3D rendering project.
Certainly I learned a lot more about
rust as part of writing diesel um but i i definitely had a reasonable understanding of
the language when i started the project so were you scratching an itch when you did it or what
was why why is it even existing i mean it was very much so. It exists because there was not a good ORM at the time.
I was originally going to...
My original plan was actually to go full-on web framework,
which, you know, if I had infinite time,
would have happened.
And ORM was the first logical part of that.
But it was very much like I just wanted to explore Rust
as a high-level language.
I didn't know when I started if Diesel was going to ever be
a library that was good enough to ship.
But I wanted to see if Rust could work as a high-level language,
if it was a language I wanted to spend more time with.
And so to me, high-level means web,
and pretty much every web application out there
needs to interact with a database,
and I wanted a library that made it easier
to interact with a database.
So it wasn't so much to scratch and itch, per se, because I didn't have an existing
application in Rust that was missing a good database library.
That said, good open source libraries are not built in a vacuum, so I very quickly realized
I was going to need an application.
So for me, that application was crates.io, which was the,
it's Russ's version of rubychems.org,
which is a surprise,
it does some surprisingly complex interactions with the database.
I would assume that that's relatively straightforward.
You would think,
but they're doing some interesting things
in the database
that a lot of people would have otherwise done
in their native language.
For example, one of the things that Decel supports
as a result of crates.io is the ability
to use arbitrary user-defined SQL functions
in the query builder.
And that was because crates.io has a function
called canon crate name, which is just where they
canonicalize the name
replacing underscores and hyphens
and changing the casing,
and they do that in the database.
That was just one of those, like,
okay, so, you know, we would have just,
you would have made sure that you always called
.downcase
and, you know,.gsub
in your scope
in Rails.
I don't actually know
that there's a huge benefit
to doing it in the database
because you also just have to
remember to call this function.
But it was,
that was how they were doing it.
And so Diesel supports
arbitrary SQL functions.
And that turned out to be,
I mean, number one,
just because it's a good,
like, it's a good thing to support.
But it turned out to be
a really good move
because there's a bunch of functions
that have non-trivial signatures to write when you actually try and figure out the type
signatures of it.
Like just lower.
Actually, I think we have lower in Diesel.
I'm trying to think.
There's a really common one where like depending on the type that you pass.
Coalesce is probably a good one because Coalesce has no meaningful type signature that you could write in most languages.
Certainly, Rust does not have variadic length functions.
So figuring out how to write Coalesce
would have been a pain in the ass.
But because we support arbitrary user-defined SQL functions
and it's really easy to do so,
you just define Coalesce with the actual signature you need
for that one case, where are you going to call it?
And we were able to just punt that issue entirely.
Yeah, it saves a bunch of code.
And yeah, it's certainly, I mean,
and saving code is a benefit, for sure.
It's one of those things, like, if I had a good idea in my head
of how to reasonably write coalesce
in a way that wasn't painful
for people to use, I would do it,
but I don't, so that's okay,
because you can still do coalesce, and it's fine.
Right, it saves you from coming up
with a painful solution, right?
Yeah.
Whereas people can all just do it
the way that they need to.
So I imagine, Adam asked, why does it exist?
And you said it wasn't really to scratch an itch,
but you wanted to try out the novelty of Rust.
You've already tried Rust.
You wanted to provide a thing in the Rust ecosystem
that didn't exist, which is one of the ways
that languages get adopted,
is people have tools that can solve their problems,
especially if you're trying to build web stuff
and there's no web stuff or database stuff available.
It's sometimes a non-starter for folks.
I was wondering if a lot of diesels,
maybe not the genesis of it,
but maybe the design of diesel and the feature set are a product of your experience working with ActiveRecord and the history there and how much of it is a product of Rust's design.
How much of it is because of where you've been with ActiveRecord and so the things that you've seen that you don't want to do, or the good ideas that you do want to do,
and then how much of it falls out from Rust's properties
as a platform.
Yeah, so it's a combination of both.
So I have joked that Diesel is my apology for ActiveRecord,
even though, of course, I'm not responsible
for a lot of ActiveRecord's design.
I do see the effects of its design quite a bit.
So certainly one of the big things I wanted to avoid are things that are common sources of issues on the Rails code base.
And there's two kinds of issues that come in, right?
It's either a person has an actual bug, and there are features in Rails that are just more bug-prone than others,
generally because they're more complex.
Or it's because a lot of people misunderstand a feature.
And so we closed those issues, but I still noticed them getting open.
So I was very much not trying to write active record for Rust.
It was very much I wanted to build an ORM that was true to Rust.
And so I spent a good bit of time thinking about what that meant.
And to me, that means
safety. So Rust is, you know,
touts that it's a memory-safe language,
but it's also a very type-safe language.
So the
defining
vision behind Diesel was
Diesel should disallow
an incorrect SQL query
at compile time.
Now, what I thought that was going to be originally
and what it turned out to be are two completely different things.
I originally envisioned this being like you just had some fragments of SQL as strings,
and we provided an API to kind of stitch those together,
and then we connected to your database at compile time
and asked the database, like, hey, is this a valid query?
What we ended up with does none of that.
The smallest little inkling of that that came through
is that we have an optional feature where you can,
we have a bunch of data structures that we generate
that represent your database schema,
and we can generate that for you at compile time
if you give us a database URL.
That's the only part of
that design that actually survived.
Because
it was bad design
or bad idea
or there was a better way of doing it?
Didn't need it?
Just didn't need it.
Turned out that Rust-type system is flexible enough
I can represent
a SQL query in a way that is very close to one-to-one
with the underlying SQL in pure Rust
and give more meaningful
error messages that way.
The main reason you want
a query builder to begin with is that
SQL strings are not very composable.
And they're hard to reuse.
So you'll notice
that a lot of Diesel's APIs
will look very similar
to Rails.
For example, if you want to pass a select clause,
it's going to be.select.
Instead of passing us some symbols,
basically, if you're just passing
a list of columns, it'll look exactly like
it does in Rails, but there'll be an extra set of
parentheses because you're passing us a tuple.
And then you'll remove
all of the colons, because
each column on your table is an
actual type.
And so you're passing us just those structs.
But then the other big difference, though,
is that then when you want to select
a clause that's more complex than just a list of columns,
you're going
to just do whatever
you want with the query builder, because you were never passing
a list of columns, you were passing us a
arbitrary SQL expression
that is valid for the get
from clause. So if you want to
call lower on one
of those columns you just lower and then that
same column but that's all still Rust code.
We just support a much much
wider range
of what is possible in SQL.
We don't support everything that's possible in SQL.
I don't think we ever will support everything that's possible in SQL,
but I'd like to get to like 95% of at least ANSI.
I definitely don't want Diesel to be chasing every backend specific feature
till the end of time.
That is something that third party libraries can do.
And from the get go,
I've tried to design this in a way
that there is a solid foundation
of APIs for people
to add additional plugins.
Crates.io does full-text
search stuff, so sort of as my canary,
I maintain a full-text search crate
for Diesel that
supports for just the various operators and types
that are required for Postgres full-text search.
So the point being,
I guess the goal was to have it be
as productive as ActiveRecord
is, but catch errors at compile
time, within reason. There are certain things
that we just don't catch. One of the biggest ones being
we don't know what your check constraints are.
So certainly inserts are probably
the least checked part of
diesel and updates, because we don't
know enough about your database schema
to actually fully verify
are all of the invariants represented
in Rust type system and I don't really care
to because that's getting
into the really really
high cost low
gain type of things
it's heavy lifting to get that done
and then the other things are like
we sort of assume that all four given type in SQL,
the equivalent type in Rust,
all values can be mapped between the two,
which is not always true.
The two examples that really come to mind are like,
Postgres, even though it claims it accepts any UTF-8 string,
will not allow strings that contain null characters.
And there actually is a type in Rust called Cstring
that very specifically represents a string
that does not contain null characters,
but we just take a normal Rust string
because it was too painful to disallow that.
And then like Chrono, the commonly used datetime library,
I don't remember what the mapping is,
but basically either Postgres supports dates that are earlier
or Chrono accepts dates that are earlier,
but then whichever one isn't that allows dates that are later.
So there's a subset of dates that are accepted between the two,
but it's something like anywhere from 20,000 BC to 23,000 AD.
So not
something I'm worried about.
But anyway, so we want to prevent
runtime errors within reason.
So then the
other half of your question was, how much of it came
from Rust versus how much of it came from
Rails
or ActiveRecord? And so there's a lot
of things that are kind of lessons
learned from ActiveRecord. Certainly there's no semb lot of things that are kind of lessons learned from ActiveRecord.
Certainly, there's no semblance of dirty
or validations or callbacks.
Things like timestamps are handled purely at the database.
We provide database-level helpers
that you can call on your migrations.
CreatedAt doesn't need anything
other than a default in the database.
And then UpdatedAt is handled by a trigger.
And because I can never remember the syntax
for creating a trigger,
there's a diesel manage updated at function
which takes the name of your table as a string
in your migration to the SQL function
that will create the trigger
and set up the trigger for you.
And so as a result of that,
that means all of our functions around inserts
and updates use the returning keyword
so that way we actually reflect what was stored in the database.
A big thing that was just sort of a learning from,
not just maintaining Rails, but also my time as a consultant,
is that it's unidiomatic in Diesel
to use the same type for reading from the database
as you use for writing.
You guys mentioned you do Elixir and Phoenix,
so this will probably be familiar to you guys.
So we have a separate struct that you'll implement
a trait called change set for,
mostly just by putting derive change set on it,
or derive as change set on it.
And the design of DSLs meant that your structs
which implement queryable are meant to be one-to-one with the queries that you're executing.
And that may or may not be one-to-one with your database tables.
And then your structs which implement insertable or as change set,
and oftentimes you will have one struct which implements both,
those are meant to be one-to-one with your API endpoints or web forms.
Because in my experience, the needs of those two things
tend to diverge over time
and they change for different reasons.
So the design of Diesel,
it's possible to have those two on the same struct,
but it's a little bit painful and it's unidiomatic.
And the design of Diesel is meant to kind of gently nudge you
to separate those early on
to make your life a little easier later on.
That actually sounds pretty nice coming from where I've been. those early on to make your life a little easier later on.
That actually sounds pretty nice coming from where I've been.
Having those
separations, I think, definitely
would grow better with an application
than the way ActiveRecord
does it.
What about the other way? Is it anything from
Diesel, any gleanings
from Diesel that have found their way back
into the Rails code base or perhaps
will be oh yeah i mean all of i mean i i can't actually like there's only a few instances i can
list of like explicit cases you know it's very hard for me to talk about all the places where
just like because i've made diesel i'm better equipped to maintain rails um but there are
definitely a few concrete cases uh i'm eventually looking to get rid of ActiveRecord Relation.
And by get rid of, I don't mean deprecate,
I mean move to a gem.
And continue to maintain that gem,
because I don't want to do Rails 2.3.3.0 all over again.
But move it to a gem and start to explore alternate query builders,
because ultimately Relation uses a full SQL query
as its unit of composition.
And I think that is the wrong level of abstraction for a query builder to operate at.
And I think that's most apparent
in how long relation or took us to add
and why it was so tricky,
which there's a conference talk
where I went into the details of it.
We don't need to recount all of that here.
Anyway, so diesel is very much
kind of what I'm imagining
an eventual
future Rails query builder might look
like. So there's that.
A slightly more concrete
example is
I'm working on a new
Postgres driver for
Diesel right now. The current Postgres driver is built on
libpq, which is the
C library
for interacting with the Postgres wire protocol
that is shipped with Postgres itself.
It is also what the pgRubyGem is built on top of.
And very specifically,
so Postgres allows you to transmit values
as either binary representation or text representation.
And we use text representation always
because the binary representation is generally undocumented
and not considered to be stable,
even though it is effectively stable,
at least for the most common types.
The only thing that is actually documented
is that the binary representation of numeric types
is network-endian and everything else.
You go look at the C source code.
Diesel right now always uses the binary representation
and there are certain data types
that I've just
not added support
for yet because I don't want to figure out
their binary representation.
On the flip side of that
the binary representation of the
timestamp type in Postgres is a
signed 64-bit integer
representing the number of microseconds
since January 1st, 2000,
and an unsigned 32-bit integer representing the,
or I'm sorry, the number of seconds,
and then an unsigned 32-bit integer
representing the sub-second portion of that,
which is, I believe, microseconds,
but it's actually dependent on a compiler flag.
But basically you can assume it's microseconds, but it's actually dependent on a compiler flag. But basically you can assume it's microseconds.
So time or date time is the most expensive kind of core type
supported by Rails in terms of typecasting.
Because the difference and the difference in performance
between turning those two numbers into a Ruby time object
versus doing arbitrary string parsing is enormous.
So the actual main driver for me doing this for Diesel
is adding async IO,
which Rails would not be able to take advantage of.
But then one of the other drivers is
libpq allows you at the per query level
to say I would like all of the results back
as binary versus text.
But the wire protocol itself actually allows you to specify per column
whether you would like the results back as binary or text.
So when I finish this driver for diesel,
I'm then going to pull that out and write a Ruby wrapper for that
and ship that as an optional new Postgres adapter for Rails,
which may or may not become the default,
depending on how well shipping
a Rust dependency of Rails ends up going.
And then the final, most concrete way
that diesels influence Rails,
about three weeks ago,
I made a change to Rails
where I changed how we handled bind parameters
and fixed a bazillion longstanding issues
that were mostly gone in 5.0,
but kind of still hung around
and now are completely impossible.
And it was a change to A-REL
and basically just how we manage
bind parameters with our AST.
Since we're running short on time,
I won't get into the technical details,
but the way I implemented it
was I took some code from Diesel
and I pasted that into A-REL
and I converted that to Ruby syntax and then just followed the
test failures until until everything was green the old copy paste and now i've realized i can do that
and it works legit strategy that's that's just my strategy now that's your first step on all bugs
just copy some stuff out of diesel paste it in yeah first implement it in
diesel and then copy paste it into rails that's hilarious all right so that's probably a little
more wordy than you were looking for no that it those are great concrete examples and i just love
seeing you know basically the fruits of labor in one place get applied across especially in a
project like rails which so many people benefit from your work there so
that is good stuff. We're running real short
here. Two really quick questions
and we'll call it a day. The first one
is DieselHits1.0
are you ever planning on
picking up the full web framework
in Rust? No. I had a
baby. I can't do that. I don't have time
for that anymore. You have a baby.
Ruby takes up all my time now. My baby's named Ruby by the way. Your baby't do that. I don't have time for that anymore. You have a baby. Ruby takes up all my time now. My baby's
named Ruby, by the way.
Your baby is named Ruby. Yes.
That's a great name.
I'm not sure if this is going to be a thing or not, but
if I can just plug this and then maybe we'll cut it
if it doesn't end up happening. Also, if you're interested,
I have these really cute baby Ruby
stickers, which are for sale on DevSwag
and there's a link in the show notes if you want to support
my development of Rails and Diesel.
There you go. Get out there and support
Sean. Buy a baby Ruby sticker
or 10, if
it's a thing.
Last question.
Hypothetically, you're stranded on a desert
island. You only get one programming language
that begins with RU.
Which programming language
do you pick and why?
I'm trying to think of a third programming language that begins with R-U to give a facetious answer.
Yeah, if I would have said R, you could have just said R.
Yeah.
Probably Rust because Rust can do everything Ruby can,
but the inverse is not true.
Smackdown.
There you have it.
And I also, like, I love Ruby,
but I do often find myself working through problems in Ruby
by prototyping it in Rust.
Good answer.
Sean, thanks so much for joining us.
It's a lot of fun.
Yeah, thanks for having me.
All right.
Thank you for tuning into The Change Log this week.
If you enjoyed this show, share it with a friend.
Rate us on Apple Podcasts.
And thanks to our sponsors, Linode, GoCD, and TopTile.
Also, thanks to Fastly, our bandwidth partner.
Head to Fastly.com to learn more.
We host everything we do on Linode cloud servers.
Head to Linode.com slash ChangeLog.
Check them out.
Support the show.
The Change Log is hosted by myself, Adam Stachowiak, and Jared Santo. Editing iselog. Check them out. Support the show. The changelog is hosted by myself,
Adam Stachowiak, and Jared Santo.
Editing is done by Jonathan Youngblood.
And the awesome music you've been hearing
is produced by Breakmaster Cylinder.
You can find more episodes just like this
at changelog.com
or by subscribing wherever you subscribe to podcasts.
Thanks for listening. Thank you.