Software at Scale - Software at Scale 38 - Hasura with Tanmai Gopal

Episode Date: December 2, 2021

Tanmai Gopal is the founder of Hasura, an API as a service platform. Hasura lets you skip writing API layers and exposes automatic GraphQL APIs that talk to your database, trigger external actions, an...d much more.We talk about the implementation of a “compiler as a service”, the implications of primarily having a Haskell production codebase, their experience with GraphQL, hiring product managers technical enough to build useful features, some new and upcoming Hasura features, and riff about the current state of front-end development and `npm` hell.Highlights00:20 - What does the name Hasura mean?02:00 - What does Hasura do?04:00 - Why build this layer of the stack? 08:00 - How to deal with authentication if APIs are exposed directly via the database.26:00 - Does Hasura make production applications faster?33:00 - JSON Aggregation in modern databases38:00 - Why Haskell?44:00 - How do you write quality Haskell? How does hiring for Haskell positions work out in practice?55:00 - Application servers do much more than just talk to databases. How does Hasura provide escape hatches so that non-database interactions (for eg: talking to Stripe) work out? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.softwareatscale.dev

Transcript
Discussion (0)
Starting point is 00:00:00 Welcome to Software at Scale, a podcast where we discuss the technical stories behind large software applications. I'm your host, Utsav Shah, and thank you for listening. Hey, welcome to another episode of the Software at Scale podcast. Joining me today is Tanmay Gopal, the co-founder of Hasura. I would say a developer productivity platform using GraphQL is the best way I can put it down. Thank you for joining me. Thanks for having me. It's great to meet everybody. Yeah. Can you start with the name Hasura? I think it's an interesting name. Yeah. Yeah. So there's an internal origin story and then there's
Starting point is 00:00:47 the external story right so let me start with the external story first um asura is a common um it's a common kind of south asian word that you'd see for demon which is not actually evil it's just a class of divine being that uh that you'll see if you travel in southeast asia and india you'll see like these demontype faces outside doors and on vehicles and stuff, and this kind of warding off evil. And the reason I chose that name was because demons are also background processes and servers.
Starting point is 00:01:16 And we use a lot of Haskell, and so the H came from there. And so that's kind of how Haskell, we decided that the name Haskell is a decent name for something that's kind of automating like we decided that the name was a decent name for something that's kind of automating away these demons that are running in the background um and and the internal story is that we were uh we were in a car and one of our one of our founding engineers was like we should have asura in the name because demons are cool uh and then the one of the other
Starting point is 00:01:43 engineers was like hey but we should have h in it because we use Haskell. And that's why the logo also, if you notice, is kind of like, almost like it has these tiny little horns, right? And there's a lambda in the middle, which I think is scaled well as a logo, especially with the rise of serverless and stuff that's happened. So that's how the name came to be and the name stuck. Yeah, it's a pretty unique thing. and i can't help but notice the tiny like demon smiling at me whenever i look at the product yeah so what exactly does hasura do if you can just explain for listeners yeah absolutely absolutely so um normally when you kind of think about um you know wanting a data api right like you let's say you have one or more databases,
Starting point is 00:02:26 and you have these sources of data or systems of record, and you have developers who are building services, maybe they're inside the same organization, outside the organization, maybe they're developers building front-end applications. These developers will need access to that data, and you typically build a data API in the middle for that access to happen. So maybe you would write something with Node.js or PHP or Python or Java or whatever. And you kind of spin up an API server, you build an API server, you deploy it, etc. And you get an API. What Hasura does is it automates that piece.
Starting point is 00:03:00 So instead of you writing code, what you can do is work at a higher level of abstraction. You work with this mental model of saying, here are my data models or logical models that map to the underlying physical models. And here are the connections between these various models. Here are model level rules for securing them or authorizing certain operations on them. And then as soon as you specify these higher level constructs, Hustler gives you the API. And this API now gets generated by Hustler, it gets maintained by Hustler, Hustler handles scaling it, guarantee performance SLAs around it, caching it, all of those other kinds of things that you would have to do by hand. And so the folks who kind of want to consume this data or kind of use that API, they basically get that ability to work with it
Starting point is 00:03:46 instantly. So for our larger customers or users, what happens is that instead of spending three quarters of a year standing up an API service and maintaining it and whatnot, that time period gets reduced to just a few weeks from the thought So from the thought of like wanting an API to going into production. So that piece in the middle, that data API in the middle, that automation is kind of what Hasura does. So maybe one way to think about it is
Starting point is 00:04:15 you can specify just like a table schema in your database and you click a few buttons and then you have an API server exposing data. Exactly, exactly, exactly exactly and that works also for existing databases right so even if you don't you don't have to create the table you can take Hustler and point that to a system that has thousands of tables and billions of roles and that will also just work instantly so why did you decide to build this layer right like if you go and hack a News and you see people,
Starting point is 00:04:45 people complain a lot about, oh, I hate MongoDB. They'll complain about the database layer. People have choices on, you know, oh, I don't like Django, I like Express or whatever, right? So why build this layer, I guess? Yeah, that's a really good question. So when we started, like how Hasura kind of came to be, to be was when a bunch of us were, we had this crazy idea that the way that we're doing application development right now, it's at a lower level of abstraction than it ought to be.
Starting point is 00:05:19 The building blocks are there. It just ought to be faster. There's a lot of graft in the middle that we have to go through to make this whole piece work. And as we were diving into pieces of where is that extra piece coming in? Our key hypothesis was that
Starting point is 00:05:39 this data access piece is this layer where we're doing a lot of repetitive work. It's like so much of our applications are essentially CRUD. If you think about your application as kind of data models and algorithms, so much of it is just powered by these data models. And there's so much repetitive work that needs to happen here. But it's not just repetitive. It's also domain-specific. It's not cookie-cutter. It's not the same repetitive, right? It's also domain-specific. It's not cookie-cutter.
Starting point is 00:06:05 It's not the same thing for everybody. It's unique to everybody's domain, but the work is somehow still kind of very repetitive, right? And so we came up with this slightly crazy idea of saying that what if we automated this piece in the middle? What impact would that have on application development, right? So the very first version of Hasura that I built was actually
Starting point is 00:06:30 on top of Mongo and it was kind of structured as this thing where you would specify this schema in a DSL and you'd kind of run Hasura, it was not called Hasura then, but you would run it as a, you'd run this kind of compilation script. It would take the schema and then it would compile out an artifact, a binary, which would be the API server. So you wouldn't actually have to build the API server to become the API server. And then as we were exploring the database space more, we were like, I think it makes a lot more sense to start off with Postgres. And so we then chucked away that whole code and decided to support Postgres. And then when we kind of built that piece out where Astra essentially became this configure and get an API server instead of build an API server by hand, we started using it for building all kinds of applications. So we started working with some of the largest companies in the world, tiny startups, large MNCs that were building their first digital assets, their
Starting point is 00:07:27 first digital applications. And as we worked with Hasura to build those applications for our customers at the time, this is way before Hasura existed, we realized that, hey, this piece actually works. Instead of spending so many months doing this kind of grunt work right you're now spending time doing the interesting stuff around the data modeling and the application logic right um and this was the time when containers are starting to get popular and whatnot um and so so that's kind of when that code technology came to be and we kind of validated the fact this idea of saying that we can we can automate this layer in the middle by doing some interesting things that perhaps we'll get into in a bit, we could save this massive amount of time
Starting point is 00:08:13 in building and iterating on this API piece to build applications. And so as that happened, this was before GraphQL as well, we had our own version of GraphQL. And this became successful, we were like, hey, you know, this tech I think actually works. So a few of us would kind of build that we shut down kind of a consulting arm. And a few of us came out and we started Hasura, we raised VC and we started Hasura. And then we kind of went through the motions and you know, launched the open source project in a few months after that. Yeah, I think that's an interesting story. You had so much interest even before you had launched anything.
Starting point is 00:08:55 Clearly, there was something there that people liked, right? So one of the key pieces of why generally people have that middle layer is this notion of authorization and authentication right how do you deal with that in hasura yeah yeah that's a very good question um so um the the lens kind of here that we took was saying that you know if if the end api consumer uh got access to all of the data right that they could get access to. What are the caveats there? What are the primary caveats there? What are the biggest problems? Why is that something that we don't do?
Starting point is 00:09:33 Just from first principles. Forget everything that's happened in the industry for the last 20, 30 years. But just from first principles, why is that a thing that we don't do? And there are two kind of key things that emerge. The first, like you rightly said, is authorization. The second is scale and concurrency, which is that applications like speaking HTTP and databases don't speak that. But this first layer, this authorization piece is absolutely key because the problem is that as an API consumer, if I'm interacting with a
Starting point is 00:10:03 certain piece of data, I need to make sure that I'm interacting only with a piece of data that I own, right? And I can't access something else, right? Whether it's read or write or whatever. And when you're trying to do that operation, this logic of deciding that I can read this document because the document is public or the document collaborators contains me or the document is in a folder and the folder belongs to me or the document is an organization and i'm a member of this organization right there's so many kind of these conditions um that decide whether you get access to a piece of data or to certain items of data or not right um and so what we did was we we took that model, that style of thinking and it's similar
Starting point is 00:10:48 to what we've seen in the database world is something called a concept called row-level security, where the idea is that you have a row or a particular entity and getting access to that piece of data,
Starting point is 00:11:04 that particular row or document or whatever you want to call it, that particular entity is a condition or is a function of certain properties of the data and certain properties of the session. So it's like F of session comma data, that gives me access to this data or not. That's the logic. At the core of it, that's what this logic is. It's some properties of the session mixed with some properties of the data,
Starting point is 00:11:34 and we operate on that together, and that gives us access to this data or not. And that's the core idea. So what we decided was we'll take this core idea and we make it declarative. So as long as you can specify a declarative condition that takes some property of the data
Starting point is 00:11:51 and some property of the session and there's a Boolean operation that decides whether you get access to this or not, that's what our authorization model would do. So that's how we decided to build it. And this is kind of similar to the low level security model that you'd see inside older databases like Oracle and you'll also see in Postgres. The newer databases don't have this and they don't do it at the application layer and stuff like that. They do it kind of in the DBA layer but we decided to do that at the application layer. And that piece just worked. It sounds restricting at, but it turns out that almost any kind of access rule that you can imagine is actually just a property of just that. It's very rare that it will be a property that is not of the session comma data. It is almost always just a property of that.
Starting point is 00:12:38 And more often than not, it can be something that's declarative. It might not be a property of that data, but something that's related to that data, right? So just by looking at the document, I can't decide whether I get access to it or not. But if I go to document.organizations.users, right? Then I'll be able to find a condition that matches this, right? So there's relationships from that piece of data.
Starting point is 00:12:59 But as soon as you kind of start putting together that model of saying there's models and models have relationships and from a particular entity, if I can traverse my relationships and I can compare those properties to the property of my session or to a property of the data within itself, I should be able to determine access or not. And that resulted in almost handling every single use case that you could imagine. So that became the authorization.
Starting point is 00:13:27 So you provide some kind of web interface or some kind of interface for people to specify these rules. Exactly, exactly, right. So this rule basically becomes like at a model level, you go to a particular model. And when I say model, the model could be a table. The model could even be a logical model right so maybe underneath it it's actually five or six tables but you have a view right you have like a view of those underlying five or six tables that's one logical model right so you're saying that this is a user and the user has like a hundred attributes but but all of those hundred attributes might be coming in from five
Starting point is 00:14:01 different tables right doesn't matter this logical, you can attach rules to this logical model, right? So you can create roles and you can say I have an editor and editor has access, you know, a user or public or private or whatever. You can create these different scopes. And for each scope, you can attach a rule. And the rule is a declarative condition
Starting point is 00:14:18 that kind of maps certain properties of the data to properties of session, right? So a trivial rule would be for a profile model, I would say if profile.id equal to properties of session. So a trivial rule would be for a profile model, I would say if profile.id equal to sessionUser.id, then I get access to this profile. So that kind of gives me the ability
Starting point is 00:14:34 to read my own profile. And these rules, like you said, when you use the Hustler kind of UI, you can configure it via web UI, so you have a little permissions builder that helps you do it. But it's fundamentally just a JSON DSL. So you can skip the UI and then you can just write out a YAML file called profiles.yaml.
Starting point is 00:14:52 And you can say permissions user and then write out the user role, right? And it's just declarative in JSON or YAML or whatever declarative. You can even write it in code as long as it as long as it generates that JSON syntax. And arguably, it's even more secure than everyone building out custom authentication. Exactly. Yeah, exactly. Exactly. Right. Because it kind of solves two very risky, two important security problems, right? The first is this authentication problem, which is that building good OAuth2 or JWT authentication is hard. It's not hard because it's technically hard. It's tedious. There's lots of different edge cases and you can have token rotation and there's expiry and there's certain
Starting point is 00:15:39 ways that claims interact or the way that subs interact. And there's a set of things there that you want to do per spec. And that's a place where you can have trivial mistakes that can have bad ramifications. So standardizing that piece is important, which is one part of this authentication challenge. The other part is the authorization challenge, which is that if you look at any particular API, and let's say you're the product owner, you're the product manager inside this, the team that's building out this API, and you go to your team and you ask them a question,
Starting point is 00:16:12 hey, what are all the rules that give me access to a particular profile object, right? Nobody can answer that question, right? A developer will have to go inside the code, they'll have to look at the controller that might be touching the profile model, and you'll have to see what the access rules are. But you might not just it might not be one controller that accesses profile, right? You might be accessing profile through some other API endpoint, like maybe you're accessing your last 10 orders in an ecommerce
Starting point is 00:16:37 situation, and order has an order.profile, right? So that's also accessing the profile entity. And now you kind of have this thing of, am I using the same authorization rules in both places? Have we written the same code? Is the code written properly? Can I prove that it's written correctly? You run into all of these issues to verify what those actual rules for every model are. The problem doesn't stop there. It becomes worse as soon as you start thinking about scaling, right? Because as soon as you start thinking about scaling and caching, you open up another potential can of worms, right? Because once you start caching these elements, right? What if your cache rules bypass those authorization rules, right? And you're like, yes, let's aggressively cache all this profile information.
Starting point is 00:17:26 But, you know, somebody was caching it somewhere and somebody was doing a lookup somewhere and the person doing the lookup forgot to use the same authorization rules, right? Or the authorization rules evolved somewhere else, but you forgot to make sure that they get updated in the cache part of the thing, right? And all of these problems are solvable, right? It's just tedious, right? It can get really messy. But now with the Hustle approach, you can literally have a UI and some of our large
Starting point is 00:17:51 enterprise customers have done amazing work in building out a lot of tooling for their own teams on top of our declarative kind of metadata. Because all of those rules are essentially declarative JSON metadata. So you can take that metadata in, and you can build your own tool that gives you a really nice UI that shows you all of the different models. And for each model, it tells you what the permission rules are. And if that's changed, it can tell you what's changed. And there's a comment in English that tells you why it's changed.
Starting point is 00:18:22 And you can observe that, which is kind of a superpower that you've not had before. And it's tedious to build that in if you don you why it's changed, right? Or, and you can you can observe that, right, which is kind of a superpower that you've not had before, right? And it's tedious to build that in if you don't have it. It's kind of like programming without a compiler, like you can do it. It's extremely tedious. It's going to be problematic across platforms. Exactly, exactly. So all of this just makes it sound like everyone should be using this tool for the next project or tools like this, right? They don't have to think that much. Another key decision, you mentioned that initially you were not exposing GraphQL, you had your
Starting point is 00:18:55 own version and then you're clearly exposing GraphQL today, mostly. So we've recently added support for REST point. It's kind of like GraphQL first. So I'd love to take get your take on why GraphQL. Yeah, so I so the API that we had, because it was a data API, right, and it would do you could kind of traverse relationships, right. So you could say one profile and profile of recent addresses and addressed of city or whatever, right, like you can traverse these relationships and stuff like that. Our API was actually a JSON API, not a REST API, but a JSONish API.
Starting point is 00:19:33 So it was one endpoint. It was like a slash API or the slash query endpoint. And it would take this JSON and the JSON would literally have something like type query fields and the fields would be an array of fields that you wanted. So the fields would be an array of like profile and profile dot something or whatever. And it kind of looked like a JSON version of GraphQL. And so when GraphQL came out, I remember in the early days that the kind of specification was becoming popular.
Starting point is 00:20:04 And we looked at it and we were like, man, this is such an amazing specification. This fits in so well with the kind of stuff that we're trying to do. But it's really hard. It's going to be hard for this to take off. And so we decided not to do GraphQL. And we kept our Json API for a bit. And then in 2018, just before we were deciding to launch
Starting point is 00:20:26 Hasura to the world, GraphQL was just starting its upward trajectory. We were like, hey, people are using GraphQL. So they've crossed the hump on there's enough community tooling to help them use GraphQL now. So using GraphQL as an API because it's just a better way to do exactly what we're doing. It's JSON native, it's neater syntax to specify like you want to query something or mutate something and traverse like joins and have arguments
Starting point is 00:20:56 in different levels. And then we decided to add GraphQL support for it. So internally, we still have our API which is called RQL, but it's undocumented. It's only used internally. The Hasura UI in portion still uses the RQL API, but everything else is GraphQL.
Starting point is 00:21:12 Okay. And what has your experience been, both in terms of developing support for GraphQL and what is the developer experience? What have your customers thought about it? Yeah. I think it's been actually very fascinating to see, right? I think, and this is kind of my take from a few years ago,
Starting point is 00:21:35 and it's probably still my take to a degree, which is that I think GraphQL is just an amazing API to use. If you're an API consumer, then GraphQL is the most sensible API format that I have today for a variety of reasons. It's JSON native. There is so much tooling because there's a GraphQL schema.
Starting point is 00:21:57 There's so much nice tooling around being able to use a GraphQL API, exploring the API, incrementally evolving the API. I don't know if you've seen the Graphical Explorer, the little sidebar that tells you how you can expand and use what fields are available, what arguments are available. The documentation, which is not necessarily the semantic usage documentation, but the code, the reference documentation for an API
Starting point is 00:22:23 is just auto-generated. And that means that the impact that you have on code gen, on writing SDKs, or making sure that your API integration is type-safe and works well on the client, whether it's a mobile app or a front-end app or even a service, is just very neatly solved. It's very nice. It's all been solved before. It's not been solved before. It's not like fascinatingly new. It's not like with GraphQL. It's like we've not seen any of these technologies before. We've seen all of it before.
Starting point is 00:22:52 It's just a really nice packaging of all of these different ideas brought into one place, fundamentally JSON native and type, that makes it just absolute joy to use. Now, the problem with GraphQL, though, is that it's a new kind of thing to build on the server side. And the way that you think about GraphQL,
Starting point is 00:23:13 which is giving a little more power to the customers from what you were doing before, introduces some challenges. It's a slightly different way of thinking. And this is okay for people who are just starting out with GraphQL, but even if you think about
Starting point is 00:23:25 larger folks adopting GraphQL, like large companies adopting GraphQL, it's a big bet. There's a lot of things that change for them. It's like the way that they think about performance or security or observability or the way they think about DDoS and DOS, like the way they think about their developer tooling of how they go from staging into production. A lot of that changes. And in a large organization, that change is not trivial. It's a hard change to make and to drive through the organization.
Starting point is 00:23:56 So implementing GraphQL is tedious. And I still think it's still tedious today. There's a lot of tooling today, but it's still more tedious. There's a lot of tooling today, but it's still more tedious than it ought to be. The speed with which you can deploy a REST endpoint and have it work for millions of people is still 10x more than what it would be for GraphQL. In a production quality way, in a way that does something complicated in a way that is just works with everything else, it's still so much better than what you'd be able to do with GraphQL because GraphQL comes at the cost of being a little more complicated.
Starting point is 00:24:35 So that's kind of been my take on GraphQL in general. But that said, the API, the GraphQL API is still, I think, today probably the most convenient API to consume. Now, for Hasura users, when we've seen them use GraphQL, a lot of people come to Hasura for GraphQL because they hear about GraphQL, they're like, how do I try this out? How do I use GraphQL? Hasura has this thing that people are talking about, let me use it. They point it to their data systems, especially if it's a large database or multiple databases with lots of data. They point them to it and it just works. It's 5x faster than any API that they have today. If it's a large API, it's 50x faster. It's ridiculous. I say this as an engineer with a
Starting point is 00:25:22 straight face. It's so much faster performance-wise and the ability to handle concurrency that you're like, okay, this is amazing. I love this API. It's really nice. It gets me super productive. And then as people start using Hustler, what we hear from our users is that they kind of came for GraphQL and they stayed for the fact that it's just a different way of thinking. It's just a faster way of thinking. It's a model kind of approach to thinking. You're thinking about models, you're thinking about authorization rules, you're thinking about higher level domain level concepts.
Starting point is 00:25:55 You're thinking about your domain and you're just getting the APIs and artifacts. So at that point, GraphQL just becomes an implementation detail to you. You're like, yeah, I do things and I get a nice API. And GraphQL is a nice API, so I get a nice API at the end of it. So for users, using HustleRite evolves from GraphQL into just being an API that is flexible, that just works really well from a model level way of thinking. And so the whole story just comes together really well, right? People really, everybody who's using kind of that API,
Starting point is 00:26:28 get the GraphQL API, they really like it. People who are maintaining Hustler, they really like it because they get to work at a high level of abstraction and kind of comes together really well as the product is built out and as the product is evolving. And I think it just worked out really well as like a go-to market strategy in a sense that you have GraphQL that people are familiar with, but it's hard to deploy that on servers.
Starting point is 00:26:50 Exactly. So you make that really easy for them. Exactly. And you provide the superior product. Okay, that's great. And you mentioned a couple of times that Hasura is just simply 5x faster or like even 10, 15, 50x faster than deploying your own API. I have maybe a very rough intuition of why that might be, but maybe can you go into detail
Starting point is 00:27:10 on why that is? For sure. For sure. So like a few of those WACO ideas that we had, we built us together, came together early when we were building this up. So the approach that we decided to take was one, when you're kind of traversing related objects, right?
Starting point is 00:27:32 In a data system. If the data system is already denormalized, that's fine. But in most cases, it's probably not denormalized, right? It's normalized to some degree. It's somewhere in the middle. It's not fully normalized,
Starting point is 00:27:43 but it's also probably not fully normalized. And in those situations, what happens is that when you're thinking about traversing, like you're thinking about taking a piece of data and fetching something that's related to it and fetching something else that's related to it, the fastest, the theoretical fastest way of getting this data would be to do a join. But this doesn't work in practice. The join doesn't work in practice, right? The join would be the fastest way of doing it. Faster than your application server trying to make multiple requests to the database
Starting point is 00:28:13 because you're making a request to the database, network transfer, making a request to the database, network transfer, right? It's much faster to kind of send that to the database and tell the database, can you please just put this piece of data together for me and send that back to me, right? That would be faster, but there are a few impediments
Starting point is 00:28:30 to actually doing this in practice. The first impediment is that we talked about authorization rules, right? You take a, you can't simply just take a document and say document.organizations.users and join the whole thing together. Because what document you have access to and document.organization and what fields of the organization you have access to and organization.users and what users of the organization you have
Starting point is 00:28:57 access to, each of them have separate rules. So constructing this join query that would take into account all of those authorization rules that are a part of your application logic, like your API server logic, makes it really hard. In practice, this becomes really tedious to create an efficient query from these model level rules that you have that are spread over different parts of your application logic. So constructing that becomes, again, it's possible, but just the speed and the way that we develop these pieces of code, it makes it hard to put that together. So that becomes a real challenge in making sure that you're making one query that is as efficient as it can be. The second real-world challenge is that when you're fetching a list of, not just a point
Starting point is 00:29:47 object, right, but you're fetching, say, a list of items, like you're fetching a list of orders or something, right, this problem becomes a little bit worse, right? So let's say we're fetching articles and their authors, right? So you're trying to fetch, like, I want the top five authors, and for each of these five authors, i want their top three articles and their titles to show up right you're showing kind of this author page right and you want to show author cards and for each of the author cards you want to show like the top two articles right that's what you want to show now if you made a join query assuming that you've got all the authorization pieces figured out right you did a naive join query with a relational system. So you'd do something like select article, authors from article,
Starting point is 00:30:30 like select star from article from authors, where author.id equal to article.authorId. So you'd make this join query, assuming authorization is figured out. It's nice, you made a single query, you've got this result. The problem is that this result that you're getting is what is called a Cartesian product. So you're getting an m cross m result. What I mean by that is when you make that select query, right? Imagine the data that
Starting point is 00:30:56 you're getting. You're getting author1.name, article1.title. You're also getting, for the second article, author1.name, article2. Author1.name, article3. So you're getting repeated information. You're getting repeated information for that author because you wanted each discrete article. So you're getting a lot of repeated information for your top-level, your left join, your top level node.
Starting point is 00:31:25 You're getting way too much information than is necessary. So now if you run a large query, right, where you're trying to fetch like 100 articles per author, and you're fetching just two authors, but 100 articles for each of those authors, the amount of data that the database has to process and send over the network to you becomes large just because it's repeated, useless information, because it's returned as a Cartesian product, right? Whereas actually the JSON result that you'll send back ends up having author name only once because it's a nested JSON structure, right? It has author name just once
Starting point is 00:31:57 and then an array of articles, right? So Hasura does this thing, we call it JSON aggregation. Hasura does not make the database return a Cartesian product. Hasura compiles it, we call it JSON aggregation. Hasura does not make the database return a Cartesian product. Hasura compiles it into a query, right? When it's talking to SQL systems, it compiles it into a query that makes the database create the JSON.
Starting point is 00:32:15 And so the shape of the result that you want at the end is actually assembled at the database itself. And then we just take that JSON response and stream it back to the client. So that means that that has several advantages, right? The first, we're not processing as much data on the database. We're returning lesser data from the database itself. That's a massive speed up between the database and the API server. The second is that the API server doesn't have to do this work of taking that result from the database, reading n bytes, converting that, right? Imagine that you're reading it in Python.
Starting point is 00:32:47 So you talk to the database, you get the byte stream, you load it up as a Python object, right? You get a Python tabular structure. Then in Python, you convert that to JSON, like a JavaScript object structure, a dictionary object structure. Then you serialize that into the byte stream, right? And then you send that byte stream. So that's order 2n of serializing, deserializing that you have to do.
Starting point is 00:33:12 Now, again, imagine for larger responses, like larger pieces of data, like I fetch one MB, it's going to be that much slower. I fetch 100 MB, it's going to be like 100 times slower, right? Just straight up 100 times slower. I mean, I would say 100 times slower, a constant factor of 100 slower, right? Just straight up 100 times slower. I mean, I shouldn't say 100 times slower, a constant factor of 100 slower, right? Because you're serializing those many more bytes and deserializing those many more bytes. Hasura doesn't have to do that because it's just constructed the right shape and it streams that back.
Starting point is 00:33:36 I mean, as far as possible, if you're joining across multiple sources, it has to unpack them in parts, right? So that's a massive amount of time save as well. So these are kind of few fundamental things that result in a massive speed up when compared to kind of building an API and forget GraphQL. GraphQL, of course, because you're doing arbitrary joins and stuff like that, it's even better. But even compared to like a vanilla REST system, it ends up working really, really well. Does that make sense? No, absolutely.
Starting point is 00:34:06 So basically, definitely the serialization and deserialization completely gets skipped. But yeah, this whole overhead, especially if you're working on one of these faster languages like Ruby, Python, or TypeScript, faster in terms of development time, loading all of these things up in memory and all that is also extremely slow and you get to skip all of that but what is intriguing to me is this idea of json aggregation directly on on the database itself so are you actually making the data fit the json shape in a sense at the database layer how exactly is that just like string concatenation or like like other like the functions? What do you
Starting point is 00:34:46 know, the databases, most most modern databases have these inbuilt functions, which, like, you know, like developers, API developers have not heard of because they're like, we are database function, who cares? But the database is actually really good at this. So the database does the JSON conversion for you. So what you can tell the database is that instead of returning kind of that set result to me, right, that has like, ID, comma, name, comma, age, or whatever, instead of returning that as a set to me, return that as a JSON object.
Starting point is 00:35:14 So that the database will return that as a JSON object. And this process is called JSON aggregation. Okay, so there's there's two things that come to mind because of that. So the database is clearly doing more work in some sense, because it's doing this JSON aggregation. But you are sending it a hyper-optimized query, which makes sure it doesn't do this Cartesian product in a sense. So does the amount of work that a database generally does, does it change over time? Or is it roughly the same as what people have before?
Starting point is 00:35:45 Yeah, that's a very interesting question. And I think it depends on the kinds of workloads that you have. For some kinds of workloads, you might notice a little bit of an increase in the amount of work that your database is doing. But NetNet, it ends up being very similar. In practice, it ends up being a lot lower than your own API server. And that's because Hasura is just not doing the basic stuff that you ought to have done anyway. Like, for example, just compiling that into a single query instead of hitting the database multiple times.
Starting point is 00:36:17 That means that when you look at Hasura processing the same query, and you look at the database load, and you look at your own API server hitting the database with their own load, the first thing that our users tell us when they've moved away from systems like Rails to Hustler is that, my database load just went down. It used to be 60%, 70%. It's now like 5%. What's happening? And the reason is that Hustler is using lesser connections with the database. It's like asking the database to process it in one shot, in one query, instead of hitting with five, six queries that are concurrently running. You can imagine that as concurrent load increases, the number of hits that you're making to the database also increases. So what will happen is if you've not done prepared statements and you've not tried to compile queries and you've used point queries,
Starting point is 00:37:02 you'll notice that as concurrency increases, the work that the database is doing actually increases for your own API server, right? So in practice, it ends up being that the database footprint is much lower. Theoretically, though, your question, from a theoretical standpoint, I would say that in some situations, it's equivalent.
Starting point is 00:37:20 In some situations, the database does do a little bit more work. But again, with modern databases, that's not too much of an issue, right? You vertically scale it out, you scale it out horizontally, right? Or you add read replicas, it's fine. And generally in API servers, you also have to try to optimize for readability, right? So even if you have a super complicated SQL statement, like should I, will this really pass code review? Should I just simplify it and break it down into three steps?
Starting point is 00:37:45 Cause like developer experience and readability is more important. Exactly. Exactly. And that's the piece that we can kind of get the skip with Hasura, right? Because it's like Hasura is structured as a compiler, right? It's not, it's not, it's not handwritten kind of translation, right? And because it's structured as a compiler, that means that there is kind of almost a provable transform, right?
Starting point is 00:38:10 From this kind of GraphQL-ish syntax to SQL-ish syntax, right? So that means that there is a lot more kind of predictability and observability into how that is happening. And then Hasura, when it's generating those queries, does put in a little bit of effort to make sure that those queries are readable. So that means that when you notice a performance problem with a query, Hasura gives you an explain
Starting point is 00:38:30 plan. So almost every GraphQL query, it gives you a plan and tells you these are the exact queries it's going to hit the database with. If it's like a multi-database thing, then multiple queries, a single database thing, a single query. And then now you can take that query, you can take that query,
Starting point is 00:38:47 you can analyze that query, and the database will now tell you that you're doing this join and it's a sequence scan. This ought to have been an index scan. You're like, oops, I forgot to add this index here. You add this index, it's passed again, which is what you would have done anyway. It's just that your time that it takes you to get to that insight of adding that index is much
Starting point is 00:39:03 faster because it's one query, you look at the bottleneck or the slowest point of that query the database will tell you what the slowest point that query is you go add an index if you can you're done and speaking of compilers another like interesting decision you made of course is haskell like it certainly makes sense from a theoretical standpoint and that's kind of what the language is built for. But you don't see that many companies, especially high profile ones, using Haskell as like the primary go-to tool, right? So I'd love to walk through that decision. How do you decide this makes sense?
Starting point is 00:39:39 And what has your experience been over the last three, four years? You know, hiring developers, onboarding, all of that. It's been extremely interesting. Um, the, the, we started off with Haskell because, um, it was two of us building, building Haskell out and we were like, Ooh, Haskell, uh, and, uh, we'd always wanted to use it and it seemed like the right fit, right? So we just started like, uh, writing it up in Haskell and it worked really well from a performance
Starting point is 00:40:08 standpoint, from the correctness that we got, the ability to iterate on the product it's such a core piece if something goes wrong in Hasura, your generation is wrong. If something goes wrong in Hasura there's a security problem so being able to enforce
Starting point is 00:40:24 correctness in what we're doing while iterating rapidly, right? Like building stuff rapidly, refactoring rapidly, right? That experience was amazing with Haskell, right? And because Haskell also has a very interesting runtime. It's a
Starting point is 00:40:40 multi-core runtime, but it's also multi-threaded, like real threads, but it's also asynchronous. So it's multi-threaded, like real threads, but it's also asynchronous. So it's multi-threaded with real threads and then there's green threading that they have on top, but the runtime also has like a async IO type mechanism. So it's kind of like the Node.js async
Starting point is 00:40:56 IO mechanism that's also available in the Haskell runtime, right? So in practice, the performance kind of benefit that we get by just vanilla writing code is really good. So that was really kind of good in the initial stages when there was a few of us writing Haskell code. Getting our early folks to onboard with Haskell
Starting point is 00:41:16 was also not hard. You know, multiple reasons, right? Like people who really wanted to write Haskell wanted to work on a Haskell thing, right? So it was easy for us to get like the first few engineers on the team and start to write Haskell, wanted to work on a Haskell thing. So it was easy for us to get the first few engineers on the team and start working with Haskell. That wasn't hard. Even for folks who were not familiar with Haskell, people who are familiar with a little bit of functional programming, we had a really nice kind of onboarding mechanism.
Starting point is 00:41:40 And within a few weeks, they would get productive with Haskell. And they'd be able to start writing code. The nice thing with Haskell is that it's totally okay to write bad code. It's totally fine because it's so easy to refactor. So you can write absolutely terrible, crappy code and it'll work and it'll be correct. And then a month later, you're like, ah, this is really bad. You refactor the whole thing, right? And it still just works and it's still correct right um in just the way that the way that you write has to code if you follow a few principles it makes that refactoring and stuff like that really easy so it's kind of getting started with that really now the the downsides that started emerging a little bit into that journey right was the reasoning about complicated memory consumption requirements.
Starting point is 00:42:28 That piece became a little bit hard. Because Haskell is lazy, it follows an internal mechanism of lazy evaluation. It makes reasoning about memory consumption a little bit hard. The ecosystem is relatively nascent compared to the ecosystem of libraries that you would get with say Golang or Java which are just extremely battle-tested especially for core parts like systems-y parts of systems interfaces that you would get to other data systems or for I.O. and stuff like that. And that was a little bit immature with Haskell. So we had to invest a lot more effort on that
Starting point is 00:43:11 side of things and making sure that that's instrumented well so that we can observe kind of its performance footprint much better. So early on, we had to dedicate a few people in our team to actually work very closely on building out internal tooling for that kind of performance and memory observability, right? And over time, that kind of engagement and that effort scaled into a lot of open source contributions that we've done with the Haskell community. We work very closely with the GHC, with the Haskell runtime and Haskell compiler kind of core devs. And our engineers kind of work with the core dev runtime and Haskell compiler kind of code devs and our engineers kind of work with the code devs we've submitted kind of a lot of improvements there we've helped move things along
Starting point is 00:43:51 and we've contributed a lot to that kind of part of the ecosystem so I think and we fund that quite a bit right so that has been a really nice piece that kind of came out of this but was necessary for us right as we were kind of scaling out because that tooling was absent. So we had to invest a lot of that, both people and money. And I think then kind of scaling beyond that, I don't think, I don't think hiring has been a problem again, because a lot of people want to write code master. And so it has been a good place for them to come and write code in Haskell. But just given how much smaller the Haskell ecosystem is and given how much smaller the
Starting point is 00:44:42 number of folks who've used Haskell in production before or who've had say engineering management experience with Haskell, those kinds of skill sets are relatively rare. So that has been something that we've had to be very thoughtful and gradual about. So we've had to put in a lot of effort there to make sure that that piece, that part of engineering and that part of the engineering process
Starting point is 00:44:58 is also scaling well. It's not been a challenge, but it's been something that we've had to be very thoughtful about. I have several questions that just jump out of that. But the first question is just around the principles you mentioned. You mentioned there's like a few principles if you follow. It's easy to write code that is easy to refactor later.
Starting point is 00:45:15 Just a couple. What are those principles? Yeah. So I think having an internal dialect of Haskell helps a lot. So there is, you know, Haskell can be, you can write very simple Haskell code, but because Haskell makes it so easy to layer on abstractions on top, you can start writing very complex Haskell code very quickly, right? And there are certain points then that parts of your code base will start looking like
Starting point is 00:45:40 you have no idea what's going on. It is so abstract and so torse it's like f weird function like angular bracket dollar angular bracket like x and then another weird symbol y right you're like what is happening um and it's hard to get into that right there's lots of layers of abstractions on top of each other and there's a lot of type level programming that you can do, which it's like people talk about kind of having generic types, right? But then imagine
Starting point is 00:46:12 programming at like where you're the type itself is being created almost as the program is running, right? The type is not available at compile time almost, right? So that kind of thinking then becomes much harder. And it's much harder to then onboard people into that
Starting point is 00:46:28 because that part of the code base people can't understand. So having a few principles where you decide kind of how quote-unquote simple you want to kind of keep your Haskell code base, what libraries you want to use, what abstractions you want to put in place, helps a lot. Having a particular way of approaching typing because with Haskell you
Starting point is 00:46:48 can kind of get into this world of saying that I will have strong types and contracts for everything everywhere. That can become challenging because then you feel as a programmer that you're fighting against the type system all the time and it doesn't make you feel as a programmer that you're fighting against the type system all the time. And it doesn't make you feel productive. Whereas what you really want to do is there are parts of your code that you are okay not having great kind of strongly typed contracts with and being a little bit loose. But the interfaces to the rest of your code kind of having better domain types. And then those domain types are a strong interface that then percolates your entire code base. So deciding kind of what those boundaries are is again very helpful. So those are kind of like a few different types of principles that
Starting point is 00:47:40 really help. When you start thinking about this going into production, there are a few kind of, again, best practices in the way that you would deal with certain operations, you know, whether you should be strict upfront or whether you want to be lazy, right? Whether you want to make sure that pieces are, certain pieces are instrumented for observability or not, right? Those portions also become important to, again, have conventions around in the code base, right? So those kinds of principles really then help. And then it becomes easy to onboard people and then kind of scale out that code base. So we've made lots of missteps. We've done lots of things that we like, oops, we really screwed up here. We shouldn't have done that. That piece is now like a hard piece to refactor because it's hard for other
Starting point is 00:48:19 people to onboard into. But then we're kind of taking that learning and making sure that we apply those learnings in other parts of the code base Basically, biasing towards simplicity as much as possible, it sounds like Exactly and the hard thing about biasing towards simplicity is that arriving at a common definition of
Starting point is 00:48:38 what simplicity is, is hard especially as the engineering team becomes larger what is simple what is the, right? Like what is the right line that becomes hard. Right. So that's the challenge, but I mean,
Starting point is 00:48:52 but I'm sure flavors and versions of this challenge exists everywhere. Right. It's not unique to Haskell. Right. It'll be the same challenge will appear in different forms, no matter what code base or what style you're writing in. And it's, it was surprising when I first heard it, but now as I think a little more, it's unsurprising.
Starting point is 00:49:09 You basically self-select your way into engineers who are willing to try something new, willing to do something different. It doesn't seem like a problem at all. Yeah, yeah, yeah. Like, I mean, maybe we have to worry about it when we're like 2000 engineers. But I think by that time, then we would have just got like our Haskell onboarding parts figured out so well, right?
Starting point is 00:49:30 That it won't be hard to, it's not hard. Like, it's not hard to write code in Haskell, right? It's not hard to get productive in Haskell. You just need really good onboarding guides, good conventions, good people in your team to pair with, right? And then people get productive really, really fast. Yeah. So, Houserad clearly is like a deeply, deeply technical product. You're solving for developers.
Starting point is 00:49:57 It's an interesting take. It's using all of these different languages and paradigms that are not commonly used. And clearly, you need to think about product development right how do you decentralize product development in a way that you're continuously shipping value to users in a sense right i don't think you can hire the standard product manager at least i would not think that is the case how do you how do you go about doing that like who do you look for what's the difference you know yeah that's that's that's a really good question it's a question that we're kind of working through at
Starting point is 00:50:29 the moment right so as as we're kind of scaling the company scaling the product scaling the team um it's it's been a it's been a very interesting journey again because i think the early um the early engineers on the team, right, including me, and I was obviously the first people to move out of engineering. Some of the folks from the early engineering team, right, from the early senior engineers who were more user-facing
Starting point is 00:50:57 and product-facing, right, they kind of naturally became the technical product managers, right? And so that kind of helped us solve some of those problems that a product management function would have solved for, right? Some of those elements around kind of the API design, right? It's kind of similar to if you were building a language
Starting point is 00:51:15 and your startup was essentially language, who does the language design, right? Like how do you decide what the right language design is? It's kind of similar to that kind of problem, right? So these early product managers and technical product managers were actually just senior engineers who knew, who understood what Hasura does inside out, who understood the philosophy at a very kind of gut level, right? And who didn't even have to write it down. And in fact, like we made this mistake of not actually writing it down, which we should have done,
Starting point is 00:51:43 because it just stayed in the gut and the mind of these early technical product managers. And so that kind of worked in the early days really well. Now as we're kind of scaling beyond this, the product management function has a few folks who are a little more on kind of the product management side who are interfacing with other functions who are interfacing with customers and and then there's also technical product managers right who are doing a little less of that side of the work but a little more on the kind of design side of the work and working closely with the architects right it's a very collaborative relationship from a kind of more customer facing product manager to a technical product manager to an architect to an engineer
Starting point is 00:52:29 right and they all have to work really really well very closely together right and boundaries are very blurred right it's not kind of the traditional boundary of like product versus engineering that boundary almost doesn't exist and for different parts of the product like the responsibility of kind of the more product side the more engineering side is very blurred and that's good because it's so dependent right it's like we come up with a particular way of doing something it would be an amazing feature but it depends so much on what is technically possible right and what is possible to get as the mbp and what is possible to even iterate and knowing that this design will never scale is information that we would only get from engineering.
Starting point is 00:53:08 The product team cannot take a call. So it's a very collaborative process in doing that. But again, that said, because it's early days for the product team, the engineering team is a little bit larger. The product team is still quite young. It's been amazing. I think our head
Starting point is 00:53:24 of product now, he was the head of product now uh dago joined us from he was he was the vp of product at a vp product and vp of engineering at new relic right and he joined us as a product um he when he joined us right for him it was amazing and he'd been playing around with himself um and when he joined us it was amazing because for him, it was like this perfect mixture of a very deeply technical product, but then also a product product. So it also kind of self-selects similar to the Haskell piece. The folks who really like to be at that, that, uh, boundary, right. At that junction of, uh, a very technical product and the technical design and the product design. So, uh, so, you know, so far so good. Uh, what, what we're learning a lot of is obviously like, you know, we should have written a lot of stuff down and we should have written all of these and
Starting point is 00:54:19 it talks around how this was designed. Uh, uh, much, much, we should have done it much earlier. We should have done it much,, we should have done it much better, which is again, I think typical problems for like, maybe more typical on the technical architecture front. If you're thinking about scaling a code base, and then you're like, what does this module do? Why is this design the way it is? Where are the architecture documents? What are the design docs? It's kind of that version that that piece that you skip in the early days of a startup or the early days of a product development that really kind of bite you that's
Starting point is 00:54:51 kind of been what we've realized that we missed doing that part we should have done that part because that would have made the product management portion so much easier right it allowed us to rapidly bring on new people who quickly looked at this design and be like, Oh, I see how Asura works. I see why we compile things. I see why we batch things. I see why we parse and validate things or whatever. They start understanding all of those portions very quickly. Right now, it's a little more tedious.
Starting point is 00:55:16 So standard product, standard problems that I think would exist with product management anywhere. But it's just that, of course, we do need kind of more technical product managers. The technical product manager role is very important. That collaboration between product and engineering is very important. And then, of course, having a community is great because that means that we can, again,
Starting point is 00:55:36 we self-select people who want to work at that kind of junction, right? It's your version of, we should have written down our company values and set up a company culture and all of that stuff but yeah okay so that's
Starting point is 00:55:53 fascinating and then the final piece that I wanted to ask about was this idea of Hasura taking over more and more parts of the workflow, like these escape holes, Hasura fulfills an important part of application development. By Hasura, I guess I mean in particular the Hasura engine.
Starting point is 00:56:18 It goes from exposing an API to doing database queries. But it's clearly things like a good example that even you have on your docs is you need to talk to Stripe in order to set up subscriptions and set up billing and all of that. So you need these kinds of escape patches. What's your philosophy on that going forward? And just in general, like what's the greater vision that you have for what Hasura can be? Because clearly it's no longer just a database compiler engine thing anymore. It's how do I build my application using this platform?
Starting point is 00:56:52 Right. So that's a, that's a, that's a very good question. So I think like when we think about like kind of application design, right. The, the,
Starting point is 00:57:01 the kind of product vision or product philosophy there is to say, if you look at a particular domain, you take a part of your application and let's say it's one domain and it has multiple subdomains, or you have multiple domains in the same application, whatever word you want to use to describe a logical grouping. Now, each domain is typically going to have essentially a set of models and a set of methods.
Starting point is 00:57:26 That's what every domain is. It's a set of models, set of methods. Models are the ways that describe the structuring of your data. And methods are things that operate on data. They do things to that data. And you don't control... Those methods are very custom. Those methods are very unique to your domain right
Starting point is 00:57:46 if you think about stripe even stripe itself is a domain which is a collection of models and methods right stripe has these models that you can read about like here are my save cards here are my users right here are my 20 transactions there's a transaction history for this thing for this organization and then here are methods which is like apply billing, right? Like make payment, right? These are all methods that happen and they all kind of affect models, right? So now in this view of the world, the way Hasura kind of works is to say, with all of the models that you have, Hasura will provide kind of the read API for all of these models. Hasura will also try to provide the simple kind of right APIs to these models if
Starting point is 00:58:26 these models are your right models. But in more complex situations, that might not be the case. And so that's kind of one piece of what Hasura will automate. The second piece of what Hasura will do is that it will allow you to invoke methods. It won't be the method, it will just allow you to invoke the method. Your method will run and do something and after your method does something it will return a reference to a model or it will return a list of models right so after you make a payment you get a payment id as a response right after you you create an entity you might get like a entity id or a list of entity ids as a response of that method what hustler does is it takes the response of that method and references it back to the models and then returns the full graph of the models, right?
Starting point is 00:59:11 So that means that your method is really simple. Your method just does the create payment logic and returns the payment ID. It doesn't care about the payment model. It's updated the payment model, right? It's done some work and it's returned a payment ID. What Hasura does is instead of in the API, instead of just returning the payment ID, it actually returns the entire payment object. It returns the entire payment model because Hasura has all the information about what this payment model
Starting point is 00:59:34 is, what fields are accessible, what's readable, what's tokenized, what's secure, etc. So that's kind of the view that Hasura takes for each domain. Each domain is models plus methods. All the model work is automated. Methods Hasura will help invoke on and Hasura takes for each domain. Each domain is models plus methods. All the model work is automated. Methods Hasura will help invoke on and Hasura will help enrich those methods. Now when you come to kind of multiple domains that you're bringing into Hasura, the same kind
Starting point is 00:59:54 of concept scales, where you're saying that now it's not just models that are within a domain, these models themselves will have relationships. And any method which will return a reference to a model maybe in the same domain, that model might have relationships to other parts of the domain. So that kind of then results, the net result that you get out of that, it's kind of this quote unquote unified API that semantically makes sense for the API consumer. But on the back end, it's pretty complicated. It's made up of several different subdomains, right? Whether it's search and transactional and Stripe as a SaaS service, right? Or some legacy service that you have.
Starting point is 01:00:30 Whatever it's composed of, it kind of then comes together as one kind of semantic API that makes sense for the API consumer, right? So that's kind of how we think about what Hasura enables, right? So what we want to do is get out of the way as much as possible of the repetitive kind of work so that you move from the thought process
Starting point is 01:00:52 of model authorization done, connect methods done. So nobody's thinking in GraphQL, nobody's worried about caching and joins and this and that. You're just writing the tiniest portions of logic, you're just defining your models and everything else should just work right um the next piece of that is kind of technically when we think about how we're evolving us in that direction technically right um one of the things that we're going to launch uh i think like maybe like first half of next year which i haven't talked about at all. So I'm super excited to be talking about it. Still a little bit hush hush.
Starting point is 01:01:28 Is the effort that we're calling GraphQL data wrappers. It's inspired by, if you've heard of foreign data wrappers for Postgres, it's inspired by that idea about Hasura. So the entire kind of Hasura machinery and magic, right? It does, it takes a GraphQL query, it does a bunch of things. And then at the end of Hasura machinery and magic, right? It does, it takes a graphical query, it does a bunch of things. And then at the end of it, the last point,
Starting point is 01:01:49 it has the piece that knows how to generate its internal AST into the specific SQL syntax that Postgres speaks, right? And so it's in the compiler pipeline, that last piece is the piece that does, that bridges it to a particular database. And so we've kind of now added support for SQL Server and BigQuery and MySQL that changes the last portion of that. So now what we're doing is we're introducing
Starting point is 01:02:14 this concept of GraphQL Data Wrappers that allows you to bring in support for any system to Hasura in any language. You can write it in Go and TypeScript or whatever, and you can build your own abstraction for Airtable or S3 or CSV file or Redis or Cassandra or whatever you want, right? And get like a Hasura-style API on that. So all of the Hasura magic of the joins, the compilation, the caching, the querying, etc. will all work
Starting point is 01:02:46 and then your adapter, your data wrapper will actually bridge that last portion to the database that you have, right? So that's the piece that allows us to dramatically accelerate the speed at which we're adding support for new databases and of course making it kind of available to the community,
Starting point is 01:03:02 right? So that the community can kind of add data wrappers for their favorite data services in their own language and whatever style they want, whatever data service they want and they can just get all of the benefits of Hasura but for their data service. So we've been
Starting point is 01:03:18 prototyping this internally. It took us like a week to add support for Athena SQL and get all of the Hasura benefits on it. So we're now kind of stacking that out, documenting that well, and then we're going to release that interface for our contributors and developers
Starting point is 01:03:35 so that everybody can just be like, go crazy, bring in whatever data source you want, anything on the interwebs, convert it to like a H Hasura GraphQL API. And you get a bunch of these things. It's kind of like if you can, it doesn't have to be maybe even as raw as S3. You can do things like get me a pre-signed URL where I can upload a
Starting point is 01:03:58 document, right? And that now you have a GraphQL API that can expose, download URLs, upload URLs, everything like that. Exactly. And the only thing is, as long as you can model your domain in the Hasura model method way of thinking about domains, you can bring in anything. You can bring in any concept and say, here are my models. Here's a list of models. Here are the methods. And so now at the Hasura level, I can do joins between models and I can connect them to each other.
Starting point is 01:04:32 Methods can return references to models and all of that. Yeah. This is another sign where knowledge work is disrupting other knowledge work. Exactly. I don't know if I'm going to have a job in 15 years or so. I think, I think, let me,
Starting point is 01:04:50 let me propose an alternative, which is, everything seems to be super GraphQL centric. I'd be amazed the day where you can automatically generate
Starting point is 01:04:59 React components to us direct from the data model. And that's when that's the day we arrive yes so the amazing thing is that it's already happening
Starting point is 01:05:11 in the community right like I know a bunch of projects that are perhaps in the early days where you can go from a GraphQL schema right a GraphQL API schema right to the appropriate React components automatically, right? So like mutations will generate forms automatically, right?
Starting point is 01:05:30 Queries will generate kind of list type components automatically or object type components automatically. That people are already doing that, right? Because once you can generate the React component for that base data structure, after that you can have higher level tooling to help you style that component also very easily, right? Stuff like that. So that's
Starting point is 01:05:48 already happening, which is just awesome, right? Because all of this just fits in, right? The recent stuff that's been happening in the low-code ecosystem on the front-end side is amazing, right? It's just super productive. And it's
Starting point is 01:06:04 hard. It's hard to scale that, but there's a tremendous amount of innovation that's happening that does make it seem like it will be scalable. So I think it's going to be amazing. I think 10 years from now, we won't even recognize what has happened to the industry. We're just like, what is going on? I can't believe we used to write this code 10 years ago. The philosophical thing, and I've been working through a side project, right? It's like, there are all of these tools in the middle
Starting point is 01:06:34 that help you automate parts of your workflow that would be painful. Things like that are memorable for, you know, user authentication and all of that. But they're so fragmented in a sense that I'm kind of waiting for some player to come up and say here are all of these things
Starting point is 01:06:51 in a bundle. You write your database schema or whatever lower, higher level abstraction than that. We expose all the APIs, all the React components. You decide your prime, your primary colors in your web app and we decide everything else.
Starting point is 01:07:09 So that's the only choice I need, which colors I need my app to be and everything else you can decide for me. Yeah. That's the future. I think that should, that should happen. Right. And maybe, maybe like that for me is my wish of what happens to engineering going like especially like software engineering going forward right like it goes from engineering straight up to art and design
Starting point is 01:07:31 right it's like we stop worrying about the details of how all of this works right we just work at like the highest level concepts right we're like yeah here are my data models here's how i want my logic to work and you want like to write that logic in the cleanest simplest way possible and you just want it to scale and work forever and then when you kind of come to that that experience of what you want to have you want to be as abstract as possible right like you said colors like design themes right like i i want it to be like i wanted to have like an abstract modern design theme right i wanted to have like a retro design theme, right? You want to work at those levels, right?
Starting point is 01:08:07 You want to work at like that level of creativity. You don't want to muck around with like hex format of colors with what alpha value has. No, no, that's the future. Can't wait for 10 years from now. Yeah. And when I talk to people like you, I get very excited, but then I go back to my day job but i have to fix npm packages that don't follow some somewhere
Starting point is 01:08:31 and um hopefully it's coming soon um but yeah we'll all we'll get there anyways yeah tanmay thank you so much for being a guest this was an amazing conversation we have i feel like hours and hours of stuff to talk about more, but I'll hopefully get you for a round two at some point. Absolutely. Thanks so much for having me, Uttav, and hopefully you all enjoyed the conversation.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.