The Data Stack Show - Re-Air: Why Traditional Data Pipelines Are Broken (And How to Fix Them) with Ruben Burdin of Stacksync

Episode Date: August 26, 2026

This episode is a re-air of one of our most popular conversations, featuring insights worth revisiting. This week on The Data Stack Show, Eric and welcomes back Ruben Burdin, Founder and CEO of Stacksync as they together dismantle the myths surrounding zero-copy ETL and traditional data integration methods. Ruben reveals the complex challenges of two-way syncing between enterprise systems like Salesforce, HubSpot, and NetSuite, highlighting how existing tools often create more problems than solutions. He also introduces Stacksync's innovative approach, which uses real-time SQL-based synchronization to simplify data integration, reduce maintenance overhead, and enable more efficient operational workflows. The conversation exposes the limitations of current data transfer techniques and offers a glimpse into a more declarative, flexible approach to managing enterprise data across multiple systems. You won’t want to miss it.  Highlights from this week’s conversation include: The Pain of Two-Way Sync and Early Integration Challenges (2:01) Zero Copy ETL: Hype vs. Reality (3:50) Data Definitions and System Complexity (7:39) Limitations of Out-of-the-Box Integrations (9:35) The CSV File: The Original Two-Way Sync (11:18) Stacksync’s Approach and Capabilities (12:21) Zero Copy ETL: Technical and Business Barriers (14:22) Data Sharing, Clean Rooms, and Marketing Myths (18:40) The Reliable Loop: ETL, Transform, Reverse ETL (27:08) Business Logic Fragmentation and Maintenance (33:43) Simplifying Architecture with Real-Time Two-Way Sync (35:14) Operational Use Case: HubSpot, Salesforce, and Snowflake (39:10) Filtering, Triggers, and Real-Time Workflows (45:38) Complex Use Case: Salesforce to NetSuite with Data Discrepancies (48:56) Declarative Logic and Debugging with SQL (54:54) Connecting with Ruben and Parting Thoughts (57:58) The Data Stack Show is a weekly podcast powered by RudderStack, customer data infrastructure that enables you to deliver real-time customer event data everywhere it’s needed to power smarter decisions and better customer experiences. Each week, we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data. RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Hey, everyone. Before we dive in, we wanted to take a moment to thank you for listening and being part of our community. Today, we're revisiting one of our most popular episodes in the archives, a conversation full of insights worth hearing again. We hope you enjoy it and remember you can stay up to date with the latest content and subscribe to the show at datastack show. Hi, I'm Eric Dodds. And I'm John Wessel. Welcome to The Datastack Show. The Datastack Show is a podcast where we talk about the technical, business, and human challenges involved in data work. Join our casual conversations with innovators and data professionals to learn about new data technologies and how data teams are run at top companies. Before we dig into today's episode, we want to give a huge thanks to our presenting sponsor, Ruttersack.

They give us the equipment and time to do this show week in, week out, and provide you the valuable content. RudderSack provides customer data infrastructure and is used by the world's most innovative companies to collect, transform, and deliver their event data wherever it's needed, all in RutterSack. real time. You can learn more at rudderstack.com. Welcome back to the show. We are here again with Ruben Burdine, and he joined us at Data Council. We did a little bit of a lightning round at Data Council, Ruben. So we'll take our time to dive deep. Thanks for joining us again for your second to second slot on the show. Yeah, thank you so much for us. Great. Well, give us just a little bit of background for those who didn't hear the Data Council show. Give us a little bit of background on yourself, and then just

the one or two sentence overview of StackSync. Yeah, perfect. So, actually, my name is Rubin. I'm confederate and see you at Staxsync. So I am based here in San Francisco, California, building Staxank with our team. I'm originally from France. So a bit of my background. You know, I did study the computer science, double degree in computer science and one degree in business as well, back in Switzerland.

And then I worked as well, you know, in Germany and in Singapore. And actually, this is also where I got really in touch with the world of two-way syncing. because I was working, you know, in a company and I was as a consultant and was in charge to fooding everything in place, you know, from accounting software to ERP to CRM. And I, you know, all of these tools to work in two ways sync. So what I did in the CRM reflect into the ERP and vice versa? And there were no products on the market, you know? Like then I searched, you know, I tried to build, you know, some alternatives myself, you know,

with somehow, you know, workato, et cetera. And none of them really worked was really complex. and I just couldn't leave the company because everybody was afraid to take this work over. And this is where actually I realized, you know, like this is where I realized, you know, this is a big whale problem. Everybody's complaining this should exist. And so, and they are committed, you know, I started an entrepreneurial journey. And now here we are.

We did YC. So Staxing, basically, we were running for a year and a half. Then we did YC, Y Combinator in the winter 24 badge. And then, you know, we moved to San Francisco and really got this explosive growth that we have at the moment. Awesome. Well, one thing that I'm excited to talk about is where two-way sync fits into the stack because there are a lot of companies who sort of use a traditional sort of in, transform, out type loop. So I'm excited to dig into that and just learn more about two-way sync in general, dive deeper than we went last time. How about you, Rubin? What do you want to talk about?

Absolutely. So I'm super excited to talk about this two-way sync where it fits in the stack, but as well as like, you know, I'm very, you know, surprised, you know, how marketing is actually reshaping the perception of people on, on, you know, zero copy ETL, this kind of trend, you know, and how actually exists. Right now, as we stand, no, it's most of marketing and little tech, right? And it's crazy how much, you know, this tech people actually go into this, this fantasy of vendors, you know, selling it. And so, yeah, extremely happy, actually.

also to decode that a little bit further. And yeah. Great. Well, let's dig in. Let's do it. Rubin, I love having second-time guests because I can say, if you haven't heard the first show, go listen to it. And you'll get context and listen to them in a row. So we can kind of dig right into some of the spicy topics.

The first one of which we'll cover is that zero copy isn't a real thing. And so very excited to dig into that. But first, I want to do two. things. One, I want to ask a little bit more about your background, and then I want to give our listeners a high-level overview of StackSink. We'll get way more into that later. You mentioned when we were recording the intro that you were a consultant. You were trying to get all these tools to talk together, you know, CRM to talk to the ERP, and it was a really brutal problem. You tried all these tools. What was the worst integration problem that you faced during

that period where you just, you know, you just said, this is so gnarly and bad that, you know, did you ever think about giving up? I mean, what was the nastiest problem? Absolutely. I mean, like this is, I mean, you know, building to sync with workflow automation tools or code was, as you mentioned, like really brutal. I mean, like, you know, like you, first of all, you know, you have to think about the whole hyperticture. Then you start building. So after maybe like a day or two, you get a workflow. And then you realize, okay, now when I edit, record on one side it goes to the other side and vice versa and then you say okay well cool this is great so now what okay so this is for one record which i just created what about for update oh and now you have

to figure out an entire update logic right then you realize you know okay i'm going to take the entire record so whenever you know i have a record updating sales force i'm going to update into my hub hub spot and vice versa and then you say well okay so i just because says force only tells you that the record has been updated but not which field so now you say well intuitively, I'm going to take the entire record. I'm going to push it into a hot spot, but then you override the email. So now you have a, for sure,

you have a marketing person in the company who created a sequence and say every time email is updated and rolls this into a welcome email sequence. So the guy, you know, so customers start now receiving welcome email every time something is updated on the Sierra. And so that's, as a representative,

you have to know exactly what changed. Now you have to run an entire system which detects which field was updated into a cell. a record. And now you start storing data and then you have deletes as well, right? And once you have done it, say, all the credit operations, you're like, well,

that's great. But now this is for one record. Let's backfield the data. And now you have to backfill all the historical data of the same system. And now it becomes extremely complex. And now every time that your system misses one event for any reason, one

custom field added, you know, someone change your name of a field or something like this, it bugs and you love the data. There is no monetary. And the maintenance over like the first three weeks were so huge. It was taking my full-time integrator position for a single contact sync pipeline. And so, well, I'm laughing. It's a painful laugh because this is over 10 years ago.

But I have to confess that I was a victim to the promise from the sales team that I can't remember if it was sales force or her. HubSpot, but it doesn't matter which one it was. But they're like, oh, yeah, we just integrate with, you know, let's just say it was HubSpot. They're like, we have a direct integration with Salesforce. All your data just goes back and forth. You just configure a few things, right? And I was like, oh, well, that's great. And, you know, I could not have been more wrong. And actually the thing that, and I'm sure things have gotten better since then, because it's a long time ago. But it was impossible to troubleshoot, especially with large batches, right? I mean, you upload a bunch of leads. You do all of

that. But the other thing actually that I'm interested in is that was a one, that was one contact sync flow. But if we take Salesforce and HubSpot as an example, any data engineer or analyst who has worked on data from either or both systems knows that like a contact record and what that means is not the same in these different systems, right? And the database design is different and they can actually have like pretty dramatically different meanings even across teams, right? And so like that's what you described sound really painful with the assumption that it's a shared definition across the two systems and across the two teams, which is actually more rare than it is common. And I think probably what you had 10 years ago is probably even worse now because the systems are more complex instead of getting simpler because technology evolved. And like because it's even worse.

and I'm telling you, like, now, and now we're just talking about how to sync systems, you know, databases, CRMs, between each other, etc. But now you're mentioning the problem of the definition of a sync. So it means you need to have filtering, right? But filtering from A to B is not the same as the between from B to A, even with the same definition, right? So this goes very crazy.

And also, if you want to really go even deeper, no, contacts are not standalone. Contacts belong to a company. So now you have to associate. Yeah, yeah, yeah. Yeah, yeah. It's insane. Different association models.

Also, you know, like you have ordering, right? Because you have to first create the company. Then the contact, because of contact, then it's the company. Yeah. But like, you know, in other systems, you know, you might have different association systems. I mean, so where you have to create the record is your company and then associate them, you know. So it really, it's really different. And so you have all of these complexities to actually maintain.

And this is what makes two ways think very hard. So in the very beginning, you know, it's very hard to go into the market. and say, yeah, you know, it's very easy, you know, we integrate, et cetera. And for example, let's say there is it's a very big, very big pain on the market at the moment, right, which is, you know, HubSpot Zendesk, for example, right? Integration of HubSpot and Zendesk. You know, the team as HubSpot is going to tell you, yeah, of course we integrate with Zendesk, you know, you can use Zendes for your customer support, HubSpot as your CRM,

and this is going to work fine because we have to a sync. So first of all, Omni Contact and companies and tickets are associated, But actually the tickets, it's not the tickets as you imagine them in the HubSpot. So the Zendes tickets is going to have to be synced to the service, you know, I mean, to the service hub of HubSpot. So you have to subscribe to the Zen Desk of HubSpot to case it. But because you use Zendesk, you don't need the service hub of Hotslots. So you're actually just buying towards the same product to actually use it.

And this is not the tickets you want to sink. So eventually, like, you're just not sinking. And, you know, like, you know, you won't have, for all of your marketing contact, right? So you won't, so because, you know, contacts will sync, you know, have different definition in HubSpot and in Zendesk. Every person you send a marketing email, you know, even a call lead from HubSpot. You don't want it to sync to your Zendesk's server system, which is made for your customers or the people with very high intent.

And so what about this transformation which happened in between? You know, custom objects are not supported. Associations are not supported. Like, it's completely crazy. And so, and this integration are very costly and they still sell. Yeah, it's wild. Yeah. That's why the original two-way sync is called a CSV file.

Exactly. Exactly. Exactly. I mean, that's, you know, a CSV file and like lots of V-lookups and stuff just, you know, and there's usually, you know, there's usually someone, you mentioned, you know, in the intro that it's hard for you to leave the company just because no one knew how all this worked, right?

And CSV files can be that way where it's like there's one person who knows how to get it just right. Yeah, it's crazy. And this is where Staxing, you know, came all about came to be, right? So Staxing basically also, you know, really deep dives into this nature

into this reality of two-way syncing between enterprise systems and where we really are, you know, release a leader is really when it works at scale, right? So Staxing basically really gets this two-way sync at scale, and it requires a whole complex engineering, which is almost a database label, you know,

conflict management, you know, technology, which has been developed over like tens of years. And so, so, yeah, so just more about Staxing, you know, Staxing is today, the leader in real-time and two-way syncing between enterprise systems and databases. So, you know, Staxing supports CRMs, like Salesforce and HapSpot, Zoho, et cetera, but also ERPs like NetSuite, you know,

ACAP, you know, Acumatica, and all of these tools, basically, they can be synchronizing two-way sync with databases, such as Postgres, Snowflick, BigQuery, MongoDB, MySQL, Oracle, D, you name it. And so what really Staxing and enables to do is really to actually bypass

all of this iPass tools, all of these complex in-house code, you know, custom code, logics, and just have like a two-way sync as you would think it can is in a human manner. Just simplify your architectural diagram to a very baby level, right?

Just like, you know, two, one CRM, one database. Whenever you modify something into the CRM, it goes into your database. And when you modify that same table, not not on the table, the same table, back, it's actually going to write back into your CRM or ERP. And that's what stacks and, you know,

really offers in real time with millions of records per minute, technically at big times. I love it. Okay, I have a ton of questions about that, but I promise the listeners we would do the spicy. We would get to a spicy take early, which is zero copy isn't a thing. So I want to move around back to Saksing specifics, but this really picked my interest when we were chatting before the show. I think you use the phrase, zero copy isn't real.

I think that was the phrase. Okay. So give us the spicy take because it has been a. major, it's been a major topic of discussion, you know, product launches, feature launches, a lot of ink has been spilled. So, and I'm sure all of our listeners have heard of it, but for those that haven't, what is the promise of zero copy? Give us like a baseline of what a zero copy mean. Yeah. So let's get zero copy or maybe in its full term zero copy ETL, or zero ETL even sometimes. It's basically the fact that's right now, basically, you have to

use, you know, 5Trent, Stitch, AirBIT, etc, to transfer your data from, you know, an external source to the warehouse, right? And, and so, you know, with this, there are a lot of recent tech developments, which actually tells you, okay, maybe we can agree on a common data format, and actually every system would pump on the same storage, so that there is no copy between systems. Actually, you have, we have one source of truth of data, and, you know, the CRM would actually pump on this data. So the warehouse would pump on this data. there is no transfer, just a single place or single storage, right? And through this, there are technical and business challenges that, you know,

at least 10 years or 15 years before it's solved, if it will ever be, right? Because it's really a business problem, actually. It's a very root. And every take problem is a business problem in the end. And so... Dig into that a little bit more. What is the business problem? What is the business problem?

And why will it potentially not get solved? Yeah. So basically, ETL in general is basically like we need to have different data sources and we want to bring everything in the same place. So we can actually have, you know, data available for insights and reporting and do all sort of like operational things. So it's a business problem. Like we need to do some real stuff and make some real money with us. And now what happens with this zero ETL is okay, well, shipping data from A to B is actually very long, costly and hard to get very high. accurate, no at scale. Yes. Yep.

So, so, so, so, so, so, so, so, so, what happens is that taxing will, I mean, taxing or like any vendor actually would just transfer data, but this can get long. So, so people say, well, let's agree on a common data format and a common place of storage. So we don't have to copy everybody can just come and grab what they want, but there is no transfer.

You know, there is no transport. It's just a common place. Okay. So, so, so that's a very good idea. But then this has a very big business. issue why this almost cannot happen is because data warehouse, you know, I mean, this common data format has to be sort of efficient for everybody. Data warehouse work in a very different way than, you know, which is to make some query

at scale. Sierra, which is to retrieve records with different indexes, you know, to make it very fast for users which work on a daily basis, right? So this means that the storage of these two different, you know, this Salesforce and this snowflake have to be very different. in nature. So just performance-wise and business-wise, it's a problem. But also strategy-wise, right?

I mean, like, what do you think? Do you think, like, saleswoman is going to open up their backend and all of those are business secrets to everybody, right? It's like, you know, yeah. This is not, you know, the schema will never be exposed because also, like, in the database schema, a company has much more than the data which is exposed to the customer. They have also a lot of metadata, a lot of organization,

relations, optimizations. And, you know, like, so the storage just cannot be the same because some part of it needs to be messed. So now you have to be a, you have to have very deep, a role level and column level, you know, access rights, right? So this caused another problem, right, of security. How do you make sure this never leaks? And so, so you have all of this business problems, which actually make sure you always have to make copy simply because, like, the data, which you operate in cannot be endangered. And also, like, we have a common place of storage. And I'm throwing another question to the industry.

So Salesforce and Snowflake would share the same storage. Where is this storage? You know, who pays for this? Where is it? You know, you know, who accepts to have latency, right? Do we all locking the same vendor into the same Amazon S3 bucket, you know, or Google blog storage? Do we have to lock it into that storage?

Because now if we move, we have to move everybody. Right? So it's even more locked in. So we have so many problems that happen. And so all these issues, right, are some. something which make zero ETL or zero copy really something which is, you know, almost fantasias today. Yep, yep.

It was interesting. We had a guest on the show from, it's a company called Wild AI, and they do, they facilitate some data sharing between an e-commerce store, you know, like a digital e-commerce store and then like a physical retail store where their products are sold. So, you know, you can, of course, see that there would be benefit in sharing, you know, having some crossover data there that benefits both parties. So they had really similar things to say about clean rooms because they thought, oh, well, we'll just use clean rooms to facilitate this. But when they really started to dig into it, they found similar things to what you are saying where it is that you can do some things with it, but it is not, you know, it doesn't quite live up to everything. thing that the marketing says it is as far as this full seamless functionality where you can

just dump data into a clean room and all this magic happens. It's actually, you know, there is actually a lot of work in order to figure out how to make it work well with both parties. And so they ended up actually building a, you know, sort of a different architecture. But fascinating, fascinating. It's the power of marketing. Yeah, the power of marketing. And this is, and this gets even more critical, right? Because like, you know, if you really deep dive into how it really works. Okay, so you go to the Salesforce to Snowflake Data Sharing, right? It's just zero copy, zero, zero caffeine, zero, everything.

You know, it's like, you know, zero nothing, right? Zero calories. Right. It's very Buddhist. You know, just, you just blank. Yes, pure. Yeah, yeah. Pure data, right?

Just pure data as a right place. And then you go into this and you say, well, in the documentation, you have a five minutes replication like. So I mean, like, if I really read zero copy and I really understand it as a human would, five minutes, you know, it's already, there is a problem. If it's in the same place and sorry, five minutes, and replication like, so if you have the word replication into the documentation of something, which is zero copy. Yeah.

That's concerning, right? That's concerning. And so eventually, what is all of this, you know, like my take on what is data cloud, you know, data sharing with snowflake or HubSpot, you know, snowflakes. you know, snowflake data share. This is just Salesforce Snowflake account.

No. That's the only thing is. You know, it's just like, they manage the entire ETL pipeline for you. And they give you a Snowflake account which you can actually just

go grab the credentials. You can grab the credentials and just query it. That's it. You can't try it to it. You can do anything. You can add custom field. You can just query it. It's just,

okay, it's just a dump of data which is locked into your snowflake. And so that's, So what that means, also, it's a, for companies, you say, well, it's great. You know, instead of going to FavTrain, I can actually buy, you know, all of these tools, right? But it's a very big problem now. Because when you have plenty of pipelines with Favtran, et cetera, you have both discounts. But when you actually have to buy this small item from Salesforce, this small sink from HubSpot, this small thing from Zendes, you have to actually contract with 10 or 15 different data sources.

I mean, I mean, vendors to actually get data into your warehouse instead of just having one ETL, which is maintained by your data engineering team. So now you have a distribution of ownership of these pipelines, which to people who have nothing to do with pipelines because they are just working in Zendesk or just working in Salesforce. And so that's a very important, basically, that's a very important strategy and cost problem as well, which also make zero ETIL. So zero ETIL is complex from a technical standpoint, if not saying almost impossible. it's complex from a strategy standpoint and it's complex also from cost standpoint, right? So actually all components which drive a business in reality are just not present into this zero copy landscape.

So this is, it's the most not existing, basically. Yeah, it is a really unfortunate term because I can see a narrow use case. You know, when I say narrow, what I mean is there's a business team working in some tool and usually have an operations person. And there is a use case for being able to write a query in that tool and, like, pull in some data set or something, right? And actually, I remember we had someone from Braise on the show.

Braz is a marketing tool. You can send customer communications, you know, and create, you know, customer journeys through Braise. And they launched a tool that had, that allowed people to write SQL queries and you could, you know, sort of, you know, pull data in. and he thought, you know, power users will love this, but like adoption was way more than he thought and like a ton of people use it, right? And so there is this interesting use case there where it's like, okay, you need some specific thing.

Your data team has probably materialized a couple of views that have some things and, you know, some valuable data fields. Just pull them into your tool, right? I mean, that's actually a totally valid use case. But calling it zero copy deal is really misleading because that's not actually the value that it provides or even really a good description of what's actually happening, right?

You're just querying data from a very high context individual system, right? So it is unfortunate. Absolutely. And I see, though, a value in zero ETS and a legit point, I mean, which is not legit, you know, really per se, but at least it can be legit from a business perspective. Let's say, for example, say you have a system, like say, in your P with a very complex data structure,

a complex data format, you know, which is quite hard to actually expose over APIs. Or, you know, like the vendor just doesn't do it because of strategy, like SAP, right? So, you know, say it's very hard to get data out of SAP for no real apparent reason, just because I don't want you to get out of the ecosystem. And so maybe like, you know, for vendors like this, selling this zero, I mean, I mean, data share, this is really data share. The data share can be valuable because they can expose data, which you cannot get access

via APIs independently, right? You need to get access to your own data in some way, and it's not possible to make it accessibility APIs. So that, maybe because of scale, because of complexity, because of data types, you know, this kind of things, you know, could make sense,

but it's because of technical or business limitation that the business has, and this is where data share makes sense. But, you know, like data sharing, you know, for HubSpot or for Salesforce, It doesn't really make sense because, you know, the APIs still enables and fall off real-time thing. And so that's why, you know, like, there is no real need for this.

You know, so actually, you have to really introduce hard limitation on the API. So at least it would be the only one, but this is critical monopoly, which would be a very big scandal. Right, right. We're going to take a quick break from the episode to talk about our sponsor, Rudder Stack. Now, I could say a bunch of nice things as if I found a fancy new tool. But John has been implementing RudderStack for over half a decade. John, you work with customer event data every day and you know how hard it can be to make sure that data is clean and then to stream it everywhere it needs to go. Yeah, Eric, as you know, customer data can get messy.

And if you've ever seen a tag manager, you know how messy it can get. So RudderStack has really been one of my team's secret weapons. We can collect and standardized data from anywhere, web, mobile, even server side, and then send it to our downstream. tools. Now, rumor has it that you have implemented the longest running production instance of Rudder Stack at six years and going. Yes, I can confirm that. And one of the reasons we picked Ruttersack was that it does not store the data and we can live stream data to our downstream tools. One of the things about the implementation that has been so common over all the years and with so many Rudder Stack customers is that it wasn't a wholesale replacement of your stack. It fit right into

your existing tool set. Yeah, and even with technical tools, Eric, things like Kafka or PubSub, but you don't have to have all that complicated customer data infrastructure. Well, if you need to stream clean customer data to your entire stack, including your data infrastructure tools,

head over to rudderstack.com to learn more. Okay, well, that was great. I love digging into the zero copy ETL stuff, zero ETL. It's just going to be zero at some point. Yeah. You implemented Z, you know, talk about two-way sync. And let me frame this a little bit because there's a very common loop that has been reliable for a really long time.

And actually sort of pre-existed the modern data stack, right? Because people did it with, you know, whatever, they would write pipelines in Python or whatever, you know, if you were completely hand-rolling it. But let's just, we'll frame it in the terms of the modern data stack. Okay. So I have Salesforce. I'm on a data team, right? I have sales, like, my, this, you know, the GoTo Market Team uses Salesforce for all the business stuff, right?

It's where they track leads and campaigns and opportunities and everything. I need to combine that data with other data, you know, some model. I need to, whatever, right? Enrich it. So I use Fivtran. I pull the data into some data. to source snowflake, data bricks, etc.

I model the data. I'm using DBT or some transformation layer to enrich the data point, do whatever transformations I need to run, and then I reverse ETL back into Salesforce, right? And then, of course, that gets pulled in 5TRAC, you know, 5TRAN and that's the loop,

right? And that's been a very reliable loop for a long time. Again, there are sort of tools to do that now, but again, it's been going on for years. What is the problem with that loop. Yeah. So that's a very interesting question. Because then you say, well, you can build, you know, if there is no problem, you can build two-way sync, you know, manually, right? And as we were saying in the beginning of the podcast, you know, there is this, you know, it was an absolutely

mess to maintain. So the problem with two-way sync is that, you know, two-way sync is an extremely hard problem to actually get because of the limitation of current tools, right? So for example, I was mentioning, you know, okay, so if you modify your record, you know, you know, in Salesforce, it would tell you that record changed, but doesn't tell you which field changed. So now what do you send to your hotspot or NetStreet? You override the entire record? So every time that you have a, you know,

if you override the email, even with the same value, you know, even though it might look the same, you might have a rule which every time you update this field, you know, send a welcome email. So now every time, like, you know, you update something in the CRM, with even like something, a completely unrelated field like first name or last name,

you would actually send a welcoming mail because you'd overrides an entire way for it, and you lose data. And so if this is one way... But in the loop of like Salesforce, ETL, transformation, reverse ETL, Salesforce ETL, transformation, reverse ETL. I mean, that's not really two-way sync, but you kind of handle all of the nasty stuff in the transformations, right? I mean, that's kind of where... That's where you use like most of the work happens. And is that like, I mean, ultimately those things get crazy. Is that like the big challenge with that and like why the value proposition of two-way sync is attractive?

Or is that loop not suited well for like for specific use cases, for example? Because I think that people use the loop for everything, right? I mean, it's kind of the it's the go-to for any, I get this is a probably a dramatic oversimplification. right? And it's like analytics, ops, you know, operational data, whatever. It's just throw it into the loop and we'll figure it out in the transformation layer, right? Yeah. Which gets complicated and expensive. I know is one issue, but walk us through the other issues. Absolutely. So, I mean, like now we're talking about one pipeline, right? Maybe the Salesforce contacts to Snowflake, you transform DBT, you put this into another table, which is a staging, and then you have. have like an ETL which is scheduled, hopefully coordinated, but you know,

his orchestration is still like a piece of problem. And actually, which would put it back. Huge, actually, yeah. And send it to, you know, query from Snowflake and send it back to Salesforce. So here is the reverse ETL vendor. They have a data storage in between, which actually compares what you send as a query. Yep. So they're running the diff.

Yep. Is it running the diff? And then like the difference is actually sent back to Salesforce. So this is all the fix the problem. But so this means that you have one ETL vendor, one reverse a TL vendor, you have at least two tables in your house.

That is the, yeah, I mean, at least two, you know. For a baby startup company, you know, very simple, yeah, at least two tables and this and you know, and a DBT engine, right?

So that's cool. And now you only talk about contacts because companies is another one. And if you need to add contacts and companies and associate them, you need to also make sure the company's feedback first after, you know, before the contacts, because the contacts don't have companies.

And so, but like if you want to associate a contact in a company, you need to have the Salesforce IDs in Snowflake. So you can send, you know, create the record in Snowflake, right? I mean, Salesforce. So it means that, you know, you need to first do the first loop of companies, get the IDs back for the companies you created into Snowflake. And only then you can create, you can start with the contract. But you have this kind of managed fields which needs some feedback because you first have to create a company into from Snowflake to Salesforce.

Get the ID back into Snowflake. Use that ID to create a contact and get the contact back because then you need to create opportunities or something like this. And so all of this orchestration now gets you tables and tables, et cetera. So this is the promise of having a simpler architecture is wrong. and what this means concretely for a business, because complex tech is not a problem. But heavy maintenance, that's a challenge.

And this is why the loop is not working. And this is not even real time. So this is only for analytics use cases where you need to ship some sort of aggregating metrics once a day, some stuff like this. So it's a big deal and one more vendor and all of this organization you have to do

just for shipping some metrics back. So don't have to be. this is easy, right? This is pretty complex. And now... The other thing, sorry to interrupt with the other thing now that you're talking through that is really interesting about it, is that I said earlier, we'll just fix everything in the transformation layer. But in reality, actually, for anyone who's built these systems inside of companies, what actually happens in is a very pernicious problem is that the

logic lives in different places, right? So you may have logic and Salesforce that runs on load super common, right? I load a bunch of data. I run a bunch of Ajax to do some operation to apply some business logic, right? You may have logic that runs on the ETL pipeline on ingest into snowflake, and then you also may have logic that runs in the individual reverse ETL jobs that are coming out of it, right? And so what's the thing about that is you don't really have a single source of truth. I mean, maybe you have one giant table that sort of represents it and then you materialize things on top of that. But it is actually very difficult to ensure that all of the logic does actually live in the transformation layer in the data store. That is pretty rare, actually,

because different teams will, you know, whatever, that you set up a different use case. Oh, I need this data for this thing. Okay, well, who's going to go in and update the DBT model and check all the dependencies and all that sort of stuff? It's like, well, we don't really have time to do that. it's like, okay, well, we'll just do it in the reverse ETL pipeline, or we'll just write like something in, you know, in Salesforce, right? And so then your business logic starts to get spread all over the place. Absolutely. And so basically by grouping, you know, this ETL and reverse ETL vendor

and putting that in real time, actually, you decrease basically the number of tables from hundreds to just one, you know, because it's the same table that you read and write data from. You also simplify all of these, you know, complex transformation layers, which now all sit within your DBT or COALAS transformations. You know, Coales is a different, is another DBT sort of, you know, competitor. Very popular as well with some no-code features.

And so what I really see is that, you know, the architecture get completely simplified by this, you know, having like, by having loops, you actually end up just having by a bi-directional arrow. And a bi-directional arrow is all the same as two arrows in opposite directions, right? Because it's two tools that don't talk to each other. And also what people don't think is that if you have a conflict, you know, a data conflict which

happen, you know, same data at the same time or et cetera, if you're badly orchestrated, you know, like you're just going to swap values. And actually some, you know, your data will just not lose same anymore. It's going to revert values. You're going to swap values. It's going to be really complex to maintain. And now the CMO walks to you and say, why? You know, we texted this customer on a different segment.

We lost a deal and say, well, because actually there was a technical, we don't care about the technical issues. Like, yeah. You have to make this pipeline work. So if you work, if your position is at stake, you know, you would not use this kind of tools. And that's where, you know, like the robust tooling, like taxing, which really works to scale. Because scale also, like, is a very different impact, right? It makes, like, it's going to take ages.

It's going to take ages, you know, just to ship, you know, to make the pipeline run. And if it takes four hours to run from HapSpot to Snowflake and four hours from Snowflake to Hapot, It just means that, you know, it takes eight hours just to run a pipeline. And when you grow too much, it becomes impossible, right? Because it makes more than 24 hours. So that's where really the challenge is. So actually you have the problem of loops, the data types. You have the problem for authentication, managing two different vendors

with potentially two different people in the data team, which are responsible for each tool. You know, like it's really an exposure of leadership to bad decisions. And so, you know, people are not. not really responsible for choosing bad tooling and we see this like, you know, the industry is filled up with bad tools. Like, honest, like it's a lot of tools are crap.

But like, leadership is actually responsible for having bad business results. And this is just digging your whole by having the wrong tooling. So actually like the IT investment is actually an IT investment into your own leadership. So as a data leader or like even as a CEO or CFO, investing in the

right tooling is actually like ensuring your business performance will be driven by the right data at the right piece. So actually like your company will not be, you know, underwater with, you know, simple, you know, normal growth. So let's walk through your use case because I agree that, I mean, the loop has been a reliable, it's been a reliable architecture for analytical use cases. and will continue to be, right? I mean, it's actually wonderful in many ways for that, you know, and then reverse CTL sort of adds,

like, this sort of slight operational benefit to it for, like, analytical type stuff, right? Where you need to get some data point into some tool or whatever. So, I mean, it's not going to go anywhere, but I think we should talk about the operational use cases in, like, in how you do two-way sync. And so what I'd love to, what I'd love to do, is like let's pick two examples.

So one may be easier and then one harder. So for the easy one, let's talk about the one that, you know, I made the mistake on, which is HubSpot and Salesforce years ago. So let's just say I have, you know, Salesforce, I have HubSpot, I have Snowflake. And my use case is, I'll just make up a use case. You can tell me if it, you can tell me how close I am to what your customer's experience. But I have this use case where we're doing. doing lead intake on some website or app,

and those leads are coming into HubSpot where they get marketed to, they're sending emails, all that sort of stuff, right? And so the marketing team or the demand generation team is using HubSpot to do all of that. And then some subset of that, you know, of those need to make it into Salesforce, or let's say all of those need to make it into Salesforce,

course, but the sales team or whatever, you know, or the support team or whoever really only is going to focus on some subset of those based on some characteristics, you know, whatever those specific fields are, et cetera. Okay. So I could theoretically run the loop, but the problem is, let's say I'm a pretty big company. I'm running it, you know, I have these huge bash sizes. Daily is it quick enough? Because if one of those leads, has some sort of characteristics, or let's even say, like, perform some specific action.

There's a really particular form that they fill out or app that they sign up for or something like that, right? And so those things, those actually, let's say the customer support team, they have an SLA of 10 minutes in responding to some action, right? But HubSpots the point of intake.

So the loop's not going to work for me because that's actually not even analytical. It's purely just, you know, this needs to show up in the sales point, support hub and they have a 10-minute SLA. Okay. Is that, am I thinking correctly about the challenge that I would have as a data team of like, okay, well, how do I actually make that work?

Is that? Yes, this is actually a correct time. And I would even say like, even clearly, right? It's like, okay, so now you make a marketing campaign, you know, on hotspot and this person immediately logs in, you know, respond to the email. It's an absolutely, you know, sweet spot. This person signs up and actually now goes into Salesforce. If your pipeline didn't run yet,

it will be inserted in Salesforce by the sign up, as well as from the HubSpot to Salesforce pipeline. So basically now you're going to have a duplicate. So you have to make sure that your pipelines are so robust to absurds and not only update, right? And how do you do absurds? Well, you need to query the data first and write data second, right? So actually you need to first, so you need to first,

so you need to first basically get the data, basically get the data from HAPSpot to Salesforce in Obsc, right, which is a very big, I mean, inquiries the data, to actually know if the record is actually present into the Salesforce. And then if it's present, update or then insert, right? So this is two API calls. And so at scale, you know, this is extremely hard and doing this on batch, batches.

It's also complicated. It's very complex. So now you have to make, let's say, Most buy plans do query records like one by one, which is a very problematic because if you have matches of 10,000 or 100,000 contacts, which is the site of a marketing campaign, you're going to be, you're actually going to be like over, basically overconsuming your API at limit for the entire week, you know, on Salesforce. Yep. And hot thought is also limited, right, per second. So it's not going to work. Now, what is the, so this is challenging.

In terms of setup, it's complex setup and it's complex. it's complex also maintenance. Well, now with two-way sync, basically, Staxing tells you, okay, you create a two-way thing between Salesforce and Snowflick. You create a two-way thing between HubSpot and Snowflick, right?

So now you have a place in set. So now you just have two bi-directional arrows, right? So now it's very just a, it's not a triangle. You have a in-since, in Snowflake, you have all the contacts, companies, extra, all the tables of Salesforce and all the table of HubSpot. whenever, you know, because Staxing is real time, right,

is this very important because that was not real time, right? So it's even problematic. They would run on scheduled jobs, yep. Exactly. So because StaxSync is real time, let's say, a new contact is created on Hap's Bob. Yep. As well as it's graded, Staxync will ship it, right? It's sub-second or maybe one second, two-second latency.

It's going to go directly into your snowflake. Staxing, we also have a feature called triggers, which actually enabled you to say, when you observe a certain data event to be transferred also triggers this workflow or this database query. So you say when a contact is created or updated right into Snowflake, from HubSpot to Snowflake

and also create this query which tells you, because Staxing also runs a diffing, so it tells you, take the if email was updated, you know, and ID is a field that changed, actually it means it's a new record, write this record into the Salesforce table into my Snowflake. So now you made one simple SQL query

because the entire Salesforce data is also real time into a sync. It's fresh. So now you can do an absurd, right? And be pretty confident that this will not really duplicate. So with a simple absurd operation, you actually know exact.

And you can actually upset on many fields, which you cannot do on Salesforce because on Salesforce you might not even be able to query based on a given field. Right, right. In Snowflake, can do... Can you filter as well? So, like, I totally understand that example where you offload, because that's actually a very, that's a pretty efficient, that's a super simple query, right?

I mean, that's about as simple as they come. Absurd. Based on email or ID or whatever. Yeah, yeah. It's super easy. But could you also filter, right? So let's say I want to modify the filter going back to the use case. I want to modify the filter so that I can say,

even if this exists in Salesforce, I don't want to send it because it doesn't need some sort of qualification, right? Or, you know, I want to send it with a flat, whatever it is, whatever that sort of filtering is, right, so that I can kind of determine, like, when different types of things get sent. Absolutely. You can also make this filter, and you say, for example, let's say, when a contact is created and the segment is X, Y and Z, send it to Salesforce, maybe it's a very large company and send, maybe send it to the sales force of this company for, you know, this subcop, this child company which serves the large company's network.

And then I say, if it's a small company, send it to the sales force of this other company, you know, subsidiary, which serves. Oh, like literally a separate Salesforce instance. Exactly. Because now you can synchronize, you know, you know. Oh, right. So even if you had single marketing intake, you could send it to... Oh, interesting. Okay, yeah, I was thinking about a way too of a way in a far too linear way.

Wow. Exactly. And this is just a simple SQL query. So now what we did, there is no table transformation and everything. There is just like a query which is triggered at the right moment to maintain this real-time feeling across your system and a simple SQL query. Or even a DBD transformation which can run, right? You can also do batch once an hour, once a day as your current use case.

But now you know, this is really real-time. And now, you know, when you insert your record, so now I'm going to say, a record has been created in a half-spot. It has been synced to Snowflake. Stack Sync, you created a trigger which says, okay, when a contact having these properties, you know, is created an email as a field that changed, then also created into Salesforce. So into the Salesforce table in Snowflake. So now we run an absurd, which prevents the use of the emergence of duplicates into Snowflake. And with 2WSync, I just want to just send this, I mean, because of 2Sync, this data is going to be inserting into Salesforce.

Yep. And so now you just create a contact in HubSpod and you have it in Salesforce and it passed into your snowflake. So it's also available to all of your other systems and analytics and dashboards to actually be observed. So you have real time analytics or rational purposes, because now, you know, you can literally, because this takes maybe two to three seconds because every query time and all this. Maybe it takes two seconds or three seconds at maximum. So three seconds later, you created a contact in HubSpot and you have it in Salesforce and you can trigger like a welcome sequence or whatever. That's really operational.

And this entire pipeline has been built by one SQL query, one trigger with two to with things. And you're handling all the API calls. everything. And even batching. And so in Staxing case, even works in a very clever manner, is that if you go into your snowflake and you modify, let's say, one million records at a time, because Salesforce has a different rate limiting and HubSpot 2, Staxing will just, you know, batch all these records as fast as possible within your allowed API rate limits, which you can also configure and send this data. So it might take a bit more time. But, you know, for example, for Salesforce, we go up to 1 million records per minute, you know, alpha million records per minute on half spot,

So it can have very fast. And so all of this, you know, this architecture, and therefore so maintenance, is simplified with a single vendor. So one deal, which has just triggers, SQL queries, and DBT transformations. Yep. Nothing else, you know. Okay, so let's, I said we'll do easy mode and then let's do something harder. And maybe this isn't, maybe this isn't harder, but I'll try to, this is the example that came to mind.

Okay. let's move from Salesforce to NeftSuite ERP, any ERP, but whatever, let's just say net suite. Measured, yeah. Things get more complicated when you think about operational use cases where you have a billing department who's using the ERP to send invoices, you know, manage payments and receivables, right? But in Salesforce, let's say that the salesperson or the customer support person is working with an individual, right? You know, this contact, and the complication, you know, it can get crazy. But the complication is a lot of times you have to make multiple hops to go from the contact in Salesforce to a unique invoice ID or purchase order number, right? because you know, you have the contact, and then, you know, they have a company in Salesforce,

and it's not always a one-to-one match of what they're called in the different systems, right? Because in NetSuite, it's, you know, Acme LLC, and in Salesforce, it's Acme, whatever, right? And so those two names don't match up, and it can create problems. So I'm breezing over the complications because that's just a simple example, right? But here we're doing two-async between two systems. there are some discrepancies in the data, right? Like, at least with HubSpot, with a lead, you know, a person in HubSpot has an email address,

and a person in HubSpot has, or a person in Salesforce has an email address. And so you can fairly reliably use that key. You know, there are some applications there. But there are keys that you can use, right? But if you have to make a hop for that operational use case where there are, like, different keys, and then like a physical asset, like an invoice, you know, related to a company.

How would you handle a situation like that? Because there we get into some really interesting data modeling challenges and some data discrepancies. Yes, absolutely. So basically, like, if you have several hops, for example, say you have to query first, you have an email in Salesforce, but also in Nesuit,

so you have to query the contact by email, which might not even be possible. Then you have to get... Right, that might not be possible. So then you have to do company name and then... Exactly, exactly. So then you have to get the contact, then you have to get to a company,

and you have to get the opportunities, and you have to get, like, the list of all the invoices for opportunities and then get the ID, all that invoice, to actually get the bottom payment. So all of these are very complex workflow. So if you are in the illegal world,

you have to get maybe like a workflow with 15 steps and merges and... It's brutal. Yeah, it's very bad. And so what happens is that now if you have your entire data real-time fresh into your snowflake, you can craft your SQL query

to actually get very powerful, you know? So actually, like, you have, in your workflow, you would have one snowflake query. Then you have a check to check if it returns a result because maybe, like, you know, you are at a very, you know, millisecond point in time where, you know, data wasn't available or something. So make a check, right?

And then, like, once you have aggregated all the data, so with a single query, you know, you're not overloading your snowflakes, your net suite with many API calls, right? You are just querying Snowflake, which is much more, you know, load able, you know, to speak. Then you have your insight.

You take your data and you insert that into the right Salesforce or HubSpore or NetSuite table. And that creates an update into your NetSuit. So I agree that. So two ways it really makes it that your tables into your snowflake or into your Postgres even are really a real-time, you know, read and write interface to your enterprise. to your enterprise system data. It's an equivalent to using an API. Basically, you're just using an API via SQL. That's the only thing you're doing.

That's a very easy way to understand it. So actually, every time an insert is going to make, actually, you're making an API call. But actually, this API call can be one million rows. You know, this read can be filtered with any kind of business and custom logic that you have. It's really extremely, it's an API, which is as flexible as SQL,

and as well documented as SQL. SQL. And you can see your data directly. Yeah, yeah. No, I think that is the, yeah, Brooks is telling us we're at the buzzer here, but what a great, I mean, I think that's the paradigm shift. And it, you know, it took me a minute to get there, maybe because I'm a little slow, which is why I fell for the marketing type on, you know, Salesforce and HubSpot Sync. But I think that's actually the real magic is that most of most integration challenges that you try to solve, you're doing it by trying to stitch together. There's basically two ways. You try to stitch together APIs, or you run the loop, which isn't really great for operational use cases,

especially ones that are time sensitive, right? Yes. And then your logic has to grow as part of like a gigantic Frankenstein model that gets crazier and crazier over time, right? And so it's really interesting because I would almost describe staff. as inverting the problem, right? Where it's like, you literally don't think about APIs. You just write a SQL query for the use case that you want to solve. It's super fast because it's a really low, it's not a heavy query. And then you don't even worry about the APIs.

It just happens, right? You're just modifying, you know, you're adding these queries over time and modifying the logic is isolated. So it's like very easy for anyone to reason about, you know, what's going on with the, like, like Salesforce contact to invoice, pass, do whatever use case. Super cool. Okay. I'm going to squeeze in one.

Oh, sorry, go ahead. Yeah. I'll just like it's a very visible logic, which is easy to build and to market and to debug, right? And especially it's a very declarative way to operate, right? API is very event driven. Do this, do that. And SQL is very declarative, right? It's like, you know, just of everything, every data you can see, you know, from a top level perspective,

just like pump everything you have. You need and get it back, you know. And this is very declarative. and which really enables you to be much more robust pipelines. So that's where we stack simply puts this declarative and SQL way to operate on top of what traditional event-driven dirty APIs, right? But also like, just from a simplification perspective, just try to connect to the NetSuite API.

And in a week, we're going to be still there. And I'm going to say, okay, well, maybe it's useful, you know, to have something that manages to me. That is a great, that my friend, is a great sales pitch. Yeah, just not talking about like the integration piece, right? Just like the authentication, right? Oh, yeah, that's brutal. Documentation, you know, like, I mean, like, I know a few survivors back in the 90s,

which actually understood how the API worked, you know? Yeah. So it's a bit like the statement, you know. Yeah, yeah. Okay, one more question here. Can you give our listeners a sneak peek? What feature are you working on that you're most excited? about. So right now, basically, at StackSync, we are making, basically, we're also launching

a workflow automation tooling, which actually plugs in into the sinks. So for example, say you have a two-way sync, you know, some data events are transferred in real-time. You can trigger, actually you can say once, when you see this transferings, you know, when you see a new contact, tell me, you know, and this tell me can be anything between a sequence of Dmitter transformations can be a workflow automation, can be a Slack notification to your sales team, can be anything, you know. And this is really something,

or say a contact change status, boom, notification to the relevant sales rep. And all of this kind of enrichment. I was holding a webinar last week about like, how can you actually say, every time there is a new contact created, go to LinkedIn, you know, get real-time live data

about the entire LinkedIn profile, make a summary, and fill up all of this, you know, database fields, which are actually like the CRM fields. And now every time I would just type an email into my CRM, I would see all of this field populating immediately, instantly. And so this is really with live LinkedIn data. And this is all these enrichment use cases are exactly what we're building.

So really, this upgrade into enterprise scale mode of your operations. This is what Staxink actually does. And Staxi now has become also the leader into the Netsuit toolysink. So if you have any struggle into your team, team, which has NetSuite, Shopify, Zendesk, have fun, Salesforce, and involved, you know, happy to chat and actually help you architect your best use case in your precise, you know, business scenario.

Cool. Rubin, this has been awesome. I really appreciate the time. Love that we dug in. Love that we demystified zero whatever it is. Zero something. I guess zero something is an oxymoron. But this has been great.

Congrats on all the success and we hope you have much more of the future. Thank you so much for hosting. The Datastack show is brought to you by Rudderstack. Learn more at rudderstack.