Tech Brew Ride Home - (Bonus) Building An AI Product With Uptech

Episode Date: September 30, 2023

Check out what we came up with at ResumeWriting.com. And check out Uptech.team. They can help you with any project! Learn more about your ad choices. Visit megaphone.fm/adchoices

On April 4th, 2023, around 2 in the morning, a man was found stabbed multiple times on a sidewalk in downtown San Francisco. Hey, who did this to you? What happened next turned the story into a political firestorm. Reports have identified the victim as Bob Lee, the founder of Cash App. From Bloomberg Podcasts, this is Foundering, the Killing of Bob Lee, beginning April 16. Welcome to a sort of bonus conversation for the TechMeme Ride Home. You all have heard me talk about this AI experiment that I was working on all summer. And finally, as you heard earlier this week, we took the wrapping off the beta test.

There's real customers playing with the tool now. Again, you can check it out at resumewriting.com. You also have heard sort of the whole saga of trying to bring this to life and how we hit sort of a roadblock halfway through the summer. Today, we're talking to one of the team members of the company that helped me overcome that roadblock. We're talking to Andrew Boss of uptech.tec.team. Hey, Andrew.

Hey, Brian. Nice to see you. Well, first of all, you know what I want to do? Because I want to thank everyone at Uptech for bringing this to life. So as I expressed, we got the project so far. But the original developer that I was working on had a job, got a job, and didn't have the time to fill in all the details. I said on the show on Monday that I'm so impressed with uptech. Like you all were amazing to work with. Literally, you just took the baton and ran with it.

I didn't have to even describe very much. I was so amazed that your ability to sort of anticipate all of the sort of rough edges. and things that we needed to sand away. So before we get into working on this project, you can find out more about uptech at uptech.tec. But in your own words, tell me what you all do at uptech. Thanks. Thanks for the kind of words, Brian. It was a pleasure working together, and it's been really fun.

At UpTech, we are a world-winning design and development product studio, originally from Ukraine. We are north of 100 team members, now spread, of course, remote all over the world and working with clients primarily from the US in Europe on such brands that you might heard, like Aspiration.com, dollar-shape.club or vote and a lot of others. Right. And let me stress that, like, you guys are full stack,

full everything. And, you know, you're listeners, so you were kind enough to sort of answer the bad signal. And when I said, you know, we need to bring this to a. conclusion, you stepped up, but you were able to do it all. And I mean, I want to stress this to anybody listening, please, if you've got a project of any sort, like from my wild crazy AI resume idea to like he said, a dollar shave club, I was just so thrilled with uptech could do everything from the design to the front end, back end, everything. And I didn't know if you'd be able to handle AI stuff. So I'm just, I want to stress again, I was so impressed with

ability to basically do everything. Yeah, thanks again. Glad it was noticed. It's basically, to be honest, what I've been doing in the last 10 years, and that's what we exactly focus on at UpTech to take any projects from idea stage to successfully release and ongoing support and grows. Me, myself, I love starting new projects, so I literally indulged my own wishes in working on this project. I started and run a couple of companies before and now running like UpTech, the biggest one, a design and development studio, as well as NoCode Studio called Somo. We have a couple of

projects in the HR sphere in people in performance management. For example, Ply that we recently exited a bunch of products in the AI sphere, a group of products called Divo. It's like miracle in Ukrainian that does AI avatars, e-commerce product photos, professional headshots, and a bunch of other stuff. So I really love starting new projects and this kind of person who likes taking the project from zero to one from the initial idea, researching, doing customer interviews, prototyping, bring it all to life, releasing and iterating. Again, Uptech. dot team, U-P-T-E-C-H. Team. We're going to talk about AI stuff, but literally any product, any sort of, like, they

can do it all. I'm just so impressed. But so this was an AI sort of experiment, a sort of tool. Are you all, when you're working with clients, are you seeing a lot of that now people looking to implement AI stuff into whatever it is, whatever product it might be? Are you seeing a lot of adoption of AI stuff right now? Yeah, of course. Like literally every new project in some way or another talks about AI

and discusses ways to implement some sort of AI features in their projects. Typically, it's not super low-level or super hard, like a couple of generative AI requests to open AI APIs or similar. But sometimes it's much more low-level stuff where we need to gather our own datasets, design different kind of NLP or other models, train them, measure their temperature output, and fine-tune where needed. For example, in our own products, we added AI on almost every one of them. Like, it answers our support tickets, for example, for our tool called natively, that wraps

websites into the mobile applications. It's a lot, a lot of documentation and details on how to do it correctly because of different Apple and Google policies. And we were like literally drawing in support requests from users like how to do that, how to do this, and now plug in and training model on our content and using it to respond to users. It's it lifted like I don't know 50% of work for the for the whole project. Yeah, we were just talking before this call, actually, about implementing some sort of like an AI bot for customer service purposes, for people trying out the resume project and getting stuck and having questions

and having the AI pop up and answer those questions. So, like, was this something, you know, that there was a learning curve when you guys have to start to incorporate this stuff in what you're doing? Or is it, are the APIs, my sense of this, again, not as a developer or whatever, but it's sort of like the APIs, it's just plug and play like anything else,

like any database or any other tool that you integrate into what you're developing. And as long as you can do those API calls, on the lowest level, it's not super complicated other than plugging it in. Is that sort of the lay of the land right now in terms of, especially working with Open AI? Yeah, especially. You're totally right.

Especially working in the OpenEI. You can start super fast and very easy just plug and play and you get simple requests respond and you can plug it into almost any sort of project to help you do summarization some suggestions etc but of course the devil is in the details and with this kind of approach you can do fairly limited and simple sort of functionality if you need to do a little bit more advanced stuff you need to do much more work and have much more understanding how it works and to be able to leverage like temperature, frequency penalties, able to convert your data sets into vector DB and use it, plug it into the existing models,

etc. So yeah, it's great that you can start like literally in a couple of hours, but then to get it actually working and working nicely, then you need to spend a little bit more time. For example, on the EI resume, to get you a quick summary of your resume, it's literally five minutes of work. We do one request. But in our case, we needed to do it in a structured way to make it output a well-formatted Gson with a bunch of keys and a lot of data at the end that went out of the top and the default token limit of GTP4 and even the GPD 3.5 turbo.

16k limits so that we had to be much more creative in terms of what kind of prompt we write, how we split it into separate chunks, and what kind of input information goes into each request so that it both has the context in order to give the response, give it in the style and the level of details that we wanted, as well as preferably hallucinate a few as Right. Yeah. The thing that I found, a couple observations from my end, you know, one of the reasons why I did this was to learn on a fundamental level what this is like to create products using this tech. What you described is like when Chris and I talk about like AI varietals and like it's, like you said, you can get started really easily, but then to get the quality on the back end, the output, the way you want it.

That's where it's sort of like an art, not a science. It is sort of like massaging the prompts and things like that. Now, you were describing that we were running into rate limits and things like that with the Open AI API. So we were almost by necessity having to get more creative and more efficient. But the same can be said, like you were saying there at the end, for it's all about massaging the AI to make it more accurate or to get the results in a more formatted way. or something like that. So is that your sense, too, that there's no real hard and fast rules for how this, it is sort of like seeing what works, massaging it until you get the outputs that you want? Yeah, absolutely right. This is both what I allow and hate about working with AI. It's more art than a science,

and as you well put it, it's like massaging. We had to do lots of, lots of iterations of prompts to get it work for AI resume, for example, and it's good that we didn't yet have to fine tune the base model, because now you can fine tune GTP4 and it works quite well. And when we were working on the projects, when we did have to fine tune the base model, for example, like AI avatars or other products, it's like you have 15, 20 different parameters that influences the fine tune. You change one of them, start the fine tune. two hours later you get the fine tune model and then you start like 10 20 prompts to see how it works

and then you change another parameter and repeat the same again like in a couple of hours you get the output and you repeat it for like dozens and dozens of time i was at once at one time i was thinking about quitting this old field altogether and not getting getting back to it so that's what you have to be ready for when you're working on some level that's all sort of software developing is like you think you know if you if you write the code this way you think you know what the output will be the end result and it's not always perfect but that's what the refining is but it's almost like it's it's several degrees beyond that because like you're describing you refine the model and you don't

necessarily have any idea what that'll do until then you can actually test it like that I can see that that's like a a sort of a radical shift in like how you work. If you think you know what the stuff will do, but you really have no idea until like you've massaged it, like that can either be frustrating or like amazing or interesting, I don't know. Yeah, you sometimes just have no idea and it's like you're hoping it will work well.

And another paradigm shift that we had to accept in working and developing AI products is that before you get, for example, like cover with tests and be like 99.9% sure the program behaves well in the use cases that you designed for with the AI you can never be sure than more than 90% or even 7080 that it will behave well you have to accept that in certain cases it will hallucinate it will give wrong responses strange responses no no responses just I'll try break and you have to incorporate your product and messaging to your users and the feedback loop with this in mind so that you first warn your users that it might happen so that they are not disappointed and don't start immediately

asking for refunds or writing bad feedback. When it does it like that, you need to give users a way to redo the resume, the prompt, whatever they were doing for and provide lots of feedback loops so that they can give you a response of what work well and not, so you can incorporate this feedback into further fine-tuning and refining. Right. And let me, let me, let's, I'm going to use, I'm going to use what we have done as a tangible example. So, you know, listeners, if you check it out and fiddle around at resumewriting.com, the idea is

that once you feed the AI your career history, going forward, what you would do is cut and paste in a job opening. And then the AI would tweak what we call your foundation resume, let's call it 15% or 20%, to incorporate some of the keywords and some of the expectations that that job opening would present. So that if you've created the foundation resume that's based on your career history, but you are applying for a job that has XYZ requirements, it would tweak your resume slightly to rewrite the language of, okay, here is a resume. that would be targeting, more specifically targeting this job opening. So what you just said about the hallucinations, it's never going to be perfect in the sense that we knew that like it would say something like it would invent, let's say you're applying to a job that has a, you know, a job requirement that maybe you hadn't done before. Well, the AI would change your resume to say that

you did it. Well, we know that that's not the, what we built it, we built it, to the product was that people would have to assume, all right, what the AI has done has retargeted your resume. It's up to you to refine it. So that's such an interesting thing that you're saying is that we, from day one, assumed that it wasn't going to be perfect. Because in a sense, how could it be? Right? It doesn't know necessarily what you have or haven't done. It only knows what you have done because we fed that into the AI. So this concept of when we were thinking about, this as a product, it's doing something that wasn't possible before, which is, it is with a push of a button in like 30 seconds, rewriting your resume to retarget it to a specific job, which in the

past, in real life, would take you at least a half an hour to do or like, you know, tweaking and editing and refining. So this eliminates, you know, a lot of the busy work step. It's still not perfect. It still requires the human to sort of guide the AI, number one. But then also, so correct the AI's mistakes. So that was sort of interesting to me that we sort of knew that from the beginning that we can't guarantee you this will be perfect. It'll get you 90% of the way there and save you time. So that's at least for certain things, that's where we are now with this technology, which

is if it's something that can save the user time and get them 90% of the way there, if that's enough of a trade-off, if the time saving is worth the 90%, so that you have to still go back and edit the 10%, refine the 10%, that's still a decent trade-off. You just have to, as you're saying, be intelligent about preparing the users for that, which is like, this is not a 100% solution, but this is a shortcut solution. And so you have to think about that in terms of product design, in terms of the user expectation, and in terms of like helping them help you refine it to get to that 100%. Yep, yep, exactly.

And this, for example, in case with AI resume, with resumewriting.com, we output the final resume, not in PDF format, but in the world document, so that you can go and fix this last 10, 15% that you would like to tweak, remove with hallucination or be more specific in terms of what you're experienced. but the core, the foundation of your resume is already there. You have the formatting, you have the well-written, professionally written summary of your resume, your responsibilities, your achievements, and your career goals. Right.

And so I'm using us as an example because if you think of the product as that, where you can prepare the users for, there's still going to be a little. little handholding that they need to do. But if they get enough of a, again, shortcut out of it, I think that there can be value there. So what we built in was if it were 100% perfect, we'd deliver you a PDF, right? But because we know that you're going to need to tweak it a little bit, then the way the product was designed was, okay, cut and paste in nine different jobs. Here are nine different resumes. Download them immediately. And then when you're ready to apply, go in, spend five more minutes tweaking it to keep the wording that the AI did for you,

because it wrote it more professionally than probably you would have. But it also maybe lied about what you did trying to sell you. And so it's incumbent upon you to keep the instinct of the AI to sell you, but make the selling accurate. So, again, I'm thinking of anybody listening out there that is thinking of designing with a similar AI product. We tried to make it easy not only to then do the extra tweaking to get 10% of the way there, but prepare the customer to be like, this is the process. The process is shortcut and then we'll make it easy for you to tweak the AI till it's perfect. Yep, yep, exactly.

That's definitely we need to be aware and prepare your users and communicate it well, because users definitely won't like the bad results at the end. Another approach that we did in some other of our products is to just write from the beginning, generate and produce not one, but four, 10 results or examples so that users can choose. Select the best one, yeah. Yeah, select the best one and just trash all the others. And this is it. Right. I think, I think, yeah, the message that Andrew and I are trying to convey here is that you as a developer of the product,

shouldn't expect the AI to be perfect. And so you need to bake into the product, that expectation as well for the user. But if you can shave down the rough edges and show people that you're, again, it's something that wasn't possible before. Like, you can sit down at Resumaywriting.com and generate 15 resumes and a half an hour for 15 different jobs. Like, that would have been hours and hours and hours of work before.

So I think that the big lesson, the big takeaway for anyone listening, is you need to assume in your development that it's not going to be perfect, but you need to build that imperfection into the product in such a way that the user feels that they're empowered, because that's where we are with this tech right now. It's never going to, it's not perfect. You can't expect it to generate the ideal document. But you can't, if you build those constraints into the product in an intelligent and empathetic way, then again, you can show the user that there's, there's real benefits to it. Let me ask you one more thing. And to the degree, obviously, we're not going to share sort of our secret sauce of the prompting that we were doing. I still find it fascinating that even on the developer side, we're literally prompting it by saying things like, imagine you're a professional resume writer and you're writing

a professional resume for a nurse. Now, again, we're not going to share the other ways that we tweaked it to be a little more sophisticated from that. But on a base level, that's still what we're doing is we're telling the AI to imagine this task. Imagine your resume writer. There's not anything technical about like, obviously when you get deeper into the weeds of like changing temperatures and things like that.

But on a fundamental level, what we're doing for this product is asking the AI to pretend to be a professional resume writer. I just find that so fascinating, again, that there's, it's more art than science. It's literally asking the AI to play pretend in a way. Yeah, it's kind of fascinating. It's almost like talking with the person who has feelings and like if you ask it kindly and gently, it will be more responsive and better. I guess it's just helping in the massive training data set that it was trained on,

it's just giving you the keys or the vectors in what direction you should. should go to so that it outputs the completions. Right. And it can be things like, OK, say this, but more professional sounding. Say this, but more efficient. Sometimes like you can get. Concise. Right, concise, right.

Where it goes on too much. And so then it's like, OK, still be professional, but a little more concise, you know. And it's. Yeah, or don't repeat the company name in every sentence right, right. But then like so, OK, I, you know, when we were testing this and I would put in my own career, and I would say,

I would literally be like, OK, podcast producer, right? And so my last three jobs, whatever. But it would do things like if you know the podcast industry, there are companies like Pineapple Street and obviously, who's the company that did cereal? And anyway, when it would write the job history, it knew enough to know. It would like say, OK, he worked at Pineapple Street

And he did, you know, he created daily new shows. Like, it knew about ride home media. It knew about Pineapple Street. And so, again, that was like the hallucination. Well, I never actually worked at Pineapple Street. But that's the sort of massaging that it can do. Like, it would come up with things that I was like, how does it know that?

Right? So I just find it so fascinating that it is sort of still a black box where you're sort of pushing the levers, but you can't see what the levers are actually doing. So it's all about sort of like playing the piano, but not being able to see the cables on the inside of the piano, you know? Yeah. But it's still super impressive. Like, the quality of the results, the esume, like, at the end of all the prompting, as you mentioned, me and all my friends were a stunt of the quality

and the professionalism. And that's why it actually works because, so again, using our thing as an example, when you cut and paste in a new job opening, we were like, well, what is it going to do? Just grab some of the keywords from the job opening? No, it fundamentally rewrites your foundation resume to make it more targeted towards what that job is asking for. So it wasn't, it's not just a fancy copy and paste that does it for you. It's like, okay, yeah, here's your career, but what this job is asking for is, is, you might be a nurse, but what this job is asking for is a pediatric nurse, right? And so without you having to tell it, it would rewrite what your career history is like,

to make it more targeted towards, it would say things like, well, have worked in OR situations before, which might be applicable to, you know. And so it would know intuitively to refine it in a way that maybe if you're not a great writer, you wouldn't think of. And then so all it was was your job to go to do. through and be like, okay, I see how it's targeting in this direction. So all I've got to do is keep sort of the direction that it has, keep the wording that it has, but replace it with the actual stuff that is accurate, right?

And by the way, saying that it's hallucinating, it's not like it hallucinates 100%, it hallucinates here and there. So it was just so fascinating how it fundamentally reworked things based on the job posting you would put in in a way that was like we're saying, like 90% would you needed and then it's just incumbent upon you to go in and tweak it a little bit. Yeah, yeah, exactly. And the way it picked from the focus of the job opening, what will be your areas of responsibility, and from your history, pick the same areas of responsibility and highlight them instead of like others that you highlighted for other job posts and not only like areas of responsibility,

but as well as your previous experience, your previous education, your, your, tools that you worked with and the technologists and just to reward it and emphasize the one that would likely be noticed and matched with the hiring manager. Right. Like, let's say, here's another example. And again, you can try this out for yourself to see it. Like, let's say you're in sales and you're going into pharmaceutical sales targeting doctor's offices, but you only have previous sales experience selling used cars or something

like that. Like, what it would do is it would show. you like, okay, but to sell to doctors' offices, sell to pharmacies, you would want to highlight that you have experience selling to enterprise, not just consumer. You see what I'm saying? It would tweak in a way where it would sort of guide you in the direction of, okay, you want to, we're going to use your experience, but we're going to sort of rework it in a way that makes sense for the fact that you're targeting pharmaceutical sales, even though you come

from use car sales, right? And so like, that's such a big thing. again, having worked in the resume writing industry for 20 years, like, you know, especially for like ex-military people, you know, rewriting a resume using military skills, but then making them applicable for, what's the opposite of military? Civilian.

For civilian experiences and things like that. So like that was the part that was so magical is it's like, it knows to do that for you. And so it gets you there. and you can do it over and over and over and over and over again. And then just when it's like, all right, it's time to apply, I need to spend five minutes tweaking this and boom, I can go. So anyway.

Yeah, and just two years ago, it was like unimaginable. Right. It's amazing where we come through in two years. My only regret is that it feels like for me that even though it's already feels like magic, this last 10, 20%, I would say it's probably not, going to be fixed in the nearest couple of years, or maybe even a couple of dozens of years. It's like with the AI, with self-driving cars.

They were almost working for decades. But this lost few percent are still not there. That's funny. I've thought about that so many times, too, in terms of, will that be a similar thing for this generative stuff? But at the same time, because of rate limits, we had to pull back to 3.5 turbo, from GPP 4.

And the difference between GPD 4 and even 3.5 turbo was noticeable enough that if we could afford it, I'd go back to 4 tomorrow. So it makes me think, like, OK, but what will 5 be like? Do you know what I mean? Will 5 take us from the 90% of the way there to 95% or will it jump from 90% to 99%? Because like I said, the fact that you can notice

the quality difference from 3.5 turbo to 5% to 5%? suggest to me that their step changes are possible. You can get exponential with this stuff. But, I mean, we'll find out. Time will tell on that, I guess. Yeah, we'll find out. Well, you'll upgrade. All our users will see.

Yeah. So, Andrew, again, I want to say again that, you know, thank you for being listeners and hearing that, you know, we had gotten sort of stuck. I cannot stress enough that I was blown away by your ability to unstuck the project and intuitively know sort of the things that like, oh yeah, we probably need a dashboard here. You all were so smart about that because clearly your experience, but you're also talented

at this. So I want to stress again, any product, project website or whatever, like if you're a listener to this show, UpTech is our official partner for anybody that wants to do a project, please get in touch with them at uptech. We're getting touch with me and I'll put you in touch with them. Again, I'm just singing your praises. I love uptech. So anything else you want to say about your team?

It was a pleasure working together. I just want to mention that we work not only with stock projects. We work with other project as well. If you're just a brand new projects, feel free to come to us. If you're dreaming something up, again, they're so intuitive. Like, go just have a conversation with them and whiteboard with them. and they'll help you figure out if you're at the earliest stages. If you just have an idea, they're amazing.

Yeah, we'll go ahead, take you from the idea, help you figure out the feature set for who you're focusing your project, what should be included in the first version, what should we postpone, help you prototype it, and then develop and obviously launch and scale. We've been doing it for 10 years. I love it. We designed the whole project, the whole team, around this idea of being able to assemble the product team,

get into the context, talk to users, become like through owners of the idea and the product and get it to market as fast as possible and as efficient as possible. Right, that's one more key point. I'm describing them helping me get it to market, but what we're in the process of right now is, you know, we've been testing it, so now we're refining things.

And so, you know, they're very good at that as well, like helping us figure out where their bottlenecks are for user adoption or where users might have problems and tweaking the product to make it as smooth as possible. They can do it all. Yeah, so we did the first 90% of the product now we're in the process of completing the 90%. It keeps, that's the theme of this conversation is getting that last 10%. So anyway, again, anyone listening, UpTech is our official partner of the podcast. at this point because they're so amazing. Any project that you need help on,

get in touch with them at uptech. Dot team and see what they've done at resumewriting.com. Check it out. And I hope that this has been as interesting to learn about sort of the nuts and bolts of how many times at the beginning of this year did I say, is there a real business there? And I've learned so much in terms of what it takes

to create a real business and a real product around this tech. So Andrew, again, thank you for helping me run this experiment and uptech. That team, right? Yep. Pleasure working together and looking forward to working on more projects. Yep. Thank you much.

Thank you.