Programming Throwdown - CUDA and OpenCL

Episode Date: September 4, 2013

This show covers CUDA and OpenCL, languages targeting the Graphics Processing Unit (GPU). Tools of the show: NES/SNES Together https://play.google.com/store/apps/details?id=com.github.mistertea.android.emu.nes https://play.google.com/store/apps/details?id=com.github.mistertea.android.emu.snes, JODA-Time http://www.joda.org/joda-time/. Books of the show: Understanding Computational Bayesian Statistics http://amzn.to/1cGrjEX and Going Postal http://amzn.to/13egaIw ★ Support this podcast on Patreon ★

Hosting provided by Host Tornado. They offer website hosting packages, dedicated servers, and VPS solutions. HostT.net. Programming Throwdown, episode 29. CUDA and OpenCL. Take it away, Patrick. So I'm a little under the weather. So if I sound stuffy or terrible, I'm gonna

blame that. Yeah, I dragged him kicking and screaming into the Programming Throwdown studio to record this episode. The studio? Wait. The studio. Am I taking too much medicine? Alright, so I do have an opening topic, and then maybe I'll lose my voice already, and

be quiet for the rest of the episode. Hey, stop cheering. All right. So I link a website here in the show notes, origamitesolations.com. But more specifically, my mom was in town to visit, and she likes doing puzzles or whatever. I don't know how I got on it. My mom likes doing – maybe it's a mom thing. Maybe it's a connection with moms and puzzles.

I mean, I think it's – I like puzzles as well. I just don't have patience for them or that's what it is a connection with like like you know for example like like my parents are close to retirement my dad's already retired so they have plenty of time that we don't have because we're uh working kids all that stuff yeah so i think we could probably be puzzle masters in 40 years and how many 30 years 40 years 10 early retirement oh all right it's a puzzle if you can solve that puzzle let me know yeah yeah uh i'll sell a book yeah that's right um so anyways so so she was out to visit us and uh we somehow i don't remember i think i had some origami paper i had found at a store or whatever and i was like oh off and on throughout you know childhood i had tried doing this amazing art of

folding paper and never been very good at it and recently i found some and tried it and decided i was going to search on youtube and surprisingly enough youtube is an excellent teacher for origami right sometimes now sometimes people do stuff they're like oh you just do this and then they like put their hands in front of the paper as they do some crazy trick awesome voodoo stuff. And then it's just as puzzling. But for, like, straightforward things and for the most part, rather than looking at the diagram, you know, it's a really useful thing. So my mom and I started doing some origami in the evening as, like, something to do as opposed to just sitting and watching TV or whatever. And she was really, really excited about it and really excited to watch youtube videos um and see like oh this is

really helpful like i never knew how to do this before or whatever so we folded some animals and some complicated shapes and some simple shapes it was a lot of fun but this website origami tessellations.com i had come across probably a couple years ago and the guy here i believe his first name is eric i won't try to pronounce his last name uh he he does uh tessellation origami which is just like you would think tessellation is like the same shape triangles triangles or squares oh that's right it's shifted around and of course once you have triangles you can do hexagons and so it's basically you first fold the paper into pre-creasing it with like let's say a bunch of squares then you do a set of maneuvers probably not using the right word anyways to fold those into something else or like

also a repeated pattern or a new shape and i just find it really interesting because normal origami is cool but sometimes you know it's like how many different ways can you fold a penguin well they don't look that much like a penguin or whatever. But this is more like artistic and abstract in a way because you're not making specifically something that looks like an animal. You could be making something that is just beautiful for the repetition of it. So I folded one. Oh, yeah, I'm looking at some of these pictures now. It looks pretty awesome.

Yeah, so I folded one. So it takes a long time to get the paper precreased. But once you do, like I folded this one, it was like a series of repeating overlapping squares. Can you precrease like 100 sheets at 100 sheets i mean like 20 sheets at once i tried yeah or like doing multiple like once you fold it in half doing the rest and essentially doubling it yeah you can do it to some extent but you end up building up an error because when the paper bends right like it bends around a radius that's increasing and it causes problems. And you do learn precision doing this.

And I fold this overlapping square thing. And it was beautiful to look at initially. And then I like held it up to a light and actually see because some places only one layer. Some places are three or four layers. Some places two layers. And it produces like a grayscale picture. I don't know if you've ever seen like a lithograph. Is that what it's called?

That sounds familiar. Lithopane. Something like that where they take wood and carve a negative of an image in it. And when you hold it up to light, you see the picture. I did that one year with a pumpkin. It took forever, but it was awesome. That's impressive. Yeah, I actually had a shape where at no point did I puncture all the way through the pumpkin.

But it was just like a different depth. So it came out pretty cool. Yeah. Yeah, so I did this with origami. And it was mathematical, right? T. So it came out pretty cool. Yeah. Yeah. So I did this with origami and it was mathematical, right? Tessellations are, are very,

are very mathematical in nature. You can kind of apply that there. And even just in general, there's a lot of work, which I, a guy, I think his name is Robert Lang has done work on like crease patterns. So what happens when you fold the shape and then you unfold it and you look

at the creases or whatever, and how do you describe that when like treatments of it? And can you find more optimal ways of folding or crease patterns and that kind of stuff it's just uh kind of the nerdy way of uh that thing you remember doing when so many years ago of like folding the paper and making the pirate hat that's pretty awesome okay on to the news so first news article html5 this is uh html5 is pretty much ubiquitous now i mean you'll hear the oh not supported in ie6 or something like that but people are giving up on that um and that might not even be true actually it's supported on a ton of browsers and it's got a bunch of really cool stuff in it there's some things that didn't quite make the spec and it's

still kind of evolving um one really cool thing that's pretty subtle is webrtc have you seen this i heard it but i don't know what it is yeah it's actually it's like a thing in javascript where on your website you can have a video chat like between or among the people like that are visiting your website and so the i think html5s is a link to this. There's a game, a 3D game of Pong, where the paddle is big enough where it has a face on the paddle, and you're actually playing your face against the other person's face. So it's like some kind of video conference where the ball's hitting you in the nose every time.

Yeah, it's pretty awesome. So yeah, HTML5 Rocks is a website that is sort of like an amalgamation of all the cool things you can do in html5 and all the updates to the standard so many of these things like webrtc you know when you hear html5 your first reaction is oh it's like new types of tags or like new dot HTML structure, right? And that's also true. But along with it comes a whole suite of JavaScript libraries that let you do just really cool stuff.

And, yeah, there's a link to check it out. Yeah, so I keep meaning to have – I keep bookmarking all these cool sites and things I run across in other news articles and do that depth-first search or breadth-first search. Either way I go, I just end up with all the stuff that I need to read and learn. And yeah, this is on my list. Cool, cool.

So the next thing is Nissan announced that by the year 2020, good vision, I guess, they're going to release a car that drives by itself. So Google is infamous or famous for doing this and working on their self-driving car and driving all around california and other places and now nissan has said that they are going to build like a test facility and they're going to have multiple affordable cars that drive themselves by 2020 and i mean this is something i get very excited about i remember for you know even growing up and hearing about this, they were going to embed pucks in the road and have cars that, you know, drove themselves.

And it just never happened. And you needed a lot of adoption. Like, everybody needed to switch over at once. This was a massive problem. And how would you ever do it? It always involved some ridiculous infrastructure. As you mentioned, like, put a puck every foot of all highways in the U.S. Yeah, it just seemed like

it would never take off and so i mean nissan announcing it google working on it like i mean these things give me hope that like maybe by the time my kids learn to drive they won't have to learn yeah that's right so it's pretty it's pretty amazing that i mean that's one of those things that's really going to sort of revolutionize the way we live i mean one of those things that's really going to sort of revolutionize the way we live i mean one of the things that's really done it for me is along the same lines is just a smartphone and the fact that you can just look anything up at any time and there's really no arguing over the other day we were arguing over stephen hawking whether i thought he died from parkinson's or he had parkinson's he's not dead yet but a lot of people thought he had lou garrick's

disease and it turns out they were right but like we saw this immediately by looking it up on the phone and that's one of these things where it's like over let's say I guess 10 or 20 years you know something has come about which has like completely changed our lives and the way we the way we you know the way we work the way we think this is one of those things where a self driving car I mean we'll be able to play board games on the on the drive to work right okay if you might be willing to take a job that's an hour away because you don't have to drive yourself right i mean it's just like lives i mean taxi services will be totally different right yeah i think that's the big thing we don't anticipate i think the short-term vision is

like oh good when i get on the highway i can go to sleep right and okay actually in the short term probably not because most of the laws seem to say you have to have a driver who's licensed and ready to take over you know basically there at the driver seat and ready yeah but eventually you know that'll adapt but i mean i think the huge impacts will be like how much safer the roads are right like i don't know what happened the first time somebody gets sued but um until they're like i mean i think it there's drastic room for improvement and reducing the number of fatalities that happen on the road right and then additionally like you said like you know further like taxi service right like why would you need to own a car that you

don't use most of the time anymore if your car a car could just be there when you're ready to take it right take you where you want to go and it's like a taxi but no taxi driver and you kind of remove a lot of the stigmas or even like what if a hoa you know your homeowners association owned like three cars or whatever and that was part of your community right or i mean i don't even know but there's all these different things you could think about or even just driving you to the door and then going home right like it's your car but it takes you to work and then goes home yeah or even like a you know so like there's no parking problem anymore or some weird parking structures where your car is like the horse,

where you whistle and the horse comes and gets you up. It's like that. Yeah, or even for truck drivers, they could do something completely different while they're on the road. They could be surfing the internet or something. I don't know, but it's exciting. I'm glad to see more people getting involved in it, and I hope it'll work out.

It hasn't worked out that great for electric cars thus far. Like, I mean, I think we're getting better, but... Really? I think electric cars are doing really well, no? Yeah, but I think, like, in my opinion, I'm not a car enthusiast, but, like, I mean, Nissan produced the Leaf, which was a failure. Right, right. And it kind of seems like maybe they regretted it wasn't tesla's doing well yeah but like other car manufacturers haven't like gotten excited and also actually made legitimately good electric cars one interesting thing i was reading an article on quora and the the question in quora was people don't know quora is a social kind of networking

type stack overflow for non-programming questions specifically that's a great way to explain it that's exactly what it is so someone asked the question what is the thing about your job that's like pretty common knowledge but to the rest of the world would blow their mind and the number one answer was that batteries and your storing energy are following moore's law which had never happened in the past it's just kind of over the past 20-30 years they've been following Moore's law. For people who don't know what Moore's law means is for the same price you can do twice as much every 18 months. So in other words for a long time processors were following Moore's law. So if you had say a 100 megahertz processor was $100

18 months from now you could be pretty certain, a 100 megahertz processor was $100, 18 months from now, you could be pretty certain that a 200 megahertz processor would be the same amount, $100, whatever I said. So batteries are doing this now. So what that means is, you know, effectively, however much of the Tesla car is the battery, let's say that's half the cost. I don't know what it is. Let's just pretend. That is going to go down in half.

Like, you know, in in three years it's going to get cut into a fourth of what it is now or it'll go down by half and be twice as high capacity right yeah like yeah that's right or you would have twice the capacity at the same price yeah that's another way of looking at it yeah but uh but yeah i feel like the the it's still early but uh i regret not buying tesla stock Let's just put it that way. I'm pretty excited about it. Don't get me started on that. I don't know what it will be in the future, but it's done well so far.

Yeah. I hope you'll say the same thing about a self-driving car startup, right? Yeah. It'll do something awesome or Google or Nissan or whoever. I mean, I think it's one of these things that it's good to have it happen. Right. Definitely. Anybody who's working on it, i'll be a cheerleader for

at some level yeah definitely i think it's great um so okay my article is a analysis of flight data set and this was pretty cool it was from mongo db's blog um for people who don't know mongo db is a no sql database i think we talked about it. We talked about this on our SQL episode. But they also have this blog where their interns kind of like write about cool things they do with MongoDB. And this blog in particular, I think is really interesting

because it's very simple, very easy to read. And, you know, a lot of people don't really know what data mining is. It's sort of this overused, very nebulous term. And, you know, this is sort of a pretty easy example where they got overused, very nebulous term. And you know this is

sort of a pretty easy example where they got access to... they didn't get access, I mean to open access to all the flight data from all the airplanes that have that have flown, commercial airplanes that have flown in America over I think past year or so. And it's just a series of cool things they do with that data. Like they explore the cascading effects of delays and things like that. I thought it was a really good read. Pretty simple, pretty fun. And it uses very simple MongoDB queries. It's not doing some kind of crazy, you know, logistic regression or something like that that you have to, you know, pour through a manual to understand. So yeah, give it a shot.

Take a look at it and you could actually redo all of the experiments yourself. Yeah, this is really interesting looking right now. Yeah, yeah, totally. it kind of makes me want to have a lot more time yeah if they can figure out a way to data mine another hour in my day that would be pretty epic and that well okay i don't want to get sidetracked that's something interesting they have that what is that the quantified self movement which is not exactly what you're saying but similar which is like in order to find where you should improve you first have to track all this stuff right like you need like if you had more information about how you live your life you could do that like right now if you ask that question how would you even start you don't have any data right so there's

like a group of people out there trying to like track basic metrics about them i'm probably butchering this but this is the high level view people out there trying to track basic metrics about them i'm probably butchering this but this is the high level view people out there trying to track basic metrics about themselves their weight their you know what their activity level was how they feel like any sort of you know varying set of parameters over time so they can analyze themselves and say you know how do i do better oh on days i drink coffee i feel better immediately but worse overall right you know so i should stop drinking coffee or things like that yeah totally yeah i mean if they can find out like certain things you do that cause you to lose a lot of time then yeah it's basically giving you free time if you can fix these things yep although maybe you'll spend you'll be more efficient and less happy and then you'll wish yeah that's right you shouldn't

have surfed the internet ever so the last thing of news here is i uh recently i saw this book well i guess book online book ebook released as pdf all these different formats about bayesian statistics was released called probabilistic programming and bayesian methods for hackers i hope i messed that up too that sounds good and it uh is interesting because not only is a good, I've only read through about I think the first chapter so far. And so far it's really good. Mostly stuff I know about covering it really well.

And they have this cool thing which I had seen several times since now where you kind of embed Python charting in the HTML or whatever. And so it actually will run different per run or whatever but actually show you like a graph that is generated from Python when you load the page. At least I think that's what it's doing. And the PDF, of course, has them just statically generated. But this is pretty cool.

And I think later ones, I believe, have even some interactive things where you can change some stuff and see what happens and able to tell different parameters, effects. Like if you flip a coin 50 times, what about 500 times, you know, what happens? And just in general, Bayesian analysis, there we go, is left off. I mean, maybe Jason could talk more about this because I think he's been doing some stuff with it. But, you know, when I was in high school, we learned a little bit about statistics, but definitely nothing about Bayes.

And even in a statistics class, I think it was mentioned as like a side note. I didn't take in-depth statistics, which is a basic statistics class. And we did a lot of the conventional, what is it called? Stochastic statistics. Is that what it's called? Frequentist.

Frequent, okay, whatever that thing is. We did a lot of that, but we did almost none other than just cover Bayes' theorem, like what it was. So do you remember Bayes' theorem? I'm going to hold you to the chopping block. It like the probability of a given b is equal to and then yeah I could probably sit there and think about it for a minute but it's like probability of b probability of b

given a divided by something yep divided by probability of a yeah so just to recap yeah I got close I didn't learn something so Bayes rule is like kind of like the fundamental theorem behind sort of what's called inferencing which is you know the which is like the basis of prediction and machine learning right so the idea here is you know so the equation is as patrick mentioned is probability of a given b so imagine like probability that it rained today given that it rained yesterday is equal to the probability that it rained yesterday given the probability it rained today. So that's flipping those two around. Which was the hard part for me because it was like how do we improve things by flipping it? It was not intuitive to me.

So I'll get to that. Multiply by the probability of a over b and so um those two are pretty easy to figure out right so the idea here is in one you're going forwards like like a given b means what is it today given yesterday you're going forwards in time and so if you were to go another step forward, like, you know, tomorrow given today, that's really sort of, that's forecasting, right? But the probability of B given A is going backwards. So that's like saying, you know, using data that you've already seen, sort of, how can you, or, wait, yeah, I got this right. So yeah, probability of A given B is going

backwards which means you know given data I've already seen how can I revise my new probability and so what it does is you know obviously if it rains one day it's probably more likely to rain the next day because because typically clouds hang over a place for more than 24 hours right depending on where you live but so you might start off with say what's called a prior distribution, which is I don't know. Like you could use, for example, say a beta distribution or a normal distribution, which says, look, I don't really know. I have a 50-50 chance of it raining and I don't really know if the fact that it rained

yesterday really matters. But then by observing several rainy days in sequence and applying bayes law you can come up with a new distribution which is skewed towards you know it raining several days at a time so it's kind of uh it's kind of a hard thing to grasp there's definitely a ton of awesome articles on bayes um but this uh i get the impression that this these guys also do a really good job i haven't read this one in particular but this uh i get the impression that this these guys also do a really good job i haven't read this one in particular okay yeah so i mean the thing for me too like at least in this introduction and i intend to go through more given more time but recently that's been in short supply um is that you know it is kind of jason was saying there i

guess frequentist is the other way where you you know something happens and you count how often it happens or whatever versus kind of the base is actually a different but equally valid way of looking at the world it's just like two sides of a similar coin like they're both trying to describe and some have advantages and disadvantages but the fact that i only ever had one i guess i kind of felt like i was missing like there were certain things like i would get to and i wouldn't know how to like even think about them properly it was like like, oh, that's just because I only had the tool of applying the normal probability that you think about. It's 50% likelihood that if I flip this coin, it'll land on heads or whatever. And that's the only way I had to think about it.

So having another way to think about it, even if I don't remember the specific formulas or code it up, is helpful because sometimes it reveals nuances to a problem or ways of thinking about something that's been really helpful to me yeah there's many domains which are i guess for lack of a better word i'll say soft like for example poker with poker it's very stochastic i mean you don't know what card you're going to draw if you did then you just win right i mean well you wouldn't always win but you would play very well and make a lot of money right the reality is you don't know what card is coming next and so when you have play very well and make a lot of money, right? The reality is you don't know what card is coming next. And so when you have these very soft environments, what's more interesting than, say, a frequentist,

which will tell you, oh, you should probably bet more, you should probably raise, is sort of having a confidence. If your output is a distribution, then you can say, you know, the mode of this distribution, so in other words, like, what I think you should should play is, let's say, raise $15. But you have this huge uncertainty or, you know, I'm holding pocket aces. There's two more aces on the table. I have four of a kind aces. Now I have virtually no uncertainty.

Right. So you can think of Bayesian as sort of living in this realm of distributions instead of numbers instead of like what's called point values and that lets you do a bunch of really interesting things yeah and you can do some it's been my impression

it's like the stuff that you can do in one you can do in the other it's just how easy or hard it is so you can do confidence intervals and distributions you know and then I don't want to call it traditional because i don't think it's the right way but in the frequentist view right like you can't do that stuff but it was always like the most complicated thing we always did at the end of the semester yeah like you know it was very difficult to do but in bays it seems to come naturally like once you get everything set up which might be a little more tricky like all this other stuff comes for free essentially yeah i mean like the unit of your operations is a distribution it's like you're

working with functions so in other words like for example if you add to normal distributions you get a normal distribution with the you know average or i think it's like the sum of means being the new mean or the average of means and the sum of the variances being the new variance but the point is like there's operations that you do on distributions rather than on numbers and that can be really hard like for example you can't take the product easily of two different distributions like if you have a logistic distribution and normal distribution and try and take the product there's just no closed form for that and so um in these examples like you have to do like monte carlo methods i don't know does this get to those uh i don't know mcmc and all of that mcmc yes opening the black box of mcmc oh

there you go yeah so so it's hard but mark of chain monte carlo yep i know i know what you said nice but then the output is just so awesome, right? I mean, think about that. Like now you have not only an answer, but a distribution over all the possible answers that could exist. So if you're writing, say, an AI to play Monopoly or something, you know, you don't want like you don't want something like what's the minimum amount of like how many houses should i buy so i don't get you know knocked out and have to mortgage right well that there's no clear answer because it depends on the dice right but if you use bayesian now you can come up with some kind of distribution that says like oh i have say a 90 confidence i'm not going to have to mortgage

and i can buy three houses in my monopoly game whatever right so um yeah it's pretty awesome i'm definitely gonna have to check this out and brush up on my bays all right with that in mind let's go to book of the show of the show show show my book of the show is i have also been looking into bays for a variety of reasons which is why you have so much to say about it one of them being that i need to get uh confidence intervals for some systems i've been working on and so i've been reading a book that is pretty heavy so i'll i'll give you a little bit of a disclaimer it's it's a heavy book i definitely recommend uh you know i think google books has it you can read the first like 50 pages without buying it right um on the flip side it's very thorough and you need no background so in other words if you're

willing to slug it out um you won't have to like be a bayesian expert to understand this book but uh it is going to make you a bayesian expert so uh it's definitely a heavy read um computational bayesian statistics i think it's by uh let me look it up it's by a heavy read. Computational Bayesian statistics. I think it's by, let me look it up. It's by a guy in New Zealand, actually, who's pretty famous for this stuff. So this will cover what we've been talking about in exhaustive and dry detail, it sounds. Yeah, and I mean it. But that's good. It has one cool thing.

It has a bunch of pictures. It has pictures of distributions, things like that. And you can see sort of... So the author is William Ballstad. Okay, Understanding Computational Bayesian Statistics. Exactly. And it... Oh, yeah, it has understanding.

But I will post a link in the show notes that you guys can click on and check it out. I'm reading it now. It's an awesome read. It covers everything in great detail. You don't have to be a math genius or anything to figure it out or to follow along. And it's highly recommended. But there's this other one with puppies doing Bayesian data analysis.

With puppies? It has puppies on the cover. Oh, man. Is it a distribution of puppies? There are three puppies. Oh, man. Okay a distribution of puppies there are three puppies oh man okay get the puppies okay never mind i was playing along at home uh so my book is one called going postal no it is not a serious book by terry pratchett so this is uh part of the disc world series which i haven't read a lot a lot of books but i've read

like i think three or four same here books from terry pratchett and there are a series set in this universe of craziness that's all i can describe it maybe i'm missing a lot of the jokes probably am it's not unusual but uh it's kind of just like silly adventures in this fantasy world right but the fantasy world makes fun of itself so yeah there's like trolls that are making fun of troll tropes or whatever right like you know there's a there's a hydra um used chariot salesman and you know hydras have like seven heads or whatever yeah yeah and his name is cut me on throat dibbler and if you can find a cheaper deal you can he'll cut one of his own heads off and grow 10 more and grow 10 more

yeah but he's like that's his threat you know you find a better deal and cut me on throat yeah so i mean there's some british amount of british humor in it so i equate a little bit to monty python style yeah um that's probably terribly unfair to someone i'm not sure who but uh this book is interesting and if you like a you know kind of a light-hearted read that's just fun when you get to the end it's not like you can tell somebody you had this deep you know story that was epic it's just kind of like a fun ride to read along and see what happens and the adventures of the people and the characters will recur in various stories even if the stories themselves aren't really sequential yeah definitely i mean in fact there's an order uh there's a reading order guide

online okay it's called the l space i should probably follow that i just read them randomly yeah it doesn't matter but if you read in a certain order you can catch all of the cameo appearances in the right order like you can see a cameo appearance of a character that and know what they've done already oh yeah that'd probably make more sense but and it's not like a normal order it's kind of like you have to read in kind of a wacky order so okay but yeah the books are amazing yeah that's it's definitely a light-hearted read so so is this one about like the postman or something yeah going postal yeah yeah so about this postal postal guy i don't like sharing too much about yeah we don't think about this because then i feel like yeah i definitely

spoiled my computational vision so you guys know the ending in the ending uh you understand bayesian statistics spoiler well hopefully you do anyways yeah or at least you've seen a lot of cool pictures all right time for tool of the show tool of the show so this is a total cop out i'll be the first one to admit it my tool of the show show is two apps. My tool of the show are two apps. Anyways, my tools of the show are two apps that I wrote. One is NES Together and the other one is SNES Together. And as the name suggests, it's a way for you to play Nintendo and Super Nintendo games together with someone else on another Android device. So for example, I can take my phone, start up NES together,

pick an NES game like ice hockey or, I don't know, Contra or something, and then, you know, invite Patrick. He can join, and we'll both be playing Contra together. I see you post about this sometimes on Google+, and I'm very envious that you have so many friends to play old arcade games, old Nintendo games with. Yeah, I've been doing emulators for a long time, so I've kind of built an audience and a community of people

who are really into emulators, so there's always kind of somebody there. But you have your own hacks in the system, so you always win, right? Totally. Ah, nice. Yeah, I have this ice hockey hack, or it just sets the score to whatever I want. Every time I press volume up on the phone, I score a goal. How did that happen? Yeah, trick shot.

But yeah, I think it's pretty fun. You know, a lot of these NES emulators are great. Like NESoid, the Android NES emulator is phenomenal. I'm not trying to compete with them. They have a great emulator. But none of them use the phone's internet. Like all of them require Wi-Fi. Oh.

And so that's sort of the niche there, is that you can just be on a bus and start an NES game with your friends. So you can rack up your bill while you play. Exactly. Jason's getting a cut from the cell phone carrier. Yeah, Verizon totally has a deal worked out. So my tool to show is Joda, the Java

time library. I actually don't know if they have one for C++, but I know of Java for sure. No, just Java. And so I'm terrible at calendar math and doing all things time related. Oh yeah, it's so hard. And handling things like leap years and daylight

savings time. Oh yeah. It's really messy to begin with. And it's still messy when you use Joda, J-O-D-A. Didn't they change daylight savings time? Like something happened. It was like leap seconds or something? Oh, yes, like a few years ago, yeah. Yeah, something happened. They changed how it's done.

Yeah, right. So, I mean, like trying to keep up with that. Anyways, I don't know. Yeah, it's just that's kind of like a nightmare. Yeah, so this Joda library is a set of Java utility function things that allow you to manipulate times and do things like, what is today minus five days or minus one month, and set it and handle it appropriately for things like if I'm at March 1st

and I subtract a day and it's a leap year, it's different than if it's a non-leap year when I go to February, and handling all that stuff and doing it in a way which is at least semi-readable and approachable and i really have been doing a little bit more than normal with it and appreciate how much less time i have to sit there thinking about am i doing this correctly when i can just use the library yeah i mean this stuff is like a total disaster like time time so it's just it's just so hard and uh this library makes it so easy i mean i think that i don't know if you do this but i always do everything in i think what's called like usec or something or msec but it's

basically like microseconds dot current time epoch yeah yeah right so i do everything in that and at the last minute i use joda time to like convert that to something reasonable yeah so we have some stuff that like looks at data over the past and tries to accumulate like in the last seven days or last calendar month or last quarter and for a long time before you know that we undertook an effort to really make sure we were stringent about how we treat time it was like without fail every end of month or every quarter unit tests would just start failing you know and like it wasn't really that there was a problem it was normally that just the test was broken but it sometimes did reveal that we had legitimate problems where our data wasn't being

calculated correctly or we'd miss a day here or there of course that's really really bad yeah totally totally yeah that's just totally really bad. Yeah, totally, totally. Yeah, that's just a total nightmare. I mean, you think about it, like, let's say you were, say, Groupon or something, and you tell someone, oh, you know, this offer is going to be here until midnight. Well, like, what does that even mean? I mean, like, if I'm in Florida and I take a plane to California, my midnight has just changed, right? I mean, it's like, like's like dealing with this stuff is such a

nightmare, and Joda definitely makes it a lot easier. Yep. So I think we're on to CUDA and OpenCL. Alright, so this is something that Patrick and I have done a lot of in our past life, and hopefully we remember a lot of it, but yeah, it's a pretty

awesome, very powerful set of utilities. So I'll cover some of the history maybe. So firstly, I guess we should say that CUDA and OpenSALE both are ways of using the GPU, the graphics processing unit, in your computer to do something besides draw your 3D video game. So some of it may work with images or representations of images, but for the most part, you're doing general purpose, things which aren't related strictly to what the GPU was initially designed to do. Yeah, exactly, exactly.

So when GPUs came out, they really revolutionized gaming, right? I mean, I remember Dungeon Keeper 2 or Half-Life 1. Yes. Okay, yeah, so those were like right on the era where GPUs were coming out. Like Half-Life 1, you could either use a software renderer, which used your CPU, or it supported Glide, which was one of these early GPUs. And the idea is they're very specialized.

They work with, there's two sort of elements in this pipeline. There's a vertex shader and there's a fragment shader. So again, these are meant for doing graphics and we'll get back to how they can do general stuff. But the vertex shader, so we're real quick on how computer graphics works. Everything is a triangle in computer graphics. Everything's just a bunch of triangles. The three

points of the triangle are called vertices and so you have sort of an army of these vertices, just these triplets of vertices, like vertex A, B, and C make a triangle, D, E, and F make a triangle, and they all flow through the system and get transformed. Like one possible transformation is, you know, if you're playing like a first-person shooter and you turn your head 90 degrees, well, the way the graphics processing unit handles that is it rotates the whole world in the other direction 90 degrees.

And so every vertex gets rotated in space to like counteract your rotation and make you feel like you're turning, right? And that rotation is a matrix operation. Right, that's right. So the vertex can be represented as a vector of three elements, you know, an X, Y, and Z component of the vertex. And then, right, any operation that transforms a vertex is a matrix operation.

So like rotation, scaling, transformation, these kind of things. So there's a vertex shader. Once all the vertices have been transformed then they get crushed into the screen. So you can imagine sort of like all the vertices in your screen but they're all in 3D so they're all kind of these projections. Then they get crushed in a 2D and then what happens is something called a fragment shader starts at, looks at your three vertices and looks at every pixel that is within that triangle and then based on a bunch of different things figures out with the color of that pixel so for example a triangle might be red but there's a blue light shining on it and so the friend and it's facing that

blue light so the fragment shader will sort of use all this data to sort of figure out like for this particular point in space like what the color should be so is that also the part responsible for determining like oh it's on the shadow side of the light or facing the right yeah and also you know some triangles might be one-sided so if you're facing like the back of the triangle then it won't draw it this is called like back face culling and things like that so this is like oh you're looking at a cube but you can only see some portion of any given time exactly where you can stand where you see all of it right so the back part of the cube they won't draw okay

and so uh that's actually that might be done in the vertex shader but at some point like one of these shaders is responsible for saying which triangles exist like should be drawn and which shouldn't which is really hard because i remember i think it was in the first half life they had like a game editor and they would talk about this so like you needed to be careful to place things to block so that the algorithm could decide like the thing in the next room isn't visible right so like if you drew like a door and a wall and you left like a tiny gap like just two things kind of touching each other but there was a slight gap there the algorithm might start attempting to draw all the things

in the next room yeah and that would make it go really really slow yeah definitely so um so the point is that the people found that this was really useful and it's very fast. So one thing that you can think about is like as I mentioned with all these operations, like let's look at the vertex shader. You want to take sort of all these vertices and rotate them to fit the way your head is rotating. But you want to do the same rotation on every vertex. So we're here. You're saying head and also meaning camera right oh not just like the model's head is turning but you're talking about like the camera is turning

exactly yeah so yeah so you're playing some fps and you're in the the camera is sort of where you're looking at yeah yeah and your eyes are turning what's really happening the whole world is turning and so they're doing the same thing on wait that's what happens to me in real life the whole world turns into your head? Really? I'm pretty sure. This is one of those, like, unprovable things, right? Like, nothing exists.

Like, I'm in a dream world right now. Have you seen this? Like, an undefeatable arguments? No. I think there's a Wikipedia page on this. Anyways, so the GPU is highly optimized for doing the same thing on a bunch of different input and you know that is actually really useful. I mean look at say machine learning. You might do say linear regression where you want to compute you know huge dot products of many variables. So you want to take a bunch of variables

and add them together or take a bunch of variables and multiply them by something. So you could do this much, much faster on a GPU than on one CPU. So people started realizing this and started coming up with all sorts of cool things to do on the GPU that aren't graphics related. But this involved a really, really painful process. So what they had to do was, you know, convert all their input data to images. So find some way to put, you know, their math, their like, you know, going back to the machine learning thing. Find some way to put the weights of their matrix into, you know, a texture on an image. And then you have to turn off all the special effects because they don't want, you know, lights to be shining on their weight matrix or something like that. It just doesn't make sense, right?

Or, oh, as you as the triangle gets further out your weights get smaller because of fog i mean it's just like crazy so you have to turn off all these render a frame and then the fragment shader you know which is drawing all these pixels on the screen that fragment shader will be doing something crazy like a dot product or something and putting the answers on the screen in pixels then you have to read the screen and interpret that yeah because the gpu often draws directly to the monitor like it right it is the thing you plug the monitor into so it has a little bit of memory that holds what everything on the monitor should be right yep so this is really hard and uh so cuda and open are... And really nobody did this. I mean, there were people doing it, like crazy hacker people,

like awesome guru people, and very specialized one-off custom things that they would do. But then any time there was a change in the library or a new graphics GPU came out, any of these things would completely just mess them up because they weren't doing something supported. Right, they were doing something that like specific to one particular card and also like many of the tasks where were graphics related anyways because those are only people who knew how to program them like i

did one thing as part of my master's thesis which was like a bouncing light from one place so in other words like a light shines in and it radiates the whole room. And I was only able to do this because of the background in graphics. Nobody could understand this crazy mess of code. You have to manually do things like disable fog. Otherwise, your answers were wrong. You didn't know why. So Kudo and OpenCL are a way to sort of relieve us of having to be graphics programmers to do

basic things. Which is a good thing because otherwise I never would have been able to do it. Yeah, totally totally. So you want to cover? Yeah, so I mean, so the idea is it's an abstraction on top of this thing. So there now it is true that I guess there's enough people doing this where they make

special allowances in this hardware this these very specific pieces of silicon that are made to do this graphics processing to allow for this general purpose processing but initially it's just kind of an abstraction it's like we have this very specific and maybe we'll talk about this in the next episode but kind of like you have a general purpose cpu which can do anything then you have these very specialized CPUs which don't have a lot of operations because you either have to simulate them or have to add essentially AND and OR gates to represent how to do that thing because the processor just isn't used to doing it

because it's not needed for shading and lighting. But as they got more and more sophisticated and more capable, it turned out that people wanted to do crazier and crazier things in both the graphics arena and the normal. and so people realized it was beneficial to have this abstraction where you could essentially shove any array into the graphics processor and then have what CUDA and OpenCL are mostly about is a kind of a way to compile normal C code that operates on a matrix, compile that into a series of graphics operations for those graphics processors to operate on,

and then a way to get the data back out without having to, a supported way as opposed to, tell me what should be on the screen right now. Yeah, right. And the benefit of that is that these GPUs, not only are they really fast because they do very specific things but there tends to be a lot of them like right i mean just now we're getting into like people having like eight cpus in our computer or maybe 16 i don't even know i don't even keep up

with it a lot but i mean you can easily have like i don't think like when we were doing it it was like 128 256 512 now process like 10 24 processors those processors can only do very specific things but you have a lot of them yeah which could be a huge advantage yeah definitely i mean the big trade-off here you know people say well water isn't why isn't everything in a gpu right and so the trade-off is as product was alluding to um so there's only, so the threads are grouped into warps. There might be like 1024 processors or 1024 threads, let's say processors, but they're grouped in blocks of 32. And for a particular block of 32 threads they can only be doing one thing at any one time. So in other words if you have some

data and on you know the first index you have to do one know, the first index, you have to do one thing. On the second index, you have to do a second thing, so on and so forth. Like it's specific to each index. It's going to take you 32 times as long to do that because, you know, the first processors are going to work on the first, you know, bit of data. And all the other ones are going to wait. And so in this case, the GPU would be really slow, like much slower. Yeah, it's like if you had a single number. Like if you had, you know, I have one number,

and I want to do these couple of math operations, then some branching, and then some other math, right? And then I want to run some, like I'm starting on my program or whatever. That set of operations isn't suitable to the GPU because although there's many of them that can run in parallel, each one isn't necessarily as fast as your as a regular processor at getting a single thing done right so the advantage is that you can do a lot of things at once and also it brings out they because this graphic stuff doesn't

have a lot of decisions that need to be made they're typically their performance at having branching operations like an if statement or a for loop like those kinds of things start to get a little hairy. So some for loops are good because you can unroll them and run them. But like a for loop where you're – So what's unrolling? Oh, okay. So initially a for loop is – and this gets into like understanding how computers operate. So the processor doesn't understand what a for loop is really.

So what it knows is, okay, I'm going to consider some conditional statement. Is this number in this register more or less than this number in this other register? If it is, branch. If not, execute the next following statement. Increase the program counter by one or whatever. And so loop unrolling says instead of having to, when you get to the end of the for loop, jump back up to the beginning and execute this conditional again if your for loop is for instance i have a known 1000 entry array and i want them to be the number one the number two the number three you know one plus their index essentially if i

want that to happen i already know in advance i'm going to have to do this what i say a hundred times or a thousand times a hundred times. So instead of having a loop which says, you know, check if, you know, my current counter is less than a hundred. If so, do this, then jump back up, right? This code running over and over. You can just do all of that all at once. So, you know, set array zero to one, set array one to two, set array two to three, set array three to four. And if you listed all those in order, there's less instructions. If you count actually a number of cycles taken by the CPU, it's a lot less. It used to be much more beneficial,

but then CPUs got better and better at understanding that, for instance, if I'm in this loop a thousand times, there's a very high probability the next time I come to it, I'm going to do the next thing already. And so it knows that. So loop unrolling in this case would say, oh, if i'm going to do this set of operations in the gpu like actually just make them set instructions instead of this branching which confuses the gpu at some level because it's expecting all of the pixels to be going through the same operations at the same time right exactly and if you want to know uh you know based on how many times you've been through a

for loop what is the chance that you'll go through the for loop again, you can read understanding computational patient statistics where they go over exactly this kind of problem. And the reason you need that is to do pipelining and all this stuff. So to your point, the GPU does not have very good pipelining so in other words like a cpu will do all sorts of crazy things like i think branch prediction where it will actually take both paths of a branch as long as it doesn't have to write anything to memory well yeah what it'll assume is it'll assume in that for loop it'll assume you're going to go ahead and do the next array operation and actually start doing that operation right but it still hasn't figured out if it should or not right and then once it figures out it should then it just goes ahead and keeps that answer if it figures out it

oops i wasn't supposed to do that it has a way to essentially flush the pipeline right and then load the whatever the other instruction was supposed to do instead right like load that you know now but the problem is you lose all that uh time you should have been doing those other operations. Right. Yeah. So the GPU, because it has so many threads and because of the ways designed with this intuition that all the threads do the same thing at the same time, it doesn't have any of this. Like it doesn't have branch prediction.

There's a lot of things like this that it does not have. And so to Patrick's point, if you have a branch, let's say you have a branch, like if I am, you know, operating on the first of this array of data then do something special while you're doing that special thing on one thread the other 31 threads are just waiting for you to finish and uh and so that's bad so uh so but you know it turns out many very cpu intense programs don't really need a lot of branching no so like you know video transcoding and things we're thinking about like all the like photography i always think about like photography stuff because it's something i understand a little bit about but like if you want to apply a blur

right right you take you have one image your input image and you have your output image and you have your thing you're going to to use to do the blurring which is a matrix you just need to add every pixel apply that matrix to the input image yeah there's isn't really a lot of branching to be done there you just know what needs to be done at every at every pixel just go ahead and do it and you can do them all in parallel right exactly right like just for every pixel do this thing as opposed to for this image go through it and do this thing yeah exactly exactly um you hear a lot of crazy things about uh you know gpu programming like oh i have a 60x speed up i have a 200x speed up right most of the time those were things that

were programmed badly you know well i mean things that are what i the term i like is embarrassingly parallel or whatever embarrassingly suited to gpu like if you give the gpu something that's almost exactly like what it was built to do it's awesome yeah but it's a very sharp descent like if you get outside of its very oh now i'm talking in like bass if you have this like you know this thing is really good at anything near that thing right it will also get a really huge speed up but even if you just move a little outside in any one of the directions you get like the performance starts to drop drastically right and there's a lot of things that you know cpus with simd and these kind of things uh can do can do pretty well but but with that the gpus are

pretty awesome the speed up is almost always worth it um to for the extra you know engineering effort if it's yeah if it's a matrix-type thing and you're using images or video or any of these things, yeah, it's going to be awesome. Yeah, totally, totally. For the most part. So somebody will counteract this.

But you do have to pay a penalty because you have to move from system RAM to the GPU's RAM. Right. And GPU RAM is really fast, which is great, but you have to pay that penalty to move back and forth over whatever the bus, PCI Express or whatever.

And so that is a non-negligible amount of time. Yeah, and that's to your point, you have to do a lot of pipelining. Like you have to say, okay, GPU, I want you to do this on this part of the image, but then like while you're doing that, I'm gonna be sending you the next image. And as long as you can sort of,

so that way it's sort of like the GPU is working and the bus is sending things at the same time. You're not just doing things serially. So like any parallel program, much less one that's so crazily, highly parallel, debugging is crazy. Oh, yeah. Debugging is kind of. You don't have a lot of the weird, like, at least if you stick to the most basic kind of stuff that most people do, you don't end up with a lot of race conditions because you shouldn't be doing that kind of stuff um in most of this uh gpu programming but even still like you're it's hard because if you go through a loop of the program what does that

really mean like you're in one pixel of one image and one processor and you're looking at something but the interaction is probably something slightly higher level than that. And you need some way to simulate it on your CPU because GPUs don't have the same kind of breakpointing ability as CPUs do. So yeah, it's getting better, but it is definitely hard. Yeah, yeah. I mean, when we were doing it,

it was kind of a nightmare. I think it has come a long way though. It's really gotten a lot better there. One interesting thing is for data on the GPU, there's three different types of arrays that you can allocate on the GPU. There's a read-only, so the idea is you're allowed to write to this array once and that's it. You can only read from it. And only the GPU can read from it, not the CPU. And so if you go there, it's going to put it in some structure which, you know, the bus isn't allowed to talk to this structure anymore.

It sort of locks it off, but it puts it in a place that's very easily accessible by the GPU. Also, it knows that this data is never going to be written to by the CPU, so it can do some optimizations there. Like it can do some operations like in a pipeline, knowing that it's not dependent. Yeah, there's no data dependency. There's write only.

So for example, like all of your input data, you don't want to read that back from the GPU. I mean, you have it already on the CPU and it's completely in its pure form, right? So all your input data, you mark it as write only. And again, the GPU knows that you're never gonna have to pull that back. So you can do some crazy things,

put it in some volatile memory and things like that. Or read-write. And so read-write is the slowest. The GPU expects at any moment, you're gonna read that data. And so it has to sort of keep it in a state that's stable. But sometimes you need that. And you bring up one of the things which is confusing at first is whose perspective are you talking from? So if it's read-only, is it from the CPU or GPU?

Write-only, and you just have to be careful to balance that in your head. Like when you say something, whose perspective are you talking from? Yeah, that's a really good point. Yeah. I mean, so just to recap, read-only means, you know know you send it to the GPU and you never expect to get it back so the GPU can read it and you can't so it's GPU read-only and then GPU write-only is something that the CPU only will read not the GPU write GPU write so write-only is where the CPU writes it,

but the CPU will never read it from the GPU. I think that's what you just said for the other one too. It's okay. There are opposites. There's two, one for each of the opposite sides. It's okay. My bad. We got it. We got it.

Yeah, I think we've beaten this one. You should look it up. So the first one, first library we talked about CUDA is something written by NVIDIA, specifically for NVIDIA GPUs. Right. So that has some advantages. They know certain things will be present or that they can rely on our architecture. And they're a company that is very highly invested in keeping you locked into their platform.

But in turn, they kind of give you a lot of niceties. Yeah. A good analogy would be directx yeah like directx it's windows only and uh so it's like the os and the gpu can sort of have an agreement which open gl isn't can't do that because it's supported on so many different devices yeah you can have it linked in with your programming language of choice but then you have specific components, parts, files which are flagged essentially as CUDA parts which are

C programs but written in a restricted set so you can't have function pointers, you can't there's a couple of restrictions I don't think you can have global variables or maybe you can over time they tend to become more and more relaxed they figure out how to do more and more things.

But you might get a penalty for doing them. So there's a little bit of, and all of these we'll talk about. There's a, more so than normal programming, there's a strong responsibility on the programmer to understand what they're doing and whether it'll be expensive or not. Right. Yeah, like, one thing that you might not expect, integer operations are really expensive. And it's because everything in graphics is floating point like where points of a triangle are in space and things like that have always been done in floating point and so there's i mean now this might be different but when we were doing it

there wasn't really like an integer in operations and hardware and so that made it actually is much faster for us to do floating point division and if there's a little bit of loss. Yeah, it's okay and Like I said Nvidia provides some special libraries on top of just CUDA itself that are really good at some of the things a Lot of people want to do like fast Fourier transform Linear algebra stuff so the blast stuff They provide their own versions of this where instead of handling all these things we've been talking about, if you know I just want to do a Fourier transform on this data, you just call it and it transfers the data over,

performs the operation, and brings it back. And when you get done, you just have it there. Yep, so for people who don't know what fast Fourier transform is, I'll give my two-second or five-second explanation. Okay, go. Okay, so if you have a series of data, let's say it's a time series or it could be spatial series, you know, like an array, you know, like a 1D image or something like that, right?

What a fast Fourier transform tries to do is turn that image into a frequency space. And a really simple way of explaining it is, you know, it tries to find a bunch of coefficients such that you can add a bunch of cosine waves together and recreate your original signal. You described Fourier transform. You've got zero seconds left to describe the fast part. I have no idea. Do you know fast? Yeah, so I think it's basically Fourier transform is a mathematical notion for continuous signals, for the most part. But yet in the computer world, a lot of times we're dealing with discrete signals. So you want to convert your wave to MP3, or at least analyze your wave or whatever.

But you only have sample points so far apart, and the signal ends at some point. You have an end of your file. So the Fourier transform will turn your discrete signal basically into sort of a continuous signal by storing all of these coefficients of a bunch of cosines that you can then add together at any point. So fast Fourier just exploits some notions that because it's discrete, because it ends, and there's some symmetries, how do you do it faster? And I don't know the specifics of it. The math always escapes me. I try about once every month to read some article that's supposed to explain it to me.

And I still miss it. But it has to do with symmetries and optimizations because of, you know, what you're working with. Gotcha. So, yeah, if you ever wondered how MP3 works, I'll give it to you really quickly. Oh, dear. I mean, it's really like a discrete cosine transform, so it's a little different. But it's like an exact FFT. But just imagine, you know, instead of storing.

So, you know, music comes in like a wave. Everyone's seen like this. What is that machine where it like lets you show different pitches and it shows you the wavelength of the pitch and everything? It's like a... A tuning fork? No, no, it's like a machine and it has graphs and it also has a pitch. And you can like turn a knob and the graph will like...

The wavelength will shrink in the graph and the graph will like the wavelength will shrink in the graph and the picture go okay i don't know oh we had one that uh okay anyways so uh so music comes in waves oh i do okay oh you know what i'm talking about now yeah you don't know the name either okay oscilloscope oscilloscope that's exactly what i was thinking okay so everyone's seen an oscilloscope which has sort of those waves right and and it's how that's exactly how I was thinking of. Okay so everyone's seen an oscilloscope which has sort of those waves right and and it's how that's connected to music. So you can store these waves and the way an mp3 does this is if you have let's say like a cosine you can find a cosine that kind of fits your wave and you can you know pick a certain amplitude which is how high the cosine goes and a certain frequency which is sort of how narrow each of those up

down hills are and so you can find you know a frequency and amplitude such that it tries to approximate you know like kanye west's song or something like that right and you you won't get all of it of course with one cosine you'll get a little bit of it so then you take that part out of kanye's song and now it sounds a little different, but hopefully you find another cosine wave that can now match that new song. And then you basically keep doing this until you end up with nothing. And you have like all these cosine waves

and when you add them together, you get Kanye West song. Yeah, so essentially relies on the fact your ear can't hear all of the little differences. Right. So if something's changing really, really fast, it's probably unimportant. It's probably just quote unquote noise or whatever that could be filtered away. Yeah.

And the same thing is true for images. JPEG is the same thing. It tries to say in this area of a white wall, there's almost no difference between two pixels. So if I just approximate it, you won't lose very much information. It tries to very carefully pick what information to store as opposed to storing everything. Yep, exactly. And so get back to NVIDIA. So NVIDIA has a

CUFFT library. You could totally use this to write your own GPU based waved MP3 transcoder. That would be super fast. Yep. Okay, so there's also bindings for CUDA for many languages. It's pretty easy to write your own.

OpenCL, a standard, very close to OpenGL in that it supports multiple CPUs and GPUs. Interesting thing there, I said CPUs. It can also, your multi-core processor can be treated just like GPU. So you probably will lose out on doing something specific, but you could approximate something on a computer if someone doesn't have a GPU. Right. And it makes it much easier to debug, too. specific but you could approximate something on a computer if someone doesn't have a gpu right and it makes it much excuse me it makes it much easier to debug too you can actually debug on your cpu catching you know divide by zero errors and things like that and then push it to the gpu once

you're confident and you can support multiple brands of graphics cards right like if somebody you know imagine like you're releasing an app uh you know like some program you want people to download a video game you're writing or who knows right um you know you don't want to just give your game to people with nvidia graphics card you want everyone to have your game and so with opencl it'll use you know nvidia to use ati or i guess they're now amd graphics cards i don't if it doesn't have any of those graphics cards, it can fall back to the CPU, and so it's much more versatile. Similarly, my experience has been

that they tend to implement things a little slower because it's a standard, so it takes a little longer to get everybody on board. And people tend to prefer their own implementation over the OpenCL, talking about manufacturers. So, you know, NVIDIA will support OpenCL at some level, but it's going

to support cuda better and so that also causes some problems yep yeah totally well that's pretty much it i it was a high level we had a little bit of a venture off into fourier land yeah we started with origami and we ended with fast fourier transform so this is definitely an awesome show all right well we've been getting emails still people were wondering where the show went we've had busy lives having uh jason had a kid i had a kid so there are two more people in the world who want will want to listen to the podcast yeah maybe or not yeah we both had children in the past what two months no three months four months i don't know okay anyways it's all become kind of a blur yes we're definitely we probably get eight hours of sleep between the two of us on a good day yeah on a good day um but you know our our kids are

getting like how old is your like four months now four five months five months and mine's almost four months and so now we're starting to get our lives back. A little. And as part of our lives include doing this show for you guys. So definitely thanks for all the love and support. And hey, when you guys are doing an episode, that's really actually good that it sort of motivates us. Yeah, keeps us honest. Yeah, definitely. So we appreciate all the support. Thanks for looking at the books.

I haven't looked at the numbers lately but uh there's a good amount of people buying the books we're recommending which is great because at least clicking on the link and then buying something else which also helps that works too but yeah i mean all of these books are ones that we've read they're not both of us but but the one who's recommending the book is right and uh it's really good that you guys are getting into it too so all right till next time see you later the intro music is Axo by Binar Pilot. Programming Throwdown is distributed under a Creative Commons Attribution Share Alike 2.0 license. You're free to share, copy, distribute, transmit the work, to remix, adapt the work.

But you must provide attribution to Patrick and I and share alike in kind.