Programming Throwdown - Mailbag

Episode Date: November 19, 2012

This show covers some of the top questions asked by the fans. ★ Support this podcast on Patreon ★

Hosting provided by Host Tornado. They offer website hosting packages, dedicated servers, and VPS solutions. HostT.net. Programming Throwdown, Episode 21. Mailbag? So, this is a pretty exciting episode. Mailbag episode. One quick thing before we get started, if this is your first time, if this is your first

episode that you've listened to, listen to a different episode. So basically, this is, you know, we've accumulated a lot of questions throughout the other episodes. And so this is our chance to sort of answer a lot of those questions and sort of engage in a dialogue about that. But, you know, this is deviating highly from the original format of the show. And so if you kind of want to get an essence of, if you want to learn a programming language or get essence of the show, watch the one four. Okay. With that said, well, maybe actually the guests will be funnier than us. So then it'll make this episode better than the other episodes. Oh, that's true. If you didn't like the last episode, watch

this one. Okay. And if anybody who ends up on this episode is better than us, we're going to kick you off and make you start your own podcast. Yeah, totally. So, anyways. Yeah, so basically... Take it away with the news. Or a personal... We always start with a personal story. So, you know,

they're having the hurricane scene. Actually, they had the hurricane. And, you know, as having the hurricane hurricane scene actually they had the hurricane and uh you know as floridians we've been through plenty of hurricanes ourselves and one thing that i always wondered was um why don't they set up like why don't they have a mode in your cell phone where you can do like some kind of ad hoc so for example let's say the cell phone towers go down right um wouldn't there be a cell phone mesh yeah like a cell phone peer-to-peer network where you could call people and as long as one person in the network had access to the tower um then they would you know you'd be able to get your call through

and so it turns out they have something like this but um for like policy reasons it's not completely implemented. So in some phones, like in some circumstances, you can actually call 911 and it will route you through other phones, which is pretty incredible. And it's something that's not on by default. But like so just a heads up for real, the technology behind broadcasting and correct me if I'm wrong. You probably know more about hardware stuff. But so any cell phone can also broadcast. like if you can receive you can also broadcast something to do with like the physics behind it well it's a two-way communication yeah but so i mean like the tower can talk to you and you could talk to the tower but you can also talk like broadcast

outwards well i mean if you're talking to tower you are broadcasting oh really you don't know where the cone or something i mean i don't so yeah that technology exists but i mean i think in general you just have a sphere around you of you know gradient just you know going down by the cube law or whatever power farther away you get oh i didn't know that okay so that makes sense then so yeah you can totally do it from like a physics math but i think the reason it's not implemented overall is a couple reasons. One, if you ever study sensor networks or whatever, you know certain nodes happen to become gateway nodes

or key nodes that a lot of stuff flows through. And if you become that guy, your phone's going to drain. You're just going to have your phone in your pocket, not using it, and go back to it and be like, why is it all gone? Oh, that's so true. So you have to have all this extra stuff to make it not be that the second thing also is that when the cell tower is broadcasting to you it's got for all intents and

purposes infinite power right it just sits there and beams out really strong as whatever limits they have set but your cell phone has very limited so it hears quietly but then it also talks quietly but the cell tower has very expensive antennas and equipment to be able to you know receive that quiet message and pick it up from the noise so i think that's the whole mechanism versus if you're talking to other phones it's not going to have the same range as you did the tower gotcha because the receiving antenna on the other phone isn't as strong so it would probably work and in some instances probably would be better especially in densely populated areas like maybe like New York City or something. But in most regions,

like I don't think you're going to, I wouldn't know what the range is, but it's probably not that far. Oh, that makes sense. That makes sense. That's my top of the brain response. But it would be cool for emergencies like you have a special mode, you put it in. So I think the way it works is the carrier can put your phone into this mode. And so then you can... Only if they can talk to your phone. Oh, yeah.

So how does that work? Oh, so I guess you have to get put into this mode, like, before a disaster or something. I'm not really sure. It's interesting, though. I guess it works for hurricanes and not for earthquakes. They anticipate the disaster coming. Yeah. For earthquakes, it might not work.

One of our listeners was pointing out Batman. Yes, this did happen in Batman. Did it really? It's totally awesome. They subverted the phone network to detect where everybody was. Isn't that one of the movies? Did you see the... Oh, we should say spoiler alert.

Sorry. Oh, yeah. Well, there you go. That's a really old Batman. Oh, it's the original? Oh, then it's totally cool. Yeah, not the new one. So do you see the Die Hard?

What is it? This is the last Die Hard. It's like the Die Hard and Never Stop Dying or whatever. I don't know. It's some cheesy name. But in the last one, they were going around trying to catch this hacker. And the terminology was like palm forehead the whole movie. They've gone into our interwebs and they're stealing all

the webs. There won't be any webs left and it's just like, come on. Okay, actually we're getting a correction. I don't like this. People are correcting us while we're talking. Normally I like to pretend that we're always right. Yeah, every other episode we're always right. It's actually a mesh protocol

network. Ah, okay. So I listen to the Twit. You ever listen to Twit? This guy Leo Laporte or whatever, he has a pretty popular general tech broadcast. And he always has people in his IRC chat room, and sometimes they have conversations back and forth in the chat room or whatever. And then, yeah, I understand now why he sometimes gets detracted. Yeah, totally.

But it's good. It's good. Live feedback. Yeah, this is pretty awesome. So the other thing that we should talk about is the election. In case people didn't know, there's actually an election going on in the U.S., which some of you might not know or might not care about. But join us.

No, I'm just kidding. Wait, I'm not saying that. I'm totally kidding. But, no, it's super interesting. And the thing that's interesting to me is how every election, the technology changes, you know, drastically because the entire four years has gone by. And I feel this also with the Olympics. So, you know, as we talked about on that show, you know, we went from, you know, the Olympics only being on TV to then you had TiVo and you could record the Olympics on TiVo. That was cool.

And then it was starting to get on the internet like nbc had this deal with microsoft where they had like some of the olympic events there and then now with youtube this time broadcasting the entire olympics in certain countries yeah in certain countries yeah there's still policy problems but from a tech standpoint it's cool to see every four years like a massive leap like not just some quanta of leap like a gigantic leap i was watching katie couric street uh on a rebroadcast on abc being rebroadcast live on youtube talking about the number of twitter follow twitter users this year versus last election yeah and how that has ballooned you know a lot and i would say even twitter's what is the cliche,

jump the shark, right? Even me, I already see people don't use Twitter the same way they used to to have conversations, except on those TV shows and people are like, hey, Katie Couric, talk about me. Yeah, exactly. Hey, mom. And they have terrible filtering on that, by the way.

I saw one where it was like, it was literally like this guy was talking and the Twitter feed was scrolling by and this one guy was like i think aliens are like retarded and it's like had nothing to do he's talking about hockey and i was like what spammer yeah it's like you just detected like oh like this is a upcoming hashtag like start putting it in my messages i don't know yeah maybe how did they don't have like a person you would i i'm sure the like high level ones do yeah like presidential debate or whatever they probably did for that so but um they were talking about social media and i think

that is a good point like people hash this out and it's interesting to see when people post politically which some people like and some people don't like political posts on social network but it's interesting to see the engagement from not only like we said people in the united states but people in other countries yeah it's really interesting actually people are fascinated by our process here. It's not straightforward. It's very confusing. Oh, it's so confusing.

I mean, I tried to explain the electoral college. I had to explain that at lunch today. It's so hard. It was a nightmare. And it's like, I don't even, like, I mean, I understood it, the process, but I don't understand. People started asking questions like, why is it like this? Oh, I mean, the why is, it's fairly, I mean, it's straightforward to say why.

Like, you know, it's an alternative, but some people say it shouldn't be that way anymore. Somebody said why, so for example, if like like, let's say California has, I don't know. People are going to be bored about politics. They're going to be like, I thought we were over with the election, now we're listening to programming, throw it out, don't talk about politics. Okay, but we'll do one, one real quick.

Like, if California has like 50 electoral votes, I don't know how many it has, but let's say 50. If half the people voted one way and half voted the other, like shouldn't 25 go? Like the winner take all. I had a hard time like explaining like why that makes sense. So I guess in theory like it used to be that way sometimes. Like people could decide. And I think in some states it's still, it's like Maine is still that way partly.

But the, or in some states it's district by district. So it's not like you say, but it's like, so you have, so electoral college is the two senators plus however many representatives you have. And so the representatives are geographically located by a certain region and they can cast votes by whatever the popular vote in their region is. And so that's even a more highly divided way of doing it. But I think that the, I mean, the whole reason why it's winner-take-all,

the whole reason that Electoral College is kind of there, aside from some legacy reasons also, is to provide that you have a state like Alaska or North Dakota or something that are very sparsely populated, very few people, but they still have a lot of cares. It still matters a lot to them. So if you give them only a percentage vote like you get in the popular vote they essentially get no say and who gets elected a president even though they're a part of the united states arguably like other people are because there's a lot of land

resources there they might produce a lot like there's all these different measures of how important various states are and so this is kind of trying to be a compromise between every person is one person and like people are proportional to the importance of their state. Gotcha, gotcha. I'm probably butchering that, but that's okay. It sounded good. I should have used that argument at lunch. It would have made a lot.

Instead, I was just like, I don't know. So on to the news. Yep. You got the first news article. Okay. So I think this is pretty awesome. So I run Linux at home. I have a Windows machine that I use just to play games and mainly because my games are on Steam. And I heard you can run Steam through wine, but it's like, it's like,

I mean, you can whine about running Steam under wine. Yeah, exactly. Bad punss i'm sorry so so you can you can do this but it's like i had a windows machine from when i was in college and it's kind of like why not just just keep it for this right rather than deal with it but um steam is coming to linux very shortly and so this article is actually a press release from nvidia um in the press release, they talk about how Steam is going to be like, they're already testing Steam on Linux with their cards and things like that. So you can sort of insinuate from this article that it's eminent, the Steam.

I also heard that it was faster. Really? Like, I saw something. They were saying it runs better. Yeah, because I have issues with Steam. Now, granted, my computer is rather old because it's an old Windows box, but sometimes if an app or if a game on Steam decides it wants to update, instead of updating in the background, it'll just lock my Steam.

I can't do anything with it. Yeah. Maybe they fix the lock. No, it'll be great. I mean, it would be nice. A big hindrance to using Linux for everything is this, right? A lot of the applications and stuff just don't work very well or aren't supported at all.

And so as that becomes less of an issue. So I think things like also the Humble Indie Bundle, which we've talked about several times. And their big thing is it has to be cross-platform to be there. I believe that's one of their things. It has to be on every platform to be featured in one of the bundles. I think that's really good, having

developers address this. Also, as a buyer, if I buy that, then I'm less locked into what... It's not just DRM. Steam has some DRM issues. Now that they're going cross-platform, maybe some of that will be a little bit better. There's the DRM locked in, but even non-DRM, you can still

be locked in because of what the only system that supports the games or applications you have. But as more are available across platform, then I'm less locked in, and I can move back and forth how I want. DAN GALPIN- Yeah, imagine when Windows 98 came out, it supported USB. And Windows 95 didn't. That was one of the big changes.

And so, you know, if something like that happens again, and if all your games are on Steam, and now Windows 8 comes out, and Windows 8 supports some hardware that you need that, you know, you can't upgrade, you know, so if you end up in this position where, like,

you have to upgrade your computer, you don't want to be forced to upgrade into Windows. if you have the opportunity to upgrade into linux and keep all your existing apps like that's going to be very enticing you know yeah i mean this is a whole like shift in the way we purchase things so it went from scarcity right like i buy bread then i take that you know those atoms and molecules home to my house right like this is the scarcity thing and not to get into economics of it um because don't know anything about it but uh but it's just like now we have less and less of that like we buy things which are digital which okay i mean that's pretty straightforward but then things like netflix like i pay every month but i don't really

own anything if i stop paying i lose no matter how much money i've spent total like i lose all of that the day i stop paying paying And or the next month But yeah, so I mean I lose that but then you have these kind of hybrid things Like a steamer there's been video services where you purchase download things But they still have this DRM or this connection back to where if that company goes away You can no longer authorize your stuff to play you may or may not be able to get it kind of, you know, unlocked so you can keep playing. So you've bought a game,

but the game is not exactly yours to do what you want with. And even we talked about previously about the new consoles going to also try to do this kind of thing about linking or locking your game to your specific console. So it's not, if that one dies and you go buy a new one or you upgrade to the version 2 like you may or may not be able to take it depending on the good graces of the console developer and the game developer yeah so from like uh like security cryptography you

know point of view if if you're not hitting the internet so so if any security measure is just on your machine then you should be able to crack it. Like from a math standpoint. Yeah, theoretically, eventually, with enough effort, you could crack it. Yeah, and I mean, these passwords and things are cracked all the time. But as soon as you make it to where you have to transfer some information to and from the cloud, then you can end up in a situation where you can't crack the password, at least not easily. And so, yeah, that's...

There's a lot of problems. And I feel like it's one of those things like and maybe it's just but you have like the 99.999 percent separate from the like one percent but yeah we just have like all the people out there and there's oblivious to this they're just giving the monies to the companies and they don't understand the implications long term right yeah there's the classic thing like the the mother who bought for their kid like all this music on their ipod and then wanted to buy their kid a zune and like oh wait nothing works on here like well i don't under like why not yeah it's just confusing and like in some ways it's like well what are you gonna do like i mean that's you know caveat

emptor right buyer beware right but it on the other hand it's like you feel i don't know like we're not having a lot of like i like when things bring choice yeah so i think things like the humble indie bundle which is you know no drm or, or even Steam, now they're making available other things. I still am not 100% comfortable with that. But if they're bringing me value, right? So they provide an incentive for games which were no longer supported to become re-supported so they can be resold at a low value on Steam because of the distribution mechanism like that. Okay, I mean, there's a value there you know i may still want if it's if a game is the same price on steam and not on steam i'd rather

buy it not on steam but now that's even harder and harder because i buy it not on steam oh without the drm then well right well well that's the idea right like but you do because now all these things that are like also have some other weird thing that you've got to register with this thing or whatever. And it's like, oh, well, if I'm going to have to go ahead and do that for this, I might as well just have it on Steam where it kind of handles all that. Yeah, I got the flight simulator the other day, the Microsoft flight simulator, the new one. I think it's called Flights. It's, like, rebranded. Okay.

And, yeah, it wasn't on Steam, but it still required that I have this, like, Windows Live account in the cloud somewhere and all sorts of DRM. Okay, well, enough talking about that. So I checked out the Microsoft Surface, since we're talking about OSes. Did you really? Yes. So what's it like? Yeah, tell us.

So I was at the mall, and I was like, I'm going to go. So a couple of things to note. First of all, some people out there might roll their eyes. That was the busiest I'd ever seen the Microsoft Store. Yeah, and isn't it so... So many people. So at the mall locally... I don't know if it's the same one.

It looks exactly like the Apple Store. Well, and isn't it so... So many people. So at the mall locally... I don't know if it's the same one. It looks exactly like the Apple Store. Well, yeah, I mean, yes. But it's at this specific mall that I went to here in the San Francisco area, the Bay Area. It is right across, literally right across the street from,

or the mall aisle from the Apple Store. And it's crazy. It's like you can just like literally see out the front windows of one and into the other. And, okay, we're having some. Oh, go ahead. I'll sort it out.

Okay. Anyway, so you can see from one to the other. So you normally stand in the Microsoft Store and there's like very few people around. There's a couple of people, somebody playing Connect, the Just Dance game in the front. Oh, yeah, yeah. And you look across the Apple Store and and you see, like, people! Everywhere, people! So, but the Microsoft store does have,

it's pretty nice, like, going in there, you know, and it's reasonable. And when I was looking at the Surface, there was, like, a line, like, every Surface, they had so many Surfaces set out, like, so many of the tablets. We might have confused people. So many of the tablets set out,

and, like, every one, there was, like, a line of one or two people to, like, wait to try it after the person who was using it was done so we kind of walked around and i found like this other area that like oh okay like this was like to try out the keyboards but i don't really care like i just wanted to see the the thing so is it running windows 8 or was it so it runs windows 8 rt which is the arm based version of windows 8 so you can't just like bring your own applications, your old Windows 95 applications, and just run them. And when you are using it,

it's not obvious that that's actually a problem. Everything just seems to work just like when you can get to the little start menu. It's slightly different Windows 8 differences. But for the most part, it just worked, and stuff is there, and it was nice. But the form factor of it is like, we're in like stuff is there and it was nice. But you know, the form factor of it is a little bit strange to me. Like the squareness aspect ratio of it is a little bit strange to me, like holding it.

But maybe that's just because I have an iPad. So I'm used to like the iPad ratio. So it's just something different. The keyboard was, it's pretty nice to type on. So how does the keyboard like, give us a mental- Okay, so there's like a, I mean, most people can go look on the internet.

It's much easier to see it. It's slightly more rectangular than the ipad so like you know the back has a little kickstand that flips out so it stands on its own and then it has the cover folds down and has keys on it oh got it got it and then you can type on it and um they have two versions one that's like a touch typing and one that has like actual kind of mechanical keys oh so you get the haptic feedback yeah so so it's actually you know it was pretty nice there was a couple like instances where we were playing around so it's my brother and i were trying stuff like and it seemed like it wasn't registering our clicks but you know and i've seen some reports on the internet about people saying like that also happened to them like it kind of only half registers the click like the animation

plays but not the underlying feedback of what's supposed to happen. But also it's one of those things, like when I first got my iPad, I was completely like, ah, that's not working. Like I can't slide it to unlock. Like what's going on? Like it took me a minute to get kind of,

oh, you need to hold, like get just below the button or just above, like, you know, kind of what, you have to mold your brain to kind of work in the right way. So, you know, it was, yeah, it's pretty nice. You know, it's a, you can kind of work in the right way. So, you know, it was pretty nice. You can kind of, in some ways, almost, maybe

as a cop-out, you want to say it's a version one, right? It's the first thing that they've done. How does it work from a developer standpoint? Do you, I guess you build in Visual Studio or something? Yeah, I haven't studied it as much as I probably should have to start talking about it. I think there's essentially

Windows 8 SDK, whatever, equivalent, right? And then you essentially can build for either platform. And I'm sure if you're developing a new app, it's probably fairly straightforward to use the limited subset of things that are available maybe on the ARM process, right? So you can just develop for both pretty seamlessly. Cool.

But then there might be certain features that are only available when you go to the full Windows 8 stack or whatever. So that's pretty nice. So then I went over and tried the iPad Mini. So having the full iPad going to the iPad Mini. So I have a Nexus 7, and the Nexus 7 is much cheaper than the iPad Mini. It's $100 cheaper.

The iPad Mini is about the same size, right? So I guess it's about the same size as the Nexus 7, except that the screen takes up more of the real estate. Oh, gotcha. So, like, the screen is bigger. And the aspect ratio is slightly different. So you want to show us your awesome Nexus 7? Oh, this is my Nexus 7.

Oh, people on the audio won't be like, what's going on? That's the back of it. This is the front. Oh, pretty awesome. Oh, okay. There's the time. I think, oh, yep.

Time is pretty awesome. Time is pretty accurate. COLTON OGDEN Down to the minute. COLTON OGDEN Yeah. COLTON OGDEN Yeah.

So how does it compare? COLTON OGDEN But it wasn't compelling. For how much more expensive it was, it was just like, you know, if you, so for me, the first world problem, is that what they call it, right? Like I have an iPad, I have an iPhone, I have a Nexus 7, I have a laptop, I have a desktop, and I have all the spectrum. It's like if I just take like iPhone or I've used an Android phone before and this also,

you know, fairly equivalent to me, you know, just Just some subtle reasons why I chose one over the other. And then having a tablet, which I do think for when I got my iPad, which has been almost two years now, is that at that time, I felt like it was much better than the competing tablets. Now it's a little more even. But then having that middle one is kind of like, eh. so if you're gonna get a middle size one so i use it kind of like as an e-reader right instead of a nook or a kindle or something um that middle

size one is nice but you don't want to pay a lot yeah because you're only going to use it's like if i'm mobile mobile i'm using my phone if i'm relaxed you know at home i'm using my my big screen my tablet is much better for web viewing. So it's only like in the car, at the airport, right? Like a few, you know, like taking it around to work meetings when I don't want to have to carry a big, like that kind of stuff. The thing is the middle one fits. The thing that's kind of funny is I've sort of seen over the years our transformation.

So I remember I was thinking, oh, I'd never really use a smartphone. And now it's like I chop off my arm before I'd give up my smartphone. And it was like, oh, you know, an iPad. You remember debating. I remember debating getting a smartphone. Like, oh, I want a paid data plan and have the internet hot. So expensive. Will I use it?

Oh, yeah. Yeah, now it's like every minute. And then I remember when we were talking about the iPad on one of the very first episodes, we were like, oh, I don't know if the tablet, like, really makes sense. And now it's like we use tablets about the iPad on one of the very first episodes. We were like, oh, I don't know if the tablet really makes sense. And now it's like we use tablets all the time. It's really a testament to how audacious you have to be as a hardware developer. You have to make something that nobody thinks they want. And then convince people they want it.

Yeah, and then have people's lifestyle adapt in a way that fits that device. It's really amazing that any of this stuff gets as widespread as it is. Yeah. Oh, so people of the podcast, listeners of the podcast may remember I have several rants against Kickstarter. I'm not as pro-Kickstarter as a lot of people. I thought you were super pro-Kickstarter. Really? I get accused of being like anti-Kickstarter.

I remember you ranting about this. I'm I get accused of being anti-Kickstarter. I remember you ranting about this stuff. I'm like kind of, I straddled the fence. So I did see, which you know, it's anecdotal, right? So it's not proof. But they had this really awesome iPhone dock. I think it was called the Elevation Dock or whatever. It was this really slick machined iPhone dock. It's like really easy to put your iPhone down in and it would charge it. It had like an audio out port as an option

It was just really really nice like really high-end design, right? Yeah, so you would have bought it like over 12 months ago, and it was supposed to ship I'm gonna mess with the exact days on it in front of me, but like April right So it didn't end up shipping until like only like four or five weeks ago So like let's call it five four or five months late six months late on top of it You would have paid a fairly good price I believe like the cheaper ones were you know still like I want to say like $75 type of range I'm probably wrong on that but um that kind of range as pretty nice, but all is nice, right?

They didn't end up shipping which just could have happened at any time, but they delayed they didn't they got way more orders And they thought whatever right they delayed it was a very profitable Kickstarter They they ship sold a lot of them. And then right before, when they finally shipped, some people got their Docks literally the day before they got their iPhone 5s, which changed the Dock connector. Oh, no. Are you serious? So some people were getting them who bought iPhone 5s. Well, arguably, if you're buying a nice Dock, you're a, you know, you really like the iPhone. You're using it. You probably upgrade. And then, like, you're buying a nice dock, you're a, you know, you really like the

iPhone you're using, you probably upgrade. And then, like, you're buying it and it's, like, obsolete. It's like, ah. So I saw that. I felt so bad for this. Yeah, so they changed the dock connector in the most recent one. I wonder why. A lot of reasons, but we won't get into it right now. I mean, it makes sense. I'm not faulting Apple for doing it or whatever.

But just another one of those Kickstarter things. You're not buying it today. You're pre-buying something that's going to be made then it might slip and then you finally get it it's like it's already obsolete versus like you know if you knew you were going to buy the iphone and you saw that in the store like let's say you know the target or walmart right you were gonna you knew you were gonna buy the iphone 5 you might wait before buying it because you know you're gonna buy the new iphone and and there's rumors that it might be changing. But back when people bought this thing, there were no rumors that the iPhone dock would be changing.

Yeah, totally. I don't think so anyways. You have to invest so far in advance and technology changes so quickly. I'll still go for board games though. Board game kickstarters? Pretty timeless. Well, I could get into some negativity on that too but I have the one

I ordered it did turn out well and it is nice there you go which one did you order what is it called for the wind I haven't heard of that one for the wind so it's like a abstract tile laying game about pirates ninjas zombies amazing and one other what is the monkeys

amazing I think and so it's the tile abstract thing and you play, each one has like a special effect and allows you to flip over the tile or not. And so it's kind of, it's pretty good. It was a good deal. So I've enjoyed that. All right, so is it time for viewer questions?

I think it is. Let's get to viewer questions. So we had a good response. We had a lot of people sending questions. Yeah, yeah, thank you guys for the questions. Pretty awesome. You have a lot of people sending questions. Yeah, yeah. Thank you guys for the questions. Pretty awesome. You have high quality, great questions. And sound effects, pretty awesome. So let's just get straight to the mailbag.

So we'll kind of mix up some questions from the audience and some questions from the mailbag. Okay, yeah. So if you have a question and you're on the hangout right now because we do have some live listeners so we have like three or three people four people um and if you have a question just type it into the chat box and uh we'll either like have you read it out loud or read the question yeah definitely so yeah type your question out in the chat box plus that will keep us from answering your question with like without knowing it's you and you have to ask the question again that's super weird so yeah if you have your question go ahead

and like copy it from your email and like paste it into the into the chat from the hangout so that we know not to ask it because we have actually all the questions from the email here so the first question is from and I really apologize if I butcher your name but I I'm going to try my best. It's from Marco Aurelio da Silva Saraiva. And he's from Brazil, I believe. Yes, I got it right. It's a very Brazilian name.

So he's from Brazil. He says, listening to you guys every episode, and it's been amazing. Thanks a lot. So he wants to know how internet crawlers work. So how web crawling works. So I guess I could take this one. Okay. So the way most internet crawlers work is they typically start with a seed set of websites.

And so with most of these crawlers like google and bing and things like that you can actually submit your website so you can actually tell like google bing yahoo whatever like you know i have this website and you should add it to your list of seed sites so it goes through these seed sites and and for each one of these sites, it scrapes all the HTML, puts it in a reverse index, which I'll explain later, and then also finds all of the tags and then adds it to this queue. So in the beginning, the queue just has the seed websites in it. So that would be stuff people submitted on their own, but also like hub sites, like Wikipedia or sites that link to a lot of other sites. Yep, totally.

So yeah, so it'll have like Wikipedia, AOL.com, Yahoo.com, et cetera. And so you can imagine this queue that has all these websites on it. And the system will go, pick one of the websites off the queue, and then it'll store that HTML somewhere and any images and things like that and then for all the links it'll add that to the back of the queue and then it'll keep pulling web pages off and then adding more on you can imagine this queue getting really really really big you know of course there's all sorts of what's called stemming which is for example

let's say you're but you know sometimes you go like on youtube for example you go and you search for something and there's like a million results and you could just keep paging through right so a stemming will know that oh this is really just millions and millions of these like results but it's it's just kind of like a search query and i don't really need to scan these like it doesn't really make sense for me to scan these. So it'll stop there. So it's not going to go forever. The stemming will make sure that you'll eventually run out of pages.

So this queue keeps growing and shrinking. And then finally, when the queue's down to zero, you have- DAN GALPIN- It also marks. Because you can have it with cycles. So it has to mark and know who it's already visited. DAN GALPIN- That's right. So when something gets pushed off the queue, it gets put onto some key, some hash set somewhere,

which just says, yeah, this is visited. So eventually the queue gets down to zero. And the other thing is that this is distributed, right? So there's this queue, but you're not going to each one of these websites like on a single computer you're pulling things off of the queue and then sending them to other computers and then they're like going pulling all the images and stuff and then they're just like telling you like what new things add to the queue so you know it sounds like this would take forever but if you have like a ton of machines you know well no i mean it basically i mean it can grow faster than

you could process it in some ways. That's true. Depending on, like, what your scale of processing is. So you just limit it to some time or, you know, probabilistically you're going to visit the most popular websites will have the most links. So you'll kind of get them early on. Yep. And the other thing, too, is you don't want to completely cream some, like, I run my personal website where i have like hobby projects and like that well like some search engine like bing or something like hits all of my web pages all at the same time and like downloads all of the images like it'll completely crush my computer right so

they're pretty clever about you know you know like like oh i just went to this computer let's put this one back in the queue don't process it right now and so there's all sorts of clever details there okay so now you have a ton of these pages and you have like think of it as a huge map where the key is the URL and the value is all of this HTML and what you want is for somebody else to say type in so in this case Marcos he actually wants to make a magic the gathering like like subnet so yeah he actually wants to make a Magic the Gathering subnet. So he wants to have all his Magic the Gathering data that he's mined from the internet. So if you were to do this, Marco, and somebody types in, I don't know, ethereal dragon. I'm trying to remember my

Magic the Gathering cards. Or phantom lancer. I don't remember. Anyways, so if someone types in one of the Magic the Gathering cards, you would want to pull up all the pages that have that name in it. And so that's what a reverse index does. So a reverse index is essentially a way where you can put in a word, and it'll give you all of the values, all of the documents that have that word in it. And there's a whole bunch of crazy information theory, information theoretic things that you can do you can actually um if you have the documents arranged in such a way like if all the documents for one server are together then you can actually with 99.9 accuracy um create a reverse index without

scanning all the documents so you know it's true that like this guy's website on antique cars might have magic the gathering, but it probably doesn't. And if you scan five pages and it's all antique cars, and you know there's like a very low correlation between that and ethereal dragons, then you can just skip his entire website. And so building the reverse index

is all sorts of cleverness there, too. So there are also, I mean, people have already written web crawlers that you can use in various languages, which is really nice. Yeah, totally. And also, and I'm trying to find it now on the fly, we didn't do a lot of prep going through the questions

and answering as well as we could have. But there's also, if you search for like, web crawling and Amazon EC2, there are articles where people have written about how they do it. And now you get into cost, right? So you spin up an EC2 instance, so you might have to pay for that. But there might be also data stores.

Other people have already done the web crawling part, which is intensive and expensive. And you can essentially just use the data. So they'll create some data store. And then you can just process that data store. So at least you save yourself half the effort. CHRIS CARTLAND- One that's really great is called Common Crawl.

Highly recommended. Basically, it's a totally open source reverse index of the internet. And so I think it even has images and things like that. So Marco, Aurelio, check out Common Crawl. And that way, you don't have to do that part. You can just take advantage of the index they've already built.

MARK MIRCHANDANI- But inevitably, this is one of those things you find out is really hard to do. DAN GALPINI- Yeah. It's super, super hard. MARK MIRCHANDANI- It's actually super, super hard. And if you look at Bing or Google or Yahoo, how much money they're spending trying to do this quickly and accurately, because websites change, and they go down,

and links come up, and links go down. know it's just it's a really really big mess and so a lot of time and money is spent by really big companies to do this really well so depending on what you try to do sometimes it is just is not feasible for an individual you know without a good profit reason for doing it to actually do it yeah i. I mean, you know, I think they said on the Common Crawl website that just going through and looking up the number of times a word is there, like if you want to see how many times the word does on the internet, that'll basically cost you like a hundred dollars, like an Amazon easy to use it. So, um, this would be an expensive hobby, but, um, it can be done. It can totally be done. And it can be done in a matter of days, which

is pretty amazing. So Common Crawl has all their source code available online. So I also believe that Sebastian Thrun, am I saying that right? I think we've talked about him before on here. He does one of those massively multiplayer? No, no, massively whatever online learning things. Oh, oh.

And so he did two, I believe. One in how to create create a search engine and one in how to create an autonomous car it's like you know the the kind of uh the background behind that and i believe that one about creating a search engine i would assume talks about web crawling at least at some level so that might be something else to check it out i'm trying to bring it up now and i'm not seeing it exactly um but yeah so i would definitely check out um if there's like oh udacity is the name of his platform yeah that's right and so i believe one of his is like how to write a search engine i don't know if it is applicable to this or not but i um i would check it out because that might have a lot of really good information okay cool so let's take one yeah let's take one. Yeah, let's take one from the... Here we go.

From our live viewers. And I'll read it off and you answer. Okay, all right. Oh, yeah, this is a great question. So this is from Aaron Chapman. And let me see. I actually have this Chapman. He's from Eastern Alabama.

So hey, Aaron from Eastern Alabama.ama um cool so it says i love everything apple i've written five iphone apps on the app store study objective c and coco touch constantly and i find it difficult to continue doing this because there's no one near me that's interested in programming i'd be happy to learn other languages but there isn't anyone near me so so the general gist of the question is sort of is, how can you get into a hobby like software engineering, programming, this kind of making apps, these kind of things, when nobody in your physical vicinity is into it? Yeah, so there's a couple of different ways. So one is these massively online learning platforms often have forums.

And so you already know that people in some, if there's one that's in that domain, right, that they're like iPhone development, I'm sure there's probably some, you'll probably find that there are people there who are also posting in the forum and just kind of like start chatting with them. If you have ideas for things, try to say like, hey, you know, I have this, I'm sure there's an area devoted, like, hey have this idea like anybody want to you know work on this with me or whatever the other thing i would recommend is um trying to find open source people doing something similar totally so for mobile development it's a little hard um because

the open source thing gets a little touchy depending on exactly what you're trying to do yeah like most open source people want to write tools and libraries and things like that, whereas like most apps are kind of end products. But there you might be able to find something in that region, so that would be something else. And then another thing would be like if depending on what you're interested in developing and then what you're trying to learn. So if you're trying to get connected with other people kind of to just work on something together, which is excellent, you know, resume builder for college, you know, admissions

for getting a job, you know, you might be able to go to something like if you're interested in game programming, going to a game programming forum and finding people who are on a project to need help. Now you have to be careful is people there may not be as devoted as you are, or maybe more devoted than you're willing to be. So you might have to do some kind of, you know, be just be up front about like what your level of, you know, interest is and how much time you'd be willing to put into it. But you might be able to find like a match, right? So a lot of people are there looking, they have ideas and they're looking for developers. That might be a good match. Or they're, you know, developers looking for other developers, right? So

that's another thing to explore. And depending on what niche you're in, yeah, there could be a lot of people interested in that kind of thing. Yeah, of yeah totally i mean so a little anecdote about this is i um when i did a lot of game programming especially when i was younger i just kind of went on my own i didn't really like go to forums didn't really pay much attention so because of that i wasted a lot of time you know like i made it wasn't wasted because it's kind of valuable, but personally, but as from a purpose of making something, it was kind of a waste. Like I spent all this time, you know, dealing with writing things to a file and reading them. Whereas like I could have just used protocol buffers or thrift like we talked about in earlier episode. And I just, I just wasn't a very like efficient programmer.

And on top of that, the even worse thing was that i made things that people didn't want to play and it's like and i had nobody of myself to really go off of you know and so and i'm not even your mom even your mom wouldn't blame no even my mom yeah she started and she was like no she's like she's like this isn't for hardcore players so i'm out she uh but no so it's like i'm not tarn adams i can, so it's, like, I'm not Tarn Adams. I can't just, like, make genius, like, by myself. So it turns out that when I started getting into, like, 3D modeling and more artsy side projects, I got together with, like, Pixel Junk and some other forums,

and I was kind of, like, sharing pictures, and they gave me a ton of feedback. Like, oh, you know, you're doing this totally wrong. Like, you're using this kind of, like, I was using an iconosphere instead of a uv sphere and it became really hard to texture and they're just giving me tons of feedback and that is incredibly useful like now i feel like i can make 3d models like really well interesting yeah that's a good point put your work out there so i was going to mention something similar as well like start a blog right like yeah it seems kind of weird but i mean if you start and and then just start posting like to various you know people that you admire look up to or other people are doing something similar and say like hey you should check out this like

what do you think you have any comments could you like look at this and and critique this for me or whatever right like i mean it's hard to put yourself out there to you know be criticized but you know just remember you're just like learning right like we're all learning like we have to remember that there are always somebody better than us yeah totally and they are on that forum no doubt so yeah i mean just going to a popular forum say hey i made this like anybody have feedback and you'll get some jerks on there right like people like ah this is terrible i mean you got a deal that's the internet right like exactly that's what happens to me every time i try to play a game online oh that's the worst you worst. You're the worst! I actually, I was playing Defense of the Ancients,

which is known for having, like, big-time, like, trolls, and other trolls and things like that. And somebody told me the other day that, like, I should kill myself because, like, I wasn't doing well in the game or whatever. And, like, you know, of course it doesn't matter. But, like, I just, for some reason it, like, resonated. And I thought about it, and I was like, you know, like, this person, like, they'll just say absolutely anything. Like, they don really like, it's not even like, I don't think the person,

like what he's saying is even going through his head. You know what I mean? Like it's going through a part of the brain that just like blurts out, you know what I mean? Like doesn't actually think about it. That's a shame. So you'll get, you'll get all kinds if you go to forums, but you know, a couple of forums I've been to um i've been to tig source tig source which is a game developer yeah yeah and they also have artists uh forums if you're into games yeah that would be also really like the independent but also the ones where they do like 72 hour thing right like oh yeah develop a game in 72 hours right yeah yeah yeah totally and then you submit it and people review it like they have a judge panel or popular

vote or whatever and people will give you feedback. Yeah, and that's the other thing too is you realize, wow, this guy made an entire game in a day, and I like got my code to compile. Ludum Dare. Is that what it's called? Yeah, that's right. Ludum Dare? Yeah, check that out.

So, yeah, you learn a lot of really interesting things from that. So, yeah, hopefully that gives you some advice, Aaron. All right, so I had another question here. Okay. So we'll try to respect the names that are in here, but hi, my name is Brian from the Detroit area. Nice. I love the podcast.

Okay, maybe we won't read. It sounds like really like self-patting on the back when we read the... Quote, unquote, air quotes, Brian, a.k.a. Jason. No, we're just kidding. Okay, so his question isn't really about technology, but he's asking for our opinion. So he's in high school. He's got a good GPA, and he's got several friends

who aren't really doing well with all the pressures of doing school. And he's doing research, and he discovered that the current school system is meant to prepare the students for basically a factory job. So this is like a – I've seen videos about this on the internet. Like this was meant to be a trade. We would go to work in a factory, get a pension, retire, like this whole thing. And then that paradigm is changing. The nine to five.

Yeah. And they were meant to learn this way because this is the way the factory was. And now we're knowledge workers. This is different. You'll hear that term. So it says this worked perfectly for my grandfather who worked in a factory. But seeing as there are no more factory jobs, at least in the United States,

should it still really be set up that way? And should schools change to follow what occupations people will actually follow? So this is something I'm actually really interested in. So having a young daughter, I'm interested in what should I save for college?

College is going to be really expensive by the time. Is college going to be different? Should I just not bother? What should I encourage her to do? How should I help her? And I think there's a couple of things you have to balance here. Well, I guess I should have given this to you to answer, sorry. No, no. Because I read the question. We'll trade off. It's okay. So I like this. I'm glad that you're passionate about this question. So there's a couple of things you have to balance. So one is when you're young, you often don't know what you want to be when you grow up, right?

So I really don't really know what I want to be when I grow up to some extent. But, you know, younger you are, if you start too early, you're siloing people, right? Yeah.

And then people kind of get too far into something before they realize it's not what they want to do. And then it's hard to switch because if you switch, you're giving up years of, you know,

experience in that thing that other people have and you won't have. And actually, so having a family from Europe, most of my cousins, my parents are originally from Europe. I noticed that they do this much earlier in Europe. So very early on, even as early as in what would be middle school over here, you know you're you pick a discipline and i noticed as a result that the um the a lot of my cousins from europe who also study computer science because we're kind of like a nerd family i guess it's in the genes but um they they have they're extremely good at like math and theory and computer you know science but they don't really have like that creative like the very broad creative like they don't know have like that creative like the very broad creative like

they don't know the things that we know about like psychology so see all these like gen ed i think there's what it was called when i was in college they don't have the breadth that we do you know and so i think it's actually just a great thing that we don't specialize early so there is still like another aspect of it which is so you have that right it's like specializing and there's an issue you have another which is the way like different people learn different ways right and so a lot of this is hard because the it's a chicken and egg problem right so uh companies are set up to hire college graduates or you know take like a you know a highly skilled knowledge-based job like going to work in silicon valley for a large let's say yahoo you're going to go work for yahoo

and um there's a programming throwdown maybe we'll pick a programming company software company so you're going to work for yahoo is set up to hire people out of college right right you know even if they've been in the workforce a long time like they'll still assume they went to college at some level or have a really good professional background to hire them. And then colleges expect people out of high schools and they expect them with a certain amount of level. And high schools have to expect middle school and on down. But as you're growing up, if you're going to go to college and college is a certain way, the high school has some obligation to help you prepare for a college that's of that type so you know things like oh people might learn better by you know um different methods or going online or playing games or watching videos but if you if that's not an option for you when you get to college now it's

becoming less and less of that's a problem but if you can't learn that way and get what's accepted of society today the high school has failed you in some way right because you're not prepared to go into college and learn the way like learning how to learn that's what you hear it and it's society today the high school has failed you in some way right because you're not prepared to go into college and learn the way like learning how to learn that's what you hear it and it's cliche but like oh you're being taught not the skills but learning how to learn so that when you get wherever you're going you'll know like how to figure out what to do yeah and so you have to balance that that other aspect of it and you know some people may learn really good in the current system and some jobs are necessary to do that way. And it is also important that we educate a large

population to a certain level, like, you know, to have a good, learned citizenry. And so that's important as well. But I mean, going forward, I think we are going to see changes, we're going to see shifts and how education is done and what's going on. And until then, we kind of just have to sit tight because big changes, you know, are also dangerous because although you may really like the idea of, you know, just going out and doing your own thing instead of going to college, you know, at this point, like, you could do well. Like, I know people who have done well doing that,

but you can't really recommend that because it's not – we don't have a lot of statistics on how well people who do that do. That's the thing. And, and, and the thing is, is, you know, with the exception of people like Steve Jobs and things like that, but for the vast majority, um, you know, chances are when you finish with your schooling at whatever level it is, you're going to apply for jobs. Um, now if, you know, if you're this entrepreneur like Steve Jobs and you're willing to go broke broke to try and start a business, things like that, then it's sort of a little different. But for most people out there, even Steve Jobs worked at Atari for a while before he started his own business. And he was in college, in and out of college during that time.

So a lot of businesses, they get thousands like yahoo probably gets thousands of applicants every day maybe maybe millions right who knows and so it's just not worth it for them to to to take a chance with somebody who like let's say didn't graduate high school right so it's just it's just not worth it because they know that like they would to – think of all the extra HR people, right? If they could throw out, let's say, 10% of the resumes, then that's probably going to save them, what, like $60,000, maybe $100,000. So we had this conversation. The problem is, and this is depending on the size of the company,

but if you take a normal medium to large company, it is so hard to actually fire somebody for so many reasons. So it is really really really risky to hire the wrong person right right and so what companies do and it's to a shame they miss a lot of good we were having this conversation the other day about test anxiety right yeah so like some people just have interview anxiety and it's a shame but that company just can't there's another person where that one came from right right and somebody may look down the line and say that person went on to become the next steve jobs like you should have hired them

maybe but at the time they're probably making the right decision by not because totally you know they got to go with in some ways they're missing on the best worker because they have to take a worker that they know will fit yeah um and that's changing some with you know remote working and you know people able to do other stuff. You can set up a presence online, which you couldn't do before. Yeah, you can prove, like, here I have a portfolio of stuff I'm demonstrating. But at the same time, if you have that much initiative to not go to college and to do really well and still learn, you know, you probably don't want to go work for one of those big companies anyways. Yeah.

But I'm a big proponent of apprenticeship. Like, I think that would be really awesome and a good thing right which is basically instead you would go to college but college would be much more limited maybe online general education type a little bit of specific but then instead of all that money being spent you know to be taught you would go apprentice with somebody so people would offer themselves up for apprenticeship and they would be trained as apprentice mentors. Apprentice. No,

I think mentor. I don't know what the master, I think a master, maybe master. Okay. Anyways, whatever. Right.

And so like you would be able to find one and like, Oh, I want to be a software engineer and you would pay him. Why? Because he's going to teach you some of the secrets. Yeah. He's going to be less efficient because he has to teach you, but you're going to work alongside him. And in some ways I think that would be better. But but in some ways if you go back and look at what apprenticeship

was like then maybe we ought to be careful and like indentured servants and well basically you you know and uh so i read read a little bit about people becoming like uh chefs and so like you start off and like you're doing like that you're like scrubbing mushrooms all night long for like weeks then you're like you upgrade to peeling carrots for months. And then you get to dice something. And you keep getting yelled at by the chef and working horrible hours. And you do this for years in the hopes that you might get a chance to demonstrate yourself. I don't know.

That's better. Yeah, that's pretty rough. Okay. So there's a balance, like everything. And we need to explore options. But I think for now, I think in the short term, I don't see any big, big changes in the way companies will do hiring and stuff. And the other thing, so the email seemed to indicate specifically, like, schedule. Like, I remember, you know, when I went to high school, it was like when I was in high school.

Walked uphill. So this is no joke. I had to get up at 4.30 in the morning to catch. So real quick anecdote, I went to this kind of special school. The way my area in Florida was set up was they had special schools for people who had special interests. So my interest was engineering, so they had an engineering school. And anyone within like

i think like 50 miles or something could go to the school for free they would bus you and everything like if as long as you had like certain aptitudes so i went to school that was super far away the bus only picked up like six people and i had to get up at 4 30 in the morning to get there by seven right it was crazy and you know i've i don't think i've ever woken up at 4 30 since high school like ever um so it's true that like the schedule you know starting at seven is kind of you know as computer science we were kind of used to just getting up whenever starting whenever and and you know it's like and then going you know working our time it's hard right maybe i'm getting older it's self-control right like it builds character but yeah that's the big thing is i think that it's a combination of two things one is school is as

much a social like like learning platform as it is a way to teach you algebra and geometry and these things so you know the fact that you're all in it together really means something and i got a lot out of just being in a room full of people who are all doing the same thing. So I felt like that was a great thing. One thing that I'm thinking of right now is I made a calculator program, which could do the quadratic equation and things like that. I think you told us this story.

But yeah, go ahead. Oh, yeah. Real brief. And then the teacher. Maybe you just told me. I don't know. Possibly. So the teacher, she realized we were doing this.

She realized I had given the program to everybody. And she made us all wipe our calculators before every test. And then of course I wrote, I wrote that program again. And then I wrote a program that looked like you were wiping the calculator, but you weren't. It was pretty all little, little arms race going on there, but you know, it was cool to get everyone in on it and something fun about that. And so to do that, um, everyone has to be there at the same time. And, you know, as parents now, we can start to understand, or at least you as a parent, can start to understand how hard it is to, like, work around your kid's schedule and, like, take them to school and get yourself to work and things like that.

So, you know, high school schedule, the whole school schedule, as a receiver of that schedule, it sucked. I hated getting up early and things like that. But now I can sort of understand why it is that way. And, you know, it's Brian, you know, it really sucks getting up early. But then you can go to college in, like, did he say what year he was? No. No, but in at most three years from now, you'll be in college and you can set your own schedule.

Yeah, no, I mean, I feel it's like the angst when you know something better is coming. And like you can see, like we can see now, colleges will change. Like they'll change. But you're not sure quite how they're going to change. There will be some people who will take bold steps. And down the road will be, you know, praises heaped upon them as pioneers but there will be a lot of people who try crazy things and kind of fail and you'll just kind of forget about them but you know their lives might be is the what is that called i don't know there's like a

name for it but basically like the personal risk to you is like very high so like if a huge amount of people try crazy things some people will succeed and then you have you know survivorship bias like those are the people you remember but there'll be a whole bunch of people who fall by the wayside as kind of crazies totally you know so there's a risk but yeah we feel yeah i remember that yeah it was rough it does get better all right so here we go you want to read this one uh sure this is from joshua brown uh and unfortunately josh had to go so he's going to be watching this later. What's the word called? Post-talk? Or post-talk? He's going to be watching

this later. But his question is, I'm just learning programming using your recommended Learn Python the hard way. My goal is to integrate Circos into Solus OS as a file manager. What sort of language should I be learning

after I master Python to accomplish this? So I actually looked this up. Oh, okay, good. I was about to say. No worries. So yeah, I found this question kind of interesting because I didn't know what either of those two things were. So Solus OS is a Linux distro forked from Debian.

Okay. So effectively Debian. And Circos is a pretty cool visualization library written in Perl where you give it some XML in a certain format. It digests the XML and produces some really beautiful pictures. Yeah, I'm on the page now.

This is cool. Yeah, so the short answer, and then we can always jump into more detail, but basically the Circos accepts XML, and so you can use any programming language you want. So there's XML writers for just about any language. Don't try and write your own XML, like write the open tag, write the greater than sign. Don't do that. There's XML writers.

If you're using Python, which you said that you're studying, you can totally use Python. There's an XML library you can import. And it's pretty awesome. It lets you write XML. I've used it before. It's great. You can write your XML,

and then the Circos program can digest it and produce those awesome graphs. Yeah, I mean, I think part of, like like learning programming, and this is something I struggle with, but it's like struggling to do things in the programming, like the thing you know. So everything's a nail when all you have is a hammer. Yeah, right. Right. And part of what we talk about programming through now is learning like other languages,

but go for it. Like try to, when you're learning a language, part of it is going for, if you only know one language, like trying to solve something before you're learning a language, part of it is going for, if you only know one language, like trying to solve something before you realize like, oh, this is bad at this. And then you know what to go look for the replacement. Like what characteristics of the replacement should be. And if you know multiple languages,

you can't learn what the pros and cons of that language are until you try to do something and kind of realize like, oh, it doesn't really work for this. Oh, it's really good at this, but really bad at this. Right? And so, you know, I think that's like part of like the learning is like you kind of just have to go for it and see how far like see where you'll get stuck because until like i haven't done this before like i've never you know integrated these two things together it sounds kind of cool like when i look at that website it looks it looks really nice yeah josh real quick talking in

if you do this let us know we want to see. And if it works on Ubuntu, even better. So, yeah, I mean, like, I think part of it is, like, nobody's done it before, so you kind of got to try something and then ask for help, right? And then be like, oh, like, how do I do this thing? Like, oh, Python's not good for that? Like, oh, okay. And as you grow as a programmer, part of that is, like, doing that sooner.

Right? So, like, you know, you've talked about this with some of the machine learning algorithms, especially AI stuff, about pruning that tree. When you're playing chess, understanding this path of plays goes nowhere good. I don't need to go process the whole thing. Part of it is learning as you program. That's heuristics. Like, oh, hmm, this language looks like it supports regular expressions really well.

I don't care. I'm not going to use regular expressions you know or like oh this this thing doesn't isn't performant it doesn't run really fast but that's okay because i'm doing something only once like i only need to do this thing once so yeah that's not a huge thing for me yeah totally totally yeah but that's a good question though yeah and joshua brown uh was from los angeles he said okay pretty cool getting questions from around. Yeah, but that's a good question, though. Yeah, great question. And Joshua Brown was from Los Angeles, he said. Ah, okay, cool. Pretty cool getting questions from around the world. Cool, and that looked like open source projects then, right? Yeah, both of those were. A great way to get involved.

It's really intimidating at first to get involved in open source. It's always overwhelming and you kind of want to quit, but you just kind of have to stick in there for a while, try it, and eventually you'll start to make at least a little progress and then yeah the cool thing that josh is doing is that he already has something in mind that he wants to use this library for so you know if it turns out that oh uh the library doesn't work when i give it you know file names because file names have colons in them and the library crashes on colons well he, he's going to be really compelled to go into the open source library, fix the colon problem, submit a patch to GitHub where this is located,

and get them to fix it. And then you have a great feeling afterwards. It's like, I made a contribution that made this library even better. And so it's pretty awesome. It's a great way to sort of get started. Yeah, so as a slight segue, this is actually really interesting. So going from one company I was working at in Florida, which didn't do that much with open source stuff really,

and didn't encourage people to kind of modify the source and change something to back to a company where it's actually been slightly a struggle for me. I've been criticized a couple times and also find other people doing it just blowing my mind where it's like oh the library doesn't support this this thing doesn't work recompile it change it submit it back you know like fix it like where are you like just changing what you do or working around it just

fix it and it's like what yeah exactly are you crazy like oh and if you don't fix it at least let them know like you should tell that open source project hey i would really love to use it this way and like you know you guys should fix this yeah um yeah that just yeah yeah it's pretty wild that's a good thing about working i guess out here in silicon valley is like people are more open to those kinds of things yeah totally i mean i you know silicon valley has this thing where you know there's many startups like all these companies like twitter was a startup with like who knows three four people and then it grew right all of these companies started off very small and so they're sort of indebted to these open source projects otherwise you know they a small

team wouldn't have been able to start something that investors could appreciate so because of this indebtedness there's like this philosophy in silicon valley that you give back now you keep a healthy ecosystem so that the next startup can get going, right? And it's probably like that in other places besides Silicon Valley. Oh, yeah, totally. That's just, you know, I can't speak to other places because I don't know. Yep, yep. So do you want to read this one?

Sure. All right. So, hey, guys, love the show. Okay. Was this you again? You wrote all these questions. So this is Mike from Virginia. All right.

So, hey, guys, love the show. Your discussion about the automated paper scanning to detect cheaters reminded me of one of my friends that is a teacher for an online school. She told me that they were looking to expel some of the students for plagiarism. They suspect the students are submitting the bare minimum to remain enrolled so they can collect the student loan money. Unfortunately, the teachers have to waste time dealing with these creeps instead of the actual students get less attention girl yeah i've heard of this over and over again from teacher friends

and stuff really i've never heard of yeah basically like it's really hard to expel students and you spend way more time than you can really merit and so like even people who are doing deliberately bad stuff or cheating or whatever like it's almost not worth your so let it go this is sort of digressions i'll keep it short but the same is true with renters so i have a friend who's renting his house um to someone else because he moved and um the person just stopped paying their bill like stop paying their rent and it turns out that it takes like on average something like over a year to evict somebody who's not paying their rent it's just absolutely insane now like if they're if they're doing other things like if they're drinking or something then like that can expedite

it but if they especially if they have a lawyer or if they know the system pretty well it can take over 12 months to evict somebody who hasn't it's's just insane. Yeah, it's crazy. All right, so there's a question. Do you know of any free open source methods to sync mobile data between mobile devices, more than one, and a remote server? He's using Air Mobile App and Microsoft SQL on the server, but he's open to other options. Seems like everything requires big time customization

to do conflict checking. So it sounds like what he's trying to do here, I would assume, is for an app he's developing. Right. So trying to have a remote server and synchronize. So I've not really done this before. So let's talk about files first. If you want to do files, there's a number of options. There's Dropbox, SugarSync, these kind of things.

And those are pretty awesome. You can actually put a file in your Dropbox and it doesn't exactly just show up magically in your phone, which I wish it did. It's actually a big complaint to have with Dropbox. You actually have to go in your phone and say, hey, sync these files. So if you add a new file, you have to,

whoop, lights just went out. Okay, they're back. So you have to actually like sync those files but once you've checked the box for the for a file then any changes you make on your phone like immediately go to the to the computer and so this this works really well for a lot of like if you want to make make notes to yourself things like that with respect to actual data this is kind of tricky, right? So the way most apps work is the phone might keep some local cached copy, but then it frequently hits the server, right?

And so you might have some server that has one database, and then as the phone needs different pieces of the database, it can grab those, and the phone needs to make updates. That's how most apps work. In your case, it sounds like you want to keep the entire database on your phone and on the server and keep the transactions in sync between the two. That's going to be pretty tricky.

Yeah, I also don't know of a. OK, so Air. That's Adobe Air. Yeah, I don't, I don't, I don't, I also don't know of, uh, um. Okay, so Air. Oh, that's Adobe Air. That's right, that's right. Yeah, Adobe. Okay. So I can't really think of anything off the top of my head.

One thing you could do is you could have, like, a handshaking, like a negotiation thing. So basically, let's say, let's say you make the assumption that if there's a conflict, the server wins. So if you can work under that assumption, then you can have the phone and the server send messages to each other using Thrift or whatever. Maybe even Air has an RPC, like a remote procedure call framework. And so the phone would send a message to the server saying,

hey, change this record. Update this. Like a new student has a test. Here's the student and test an essay or something, right? And the server can also say, hey, here's a student and essay. If you get a conflict, like the same student gets created with two different essays at the same time, you just let the server win.

So the phone would just take the server's answer. And if the server gets a conflict, it would just throw out the phone's answer. You could probably code something like that up. DAN GALPIN- So it's a little hard, I guess, the details are a little loose. But another thing which I've used before is Cheesy. And people don't like it.

So if you have one account, one person should be editing the data at a time. And so if that's the case, you can just have a method for basically determining if somebody's currently accessing or modifying the data. And other people have to be read-only. Right, that makes sense. So essentially, the server is only a server.

Everything else has to become a client of some sort, even if it's on that computer. And when one is connected until it either logs out or has some sort of timeoutout everybody else can only be in read mode yeah makes sense and that you know that also will prevent it's a little cheesier and you know people might not like it but it is simple and straightforward the other thing would be for this is like general advice i guess when when you come across problems like this is you know really just working on the google for a while and like trying to like search out like other people have

had similar problem try ways of rephrasing it i don't like i find maybe this is not true for our listeners but i find like people i know like in day-to-day life like just aren't good at so i say googling but i just mean searching like being whatever yahoo it doesn't matter what you use but like understanding like how to try permutation state it different ways because you don't know like part of the hard part is finding out what the word is that most people are using because it might not be the same word you're using to describe the problem um and so again forums can be a help there because other people can help you with that if they're in the the domain but also just trying a little longer at searching and trying to find somebody else who

may have encountered that problem yeah and the other thing is keep it simple right so in this case you know maybe where he's going with this is he wants to have all the students' tests on all the phones so that they can scan, so that they don't have to hit the server every time they want to check if someone's cheating. But that might be a nice optimization, but that might not be necessary.

Maybe the server could check for cheating like from every phone, you know. If you have like a thousand users, then the server can check like a thousand times and it could even have a delay. It could say, hey, I'm checking 10 other people right now. Like yours is in the queue or something like that. And that might be way easier than making some kind of crazy synchronization, you know.

So keep it simple. Try and keep everything on the server if at all possible. And if you can't do that, yeah, then you could do some kind of hands synchronization. So keep it simple. Try and keep everything on the server, if at all possible. And if you can't do that, then you could do some kind of handshaking. CHRIS BROADFOOTEN- All right. So we're running kind of long on time already. We're at an hour and 10 minutes. So we'll do one or two more.

CHRIS BROADFOOTEN- Yeah, let's see. So we can do two more. CHRIS BROADFOOTEN- All right. CHRIS BROADFOOTEN- So we have Corey Reese. Corey from Texas. And his question is, what programming language would you recommend for a beginner? Yeah, so I mean, we've talked about this a little bit on past shows. It kind of depends on what you want to do. But I like Python for beginners.

Me too. I was going to say Python. But also, I mean. We didn't rehearse that. No, no, no. But I mean, actually Erlang. Yeah, Erlang's good. No, no, no, I'm just kidding.

Not so much those. I mean, you can, I guess. If you started with that, you'd be like a reaper programmer. Yeah, you'd be really good at stuff that most people aren't. So I mean, I really think Python is a good blend between easy to use, but also, and the batteries included we've talked about before in that episode, and also being useful for real world things.

If you want to jump straight in, Java or C++. That's what still most of the world's software houses are using. And so learning those from early, it's never a bad thing. But it's not as much reward up front.

You don't get that immediate satisfaction the same way. Yeah. But you know, I'm also inclined to say JavaScript. Really? That's interesting. Why JavaScript? So, I mean, think about like all the, if you're, if you're going for like, if you're the kind of person who needs victories, like what better way, right?

That's true. You know, so like even if you write client server now with like Node. way, right? That's true. So, like, even if you write client server now with, like, Node.js, right? This is actually on a task of mine probably longer standing than it should have been. So I don't know JavaScript that well. I haven't really done more than just very simple stuff. But, like, all the stuff I read, I have a task of mine to learn it and to actually implement client server stuff completely in JavaScript. With, like, Node.js and a JavaScript front end and web page development.

That is just so, like, I mean. You could make good, earn a living at just doing JavaScript programming. Yeah, totally. And then also, there's a lot of good tutorials out there. Because a lot of people who aren't expert programmers, that's the first thing they learn because that's what they need. So there's a lot of good resources out there. And you can do more and more than you ever could before with it. Yeah, it makes sense. And plus, you don't have to deal with deploying.

Like, if you make some Java app, you have to, like, have people download it. And then half the people, they don't have Java installed. You have to deal with that. If you're writing something in JavaScript, you just make a website. And you're like, hey, everybody, check out my website. And it just works, you know. So as far as, like, going back to what we were saying earlier about getting feedback and being part of a community and things like that. It's way easier to say go on some forum and say

everyone go to this website than it is to say... Forget that. Email your mom or dad or your brother or sister or your cousin or your aunt and like, hey, what do you think about this? Yep. It's much harder to be like download this Java and run this. Download this 100 meg jar file that I have

and then double click on it. Yeah, yeah. So JavaScript's actually an I have, and then double-click on it. Yeah, yeah. So JavaScript's actually an amazing choice. I didn't even think about it. Okay, so that's a non-personal recommendation. But as far as everything I read, I mean, I wouldn't feel too bad about somebody taking that advice. Yeah, totally.

So one last question. This is from Joey Andres. And I'm using his email address to infer his location. His email address says he's from Alberta, Canada. All right. So we'll assume that that's true. His question is, he's a first-year undergrad majoring in computer science. He's starting to realize the importance of number theory and invariance. what path should he take to

sort of understand more about like the math behind computer science? So how can you get into sort of the theoretical parts of it? I think this is your, this is like the bowling ball is going straight down your lane. Yeah, this is awful. Okay. So I'll totally take this. Great question, Joey. So yeah, the mathematics of computer science is super, super interesting.

You know, number theory, of course, incredibly important. A lot of interesting things there. I would think, like, one of the best ways to get into this kind of thing is to get into complexity analysis. Because that's something you're going to have to do a lot of anyways. Interesting. And that is something that's something you're going to have to do a lot of anyways and that is something it's sort of it's very intuitive like the way it starts is imagine if you have like a list of you know the way they do it typically is with sorting right so you have a list of numbers

and then they show you this thing called bubble sort where you have two for loops and then inside the two for loops you do some swapping right writers there's some there's some conditional swapping there and when you're done the list is sorted and you can see this takes n squared time and you can actually see it in the for loops like you have the end and then inside the for loop you have another you have another end so in square um then they get into like n log n and things like that and then they show how things like a like a heap are order one on amortized order one even though there are cases where you know heap

insertion and deletion actually heap retrieval is always order one anyways there's some algorithms like heap insertion say which can be order log n but uh but if you do it in a certain way amortized, which means kind of like on average it's order one so anyways, these complexity analysis is a good way to get started and it's really useful as an engineer when you're designing new algorithms

to say if I design it this way it can be way faster we're talking, instead of taking years it takes three minutes some of the even more like deeply theoretical things about like you know complexity not complexity but like information like how much information is there like Kalma Gaurav complexity of algorithms and data and

things like that that that stuff is super interesting but that could be like a higher barrier of entry, you know? So by starting with complexity analysis, it's something like sort of easy to grasp. Then you can sort of jump into some of the other things. Yeah, so I mean I guess there's kind of two ways to view that question. I think that's probably like the way you're viewing. But the other thing that, I mean, you run into a lot in computers depending on what you do, is even stuff that's not directly computer science but is related related so like like discrete math or whatever like how floating point numbers work so that can get interesting but even less obvious things like prime numbers

it's like right like calculating prime numbers or you know uh geometric algorithms right like kind of doing stuff that's like a convex hole right like these kind of things involve math and if you don't know what they mean you can can look them up. And so, you know, those things you do encounter in programming fairly frequently and you hear them discussed. So those are, I guess, kind of related to like number theory type stuff or whatever. And those are interesting to me. I never got very deep into them, but kind of understanding them because a lot of stuff like cryptography you come across has a lot to do with, with prime numbers and that kind of stuff. And then just getting more of an understanding of that kind of level of math can be really useful.

And I find for that, I would recommend something like Project Euler, depending on how you say it. You can search it, Project E-U-L-E-R, and they have various programming challenges, problems. And a lot of those, if I recall correctly, will involve to understand things like you know how to find a prime number or they'll actually give you kind of like a little bit of like here's the certain kinds of numbers we're looking for you should calculate these you know these are highly related to like ACM programming competitions if you see those or something called top

coder you can go on there and those very deeply a top coder for instance or even ACM will combine what Jason's talking about with what I'm talking about so you'll need to know on the one hand like how our prime number is calculated and on the other hand the default algorithm for doing that unless the problem gives you some cheat like way to shortcut it like what would that complexity be yeah and could I do it in enough time Or do I need to look for that shortcut, you know, and find out what it is? Is that the trick to solving the problem? And those methods are often they'll have solutions for you. So you can see what you should have done. And then you can learn from that. Yeah, another, just to recommend some reading material on this is

Algorithms in C++ by Robert Sedgwick. And maybe we'll post a link on the blog or something like that. But yeah, Algorithms in C++, it's actually two books. There's a volume one through four, which got assembled into one book, and a volume five, which takes up an entire separate book because it's so epic. But yeah, Robert Sedgwick wrote this Algorithms in C++ series and has amazing theoretical theoretical and application based like framework in it. So he actually talks about like max flow. So in other words

given a graph and assume the graph are like pipes and and so they show how much water can go through these pipes. Like how much what's the maximum flow between two nodes? Like assuming that you could use all the pipes right? So he actually talks about the mathematical complexity and then mathematical intuition behind how the algorithm works. And then he shows the algorithm. So highly recommended reading. Yeah.

I mean, it all depends on what kind of programming you end up doing and how much of this you need to know or don't need to know. But yeah, I mean, there are people who major in math, right? Totally. A lot. So yeah, and computer science is closely related to math in many, many ways.

Yeah, it can't hurt to learn that stuff, at least at some level. Yeah. So all right, well, I think that's a wrap. Thank you, everybody who was on at one time or another or is still on the Hangout. Jason's giving you a wave hi. And we appreciate it and we are uh going to

be back to our regular episodes we'll have a language for our next time and maybe we'll do another one of these mailbags we'll see give us some feedback let us know what you think did we ramble too long were we too off topic we're always pretty rambly so yeah so it feels good but you should be used to that thank you for all your feedback thank you for all the questions yeah definitely it's uh it's been awesome doing the be used to that. Thank you for all your feedback. Thank you for all the questions. Yeah, definitely. It's been awesome doing the mailbag, answering people's questions. You know, it's great that we sort of like aggregated

a lot of these questions and then we can sort of dedicate a whole episode to it. I think that this worked really well. I appreciate everybody's contributions. And yeah. And thank you all for subscribing. We noticed increased downloadership. Yeah, yeah.

So a lot of you are either downloading it multiple times or... I can show. Yeah, or... I can share with you guys. Or yeah, we're just becoming more popular. We're not sure what it is. So if you're a new listener,

again, like Jason said, well, too late now. You've already listened to this episode. But check out another episode too. Yeah, so we released episode 20, you know, what, maybe three weeks ago or something. And hundreds and hundreds of people

have downloaded the episode, which is totally awesome. And we appreciate all of your feedback and questions and comments. So we have a few new languages. Some people said, hey, I don't have a question, but I want you to cover this. And we'll definitely be covering that for the next few weeks. So, yeah, keep your emails coming to programmingthrowdown at gmail.com

and keep posting on G+. All right. Thank you, guys. Cool. See you later. The intro music is Axo by Binar Pilot. Programming Throwdown is distributed under a Creative Commons Attribution Sharealike 2.0 license. You're free to share, copy, distribute, transmit the work, to remix, adapt the work, but you must provide attribution to Patrick and I and sharealike in kind.