Embedded - 518 Nothing We Can Do About Frogs

Episode Date: January 11, 2026

James Cameron spoke with us about programming for and operating a large telescope. The show is a blend of astronomy, engineering on the fly, and weird lady bug habitats.  The Anglo-Australian Telescope (AAT) is part of the Australian National University's Siding Spring Observatory in Coonabarabran, New South Wales, Australia.  The AAT has an all sky camera where you can check in on a very dark sky.  James was on Embedded Episode 172: Tell Forth You Me Please where we talked about the Forth programming language and his experiences with One Laptop Per Child.  Transcript Unrelated to the AAT, Chris took this image of the Andromeda Galaxy (M31) from his Zwo Seestar 50 over 9 hours (multiple days), stacking the images and processing the data.

Welcome to Embedded. I am Elysio White, alongside Christopher White. Our guest this week is returning guest, James Cameron, who we talk to about one laptop per child and fourth. And this time, we're going to talk about telescopes and whatever that bird in the background is. Hi, James, and extra guest. Welcome to the show again. They're frogs. They just started. As soon as you started recording, they just started. Nothing we can do about frogs.

They're about six meters away in a drain pipe. So, yeah, I don't know how to fix that. We've never been able to fix frogs. They're in chorus now, so it's probably hormonal. We are proceeding despite the frogs. It's like we've got all the show titles all set out. James, could you tell us about yourself as if, we met, I don't know, on the embedded Slack channel?

Okay, yep, sure. I'm hard to describe because I don't often describe myself, but I think the best one is scientists with a business degree. I had childhood exposure to music, physics, electronics, and computing, and science seemed to be the way to get things done. It seemed to be the way I operated myself. and so that's the way I always thought through problems. When I was looking for something to do as a job,

computer programming was the bees' knees. It was really well thought of and therefore well paid, and so that's what I started doing. So computer science is what I've focused on, but not with a qualification, because I'd done all that by the time I was in middle school, and so the next thing to do was work out how to make money from it, and hence the business degree.

We want to ask you lightning round questions. You are familiar with the show. Are you ready? Yep, go ahead. Are the frogs ready? I don't hear them, so I don't think they're ready. Have you seen a total eclipse? Yes, yes, I have. Of various bodies, mostly an eclipse of the sun.

Have you seen the green flash? No. Have you seen the zodiacal light? Yes, yes, I have. But someone had to point it out. I wasn't actually trying to see it. It was just sort of, oh, look, here it is. What's that? Zodiacal light, what's that? Oh, all right, okay. Have you seen a platypus in real life? Yes, I have. I was walking in bushland and came across a little regulate of water.

And there was this weird thing swimming in it. And it just didn't seem to swim right. And so I focused on it visually for a while and tried to work out what it was. And eventually I was rewarded with a large nose, which was what's in the books. So, yes, it was surprising. Have you ever touched a kangaroo? When I was young in visiting an animal petting zoo, yes. And when I've had to remove one from the road after it's dead, yes.

But on that, no, you tend not to get close to them. Yes, I've seen their upper bodies. They seem very boxy. Boxing. Like good at boxing. They have a radius of comfort. And the radius of comfort, for a kangaroo here where we are is about 50 metres,

which is to say if you're within 50 metres, they'll move away. Up at the observatory where I worked, the radius of comfort was about 10 metres. And that's unfortunate because you get closer to them. And if you're driving, it means they're unlikely to jump away, and so they might jump into you as you're driving. And so that's not nice. But, yeah, they're not friendly little critters. They've got their own agenda.

And the males, when they're wanting to mate, are particularly irritating because they're chasing after females at 40 kilometres per hour as they go past. You know, the females running away, the male is chasing them, that seems to be their thing, what they like to do. And the males in that state are without thought and strategy. Well, they have thought and strategy, but they're concentrating on something else. Doesn't involve motor vehicles.

Yeah, they don't notice you. And that's on foot, especially where that's dangerous. Favorite Australian animal? At the moment, it's echidna. We have numerous echidna around here. Very spiky ant-eating type animal. They dig up bugs and ants. They make holes all over the place.

Not as many holes as... Akednas also have cloacas. Yeah, well, I don't go into that detail. Sorry, that's... The reference. The reference. It won't make sense to anybody, but Christopher. Moving on from Cloakas.

If you get close to an echidna, like within one metre, that's when they can actually see and sense you. So their range of comfort is one metre. So you get right up to echidna before they even do anything about it. And their reaction is to try and dig a hole in the ground right where they are. Or if they know they can't dig there, then run as fast as they can for 95 centimetres and then dig a hole. And they then have...

have all their spikes pointing up and their soft body parts in the hole. So they don't actually hide in the hole. They sort of use the hole to protect half of themselves. So they are weird. And when they're in their mating frenzy, they're a train of echidnais traveling across the landscape, one after the other. That's so funny.

I feel like we should wrap up lining around, but we have many more. Do the next one? There's three next. The mount question. Oh, okay. Equatorial, alt-asimuthal, or fixed? I don't mind. I mean, you can mix a match.

Where I worked, the building was azimuthal, and the telescope was equatorial. Okay. Exactly. It causes interesting results. But they're just different ways of compensating for the rotation of things. I prefer my chairs to be azimuth. Yes.

So don't slide off them. So tell me about this telescope. Forget lightning around. We want a real answer this time. That's not real answers. They were. Sure. So one day I was looking for something to do.

and heard that the team up at the telescope needed someone to do things for them. And so I popped up there and they said, come join us. The telescope that I'm talking about is the Anglo-Australian telescope at Kuta Barabran, northwest of Sydney in Australia. It's the largest optical telescope in Australia, as far as I know. It's not the largest telescope. There are radio telescopes that are much bigger, and we use. the whole of Australia when we link up those radio telescopes.

It makes a very large aperture. But it's the largest optical telescope. It's also quite old being built in 1974 or so and has been maintained since. I was also built in 1974. Not feeling quite old yet. But you've got automatic systems, which... generate your components, whereas this telescope, we used humans to maintain the components there.

So when I first encountered that telescope, I was a tourist and just visiting and see the visitor's centre and climb the stairs into the viewing gallery and gaze with awe at this gigantic thing and then go away and go to the next tourist trap. But when we moved up into this area, it's the white thing on the mountains over there. And so you can see it as you're driving along. It's a very recognisable white dome. And the density of settlement out here is very low.

The population density is very low. And so anything mounted on a mountain is going to look very strange. And it does look very strange. It's out of place in that respect. We're all used to it now. But it's weird to see a building on top of a mountain. And so it attracts all sorts of people. And how big is the scope itself?

Like how many meters is the aperture? I think it's 2.7 meters. or something? It's a very big mirror. It's one of those telescopes where it has a primary mirror facing the target and the light is then concentrated into a secondary mirror which then reflects the light back down into a hole in the primary mirror and then you put your instrument down the bottom below the primary mirror. That's one of the modes it has of operating. Another mode is that you put the instrument up the top of the telescope facing. downwards and then the starlight is only

reflected off the primary mirror before it reaches the instrument. That gives it a much wider field of view, which is great if you're looking at galaxies, lots of them at once, but terribly if you want to look at a single star. So the first mode where it's using a primary

and secondary mirror and the instrument is at the bottom, that's when it's being used to look at a single target, like a galaxy or a star. I want to pull a couple of stats from Wikipedia. 3.9 meter equator equatorial mounted telescope. That's it, 3.9. Sorry.

One of the most scientifically productive 4-meter-class optical telescopes in the world. It has a seven-story circular concrete building topped with a 36-meter-rotating steel dome. The top of the dome is 50 meters above ground level. And the moving mass of the dome is 260 tons. That's incredible. Like I would, if you ask me how much, well, the dome itself is probably weighs a lot because it's a giant dome.

But yeah, a four meter telescope is big, but I guess I just don't have any reference for this because I, you know, I work with small amateur telescopes. and like, oh, I've had an 8-inch telescope, and 4 meters, you know, that's bigger, much bigger, but it's not 260 tons of support bigger, but I guess it is. It really is. Well, firstly, imagine taking your telescope and placing a dome around it.

How big would that dome be? Yeah, that's true, several feet, yes. It's already going to be several meters because you need to be able to walk around the telescope. You need to be able to hold the dome up. You need the dome to rotate to face the target, so you need, you know, a tracking system to rotate on and all that sort of stuff. So the size of the dome itself is not terribly surprising. It's the size of the telescope which actually set the size of the dome.

These days telescopes are made slightly differently and that it tends to be cheaper to make the whole thing capable of living outdoors and just have bits and pieces open up rather than have the entire thing. squirreled away inside the dome. So the way in which things are done has changed. But, yes, it is very big and very heavy and has some really good motors which make it rotate. So, yeah, it's big.

Well, and it has to do, the steel dome is partially because you have high winds where you are. Yes, definitely. And the reason we have high winds is that mostly the area around these mountains is very flat and these mountains protrude into the airflow. And we often get a condition and we have it today where you look at the mountains

and there are clouds downwind from the mountains that have been formed by the wind passing over the mountains. So you get local weather, local rainfall around the mountains as a result of the mountains being there. And so, yes, when you put something on top of the mountain and you're dealing with very high-speed winds, the thing had to protect itself. We did have, for example, an engineered maximum wind speed that we're allowed to keep operating with.

And if the wind speed got above that, we had to shut the dome. And if the wind speed got above something like double that, I imagine we'd evacuate. Right. You know, it's one of the things you have to do with when you put something on top of the mountain is you get high winds. So as a developer for a telescope, I would think you'd have all of these different control problems. But the telescope was built in 1974. Hmm. What exactly do you need to do?

Like, what's a day in the life? when you have a fault with the telescope, everyone arrives in the morning. We have our 8 o'clock meeting and we get the list of faults that have been detected by the operator overnight. And some of the faults will involve the control systems. And we need to know if the fault is in the control system or in the hardware or if it's a false report. And so we need to be able to maintain the software of the control system to some extent, or at least to be able to probe the software to understand it. And so sometimes we'd get a fault and we'd have to write a bit of code

in order to probe the understanding of how the control system works, or we'd have to read the code of the control system to figure out how to tweak what's going on, or we'd have to go find the documentation and the archives and try and work out how it's meant to work. And every now and then there'd be a new system to develop or a system to replace because we just can't get the parts anymore to make it run and we need to replace the control system with something else.

So there's an ongoing maintenance and there's also the daily fault finding. And so yes, there are things that have to be done. The fault finding. When I think about moving 260 tons, I think one of the faults may be like overcurrent. Yes. But what kind of faults are we talking about here? Oh, the awful range of faults.

Everything you can think of goes wrong because this is a one-of-a-kind thing. You can't take a list of faults at other telescopes and make any sense of it enough to be able to predict what sort of faults would happen. This telescope, for example. They're just not enough telescopes in the world of this scale to be able to do that kind of analysis. But things that go wrong, I mean, everything from a part wearing out to electrostatic discharge to lubrication was insufficient. And so the current was too high and it tripped and the operator didn't have the qualification to, reset the trip or something like that. I don't know.

The faults, we would typically get two or three a day, and we'd have to work through them and figure out how to fix it, preferably so that the fault doesn't come back the next day. Every now then we'd have a fault keep coming back because the overnight operator... Just keeps pushing go. Do it anyway. It keeps trying to use a telescope, yeah.

I mean. I'm trying to do science here, man. That's right. There's all sorts of things like false alarms, like an alarm goes off saying there's rain, but there's clearly no rain, no clouds. Why is that happening, that sort of thing? Or trips of circuit breakers that we've never had trip before.

We have to go and find the circuit breaker and reset it and that sort of thing. There's all sorts of instruments. faults as well, not just telescope and dome faults. There's ways in which the instruments can fail to thrive, like they might be the wrong temperature, or the supply of liquid nitrogen might have fallen too low, or can't turn it on, or can't find the key to get in the room. All sorts of things go wrong.

And it's not for lack of attention to detail or design, It's because it's a really complex system with lots of subsystems that all interact in weird ways. So, yeah, it's weird. It's built up over decades. Yeah. Would it be simpler to just start over? Well, I was going to kind of ask, is this the ship of Theseus? Like, how many of the original systems when it became operational were there by the time?

you left. Grandfather's axe. Yet quite a few of them. Very little was actually replaced really when I was there over the years that I was there if you counted as percentages of total number of subsystems mostly maintained. So yeah that's so to Elise's question it that's kind of normal to answer the question would it be better to knock it down and start again people do make that kind of calculation. The people in charge of the funding, they work out how much it would cost to replace it, how much it would cost to do something comparable

and whether they would be better served to do that rather than to proceed as they are. And almost always they find it's cheaper to proceed as they are, in that they can get more science done if they keep the thing running than if they stopped for a while and built a new one. And if they stopped for a while and built a new one, the next question is would they build it there?

Right. Because there might be reasons not to build there now. The light pollution is different to how it was in the 1960s when the decision was made. The pollution from the satellite clusters, constellations is different. And also the science. What science do you need to do?

What kind of field of view do you need and what kind of analysis of the light do you need to apply to it to be able to make the research results that generate the science papers? That changes over time as well. You mentioned before a four-meter-class telescope. The classifications are, I think, arbitrary.

And there is, I think, one other telescope, which is the same fundamental design. And I don't know if it's still in use, but there are other bigger telescopes being used for different things. And so this telescope is sort of in a niche scientifically. It's used for mostly galactic survey and for spectrographic analysis of stellar sources. It's not used for planetary science much. It's not used for looking at asteroids, well, much. Not intentionally necessarily, but sometimes those pop up anyway. I was operating at once and we were asked to look at asteroids.

And so that was novel and unusual because it wasn't imaging of the asteroids in terms of looking at the shape and size and how they're rotating. but it was chemical analysis of the reflected light using the spectrograph compared against sunlight. And you look at the subtraction between the two spectrum and you can work out the absorption spectra of the surface and from that you can work out what chemicals are present. And if you compare those with the results of the last time you did it, you can see if anything interesting has boiled off

or if someone's been up there mining and you hadn't noticed. So this was built in in the 70s, and you said most of the control systems or a fair portion of them remain. What were the computing systems and control systems? Like, how were those built then? And have you replaced them with the Raspberry Pi? I mean, is this an IBM mainframe, you know, or was this totally bespoke? Raspberry Pi's got a bad reputation because of the microSD card. tendency to fail.

And while microSD cards were doing that for certain years of production, it was those years that the telescope first started using Raspberry Pi's. So they had a bad experience. The modern microSD cards don't do it as badly anymore, and there are ways to make sure your Raspberry Pi stays alive. But yes, the really old control system that operated the telescope drive which points where you want to see the target, that system's been replaced two or three times.

It's currently an Intel-based Linux system. Previously, it was a Vax-based VMS system, and before that was something else. But each time it was replaced, it grew new features. it retained the same pointing accuracy so that the science could continue. But its main purpose as a subsystem is to point the telescope at the target and to keep it pointed there as the most irritating thing in the world happens, which is the Earth rotates.

And as the Earth is rotating, your telescope slides off the target. And so therefore you have to keep the telescope moving at exactly the same rate as the Earth rotating. and if you don't keep it at that rate, then your pictures become blurry. Not only that, you've got to make sure that things aligned correctly with the celestial pole. Yeah, that's right. And that's one of the advantages of the equatorial mount

is that only one motor is running mostly when it's tracking like that, because it's one motor which compensates for the rotation of the earth. Whereas if you're using an alt-asimuth mount, you're running two motors continuously at different rates. And so mechanically, that was an advantage back then. But these days, you would build alt-asimuth because it's easier. Because it's easier to build.

Yeah. And we have other technology. Like when you do alt-azimuth, even though you can track an object, the sky still rotates over time some. And so as you take images, you have to compensate for that. by rotating the entire sensor to keep, yeah, so there's other stuff you have to do. There are downsides as well. Like an alt-asimuth telescope can't look directly upwards because if it does and you're

tracking a star at the same time, there is a point in time where you have to rapidly rotate the telescope so you can start following the star down the other horizon. Gimble lock. Pretty much. It's called Gimble. Yeah, that's right. And we have that happen with the dome. And it's rather slow and subtle to begin with.

And everyone's sitting in the control room, the astronomers are reading their articles and doing their exposures. And they've invariably picked the star, which is going to go right overhead, because that minimizes the air mass, maximizes the photons that are collected by the bucket. And so the spectrograph results are the best. And so as the star goes directly overhead, what happens is the dome zooms goes faster and faster trying to keep up.

And it always does. There's enough gap in the aperture so that the dome does keep up that it gets so noisy. And at some point, everyone looks around and says, what's that noise? And oh, yes, the target's right overhead. And that's because the dome is trying to catch up. of the telescope. And then it gradually slows down and calms down. And so for the particularly young astronomers who've never seen this happen,

I'd take them outside in the dark of the telescope and let them see with their small amount of light, the dome racing past. So, yeah, it's one of the side effects of having. having that kind of mount combining with another kind of mount. And there are other limits as well. Like the telescope can't point too far down, otherwise it'd fall off its mount.

And so we don't want that to happen, and so therefore there are limits. You mentioned the dark energy survey, or the two-degree field? That has been mentioned, yes. This is, again, Wikipedia knowledge here. a robotic optical fiber positioner for obtaining spectroscopy. Basically, you have...

Basically, tell me how this works. Yep, sure. So it's a big plate. It's several handspans in diameter, very flat. It's been machined to be very flat. It has a metal, which, has been chosen to have a very low coefficient of expansion as temperature changes. And so therefore, as the temperature changes inside the dome, it doesn't change shape and size.

And there are these 400 fibres that are in retractors around the edge of the plate. And what you do is you pick out the end of the fibre, which has a magnet and a mirror at 45 degrees. and you put the magnet onto the plate and that collects the starlight at that point on the plate because you focus the image of the stars onto the plate using the telescope. And then the starlight from that one star or that one galaxy goes down the fibre to the spectrograph,

where it is presented to the spectrograph for analysis. And to put out 400 of these things by hand has been done but is very boring. and takes too long. And so there's a positioner, an X, Y table, and a gripper, which picks these magnets up from the edge and puts them onto the plate in the position according to where we think starlight will appear. Okay.

And so... I want to see if I understand. You have 400 fibers. Yep. And where naively, you might make a lot of... like a pin screen where you just have 400 in a grid. Yep. That doesn't make sense because stars aren't in a grid.

Stars are in, let's say constellations, even though that's not really right. Instead of putting your 400 optical fibers, single one optical fibers, in a grid, you're going to put them in the constellation pattern with great precision. Do you want one fiber per target? One fiber per star. Yeah. You decline to put the fibers out where you don't want the light collected. So if there's a nearby star which is obscuring your view of all these other galaxies,

because it's too bright. It's too right. You don't put fibers there. And so you're blocking the light and therefore you get to analyze the light from the things you are interested in. Of course, this takes a survey view of the sky first. people who prepare these lists of targets for the positioner, they've gone through a lengthy analysis process to work out what targets they're interested in and what shading they may require. And all that's done by the astronomers long before they get to the telescope.

And they load, they give us the data file, they upload it into the Linux system which handles the positioner, and it then positions the fibers into all those places. And yes, you can be up at the prime focus axis, which is where we lean the telescope over so we can service this instrument during the night. And you can look at the plate of fibers and you can see the constellation pattern. But since it's only a two-degree field of view,

it's probably not going to be a constellation you recognize because two degrees in the sky is not the shape and size of constellations that we have known. About four full moons. So two degrees is four full moon? Yeah. And full moon is a half a degree. I think so.

At the moment, it's very bright. Yes. Right, because we're at perihelian now. Yes. For us down here in Australia, the moon is in the northern sky, which is, it seems weird and rare. It does that occasionally. The Earth-moon system, geometries and patterns are frankly strange. But yes, at the moment, it's leaning over quite low.

And so as it rises, it's rising quite low. We're in the middle of summer. We're about to enter into a week-long heat wave, so we know about it. Okay, so I wouldn't recognize the constellation unless it was some tiny constellation. like maybe the Pleiades I might recognize. I mean, I wouldn't because I don't, but Chris would. Yes, that's right. And I have been up there trying to sort out a tangle

and recognize the pattern of what it is. But on the other hand, I'd probably been primed by seeing it on the computer screens down the control room. So, yes, you get to see where the astronomers have chosen to select the light using this instrument. then the light goes down those 400 fibers into the spectrograph and they go into a long row of fiber terminations. And so the spectrograph is looking at this vertical slit containing fibers.

And each of those fibers, the computer knows where they are up on the plate. And so therefore can identify which target they are. And so the spectrograph processes these 400, stars all at once. And as a result, you get to do a spectrographic analysis of 400 stars each time you use this. And so 20 minutes

of exposure and there you have it. During that time, the plate at the other side of the instrument is being prepared by the position of while it's doing the exposure. And then when we're finished exposing, the thing is tumbled while we move

the telescope to the new target. And so a new set of stars is then collected. And so it's a very efficient way of collecting spectra of stars and galaxies in terms of the number of spectra you can collect per hour. But the downside is you can only look at things you expect to see. Yes, you have to have done a survey first so you know where the things are or you'll be using the databases to work out where the things are. And there are certain things that you can't look at at the same time. You're tend to to select all the stars of a certain brightness.

So the exposure in the spectrograph is appropriate nominal. Because if you accidentally collect a very bright nearby star while trying to image very distant galaxies, you'll just have a lot of light coming out of one pixel in the spectrograph, and that will disturb your results. So there are a lot of things you've mentioned here. There's the control system of controlling the whole building, there's the keeping it cool,

which has to have a control system too. There's the robotic positioning of these optical fibers based on what astronomers give you, which probably is never in exactly the format you want. There's all this data collection. And then, of course, eventually there's going to be this pile of data that needs to be turned into a scientific paper depending on what they were looking for.

What parts of this? Are you involved with when you worked there? For the break fix, pretty much anything that involves software or control systems with the software component, for everything being an operator of the telescope overnight, I was responsible for everything. But not necessarily having to fix it. I would just have to find a workaround.

So if the rain alarm is going off when it shouldn't, then I just ignore it. Or I'd go outside to make sure it isn't raining. If the internet was down then I'd find a work around. If there's a power failure, I'd make sure the generator started. If the dome doesn't open,

I'd kick a few things and get help from the technician on duty. And if that still didn't work, we'd have to abandon the night. If the dome doesn't close, that's a fire risk because you don't want the sunlight shining on the mirror because it can reflect off and cause fires. So you make sure everything's closed down on a telescope and you point the dome away from the sun

and you tell the morning crew that, sorry, we couldn't close the dome. If that 2DF instrument faults in a way where it tangles the fibers up, then you have to get up there with a set of pliers and move the magnets around and put them in their parking positions and then get the software to survey them. And sometimes there would be things where we're not sure why it's happening and we've got three hours spare because there's clans, because there's cloud, and so I'll read the software which drives the control system,

read the source code before I put in the fault report to see if I can figure out what's actually going on. So it's a lot of live production DevOps kinds of things during the night, but when you're on day shift, it's a matter of what the fault is, which subsystems are involved, which ones are those are software, hand them off to the software technicians, which one of those are mechanical, hand them off to the mechanical technicians. and we go our separate ways, do our separate things. They put the oil in.

I do the upgrades or analyze the data. And then hopefully we get it working again. But yes, a very large collection of very complex subsystems with all sorts of interesting relationships makes for some exciting work. It's funny how sometimes people think that engineering is just sitting with your butt in the chair and typing at your computer. It doesn't sound like that's been true for you. No.

I mean, at one laptop per child, we had this philosophy of dog fooding where if we're making a laptop for children, we really need to make sure we can use them ourselves. And that way we find all the faults because there's nothing like finding a fault when you're trying to use a laptop for your own use. And the same thing,

when operating this telescope, the software faults which were particularly annoying to me as a telescope operator were the ones that I would tend to concentrate on and fix. An example is there's a acquisition and guide unit underneath the telescope where you used to put the glass plates back when we did that kind of photography in the 80s, 70s. the image of the stars is focused down there and that's where you put the smaller instruments rather than 2DF.

But this X, Y table down there can position probe cameras which are used to guide the telescope. You have one of the cameras looking at a star off the center of the field of view and guide the telescope based on the movement to that star. But this XY table has a control system all of its own right.

and it was occasionally not finishing the job when you ask it to move a camera. You tell it move the camera over here and it would move only one of the coordinates, you know, the X coordinate instead of X and Y. And so I got curious one day and said this is costing us a lot of irritation while observing because you try to do something and it doesn't always complete. And found faults in the software where interrupts were occurring and they shouldn't have been. And once I'd fixed that fault,

which took a few nights of rainfall where we couldn't do anything else, the acquisition and guide unit motor control was now reliable in that 100% of the time when you ask it to move, it would move, whereas before it was only 75%. And so, yeah, I can't remember the question now. I think you answered it just fine.

So this is a big organization. It's not just you. Yeah, it's not just you. You mentioned other software technicians, mechanical technicians. I'm sure there's many operators who are ostensibly operating, and then there's the astronomers behind. And one of the things you mentioned about the fiber thing was efficiency, like being able to reconfigure while something else was happening so that it's ready to go.

as quickly as possible. I imagine there's not a lot of big observatory telescopes in the world, and there's a lot of astronomers. And so there's probably a big backlog. There's a lot of people who want to get time on these instruments. And so having downtime is probably, you've got people yelling at you, right? Like, oh, the schedule, you know, I've got to observe this thing. And if I don't get it this week, then it's going to be out of view or unfavorable.

Like, how much does that pressure come into the work? How much do you really enjoy rainstorms? Yeah. Well, the astronomers compete for time on the telescope. They put in their draft papers, if you like, what they plan to do with it. And a team of scientists works out the priority of which these observation plans should go ahead and when.

then they have to be fitted into what the moon is doing because some observations can't be taken when the moon is bright or when the moon is behind the target or those sort of things. And then there's the funding. Some of these observations are funded by certain sources and some by others. And that generates a schedule of observations. And so an observer who wants to use the telescope

for four nights to look at a kind of. cluster of galaxies might only need only one night to get that done, but they're booked in four nights because they really, really need to get that done. And the team doing the scientific decision-making has agreed with them that they really need to get it done. And so booking it over four nights makes it more likely that they would get it done. And they would have secondary observing tasks that they complete within that time. So other less important targets. And so for those nights we'd notice they would be quite calm and collecting very boring targets for six hours until their target of preference rises high enough to minimize the air mass for

the science and then it's everyone's excited and point the telescope and capture this thing and eagerly wait for the results and then okay that's done they have dinner while they did the last exposure and then go back to boring targets again. And so it can change during the night. We had six telescope operators, and so every six weeks you'd get a week on, and then everyone would get a turn one by one,

some of them more frequently than others according to their contract. But from the point of view of the astronomers, they'd booked their time. And if they don't get their time because it's wet, the agreement they have with the Science Committee may mean that they'll get time reserved to them in the next schedule. And so sometimes we'd have astronomers come back

to complete the work that they had hoped to do before, but they've had to come back, you know, a month later when the moon is back to how it was or the next semester when the schedule is rewritten. or they'd be reserve nights where that sort of thing might happen if the director so desires. So, yeah, there is a whole complex of allocation and scheduling that happens long before the telescope operator gets to meet the astronomer on the night. Listener Bailey asked a question that works well here. Did you ever get to look at something you wanted?

Yes, that was fun, actually getting to look at something I want, because. you're in charge of this gigantic telescope, and your job is to make sure the astronomers get what they want to look at, but every now and then the astronomers would stop. They'd not want to look any further because something was stopping them from looking. They've seen the horrors of the universe. Well, they're usually over that by the time they get. It's when the atmosphere is particularly noisy,

when the light is not arriving cleanly. It's called when the seeing is really bad, or when the clouds are so prevalent that they can't get an exposure in without also collecting moonlight or reflected light from cities or something like that. And so therefore the science they would do is hindered by the conditions

or if it's just plain raining or it's cloudy. But situations where I can keep the dome open and the only thing stopping us from doing something is that the clouds aren't right. If the astronomer agrees, then we might swing the telescope to something else that we can look at for some other reason. Like, for example, if there was a fault with the telescope, which said that it was unwise to run it in the southwest sky for some reason because of the dome catching or something like that,

then we'd give that a swing and we'd pick an interesting target down there and run down there while we're waiting for the clouds to stop being all furry and wobbly as they go past. And so, yes, every now and then that sort of thing happened. There were one or two cases where the astronomer just bowed out because the seeing was too bad. And so they'd go off and sleep,

which they really need to do I'd be left with a telescope and so look at the moon or a planet or there's also other things that have to be done like updating the mathematical pointing model which means looking at 20 or so stars all in a row and making sure that they're exactly where they meant to be and that lets you make sure the telescope's going to find the star the next time

so there's that sort of job to do as well and so along your way you might see oh, if there's something over there, have a look of that. At one stage, I knew where Voyager was, and so pointed the telescope in that direction and entirely could not see it. It'd be so small. Oh, damn.

Exactly. There's no way to see it, but to know that you've got a picture of the sky, totally black, where Voyager would be if it was big enough to see. That was fun. there's other objects around like Vesta and unusual objects that are worth looking at sometimes but it is not a telescope that's really good for looking at things that are close by.

Like you can look at a mountain or two on the moon because the field of view is so narrow. It's for looking at very distant objects and it's a space. spectrograph telescope. It's got a bunch of spectrographs behind it. So the only cameras we have for looking at things as a human are the guide cameras. And they only really need to approximate the centre of a star in order to guide the telescope to keep it on track. And so they're not very good cameras for this sort of stellar photography sort of thing that you can do with mobile phones and smaller telescopes. And so it's not as rewarding as getting your own telescope

and pointing it to the sky. But it is really good at seeing very distant objects. And so that's the sort of class of telescope it is. It's not a fault. That's what it's designed to be. It's designed to collect spectra. And while this is an old telescope, it is not such an old observatory

that it was a telescope with optical viewing with an eyepiece that you could go look at. This is all camera-based stuff from the get-go. We do have eyepieces, but I must admit, we have not used them frequently, and I'm not even sure if they all work. It's easier to use the guide cameras, because the problem with using the eyepiece

is they're in the telescope, and you've got to get into the metal cage. And you can't do that alone, because you need to have someone to get you out again afterwards, and with safety and human health these days, you need to not do that if you can at all avoid it. Back in the 70s and 80s, the astronomers would ride the telescope in the focal positions

with their stack of photographic plates and collect all their targets that way, communicating with the operator. And you'd wonder how they'd get out of the thing if they needed to. They'd bring a rope ladder in case the drive fail. and I never found out that. And I never wanted to be in the telescope, like being in a cage.

It is actually called a cage for that reason. You can't easily get in and out if it's not in its parking position. William asked if the proliferation of objects by the commercialization of space has impacted your work. Do you have to adapt to the increased orbital traffic? Is this going to become an issue more so in the future? Living on the mountain, as a resident, you have to secure your own internet access, which was always mobile phone service.

And when Starlink arrived, we were very happy because we could each get our little Starlink kits and have full internet access instead of having to walk up to the telescope and using the Wi-Fi there. So in one respect, great. In another respect, it gives you something to look at when you see the trains of Starlink satellites going over. They are pretty. They can be pretty. They appear on the all-sky camera, which is available on the internet.

You can actually look at what the sky looks like at the observatory. There's a link to it somewhere. I can provide that. The sky camera also captures all the satellites as they go over a particular point in the southwest about an hour before dawn when their solar panels reflect the sun. And so there's a glinting effect that appears as well. We noticed that. We've never had to, in my experience there, of many nights observing, never had to point somewhere else because of one of these satellites. Because the satellites only obscure the field of view for a half second or so, and our exposures are typically 10 to 20 minutes long. And so if you're getting the wrong photons

for a couple of seconds out of 20 minutes, it doesn't really matter. You can work out mathematically what the effect of that noise is and you can compensate for it by adding a few more minutes of exposure. If we're a telescope that was trying to take astral photography, photographs of wonderful-looking things in space, yes, it's a problem. But with the field of view and the type of spectrographs we use, it was not a problem. It's a problem for astronomy in general, and the astronomers are, of course, rightfully upset about it,

and that they're losing their dark skies. And it's also happening in radio astronomy as well, where many of these sources going overhead are now emitting in fairly wide bands of radio frequency. So they've got problems too. But for this particular telescope, apart from knowing that it's a problem for everyone else, and apart from being able to see the things as they go over, we didn't really have anything where we needed to avoid a target.

Christopher, you have a telescope, a small telescope. I have a telescope in the sense of I have a hot wheels car compared to a semi to compare these two, yes. And it used to be, when you had a different older telescope, you needed it to be as dark as possible. You were really annoyed by the neighbor's light. But now you set it out and cars drive by. Yeah. And you take hours and hours of exposure and then stack it. So you aren't necessarily affected by Starlink lights either.

I have to, that dictates how many of those I have to throw out. Yes, it sets a number of hours. And so is this spectrograph sampling similar to your stacking? I don't think so, because imagine I'm just looking at individual stars. I don't care about the whole sky. Like if I'm taking a picture of Entromeda, that's a big picture of the sky, I want to see the thing, and I don't want 16. lines going through it.

Which is what the satellites would make. They would make straight lights. Now imagine I just want to look at 400 stars in that view or galaxies, which are pinpoint sources, but I just put the fibers over them to look at individual ones or something. The chances that those lines cross one of those is much smaller than distorting my entire image. Okay. I think. Remember also that when we're observing at a big telescope,

we're typically interested in full darkness, so after twilight is finished, and when the target is almost directly overhead. And so most of the satellite constellations are then invisible. If they're glowing at all, they're glowing because of reflected light from cities, not from the sun. So you really notice that in the evening,

if you're trying to do astral photography before you're going to bed, because you've still got the sun up there illuminating the satellites. But much later in the night, it's much less of a problem. One of our spectrographs, Belche, is for looking at single targets, and it measures the rotation of those targets very accurately using spectrographic analysis. And for that one, I would imagine that we wouldn't want to use it too early in the night because there's too much reflected light from the sky itself,

that alone reflected light from satellites. But even then, the exposures are sufficiently long for most of the science targets that even if you do have a satellite going across it, probably wouldn't notice. And you'd do another exposure if you weren't sure. Yeah. Okay, I have two more big questions.

Christopher, do you have any before I go on? No, go ahead. What is a starbug, and how can I get one? That's one question. Well, I always thought the starbug was the little shuttlecraft from Red Dwarf, the British science fiction comedy. But when I was up there, I learned that Starbug was a re-engineering of the 2DF target mirrors, which instead of using a positioner to position the mirrors on the plate, the mirror would position itself by being vibrated into position. And we had people who had to go and clean those things once a week,

but I never actually got to see it myself. That was at a telescope down the road at the same observatory. Starbugs just popped out at me as something that sounded like a lot of on. You have been an engineer or been in engineering roles for quite a while. And you mentioned you started, you have a business degree, but you've been doing development for a few years now. Yep.

What advice would you give to folks starting their engineering careers or considering engineering careers? Firstly, it's rare to be asked for advice. Secondly, there are insufficient cases of my giving advice and seeing the outcome that I can determine if my advice is reliable or useful. Thirdly, it's against my interest to give advice because I prefer to be the only one on the planet we can do this. That way, the price that I can charge is the highest. But in general, engineering is best done repeatedly and in as many parts of the engineering as possible. So while I started in financial software, I moved to building complex systems out of

existing products when I was at digital and working in areas of engineering which I hadn't been experienced with before but utilised the skills and I just kept increasing the skills that I had and those charts of skills to learn are a good guide as well the other thing I learned from experience is that as I was already very good at engineering before I even entered the workforce because I had an engineering father, I was able to concentrate on a different degree, a degree that would serve me better, and that's why I got a business degree.

And since then, I've worked in every part of engineering, sustaining at one laptop for child, QA at one laptop for child, support when I was at HP, and support of software is funded by the engineering group. So when an engineering group gets the software wrong in some way and raises the support costs, they very quickly learn what they've done wrong

and they issue a release to reduce their support costs. And we would very gladly show them how they've done it wrong because we're all engineers too. So, yeah, get another degree. if you fixate on engineering, what you're doing is selecting your potential employers. You're reducing the field. And I never thought that was a good idea.

Because I'm working in Australia where there is so little population compared to other countries that my situation is different. Where most of your listeners are, in highly populated areas, you may need to use different strategies of specialisation and labelling and hence get an engineering degree. But once you got your degree, go get another one in music or something else and then

combine them because someone with a combined degree is in a smaller population of potential candidates. And so when you find that perfect job which combines those degrees, off you go and get it. The other thing I learned was not to overly optimize my performance because the rewards are not necessarily useful. And so I, how can I say this? I would tend toward picking roles which have an engineering component rather than being purely engineering. Yeah, that's what I've done. with my career. I haven't planned my career. It's just something that happened.

I think that's true for a lot of people. I've had plans at different times and pretty much none of them have panned out and I'm perfectly happy with that. Yeah, planning is a wonderful way of failing. That's a wonderful way of taking stock. Just because you plan it doesn't mean that's where you're going to end up, but at least you have an idea of possibilities. Still, my plans really haven't worked. Oh, one more question about telescopes, sort of. Simon wanted to know

if the Aussie wildlife is as dangerous for the telescopes as it is for the humans. Do you have any good animal stories that involve the telescope? Oh, no. Yes, yes. One of our technicians was bitten by a brown snake as he was driving

to the site, I think. He's written about that. in the car? He got out of the car and the snake was on the ground next to where he'd parked his car and it bit him immediately. Brown snakes are bad.

I would assume any snake in for... In Australia's. The Australian brown snake is the worst. We also had brown snakes living in the car park at the telescope and so we have signs at the doors reminding us as we leave

not to go that way. And we've had some of the holes sealed up. That's good. So that the snakes would go somewhere else. Another big one is ladybugs. Excuse me? Ladybugs. They're so bad.

What? They're venomous ladybugs in Australia that I don't know about? That makes sense. Fenomous ladybugs. No, I don't think these are venomous unless you eat high the LV-50 for them. But they get into everything. They've been buzzing around this National Park next door and see this white thing on the hill

and think it must be the moon, so they head in that direction. And they get into all the nukes and grannies of the equipment. And one of our telescopes on the mountain had such an infestation of flated bugs that it was skid out the broom and the vacuum cleaner and try and remove them. And there's, on the 2DF instrument,

on the way it's assembled onto the telescope because these instruments get craned on and off the telescope at different times. There's this couple of big cylinders about a metre or two long and half a metre diameter which you use to roll up the hundreds of metres of optical fibre bundle so that you can take the instrument off the telescope.

So just to explain, we pull the fibre out of the fibre. building, wind it onto these two spindles on the telescope, and then take that bit of the telescope off the telescope and set it aside on the floor and then put something else on the telescope. It's part of an instrument change, which happens every week or so. And in these cylinders, someone has written Ladybug Storage as a joke in that it's one of the things which would collect the ladybugs during observing, and you might have to do something

about it if there's too many of them or they give off a stink. The kangaroos are always a menace because when you're driving past them, if they haven't done their lift-off to get away from threat, then the direction that they're going to jump is going to be random and they don't always pick the direction you expect and in a given herd of 10 kangaroos they almost never pick the same direction unless one of the others has already picked their direction

in which case the others may follow. So if you arrive at a herd of kangaroos too quickly they will jump in all directions and one of those directions might be into your car. So yeah, that's where life at the telescope can be irritating. Got to hope the kangaroos and the brown snakes never joined forces. No, they don't seem to be very happy with each other.

Kangaroos will very much stay clear of the brown snakes. And the birds can sometimes warn you about the snakes. Like birds will react if you get too close to their nest and they'll persuade you to leave. But the same alarm call happens in a different way when you get close to a snake, which is weird. They're trying to protect you as well as protect their nest. Okay, I didn't realize ladybugs went towards the moon. But can you imagine being a moth and you get to the telescope and you're like, I did it.

I got to the moon. I did it. And now crawl into a hole somewhere. And that hole might be important. James, it's been wonderful to talk to you. Do you have any thoughts who'd like to leave us with? keep engineering. That's how we make the world.

Our guest has been James Cameron, developer of many things. Thanks, James. Thank you. Thank you to Christopher for producing and co-hosting. Thank you to our Patreon listeners Slack group for their many questions. And of course, thank you for listening. You can always contact us at Show at Embedded FM or hit the contact link onembedded.fm. There will be many links in the show notes and, of course, links to being able to support us.

We really appreciate it if you do. Now, a quote to leave you with. Victor Hugo. We see past time in a telescope and present time in a microscope, hence the apparent enormities of the presence.