@HPC Podcast Archives - OrionX.net - @HPCpodcast-72: Vanessa Sochat and Alan Sill – HPC.social Community

Episode Date: October 27, 2023

Vanessa Sochat and Alan Sill ,the creators of the HPC.social project join us as we discuss the broad HPC/AI community and their efforts to enable it digitally through a broad multi-channel platform that includs Slack, Discord, Mastodon, GitHub, a jobs board, a community map, and the effort's main site HPC.social. [audio mp3="https://orionx.net/wp-content/uploads/2023/10/072@HPCpodcast_HPC-Social_Vanessa-Sochat_Alan-Sill_20231026.mp3"][/audio] The post @HPCpodcast-72: Vanessa Sochat and Alan Sill – HPC.social Community appeared first on OrionX.net.

So I think two really important attributes of a community that I was looking for were just inclusivity and then having fun. We don't have to invent everything. We just have to provide a platform where people can bring their own ideas. That's excellent. It starts with engagement and activity, but it's nice that you're advancing towards collaboration and creativity. But sometimes you may also wake up and be like, you know, it would be really cool if we did this totally different thing. Something that no one's done before that I want to talk about with folks. Like, please bring those ideas, too, because it's not hard to build automation and things on GitHub and interfaces and portals and quickly turn ideas into life. From OrionX in association with Inside HPC, this is the At HPC podcast. Join Shaheen Khan and Doug Black as they discuss supercomputing

technologies and the applications, markets, and policies that shape them. Thank you for being with us. Hey everyone, welcome to the At HPC podcast. Shaheen, great to be with you again. Great to be here. Another very, very good episode that I'm looking forward to. Yes, we have with us Vanessa Socket and Alan Sill, the creators of the HPC.social project. Welcome to you both. Vanessa is a computer scientist at Lawrence Livermore National Lab. She is a software engineer working with the Converge computing team, mapping out the space between cloud and HPC, both technologically and culturally. Previously, she worked in research computing at Stanford, where she earned a PhD in biomedical informatics. And she was a researcher at Duke University, where she earned a PhD in biomedical informatics. And she was a researcher at Duke

University, where she earned a degree in psychology and neuroscience. Alan is managing director of the High Performance Computing Center at Texas Tech, where he's also been a professor of physics and where he's been for 31 years. He is also co-director of the National Science Foundation's Cloud and Autonomic Computing Center and is president of the Open Grid Forum. Alan has an extensive background in distributed and grid computing. So welcome to you both. Vanessa, why don't we start with you and tell us a little bit about the HPC Social Project. What was the genesis for the group? So Alan may be better suited to tell this, but I can definitely tell you my sort of perspective on the story. Now, probably most

of our listeners here know that Twitter or the platform previously known as Twitter now called X and no one knows what's going on. That was kind of the place where the HPC social community liked to talk. That's where we have HPC Guru and a lot of experts. And it was just a really fun environment. So as soon as all of this shenanigans started with he who shall not be named, we were kind of losing that community because many people started to leave. And so Alan was the one that kind of said, hold on, we need to first understand the needs of the community. So actually, let me hand it over to Alan, because he did some work with actually taking a survey to figure out what people wanted. And that was really the first step to creating HPC Social.

Well, thanks, Vanessa. I will say that the inspiration was partly because I knew that you and many others had expressed a desire to do something a little more inclusive than even the forum that was present on Twitter. So it wasn't just that that platform went south or sideways in various ways. It was also that there were other motivations. And you in particular had posted a number of things, including community maps and ways of sharing software ideas that were in the back of my mind when this all happened. I thought, you know, we can do better. So the background is actually pretty nicely captured on the site itself.

If people just go to hpc.social and click on the projects links or the news links. In the news link, in fact, there's a link to the YouTube version of a talk that Vanessa and I gave at the EasyBuild user group meeting this past spring. So I think for me, the motivation was partly that there is a community and my whole background has been in very community-based projects. The launch of the Open Science Grid, the Worldwide LHC Computing Grid. You know, particle physicists, I'm a particle physicist by background, you know, have lots of friends because we need them to build

these big expensive experiments that we have. And that naturally leads us to communicate with the web itself. In fact, as you know, was invented by Tim Berners-Lee at CERN. So these led us to consider when this event happened, that everything started to be in turmoil about how are we going to communicate if this platform goes south. I posted on some forum or other, who's going to buy the hpc.social domain? And people said, why don't you do it? So I did. And then as Vanessa says, we launched a survey. That's one of the first of our projects. And you can go and read the results of it. And we asked people, where are you? What do you want to do?

How do you want to communicate? How do you want to pay for it? And the results of that informed the rest. So for me, the rest started with launching a Mastodon instance. That was kind of our first outcome of the project. And that's up to over 500 accounts right now. So that's been a success. And then I basically turned to Vanessa and several others, Fernanda Forder, Felix Leclerc, several others who we knew and asked them, you know, what should we do? And so there's kind of behind the scenes, there's actually a little informal committee of advisors that we communicate with. But I turned to Vanessa because I knew she had all these other projects. And so I'll turn it back to you, Vanessa, because a lot of the rest of the site, the majority of the site is actually Vanessa's work. Sure thing. So I think two really important attributes of a community that I was looking

for were just inclusivity and then having fun, especially in communities that are grown out of academic ones. There's a tendency to be very serious, very professional. And it's not that that's a bad thing. Of course, if you're giving a talk and you want to come across that way, that's fine. But there's often this misunderstanding that, okay, if we're going to do great work, we have to be serious. And the kind of the message that I want to kind of send, or I want to kind of be embodied in a community is that we can do great work, and we can have fun too. And actually, the having fun part feeds into doing the great work, because you're doing the work with your friends, you're excited about ideas. And so those are sort of the underlying themes that I wanted to bring to HPC Social. And so

very logically, the first thing that we needed was a place, a portal, a website. So this is the main HPC Social site. It is more of a portal in terms of like you expect someone to come there that wants to learn about the community, find the resources that are available. Once we had that portal, and of course that's set up on GitHub because there's great page building there and an ability to preview things beforehand, then you just need to start to think about, okay, what resources does the community really want?

Well, one that makes sense and one that is useful is like to see where everyone is. One thing that is so great about the HPC community is that it's widely geographically distributed. So creating a community map was a no-bra. You know, this is also something that I've done before for other communities, I just really like to see and I think others like to see as well where people physically are. So that was one of the projects. And then in terms of resources, you know, you can kind of group those into buckets. So here's a bucket like, well, people need to be able to share opportunities

or find opportunity. So we should have a jobs board. it can't just be a jobs board. It needs to be really easy to submit to and update and there needs to be automation so that it posts to all the places that people are going to read Slack, Mastodon, Twitter, X, I don. I wanted to kind of one up the previous kinds of job boards that I've done before. So it has, you know, a nicer interface, better search. Then another thing is like people want to be able to share their ideas and read. One of the kind of weaknesses about the Twitter community is that there were a few very prominent voices that tended to lead to discussion. And that made sense, right? Because if you're, you know, HPC guru, I don't need to keep calling out HPC guru. They're just whoever they are, mysterious HPC guru. Yes. We all, we all love HPC guru. We love HPC guru, whoever you are. I have my thoughts, but no, I'm just kidding. I had my famous joke that I have found a very clean proof of who HPC guru is,

but this space is not big enough to describe it. Going after that format theorem thing. My take on that is that we have an obligation, if we know who HPC Guru is, to obscure that information. That's right. There you go. So to go back to HPC Guru, there were a few very prominent voices in our community. And my thinking was like, well, you know, that that's good. They have good things to say. They also, you know, inspire good discussion. But there's a lot of people, for example, that write blogs, and they'll write about something they're working on a piece of technology, and we should be able to elevate their voice too. So we first created this sort of HPC social syndicated blog, like a community blog, very quickly, we learned that

one blog would not rule them all, because quickly, we learned that one blog would not rule them all because actually there's different flavors of blogs. So imagine that you have a more commercial vendor project that is fundamentally different than someone who is hacking on the weekend and goes and writes about it. And maybe the frequency of the vendor project would be a lot more and would dominate the signal. So we wound up, I think at the end, having four different blogs. So sort of like for personal, commercial, project oriented, and I'm forgetting the last one. But anyway, now each of them has a feed that you can choose to subscribe to

and hear these different stories. So let's see the jobs board. A thing that came a little bit later is this based on this idea that when you're new to a community or an area such as high performance computing, you want to get involved, you want to have fun, you want to interact with projects. And so we wanted to kind of take the idea of a good first issue, which is a common label on GitHub

to do exactly that label a good first issue. And we wanted to go through a bunch of research software projects, find those issues, and then put them in one place. So someone new could come and be like, Oh, I'm gonna search for something with Python or something with this particular domain science. And that's gonna be my kind of entry point. And then of course, the projects that have already been stated, but are kind of obvious

and actually they're probably the most important are the places that we talk on a daily basis. So aside from Mastodon, this is also our chat places. So Slack, Discord, Actually, the Discord was brought in by another community member. Alan, actually, I think you might remember this. I didn't, actually. The Discord was started at Supercomputing last year by a student, Hayden, who has just offered it to us. In fact, this is not the first time that that's happened. The fairly well-known HPC huddle that Andrew Jones and others started some years ago has

decided to just pile in with our community and now uses the Slack and Discord and Mastodon links to advertise itself and is just part of the community now. So as Vanessa said to me early on, we don't have to invent everything. We just have to provide a platform where people can bring their own ideas. And if you have a new idea, the easiest thing to do is to hit the discussion tab, but that may be a little hard to differentiate just from Mastodon and chat. But what it does is lead to the GitHub discussions page where you can propose a new project or repository. Among all these, my favorite is the good first issues because it aggregates information. It's dynamically updated. It's a brilliant piece of coding by Vanessa and it allows people to pick ways of just piling in and getting involved with HPC software.

Yeah, that's excellent. I personally got to know more about your project through the HPC huddle, which I thought was a really good microcosm of HPC with a lot of really good deep conversations. And then I was pleased that all of that got all merged in. Could I ask, do you have sort of a membership count? I think you mentioned 500. Well, so the Mastodon is part of the Fediverse. And so what that means is we've got 500 accounts on mast.hpc.social, but lots of other people participate whose homes are in other servers in Mastodon. So the whole HPC community on the Fediverse may or may not have found us, but a fairly large fraction who have made that transition are there. Slack started later.

People said, let's have a Slack channel. And at first there was discussion, oh, well, that'll fragment the community. I think we've demolished the fragment the community argument. It's sort of like saying that you can only have one mode of transportation, right? If you have a bicycle, you can't take buses or something. So we use different methods to communicate and the community just settles on where it's comfortable. So Slack started later, but it's up to about 300 now. And Discord is hovering a little under 200, I think. So all told, and there's probably some overlap,

that's a fairly large chunk of the HPC community. In less than a year, we started this. We kind of made the paper launch. There wasn't anything physical at supercomputing last year and just made it available. And it's been growing since then. I'm really pleased to see the jobs board is very healthy. It's constantly getting new updates. And there's a nice variety of jobs. And also really pleased to see that it's global.

We have jobs posted. This is really important to me. I spent my career in highly international science. And I personally just, this is just a personal reaction. I don't like all of the things that are kind of forcing us within borders and boundaries these days. I like thinking a little more globally and I'm really pleased to see the participation is highly international and the discussion mechanisms we've discussed and also in the map and in the jobs. And that's a really good reflection of the HPC community anyway. So it also sort of indicates that you're capturing a good cross section, a good reflection of the community. That's really nice to see. Yeah, I want to second that. I found that in the Slack, it's a really kind of diverse set of people, not just geographically, but just in terms of like

people from national labs and academia and actual vendors. but it's still a place where you come and you, you know, you go on the humor channel and you post like jokes. Like I sit and I have dinner and I'm posting like memes and stuff. And I feel like I'm hanging out with my friends and really kind of at the end of the day, like that's, that's kind of what I wanted. And as a side effect, which is a great side effect, like I am learning a lot too, because people post, you know, their little niche of what is important to them. I learned something and yeah, I'm, I'm extremely pleased with that. It also feels like a place where you can be comfortable and be yourself. You can have some aspect of psychological safety where you're not going in and there's just, you know, one demographic, one kind of person,

and you have to kind of monitor what you say because, you know, I don't know, you might get in trouble for saying the wrong thing or something like that. Yeah, no, there's a space to say these things. So, I mean, this is, this to base is an important question because of course, moderation or the lack of it in other platforms has frequently been their downfall. We have had to throw out some folks who violate the terms of service on Mastodon. I don't think we've had that problem on Slack and Discord yet, but we have the mechanisms in place to enforce the rules, which are basically trying to make the conditions that Vanessa

described possible. We don't want to have people feel unwelcome, but we also don't want to restrict people's viewpoints. I'd like to emphasize another aspect, which is I've seen technical discussions emerge in each of these forums, and I'm really pleased to see that. I would like to emphasize another aspect, which is I've seen technical discussions emerge in each of these forums. And I'm really pleased to see that. I would like to see a little more of it on Discord. I'll be honest, Discord is a younger platform than me. And I don't know how to encourage that. Me too, Alan. I don't use Discord. It's for the youngins. No, I'm just kidding. They can do what they want. And this is kind of the point to my, find a space where people can do what they want, participate where they want, say what they want. There's not a membership form

you fill out. You just join a community and behave in a reasonable way. I may need to take back the comment about the age demographic of Discord. My mom plays a lot of computer games. She's a heavy gamer and she's in her seventies and she uses Discord. There you go. Yeah. There you go. But if we do have an HPC rock band, they probably are going to go on Discord too, right? That's the next project. I want to mention sponsorship just because people might wonder who pays for this. And so we do actually have a sponsors page. It's a little hard to find right now. We're trying to make it a little easier to find.

But there have been the two centers I'm associated with through the National Science Foundation's Cloud and Autonomic Computing Center and my own center have helped. The Open Grid Forum, which I'm nominally president of, but, you know, again, it's its own organization. And then the National Center for Supercomputing Applications. Those have been our big contributors.

Lately, we've seen a lot of interest. We posted a couple of pictures of some stickers that we'll have to pass out at Supercomputing, taking Vanessa's fantastic dinosaur and then giving it a base of the HPC social logo, text logo, and it really looks fantastic. So we offered a copy of this, you know, a sticker to people who made a donation, which you can do on the GitHub sponsors page. And we got a pretty big response that way. That's great. It's enough to keep it going.

And we're not looking to make money off of this. And all the costs go to just paying. We prefer to have a professional user serve. These things really are very pricey to do. And of course, you know, you put a lot of your own time and effort into it. So I heavily edited him in GIMP and I, I wanted to give him kind of like old school eighties vibes. So like that's, that kind of explains his coloring and like the sunglasses, but then I wanted to also like make him just look really cool. Like if I, if I were a dinosaur, like this is what I would look like. Yeah, man. Like he's the cool level I aspire to be. It is very cool. And someone commented

something that I had never noticed before until they said it. They said, well, you know, it's a bird. I said, it's a dinosaur. They said, well, you know, you replaced the Twitter bird with the dinosaur. Ah, there you go. Dinosaurs are birds. I said, oh my gosh. I didn't even think of that. That's very clever. Right on. There's also a Easter egg in that logo, if people look closely enough. I've shared it before, but. I did watch the YouTube video about HBC.social, and it mentions a low threshold for joining. Maybe you could talk a little bit about how people would join. And also, you mentioned, Alan, terms of service. Are there kind of rules of behavior on the platform to some degree? Well, because the first thing we launched

was Mastodon. I spent some time studying best practices there and did actually a fair bit of research on even the international aspects. Mastodon has an advantage that it doesn't do any data harvesting. So it makes it easier to run and comply with things like the GDPR and other data protection terms. But beyond that, there's actually been a fair bit of work done by people who wanted to ensure the success of the Fediverse. And actually, if you just go to the Mastodon page there and click on the about, it has a link to all our terms of service. And these are borrowed from open source policies.

We have a policies repository. It covers things like DMCA reporting. And I think it's fairly comprehensive. I'm pretty proud of it. But it isn't our own work. It, of course, leverages a lot of good work done by others in preparing that. So that was kind of the basis of our starting point. Also, as I said, we have it commercially hosted and the commercial host, which is based in Europe, though the servers are based in the US, in our instance, had its own set of guidelines and we just linked to them. We haven't had any serious pushback. We have had quite a few moderation requests from people who've noticed accounts being created for basically spam. And those are covered by these terms of service.

So it's, again, an attempt to enable conversation without interfering with it and without allowing other people to trample over it. Well, it's interesting because on Twitter, I was not really a user of Twitter until I came to InsideHPC. And you frequently hear in general talk about that platform, about incivility and insults and so forth and so on, which at least in the HPC community, I never encountered, never saw. You might see some critical comments about vendors possibly, but nothing untoward and nothing uncivil. How about on your platform?

Are you seeing generally good behavior? Overwhelmingly, we did have one instance of someone who's still a member of our community and had posted some stuff that got reported. There are, in Mastodon, there are mechanisms of reporting, which, again, it serves an extremely wide and diverse community itself. And so if you receive a report, you have to have moderation. So let me just pause here and say, we have a little moderation team of about four people that works on these things and responds to them. Most of the things we have to respond to are just spam someone advertising, using our platform for advertising unrelated stuff. So it's easy to clean up. But we had one case of a user who was posting a lot of political opinions and getting pushback.

They weren't terribly offensive, but they did get pushback. And we used the mechanisms in Mastodon to simply advise them of this. And that person chose to just separate the accounts and use the HPC one for the one on our server for HPC related content. So overall, it's been a very positive experience. And we do describe it fairly well as to what it's about. So I think people know what they're getting into. And in particular for that story, I want to point out that there was very kind of clear communication between not just sort of like the people that were accusing and then the moderation team, but also the moderation team and the person being accused. What I've noticed often happens when,

you know, there's a report to a code of conduct or some kind of violation like that, is that immediately the person who's reported is assumed to be the one in the wrong, and they are treated like the perpetrator, or maybe they're not, but maybe the communication isn't clear with them because, I don't know, some reason it's not set up well. And so it is so important, and I'm really glad to see that we have this, that we have very kind of frequent and clear communication between all parties. And we can have a discussion about what happened to get to a resolution where everyone understands kind of what the other party was thinking, maybe when they did the report or when they

posted it. And then we can figure out how to move forward in a way that makes sense together. Because at the end of the day, like we were part of the same community and we have similar desires and goals. No one really that wants to do that is going to go out and be purposefully malicious.

Usually it's something like, oh, well, I was posting my political opinions, but you're right, maybe I should make a separate account for that. There's very good solutions, but if you approach that in the wrong way, it can turn out feeling just very bad for one side. Like, oh, I I'm, I'm a terrible person and I did this wrong. And I don't want to be a part of this community. That would be the bad outcome. That's a really good point. Also just as a user of this platform, I have never seen anything like

that. I may have seen, as you mentioned, Alan, like posts that are a little bit too advertising-y or too kind of maybe not quite related, but even that has been like one or two. In general, it's a very, very solid community. We've tried to make it clear as Vanessa said, personal use is permitted, but the purpose is what we try to communicate. We're all here for being part of the HPC community. Now, if we go out for beverage at supercomputing, we're likely to talk about a range of things. And so we're trying to have space for that whole range of things on all these platforms. And by and large, I think it's been successful.

I guess the reason I bring it up is not to really any bad experience, but the opposite to say that we are prepared if a bad experience does come. We have the basis in terms of rules to take action, and we have taken action to kick people clearly who were not here for that purpose off, betting frameworks and so forth like that have been kicked off by the moderators. I generally don't have to get involved. The moderation team does a pretty good job. That's great. Let's talk about SC23.

What are you planning for that? That's like an opportunity for this virtual, can I call it metaverse, community to actually meet in person, at least those who are attending physically. Do you have anything going on there? Well, I have over 1,000 stickers to pass out. There to pass out and I'll be available to any booth. Sign me up. Thanks to the generosity of our supporters. Beyond that, I want to mention the Beowulf Bash. We will have a presence there as well. And one of my organizations that's also a sponsor of racebc.social is actually sponsoring an event I want to opportunistically take the time to mention, which is that it's the 25th year.

This is the 25 years since the first BOF that was at Supercomputing 98 that launched the grid movement, the international grid movement. Oh, wow. Yes. The open grid forum is going to be sponsoring a room at the Beowulf Bash. By the way, if you haven't been to beowulfbash.com, take a second to type it in. It's a lot of fun this year. You'll recognize the font and the logo that they have chosen. I won't spoil the surprise by saying it's going to be a lot of fun. This is an event totally in the spirit of HBCU.social, even if it's put on by others.

And as I said, we'll have our 25 years of GRID celebration there and lots of stickers to pass up. That's excellent. Look forward to that. On my side, so I'm not attending SC in person. I'm still very cautious about COVID, but I have two virtual-ish things planned. So the first one is called the Noodles Award. And I don't know where this came. Well, okay. So the story basically is that someone said something to me and it was it was about a specific product and I don't need to share the details. But I so I'm someone that has like a lot of kind of visual I have a very vivid imagination. So I have a lot of kind of funny fantasies,

just in regular conversation. And so I had a visual of being kind of flustered with a particular tool and going and finding the person that made the tool and taking a bowl of noodles and dumping it on their head. It was so hilarious to me at the time. I was like, oh my goodness, there's so many things out there in our space, in technology or tools, like when you just want to go dump noodles on someone's head. And so the Noodles Award is a place where you can kind of, it's a form you can go and fill out when you have a frustration with something very general that maybe is kind of funny to you. And you can kind of rant about it there. And so you give your contribution. And I haven't totally decided, you know, what the venue venue will look like for that. I think probably what we want to do is make some kind of website and just kind of share

the noodles and just have fun about it. Anyway, that's going to be fun. That's going to be sort of released around the time of SC. We definitely have a good number of submissions already. We can share the link with you folks so that you can share it with your listeners as well. The other thing that I'm really excited about that is still a work in progress. I don't even know if I'll be able to finish it. But I started one night singing a song, which I'm not going to tell you what song it was about a workload manager. And again, this this was absolutely hilarious at the time. So I started to

make a music video basically with all different lyrics for the song about this HPC workload manager. I've recorded the singing, I've recorded, you know, dancing and various scenes in different places. And then I said, you know, this would be a lot more fun if I brought in other people, because no one wants to see me like poorly singing and dancing for the song. So I put out a call to the community, like if you want to participate in this, like, please ping me. And I've had a couple of pings, but no one yet has sent me any material. So I'll give out another call now if someone is interested in participating in really what comes down to kind of a parody music video that will be kind of released with SC, please get in touch. I would absolutely love to have you and And the minimal contribution really is just like a video of you rocking out at your computer because we have the audio. Really what I'm looking for are just other people like having fun, video of you having fun. So those are my two, I guess, I suppose, contributions to SC coming up. I want to mention something about opportunities to contribute as well in the actual nuts and bolts of all this.

As I said, this is pretty much all except commercially hosted stuff, open source projects. And so, for example, our jobs board is actually backed up with a set of bots that Vanessa wrote that cross posts to Slack and Discord, as well as to Mastodon, the job spot. Among the good first issues that we posted are a couple about our site itself. So even if you don't have a brand new idea, you can go to the github.com slash HPC social page and or follow all the links on the hpc.social site to it and find ways of making all this better.

We certainly wouldn't turn down content and design improvements of any sort or new ideas. It's a community effort and it will be valuable to the extent that the community participates. That's excellent. It starts with engagement and activity, but it's nice that you're advancing towards collaboration and creativity, which is also an indication of the maturity of the community. Right. And the community is a reflection of its members. So if, you know, of course there's good first issues that you can find, but sometimes you may also wake up and be like, you know,

it would be really cool if we did this totally different thing, something that no one's done before that I want to talk about with folks, like, please bring those ideas too, because it's not hard to build automation and things on GitHub and interfaces and portals and quickly turn ideas into life. And absolutely, I love doing that. So if you have a new idea for something fun, for something useful, please bring it to the community and let's chat about it. Yeah. And if nothing else, track me down at the Beowulf Bash and we'll have a conversation. Very good. When you all got this thing going, did you notice or have you noticed, is there particular features or topics where people really seem to be very much into it and

that seems to be very popular, if you will, or that people are really congregating around? I would say that for a lot of the huddle meetings, the emphasis tends to be on kind of new releases of things, new hardware updates from kind of the vendor ecosystem. I would say personally, I think you'd get a different answer depending on who you ask. But the space of things that I'm excited about tends to be more in kind of containers and the integration between cloud and HPC, this thing we call converged computing. So when I post, I'm usually talking about something along the lines of, oh, I'm porting HPC toolkit into Kubernetes. Can we talk about this particular thing?

Or, oh, I'm curious about this particular thing in MPI. MPI is actually a really interesting thing to talk about it because my first exposure to MPI was through containerization. So through singularity, exposure was just like, it's pain because, you know, everything that would come in would be like, oh, well, my MPI in the container isn't working. And then sort of the mantra was like, oh, well, the version of MPI in the container needs to be the same on your machine. And then you learn about that there's different versions, there's different implementations of the interface. And it's not,

it's not so simple just to run it. Like just getting one thing working the way you want it is so complex. So I'm actually really enjoying diving into MPI and learning more about the different kind of methods and what the calls are actually doing and going into more performance analysis. So that's, to use sort of a cliche term, that's kind of where I'm geeking out at the moment. Because this is an area in HPC, this kind of work that I've never been exposed to at this kind of level of detail. But it's interesting in the way I'm approaching it. I'm approaching it by way of integrating it into a cloud environment.

Yeah. If I just quickly look at the Slack channels, people can propose or add channels to Slack, and it's easy to see in that platform what they are. There's probably 15 or 20 topics ranging from benchmarks to containers to DevOps to hardware. I've been pleased with the security one in particular, and the hardware and software topics are often quite interesting. But we get people posting just questions like, how much power should I plan for in my HPC center that we're building for RAC? And I'm pleased to see that the community jumps in and answers those. We mentioned the HPC huddle, which if people haven't found the HPC huddle, that is a lot of fun because it's deliberately scheduled following a pattern that Andrew and others developed early on to be at different times on different weeks so that people in different time zones have a chance to participate.

So that's a healthy one. Let me give a boost to the HPC-Carpentry channel and Slack and Discord. Those are part of the Carpentries project, and they have been trying to design lessons aimed at people new to HPC. And frankly, they could use a little more participation from the community. They post their meeting times every week. They're very open to input. Andrew Reed and Trevor Keller have been driving that effort. But I would love to see people dive into the HPC Carpentry channel and help them design this curriculum that's aimed at being delivered to the carpentries methods, but aimed at people new to HPC. That's excellent.

Yeah. Maybe we can throw a question back at our hosts. What kind of resources or projects, what kind of things are you looking for? What adds value to your day to day? Right. Let the record show that Vanessa has a podcast where she does the interviewing and she's prone to go right into that personality, which I love. Exactly. No, that's excellent. I'll go first and Doug, you can comment. My view is that it would be nice to cover all the aspects of HPC. Obviously it's easy to cover hardware because top 500 is there and it's not a very tall stack. The software and storage we've teed up, we've drilled down into some aspects of it. But I would say that probably the biggest gap that I would like us to fill in a more coherent, holistic way is really the

software stack. Of course, one challenge is that it just really is a tall stack, especially these days. And if you add the containers, it isn't just like the compiler libraries of the old days. So that is also because the industry doesn't really present all that software stack in a coherent way either. So with my industry analyst hat on, I've been having a side project to try to see how I can organize it and how I can present it in a way that is simple, but not simplistic. And it allows me to put my arms around it. Doug, what do you think?

Yeah, I like those comments, Shaheen. And also among publications, journalists, editors covering HPC, supercomputing, and in the analyst community, we're really focused on what's the next big thing coming out. I was really impressed last year as we approached and then did achieve Exascale. A lot of the discussion was already on what comes after Exascale, but that technology is so far ahead of what most people are using actually now. And I think this is sort of an issue, I think, among publications and analysts where we're out ahead of the actual user community. So the focus on what people are using

now actually doing now, I think is really so important. That's an excellent point and really reassuring coming from an actual journalist. This is really excellent. Yes, we did include journalists as a category in the initial survey and we got a few responses. I have seen in the Fediverse, there's been a fairly wholesale shift of journalists to that platform. It's not one we particularly host on mass.hbc.social, but there are lots of journalism-oriented posts, and it's fairly easy to find, especially now that the latest version of Mastodon includes global search capability that you can opt into to let your posts be searchable. This was a widespread complaint.

The other thing I see that people encounter when they first use Mastodon is that, unlike other platforms, there is no algorithm. There's no feed. Even to the extent that if you post a reply to someone on a post that they made, you don't necessarily see the replies to that post unless you navigate to their server. So these are kind of getting started issues. People sort of sign up and expect to have this firehose of content shoved at them. No, no, it's much more self-driven and interactive, but it's been a good match. I personally hope that someday Twitter gets fixed again, though that seems like a dim hope. I have nothing against that platform. I enjoyed my time on it.

I think Vanessa's comments about its shortcomings are there. But that's one reason we have many different platforms for discussion. No, you do have several channels between Slack and Discord and Mastodon and the video conferences that-

Mattermo, Microsoft Teams. Bingo, right? The hardest thing about our jobs is like, okay, I need to talk to this person. Where are they again? That's right. Exactly, exactly. And like you said,

some people hover in certain areas and not in other areas. So if you want to reach them, that's where you got to go. Right. And Alan, I apologize. I laughed a little bit when you said the algorithm

because I remembered back to when they put that repository in GitHub called the algorithm and it was just hilarious. And oh my goodness. I don't know if that code's still online. I need to look at that again. I'm not here to bash on the other platform, but I do think that we have to acknowledge that one of the instigating aspects of this particular shift was the deterioration. And once we got started, it's kind of like rolling a ball over the top of a hill.

You know, we just kept going and all these other things came out. You know, Alan, my view is more like what you said, is that as good as all these social media platforms are, they don't do everything. And if you want to have a video call, and if you want to have it with a group of like-minded folks, then get set up on Slack. So now you have that as an additional channel. So I think a multi-channel approach to community is the way to go. And you're absolutely doing that. If you have a blog, we will be happily syndicated. If you have a podcast, do we know anyone with a podcast that we could do our podcast space? You did offer it to me.

I acknowledge you did offer it to me like two months ago. And I bookmarked it to go to it. And I haven't got around to do it. So I will do that. But if people listening to this have their own material that they'd like to get out there, you know, contact us. We've made a semi-infinite number of ways of contacting us available. You sure have.

That's right. Very good. Excellent project. I applaud you and the rest of the team that is helping you and the rest of the community. I think it's such an important thing for this HPC community that obviously is critically important for humanity, if you there's a couple of things there. There's, you know, how to build software. And, you know, then you could mention SPAC, you could mention EasyBuild, distributing it, so you could mention environment modules, and that sort of thing. I kind of want to provide, I want to give the perspective of where I come from. And that is that users of, you know, HPC systems, research software, they often don't know what they want. And so I feel like a lot of my job,

and actually our job in general, as developers that make infrastructure that make container technologies is actually establishing some of that vision for what the future might look like. And that future can't be like this entirely new thing you have to learn, it usually has to meet the person where they're at, and then maybe take a step more towards, you know, modularity or automation or that kind of thing. And this take a step more towards, you know, modularity or automation or that kind of thing. And this is especially relevant for convergence. You know, you ask the question, like, what's after Exascale? In my mind, I'm like, well, I think we kind of need to better integrate things. And this is the idea of convergent computing, you have kind of meta

schedulers, and you're bursting work off clouds. And there's, there's so many cool things to do. And to answer your question about apps, one thing I've been doing is actually building a lot of apps. I think I'm up to like maybe 20 to 30 now into containers, getting them into Kubernetes, but doing that in a way so it's very easily reproducible. So there's something called a Kubernetes operator, which if you think of a human operator, like a person, so to actually step back even more,

to interact with Kubernetes, you write YAML files, you apply the YAML files with a tool called kubectl. And then, you know, it creates abstractions in Kubernetes. So it creates pods that have containers, maybe the pods are in deployment. There's now an abstraction called a job and actually a job set as a SIG that looks a lot, it's under the batch working group. And it looks a lot more like what we would consider a job in traditional HPC. And that's sort of where the two communities can start to have this conversation. So what I'm trying to do is take a bunch of our very traditional kind of HPC benchmarks, proxy apps, different ways to measure

performance, get them into an operator that knows how to quickly deploy these things as job sets into Kubernetes. So as a community, we can very easily kind of bridge that space, share the apps that we're developing, but also understand how they're performing. So one of the reasons I mentioned HPC Toolkit before is because this is a totally uncharted territory. The example I always give is that we're on a ship and we're on an adventure in the fog of war and we're trying to figure out where we're going. But it's really interesting because you have a perception like, okay, this app absolutely needs low latency to run. It will not work if it doesn't have that. And then you move it into a cloud environment and maybe it

doesn't work well at first. And you kind of tweak some things, you try a different instance type, but then all of a sudden, you know, on a different instance type, it works, but you step back and you're like, wait a minute, I measured the latency using this operator or however you did, maybe the OSU benchmarks. And it wasn't very good. My previous perception that this app absolutely needed the lowest of latency that only HPC can offer, maybe that was wrong. And actually, that is a really interesting kind of step that we're going to take. Better kind of understanding how our apps are running between HPC and cloud, and then how the needs they map between the spaces. It is such cool work. There's so much to be done. I am so freaking excited about it.

That's excellent. But part of part of that to go back to what we're talking about is really figuring out what are the steps that someone takes who's a traditional HPC user to using something like the cloud. So it isn't painful. So isn't hard. So it isn't like, oh, I need to learn what Kubernetes is. Like, I don't want to do that. That's like really hard and complex. So anyway. That's right.

That's right. Those are all right on. All of those issues is what makes software such a complex thing. And of course, part of it is like, what is it? And then how do you use it? How do you use it wisely? How do you use it instead of this other thing? And that's just an endless, endless set of topics for us.

Job security. Just kidding. Yeah, exactly. Yeah. So we do have the OpenMP ARB is on our Mastodon server. There's a whole bunch of technical projects on there that post. We have hardware and software channels. But I want to make a point that's kind of a level up, which is that, yes, we have the

technology and that's what we're all interested in HBC. But for me, this project is about the fact that HBC is carried out by people. And this is really a set of mechanisms to let people communicate with each other. We have a humor channel. We have a pets of HBC channel. So, you know, there's ways for you to post immediately if you get involved. And I want to make sure that we don't make the threshold too high, as you said in the other talk, you know, a very low threshold for joining. Good point. Excellent point. I love the humor channel. That is definitely my favorite.

I frequent that quite regularly. And I've even posted a few things. Yes. Thank you. Thank you both. Yeah, thanks for having us. Super fun. Okay, we've been with Vanessa Socket and Ellen Sill. We've been talking about hpc.social. Really a great talk. Thanks so much. Thanks for everything you're doing. And we'll talk again soon. Thanks so much. Sounds good. Thanks a lot. Take care. Thanks for making it happen. That's it for this episode of the At HPC podcast. Every episode is featured on InsideHPC.com and posted on OrionX.net. Use the comment section or tweet us with any questions or to propose topics of discussion. If you like the show, rate and review it on Apple Podcasts or wherever you listen.

The At HPC podcast is a production of OrionX in association with Inside HPC. Thank you for listening.