Microsoft Research Podcast - Intern Insights: Dr. Josh Benaloh with Anunay Kulshrestha and Karan Newatia

Episode Date: September 8, 2023

Every year, interns from academic institutions around the world apply and grow their knowledge as members of the research community at Microsoft. In this Microsoft Research Podcast series, these students join their internship supervisors to share their experience working alongside some of the leading researchers in their respective fields.In this episode, PhD students Anunay Kulshrestha and Karan Newatia talk to Senior Cryptographer Josh Benaloh about their work this summer on ElectionGuard, a free, open-source toolkit that enables voters to verify that their votes have been accurately counted. Kulshrestha and Newatia discuss their contributions to extending ElectionGuard to mail-in voting and rank-choice voting, respectively; what is needed for widespread adoption of the verifiable election technology; and why they’d recommend a Microsoft internship to other students.(Editor’s note: After its design and development by Microsoft, ElectionGuard is now part of the newly formed nonprofit Election Technology Initiative, which will join with Microsoft to further ElectionGuard’s growth and help advance its adoption.)Learn moreElectionGuard | Project pageThe next frontier in elections: Microsoft supports the Council of State Governments’ Election Technology Initiative | Microsoft On the Issues blog, July 2023Security and Cryptography | Group page

I think the fact that I myself came back here probably says something and I mean like both the interns and the full-time researchers they are truly experts in what they work on and it has been a joy to get to work on the projects that I did not just this summer but last summer as well I was often told that MSR was is very special place, and I did not realize that until I got here, and I completely agree. I think it is very different from a lot of academic departments, but also very similar in a lot of ways, and I enjoy that. I enjoy the differences, and I also enjoy the similarities. Welcome to Intern Insights, a Microsoft Research podcast featuring brilliant students who are contributing to the research and advances at Microsoft as part of the renowned internship

program at Microsoft Research. I'm Dr. Josh Benelow, a senior cryptographer here at Microsoft Research, and I'm talking today with two of our finest interns, Ananay Kulshrest and Karan Nuwathia, who are working with me on a project called Election Guard, a free open source toolkit that allows voters to check that their votes have been accurately counted. So welcome, Ananay. Welcome, Karan. I'm going to start by asking you to introduce yourselves and talk a little bit about your own background, where you're from, your schooling, your graduate work, what's going on there. So Ananday, would you like to start? Sure. Thanks, Josh. So I grew up in India. I grew up in New Delhi.

I moved to the U.S. for my undergrad at Stanford in math and computer science. I was very interested in public policy, so I stayed on for a master's. And I've been trying in my graduate work at Princeton, I've been trying to apply techniques from cryptography to public policy. And my graduate work is mostly techniques that are useful to public policy problems and working on the interplay between privacy and accountability and how cryptography can offer solutions there. Wonderful. Karan, would you like to say a little bit about yourself? Sure. Yeah. I'm originally from Assam, India. Then I went to high school in Delhi. I came to the US for college. I went to Cornell in upstate New York. I majored in computer science and I decided to do a PhD. Currently, I'm a third-year student at the University of Pennsylvania.

My research focuses on large-scale systems, especially the privacy and security aspects of large-scale systems. Great. Thanks. So there are a lot of opportunities that you have. You've done great work as students, and you could have stayed at your university. You could have gone other places. Why did you choose to spend the summer at Microsoft Research? So I'm uniquely interested in elections, I think. I consider elections to be a very important public system that cryptography can improve upon. My interest in these sorts of techniques predates just elections and the election problem.

I think the same techniques can be applied to auctions as well. And so when I heard about Election Guard, I think I was immediately attracted to it, and that's why I applied, and that's why I'm here at Microsoft Research. Karan? Sure. So I wanted to come to Microsoft Research because I wanted to apply research in a real-world setting, the problems that industry cares about. And I interned here last summer as well.

I interned in the data systems team, and I loved what I did last summer, so I wanted to come back. And this time, I wanted to do something that was closer to what I do in my PhD research, so I decided to apply to the security and crypto team. And then Josh sent me an email about this project called Election Guard, which I was completely fascinated by because the underlying crypto that's used in this project is extremely similar to what I'm already doing in my PhD. And I wanted to use what I already know and want to learn more about in a real-world context, such as in a selection context. So let's talk a little bit more about election guard.

I wonder if you can tell the listeners a little bit about what it is and how it works. It's okay to go into the math a little bit, get a little bit geeky, maybe talk a little bit about the fundamentals of homomorphic encryption and how that leads to things. Why don't you start, Karan? Sure. So election guard is generally about end-to-end verifiability while still maintaining the privacy of individual voters. So what this means is there's a few properties. Firstly, a voter should be able to verify that their vote was correctly counted. This just means that if they voted for candidate A, then they should be confident that the vote was actually recorded for candidate A. Secondly, this also means that anyone should be able to verify that all the correctly recorded votes were correctly tallied up. And this preserves the integrity of the election. At the same time,

other voters should not be able to know what a voter's vote was. And they should not be able to prove to others who they voted for, because otherwise they could sell their vote. Now, that's like the high-level idea. In terms of how we actually do that, there's a few cryptographic tools that we use. For example, homomorphic encryption.

This is used to encrypt the user's vote and add up all the voters' individual votes. Homomorphic encryption allows us to do computations on ciphertext. So let's say there's two candidates A and B. And say a voter voted for candidate A. Then the vote is an encrypted vote of candidate A, and similarly for the other candidate. And the election authorities can simply just sum up all these individual votes which are encrypted, and then decrypt that later. So this allows the votes to be tallied without ever being individually decrypted.

They're only decrypted in aggregate. Right. Only the final result is known. Great. Do you want to add anything? Yeah, I guess we could offer that analogy. So end-to-end verifiability is fairly easy to do if you don't care about ballot privacy. Everyone votes in a public fashion. All votes are public. And so now it is very easy for a voter to verify that their vote is indeed counted. And it's very easy for everyone to verify that the tally is correct. But of course, we don't want people to vote publicly.

So that's why we need cryptography. And that's why we need homomorphic encryption to be able to compute on private data and compute this tally function eventually without revealing anything about the votes themselves. Great. So one of the limitations of the original deployment of Election Guard was that this homomorphic encryption was done only after voters indicated what their preferences were. And that has some limitations because it can't be used well for things like mail-in voting, because there's no way for the encryption to be done in front of the voter on a piece of paper. We're dependent on voters using the devices in precinct in order to vote. So we've done some work to support

mail-in voting. And Anilil, can you talk a little bit about what you've done there? Sure. So in the current design, in the new version of Election Guard, we allow election authorities to pre-encrypt ballots which do not contain any voter selections. So these are just encryptions of every possible selection that a voter can make. And a voter can then choose what selections they would like to make, just as a normal mail-in ballot where you get to fill it in in ink. And so a voter can just select which encrypted selections they want. And later on, election authorities can aggregate voter selections

and compute an election record, which looks indistinguishable from an election record produced by in-person voting. And you can still have the guarantees of end-to-end verifiability with mail-in voting, although the challenge part is a bit different because you have to ask for a new ballot every time. So that is the main difference between the mail-in challenge and the normal challenge. Indeed. Another one of the limitations in the original deployment of Election Guard is the dependence on homomorphic encryption and homomorphic tallying

was not compatible with rank-choice voting. So, Karin, I'd like to ask you a little bit about what work you've done to enable rank-choice voting to be used with ElectionGuard. Sure. So, as Josh mentioned, certain voting systems such as rank-choice voting or write-in votes cannot be done using homomorphic tallying. So, the idea that we use is called MixNets. So, MixNet, basically, at a a high level it shuffles the order of the votes such that after the shuffling is done the random order that we get at the end does not reveal anything about the voters. So in particular let's say there's a few entities which are the mixers. Once the voters have voted their votes are still encrypted.

But then instead of being homomorphically tallied, they are sent to the first mixer. The first mixer re-encrypts all the ciphertext, which just means adding an encryption of zero to each vote, and then permutes them by shuffling the order, and then pass it along to the next mixer and so on, until we get the order from the last mixer. Once we get the final random order from the last mixer, we can decrypt the votes just

as we did in the homomorphic case. And doing so does not reveal anything about the voters' choices themselves. Okay, so while we're down in the weeds, there are a few more things that we've been working on this summer. And another one of the challenges that Election Guard has had is that when site. Can you talk a little bit about that. Yeah. So the key problem here is that I, as a voter, would like to test that this ballot marking device isn't cheating me. And it is not very satisfying, like you said, to know that after two weeks, right? So the idea is that voters should know there and then should be able to unlock ballots that they don't want to cast at the polling booth itself. And so the technique that you designed that allows us to reveal the randomness that was used in encrypting the vote, instead of revealing the key, because we can't reveal the

key, the election, an individual vote cannot be decrypted. But what can be done is the randomness that was used to encrypt it can be revealed to the voter and the voter can re-encrypt the vote using the same randomness and check whether the machine encrypted their vote correctly. So in some sense, instead of decrypting something, we're re-encrypting the same data and checking whether the encryption is the same or not. And so in some sense, we get a check on correctness of the ballot marking device, and the voter can do this using, well, in theory, a calculator and a paper. But I think it's better if they use an independent verifier app to do this. And can you say a little bit about the study that you're working on right now, feverishly

trying to get in place? Yeah, we would like to test how voters feel about doing this sort of verification in person, how they feel about using an external app that is just distinct from the ballot marking device, what the effect of using paper receipts is. We're interested in all these questions and trying to increase voter confidence. So we're going to go into the community and go into the field and try to catch some voters and ask them questions. I'm very much looking forward to that experience and what we learn. We've already learned a lot just in trying to prepare the study. Lots more when

we actually do it, I expect. So, Karin, another thing that people have raised about Election Guard is the fear of quantum computers and how quantum computers might come along and break cryptographic methods sometime in the future. It's not really an urgent concern for Election Guard, but it's a long-term concern, something we need to be aware of. Can you talk a little bit about the research that you've been doing to study alternatives to address the threats of quantum computation? Sure. So I think this is a real and a very important concern. So concretely, I guess the issue is that what a malicious group of people or the government themselves could do is

they could throw the encrypted votes right now and then decrypt them later using a quantum computer once that becomes efficient. So currently, we are using the Algamal crypto system. And while it is very efficient, easy to understand, and so on, it's not quantum resistant. So this means that we'll have to start thinking about how we could move in-person voting systems with end-to-end verifiability to ones which are quantum resistant. So I've been looking into some of the latest work in this area. And there's a lot of promising papers that have come recently. So for example been looking into some of the latest work in this area, and there's a lot of promising papers that have come recently. So for example, there's a couple of them which are built

on the hardness of the lattice problem, in particular using the BGV crypto system, which is quantum resistant. So one challenge with BGV as compared to El Gamal is that it's not very efficient currently, at least, to prove in zero knowledge that the encryption was done correctly. This is something that we had not discussed before, but since we have been talking about verifiability. Now, one important aspect of this is proving that a user's vote was encrypted in a well-formatted manner. This means that when a user goes into a vote in a voting machine and they vote for a particular candidate, the voting machine should not record their vote

to be for multiple candidates or for no candidates, for example, if it's an election where they can only vote for one candidate. So this has to be proved in zero knowledge, which means that the voting machine or the prover is able to prove that there's only one vote and no less and no more. But it doesn't actually reveal who the vote was for. So while this is fairly efficient to do in the Algamon crypto system, this is still not the case for the BGV crypto system. So one important challenge going forward would be to think about how we could adapt the existing ideas, such as the one I mentioned, using lattices, and explore the research aspects of how

to make proving zero knowledge or even encryption itself more efficient and bring it closer to being deployed in practice. So one challenge with election guard and the techniques that are used there is that there is mathematics behind it. And although we can mathematically prove something, we may have trouble convincing individuals who may not like mathematics, may not be immersed in it, that this really proves that an election is correct. We're asking voters to do things like check their votes and challenge things, and it's possible that no voters will challenge it. So what kinds of challenges do you see in getting voters to accept this technology, to use this technology, feel comfortable with the technology? What can we do here? So I think voter education is key.

Without voters understanding that their ballots are safe and secure and will not be individually decrypted, only the result will be known. Without them understanding that, I think a lot can go wrong. A lot of misinformation can be spread about these sorts of verifiable election systems, which the sorts of misinformation that is impossible to spread about current election systems, right?

And that obviously remains a central challenge. But like you mentioned, I think the biggest challenge is just voter apathy, right? What if no one cares? And I think that's a larger societal issue, right? Like general apathy about our political institutions is on the rise. And I think using this technology today is great. It can improve what we have tremendously, but there are still questions. And Karan, I know you've been looking a little bit about dispute resolution, for instance, as one of the things we'd like to find better ways to handle. Can you talk about that a little bit more? Sure. So this problem of dispute resolution basically is that, so while currently voters have the choice to either cast or spoil their ballot, this means that after they have entered a vote,

they can either choose to challenge it, which means that, as Anunne had mentioned before, that their vote will be decrypted and shown to the voter, or they can just choose to cast it, in which case it'll be encrypted. And the voter then cannot prove to anyone else that they voted for a specific candidate, because otherwise they'd be able to sell their vote. But this creates a problem because since the voter isn't able to prove who they voted for, the election authorities, if they are cheating, there's no known way right now to resolve this conundrum. So one of the challenges of dispute resolution is that even though a voter might be able to tell that something is wrong,

the voter might not have quite the evidence that would be needed to show a third party and prove that something went wrong. Can you say a little bit more about how that fits into dispute resolution? Sure. So, yeah, as Josh was saying, even though the voter themselves individually know that their vote was recorded differently, they are not able to prove to a third party that this was the case. Say the voter actually voted for candidate A, and then they challenged their vote. But when the vote is decrypted using the verification process, it's actually shown to be candidate B instead of A. But only the voter themselves knows that they voted for candidate A.

And of course, as we had mentioned before, they cannot prove to someone else that they actually voted for candidate A. So this creates a problem of how to resolve this issue. I think this is some combination of cryptography and policy because there's a broader question here of like, let's say we had a technique to resolve this. This would mean that the integrity of the election

would be in question. So how do we then resolve it? Like, do we cancel the whole election? Do we try to find the source of the bug and then let the voter vote again, or what else do we do? One thing I think that's being highlighted here is that elections aren't just about math, that elections involve mathematics, but there's a social component. There are advocacy aspects and rules that have to go into this. Election administration is a discipline unto itself. So one of the interesting things about working on elections is bringing together people from

these many disciplines to find solutions to some of these problems. I wonder if you have any thoughts or experiences on how to try to bring a mathematical solution into the world that is not so much about mathematics. I think we begin by hiding the mathematics, but making it more palatable, I think, in ways for voters that are not as mathematically inclined, I think, is a big first step. So I think WhatsApp has tried to use visual fingerprints, right?

There's an entire literature on using non-mathematical ways to allow people to deal with cryptographic objects. I think that line of work is central to deploying these sorts of verifiable elections. On the point of dispute resolution, I think one thing that I think people should understand is that the problem here is that we would like voters to not be able to prove who they voted for if there was no problem, but be able to prove who they voted for if there was a problem, right?

And that is the central issue with dispute resolution. Yeah, that certainly is one of the deep challenges that we face in this. Is there anything else that either of you would like to add? Sure.

I think regardless of the current open challenges, this is like a great step forward because I personally believe that all data that leaves your, for example, mobile device should be encrypted by default. This is currently not the case.

And I know we are talking about in-person voting, but generally I think a person's data should be encrypted by default. And this is certainly not the case right now. And this is why there's a lot of data breaches in the first place. So if everyone's data is secure, then even with breaches,

there's nothing a hacker could learn. Yay, homomorphic encryption. So Microsoft Research is a very active place. There are many other interns around doing many things. Have either of you had many opportunities to talk with other interns and find out what they're working on and interact, learn about their work and contribute to yours as well? Sure. So since we have been discussing post-quantum, a friend of mine who just finished his internship, Mihil, he has been working on sort of the problem that I was saying,

how to bring quantum resistance systems closer to practice. He has been working on sort of the problem that I was saying how to bring quantum resistance systems closer to practice he has been working on designing the hardware to make this closer to reality So you've given up your summers to some extent

to your work at Microsoft Research but it's not all work of course there are lots of other things that go on, lots of activities, and one of them is the Microsoft Puzzle Day. And you both did participate. Your team did very well. I wonder if you'd like to say a little bit more about the experience of Puzzle Day or other activities here that are intended to be more fun and social?

Yeah, I think I thoroughly enjoyed Puzzle Day. I was bummed we couldn't find teams for the other puzzle events. There's a lot that I did not expect to find so many puzzle events over the summer. Actually, before coming to Microsoft, I did not know about the puzzle culture, so this is definitely a pleasant surprise. Yeah, and I think if I'm in this Seattle area, I would like to probably contribute to this later on. Yeah, I definitely enjoy all the intern events that Microsoft has been organizing. So they also organize other events beyond the puzzle ones,

such as one where we went to Topgolf, and that was very fun, for example. And it's also a great way to get to know more about the other interns. And there are interns here who are from all over the world. And I think I've actually learned a lot from interns who work not just in the area that I work in, but also from other areas, to see what's the state of the art,

what are the important problems, not just in academia, but in industry. FRANCESC CAMPOY FLORES- In some sense, the intern events are like the social aspects of conferences, because they're pretty much just graduate students, right? Yeah. But one thing I learned in my graduate student days is that the real activity doesn't necessarily go on in the talks and the meeting rooms.

The opportunity to socialize and learn from others outside in the hallways and dinners really is where things happen. Because most of your collaborators are most likely from your peer group, and they will also be doing interesting things in the future. So it's definitely good to know who's doing what. of bumping into people in the halls and asking what they're doing and finding out leads to more interesting research sometimes than the things that you come in planning on doing and working on. So with this experience, I know you're getting near the end of your internships, both of you.

Is this something that you'd recommend to others? Why should others consider coming to Microsoft Research? I think the fact that I myself came back here probably says something. I definitely love working here at Building 99. It's probably one of the most innovative buildings in the world. And that's because of the people here. And I mean like both the interns and the full-time researchers, they are truly experts in what they work on.

And it has been a joy to get to work on the projects that I did, not just this summer, but last summer as well. This is something I'd wholeheartedly, totally recommend to anyone who's thinking about doing an internship in industry. I was often told that MSR is a very special place and I did not realize that until I got here and I completely agree. I think it is very different from a lot of academic departments but also very similar in all the ways and I enjoy that. I enjoy the differences and I also enjoy the similarities. So after you go back to your respective PhD programs, I know you're at different stages. Ananday, you're getting close to finishing. Can you talk a little bit about what you hope to do, would like to do after you complete your PhD?

Sure. I am going to be on the academic job market. So I guess this is the self-promotional part of this podcast. My thesis is, like I mentioned, is on cryptography for public policy. So I guess this is the self-promotional part of this podcast. My thesis is, like I mentioned, is on cryptography for public policy. So I apply cryptographic techniques to problems of privacy and accountability. And I'm interested in the independence that academia offers. So that is my plan currently. But that might change. I am very open to working for nonprofits-profits or working in government to deploy some of these things in the future. So I'm very open to all of those ideas, but I think my home

generally would be in academia. So Karan, you've got a couple years left, I think, but what would you like to do when you've completed your PhD? So I'm on the other side of the spectrum, I guess. I definitely want to work in industry after I graduate. And this is because while I've enjoyed working and doing research in academia, after I graduate, I want to work on bringing ideas from research into practice. And a lot of it still requires fundamental research. So some of the problems are only known in industry. And a lot of it is due to the scaling issues which companies like Microsoft face. And I think that's a very unique challenge because not only do you have to do fundamental

research, at the same time you have to potentially work with the product teams to figure out how you can integrate it into real world products. And one other thing that draws me to industry is the fact that I think it's a challenge and a joy to get to work on products and ideas which can impact millions of people around the world. So we talked about your backgrounds a little bit at the beginning and then dove into the geeky stuff very quickly. So I think I'd like to circle back a little bit and get a little bit more of a sense of what you enjoyed in your childhood, growing up, how you got to the point where you both came to the U.S.

to go to university. Ananday, you were talking to me a little bit before we started about your difficulty with geometry as a student. Do you want to say more about how you managed to work your way around geometry in school? Sure. I was very interested in mathematics in high school, and I decided that I could not really handle geometry. I focused on number theory. I got very interested in number theory. It was sad that all the Olympiad questions in number theory were very easy, but I was very interested in number theory when I got to college,

and I was very interested in doing research. And in India, we have wonderful undergrad institutions, the IITs, except they're very good undergraduate institutions. They're not research institutions. They're not research institutions. They're not akin to an R1 university in the US. And also, more importantly in India, public funding in research often goes to government labs and not the state institutions or private universities like it is everywhere else in the world. And so what this leads to is a dearth of funds in research at both the undergrad and the graduate level in India.

So it was very clear to me in high school, and I came to the US for a math camp in, I think, 11th grade. And it became very clear to me that if I wanted to do research at the undergrad level, I would have to get into a college in the US. And that's why I ended up applying. And when I got to the US, I realized that pure mathematics was way too hard for me. There were people that were much smarter that could do, you know, mathematical research. And I figured that analysis was just way too difficult. So I switched to algebra, decided to do more

number theory and decided to apply that. And cryptography was, I think, a natural extension of that. That combined with, I think, my own politics and around that time when I was in college, Phil Roggewey's talk on the moral character of cryptographic work, I think really changed my perspective on what cryptography is and what it can do. And yeah, I think it was my freshman year

when I decided I would like to do more of this. And yeah, ever since then, I've been trying to find more problems to apply these techniques to. yeah, ever since then, I've been trying to find more problems to apply these techniques to. It's great that you've been able to find interesting problems to apply these methods to and that also have social impact. One thing you mentioned,

many of the listeners might not be aware of the existence of math camps. That might seem alien. Can you just say a little bit more about what they are and how they work? Yeah. So in the US and around the world now, there are a lot of summer camps for students that are interested in mathematics that is not arithmetic, right? So introduction to college level mathematics in high school, I think is very important for people to understand whether they

would like to study mathematics because college mathematics is nothing like high school mathematics, right? And a lot of these, like SUMAC, PROMIS, the Ross Math Camp in the US now also have international branches. There's a PROMIS program in India now, but these are highly selective, you know, math camps. And most of the folks I met at these camps ended up doing math majors. And I think about 50% are actually doing math PhDs. So people know very, very early if they're interested in math. And these are some of the places where they find out. I'll say quickly, I did the Ross camp back in,

I'm embarrassed to admit, 1976 when I was in high school. So it's been going on for a long time. Karan, can you talk a little bit about your path to Cornell, what you were doing as a high school student, and maybe even before that? Sure. So I went to boarding school for high school in New Delhi. I guess I've been moving farther and farther away from my hometown. So while I was thinking about what I would major in in college, I initially actually came in as a chemistry major because I enjoyed doing that in high school. And in fact, I told my friends and my parents that I'll major in anything but computer

science before coming to college. All my friends in high school enjoyed it, but for some reason I completely hated it. I don't know why. I had never actually taken a computer science course coming to college. All my friends in high school enjoyed it, but for some reason, I completely hated it. I don't know why. I had never actually taken a computer science course before I came to college. But when I took my intro chemistry class, I happened to get like a C. So I realized it's probably not for me. So I wanted to explore what else I could major in instead. One good thing about undergraduates in the U.S. as compared to India is that in the U.S., you don't have to know what you're going to major in right away. You can have like a preferred major, but you're free to change it. In India, you have to apply to a specific major, and that's what most people end up doing because of the requirements.

So in my freshman year at Cornell, I decided to take a few different classes very broadly. I took courses in environmental science, material science, computer science, physics, etc. And I think what stuck with me was the intro computer science course. I had never taken a course before as I said, but I think it was a completely new paradigm for me, which I wanted to learn more about. And then I kept taking more and more computer science classes. I realized, like, this is what clicks for me. And what made me really interested in doing research in it was my computer systems course,

where we focus a lot on, like, the security aspects of computer science. So, for example, this is a very simple attack, but we did a buffer overflow attack. And I realized this is a real world problem because even a simple attack can be deadly. So that's when I knew I wanted to work on security in general. And after I started my PhD, I got more into the cryptographic aspects of security and privacy, such as homomorphic encryption, proving in their knowledge, mixed nets, and so on. And I guess that's what I'm

doing this summer too. That's great. Yeah. It's important, I think, for people listening to understand that we don't, as researchers, have one path that we're just following directly our whole time. I myself was a math major. I mentioned the Ross Math Camp I did before I was an undergraduate. I went to graduate school in mathematics thinking, I'm doing math, I'm doing math. And I found, as you mentioned, Adene, that I was not going to be that good. So after a year in graduate school, I looked back at my undergraduate time and said, yeah, I did okay in my math classes.

I got A's and B's. But all those computer science classes that I took just for fun were all A's because they were so much fun. Maybe I should have learned something from that. And that's when I switched into computer science and found my way into cryptography. So it's good to have the opportunity to move back and forth and find the thing that's really right for you. Yeah. One thing I wanted to add is that if you're truly passionate about learning something,

it's fine if you don't get it right away. This was the case for me, even in the intro computer science course that I took. For example, I struggled to understand the concept of recursion. And it took me quite a few months before I actually got what it really is. And my younger sister, actually, she's a couple years younger than me,

she helped me understand what it is. But I realized, even if this seems hard at first, if I'm really passionate about something and I ask others for help, that can be really helpful. And I'm glad to be where I'm at right now. On another tangent, I will mention that in terms of interdisciplinary research, this is, I think, often the case that departments don't know how to bucket you. So I think a lot of tech policy research, even today, happens outside CS departments. So I tried applying to non-computer science departments, policy departments with the sort of work I was doing, and they obviously did not have the expertise to advise me. But then computer science departments that do have the expertise also relinquish a lot of policy work to other departments.

And I don't think that's right, and I think that needs to change in academia. Is there anything that either of you would like to add about your experiences here or recommendations to other people in how to best learn and contribute and go forward? Sure. So having spent two summers at Microsoft, one thing I'd recommend to other interns is reaching out to other people here, not just other interns, but also other full-time researchers. As I mentioned before, spending time with other interns can be very, very valuable. And you can also get to learn more about what other people here are working on, both interns and full-timers. And this is something that we may think that it could be easy to do, but then as we spend more and more time researching over the course of the summer, we may not have enough time to spend time with other interns, for example. But I just want to say this is one of the most rewarding aspects of interning here. Yeah, I'll just add to that and say that, yeah, I think meeting other interns is actually a very rewarding aspect because we don't get that at our home institutions as much.

But we only get to talk with our research groups and, you know, other people in our programs. And that sort of cross pollination does not happen very naturally outside of conferences. So I think that's a very natural pathway to find collaborators, to find interesting problems to work on, even after the internship, and I think I would highly recommend that. Yeah. So Karan, I know you're deeply involved in preparing your end of internship talk and wrapping things up, and Ananday, I know that the study is pressing down hard on our time and trying to get things done. So I want to thank you both for taking the time and sharing with listeners about your experiences here this summer. Thank you.

Thank you. Thank you so much.