Microsoft Research Podcast - Intern Insights: Vaishnavi Ranganathan with Angela Busheska

Episode Date: October 24, 2024

Every year, interns from academic institutions around the world apply and grow their knowledge as members of the research community at Microsoft. In this Microsoft Research Podcast series, these students join their internship supervisors to share their experience working alongside some of the leading researchers in their respective fields. In this episode, Angela Busheska, an undergraduate engineering student at Lafayette College, talks to Senior Researcher Vaishnavi Ranganathan, about her work on TerraTrace, a platform that brings together statistics and large language models to track land use over time for agricultural and forestry applications. Busheska discusses the personal loss that drew her to climate activism, the chain of events that led to a memorable face-to-face meeting with Microsoft’s chief sustainability officer, and her advice for going after the internship you want and making the experience count.Learn more:TerraTrace | GitHub repoProject FarmVibes | Project homepageProject FoodVibes | Project homepage

Being in New York while working with people from Seattle and Brazil allowed me to have a broad range of people that I had a chance to meet. So like the New York office, it is very specifically focused on economics and social aspects that I'm not an expert on, but having a lunch with these people every single day had a chance to learn a lot. Being in Seattle and meeting with interns and researchers in Seattle had a chance to learn how other projects came to life. With our meetings, I realized how farm vibes and food vibes were once just like a small idea and now are these huge projects.

So having a chance to understand the history of all the things around me was a great way to see how I am able to build something for the future. Hey everyone, welcome to Intern Insights, a Microsoft Research podcast featuring some of the brilliant students who are contributing to research and advances at Microsoft as a part of the renowned internship program at Microsoft Research. I'm Dr. Vaishnavi Ranganathan, a senior researcher at Microsoft Research with a passion for leveraging sensing and wireless technology to help address global challenges in health, environment, and sustainability. Today, I'm speaking with my intern, Angela Bushezka, about her work this summer and her experience as a Microsoft Research Intern. Angela, welcome to the podcast. So, I've had the pleasure of getting to know you this summer, not only in the capacity as a student

and researcher, but also as a person. So, to start with, why don't you introduce yourself, tell the audience a little bit about yourself, where you're from, your academic background, what excites you in research, and most importantly, outside studies, what do you enjoy doing? Yeah, absolutely. Thank you so much for having me. Super excited to be here and super excited to have spent three months, time flies, as a researcher along with the project. So my name is Angela. I am originally from North Macedonia, a very small country next to Greece in the Balkans.

And I spent my first like 16 years thinking that I would be a mathematician, really focused on math Olympiads. And then I moved to the capital city of my country, which is Skopje, North Macedonia. And that year, in 2019, we had the greatest pollution. We were like on the top of every single list in terms of pollution. But honestly, I didn't care. I was like, I'm going to do like medal Olympias, win every medal. I moved in with my aunt who was living very close to the city center and she had a lot of cardiovascular difficulties. And then this air pollution clogged her blood and she passed away.

And that was the moment when I saw like, I love math, but this air pollution took my aunt. So I have to do something. So I made the hard decision that junior going into senior year to cut my participation in meta-Olympiads after seven great years there and focus on the climate.

I really just started to understand where is the pollution coming from in my area. I started to realize that fast fashion had a huge influence and then that's why I started like getting into this field, getting into research. I started a climate tech non-profit called Unroot where our mission is to debunk the fast fashion supply chain and let people but also corporations know what is behind their clothes. So we kind of help both sides. We help people with their shopping choices, but we help a lot of corporations and policy to understand either what's behind their supply chain and change it or how to better communicate their supply chain, because there's a lot of green

washing that is happening in this space. So it really started as a ragtag group of teenagers, and this is my fifth year working at it. So I really had a passion for both understanding the research side of it and the nonprofit, like getting people to learn more about it. So I think that marked really my presence. And then coming into the U.S. in 2021, I started to study both electrical engineering and computer science just because I love this intersection of building and doing stuff in practice

and a way to apply my math knowledge into real life. Wow. So early in life, you have such amazing goals. That's impressive. So, Angela, we connected after Madeline Depp, a researcher who was involved in the intern selection process, introduced your work and your passion for sustainability to me. I saw your passion for this area and, you know, your enthusiasm in the first conversation.

We were already working on ideas, right? I knew I had to work with you. But for you, there are many other internship opportunities. I would love to know what excited you the most about Microsoft research. Absolutely. Yeah. So as I said, like I started getting into this area of sustainability by myself, really didn't really have a structured way to learn. So when I heard that there is an opportunity from people who have previously worked in this area, or doctors in this area, to learn along them, that was definitely one but also um throughout my life i had like

entrepreneurship stunts i was selling lemonade when i was six years old started a non-profit and this was a great opportunity for me to start a zero to one project while being an intern which is i would say once in a lifetime chance like we had a chance to define the problem see solution try things i was not part of just one project or building one feature. I had a chance to drive it along you and the team. So that was, I think, the main reason why. I'm glad we were able to attract your attention here. Before we get into specifics on exactly what we did this summer,

I'd like for us to talk a bit about your internship setup, right? I think it allowed for some really unique experiences. And, you know, I would love to have you share your internship highlights. I think there is one in particular that I'm thinking about, which I really hope you'll share. Absolutely. I would say this was an internship like no other experience I have had in the past. So just for context, I was interning in New York, while part of the team being in Brazil, part of the team, you and the other researchers, being in Redmond, a couple of other researchers being technically in New York.

So I had a chance to interact with all of these different time zones. Also, for an additional context, I was a part of an undergraduate research internship group where they brought us to Seattle from New York and also their other interns from Boston to spend like a week and learn more about Microsoft, Microsoft Research, the leadership. So that was an incredible opportunity.

Also, I had a great opportunity prior to that to come to the team meeting, to see farms in real life, interact with farmers, interact with the team in person, which was also another great opportunity. And I'll come to the one that you're referring here. So the second time when I was like visiting Seattle, I remember a week prior, it was Friday, we were finishing up some stuff. You actually pointed out that I should definitely message some of the leadership people to interact with them and

hear their insights. And it was ESC times on somewhere around eight when I left the office and I shoot an email to the sustainability officer, Melanie Nakagawa. I was like, Hey, my name is Angela. I'm a research intern. I have worked in like fast fashion in the past. Now I'm working with Vaishnavi and the team on this food project. I really want to meet you next week. I'm coming for like three days. I sent that email. I was like, there's no way I'm getting a response from this. And then on Sunday, I traveled to

Seattle. And I remember during the first day, the assistant of the chief sustainability officer says like, there is this Bloomberg Green Festival that is happening in Seattle on Thursday. And we would love for you to come then meet me. It was like, I don't know, it was like real life or not. I really couldn't. It took a lot of time to process. And after that, one thing additional that was a barrier

was how to get into this festival because this festival had a ticket of like $300, $400. And I think that by that time, everything was sold out. So I was like, I would love to, but I cannot just get into the Bloomberg Green Festival without a badge. So then I realized there is this sustainability community at Microsoft

that is like 8,000 employees or something. And they got badges previously for the festival. And everyone who is working there can just get the basic badge just to get into. So I emailed a couple of people from there. They set up me with a badge. Huge props also to the undergraduate research team who extended my stay for a day because I was supposed to leave for New York the other day. So a couple of miracles happened there. I had a chance to go to the Bloomberg Green Festival, had a chance to meet with Melanie and her assistant Tyler, who was working also in this area of sustainability and her presence on this area, on like on these events. It really goes to say how an undergrad intern can like meet with leadership and learn from the leadership to the place and the opportunities we had during this internship.

Thank you so much, Melanie and Tyler, for taking out the time at the Bloomberg Green Festival. I know it was a super packed day for both of you, but thank you so much for taking the time. Thank you for sharing that. I want to point out something. It's amazing how the leadership makes time and they really value every person's work. And that's an internship which is accounted for now, right? And you know, your work is meaningful to the company. I think Microsoft is so big that people don't realize there's this community of 8,000 people in sustainability often. So it's amazing. It blows my mind every single time. And kudos to you for following up on my late night comment.

Angela, so this internship falls under larger efforts within Microsoft Research to establish a sustainable agriculture and food supply chain, a project that we know as Food Vibes. In your work this summer, you focus specifically on enabling technology to meet the new European Union's regulation around deforestation-free products. Could you tell us in your words about these regulations and how you think the work can aid in their implementation? Absolutely, yeah. So the European Union's regulation on deforestation-free products, also known as EUDR, is a legislation that will prevent food linked to deforestation to enter European Union borders.

But how will the border officials know if something is coming from a deforested area or not? On another side, it's also the farmers from this area who don't know what exactly is a deforested area, especially the ones who moved after maybe an area has been deforested. So there are a lot of questions around there. And I think that getting into and understanding the problem was a huge part to continue after that and building the solution. Yeah. And I believe that having a large team here really helped because we all often had

discussions around any of these areas. And we really valued your inputs and how you participated in these discussions, right? So through your internship, we've seen that there are a variety of tools, you know, and vegetation metrics at our disposal to help determine what exists on a plot of land because that's what we want to identify. There like satellite imagery we get the normalized difference vegetation index and then there's the usda's cropland data layer which exists purely for the us but you know we are getting an incomplete picture here at least as it pertains to this particular use case how do you think terra trace brings these pieces together and what are some of the specific challenges or motivations of the core of your work?

Absolutely. Yeah, I think like a couple of first weeks, we tried all these great machine learning models. And one thing we realized is that when we had like a Northards and when we had pine trees, and then when we had like regular forest, all of them are classified the same, which was a really great light saying like,

yeah, these are great. These are models that are trained by millions of data. They still don't work as we want to. And also I think that in an era where everyone is like training big grade models, we kind of decided to take another route and say like, models are great. What if we go step-by-step and maybe we don't need a full model round, maybe we can go step by step to a certain point to try to understand deeply. And I think I'm really grateful that we took this direction because we had a chance to understand on a granular level what is actually happening. Here we had like, we were lucky at the beginning to work with farms in Washington. We did like actual farmer who was in the team who we have like pictures and we had all the data. So that

was a great ground truth for us to understand, okay, this is how the vegetation looks on a farm. This is how the vegetation looks on a forest across years. So we had these snapshots. And then from then on, we just moved along to say, well, if we know about this what's happening in Washington, how we can scale now to other places in the U.S. We look deeply into California just because it has a lot of like

agricultural diversity. And I think that this step-by-step level brought us to create TerraTrace. TerraTrace is a platform where one can enter the coordinates of a farm and see what has been happening on a given piece of land across time. Now, TerraTrace combines a couple of different things,

combines mathematical results, combines LLMs, and combines just combines a couple of different things, combines mathematical results, combines LLMs, and combines just like the basic information of risk. And I think this is good, just because it's not a heavy computational platform that we have. And if we need to use it on like a couple of years from now on a farm, we technically can. If we need to use it like in a legal office, we can, just because it doesn't depend a lot

on data. And it's like easily portable which is another plus and differentiator compared to what exists outside um do you want to share a little bit about the signature curves that you identified absolutely so when i just like realized there is a metric called ndvi which is basically measuring vegetation it is a time series of vegetation uh what we did with NDVI is we tried to measure it across time on different places. And I think in coffee, it had the greatest impact just because there was a coffee in Vietnam, coffee in Honduras,

completely two different places. And then when we analyzed and saw the shapes, we were like, wow, they are completely two different places. However, they still follow the same trend. And that was a great calling to say, like, this is a greater metric. Maybe if like vision computer models failed, this is something that is like very rudimentary. You wouldn't expect that just a very simple index would calculate that. But we saw it working for a lot of different places.

And I'm particularly grateful for the places that were not together at the same place. Yeah, and I think like looking at this like temporal over time picture is the unique aspect of what you built. And that was what unlocked it for us, right? So you hinted at this in developing the TerraTrace platform, you ended up combining large language models

or LLMs and statistics. What did you like about this approach? What did each bring to the table? How did it contribute to your ultimate goal? Could you share a bit about that? As I said, like at the beginning, I'm a huge MAD person. And I truly believe that MAD has more power than we credit today for. So one thing that I started building when I started building,

I was just, how can we get as much information as possible without LLMs and then give it to the LLMs? That was kind of the approach that I started. So we have this, as you mentioned previously, crop data layer, that was the base truth. So I could always go back and compare to see if it has, like if we are doing the same thing. Then we had the signature curves that we could compare to base signature curves to understand, okay, it's a coffee or it's a farm. And now when we were having a farm, we had a problem because corn looked the same as wheat, multiple crops look the same. So I needed something more to differentiate what exactly is there. So by that, I managed to understand like the growth rate, the fallout rate,

what is the percentage when this is the curve is up, how much percentage is down. So every insight that we could get by just basic statistics. And after that, all this combined knowledge was given to the LLM to kind of just confirm that the math is right and give us, okay, greater insight into,

well, your math is good. And after that, based on your math and everything the LLM knows, well, there is this probability that it's corn. And after that, we came to the crop data layer to say, yep, it's corn. Referring to corn just because this was our example demo. Also, LLMs are changing very rapidly today. So like GPT-4, GPT-4 Turbo, expecting GPT-5.

So constant development needs to be done. And I think that if something is like changed and doesn't work in the platform, one thing that will always work is the statistics. So having something to always refer to was a very interesting process. Yeah, I really appreciate how, you know, we had the focus on,

hey, let's do what we can with math first, because after all, this is a sustainability project, right? Exactly. So LLMs were great at summarizing this, taking all the data and giving you the outputs. I think that was a very interesting approach, too. So there are several existing pieces of work we've seen in this area. We've relied on a lot of literature in this specific application, right? And ended up having authors who were affiliated with Microsoft. Could you share a little bit about those works that we drew on and how we leveraged the Microsoft connections? Absolutely. One thing that was something I was not expecting was reading the

papers and then hopping on call with the authors. So one thing that I would credit is first, we understood there was this like GLLM group. They actually had a presentation in our weekly meeting for the group. So it was like me just staying in the presentation and trying to learn. And the other day they send out like, this is a research paper take a look at it and after like we realized the majority of the authors were in the same building as you actually we had a great chance to like meet with them this is sad clip right sad clip and glm both of them trying to understand like how they have built it what they have built it uh why how they planning to continue how i am able to use it

and actually like because when you're reading a paper and you're reading GitHub repo, it's one thing. When you're speaking with the author and saying, this is why we've built it, here's the limitations, be careful how you're using it, it's completely another thing. So I am really grateful for this situation. But also, not only inside of Microsoft, we also had a chance to work with a lot of other folks. For example, we read a lot of papers from research groups in universities. And because our interns came from these universities, it was great to understand from them also what they have built. And the other interns whom you worked with, which we'll come to.

So for this specific work, we were motivated by the use case what were your some of your findings and also how are you envisioning this work as a foundation for other supply chain or even you know broader sustainability scenarios and even beyond those fields that you can think of yeah so one thing that we as a conclusion came out from here is that signature curves are not just by look when we tested that with a lot of different other signature curves to realize okay there is this is true because in most of the cases they worked obviously we had some failures like citrus signature curves didn't work because of other reasons that happen on a farm but for most of them

we got same same results on what happened with the signature curves with what happens actually on a farm which is a great way to further this exploration here another thing that we that we realized from this project is now we the group previously has worked on trying to understand the supply chain like having the tracking part and having this code to track along the journey but we were missing the part on what happened on the land. And now that we have this additional way to understand what happened on the land, I think is a full system of starting. This is what happened on the land.

This is what happened across the road. And this is where it is now. So that not only UDR, but even like customers in the future will know where their food is coming from, which is that total part of sensibility that we want to get to. Because I think that at this point, we know the challenges, we know the climate change challenges that we are having. And the more information we have, the better decisions we and policymakers would be able

to make. And I think we realize this, and you can chime in, Angela, but I feel that the traceability is a vehicle for data. What you build is a means to take all that data and make it meaningful to people, right? Like what does NDVI mean? I have no idea as a lay person. So you can actually get that information out. There were also a few other use cases that you gleaned out of your data input right like wildfire was one of them would you like to share a little bit about that yes so we were looking specifically about california specifically about 2020 which was which were like a season of very very bad wildfires and we could see by measuring this vegetation how like wildfires affected these regions. You could see how vegetation was dropping very quickly

from a great number of one to just flat zero just because everything was there burned. So seeing firsthand on these curves and after that linking up to a lot of other background wildfire data was a good check-in. You could see that it's not a person deforestation, it's a file fire deforestation. And also it is really helpful

for risk estimation as we go further along. We could see like this past year, climate change effects are there. It's not just like this fancy term in the future, it's there. So I think that having a chance to see in the past

and having these models, it's also giving us a preview for the future. Because ultimately we want this greater food yield, so more and more people can enjoy healthy food. So having a chance to predict the risk would help us to save more food and care better about our planet and our people. I think there's a lot of potential. The more information you have, the more applications and sustainability you can build this is merely like a footstool to launch it off right um you said one of the things you'll miss most from your internship is the density of smart people

in the building and in the company over the course of your internship what did you learn how did your actual experience compared to like expectations you had for it I think it exceeded every single expectation that I had because um I would say when you mentioned the density of smart people it was really incredible to have and I think that this maybe my uh very interesting connection of being in New York while working in uh with people from Seattle and Brazil allowed me to have a broad range of people that I had a chance to meet. So the New York office, it is very specifically focused on economics

and social aspects that I'm not an expert on. But having a lunch with these people every single day, I had a chance to learn a lot about how human data is provisioned. They were building a lot of things for Microsoft new projects and new products that are like the co-pilot and the computers. So I had a chance to hear about those perspectives. Being in Seattle and meeting with interns and researchers in Seattle, I had a chance to learn

not about like the farms perspective, but also how other projects came to life. With our meetings, I realized how farm vibes and food vibes were once just like a small idea and now are these huge projects. So having a chance to understand the history of all the things around me was a great way to see how I am able to build something for the future. And also maybe another thing is the collaboration that happens. As I said, I was surrounded by a lot of socioeconomic people that I am not really an expert in, but they were really great with providing me advice from an area I would never think of. So when I was speaking like from farms to someone who is in economics, they would raise another point on how this can hurt or

help economy or to look at it from another side, which is all great perspectives to have when you are creating a research from scratch. That's why I really like referred to entrepreneurship previously, because I feel like even though I was an intern, I really had a chance to shape the project in a way that I would get all these insights from people and then obviously decide how to go further. Yeah, and I want to add summer with interns is the highest energy level at MSR and the best time of my year at Microsoft. So having successfully completed your internship, what advice do you have for our audience, you know, when it comes to applying for an internship

like this in industry and then getting the most out of that experience i would say one of the things that really maybe helped me through the process is um like working obviously in an area that i was really passionate about but another thing and especially this is coming for specifically for the marginalized communities in tech uh although i would say msr is doing a great job in having this balance but i'm coming from a liberal arts college where we have like two girls that are studying electrical engineering. So having this disbalance can in some ways, like, I don't know, put you,

I'm not good enough and all this imposter syndrome. So maybe one thing is definitely like reach out to people. And something that they really learn heavily from you is like reach out. There is nothing you can lose most of the time. It is just the getting a no is maybe the the hardest answer that you will get so um having a chance to ask is always a good idea obviously don't go

to like every single um executives to ask questions even though it paid out and paid out um and just like ask for help for the people around you ask to learn i realized like researchers really want to speak about what they have worked on previously. So whenever you get a chance to ask them about their past project, they're super eager to tell you about like their grad days and how they have become to what they are now. So that was also great. And in terms of applying, I think when I applied this,

I was like, there's no chance I will be able to get in. But for me, as an introvert turned into an extrovert, fun fact, I was a big introvert in the past, who turned into an extrovert just because of my work. It's like, if you don't apply, there's 100% chance that you're not going to get in. So like when I interviewed after my first interview, I was like, yeah, there's like no chance. Because like I presented the projects, but I was like, there are there's like no chance because like I presented the projects, but I was like, there are so many other students who I know they're working in these areas and then it happened. And then I'm here recording this podcast. I remember it was Thanksgiving

break when I applied for this internship. If I was like, no, I'm like, if I self-rejected myself previously, yeah, this would never have happened. So I would say for all interns out there just don't self-reject yourself let them let them reject you it's my really advice and yeah have more confidence i'm glad you applied and you joined us because this was the highlight of the summer for us this way right um so your work with microsoft research is one of your several projects you've been involved with many other things right right? What else do you have going on? How does it contribute to the impact you're looking to have at this point in your life? Yeah. So one thing that I realized in the past four years is that I started,

so my journey was a very, I would say, interesting and not predictable, just because when I graduated from high school, I got trapped by all the COVID pandemic. I couldn't fly into the predictable just because when I like graduated from high school I got trapped by all the COVID pandemic I couldn't fly into the U.S. just because of all the visa regulations it was like okay a gap year at home which if you asked any of my friends before I was never the person who would take a gap year so staying that year really like without a school or official job was the chance for me to just look at and start my nonprofit on route. From then on, I worked with policy. We worked at the UN.

We even went to Hollywood in March to speak about like fast fashion. So I realized there is no one way to make an impact. So I was really glad to join research, to join policy and speak wherever I can. I think now I'm getting back into my senior year of college. Time flies. And something that I am like looking definitely into is how to continue my research in this area, but most probably would love to be through grad school, having a chance also to push further this project, just because I think that speaking out and sharing research is one of the greatest thing we can do, like

referring to all the other times when i was spoke like i was speaking on uh conferences it will be um magical for someone to respond with positive and negative feedback and maybe even get a collaboration for us to move forward so that is definitely something that um i'm looking forward to so definitely this area is something that i got in by mistake i got in in really just because I had that great time in my senior year to spend more time researching, but it's definitely something that marked my life at this point. And I think that whatever route I pursue, it will be to fight climate change in any way or another. I want to point out we are also planning to open source this

and you hope to work with people to get it in the hands of people who need it and could benefit from it most, right? Absolutely. So that's a great direction. I think every year our undergraduate researchers

set a new bar in my life expectations and I thank you for that. This year is another bump up. So you are a senior angela what's next for you and what's next for your research and work yeah so this will be a fall really spent a lot of applications because of like graduate school uh definitely something that i'm looking forward especially graduate school in this area so we kind of spent a lot of time living in an era of GPT wrappers and I think when there is a product

out there that is really based on research it can make a tremendous impact as I said I am really a person who loves to experiment and do things so something out there is definitely either like having a research that is spun out of this or having perhaps a startup down the line that is based in this area and helping millions of people around the world is definitely something I'm looking forward.

Because I saw the impact when you're helping other people. And I think that is the best thing that one person can do. So I was really excited to do it with all of you this summer and obviously continuing on to open source this project obviously for feedback but also for real use and if we can help with something that will be that will be amazing hopefully getting this out to conferences around the world um and yeah see you see where it can take us yeah i have one last question for you. You had some interesting intern mentors, myself included, and then Bruno Silva, who sits in Brazil who really speaks a lot, even more than needed sometimes. So I would say I had a great balance because I was in a time zone that was between Brazil and Redmond.

So I would spend a lot of my mornings, especially the beginning, huge shout out to Bruna for dealing with all of my technical questions. It was my first time setting up a lot of research clusters um we should not forget also the people at the microsoft core dealing with all of the uh like gpus and stuff because i think that uh managing all of these researchers for all the people out there is it's a hard job so having a chance to communicate and also understand i even though i study electrical engineering i never had a chance to under deeply understand how they're used in real lives um speaking also with people from all different like walks of life because peter was a my other mentor was someone who has been really experienced in this area so he he would reference

projects from like i don't know how many years and it was like so fun to see the history there to see his like previous experiences come to life. You don't every single day get to have three or four mentors from all different areas and aspects. So I'm really grateful that I had a chance to talk and also get called when things were not right so we can make them better and more correct for the people that will use this project. I'll tell them you said thank you. So Angela, I think we are coming to a close here.

Thank you so much for taking the time to share your experience with us. It's been a pleasure to work with you and I really look forward to seeing what the future holds for you and what amazing things you're going to achieve. So thank you, closing out. Thank you so much.

Really appreciate to have this opportunity and for everyone that is like working in this area for every future intern that is doubting themselves, feel free to find me out there and like ask, always happy to help people, always happy to chat sustainability with people. So thank you so much. Thank you.