Microsoft Research Podcast - Ideas: Accelerating Foundation Models Research: AI for all

Episode Date: March 31, 2025

Innovative AI research often depends on access to resources. Microsoft wants to help. Technical Advisor Evelyne Viegas and distinguished faculty from two Minority Serving Institutions discuss the benefits of Microsoft’s Accelerating Foundation Models Research program in their lives and research.

So AFMR is really a program which enabled us to provide access to foundation models, but it's also a global network of researchers. And so for us, I think when we started that program, it was making sure that AI was made available to anyone and not just the few, right? And really important to hear from our academic colleagues what they were discovering and covering and what were those questions that were not even really thinking about, right? So that's how we started with AFMR. One of the things that the AFMR program has has allowed me to see is this kind of ability to better visualize the terrain of creativity.

And it's a little bit of a double-edged sword because when we talk about disrupting creativity and we think about tools, it's typically the case that the tool is making something easier for us. So my big idea is to actually think about tools that are purposely making us slower, that have friction, that have errors, that have failures. To say that maybe the easiest path is not the most advantageous, but the one that you can feel the most fulfillment or agency towards. For me, I think what programs like AFMR have enabled us to do is really start digging outside

the box as to how will these or how can these emerging technologies revolutionize public health. What truly would it take for an LLM to understand context? And really, I think for the first time, we can truly, truly achieve personalized, if you want to use that term, health communication. You're listening to Ideas, a Microsoft Research podcast that dives deep into the world of technology research and the profound questions behind the code. I'm Gretchen Huizinga. In this series we'll explore the technologies that are shaping our future and the big ideas that propel them forward. I'm

excited to share the mic today with three guests to talk about a really cool program called Accelerating Foundation Models Research or AFMR for short. With me is Cesar Torres, an assistant professor of computer science at the University of Texas Arlington and the director of a program called the Hybrid Atelier. More on that soon. I'm also joined by Mohamed Idris, an assistant professor of medicine at the Morehouse School of Medicine. And finally, I welcome Evelin Villegas, a technical advisor at Microsoft Research. Cesar, Mohamed, Evelin, welcome to Ideas.

Pleasure. Thank you. Thank you. So I'd like to start these episodes with what I've been calling the research origin story. And since there are three of you, I'd like you each to give us a brief overview of your work. And if there was one, what big idea or larger than life person inspired you to do what you're doing today?

So Saar, let's start with you, and then we'll have Mohammed and Eveline give their stories as well. Sure. Thanks for having me. So I work at the frontier of creativity, especially thinking about technology could support or augment the ways that we manipulate our world and our ideas.

And I would say that the origin of why I happened into this space can really come back down to a bring your kid to work day. My dad who worked at Maquiladora, which is a factory on the border, took me over. He was an accountant and so he first showed me the accountants and he's like, look at the amazing work that these folks are doing. But the reality is that a lot of what they do is hidden behind spreadsheets and so it wasn't necessarily the most engaging. Suffice to say I did not go into accounting like my dad. But then he showed us the chemical engineer in the factory and he would tell me this chemical engineer holds the

secret formula to the most important processes in the entire company. But again, it was this black box, right? And I got a little bit closer when I looked at this process engineer who was melting metal and pulling it out of a furnace making solder. And I thought, wow, that's super engaging. But at the same time, it's like it was hidden behind machinery and heat and it was just unattainable. And so finally I saw my future career and it was a factory line worker who was

opening boxes and the way that she opened boxes was incredible. Every movement, every like shift of weight was so perfectly coordinated. And I thought, here is the peak of human ability. This was a person who had just like, found a way to leverage her surroundings to leverage her body, the material she was working with. And I thought, this is what I want to study. I want to study how people acquire skills. And I realized that moment, I realized just how important the environment and visibility was to being able to acquire skills. And so from that moment, everything that I've done

to this point has been trying to develop technologies that could get everybody to develop a skill in a same way that I saw that factory line worker that day. Wow. Well, we'll get to the specifics on what you're doing now and how that's relevant in a bit, but thank you for that. So, Mohamed, what's the big idea behind your work and how did you get to where you are today? Yeah, no, first off, I think it's a really cool story. I wish I had an origin story from when I was a kid and I knew exactly what my life's work was going to do. Actually, my story, I figured out my why much later. Actually, my background was in finance. And

I started my career in the hedge fund space at a company called BlackRock, really large financial institution you might have heard of. Then I went off and I did a PhD at Penn State and I fully intended on going back. I was going to basically be working in spreadsheets for the rest of my life. But actually during my postdoc at the time I was living in Montreal, I actually had distant relatives of mine who were coming to Montreal to apply for asylum. And it was actually in helping them navigate the process that it became clear to me the you know, the role, it was very obvious to me, the role that technology can play in helping people help themselves.

And kind of the big idea that I realized is that, you know, oftentimes, you know, the world kind of provides a set of conditions, right, that strip away our rights and our dignity and our ability to really fend for ourselves. But it was so amazing to see, you know, 10, 12-year-old kids who just because they had a phone were able to help their families navigate what shelter to go to, how to apply for school, and more importantly, how do they actually start the rest of their lives. And so actually at the time, I, you a few friends and we started to think about, well, all of this information is really sitting on a bulletin board somewhere.

How can we digitize it? And so we put together a pretty, I would say, badass team, interdisciplinary team, included developers and refugees, and we built a prototype over a weekend. And essentially what happened was we built this really cool platform called ATAR. In many ways, I would say that it was the first real solution that leveraged a lot of the natural language processing capabilities that everyone is using today to actually help people help themselves. It did that in three really important ways.

The first way is that people could essentially ask what they needed help with in natural language. And so we had some algorithms developed that would allow us to identify somebody's intent. Taking that information then, we had a set of models that would then ask you a set of questions to understand your circumstances and determine your eligibility for resources.

And then from that, we'd create a customized checklist for them with everything that they needed to know, where to go, what to bring, and who to talk to in order to accomplish that thing. And it was amazing to see how that very simple prototype that we developed over a weekend really became a lifeline for a lot of people. And so that's really, I think, what motivated my work in terms of trying to combine data science, emerging technologies like AI and machine learning,

with the sort of community-based research that I think is important for us to truly identify applications where in my world right now, it's really studying health disparities. Yeah. Eveline, tell us how you got into doing what you're doing as a technical advisor. What's the big idea behind what you do and how you got here?

So as a technical advisor in Microsoft Research, I really look for ideas out there. So ideas can come from anywhere. And so think of it of scanning the horizon to look for some of those ideas out there, and then figuring out are there scientific hypotheses we should be looking at. And so the idea here is once we have identified some of those ideas, the goal is really to help nurture a healthy pipeline for potential big beds. What I do is really about subtle science and exact art. And we discover as we do, and it involves a lot of discussions and conversations

working with our researchers here, our scientists, but of course with the external research community. And how I got here, well, first I will say that I am so excited to be alive in a moment where AI has made it to industry, I got here? Well, first I will say that I am so excited to be alive in a moment where AI has made it to industry because I've looked and worked in AI for as long as I can remember with very different approaches. And actually as important for me is really natural languages which have enabled this big evolution. People sometimes

also talk about revolution in their idea of the language models. Because when I started so I was very fortunate growing up in an environment where my family, my extended family spoke different languages. But then it was interesting to see the different idioms in those natural languages. Just to give you an example, in English you say, it rains cats and dogs. Well, in France, in French it doesn't mean anything, right? In French actually it rains ropes, right? Which probably doesn't mean anything in English. And so I was really curious about natural languages and communication.

When I went to school, being good at math, I ended up doing math, realizing very quickly that I didn't want to do a career in math, proofs all that. It's good in high school, but doing a full career, it was not my thing in math. But there was that class I really, really enjoyed, which was mathematical logic. And so little by little, I started discovering people working in that field. And at the same time, I was still restless with natural languages. And so I also took some classes in linguistics on the Humanity University in Toulouse in France. And I stumbled on those people who were actually working

in some in linguistics, some in computer science, and then there was this lab doing computational linguistics, and then that was it for me. I was like, that's how I ended up doing my PhD in computational linguistics. And the last aspect I'll talk about, because in my role today, the aspect of working with a network of people, with a global network, is still so important to me, and I think for science as a whole. At the time, there was this nascent field of computational lexical semantics, and for me it was so important to bring people together because I realized that we all had different approaches, different theories, not even in France, but across the world. And actually, I worked with somebody else and we co-edited the first book on computational

lexical semantics where we started exposing what it meant to do lexical semantics and the relationships between words within a larger context, with a larger context of conversations, discourse, and all those different approaches. And that's an aspect which, for me, to this day is so important. And that was also really important to keep as we develop what we're going to talk about today, Accelerated Foundation Models Research Program. Yeah. This is fascinating because I didn't even know all of these stories. I just knew that there were stories here and this is the first time I'm hearing them.

So it's like this discovery process and this sort of pushing on a door and having it be well, that's not quite the door I want. Let's try door number two. Let's try door number three. Well let's get on to the topic of accelerating foundation models research and unpack the big idea behind that. Eveline, I want to stay with you on this for a minute because I'm curious as to how this initiative even came to exist and what it hopes to achieve. So maybe start out with a breakdown of the title. It might be confusing for some

people, Accelerating Foundation Models Research. What is it? Yeah, thank you for the question. So I think I'm going to skip quickly on Accelerate Research. I think people can understand it's just like to bring... Make it faster. Yeah, well, faster and deeper advances. I mean, there are some nuances there, but I think the terms like foundation models, maybe that's where I'll start here. So when we talk about foundation models, is just think about any model which has been trained on broad data and which actually enables you to really do any task. That's, I think, the simplest way to talk about it. And indeed,

any tasks. That's, I think, the simplest way to talk about it. And indeed, actually, people talk a lot about large language models or language models. And so think of language model is just one part, right, for those foundation models. The term was actually coined at Stanford when people started looking at GPTs, the generative between transformers, this new architecture. And so that term was coined like to go not just talk about language models, but foundation models, because actually it's not just language models, but there are also vision models. So there are other types of models and modalities really. And so when we started with accelerating foundation models research, and from now on I will say AFMR, if that's okay. Not to be confused with ASMR,

which is that sort of tingly feeling you get in your head when you hear a good sound, but AFMR, yes. So with AFMR, so actually I need to come a little bit before that and just remind us that actually this is not just new. The point I was making earlier about so important to engage with the external research community in academia. So Microsoft Research has been doing it for as long as I've been at Microsoft and I've been 25 years. I just did 25 in January.

And so I'm's really important for Microsoft Research, for Microsoft. And so we had some programs even before the GPT moment where we had engaged with the external research community on a program called the Microsoft Turin Academic academic program where we provided access to the Turing model, which was a smaller model than the one then developed by OpenAI. But at that time, it was very clear that we needed to be responsible, to look at safety, to look at trustworthiness of those models. So we cannot just drink our own Kool-Aid. We really had to work with people externally.

And so we're already doing that. But that was an effort which we couldn't scale really, because to scale an effort and having multiple people that can have access to the resources, you need more of a programmatic way to be able to do that and rely on some platform, like for instance Azure, which has security and privacy, confidentiality, which enables to scale those type of efforts. And so what happens as we're developing this program on the Turing model with a small set of academic people, then there was this Chatt GPT moment in November 2022, which was the moment like the aha moment, I think, as I mentioned

for me, it's like, wow, AI now has made it to industry. And so for us, it became very clear that we could not with this moment and the amount of resources needed on the compute side, access to actually open AI that knew that GPT, at the beginning, you know, GPT-3 and then 4 and then. So how could we build a program? First should we and was there interest? And Academia responded, yes, please, of course, right? I mean, what are you waiting for? So AFMR is really a program which enabled us to provide

access to foundation models, but it's also a global network of researchers. And so for us, I think when we started that program, it was making sure that AI was made available to anyone and not just the few, right? And really important to hear from our academic colleagues what they were discovering and covering and what were those questions that were not even really thinking about, right? So that's how we started with AFMR. This is funny, again, on the podcast, you can't see people shaking their heads nodding in agreement, but the two academic researchers are going, yep, that's right. Well, Mohamed, let's talk to you for a minute. I understand AFMR started a little more than a year ago with a pilot project that revolved around health applications. So this is a prime

question for you. And since you're in medicine, give us a little bit of how it started, how it's going from your perspective, and why it's important for you at the Morehouse School of Medicine. For sure. You know, it's something that we mentioned that really I remember vividly is when I saw my first GPT-3 demo and I was absolutely blown away. This was a little bit before the chat GPT moment that Aline was mentioning, but just the possibilities, oh my God, were so exciting. And again, if I tie that back to the work that we were doing, where we're trying to

kind of mimic what chat GPT is today, There were so many models that we had to build, very complex architectures, edge cases that we didn't even realize. So you could imagine when I saw that I said, wow, this is amazing. It's going to unlock so many possibilities. But at the same time, this demo was coming out actually saw a tweet about the inherent biases that were baked into these models. And I'll never forget this. I think it was at the time he was a grad student at Stanford, and he were able to show that

if he asked the models to complete a very simple sentence, a sort of joke, two Muslims walk into a bar. What is it going to finish? And it was scary. Two-thirds. It was about 66% of the time the responses referenced some sort of violence. And that really was an aha moment for me personally, of course, not being that I'm Muslim, but beyond that, that there are all of these possibilities at the same time.

There's a lot that we don't know about how these models might operate in the real world. And of course, first thing that this made me do as a researcher was wonder, how do these emerging technologies, how may they unintentionally lead to greater health disparities? Maybe they do, maybe they don't, the reality is that we don't know.

Right. Now I tie that back to something that I've been fleshing up for myself, I give them my time here at Morehouse School of Medicine. And kind of what I believe is that, the likely outcome, and I would say this is the case for really any sort of emerging technology, but let's specifically talk about AI, machine learning,

large-scale models, is that if we're not intentional in interrogating how they perform, then what's likely gonna happen is that despite overall improvements in health, we're gonna see greater health disparities. It's almost kind of that trickle-down economics type model, right? And it's really this addressing of health disparities, which is at the core of the mission of Morehouse School of Medicine. It is literally the reason why I came here a few years ago. Now, the overarching goal of our program

without getting too specific is really around evaluating the capabilities of foundation models. And those, of course, as Eveline mentioned, are large language models. And we're specifically working on facilitating accessible and culturally congruent cancer-related health information. And specifically, we need to understand that communities that are disproportionately impacted have specific challenges around trust.

And all of these are kind of obstacles to taking advantage of things like cancer screenings, which we know significantly reduce the likelihood of mortality. And it's going very well. We have a pretty amazing interdisciplinary team. And I think we've been able to develop a pretty cool research agenda, a few papers and a few grants I'd be happy to share about a little bit later. Yeah, that's awesome. And I will ask you about those because your project is really interesting. But I want Cesar to weigh in here on sort of the goals that are the underpinning of AFMR, which is aligning AI with human values, improving AI human interaction, and accelerating scientific discovery.

Cesar, how do these goals writ large align with the work you're doing at UT Arlington and how has this program helped? Yeah, I love this moment in time that everybody's been talking about that GPT or large language model exposure. Definitely when I experienced it, the first thing that came to my head was, I need to get this technology into the hands of my students. Because it is so nascent. There's so many open research questions, there's so many things that can go wrong. But there's also so much potential, right. And so when I

saw this research program by by Microsoft, I was actually surprised. I saw that, hey, they are actually acknowledging the human element. And so the fact that there was this call for research that was looking at that human dimension was really refreshing. So like what Muhammad was saying, one of the most exciting things about these large language models is you don't have to be a computer scientist in order to use them. And it reminded me to this moment in time within the arts when digital media started

getting produced. And we had this crisis. There was this idea that we would lose all the skills that we have learned from working traditionally with physical materials and having to move into a digital canvas. And it's kind of the birth of a new medium. And we're kind of at this unique position to guide how this medium is produced and to make sure that people develop that virtuosity in being able to use that medium, but also understand its limitations.

One of the fun projects that we've done here has been around working with our glass shop. Specifically, we have this amazing neon bending artists here at UTA, Jeremy Skidmore and Justin Ginsburg. We've been doing some collaborations with them. And we've been essentially monitoring how they bend glass. I run an undergraduate research program here. I've had undergrads try to tackle this problem of how do you transfer that skill of neon bending. And the fact is that because of AFMR, here is just kind of a way to structure that undergraduate research process so that people feel comfortable to ask those dumb questions exactly where

they are. But what I think is even more exciting is that they start to see that questions like skill acquisition is still something that our AI is not able to do. And so it's refreshing to see it's like the research problems have not all been solved. It just means that new ones have opened and ones that we previously thought were unattainable now have this groundwork, this foundation in order to be researched, to be investigated. And so it's really fertile ground. And I really thank the AFMR program for letting us have access to those grounds.

Yeah. I'm really eager to get into both your projects because they're both so cool. But Eveline, I want you to just go on this access line of thought for a second, because Microsoft has given grants in this program, AFMR, to several minority-serving institutions, or MSIs, as they're called, including historically black colleges and universities and Hispanic-serving institutions.

So what do these grants involve? You've alluded to it already, but can you give us some more specifics on how Microsoft is uniquely positioned to give these and what they're doing? Yes. So the ground program per se is really access to resources, actually compute and API access to Frontiers model. So think about Azure OpenAI, but also now actually as the program evolves, it's also

providing access to even our research models, so PHI, I don't know if you've, like smaller models. Yeah, PHI. Yes, PHI. Okay, Vajra. So it's really about access to those resources. It's also access to people. I was talking about this global research network and the importance of it.

And I'll come back to that specifically with minority serving institutions, what we did. But actually, when we started, I think we started a bit in a naive way, thinking we did an open call for proposals, the global one, and we got a great response. but actually at the beginning we really had no participation from MSIs. And we thought, wow, it's open. And I think what we missed there at the beginning is like we really focused on the technology. And some people who are already part of this global network started approaching us, but actually a lot of people didn't even know or didn't think they could apply.

And so we ended up doing a more targeted call where we provided not only access to the compute resources, access to the APIs to be able to develop applications or validate or expand the work which is being done with foundation models. But also we acknowledge that it was important with MSIs to also enable the students of the researchers like Cesar Mohammed and other professors who are part of the program, so that they could actually spend the time working on those projects. Because there are some communities where the teaching load is really high compared to other communities or other colleges.

So we already had a good sense that one size doesn't fit all. And I think what came also with the MSIs and others is like, also one culture doesn't fit all, right? So it's about access, it's about access to people, access to the resources, and really co-designing so that we can really make more advances together. Yeah. César, let's go over to you, because big general terms don't tell a story as well as specific projects with specific people. So your project is called, I'm going to read this, AI enhanced bricolage, augmenting creative

decision making and creative practices. That falls under the big umbrella of creativity and design. So tell our audience and as you do make sure to explain what bricolage is and why you work in a hybrid atelier. Terms I'm sure are near and dear to Aveline's heart. The French language. Talk about that, Cesar. So at UTA, I run a lab called the Hybrid Atelier.

And I chose that name because lab is almost too siloed into thinking about scientific methods in order to solve problems. And I wanted something that really spoke to the ethos of the different communities of practice that generate knowledge. And so the hybrid atelier is a space, it's a maker space, and it's filled with the tools and knowledge that you might find in creative practices like ceramics, glassworking, textiles, polymer fabrication, 3D printing. And so every year I throw something new in there. And this last year, what I threw in there was GPT and large language models.

And it has been exciting to see how it has transformed. But speaking to this specific project, I think the best way I can describe Riquelage is to ask you a question. What would you do if you had a paperclip, duck tape and a chewing gun wrapper? What could you make with that? And so some of us have these MacGyver type mentalities, and that is what Claude Levi-Strauss

kind of terms as the bricolore, a person who is able to improvise solutions with the materials that they have at hand. But often when we think about bricolage, it's about the physical world. But the reality is that we very much live in a hybrid reality where we are behind our screens. And that does not mean that we cannot engage in these bricolore activities. And so this project that I was looking at, it's both a vice and an

opportunity of the human psyche, and it's known as functional fixation. And that is to say, for example, if I were to give you a hammer, you would see everything as a nail. And while this helps kind of constrain creative thought and action to say, okay, if I have this tool, I'm going to use it in this particular way. At the same time, it limits the other potential solutions, the ways that you could use a hammer in unexpected ways, whether it's to weigh something down or like jewelers to texturize a metal piece, or I don't know, even to use it as a pendulum. But my point here is that this is where large language models can come in because they can, from a more unbiased perspective, not having the cognitive bias of functional fixation, say, hey, here is some tool, here's

some material, here's some machine, here are all the ways that I know people have used it. Here are other ways that it could be extended. And so we have been exploring, how can we alter the physical and virtual environment in such a way so that this information just percolates into the creative practitioners mind in that moment when they're trying to have that creative thought.

And we've had some fun with it. I did a workshop at an event known as RCS here at DFW. It's a research weekend where we bring a couple of undergrads and expose them to research. And we found that it's actually the case that it's not AI that does better. And it's also not the case that the practitioner does better. It's when they hybridize that you really kind of lock into the full kind of creative thought that could emerge. And so we've been steadily moving this project forward, expanding

from our data sets, essentially, to look at the corpus of video tutorials that people have published all around the web to find the weird and quirky ways that they have extended and shaped new techniques and materials to advance creative thought. So it's been an exciting project to say the least. Okay. Again, my face hurts because I'm grinning so hard for so long. I have to stop. No, I don't because it's amazing. You made me think of that movie Apollo 13 when they're stuck up in space and this engineer comes in with a box of, we'll call it bricolage, throws it down on the table and says, we need to

make this fit into this, using this, go. And they didn't have AI models to help them figure it out, but they did a pretty good job. Okay, so that's fabulous. I want Mohammed's story now. I have to also calm down. So much fun. No, I love it. I love it. And actually to bring it back to what Vivaldin was mentioning earlier about just getting different perspectives in a room. I think this is a perfect example of it.

Actually, I never thought of myself as being a creative person. But as soon as you said a paper clip and was it the duct tape or gum wrapper? I thought to myself, my first internship, I was able to figure out how to make two paper clips and a rubber band into a, this is of course before AirPods, right? But something that could wrap my eyes around and it was perfect. I almost started thinking to myself, how could I scale this, maybe get a patent on it. But it was a paperclip. Yeah. Awesome. No, no. I mean, this is really exciting stuff. Let me tee you up because I want to actually, I want to say your project out loud because it's called Advancing Culturally Congruent Cancer Communication with Foundation Models.

You might just beat Cesar's long title with yours I don't know you you include a literation which as an English major that makes my heart happy But is positioned under the cognition and societal benefits Bucket whereas Cesar's was under creativity and design But I see some crossover Eveline's probably grinning too because this is the whole thing about research is how do these things come together and help. Tell us, Mohamed, about this culturally, tell us about your project.

Mohamed Nadeau So, you know, I think again, whenever I talk about our work, especially the mission and the why of Morehouse School of Medicine, everything really centers around health disparities, right? And if you think about it, health disparities usually comes from one of many, but let's focus on kind of three potential areas. You might not know you need help, right? If you know you need help, you might not know where to go. And if you end up there, you might not get the help that you need.

And if you think about it, a lot of like the kind of the through line through all of these, it really comes down to health communication at the end of the day. It's not just what people are saying, it's how people are saying it as well. And so our project focuses right now on language and text, right? But we are, as I'll talk about in a second, really exploring the kind of multimodal nature of communication more broadly.

And so, you know, I think another thing that's important in terms of just background context is that for us, these models are more than just tools, right? We really do feel that if we're intentional about it, that they can be important facilitators for public health more broadly. And that's where this idea of our project fitting under the bucket at benefiting society as a whole. Now, you know, the context is that over the past couple of decades, how we've talked about

cancer, how we've shared health information has just changed dramatically. And a lot of this has to do with the rise, of course, of digital technologies more broadly, social media, and now there's AI. People have more access to health information than ever before. And despite all of these advancements, of course, as I keep saying over and over again, not everyone's benefiting equally, especially when it comes to cancer screening. Now breast and cervical cancer, that's what we're focusing on specifically, are two of the leading causes of cancer-related deaths in women worldwide.

And actually, black and Hispanic women in the US are at particular risk and disproportionately impacted by not just lower screening rates, but later diagnoses and, of course, from that higher mortality rates as well. Now again, an important part of the context here is COVID-19. I think thereby some estimates about 10 million cancer screenings that didn't happen. And this is also happening within a context of just a massive amount of misinformation. It's actually something that the WHO termed as an infodemic. And so our project is trying to kind of look for creative emerging technologies based solutions for this. And I think we're doing it in a few unique ways. Now, the first way is that

we're looking at how foundation models like the GPTs, but also open source models and those that are, let's say, specifically fine-tuned on medical text, how do they perform in terms of their ability to generate health information? How accurate are they? How well is it written? And whether it's actually useful for the communities that need it the most. We developed an evaluation framework and we embedded within that some qualitative dimensions that are important to health communications.

And we just wrapped up an analysis where we compared the general purpose models like a chat GPT with medical and more science specific domain models. And as you'd expect, the general purpose models kind of produced information that was easier to understand, but that was of course at the risk of safety and more accurate responses that the medically tuned models were able to produce. Now a second aspect of our work, and I think this is really a unique part of not what I've called but actually literally there's a book called the Morehouse Model, is how is it that we could actually integrate communities into research.

And specifically my work is thinking about how do we integrate communities into the development and evaluation of language models. And that's where we get the term culturally congruent, that these models are not just accurate, but they're also aligned with the values, the beliefs, and even the communication styles of the communities that they're meant to serve. One of the things that we're thinking, you know, quite a bit about, bit about is that these are not just tools to be published on and maybe put in a GitHub repo somewhere. These are actually meant to drive the sort of interventions that we need within community. Of course, implementation is really key. For this, not only do you need to understand the context within which these models will be deployed, the goal here really is to activate you and prepare you with information to be able to

advocate for yourself once you actually see your doctor, right? So that, again, I think is a good example of that. But you also have to keep in mind, Gretchen, that our goal here is we don't want to create greater disparities between those who have and those who don't, right? And so, for example, thinking about accessibility is a big thing. And that's been a part of our project as well. And so for example, we're leveraging some of Azure API services for speech to text, and we're even going as far as trying to leverage some of the text to image models to develop visuals that address health literacy barriers and try to leverage these tools to

truly, truly benefit health. One of the most delightful and sometimes surprising benefits of programs like AFMR is that the technologies developed in conjunction with people in minority communities have a big impact for people in majority communities as well, often called the curb cut effect. Eveline, I wonder if you've seen any of this happen in the short time that AFMR has been going. Yeah. So I'm going to focus a bit more maybe on education and examples there where we've seen

as Cesar was also talking about it, you know, for scaling and all that. But we've seen a few examples of professors working with their students where English is not the first language. Another one I would mention is in the context of domains. So for domains, what I mean here is application domains like not just in CS, but we've been working with professors who are, for instance, astronomers or lawyers or musicians working in universities. So they started looking actually at these LLMs as more of the super advisor helping them.

And so it's another way of looking at it. And actually they started focusing on can we actually build small astronomy models, right? And I'm thinking, okay, that could maybe also we learn something which could be potentially applied to some other domain. So these are some of the things we are seeing. But I will finish with something which may, for me me kind of challenges this cut effect to a certain extent, if I understand the concept correctly, is that I think with this technology and the way AI and foundation models work compared to previous technologies, I feel it's kind of potentially the opposite. It's kind of like the tail catching up with the head. But here I feel

that with the foundation models, I think it's a different way to find information and gain some knowledge. I feel that actually when we look at that, these are really broad tools that now actually can be used to help customize your own curve, as it were. So kind of the other way around. So I think maybe there are two dimensions. It's not just I work on something small and applies to everyone. I feel there is also a dimension of this is broad, this is any tasks, and it enables many more people. I think Cesar Mohamed made that point earlier. You don't have to be a CS expert or rocket scientist to start using those tools and make progress in your field. So I think that maybe there is this dimension of...

I love the way you guys are flipping my questions back on me. But so, and again, that is fascinating, you know, a custom curb, not a curb cut. So, Sar Muhammad, do either of you have any examples of how perhaps this is being used in your work and you're having accidental or serendipitous discoveries that sort of have a bigger impact than what you might have thought? Well one thing comes to mind. It's a project that two Tupi two PhD students in my lab, Adam Emerson and Trisha Shienda have been working on. It's around this idea of communities of practice. And that is to say, when we talk about how people develop skills as a group, it's often

through some sort of tiered structure. I'm making a tree diagram with my hands here. And so we often talk about what it's like for an outsider to enter from outside of the community and just how much effort it takes to get through that gate, to go through the different rungs, through the different rites of passage, to finally be a part of the inner circle, so to speak. And one of the projects that we've been doing, we started to examine these known communities of practice, where they exist. But in doing this analysis, we realized

that there's a couple of folks out there that exist on the periphery. And by really focusing on them, we could start to see where the field is starting to move. These are folks that have said, I'm neither in this community or another, I'm going to pave my own way. While we're still seeing those effects of that research go through,

I think being able to monitor the communities at the fringe is a really telling sign of how we're advancing as a society. I think shining some light into these fringe areas, it's exactly how research develops, how it's really just about expanding at some bleeding edge. And I think sometimes we just have to recontextualize that that bleeding edge is sometimes the group of people that we haven't been necessarily paying attention to. Right. Love it.

Muhammad, do you have a quick example or you don't have to, but I just was curious. Yeah, no, maybe I'll just give one quick example that I think keeps me excited. It actually has to do with the idea of small language models. Right? And so I gave the example of GPT-3 and how it's trained on the entirety of the internet and with that is kind of baked in some unfortunate biases. Right? And so we asked ourselves the flip side of that question.

Well, how is it that we can go about actually baking in some of the good bias, right? The cultural context that's important to train these models on. And the reality is that we started off by saying, let's just have focus groups. Let's talk to people. But of course, that takes time, it takes money, it takes effort. And what we quickly realized actually is there are literally generations of people who have done these focus groups, specifically on breast and cervical cancer screening.

And so what we've actually have since done is leverage that real world data in order to actually start developing synthetic data sets that are small enough, but are of higher quality enough that allow us to address the specific concerns around bias that might not exist. And so for me, that's a really like awesome thing that we came across

that I think in trying to solve a problem for our kind of specific use case, I think this could actually be a method for developing more representative context aware, culturally sensitive models. And I think overall, this contributes to the overall safety and reliability of these large language models and hopefully can create a method for people to be able to do it as well. Yeah. Eveline, I see why it's so cool for you to be sitting at Microsoft Research and working with these guys. It's about now I pose the what could possibly go wrong if you got everything right question on this podcast. And I'm really interested in how researchers are thinking about the potential downsides and consequences of their work. So Evelyne, do you have any insights on things that you've discovered along the path that

might make you take preemptive steps to mitigate? Yeah, I think it's coming back to actually what Mohamed was just talking about. I think these are two around data, the importance of data and the cultural value and the local value. I think an important piece of continuing to be positive for me is to make sure that we fully understand that at the end of the day, data, which is so important to be those foundation models, is, especially language models in particular, are just proxies to human beings. And I feel that we need to remember that it's a proxy to humans and that we all have some

different beliefs, values, goals, preferences. And so how do we take all that into account? And I feel that beyond the data safety provenance, I think there's an aspect of data caring. I don't know how to say it differently, but it's kind of in the same way that we care for people. How do we care for the data as a proxy to humans? And I'm thinking of, you know, when we talk about like,

especially in cases where there is no economic value. Right. And so, but there is local value for those communities. And I think actually there is cultural value across countries. So just wanted to say that there is also an aspect I think we need to do more research on that data as process to humans. And as complex humans we are, right? Right. Well, one of the other questions I like to ask on these ideas episodes is about the idea of blue sky or moonshot research, kind of outrageous ideas. And sometimes they're

not so much outrageous as they are just living outside the box of traditional research, kind of the what if questions that make us excited. So just briefly, is there anything on your horizon, specifically Cesar and Mohammed, that you would say in light of this program, AFMR, that you've had access to things that you think, boy, this now would enable me to ask those bigger questions or that bigger question. I don't know what it is. Can you share anything on that line?

I guess from my end, one of the things that the AFMR program has allowed me to see is this kind of ability to better visualize the terrain of creativity. And it's a little bit of a double-edged sword because when we talk about disrupting creativity and we think about tools, it's typically the case that the tool is making something easier for us. But at the same time, if something's easier then some other thing is harder.

And then we run into this really strange case where if everything is easy, then we are faced with the blank canvas syndrome, right? Like what do you even do if everything is just equally weighted with ease? And so my big idea is to actually think about tools that are purposely making us slower, that have friction, that have errors, that have failures, and really design how those moments can change our attitudes towards how we move around in space. To say that maybe the easiest path is not the most advantageous, but the one that you

can feel the most fulfillment or agency towards. And so I really do think that this is hidden in the latent space of the data that we collect. And so we need to be immersed in that data. We need to traverse it and really it becomes an infrastructure problem. And so the more that we expose people to these foundational models, the more that we're going to be able to see how we can enable these new ways of walking through and exploring our environment. Yeah. I love this so much because I've actually been thinking some of the best experiences in our lives haven't seemed like the best experiences when we went through them, right?

The tough times are what make us grow. And this idea that AI makes everything accessible and easy and frictionless is what you've said. I've used that term, too. I think of the people floating around in that movie, WALL-E, and all they have to do is pick whether I'm wearing red or blue today, and would you drink, I want. I love this, Cesar.

That's something I hadn't even expected you might say, and boom, out of the park. Muhammad, do you have any sort of outrageous, that was flipping it back. I was gonna say, yeah, no, listen, I don't know how I could top that. But I mean, so it's funny. So as you were mentioning that, I was thinking about grad school, how at the time it was the most, you know, friction filled life experience. But in hindsight, I wouldn't have traded it for the world. For me, you know, one of the things I'm often thinking

about in my job is that, you know, what if we lived in a world where everyone had all the information that they needed, access to all the care they need? What would happen then? Would we magically all be the healthiest version of ourselves? I'm a little bit skeptical. I'm not going to lie, right? But that's something that I'm often thinking about. Now, bringing that back down to our project, one of the things I find a little bit amusing is that I tend to ping-pong between this is amazing, the capabilities are just, the possibilities are endless, and then there will be kind of one or two small things where it's pretty obvious that there's still a lot of research that needs to be done, right? So my big what if, actually, I want to bring that back down to kind of a technical thing,

which is what if AI can truly understand culture, not just language, right? And so right now, an AI model can translate a public health message. It's pretty straightforward from English to Spanish, right? But it doesn't inherently understand why some Spanish-speaking countries may be more hesitant about certain medical interventions. But it doesn't inherently understand why some Spanish-speaking countries may be more hesitant about certain medical interventions, doesn't inherently appreciate the historical context that shapes that hesitancy, or what kinds of messaging would build trust rather than skepticism.

So it's this little cultural nuances. That to me is what, when I say culturally congruent, it's a cultural context, what it is that I mean. Nuances that to me is what I want to say culturally congruence and cultural context what it is that I mean And I think for me I think what programs like a fmr have enabled us to do is really start thinking outside the box As to how will these or how can these emerging technologies? Revolutionize public health what truly would it take for an LLM to understand context and and really I think for the first time We can truly, truly achieve personalized, if you want to use that term, health communication. And so

that's what I would say for me is like, what would that world look like? Yeah. The big animating what if. I love this. Go ahead, I believe you had something. Can I expand? I cannot talk. I'm going to do like talk. I cannot talk like that friction and the cultural aspect, but can I expand? And as I was listening to Cesar on the education, I think I heard you talk about the educational rite of passage at some point and Mohamed on those cultural nuances. So first, before talking about what if, I want to say that there is some work, again, when we talk about AFMR, the technology is all the brain power of people thinking, having crazy ideas, very creative in the research being done.

And there is some research where people are looking at what it means actually when you build those language models and how you can take into account different language and different culture or different languages within the same culture or between different cultures speaking the same language. So there is very interesting research. So it made me think expanding on what Mohammed and Siza were talking about. This educational rite of passage, I don't know if you're aware, in Europe in the 17th, 18th century, there was this grand tour of Europe.

That was reserved to just some people who had the funds to do that grand tour of Europe, let's be clear. But it was this educational rite of passage where actually they had to physically go to different countries to actually get familiar and experience, experiment, philosophy and different type of politics. So that was kind of this passage obligé, we say in French. I don't know if there is a translation in English, but kind of this rite of passage basically. And so I am like, wow, what if actually we could have, thanks to the AI looking at different nuances of cultures, of languages, not just language, but in a multimodal point of viewpoint, what if we could have this citizen

of the world right of passage where before we are really citizens of the world, we need to understand other cultures, at least be exposed to them. So that would be my what if. How do we make AI do that? And so without, and for anyone, right? Not just people who can afford it. Well, I don't even want to close, but we have to. And I'd like each of you to reflect a bit. I think I want to frame this in a way you can sort of pick what you'd like to talk about.

But I often have a little bit of vision casting in this section. But there are some specific things I'd like you to talk about. What learnings can you share from your experience with AFMR? Or and what's something that strikes you as important now that may not have seemed that way when you started? And you can also, I'm anticipating you people are going to flip that and say, well, what wasn't important that is now

And also, how do you see yourself moving forward in light of this experience that that you've had so Muhammad Let's let's go first with you then Cesar and then Aveline you can close the show Awesome one of the things that I'm often thinking about and one of the concepts I'm often reminded of One of the things that i'm often thinking about and one of the concepts i'm often reminded of given the significance of the work that institutions like a more house school of medicine. I need to arlington kind of minority serving institutions right it almost feels like there is an onslaught of push back to addressing some of these more systemic issues that we all struggle with, is what does it mean to strive for excellence, right? So in our tradition, there's a concept called Ehsan, Ehsan. You know, there's a lot of definitions of it, but essentially to do more than just the bare minimum,

to truly strive for excellence. And I think it was interesting, having spent time at Microsoft Research in Redmond as part of the AFMR program, meeting other folks who also participate in the program, that I started to appreciate for myself the importance of this idea of the responsible design, development, and deployment of technologies if we truly are going to achieve the potential benefits. And I think this is one of the things that I could kind of throw out there

as something to take away from this podcast. It's really, don't just think of what we're developing as tools, but also think of them as how will they be applied in the real world. And when you're thinking about the context within which something is going to be deployed, that brings up a lot of interesting constraints, opportunities, and just context that I think is important, again, to not just work on an interesting technology for the sake of an

interesting technology, but to truly achieve that benefit for society. Hmm. Cesar? I mean, Ekhoi Muhammad, I think the community is really at the center of how we can move forward. I would say the one element that really struck a chord with me and something that I very much undervalued was the power of infrastructure and spending time laying down the proper scaffolds and stepping stones. Not just for you to do what you're trying to do, but to allow others to also find their own path.

I was setting up Azure for one of my classes, and it took time, it took effort. But the payoff has been incredible in so much the impact that I see now of students from my class sharing with their peers. And I think this culture of entrepreneurship really comes from taking ownership of where you've been and where you can go. But it really just, it all comes down to infrastructure. And so AFMR for me has been that infrastructure to kind of get my foot out the door and also have the ability to bring some folks

along the journey with me. Eveline, how blessed are you to be working with people like this? Again, my face hurts from grinning so hard. Bring us home. What are your thoughts on this? Yeah. So first of all, I mean, it's so wonderful just here live, like listening to the feedback from Mohammed and Cesar of what AFMR brings and that's the potential to bring.

And first let me acknowledge that to put a program like AFMR, it takes a village. So I'm here, the face here, or well not the face, the voice rather. But it's so many people who have at Microsoft on the engineering side, we're just talking about infrastructure. Since I was talking about the pain and gain of leveraging an industry-grade infrastructure like Azure and Azure AI services. Also our policy teams, of course our researchers, but above all the external research community is so grateful to see it.

As you said, I feel super blessed and fortunate to be working on this program and really listening what we need to do next, how can we together do better. There is one thing for me, I want to end on the community, right? Mohamed talked about this, Cesar, too, the human aspect, right? The technology is super important, but also understanding the human aspect. And I will say, actually, my curb-cut moment for me was really working with the MSIs and the cohort, including Mohammed and Siza, when they came to Redmond and really understanding some of the needs which were going beyond the infrastructure, beyond a small network and how we can put it bigger, and deployment

ideas too coming from the community. And that's something which actually we also tried to bring to the whole of AFMR moving forward. And I will finish on one note, which for me is really important moving forward. We heard from Mohammed talking about the really importance of interdisciplinarity, right? And less as not not working silo. And so, and I want to see FMR go more international, internationality if the word exists. It does now.

But it's just now. But it's just making sure that when we have those collaborations, it's really hard actually, time zones, you know, practically it's a nightmare. But I think there is definitely an opportunity here for all of us. Well, Cesar Torres, Mohamed Idris, Evelin Villegas, this has been so fantastic. Thank you so much for coming on the show to share your insights on AFMR today. It was a pleasure. Thank you so much. Pleasure.