In The Arena by TechArena - Advanced Packaging Innovations in the Semiconductor Industry with Chee Ping Lee of Lam Research

Episode Date: August 12, 2024

In this podcast, learn about the challenges of silicon advancement, the importance of advanced packaging, and how Lam is driving breakthroughs to support the future of AI and chiplet ecosystems.

Welcome to the Tech Arena, featuring authentic discussions between tech's leading innovators and our host, Alison Klein. Now, let's step into the arena. Welcome to the Tech Arena. My name is Alison Klein, and I am really delighted to welcome Qiping Li to the program from LAM Research. Welcome to the program, Qiping. How are you doing? Hi, Alison. I'm doing fine. Thank you for inviting me here to be part of your program today.

Very excited to be here. Now, Qiping, you and I have talked before and we'll get into why we've talked, but this is LAM's first time on the tech arena. Do you just want to give a brief background on the company and your role there? Sure, I will be happy to do that. LAM is founded in 1980, so it's about 44 years since then. Ever since, we have been playing a very key role to enable innovation in the semiconductor industry it is a american company based in fremont california what we do is literally designing and manufacturing semiconductor capital equipment and these equipment are used by chip makers around the world to make chips right so it So it's probably safe to say, in my view, that almost every critical chip that's used in electronic products today

that you may have at your home or with you in your phone right now, they have been processed through a system made by land research. Nice. Now, we recently published a paper on a topic that I think deserves a tremendous amount of attention, given the silicon environment in which we operate today. How is silicon innovation supporting the AI era and continued platform advancement underpinned on advanced packaging? And I think that this is just

so fascinating. You're an expert in this field. I would say that you're one of the few people in the world that understand this completely. So I'm delighted to have you share your wisdom with the audience. Let's start with the challenge facing silicon advancement today. What's your perspective on that? Sure. And listen, I think that's a very good question. Again, thank you for your compliment. I think that came with my 14 years of exposure in the industry, focusing on advanced packaging, and also part of my role to talk to customers, listening to them, introduce our product to them, and then getting some insight from them to my product group to design the right product. But that also means I need to understand what are the challenges that are being faced by our customers in silicon advancement today, right?

So let me go to the basics. Traditionally, one of the best ways to improve or increase the performance of integrated circuit or IC is to increase the number of transistor counts. And to increase this transistor count, you scale down the technology to build the transistor so they can pack more transistors on the same space. For example, like NVIDIA A100, which is a very famous GPU chip used to train GPT-3, that packs about 54 billion transistors in a size of about 800 millimeters square. The new generation, which is H100, that packs 50% more transistors, 80 billion altogether, using a next generation N4 scaling node technology.

However, we don't expect such scaling to continue forever for one very reason. As you scale down the transistor size, it's going to nanometer scale today. And as they go into atomic scale level, their behavior becomes not so consistent, does not follow macro physics in general. And this means the industry needs to find new technology to scale. And this breeds the next problem, which is high R&D costs, complexity in the design, complexity in the fabrication, and also more fabrication costs. One example given by IBS, they expect if you move to 3 nanometer to 2 nanometer wafer, the cost of the fabrication go up by about 50%.

That's crazy. Yes, it is. On the other hand, you still have this AI, which is very demanding in their requirement of what is needed from silicon technology. Because of the high speed access to high amount of memory to train the model for challenges which just increases every generation of AI which is becoming more memory hungry. So question is,

how do you put high amount of memory chip together? And how do you put this high amount of memory chip together closer to the GPU chip also to achieve what we call PPAC advantage. First P for power, second P for the performance which means you want to consume less power, you want to have higher performance. A stands for area, you want the whole chip to be small, you can't afford to build very big. And lastly, you want that to be cost effective because AI is going to cost a lot of money to train.

And this is where I think advanced packaging like 2.5D, 3D is a very promising solution. In that regard, 2.5D are about putting the chip closer together side by side. For example, the GPU and the memory. And in 3D, it's putting the multiple memory die closer together as well. Now, we've seen a tremendous amount of manufacturers shifting to advanced 2.5 and 3D packaging to solve these tough challenges. When you look at your customer base who are using your solutions, where are we with this tradition holistically? And who in the industry is notable for delivering products

based on 2.5D and 3D packaging today? That's also another good question, Alison. Today, LAMP customers are increasingly adopting a wide variety of packaging schemes that enable the logic and memory integration of what you call a 2.5D and also some form of 3D packaging in development currently. In fact, what we see is anything that's related to advanced packaging, they are growing in term of investment, in term of resource for development of the technology right now. But do you know that 2.5D and 3D packaging has been evolving in the industry, although slowly over the last decade. It's not a new invention or new innovation. As an example, 3D packaging, where you stack the logic and the sensor together,

has been used in camera image sensors that can be found in your smartphone today. That 3D stacking technology is actually giving improvement in your night shots and also in your fast-moving video capture kind of requirement that you have in your daily life, right? So these camera image sensors are made by Sony and also made by Samsung. Depends on which camera phone that you buy. They can be found in mainstream phone today besides the high-performance, expensive smartphones.

But the adoption already expanding over the years. What makes 2.5D and 3D even more important today is generative artificial intelligence or generative AI, made popular by ChatGPT. This AI modeling for ChatGPT requires a very high amount of computing power of the accelerator, as well as the memory to train the models. We have the 3D stacking being used for high bandwidth memory which are specialized type of DRAM stacked vertically together and today is stacked as many as 12 layer high and this 3D stack memory are then packaged together with the GPU to form an accelerator

using what we call 2.5D technology that leverages silicon interposer. And that's just 2.5D. In future or even today, you are seeing some processor from Intel, from AMD, inside your laptop, your PC or server that are already using 3D stacking technology. In addition, some flash memory from several or couple memory makers are already using 3D stacking to stack the Logic CMOS and the memory array on top of another. We do expect more customers of ours to follow this direction in future or in the near future to be exact. That's fantastic. Now, HBM memory, you mentioned, has gotten a ton of attention and

it's a leading solution depending on this technology. Can you explain what it is inherent in the HBM memory design and the standard that makes advanced packaging so critical? Sure. As I mentioned early on, generative AI requires a lot of memory for the training of the models. And HPM is an advanced computer memory designed to enable faster data access with less or lower energy consumption. So to achieve this, you need to use advanced packaging, what we call a 3D stacking technology, where multiple layers of DRAM die or memory die are vertically integrated. One example that I can use as a reference is to think of HBM like a multi-story library. So you want to access the data in the library.

And what's the best way to get the data from the library is when you build a multi-story library connected by elevator, high-speed elevator system, right? Instead of having one small library, another small library distributed around the city, and then you have to run around to get the information from different library, and that is taking more time, more resources, more energy.

But if you put this group of libraries into a multi-story, one center area, you accelerate the delivery of data, delivery of books from the library to yourself. And you achieve that by using what we call true silicon VR, which is similar to the high-speed elevator that I mentioned earlier on. That's fantastic. Now let's go under the hood a bit. You know, LAM delivers some really incredible technology in this space.

How did you build the core capabilities to scale silicon in 2.5 and 3D? One of our missions in the company is to drive what we call semiconductor breakthrough that define next-generation solutions that align with the development of critical technologies like 3D chip stacking for HBM and AI chips. AI chips and HBM that uses 3D technology rely on what we call building blocks. Packaging interconnects, they are creatively used to join the chip together like a Lego system. So some of the examples, one of which I have highlighted earlier on,

is what they call true silicon vias. True silicon vias are like holes that are etched from the front side of a wafer to the back side and then filled with copper material to deliver the signal and power. At the product level, LAMs have a model called CINDYON that edges this hole at high speed and good uniformity with very less what they call thick scalloped effect

on the sidewall of the hole. We have another product which is called SABRE 3D copper electroplating tool that's used to fill in the hole with copper material thereafter. As a fact, LAMP's Indian Edge today, as well as the Sabre 3D system, hold about 100% of the market in high bandwidth memory TSA today.

Another building block that is critical for this 3D and 2.5D stacking are what we call the micro-bombs. These are like tiny solder bombs that are formed on the chip for connection to the next chip. To make this micro-bomb, we need the electroplating system. Again, LAMP has a solution called Saber 3D that is enabling high uniformity or co-planarity performance and good productivity to enable these micro-bumps. One of the more emerging packaging technologies are what we call hybrid bonding technology. It's very sought after and very heavily invested by many customers around the world

to enable 3D stacking technology. Hybrid bonding are essentially made of pads. They are very flat and super tiny connection down to hundreds of nanometers scale. They are made on cheap surfaces, comprises of dielectric and also copper material. And these surfaces are then used to bond to another surfaces. Hence the word hybrid bonding, because you bond dielectric material and also the copper material.

For this very advanced packaging technology, LAMP has this model called Vector that deposits very uniform low-particle dielectric material and also the electroplating tool, SEPA 3D, which I mentioned a couple of times earlier, a very flexible tool that can also be used to fill the copper material used in hybrid bonding technology. And I hope that gives you a good overview of what LAM technology available for this advanced packaging space. That's fantastic. Now, I know that this has been a technology years

and even decades in the making, but we need to keep innovating to keep pace with the demands of AI. So where is LAM innovating to keep pushing the industry forward into the future? Great question, Alison. LAM technology has empowered substantial gain in memory density, bandwidth, empowered efficiency, and enabled new generation of applications and processes requirements.

And we are not stopping there. We continue to push the boundary of physics and material science to deliver new breakthroughs in even atomic-scale kind of processing called atomic-level deposition. As an example over here, in the high-speed elevator of true silicon VR that I mentioned, it is a need to also deposit a thin layer of isolation prior to the recuperal deposition. As the true silicon VR become deeper and smaller,

it is very challenging to deposit this thin layer. We actually innovated a system that can deposit this tiny layer in atomic layer accuracy to ensure the entire VR are covered properly. Another example is

in the hybrid bonding technology that I've shared, where two different materials are bonded together, the dielectric and the copper. We are also driving variants of materials to improve the bonding performance so that it gets stronger bond between the two surfaces. And traditionally, you might know LEM for a company focusing on the round wafer space. In 2022, November, we acquired a company called SEMSISCO. SEMSISCO is a global provider of wet processing semiconductor equipment, but primarily on the panel foam.

Panel are literally rectangular substrate that can be the size of 600mm by 600mm. This area is an emerging focus of advanced packaging today. Advanced substrate, fan-out panel-level packaging could be a game-changer process in response to the growing size of the AI accelerator. This approach of using a panel actually enables chipmaker to improve the yield and also reduce the waste because it packs more wafers into a rectangular form that is five times bigger than the wafer space itself. It is an exciting time. We are collaborating with Chipmaker to also tackle the biggest challenge in creating next

generation semiconductor devices, including HPM itself. Nice. Can you talk about how the chiplet ecosystem will tap advanced packaging and what that means for foundries as they look to markets with multi-vendor chiplet designs? Chiplet is also a very hot topic of late. And chiplet ecosystem by default or by design will

have to leverage advanced packaging to be successful because you need to connect this chipplet together and as close as possible to minimize the latency to minimize the power penalty when you break a chip into a chiplet that designed. This actually means there are a lot of opportunities for the foundries. I view that very favorably because it means the foundry have a new offering that they can bring to the market today. But that also comes with challenges. There is a need to have a standard for chiplet connection. There is a need for more collaboration among the ecosystem

because the chiplet can come There is a need for more collaboration among the ecosystem because the chiplet can come from multiple parties. And it also means there is more competition in this space to drive technology. How do you bond? How do you join this chiplet together with the best performance? This is why when you look at the ecosystem today, you are seeing the rise of industry standards. For example, like UCIE, which you have made reference in your paper that you published recently. And you are also seeing multiple foundries releasing new solutions to the market. For example, 3D Fabrics is a packaging solution coming from TSMC and you have

the cube series that are being offered by Samsung, also so on. But literally, in my view, shippement requires an industry ecosystem to work closely together. As the saying goes, it takes a village to raise a kid. And I think it's the same here. And the foundry plays an important role to drive the ecosystem to work together to make that happen. That's fantastic. I love this interview

and I learn so much from you every time I talk to you. It's wonderful. I just have one more question for you because I'm sure that folks who are listening online are experiencing the same thing. Where can folks find out more about the solutions you talked about and engage with your team

in discussions about the future of Silicon Innovation? One of the good channels about and engage with your team in discussions about the future of silicon innovation. One of the good channels for you to learn more about our LAM solution will be our website. You can reach us at www.lamresearch.com and that site gives you a good overview of what are the solutions that we offer in this advanced packaging space. And I hope you will be able to get quite good insight by getting into that website. Thank you so much, Chi-Ping.

It's been a wonderful time talking to you. And I can't wait to have you on the program again. Good to be part of today's session with you, Alyssa. Have a good day. Thanks for joining The Tech Arena. be part of today's session with you, Alison. Have a good day.