And I also really want to say that to folks, you know that we air these podcasts after we've recorded them. And so today, as we're recording this podcast, the fires in LA are still burning, and they're not contained. And I just want to say to all of our listeners, their families and their friends and so many people who are impacted and affected, and just to the huge number of ways that what's happening is truly tragic and that our hearts go out to you. The size of this fire is, it's catastrophic and beyond comprehension and it reminds us that while, you know, making a podcast about a show relating to artificial intelligence, the power of Mother Nature is really extraordinary. So, it's hard to go from that to into the podcast, but let's go ahead and take that leap.
We're going to spend some time today on a topic that can be confusing to folks. And I'm going to mention what it is, but I don't want you to turn the dial off or push the button off. Bear with me, and just let me tell you why you really want to know about this. But we're going to do an episode today on chips that are central, semiconductor chips, chips that are central to AI, and a little bit about export controls and why that all matters. Export controls, like import-export, so we're talking about export controls. And again, I know that for some folks, they're going to be thinking, “Oh, my God, I never want to listen to an episode about chips, and I certainly don't want to listen to an episode about export controls. It'll be the most boring thing ever.”
But I am going to try to make it a little bit more interesting, and to pack a lot of information that you're going to want to know for 2025 and for what's coming up with some changes in administration for you. And I promise that if you're interested in AI, and I sort of assume that most of you are — or all of you are, since you turned this thing on to begin with — that you're going to want to know this information because chips, processing power and any restraints or restrictions on getting those chips that are central to AI is critical to improvements in AI and to where we are with AI today.
So let's start with the story of why chips and export controls are both interesting and integral to the AI landscape. And I want to start with a couple of basic points. In prior episodes, we've talked a lot about the enablers of AI being things like data and processing power and energy. So one of those is the processing power. And processing power that enables AI comes with technological advances, and it comes with technological advances in chip technology along with, of course, there's lot of energy that's needed to power that. So we're going to talk about that in just a moment. So let's talk about what those chip advancements are. They are both in the design of the chip and the operational capabilities of the chip.
And so now, what is a chip? And so a chip you might think of as like a chip. The word itself is sometimes now today, in the AI context, used in a way that is a synonym for different things. So chips are also things called CPUs, which are central processing units, which carry out the basic instructions of certain programs. The CPUs are not as critically relevant to what we're going to be talking about today as something else called a GPU, which is a graphics processing unit, which is a highly efficient processor that's really optimized for large scale computations. So GPUs you can think of, for today's purposes, as a kind of chip. And so even though they're called, you know, graphics processing units, and so it sounds like it's a big machine, in fact, those big machines are contained on tiny little chips. And I mean tiny, tiny, tiny, and we'll get into some of those measurements in a moment. They can sometimes be put together into work in groups and clusters, and we'll talk about that. But both CPUs and GPUs, and GPUs again are really the chips for AI, they're a kind of semiconductor technology. And so for purposes of today's discussion, we're going to talk about semiconductor chips as these silicon wafers. It's like a silicon-based substrate, or like a wafer. And they've gotten smaller and smaller and smaller over time. And they have actually millions, or even now billions, of these things called transistors that are packed on them.
So we need to talk just for a second, and don't turn me off yet. I'm going to make this worth your while. But transistors are tiny little devices — they've gotten smaller over time — that control and amplify electrical signals and power. And the first kind of transistor dates back to the early 1900s. But then there was the first working transistor, or commercial transistor, was in the 40s, the late 40s, 1947. Bell Labs did a lot of work with transistors after that. But today, transistors of various types are used in all electronics. And that's true in connection with the chips that actually are at the heart of the processing for AI models. So AI models require a series of these chips that have millions or billions of transistors put onto these teeny tiny silicon wafers in order to work. Now, I told you I'd tell you about the size because if I talk about millions or billions of transistors on a tiny silicon wafer, I want to give you a sense of how small it is. We're talking about really almost like nanotechnology at this time. So transistors can be so small today that they can measure around two or three nanometers. And to give you just a little bit of a sense of what that means, a two nanometer transistor is smaller than a strand of human DNA. So just think about that, really, really small. And that's why so many of them can be packed onto these silicon chips, or silicon wafers.
So now we've got this sort of concept of chips. Let's talk about how all of this fits into the next piece, which is export control and why that impacts AI development and is going to be a big issue in 2025. Well, first of all, I think it's clear to everybody who's heard about a lot of companies like NVIDIA and Qualcomm, IBM, Cerebris, Apple, Intel, Microsoft, AMD and others that there's a lot of competition, huge competition, in designing and manufacturing the semiconductor chips that are used in AI, these GPUs. And so these GPUs, which are performing the computations for the powerful AI models and are able to do that in parallel and to process many things simultaneously, these GPUs are actually manufactured in a really sort of complicated multi-step way. And we're going to go through that and talk about the ways in which different kinds of regulation that impact the different steps in that process can cause issues in terms of supply chain access, what countries have access. and things of that nature. And you can start to imagine now the complexity of how ubiquitous AI technological development is all over the world, and how if you start having friction in the supply chain, how that can cause some real issues.
Now, let's back up though one more time because I want to talk about some of the competitive issues in making these chips. Okay, again, don't turn me off, all right? But the AI models are able, with faster and better GPUs, to actually process more and more information. So there's a lot of competition in making the best and the fastest GPU. And the power of an AI model can have a direct relationship to a GPU. So let's just take, for instance, to train a neural network like a large language model, an LLM, and you've got to process a huge data set or maybe modify the weights of the parameters inside the neural network. And we've talked about those concepts in prior episodes. So if you want to learn about those, you can turn back to some of our prior episodes, and we'll talk about how those inner workings are. But these multiple GPUs can work together in something called distributed training. And they can train the neural network, they can modify the neural network and the weights and the parameters inside the neural network, and they can do it with power and with speed. So after an AI model is trained, the GPU can also be used for something called inference. So the GPU, let's go back, the GPU is used for training to do all of the processing that's necessary for the training of these large language models to process and compute and do huge computations for vast quantities of data. That's at the input stage. But then, the GPUs can also be used at the inference stage. Now, inference is the word that we use to describe when the model, the AI model, is actually generating output of some kind. So when you ask an LLM a query and the LLM then sends the query off to be answered, the GPU is involved in processing the information that's necessary to answer that query. So think of the GPU as the processor, the engine that's driving the computational process that's making the AI model really be able to create the output that it can create.
So we've got these chips, and now we know we use them both at the front end when the model's being developed in terms of processing all the data, we use them at the inference stage. And I want to lead us now into the supply chain. When you manufacture literally anything, you have some amount of a supply chain. And what I mean by that is you have different stages in the chain, from the beginning to the end, that results in ultimately a manufactured product. So a simple example of this is you need materials to make a product, and so you have the acquisition of those inputs, those materials. Then you have some sort of facility to make the product. And then you've got that facility located someplace. Then you've got maybe a manufacturing process that might be proprietary, and that can have a whole bunch of restrictions around it in the AI context. The manufacturing process itself can be highly, highly confidential. And ultimately, you end up with a good and whatever has been manufactured. That may be then sent to another company who then distributes it.
So when we talk about the GPUs being built on silicon, let's talk about the supply chain, silicon wafers. Silicon is actually extracted from quartz, and so you need a supply of quartz. But there are other materials, other supply materials, that go into GPUs called rare earth materials. Now, once you've heard the phrase rare earth materials, you're going to be hearing about that a lot because rare earth materials are used in computers and they're used in these chips. And they are in fact quite rare, and you can't get them everywhere. And the kind of rare earth materials that go into GPUs are found in places like the United States or Australia, but China is a huge source of rare earth materials. Now, you know that the United States has got some trade issues in different ways with China. So you can imagine now that if there's a trade set of regulations that has restrictions on the transfer of rare earth materials, it can actually impact the supply chain. And there are also specialized chemicals that go into GPU manufacturing, and those are sourced from places like Japan and South Korea. And the actual manufacturing of the GPUs occurs in really specialized fabrication plants that can put these millions or billions of transistors onto the silicon wafers. And those specialized plants are located in places like the United States, Taiwan, South Korea, China, among others. And then, once the silicon wafers have the transistors on them and they're assembled and tested, that testing can occur in the location where they were fabricated or in other places such as China, Vietnam or Malaysia. So you've got yet another step. And then at that point, the GPUs may be considered fully manufactured, fully tested, and can be sent to companies like NVIDIA, AMD, Intel. We talked about Cerebris and other companies for distribution. Each of those companies will have its own proprietary arrangements with every point along that fabrication chain, so that they are able to have their particular chip made, And the chips will be different as between the different companies.
So advanced GPUs, really highly advanced GPUs power advanced AI. And with all of this as background then, let's talk about the export controls. So export controls, meaning controls on exporting things from the United States. They can actually enter the process relating to GPU manufacturing distribution in various ways. It can be in terms of, as we've mentioned, you know, the rare earth materials, the fabrication process itself or the distribution itself. Now, one of the most controversial export control debates that we have today is going on right as we speak during the final, sort of, couple of weeks of the Biden administration. And when this airs, we might even be just before the hump of into the Trump administration. But we've got, now, a new rule called “Export Control Framework for Artificial Intelligence Diffusion.” And we'll talk about that in a moment. But let me say as a backdrop, the United States has already restricted NVIDIA's A100 and H100 GPUs from being sold into China. So we already have, and there are other export restrictions. This is a very complicated regulatory area, but just to give you an idea, there are already some export restrictions on chips today.
But we've got this rule called “Export Control Framework for Artificial Intelligence Diffusion,” which that's just like a confusing word, phrase. But in short, the rule limits how and to whom and the number of advanced GPUs that— when these GPUs exceed a certain performance threshold — how, to whom and the number of them can be exported. And the rule imposes compute caps on certain countries. And there are, some argue that limits on those countries to undertake the processing that's necessary to compete or innovate at the highest level will create its own problems. Maybe those countries will be driven to other countries such as China or elsewhere to have alternative supply. There are a vast number of voices on all sides of this debate. And there are some who believe that because these AI chips are essential to innovation and to U.S. leadership in innovation, that export controls having any impact on that and sending buyers to other countries can have a negative impact. And one point that gets made is that U.S. companies that make GPUs are going to now have limitations imposed on the markets that they can serve. But there's also a second concern that the rule, these export rules, could drive a wedge between the United States and its partners or allies. And then there's an additional concern that among the countries that are impacted or limited in receiving a certain number of these chips, that they are also going to have incentives to go elsewhere.
So there are voices all over this debate that are, you know, not necessarily going to find a point of alliance, even as between them, even the just the U.S. voices. You know, Senator Ted Cruz has actually said, look, export controls may have a role for very sensitive technology. But this rule, according to Ted Cruz, has gone too far. So it's really a debate to be watched. And one interesting part of the rule that gets mentioned a lot is called the AI 20, which is a portion of the rule which designates 20 countries that are given sort of VIP access to otherwise restricted GPUs. There are some countries that are excluded from that, at least at this point in time, Saudi Arabia, Singapore, Mexico, the UAE, among others. So it's very complicated. And then there's a piece of the rule called the Data Center Validated End Users. Yet again, an incredibly complicated concept and set of terminology. But that Data Center Validated End Users, who are hyperscaler cloud providers like Amazon or Microsoft in the United States. They're so big they're called hyperscalers, and they're cloud providers. And they are allowed to ship GPUs to themselves, to certain facilities to themselves, outside the United States, but they have very significant reporting obligations that are also subject to lots of debates.
So in general, there's a significant issue with this rulemaking coming at the tail end of the Biden administration. And a bunch of folks are saying they may not have gotten as much stakeholder say because of the timing. Others, obviously are in favor of the rule. So we've really got a lot of debate, and we're going to see now what the Trump administration does with this rule. And we're going to fill you in in further episodes about it. But you can see now that when you've got these incredibly critical pieces of AI models, the GPUs, the chips that make the models as powerful as they are, that export controls can impact that. And if you've got national security concerns, there may be well and truly good reasons to have certain kinds of export controls. But it's obviously an area of hot debate, particularly when you're talking about the kind of global arrangements that are actually, today, used in order to manufacture and distribute these chips.
So I hope I didn't bore you too much. That’s all we've got time for today. I will return to this later at some point, and you'll have this as a sort of a base in the chip world. I'm Katherine Forrest. Thanks for tuning in to the episode that we just had on “Waking Up With AI,” a Paul, Weiss podcast. Thanks.
