Nvidia and Computational Inverse Lithography Technology

Photolithography, the fundamental technology used in almost all integrated circuit manufacturing, is pretty simple in concept: you make a mask for the pattern you wish to lay down on the chip, apply a photo-sensitive resist uniformly to the chip, expose it to light through the mask, wash away the portion the mask indicates should be processed in the next step, and then etch, deposit material, or implant material on the non-masked regions. The details of this are exquisitely complicated and require great precision, but the idea of manufacturing complicated devices by what amounts to printing pictures of them is the simple concept that underlies the greatest revolution in mass production in human history.

Since their inception in the late 1950s, integrated circuits have simultaneously shrunk the size of the devices that compose them while increasing their density and complexity. I remember, in the late 1960s, reading forecasts that this process would have to come to an end when the device geometries shrank to smaller than the wavelength of light used to print the mask onto the chip substrate. Well, that didn’t take into account the cleverness of engineers, physicists, optical designers, and creators of computer-aided design tools engaged in a global competition to produce the fastest, least expensive, and most complex and energy efficient circuits in a market which was growing exponentially in size.

If you think in terms of geometrical optics, it’s completely impossible to create a sharp image with features smaller than the wavelength of light you’re using to produce it. This is why, for example, there is an absolute limit on the smallest object you can observe through a visible light microscope—to see smaller things, you need to use something like an electron microscope, which images with a far smaller wavelength. But geometrical optics considers light as composed to infinitely small rays that may be absorbed, reflected, and bent, but do not exhibit the properties due to their wave nature such as diffraction and interference. As light interacts with small features on the order of its wavelength, these phenomena increasingly dominate, and shining light through a mask with features comparable to its wavelength will produce an image that looks nothing like the pattern on the mask.

But, since light propagation is time symmetric, if you ran the light the other way, wouldn’t you get back the pattern on the mask? Well, in principle, but of course there are nasty details to be dealt with. If you could solve them, though, couldn’t you then compute the pattern on a mask which, after all the messy diffraction and interference occurred, actually printed the pattern you wanted on the chip? The mask would look nothing like the pattern on the chip, but it would make that pattern when light shone through it.

This was one of those things that were possible in principle, but required such a hideously large amount of computation that it was long considered infeasible. Over the years, chip designers learned tricks to eke more resolution out of light waves, but eventually this hocus-pocus was approaching the limit. Here is the evolution of masks to make simple horizontal and vertical traces over the years.

image

That screwball pattern at the right is the result of “Inverse Lithographic Technology” (ILT) or, as you might say, “Beating light into submission with a computer”. When you shine light through that crazy ILT mask, what you get on the chip are the lines at the very left.

The computation required to do this is daunting, especially when you consider that leading-edge chips can have billions of transistors, passive components, and interconnects, all of which must be perfect or the chip is junk. In March 2023, Nvidia, known for its graphics processing units (GPUs) and artificial intelligence accelerators, announced a new set of algorithms, collectively called cuLitho, which they claim speed up the computations of inverse lithography by a factor of forty, making it practical to deploy widely in chip design and mask making. Here is an IEEE Spectrum article about the announcement.

Nvidia claims that using cuLitho, a cluster of 500 Nvidia DGX H100 GPUs can compute masks using inverse lithography faster than an array of 40,000 of the fastest general-purpose processors available today, producing three to five times as many masks a day.

Taiwan Semiconductor Manufacturing Company (TSMC) is said to be qualifying the cuLitho process for production use this year.

4 Likes

Nvidia is working on a new lineup of artificial-intelligence chips customized for China as the semiconductor giant tries to maintain access to a huge market while adjusting to shifting U.S. regulations.

“It’s going to cost a lot of money and burn a lot of power relative to using H100s, but if I’m China, maybe I’ve got no choice,” Rasgon said.

4 Likes

NVIDIA basically compressed 30 years of its corporate memory into 13B parameters. Our greatest creations add up to 24B tokens, including chip designs, internal codebases, and engineering logs like bug reports. Let that sink in.

The model “ChipNeMo” is deployed internally, like a shared genie:

  • EDA scripts generation. EDA stands for “Electronic Design Automation”, a core software suite for designing the next-gen GPUs. These scripts are the keys to a $1T market cap
  • Engineering assistant chatbot for GPU ASIC and Architecture engineers that understands internal hardware design specs and is capable of explaining complex design topics;
  • Bug summarization and analysis as part of an internal bug and issue tracking system;
  • Domain-finetuned retriever that achieves much better accuracy over internal knowledge.

And we publish a whitepaper to share ChipNeMo’s creation process: https://arxiv.org/abs/2311.00176

2 Likes

Skip to the emergence of sovereign GPU clusters and it starts to read a bit like Snow Crash.

3 Likes

This video has some interesting background on the enormous complexity of the latest photolithography machines. https://youtu.be/rdlZ8KYVtPU?si=Xt0L73QGlz9WO4xG

The video before this in the channel goes into more detail about the dominance of diffraction in mask imaging.

4 Likes
2 Likes

As Palmer Luckey says. Can you imagine the US tech companies selling their technology in Imperial Japan because the market was so juicy that it was more important than US national security? Yet this is the position of Apple, Microsoft, Google and NVidia

5 Likes
3 Likes
1 Like

Um, I’ve reproduced what Patel shared.

3 Likes
3 Likes
3 Likes

Hmmm. Half a trillion dollars by a single company! Total federal expenditures for 2025 is ~7 trillion. Spending one fourteenth of that is hard to imagine for a single company. I haven’t come across the time interval anticipated.

4 Likes

Does NVIDEA have its own foundry? Or is it just racking its chips in a server farm like all the other AI folks?

2 Likes

So the answer is no. Nvidia doesn’t produce chips. Its chips are produced by tsmc and I am guessing what they call a super computer is racks of their cards that are also something that some other company like cloudweave who then also hires out the work to create a facility.

One of the things it appears NVidea is doing is providing financing to its customers that then purchase NVIDIA’s product. Nothing to see here.

2 Likes

This simplistic analysis fails to capture the reality of manufacturing supercomputers or anything else, for that matter.

TSMC may be the ones fabricating the chips via photolithography but there would be no photolithography without the products non-Taiwanese firms, most notably Dutch firm: ASML. Furthermore, a bag full of chips from TSMC are more useless than Tic Tacs without the know-how and work of a firm such as NVIDIA. Presumably, there’s a reason NVIDIA is a leader in its field. If anyone could do it, anyone would do it.

Thus, there are many disparate elements that need to be brought together to make the final product. While it may be a desirable and worthy goal to bring all these together in the US, it will take more than returning semiconductor foundries. It also means that Taiwan is merely one of the many bottlenecks in the production of computing devices. Reciprocally, it also means that any effort to close that bottleneck spoils the fun for everyone, not the the US.

This reminds me of a much simpler example of the humble pencil, as outlined in the classic I, Pencil. If this reasoning applies to something as seemingly trivial as a pencil, it must go double for a supercomputer — maybe even triple!

3 Likes

It wasn’t an analysis of manufacturing. It was a statement on the statement that NVIDIA is committing to manufacture or build something. It just happens to be a super computer.

As I noted. They invested in cloudweave. Cloudweave doesn’t appear to have any special IP or competitive advantage other than they can buy NVIDIA chips. I believe cloudweave was started as a cryptocurrency mining outfit, that jumped on the AI bandwagon. NVIDIA gives them money to buy NVIDIA chips and the investment gives cloudweave credibility such that other people invest in them. They have one customer that needs overflow compute capacity (Microsoft). Cloudweave loses money, but a hyped IPO results in a huge IPO that not only allows cloudweave to buy more product from NVIDIA, but pumps up the valuation of the original investment.

We can call that NVIDIA building a super computer. We can call it an investment. However, I think it is conducting a murky activity that smells like a scam.

I wouldn’t call Panasonic investing in a startup that buys Panasonic batteries and then sells them at a loss to Ford as Panasonic committing to build electric vehicles in the US.

2 Likes

You’re quite right. It much less than an analysis; it was speculation: “guessing,” in your own language. However, this guessing involved some priors, such as the assumption that to “produce the chips” is somehow the key ingredient, more important than any other. That was the point I was addressing in my response.

Since we’re on the subject of guessing, the Cloudweave narrative you have cloudwoven has no basis. Presumably, you meant CoreWeave, which is not mentioned in the linked piece. What is mentioned is chip production in Arizona and a manufacturing facility in Texas. Last time I checked, both of these were still part of the US, though I grant that they may not remain so. The remaining narrative is pure fiction.

We can probably agree that corporate public relations representatives lie and exaggerate in their press releases. Time will tell if any of this materializes and exactly which form it will take. Just as it’s too soon to celebrate the triumph of domestic manufacturing, likewise it is too soon to dismiss as “nothing to see.”

2 Likes