What Made Silicon Valley Special?

I was recently asked by an E-mail correspondent whether I had written anything about what caused “Silicon Valley”—the Santa Clara valley of California around San Jose—to become a technology hub and hotbed of innovation. I can’t recall ever writing at length on the topic, so I put together this reply. I thought it might be interesting to the audience here, so I’m posting it, slightly modified with a few warts removed.


I believe (and this is not a unique view, but one I first encountered in an op-ed in the Wall Street Journal in the late 1980s) that a major contributor to the Santa Clara valley’s emergence as a technology hub is its proximity to two world-class research universities (Stanford and UC Berkeley) with strong science and engineering departments. In fact, if you look at technology hubs around the world, I think you’ll find that every single one is located within 60 km of one or more top research universities.

In addition, Stanford had a tradition of allowing its professors to start and work in spin-off companies that employed their students, which provided a path directly from research at the university to commercial applications (which often enriched Stanford from patent licenses on the technologies they used).

When I started working in the Santa Clara valley in 1974, there was still a substantial amount of open land (there were orchards between the building where I worked and Intel headquarters). This meant that, compared to the San Francisco and Oakland areas, land was comparatively cheap, allowing companies to locate there and their workers to find housing nearby. (This is now history, and the area is prohibitively expensive.)

Once you seed an area with technology companies, including many that spun off from previously-established ones such as Fairchild, Hewlett-Packard, and National Semiconductor, you have a large pool of people from whom a start-up can recruit their founders and early employees, rewarding those willing to take the risk with equity in the company.

Before long, this prosperity attracts the support infrastructure such as venture capitalists and technology-focused lawyers that fund the companies and keep them out of trouble.

Once again, it’s the “network effect”: having everything there attracts others to come there and the process builds upon itself.

That was then. Today, Silicon Valley is a sad parody of what it once was and only a fool would found or keep a company in that toxic environment.

16 Likes

And Clarkson had a tradition of demanding that its professors not start and work in spin-off companies. That’s not the only reason Potsdam was not surrounded by start-ups. Not being close to an Interstate and hitting lows of -20F during the winter were also problems.

5 Likes

You make me smile!

That is one reason so many tech companies are moving to Texas. One thing about Texas is that if they lack a world-class research university in close proximity to where they want a tech hub, they build one there. An example would be UT-Dallas (which is in Richardson). The Telecom Corridor companies in the northern suburbs (along US-75) needed a research university. So they funded upgrades to a satellite UT campus in hopes of growing into to Texas’s Cal Tech. This was in the 1980s and 1990s. Don’t know what Woke has done to the campus over the last ten years, but over the first decade of this century they had largely achieved that vision.

7 Likes

Internet has disrupted universities: the best scholarship is now done online, not at universities. I wonder if it’s not better to lean into this versus replicating an old model. The new physical university can be established easily at a point when a sanctuary is needed for the distributed community of scholars.

On this note, Epicurus has inspired monasteries, universities as well as Karl Marx. Here’s a short popular intro to his work:

4 Likes

And then the long slow slide into yesterday’s news begins!

It is interesting that physical proximity to a community of people who are trying to accomplish things in closely related fields seems to have been a factor in the case of Silicon Valley. If we turn back the clock, that proximity was clearly essential to the former blooming of Detroit – not only access to essential raw materials, but also access to hundreds of little machine shops. Or the English Midlands in the later 18th Century. To turn the clock forwards, during China’s great growth phase, it seems that one city would focus on shoes, another on clothes.

But did attracting lawyers ever aid an area to prosper? There is room for different views on that!

7 Likes

Though not much room.

1 Like

I believe that during the period in which Silicon Valley was establishing itself as a hotbed of technological innovation and start-up companies the contributions of some of the lawyers in the area were substantial. They helped define the employee and executive compensation plans that created the incentives start-ups used to attract talent away from secure lifetime jobs at big companies into high-risk, high-reward start-ups, then thread those packages through the regulatory and tax minefields set up to thwart them. Law firms with experience in representing small entrepreneurial companies in acquisition or technology licensing negotiations with companies hundreds or thousands of times their size played a part in many of the success stories in the Valley. And in the semiconductor business, intellectual property lawyers both kept innovative start-ups from being crushed by far larger competitors and facilitated licensing deals mutually beneficial to the innovators and licensees.

Now, you can say, and this flaming libertarian would largely concur, that these lawyers were mostly clearing minefields laid by other lawyers in politics, regulatory, and tax agencies, but given the landscape in which these companies were operating at the time, I believe the contribution of some of the law firms in the Valley in the 1970s through around 2000 were easily net positive.

8 Likes

But the relative ratios were extremely against the techno nerds. Lawyers may have some use, but among those is being targets.

1 Like

Reminds me of that old story about the solitary lawyer struggling to stay afloat in a little Texas town. He was having a hard time finding enough business, even though he was the only lawyer in his area. Then a second lawyer opened an office in that legal desert. Now both are wealthy!

8 Likes

The law firms that see success for one client become a magnet for other clients.

Many law firms tried to get into the hot IPO business of the late 1990s. The response often was: “I don’t care that you can do just as good legal work as BigNameFirm at a 20% discount. If people see BigNameFirm is handling our IPO, that gives us the prestige to clear at a $10 billion valuation. With you representing us, we’d be lucky to clear $1 billion.”

6 Likes
2 Likes

Betteridge’s Law of Headlines applies.

You have to distinguish between the transistor age of Silicon Valley and the network age of Silicon Valley otherwise you’ll miss the essence of technological innovation. I was a pioneer of the network age starting with the PLATO network but I recognized as early as 1978 that the age of network effects was going to capture the positive externalities and lead to stagnation as monopolies emerged. That’s why Steve Freyder and I started working on PLATO’s Cyber 6000 series to write an emulator of the 8086 before the first silicon samples: I wanted to be the first with an OS for reasons that are now obvious to everyone in hindsight. But I had no illusions about being a great innovator. I was just fearful of someone like Gates coming out with a piece of shit monopoly. I’ve told the tale elsewhere of how I got repeatedly seduced away from that by the largest corporations investing in networks at the time only to find that they weren’t serious.

Carver Mead essentially agrees with my view of what made Silicon Valley great, and it is also my view of what makes for scientific revolutions – an educated Jeffersonian Yeomanry capable of essentially saying “Take this job and shove it.” Well, that combined with a pioneer culture of “Take this political economy and shove it… I’m off to the frontier.” Of course, if you think you’re going to get away with terminating the bloodlines of the engineers that do that by making them buy into insanely overprice land, and then replacing the missing sons of those engineers with cultures that are anything but individualistic (even if selecting from among them those of a relatively pioneering/individualistic bent), you’re in danger of being viewed by history as a genocidal maniac and not simply because you depopulated your own.

5 Likes

This is close, but in actuality UT Dallas was established quite a few years earlier in response to a shortage of trained engineers to work at Texas Instruments. Well before telecom companies had any research or development facilities in the DFW area.

From the book of knowledge (source)

To compensate for a shortage, McDermott, Green, and Jonsson established the Graduate Research Center of the Southwest on February 14, 1961. While the institute initially was housed in the Fondren Science Library at Southern Methodist University, a nearby empty cotton field was later acquired by Jonsson, McDermott, and Green in Richardson, Texas in 1962. The first facility, the Laboratory of Earth and Planetary Science (later named the Founders Building), opened in 1964. The Graduate Research Center of the Southwest was renamed the Southwest Center for Advanced Studies (SCAS) in 1967.

See this page for an interactive timeline going back to late 1940s. Here is a photo of the campus complex in 1972

3 Likes

Feynman and Mead collaborated in the early 1980s (source). And Feynman then decided that he actually wanted to be a computer scientist…

My favorite piece is Feynman delivering a talk on hardware and software at the Esalen Institute in the mid-1980s. Lex Fridman edited the Q&A session (source) and it’s Feynman at his best.

2 Likes

Feynman got the importance of objective criteria prizes:


Although he was “disappointed” that the electric motor winning his $1000 prize did not embody any major innovations, just think about how much money Sand Hill Road loses in its many “disappointments” vs how much Feynman actually purchased with that $1000.

Some VCs like to characterize the way they operate is as “hunting packs” that identify a prey they chase down, kill and divvy up the carcass. It isn’t entirely clear to what extent the “prey” is the market gap being filled by an innovator’s business proposal and to what extent it is the innovator himself. The virtue of capitalism is, of course, the fact that these hunting packs are free to form and disperse to form new hunting packs. The failure mode is when they start to over-hunt the kind of men who produce the innovations and deplete the next generation of the sons of such men.

This gets into the evolutionary psychology of individualism in the form of barbarian pastoralists forming a market place of raiding parties that vie to attract men who have the greatest martial reputations, e.g. the astoundingly successful Yamnaya culture.

But that only works so long as the individuals are free to vote with their feet without so great a risk to their bloodlines that they will tend to submit to incompetent warlords who are ready, willing and able to gang up on individuals who aren’t willing to serve them. When the balance of power tips in that direction, you breed out of your population the kind of individual integrity it takes to think independently hence creatively.

In this regard, the portrayal of Hillis and Lenat* (by Feynman) as exemplars of individualist innovators is specious. They both relied heavily relied on government risk management – albeit in the second-to-the-least pathological form: Peacetime Defense Contracts. Total-warfare risk management is the least pathological form of government risk management since the price paid for failure is death. The worst form of government risk management is civilian.

I’m not going to state unequivocally that Cray Computer Corporation, hence a Gallium Arsenide computer industry was killed by DARPA “picking winners” with peacetime risk capital investments, but there is at least a case to be made.

Feynman may be forgiven his lack of insight in this case since he cut his teeth on the Manhattan Project and got a taste of how well DoD risk investments can work in time of total war – and he probably had never thought deeply about the consequences of not shooting corrupt bureaucrats during peacetime. But if anyone has any doubt about the damage that the Manhattan Project did to physics in the post WW II era, or the damage that the Apollo Program did to space industrialization, they need look no further than Carver Mead’s comments on physics and wild hysteria against SpaceX that conflates government risk capital with government purchases of services from privately capitalized technology development.

  • I have to relate an anecdote about Lenat here. Circa 1994, I was driving someplace with Charlie Smith and asked him about Lenat. Although I didn’t know it at the time, Charlie was the guy who spent the Systems Development Foundation’s capital on reviving the field of neural networks as acknowledge by Hinton, Rumelhart, Werbos, et al, and who took care of Mead during a personal crisis and encouraged him to return to and complete his book on collective electrodynamics. Charlie’s response shocked me. He uncharacteristically burst out in loud peals of laughter. I didn’t get it at the time, of course, despite the fact that I had been involved with the second neural network summer prior to this. But I do have to admit that Lenat did offer one thing of underappreciated value – something I took considerable personal risk in promoting within HP’s “Internet Chapter II” by blowing off the reputation I had with HP in order to bring Lenat to HP after being trained by Lenat at his facility in Austin: Cyc as an algorithmic description language and the underlying engine. Basically when I discovered HP’s IC2 project was a fraud geared simply to import H-1bs and put them through school to get MBAs, I tried to get them to adopt Cyc as their “eSpeak” engine as a fallback. It would have been vastly superior to what they ended up with.
6 Likes

spent the Systems Development Foundation’s capital on

Just how significant do you think SDFs activities were? I recently learned about them for the first time and have no sense of their effectiveness.

1 Like

I can’t speak to the general RoI of SDF but I’m pretty certain that without its revival of the “connectionist” approach to AI (ie: “artificial neural networks”), the field would likely have remained in the Minsky/Pappert induced winter it suffered for at least another decade. I recall a CS prof at the UofIA in 1974 being excited about a connectionist experiment he was doing, but the land grant colleges were in no position to challenge MIT regarding the “futility” of the “perceptron” approach to AI. As Schmidhuber has taken pains to point out, virtually all of the advances in neural networks being touted now as “new” were made during that period, with even transformer/attentional mechanisms being little more than pragmatic refinements thereof.

Below is capture from the story "The Believers: The hidden story behind the code that runs our lives:

Rumelhart’s PDP books were financed by Smith while he was director of SDF (mis-named SDC by the above article). Those 2 volumes were instrumental in the awakening the second connectist summer.

I suppose I should emphasize that Charlie was a bit of a unique event in history. The only child of GE’s jet engine chief engineer (and EE mother), from a Christian Science home, he was taken in by Princeton at an early age and was ill equipped to deal with the psychological trauma. My suspicion is this involve sexual exploitation but that’s mere conjecture on my part from my knowledge of his personality and the, shall we say, “liberal” predispositions of some of his mentors such as Oppy, with whom he had a close personal relationship. Probably the most positive influence IMHO was John Tukey. Tukey’s pragmatic approach to statistics involved giving permission to “explore data” without the stifling pedantics that attempt to cut off erroneous interpretations prematurely. We live in a world filled with spurious correlations and confounding variables, but if you’re not given permission to even consider them before you’ve gathered together a PhD-level defense of your “thesis” you can’t even begin to perceive the world as it is. This, I believe, is behind a great deal of the corruption of the social sciences as priesthood. Charlie took that pragmatic attitude with him when he was in charge of founding the Energy Information Administration where he took the job of data quality and analysis on as a kind of substitute for the religious enthusiasm that infects Christian Scientists to the point that they’ll die of treatable illnesses. It was that enthusiasm that led him to understand the importance of neural networks as a pragmatic approach to creating models and, thence, to recurrent neural networks (Werbos) in order to model system dynamics. The neural network field is still unable to appreciate the importance of the transition from statistics to dynamics – which is why it still misses the Algorithmic Information Criterion, despite Minsky’s final repentance that should have set his acolytes on the right path.

8 Likes

Another point in support of this thesis is Einstein. Here’s an excerpt from Carlo Rovelli’s “Seven Brief Lessons on Physics”

FIRST LESSON
The Most Beautiful of Theories

In his youth Albert Einstein spent a year loafing aimlessly. You don’t get anywhere by not “wasting” time—something, unfortunately, that the parents of teenagers tend frequently to forget. He was in Pavia. He had joined his family, having abandoned his studies in Germany, unable to endure the rigors of his high school there. It was the beginning of the twentieth century, and in Italy the beginning of its industrial revolution. His father, an engineer, was installing the first electricity-generating power plants in the Paduan plains. Albert was reading Kant and attending occasional lectures at the University of Pavia: for pleasure, without being registered there or having to think about exams. It is thus that serious scientists are made.

After this he registered at the University of Zurich and immersed himself in the study of physics. A few years later, in 1905, he sent three articles to the most prestigious scientific journal of the period, the Annalen der Physik. Each of these is worthy of a Nobel Prize. The first shows that atoms really exist. The second lays the first foundation for quantum mechanics, which I will discuss in the next lesson. The third presents his first theory of relativity (known today as “special relativity”), the theory that elucidates how time does not pass identically for everyone: two identical twins find that they are different in age if one of them has traveled at speed.

Einstein became a renowned scientist overnight and received offers of employment from various universities. But something disturbed him: despite its immediate acclaim, his theory of relativity does not fit with what we know about gravity, namely, with how things fall. He came to realize this when writing an article summarizing his theory and began to wonder if the law of “universal gravity” as formulated by the father of physics himself, Isaac Newton, was in need of revision in order to make it compatible with the new concept of relativity. He immersed himself in the problem. It would take ten years to resolve. Ten years of frenzied studies, attempts, errors, confusion, mistaken articles, brilliant ideas, misconceived ideas.

Finally, in November 1915, he committed to print an article giving the complete solution: a new theory of gravity, which he called “The General Theory of Relativity,” his masterpiece and the “most beautiful of theories,” according to the great Russian physicist Lev Landau.

There are absolute masterpieces that move us intensely: Mozart’s Requiem, Homer’s Odyssey, the Sistine Chapel, King Lear. To fully appreciate their brilliance may require a long apprenticeship, but the reward is sheer beauty—and not only this, but the opening of our eyes to a new perspective upon the world. Einstein’s jewel, the general theory of relativity, is a masterpiece of this order.

I remember the excitement I felt when I began to understand something about it. It was summer. I was on a beach at Condofuri in Calabria, immersed in the sunshine of the Hellenic Mediterranean, and in the last year of my university studies. Undistracted by schooling, one studies best during vacations. I was studying with the help of a book that had been gnawed at the edges by mice because at night I’d used it to block the holes of these poor creatures in the rather dilapidated, hippie-ish house on an Umbrian hillside where I used to take refuge from the tedium of university classes in Bologna. Every so often I would raise my eyes from the book and look at the glittering sea: it seemed to me that I was actually seeing the curvature of space and time imagined by Einstein. As if by magic: as if a friend were whispering into my ear an extraordinary hidden truth, suddenly raising the veil of reality to disclose a simpler, deeper order. Ever since we discovered that Earth is round and turns like a mad spinning-top, we have understood that reality is not as it appears to us: every time we glimpse a new aspect of it, it is a deeply emotional experience. Another veil has fallen.

But among the numerous leaps forward in our understanding that have succeeded one another over the course of history, Einstein’s is perhaps unequaled. Why?

In the first place because, once you understand how it works, the theory has a breathtaking simplicity. I’ll summarize the idea.

Newton had tried to explain the reason why things fall and the planets turn. He had imagined the existence of a “force” that draws all material bodies toward one another and called it “the force of gravity.” How this force was exerted between things distant from each other, without there being anything between them, was unknown—and the great father of modern science was cautious of offering a hypothesis. Newton had also imagined that bodies move through space and that space is a great empty container, a large box that enclosed the universe, an immense structure through which all objects run true until a force obliges their trajectory to curve. What this “space” was made of, this container of the world he invented, Newton could not say. But a few years before the birth of Einstein two great British physicists, Michael Faraday and James Maxwell, had added a key ingredient to Newton’s cold world: the electromagnetic field. This field is a real entity that, diffused everywhere, carries radio waves, fills space, can vibrate and oscillate like the surface of a lake, and “transports” the electrical force. Since his youth Einstein had been fascinated by this electromagnetic field that turned the rotors in the power stations built by his father, and he soon came to understand that gravity, like electricity, must be conveyed by a field as well: a “gravitational field” analogous to the “electrical field” must exist. He aimed at understanding how this “gravitational field” worked and how it could be described with equations.

And it is at this point that an extraordinary idea occurred to him, a stroke of pure genius: the gravitational field is not diffused through space ; the gravitational field is that space itself . This is the idea of the general theory of relativity. Newton’s “space,” through which things move, and the “gravitational field” are one and the same thing.

It’s a moment of enlightenment. A momentous simplification of the world: space is no longer something distinct from matter—it is one of the “material” components of the world. An entity that undulates, flexes, curves, twists. We are not contained within an invisible, rigid infrastructure: we are immersed in a gigantic, flexible snail shell. The sun bends space around itself, and Earth does not turn around it because of a mysterious force but because it is racing directly in a space that inclines, like a “marble that rolls in a funnel. There are no mysterious forces generated at the center of the funnel; it is the curved nature of the walls that causes the marble to roll. Planets circle around the sun, and things fall, because space curves.

How can we describe this curvature of space? The most outstanding mathematician of the nineteenth century, Carl Friedrich Gauss, the so-called prince of mathematicians, had written mathematical formulas to describe two-dimensional curvilinear surfaces, such as the surfaces of hills. Then he had asked a gifted student of his to generalize the theory to encompass spaces in three or more dimensions. The student in question, Bernhard Riemann, had produced an impressive doctoral thesis of the kind that seems completely useless. The result of Riemann’s thesis was that the properties of a curved space are captured by a particular mathematical object, which we know today as Riemann’s curvature and indicate with the letter R. Einstein wrote an equation that says that R is equivalent to the energy of matter. That is to say: space curves where there is matter. That is it. The equation fits into half a line, and there is nothing more. A vision—that space curves—became an equation.

But within this equation there is a teeming universe. And here the magical richness of the theory opens up into a phantasmagorical succession of predictions that resemble the delirious ravings of a madman but have all turned out to be true.

To begin with, the equation describes how space bends around a star. Due to this curvature, not only do planets orbit around the star but light stops moving in a straight line and deviates. Einstein predicted that the sun causes light to deviate. In 1919 this deviance was measured and the prediction verified. But it isn’t only space that curves; time does too. Einstein predicted that time passes more quickly high up than below, nearer to Earth. This was measured and turned out to be the case. If a person who has lived at sea level meets up with his twin who has lived in the mountains, he will find that his sibling is slightly older than he. And this is just the beginning.

When a large star has burned up all of its combustible substance (hydrogen), it goes out. What remains is no longer supported by the heat of the combustion and collapses under its own weight, to a point where it bends space to such a degree that it plummets into an actual hole. These are the famous “black holes.” When I was studying at university they were considered to be the barely credible predictions of an esoteric theory. Today they are observed in the sky in their hundreds and are studied in great detail by astronomers.

But this is still not all. The whole of space can expand and contract. Furthermore, Einstein’s equation shows that space cannot stand still; it must be expanding. In 1930 the expansion of the universe was actually observed. The same equation predicts that the expansion ought to have been triggered by the explosion of a young, extremely small, and extremely hot universe: by what we now know as the “big bang.” Once again, no one believed this at first, but the proof mounted up until “cosmic background radiation”—the diffuse glare that remains from the heat generated by the original explosion—was actually observed in the sky. The prediction arising from Einstein’s equation turned out to be correct. And further still, the theory contends that space moves like the surface of the sea… And all of this, which emerged gradually from my mice-gnawed book, was not a tale told by an idiot in a fit of lunacy or a hallucination caused by Calabria’s burning Mediterranean sun and its dazzling sea. It was reality.

Or better, a glimpse of reality, a little less veiled than our blurred and banal everyday view of it. A reality that seems to be made of the same stuff that our dreams are made of, but that is nevertheless more real than our clouded, quotidian dreaming.

All of this is the result of an elementary intuition: that space and gravitational field are the same thing. And of a simple equation that I cannot resist giving here, even though you will almost certainly not be able to decipher it. Perhaps anyone reading this will still be able to appreciate its wonderful simplicity:

  	Rab - ½ R gab = Tab

That’s it.

You would need, of course, to study and digest Riemann’s mathematics in order to master the technique to read and use this equation. It takes a little commitment and effort. But less than is necessary to come to appreciate the rarefied beauty of a late Beethoven string quartet. In both the reward is sheer beauty and new eyes with which to see the world.

2 Likes