Metacritic digest #2 - Upgrading OpenAI to neutral
Infinite games. Semicap barely works. Internal rate of return. Meta's working policy.
Last week was easter, so I traveled to see my family and couldn’t write.
Upgrading OpenAI to neutral
Anyone who follows me on Twitter knows that I publish provocative tweets about AI startups. The truth is, I am highly skeptical on companies who develop AI models can be worth significantly more than book value:
First. Companies that implement something using a foundational model are fun, but I have a tough time seeing how they could be worth a lot. Some could create some durable advantages, but even then, it’s hard to create something that competition won’t attack big time.
Second. There are foundational model companies like OpenAI and their competitors (here’s a list of many competitors in the foundational model business). In a very cynical way, they are in the business of creating a PyTorch pickle and selling it through an API (or giving it for free, like Stability does1). The question is: is this good business?
There are two main theories here. The first one, which I tend to have more sympathy for, is that your model is as good as the next guy’s model. These general models are using the open internet data to train2, so the data is the new oil framework doesn’t exactly apply here: OpenAI trained whisper using YouTube data. Because industrial secrets are awfully hard to hide in this industry (both because of idiosyncratic culture issues, but also because employees change companies often). Therefore, the model for an AI company is you buy inputs (PhDs + GPUs + data licensing), and then you get a model in the other side. Because there is a bubble chasing returns here, capitalism will be sure that you can’t make much more than your cost of capital.
The second theory is that it will resemble the semiconductor industry. Of course, your CPU is as good as the next guy’s (they’re designed by the same engineers, perhaps they even use some of the same IP and they are produced by the same fab), but because you got an eighteen month lead over the guys down the street, perhaps you can continue to be ahead of them for 30 years, as long you don’t hit a wall of new ideas. That was the history of Intel, and they could very well still lead AMD today if there was not for their strategic missteps. And now that AMD has a 18-month lead over Intel, they can continue to earn a significant profit on their chips as long the laws of physics allow them to keep improving.
In the semiconductor theory of AI, if you don’t hit a big wall of diminishing returns, a small lead could mean you continue to lead for decades if you can improve your models as fast as the competition catches up.
For current AI companies to resemble the semiconductor industry, their current model of LLMs would be good enough that just through incremental innovation3, they can eventually achieve singularity. It is not clear how long they can keep launching GPT-X with significant improvements nor if the current framework is the one to lead to AGIs or one needs to get back to the drawing board. But even in such hypothesis, foundational models are good business as long as the quality is improving, because commoditization will hit eventually4.
I continue to believe that foundational AI is a business with significantly unclear economics.
But what I was missing with OpenAI is that ChatGPT is increasingly building traction as a distribution platform. As the time goes by and people get used to using it, there is going to be value there, even though copycats will eventually emerge. Ben Thompson recent post shows that OpenAI is acting quick to accept their fate as a consumer facing company and they are creating a platform around ChatGPT. OpenAI owns distribution.
Therefore, it is silly for me to continue to call OpenAI a zero.
To read more about it:
Infinite games: chess edition
The 2023 chess world championship is happening and it’s incredible. It’s 3 for GM Ian Nepomniachtchi, 2 for GM Ding Liren. Incredibly beautiful games from both players. In five games, we already have 3 wins, which is a lot for the championship standards. When Caruana challenged Carlsen, all the fourteen games were a draw.
I am fascinated by the human drama contained in sports so I couldn’t help but be interested in the existential crisis that Ding had during game one of the match. Unlike Nepomniachtchi, who challenged and lost Carlsen in the 2021 edition, this is the 1st time the world’s 3rd best player is playing the tournament, after he finished 2nd in the Candidates’ tournament and current winner Magnus Carlsen decided not to play. This is how Ding Liren responded to how he was feeling after the game 1. (I edited a bit, you can watch it here).
I'm not happy. I feel a little bit depressed because during the game I feel kind of a flow of unconscious solutions during the game. And there were parts of the game before the middle game I didn't think about chess. So much and my mind was very strange, there are many memories, feelings, strange things happens. I feel a little bit, maybe there is something wrong with my mind. Maybe it can conclude rooted by the pressure of the of the match.
Ding Liren has worked for decades to be where he is and when he’s finally there, his mind starts to go away, he starts to feel depressed, and not to focus on the game. I wonder if it’s because of the finite nature of the sport: “was to be here that I spent the first 30 years of my life? A chess game like any other?” On the next day, Ding would be feeling worse that he lost with blacks the game 2. So far, the day off has helped him to address this issue.
One of the best of all times, Bobby Fischer, never played chess5 after defeating Boris Spassky during the 1972 Chess World Championship. He said he reclaimed his liberty, and he hated chess. The documentary of Fischer is a great watch.
Jack Raines wrote recently on Young Money about just that. It’s advisable to look for infinite games because you won’t feel the same drop as Ding is possibly feeling. Tech is like that, finance is like that, and investing is an infinite game. There isn’t a world cup. Only the daily struggle and a precise scorecard.
Semicap barely works, software edition
Someone asked what was the largest amount of bad code someone had seen work, and someone said the 25 million lines of code that makes the Oracle database, because the testing suite took 2 days to run on their testing farm. Then user skrebbel said: hey it reminds me of ASML, but we do not have any test.
Sounds like ASML, except that Oracle has automated tests.
(ASML makes machines that make chips. They got something like 90% of the market. Intel, Samsung, TSMC etc are their customers)
ASML has 1 machine available for testing, maybe 2. These are machines that are about to be shipped, but not totally done being assembled yet, but done enough to run software tests on. This is where changes to their 20 million lines of C code can be tested on. Maybe tonight, you get 15 minutes for your team's work. Then again tomorrow, if you're lucky. Oh but not before the build is done, which takes 8 hours.
Otherwise pretty much the same story as Oracle.
Ah no wait. At ASML, when you want to fix a bug, you first describe the bugfix in a Word document. This goes to various risk assessment managers. They assess whether fixing the bug might generate a regression elsewhere. There's no tests, remember, so they do educated guesses whether the bugfix is too risky or not. If they think not, then you get a go to manually apply the fix in 6+ product families. Without automated tests.
(this is a market leader through sheer technological competence, not through good salespeople like oracle. nobody in the world can make machines that can do what ASML's machines can do. they're also among the hottest tech companies on the dutch stock market. and their software engineering situation is a 1980's horror story times 10. it's quite depressing, really)
This is a good reminder of how hard the semiconductor manufacturing process is. When you ship some wafer fab equipment to Tainan, it will be fine-tuned by TSMC engineers with heuristics and a lack of real understanding on why things are working. We can imagine that ASML’s educated guesses were bad guesses and it created a bug that TSMC can only dream to correct, but they did some workaround it.
In some industries that Oracle serve like finance, defense, and aerospace, it can be bad if they aren’t exceptionally reliable, so Oracle has 2 days of testing and a testing farm. In the industry that ASML serves, you can afford low reliability: Boeing CEO would be in jail if his planes had the yield of a Samsung fab. Even under low yields, it can be economical to run a fab, so there is some acceptance for bugs. What is important for the semiconductor industry is not to be left behind, because no company that was left behind did catch up. Therefore, if to beat Samsung for Nvidia wafers you need your suppliers to ship code without testing it, you’re ready to go. 1 because who cares, Samsung will use the same machines anyway. But the most important thing is that technical debt that will hunt you in the next generation is only worrisome if you live to ship the next node on the next generation.
Internal rate of return
The lack of skill that many CEOs have at capital allocation is no small matter: After ten years on the job, a CEO whose company annually retains earnings equal to 10% of net worth will have been responsible for the deployment of more than 60% of all the capital at work in the business. - Warren Buffett
Some time ago I read a manager letter discussing how people often ignore the importance of management while doing DCFs. A key assumption on DCFs models is the hurdle on reinvestments. Many investors ignore how much gravity the minimal IRR required for projects causes on common stock performance. If a company underwrites 10% IRRs, you shoudln’t be surprised if the stock returns 10% over the long term.
The importance of hurdle rates is quite interesting when compared with how many companies don’t publicize them.
Ziff Davis, a TMT roll up of media and software assets, do share. It’s 20%.
If you can, try discovering this number whenever underwriting an investment.
Meta talk: the management that didn’t set the example
This week in Meta talk, Mass Layoffs and Absentee Bosses Create a Morale Crisis at Meta ($, NYT). I can’t believe what they are doing. Zuckerberg will ever get a pass, even though I think it lacks empathy to take parental leave during Year of Efficiency. But if you are Adam Mosseri, common, you need to go to the office every day! You cannot live in London 8 hours ahead of Menlo Park and ask your employees to go to the office and you not be there! Your employees will grow angry at you because they are under the threat of further layoffs and because you feel like a duck to all the buzzwords of recent years: NFTs, TikTok, metaverse. Oh my God, you gave a Ted Talk on a creator-led internet on the blockchain! Be humble and go to the office, please! Kevin Systron went to office, you are not Kevin Systrom, go to office.
I am sure that this advice applies to most of Meta’s senior leadership. Unless you have $30B in the bank, go to the office. Don’t ask sacrifices from the troops you are not willing to make yourself. Otherwise, the Year of Efficiency can strike back.
Should you invest in banking stocks?
Recently, I spent some time reading JPMorgan Chase & Co. letter to shareholders and listening to the most recent earnings call. It’s a great reading.
He’s still bearish banking. He thinks Basel IV will make capital requirements higher and he accepts his fate that more regulation will come because of SVB. On the other hand, the JPM franchise seems to be going great. Less than a portion of a portion of their credit book is commercial real estate, mostly focused in the multi-family side. Their provisions for a recession seems to be out of consensus into the bearish side. The bad part about JPMorgan is that they think the bank is overearning, the bank expects a net interest income of $84B this year, but $70B over the long term.
So, the answer is no.
Meme of the week
It can make commercial sense to destroy the economics of a sector. If I were to compete with Microsoft and Google, I’d be funding some money to open-source alternatives. Actually, Meta just did it. Meta is still a bit behind in LLMs, but it’s easy to see why nuking the economics of AI is aligned with Meta’s corporate strategy. 1) Meta is in the advertising business, and Google and Microsoft are the #1 and #4 players in that market, therefore anything that hurts these players is good for Meta. 2) Meta is in the social media/general entertainment business, and Google and Microsoft own some of the largest competitors to Meta: YouTube and LinkedIn. 3) Meta is in the hardware business, and Google and Microsoft are in the operating system business. Anything that hurts Google and Microsoft is good for Meta’s hardware endeavors. The same strategic framework can be applied to IBM, Amazon, Alicloud, Nvidia, and Oracle. If I were Safra Catz, I’d be offering my GPUs for ridiculously cheap prices just to screw Microsoft.
In reality, they spice the data with some private data, like a dataset only a company have access or by paying Kenyan workers to create some training data. In the image generation land, which is 6 months ahead in the hype cycle, Adobe created a generative model which the value proposition is using less data, and therefore being worse. The idea is you’re going to be protected from copyright issues.
Or if not through incrementalism, at least through a breakthrough that everyone can have access to.
Not exactly true. Some segments of semiconductors don’t see much innovation, but because over the decades they consolidated to 2 or 3 players, the competition is less ferocious. Therefore, the insurmountable amount of capital that would be needed to compete in 2042 in AI would mean that there would be a floor on returns on capital.
INB4: Yes, he played a rematch against Spassky in 1992.