artificial-intelligence

Depreciation Should Be 3 Years

The goal of Nvidia, as stated, is to produce graphics card that provide the best performance, such that is is not even economical to run a competitor’s cards. Let’s delve into this. First we need to understand marginal cost. If you build a machine to make widgets, and let’s say you buy the machine up front, the cost of the machine is a sunk cost. If you make one widget or a million, it doesn’t matter. Your sunk costs are your sunk costs.

If it takes $10 with of labor and inputs to make 1 widget, my marginal cost per widget is $10. Anything I earn over $10 per widget covers at least my marginal costs. I may not be profitable, but I’m making money on every widget and cash is coming in. If I make less than $10 on a widget, I’m losing money every widget and bleeding cash. Based on the market, I expect to sell 100,000 in a typical year. A price of $15 would cover both my marginal costs and the amortization of my sunk costs. That’s the best of all possible worlds.

What did I mean by amortization? I expect my widget machine to last, say, 10 years. I don’t just assign the cost of the machine to the first year, as it will produce revenue for the next 10 years. It distorts my performance by making my first year look like a legendary disaster, but the next 9 years like I’m a genius. I take 1/10 of the machine cost every year in amortization. If the machine was 5,000,000, that’s 500,000 a year. If I sell 100,000 widgets, I can assign $5 to each widget as its portion of the sunk cost. If I sell 100,000 widgets for $16, I cover both my sunk and marginal costs and $1 of profit per widget. If I sell 100,000 widgets for $14, I need to sell more than 100,000 to cover amortization.

But let’s say the widget business is not so good. Someone offers me $12 for 50,000 widgets. If I have no other orders for widgets, I’m going to lose money, given the amortization of my machine. Specifically, I won’t be able to cover 400,000 of the amortized cost. However, because my marginal cost is only $10, I get $100,000 of contribution to my sunk costs that I would not have received, had I not taken the deal. I could even make money if I got enough contracts at $12. If the offer is for 200,000 widgets at $9, I should not take the deal. Even if it is more money, I’m losing $1 per widget in marginal costs. I am losing more money on that deal than if I just shut down the machine. There is no sales volume that would result in anything but losing more money.

If my competitor buys a newer widget machine, that’s more efficient, and can make widgets at $8 a widget, they can undercut me at a $9 price point. They’re still making a contribution to their fixed costs (since the marginal cost is $1 less than the selling price). But I can’t match their price without bleeding money. I have no choice, at that point, but to re-tool and try to get my cost down below $8 to be competitive. Otherwise, they take market share and drive me out of business.

The cost of building the data center and filling it with GPUs is the sunk cost. Next, you have marginal cost which is largely electricity. It’s used to both power the devices and cool the devices. Then you have people to manage the data center. Then you have software costs related to that tuning or managing the model. The goal of Nvidia is to make it so that the marginal costs of operating a competitor’s chips is higher than the Nvidia chip. They want to produce new chips at a 1 year cadence, and within 2 years make the marginal cost of operating the newer chips decisively more attractive that the old chips. They want the economics to drive customers to replace on a 2-3 year cycle, to minimize their marginal costs.

One myth is you only need the latest and greatest chips for training, but when they’re two years old, they can move to inference. Training is when the model is sent data to get it to make a good response. Inference is when you and I ask the model for stuff. It’s estimated that 90% of the costs are for inference. That means the least efficient chips would be used for 90% of the cost. Moreover, as the models come on line, they are larger and push the limits of the older chips, meaning those chips have to work even harder for one unit of output. There may be some models that require too much memory or more bandwidth than the older chips provide. Using a 5 year old chip that’s struggling on a newer model is the highest possible marginal cost you can have. Granted, no one can have all new chips all the time, so you have a blend of generations with some chips being new and some chips older. You can track a blended marginal cost, and let’s say that’s $10 per 1,000 queries. [Don’t worry about the exact numbers – I’m making the math easy.]

What Nvidia is trying to do is to make chips so the AMD customer might by a cheaper chip, but their marginal costs are higher than Nvidia’s offering. For example, the AMD chip is a $12 per 1,000 queries, while Nvidia wants to be $10. But next year AMD will come out with a better chip, maybe meeting Nvidia’s $10 per 1,000 queries. So Nvidia has to come out with a chip that produces 1,000 queries for $8. And the same again next year. The two year old Nvidia chip still costs you $10 per 1,000 queries, but the new one costs you $6 per 1,000 queries. To stay ahead of your competition, you need to retire the 2 year old chips as fast as possible and buy as many of the new chips to keep your blended cost lower than your competitor’s blended cost.

But even the models themselves have a short life. Every few months a newer or improved model comes out. Most users want to use the latest model, even if it’s larger. The old model, which might run well on the old hardware, begins to see its market shrink. It may still have applications that are tied to it, but not new applications and not new users. The realistic lifecycle of a model might only be a couple of years. The chips are constantly being pushed to larger and larger models. To get better performance more and more “think time” is built into how the model is executed. This makes new models on old chips not just a little worse, but even harder because the model has to be run with more iterations to provide better output. It’s like having a truck that burns more gas than the new model, but you also have to make more trips with it.

Right now we don’t have a price war. We have a war of financing. Investors want to see growth. More than undercutting each other on cost, the goal is to be able to grow your user base with free services. If that’s the case, using your highest marginal cost chip, to offer free services, seems like putting a noose around your corporate neck. All your competitor has to do is pay more in sunk costs that have to be figured out some point in the future (buy more new GPUs) and kill you on marginal costs today. This won’t work forever, but it works in the short run.

The goal is to be the last one standing in a winner take all market. If your competitor is only bleeding $8 per 1,000 queries on free customers to your $10 per 1,000 queries, they can either offer better models or higher limits. They don’t have to be profitable. No one is profitable right now. No one is meeting their marginal costs. But if you can operate more efficiently, you have more time until the corporate grim reaper comes for you. Your runway lasts longer. What about the GPUs your investors financed? Don’t they want a return on their investment? Yes, that comes from you making it to the finish line while your competitors do not, rather than getting to positive cash flow.

So let’s think about this, again, for a 4-6 year lifetime for a GPU. I would buy 4 years, as that’s probably where the chip is clearly on its last legs, but at 3 years it may still be effective to run in a blended cost environment. That 4th year is likely when it really starts becoming an economic millstone around your neck. I think 6 years is absolute fiction, unless you have government and large corporate customers who have a fixed (and profitable) contract and don’t want you to change the model used by their applications. For the part of the market, like developer subscriptions, or chat agents, or new development, which live on the latest models, a 6 year old chip is likely killing you. It raises your blended costs and you burn through your investment even faster. So no, I don’t think a 6 year life span makes any sense whatsoever, except for a narrow type of government or large corporate customer. I think a 2 year lifespan makes a lot of sense for a very aggressive AI provider, who wants to keep their marginal costs as low as possible.

The Lack of Progress in Computing and Productivity

As “A Much Faster Mac On A Microcontroller” points out, even the cheapest processors today are capable of meeting most peoples’ needs with a couple of major asterisks (the underlying compute hardware in the article is about $10 to $30). I’m not going to claim that Word in 1995 is as good as MS Word in 2026. I get a lot of features on 2026 MS word on my Surface laptop that I didn’t get with my Mac 6100 from 1996. But a 1990s PC did that work while consuming less than 100 watts. A modern gaming PC can push 750 or even 1,000 watts while gaming, and 200 watts or more doing nothing. I would argue the biggest difference in desktop computers is in gaming and media consumption. Productivity has largely been static. And this has some implications for productivity we might want to consider before burning up the planet with AI and filling it with toxic e-waste.

What I will say is that for many use cases, the computer from 1996 is as useful as the computer sold in 2026. I have watched many people use nothing more than Outlook, Word, Excel, and their browser, day in and day out. These are tasks I was happily doing in 1995 on my DOS/Windows 3.1 laptop at work, or my Mac 6100 at home. With fewer features, but the largely the same work. The biggest change is moving from desktop applications to web based applications, real time collaboration, and apps like Teams and Slack.

From the productivity numbers, most peoples’ work computer shouldn’t be a $2,000 computer, or even a $1,000 computer burning 200 watts on average. It should be a $100 computer that consumes under 50 watts when running full tilt, but 5 watts at idle. With reasonably well written software, that cheap machine can do everything you need for the bulk of office workers, including Slack and Web-based applications in Chrome. But even more damming is it suggests that investing in computers has had little impact on productivity. I’m going to show you the best possible argument, with a slightly higher productivity when computers were being deployed at scale between 1994 and 2004.

How to read this chart. Productivity measures output per labor inputs. This looks at the percentage change, quarter to quarter. Why do we get negative productivity growth at points? Because the amount of output drops per labor our due to changes in demand. If you produce a little less but don’t lay off, productivity falls. Annual productivity has grown 1-1.5% in the last 20 years. This, combined with the growth in population of about 1% implied a 2-2.5% real annual growth for the highly developed US economy. (Setting aside the population growth was coming through immigration and the implications current policies have on that for this discussion).

But growth was not likely coming through the computer revolution. In fact, we see productivity grew more before 2000, when offices still had shared computers, than it did between 2008 and 2020, when almost everyone had at least one computer (if not two or three) at their disposal. The software and operating systems were much better in 2010 than in 1995. More computers and computer automation did not imply more productivity. This is known as the Productivity Paradox.

The 1990’s computer revolution had productivity gains about the same as the previous 40 years. So the desktop “computer revolution” didn’t meaningfully impact this measurement of productivity. In fact, the period from 2008-2020, with its ubiquitous computing and zero interest rates should have spurred investment leading to productivity growth. That period had unusually low productivity growth. More computer did not translate into more productivity. Take that in for a minute. The era when we introduced iPhones, iPads, and Android devices, and had capable, cheap laptops and desktops coming out of our ears, along with zero interest rates, had sub-par productivity growth. Maybe that’s a hang-over from the great recession? Before then, between 2000 and 2008 had at best historically normal productivity growth.

Why I focus on the desktop versus the enterprise is because I think there’s a real difference in individual versus institutional computing. I still think there is a lot of needless spending on bells and whistles that drives up the cost, but it’s hard to argue that book keeping and accounting aren’t more productive today. But if one group is getting a lot more productive, we still have to ask if this means another group is less productive to balance the average? Or the impact is so narrowly focused it doesn’t move the broader needle? It’s also possible to argue that technological change means computers are a necessary input to enable or unlock the next productivity gains through robotics and machine automation. Maybe productivity would be even lower absent computers. But I still maintain that most office workers would be only marginally less productive if you put a thirty year old computer in front of them. (Although they might complain a lot. And maybe quit).

Which brings us to artificial intelligence. I use it and find it’s maybe a 5% productivity boost. I can’t just yell at the computer to do things. I have to create a context and prompts to enable the AI to produce something that’s usable. And then I still need to refactor it to bring the quality up above a naive or basic implementation. And sometimes, it’s just faster for me to code it at a production ready level rather than doing all that other work. I could see how some developers actually find it a negative productivity tool. It doesn’t always generate the correct code and I’ve only had reliable success with very popular languages and tools. On legacy code it sometimes generates pure garbage. Adding AI may be no more beneficial to productivity than it has been to put a computer on everyone’s desk, or providing internet access, or Web 2.0 apps, and so on.

Like a lot of AI hype, that tweet in no way matches up with my experience. However, it later came out that what they did was feed the AI tool a year of context and it generated a “toy” implementation. That implies it is not code you would run in production without a lot of work. Sometimes the difference between a “toy” version and a production version is months of effort. Sometimes the real effort is figuring out what to build. A proof of concept or toy version is what I get when I use AI code assistance.

Which brings us to the question of how much money should we spend on AI and related investments? Based on the last thirty years, it’s unlikely that computing related investments will drive significant changes in productivity. And from my personal, anecdotal experience the gains from AI aren’t huge. Right now AI investments appear to be sucking up virtually all the available investment capital, along with energy consumption (causing some communities to face higher electricity and gas prices). What if, after all is said and done, we look at that same chart through 2035 and we don’t see any change in productivity? Think about all the money we would have wasted, all the e-waste we would have generated, and all the other lost opportunities we will not pursue?

Am I an AI doomer? No. We can do something an LLM can’t. We can pull a physical plug out of a wall. What I am is skeptical that this is the vehicle for this level of resource commitment. In fact, I’m for creating even less dependence on computer infrastructure in favor of other infrastructure in everything ranging from water treatment to rail to energy production and distribution. I’m all for diversifying semi-conductor production out of Asia so we have a secure source of semi-conductors if Taiwan is overrun. Some of it will require computers to automate and better manage those resources. And some will require an office worker to plug numbers into spreadsheets and send e-mails. But it’s hard to argue that spending much on new computer based tools for that office worker, or automating that work with AI, would do much for growth.

My Porcine Aviation Era

I have not had great experiences with AI development tools. I’m not a Luddite, but if the tool takes me more time than just doing it by hand, and I get the same results, it’s not a tool I want to use. In some cases, the code had subtle bugs or logic little better than the naive implementation. Or in other cases, the code was not modular and well laid out. Or the autocomplete took more work to clean up than it saved in typing. And in some cases it would bring in dependencies, but the version numbers would date back from the early 2020s. They were out of date and didn’t match to current documentation. For the most part the code worked, but I knew if I accepted the code as is, I would open myself (or whoever followed me) to maintenance issues at some later date. One might argue that the AI would be much better then, and could do the yeoman’s work of making changes, but I wasn’t sold on that idea. (And I’m still skeptical).

I would turn on the AI features, use them for a while, but eventually turn them off. I found it helped me with libraries with which I wasn’t familiar. Give me a few working lines of code and a config to get me going, and I’ll fix it up. It would save me a search on the internet for an out-of-date Stack Overflow article, I guess. I used prompt files and tried to keep the requests narrow and focused. But sometimes, writing a hundred lines of markdown and a four sentence prompt to get a function, didn’t seem scalable.

Well, pigs are flying and I found something that appears to be working. First, it involves a specific product. At the time of writing Claude Opus/Sonnet 4.5 seem to be quite good. Second, I have a better way of incorporating prompt files in my workflow (more on that below). Third, language matters. I found Claude gave me the same problems listed above when working on a Jakarta EE code base. But Rust is great. (Rust also has the advantage of being strongly typed and eliminating some of the issues I’ve had when working with Python and LLMs). Fourth, I apply the advice about keeping changes small and focused. Fifth, I refactor the code to make it crisp and tight. (More on that below). Sixth, ask the LLM for a quick code review. (More on that below).

The first topic I’ll expand on is changing my relationship with prompt files. Instead of attempting to create prompt files for an existing code base, I started writing a prompt file for a brand new project. I had Claude generate a starter and then added my edits. I believe in design (at least enough to show you’ve thought about the problem). This actually dovetails with my need to think through the problem before coding. I still find writing prompt files for an existing code base tedious. But, if I have to sit down and think about my architecture and what pieces should do, the prompt file is as good a place as any.

The other thing I want to cover is refactoring what the LLM hath wrought. Claude generated serviceable code. It was on par with the examples provided for the Rust libraries I was using. (Which also happen to be very popular with plenty of on-line examples). Claude would have had access to rich training data and pulled in recent versions (although I had to help a little). But the code is not quite structured correctly. In this case I needed to move it out of the main function and into separate modules. But mostly it was cut and past and let the compiler tell me what’s broken. Next, in the refactor, is to minimize the publicly exposed elements. Now I have code that’s cohesive and loosely coupled. The LLM by itself does not do a great job at this. Taste is more a matter for meat minds than silicon minds at this stage.

The final thing I want to touch on is using the LLM to review the code after refactoring. This gives me another data point on the quality of my code and where I might have had blind spots during the refactoring. I work with lots of meat minds and we review each others’ code on every commit. There are some good reviews and there are some poor reviews. And reviewing code is harder, if you’re not familiar with the specific problem domain. But the machine can give a certain level of review prior to submitting the PR for team review.

So that’s what I’ve found works well so far: green-field projects, in highly popular frameworks and languages, performing design tasks in the prompt file, using LLM best practices, refactoring the code after it’s generated, and a code review before submitting the PR. Auto-complete is still off in the IDE. And I’ll see if this will scale well as code accumulates in the project. But for now, this seems to produce a product with which I am satisfied.

[A small addendum on the nature of invention and why I think this works].

Peoples’ ideas are not unique. As one of my relations by marriage pointed out years go, when he moved above 190th street in Manhattan, there seemed to be a sudden run to the tip of the island to find “affordable” housing. In a city of millions of people, even if very few people have the same idea at the same time, the demand quickly gobbles up the supply. Humans build ideas up mostly from the bits and pieces floating around in the ether. Even “revolutionary” ideas are often traced back to maybe a interesting recombination of existing ideas. Moreover, people have sometimes been doing that “revolutionary” thing before but didn’t connect it to some other need or didn’t bother repacking the idea. What’s more important about “ideas” is not the property of the idea but the execution of the idea.

There is still something about invention, even if it is largely derivative, that the LLMs don’t appear to posses. Nor do they have the ability to reason about problems from logical principles, as they are focused on the construction of language from statistical relationships. Some argue that enough statistics and you can approximate the logical reasoning as well as a human, but I haven’t seen solid examples to date. The LLM doesn’t understand what to create, but it does summarize all the relevant examples necessary to support the work of realizing the idea. But even then, there are gaps that we have to fill in with human intention. Does this revolutionize coding for me? No, I estimate it makes me some percentage more productive, but in the 5-15% range. And of the time and effort necessary to realize an idea, coding is maybe 1/3 or less of the effort. And I worry that we’ll never get to the point this technology will be available at a price point that makes the provider a profit while being affordable to most people. After all, there’s a limit to how much you would spend for a few percentage points of additional productivity.

What if Data Center Space Were Free?

First, what is a data center? It’s a slightly archaic term, meaning where the firm aggregated its computer data. It’s from the age of centralized computing. While desktop computers were intended to run on household current, shared computers (mainframes, VAXes, Enterprise databases, etc) were to be installed in rooms with commercial power and air conditioning. These were often installed with special raised floors, allowing cables to run beneath your feet. The space where we put servers has changed, but not radically. Modern servers, although they use processors that are of the same instruction set as desktop and laptop processors, have constantly screaming fans and power requirements that can strain a typical 15 amp household circuit or typical office circuit. And rather than a raised floor, the cabling is now overhead.

The first big change was going from on-premise to hosted infrastructure. Prior to the 2000s, if you went to an internet company, you would likely be taken to their data center. It would be in the same building, or one building over, from their offices. Was there a server problem? Walk over to the data-center and take a look. (Tip: if the servers all looked the same, you could eject the CD ROM tray to help you find it). Starting in the late 1990s and through the 2010s, the data-center moved to a shared facility. Now, the data center might be hundreds of miles away. You might never visit it, or take the tour once before signing up. You ship your equipment to the site and they put it in racks for you, connecting cables as you specify. Or in some cases, lease the equipment from them. From the 2010s on, the move became to the cloud. In the cloud you are renting the equipment in very short time increments. The cloud provider gives you an API to manage the systems.

When you rent space in a data center you are paying for power, networking, and floor space. Power is a combination of the power your draw and some part of the infrastructure, like the backup generator. In a shared data center, you are generally responsible for your own uninterrupted power supplies. Networking is a function of how much of the bandwidth you intend to consume, or you can provide your own network pipe. Within the category of floor space you can add features like physical security, but it is essentially your portion of the footprint of the data center. All the other costs like staff to make sure the physical structure is operational, or someone to attach a cable, are either baked into those costs or billed separately. If you’re a small company looking to host a set of servers, you would likely pay for a rack (a single tower of servers) or a cage (essentially a fenced in area with a lock on it). That square footage combined with the power and network bill is your monthly fee for hosting your servers.

The cloud obscures all that and layers on management. You no longer have to set up your servers, storage, and networking. The cloud provider does that for you. You can still provision a VM or create a virtual (fake) network, but the physical hardware is hidden from you. Your interface to the computers is the API that cloud provider publishes. The costs can be per hour, minute, or gigabyte to seem ridiculously low. How can you justify managing your own servers when it not only involves all the costs mentioned above, but hiring and managing an IT staff, compared to those low costs. There are times when companies have been nearly bankrupted by their cloud spend, but for many it still feels like a deal. And it’s highly flexible, even if getting the costs down means buying into multi-year, inflexible arrangements more akin to leases.

But that’s not the real draw of cloud. Remember floor space, power, and networking? When a data center runs out of floor space, construction time is measured in years. It may not be practical to deliver more power. And even adding more networking always seems to take the networking providers months just to turn on a bit of fiber. God knows why. If you have your equipment in a data center and need to add more, the answer could be ‘no.’ It requires you to figure out where to put the equipment and how to communicate between the two data centers (although some providers had a solution for this). For all intents and purposes, the power, floor space, and networking in the cloud are infinite. And they handle other issues like fail-over, assuming you are willing to pay for it.

But we may be heading to an interesting situation, should the AI hype cycle crash. We will wind up with a lot of data center space heavily over-subscribed with power and networking, with no clients. We might be in a glut of modern data center space that is re-possessed by PE firms and regional banks lending to the projects. This would be like the glut of dark fiber that made YouTube and other social media initially affordable. You might have a group of lenders that’s suddenly trying to get rid of a largely completed data center at a fraction of what it cost to build. All you need to do is walk in and secure agreements for power and networking. The infrastructure will be there in varying degrees of completion. From powered on to cement slab.

What we lack is a view of operating systems that spans multiple computers. The cloud would still have an advantage for many companies that denuded themselves to system administrators to hire cloud administrators. It’s also hard to cost compete with a company that can smear those costs over a much larger number of systems. The idea of companies taking their data centers back in house isn’t what I think is likely. But it may open the door for newer and cheaper competitors. If not general cloud competitors, then maybe specialty providers that provide storage only or back up facilities in case there’s an outage? These new data centers would already have fat pipes to reach out to AWS or Azure.

Maybe another option is to finish the building and bring computer controlled manufacture. You would have plenty of power for laser cutters or mills. Even industrial processes like powder coating or electroplating require a significant amount of power. These data centers are being designed with more than enough power to spare. Like we divide existing data centers into ‘cages,’ these could also be divided up into cages for specific manufacturers. The data centers are also equipped with loading docks for semis. You want to make something like a bed frame that requires a CNC cutting board after board of MDF and grade ‘A’ plywood? No problem. You have a linear, football field sized building, where it gets cut, finished and packaged in one long assembly line. Maybe it would serve high-tech, multi-modal manufacture? You sit in a suburb of DC as you basically run a CNC cutter in Lousiana?

The down side is the AI chips themselves will have a very limited shelf life. Although using firms are extending the depreciation targets, the goal of the chip makers is to produce a chip so much better than the two generations ago, that it isn’t economically viable to operate the old chip. That’s a three year lifespan. Not because it can’t do the work, but because the electricity cost is too high. Maybe some of them could be used to support vision and robotics tasks related to manufacturing, but that may be only a small subset of what’s being purchased today.

Government Handouts are the Exit

It’s almost undeniable that the only reason the US economy has started slipping into a recession, or would have slipped months ago, is that investment in AI has driven about 1 to 1.5% of GDP. That’s an insanely huge figure. Not AI profits – which are years away even in optimistic projections. Unlike investments in roads, for example, about 60% to 70% of the AI investment is in chips that become obsolete in three years, but maybe as little as two years. The growth is happening so fast that power companies can’t keep up, which has lead to basically using old jet engines to turn hydrocarbons into CO2 to power those chips. All to give you an answer that might or might not be right, or just to generate offensive AI videos like Mahatma Gandhi eating a burger. Just to recap: the only thing keeping us from a recession is money being plowed into quickly obsolete “assets” (it’s hard to call them that – they’re almost a consumable), powered by setting even more climate change. The cement buildings, the data centers, left behind have a multi-decade life, but no one needs that much data center capacity. And if they’re unoccupied, they will go to shit.

So far the financing for this has gone beyond traditional investment to weird circular financing where company A invests in company B, who buys products and services from company A. Company A can then point to future orders and sales. Company B points to more investment. Everyone’s happy. Number go up. Company B then makes absurd projections of incomprehensible investments that need to be made, causing investors to snap up associated companies, and everyone happier because more number go up. But that’s okay, Company C promises (not necessarily delivers) future investments in A, making more number go up, after A promises to buy 3 times that much in company C’s products and services. At no point is numbering going up because Company B is anywhere near making back a significant amount of what it spends on short lived assets and jet fuel to power its data centers.

But surely this is good because it will make us all richer, right? Not really. If you think that, you haven’t been following along. I’ll give you a minute to catch up on how wealth inequality is both bad and accelerating. The benefits will be concentrated in the hands of the wealthy. Most of the benefit will be concentrated in the hands of people like Sam Altman or Mark Zuckerberg (who’s been searching for some new idea – any idea – since Facebook). The bonuses to execs and large share holders would be fantastic, if there any real chance of any of this earning back any money.

David Sachs and Sarah Friar made a statements which might indicate how these companies intend to square this circle of constant investment, no profit, and concentrated wealth. They will make the argument that if the government does not step in to support their narrow version of AI, the economy falters, and we go into recession. To keep that from happening, all we need to do is to make people like David Sachs wealthier, by bailing out their AI bets when the start to go bad by backstopping their loans or printing more money by driving down interest rates. (And therefore boosting inflation back up). I don’t think these are isolated musings. I think their air is probably thick with ideas in this vein, and these are just a couple of leaks. Maybe testing the waters? Or just they keep talking about it, so it’s a natural topic of discussion.

They have done everything in their power to make stochastic parrots seem like the next nuclear bomb. The country with the AI lead (whatever that means) will win the next wars. Tell that to Ukraine, who is using very much human piloted drones to attack 60 year old tanks and drones piloted by human Russian pilots. If businesses don’t adopt AI, or find that AI adoption is more limited than what they thought, and the profit potential seems to be a small fraction of what were overly optimistic projections, AWS or Microsoft’s investments in AI won’t seem like a good use of cash. Rather than lighting giant piles of money on fire, they should have bought back their stock. NVDA doesn’t look like a hot stock if the demand for their chips start to sputter. And Broadcom (AVGO) and ORCL start to falter at that point. (ORCL is already about 1/3 down. META – which has been floundering for its next idea – will also be seen to have wasted cash. The only dangers LLM based AI presents to the modern world is its ability to quickly mint disinformation and memes, and the financial crater it will leave when people no longer expect massive (or any) profits from the likes of Open AI. When that happens, and they stop lighting their money on fire, GDP shrinks and the US will probably slip into recession.

I was about to end there, but that isn’t quite the whole story. Because it isn’t just Wall Steet burning cash on stochastic parrots powered by jet engines. I feel like I would be remiss if I forget to mention all the private equity and funds that are investing in data center construction. To build the data center, largely unregulated private equity firms (which can borrow from regulated banks) have been making loans. If this all goes sideways, the 300,000,000 loan held by a PE firm for a data center could go to near zero, the small fraction recoverable only after years of bankruptcy litigation. Maybe there’s enough of these loans to make the systemically important, regulated banks sweat blood as their PE customers start to sputter. As long as number go up, the loan is getting serviced, but once number stop going up, we could have a massive, sudden influx of cockroaches. This includes some funds who buy notes or make loans as part of their income portfolios. You could wake up to read a horrible story that PIMCO is suddenly knee deep in bad loans in what should have been a safe, income generating, portfolio. And just to give you an idea of how poorly people view risk right now, you only need to look at the historically low spread between junk and investment grade bonds.

Your Mind, Their Thoughts

How does a company that’s hemorrhaging money get to profitability, when they offer a free service? You can create tiers or pay walls to funnel users to paying. This model is popular in the SaaS world, where the free version is a loss leader for future sales. But it isn’t a suitable model for every service. The other avenue to monetization is to show advertisements. It isn’t black and white, with some paid services, like Hulu, still show advertisements. The degree to which advertising is permitted is the degree to which the consumers (businesses or individuals) push back on the advertising.

Strictly speaking, Google and Meta are communication service providers on the SP 500 index. Practically all their money comes from advertising and sponsored content. Amazon and Microsoft are also making significant money from advertising and sponsored content. Your feeds on services like Linked In, X, Facebook, Tik-Tok, YouTube and so on are becoming a gruel of actual content and advertisements, either direct ads through the platform or “creators'” own advertising. New and improved with AI slop to foster more interaction and create more engagement. More of our economy is based on putting an ad in front of someone’s eyeballs than you would imagine. It’s easy to spot some advertising, such as a commercial about a laxative in the middle of a Football game. It’s harder to spot other ads, such as an influencer that doesn’t disclose a payment for a “product review.” The adage that if you aren’t paying for it, you’re the product, is ever more true. Have you thought, for five minutes, how the startups offering free access to LLMs are going to make money?

After thinking about it, I realized companies like OpenAI are better positioned to make money than we realize. First, the injection of cash has turbo-charged their data gathering. There is more investor money to harvest more and more data. I suspect this is also where the first moats for legacy chat-bots will happen, inking deals with content companies. New entrants won’t have the pockets or the bandwidth to negotiate a bunch of little deals to avoid getting sued. But that’s another issue. They are hoovering up everything. There is plenty of evidence they, or their agents, are ignoring any ‘ROBOTS.TXT’ entries that disallow scraping. When actual regulation arrives, it serves more as regulatory capture than creating equitable payments to the sources of content.

Second, we have come to accept that they can wrap your prompt in their secret prompt. These additions to your prompt are hidden, to arguably prevent circumvention. The stated reason to inject those prompts is to prevent leaking dangerous information, such as how to make explosives. They are also part of your terms of service. Attempting to circumvent or discover the prompts is a basis for canceling your account. The account that has your obsequious, pleasant friend on which you’ve come to rely. The point is we are now comfortable, or happily oblivious to, our prompt being wrapped in additional hidden prompts. The easiest way to hide advertising is to keep promotional material secret, like the safety prompts. And to make it a violation of the terms of service to avoid promotional prompting, like the safety prompting. You may even be aware that there is promotional prompting in general, but a specific prompt.

Another way is to selectively return supporting links. For example, if you ask about camping, cold weather clothing, or places to visit in Maine, you might get a link to LL Bean. This is relatively harmless, except that it is different from search, where you can move past the initial results. There is a push for search engines to move from search results to AI results. That may mean, in the future, that you only get the handful of links from the paid advertisers along with the chat response. There may be no button to show more results, or you may have to explicitly ask for more results. Combine that with the advertiser’s ability to modify the hidden prompts injected along with your prompt, and you might lose any awareness of other possibilities. And should the LLM lie about one retailer having the best price, or a particularly well-suited product, that’s chalked up to the hallucinations.

There is also the information you are divulging about yourself. Maybe you are spewing information you would never share on Facebook or even Google Search. For free users, the AI companies are likely to mine all prior conversations, building up a detailed profile. For paid users, mining may depend on the plan and the account, such as a corporate account versus an individual premium account. This is already happening through other social media, but the LLMs may have more detailed information about mental state or health. While it may be more a difference of degree than kind, the chats may have richer data. I suspect the need for vast amounts of storage is to handle the influx and processing of the data you are freely giving them about your internal emotional and psychological state.

What I fear, and may be more deeply concerning, invoving the ability of the LLM to prime you over time. In some sense, search is “one shot.” You type in a search, you get back results. Facebook and other social feeds have been shows to influence peoples’ opinion not on just products, but able to alter their mental health. Their advertising can be better concealed. You might have retweeted or re-posted what were ads in the past. To a degree people have unmasked some of the behavior. We might be more inured to it now, and therefore have a bit of a resistance, but the social media algorithmic rabbit hole is alive and well. We know to watch for “radicalizing” content. What we don’t know how to spot are radicalizing outputs from a chat bot.

LLMs and chat bots may catch us in a particularly vulnerable way. We have a bias to believe the computer’s response is a neutral, disinterested party. And the responses from the LLM are private and highly individual. Not like public feeds on various Apps. If a company that sees sufficient lifetime value in a customer, they may be willing to pay over multiple chats. Maybe a $100 for a couple of months of ‘pushing.’ Imagine if the opioid vendors had access to this technology. Paying a few dollars to push someone toward a prescription for their brand of opiate may be worth thousands of dollars per patient. And each future addict’s chats are essentially customized to that person. Remember, we have plenty of evidence that existing social media can shape opinion and even mental health. Show enough people “PSA” style ads about enough vague symptoms and people will, in fact, ask their doctor if that drug is right for them.

But the big hook is the outsourcing of your cognition. Human beings are inherently lazy. If an escalator is present, almost no-one takes the stairs. Go to the airport and watch people, without luggage, queue for the escalator. The stairs are almost empty and there is just one flight. But they will wait in a press of people. Having a tool that allows you to ‘just get the answer,’ is like your brain being given the option to take the escalator. Instead of thinking through even simple problems, you just pose the prompt to the chat bot. And just like muscle gets soft and atrophies with disuse, your ability to solve problems dwindles. It’s like the person who begins to take the escalator not because it’s a little easier, but because they are now winded when taking the stairs. Need a plan for a workout? This shouldn’t be that hard, but you can just ask the LLM. (Ignoring it may actually give you bad advice, or in a world of sponsored chats, push you toward products and services you don’t need). Need a date idea? Just ask the LLM. Is your back pain something to worry about? The LLM has a short answer.

At least reading search results might inadvertently expose you to a knowledgeable and objective opinion between ads. If I search on Google for US passport applications, the first link is actually a sponsored link to a company that will collect all my data and submit my passport application for me. Who is this company? I’ve never heard of them. It ends in a “.us” domain, making it seem US related, but who knows what they do with the data or how they store it. The second link is the state department, but the third link is not. The only reason the state department is there, is because they paid to sponsor a search result. But at least it’s there. And it’s also in the list of general results. Google, Facebook, Tik-Tok, and so on have a track record of taking advertiser money from almost anyone. Amazon’s sponsored content is sometimes for knock-off or counterfeit products. And some sites have absolutely no scruples on the ads they serve, ads which might originate from Google or Meta ad services.

The lack of scruples or selectivity demonstrated by other on-line services that take advertising, combined with the outsourcing of cognition, means you are exposing yourself to some of the shittiest people on the face of the earth. For every time you are pushed toward buying a Honda, you might also be pushed toward taking a supplement that is dangerous to your health. You will likely be unaware you are being marketed to, and in ways that are completely personal and uniquely effective on your psyche. In a state of mind where you’re being trained to expect an objective result, with additional prompts that are invisible to you for “safety,” and a technology whose operation is inscrutable, you have no idea why you are provided with a given answer. Is it your idea not to buy a car at all and just use ride share services every day? If the ride share services want the behavior to stick, they know it needs to feel like it was your idea. Is it your idea to really push your doctor for a Viagra prescription, even though you are an otherwise healthy, 24 year old male? You shouldn’t but those symptoms come to mind…

The possibilities for political advertising and opinion shaping are staggering. The LLM expected to give neutral answers is sponsored to return “right leaning” or “left leaning” answers for months before an election. Or it embeds language also used by framers of important electoral issues, to prime you for other messaging. Unlike the one-shot advertising in a search result, or the obvious ad on the page you ignore, the LLM is now doing your thinking for you. There will be people who will take the mental stairs because they know the LLM dulls their wits. But these will be fewer and fewer as LLMs get better and more common. With no evidence that on line advertisers find any customer objectionable, could Nick Fuentes be paying to inject your responses with pro-fascist content?

It will be impossible for you to determine what ideas are a product of your reason and research. You will still feel like you’re in control. You will still have your mind. But what goes through your mind will be even more carefully and accurately shaped. In a state were a few thousand votes can sway an election, how much would a campaign pay to advertise to specific voters, if they start seeing those voters adopt talking points and slogans from their LLM chats and social media posts? Would it be $500 per voter? Maybe you need to target 50,000 voters at a total cost of $25,000,000? That actually seems affordable, given the vast sums that are spent on some close elections. The free chat bot loses money. The “premium” plan at $20 per month loses money. Even the $200 a month plan loses money. But the advertising may be their pay-day. How much would you pay to get people to think the way you want them to think, each person believing this was the natural evolution of their own thinking. Casually using LLMs is essentially opening your mind to think other peoples’ thoughts.

Yes, But It’s not COBOL

Articles like these point to a multi-decade old language when something fails. Sometimes they don’t even wait for the post-mortem. There’s COBOL involved, so it must be COBOL. It’s old, right? First, let’s get one thing out of the way, and that’s the implication that the language is 60+ years old, so the computer it’s running on is old, right? No, it’s likely running on a modern IBM mainframe with modern tools. IBM makes the promise that if you write a mainframe program today, you can run it on future mainframes, without modification. That’s great for business customers because re-writing working software is expensive and time-consuming. These are highly reliable machines that are intended to run with down-time measured in seconds per year.

But the software failed, right? Because it’s old. That is complete balderdash. If you write a correct program today, it will continue to be a correct program 100 or 1,000, or 10,000 years from now. If you have an interest rate, and an amount, and compound that over a period of years, that answer won’t change. Because the program itself is applied math and logic. The rules of logic and math don’t change over time. Time itself isn’t the issue. What is the issue?

The issue comes back to maintenance. If I write a program that works today, it may not be completely correct. There may be bugs. Those need to be fixed and the effort I put toward fixing the bug impacts the long-term stability of the program. If fixing a bug is done under the gun, or on the cheap, it might cause what’s called “code entropy.” Code entropy is the de-evolution of a well written program into crap. Sometimes the bug fix must be rushed through, as customers are losing money by the minute. After that, we should go back and do a broader fix to the software. That may mean making changes to the underlying logic or other parts of the program. In doing so, we minimize the code entropy problem. But that maintenance cost money.

The next reason for maintenance is a change in requirements. This is especially true for systems that change every time there’s a change in the law. In some cases these changes are retro-active. This creates a lot of churn on short time-lines, and like bug fixes, and also results in code-entropy. The quick fix is rarely followed by the work to refactor the existing code, accordingly. The software entropy increases and the code becomes even harder to fix with the next change. Re-architecture of the old code costs money. Most places just indefinitely defer the maintenance on their old COBOL code.

Many commercial, private sector, companies rely on COBOL for high-volume transaction processing. It has many features more modern languages lack, like it’s English-like structure is legible to non-programming auditors. And modern features have been added to it, even if they have not been adopted by organizations using COBOL (especially the ones likely to skimp on maintenance). But it is not a truly modern language like Rust or Go, or even a middle-aged language like Java. And it exists in a specific computing environment (the mainframe) which is kept artificially expensive thanks to its monopoly supplier and small customer base. Getting trained on mainframe operations isn’t cheap and many companies don’t want to pay for it, as their newly trained people will leave for better offers.

Many of the problems people associate with COBOL are going to re-appear (and have re-appeared) when companies move to platforms like Java. I have been at many sites were Java programs on old, unsupported versions of Java are being poorly maintained. Or running on old, out of support application servers (programs that run Java code on servers). Databases that are so old and out of date that either the vendor has gone out of business or there is no longer a way to upgrade their database to the current versions. When out of date, poorly written Java code crashes, it just becomes a generic, bland, IT failure and mismanagement. But, because it doesn’t involve COBOL, it doesn’t get the headlines that are cheap and easy to score with an old language.

The biggest counter-example of “because COBOL” is the number of banks, brokerages, exchanges, payment processors, and insurance companies that quietly process about 80% of the worlds financial transactions on a daily basis. They have an incentive to perform routine maintenance. They have also quietly off-shored their software maintenance over the last few decades to places where a COBOL coding jobs is a good job. Offer most US software engineers a COBOL job and they will turn their nose up and assume you were joking. But in India, the Philippines, and China, COBOL is not the scarlet A that it is in the West.

I want to address something specific about the article posted above. It stated that because COBOL is by its nature defective or tool old, it cost the US 40 billion in GDP. That sounds like a lot, but in an economy generating trillions of activity, it is a rounding error. Second, re-writing that code has its own costs. That could be even more billions spent getting exactly to the same level of service provided today. There probably isn’t enough money in the world to re-write the existing, mission critical COBOL code into something else. That will take away from other budgets and, if not maintained, will result in the same problem just 10, 20, or 30 years in the future. And where will publications like FT get cheap headlines in the future, if COBOL goes away?