The more I’ve been thinking about Large Language Models (LLMs), the more apparent it becomes that they are not so much a new technology as a new industrial revolution, one as earth-shattering for the world of labour as the original. Let me explain.
They don’t just alter what we can do with computers but how we do it and, more importantly, the economics of doing it.
The most significant change from the original Industrial Revolution was not that machines could replace manual labour but that the economics of that labour changed radically, including distribution, production, and consumption. Factories cropped up anywhere there was access to power; all forms of labour that could be economically powered by machines became so.
Consumers got access to whole new categories of products. Products that were unthinkable before the Industrial Revolution: cars, flight, and washing machines. The world changed.
I’m a kid of the internet; I started programming early when I got my Commodore 64 and was hooked once I got my first access to the internet. We made primitive websites, then the first web applications, and the modern web computing stack was born in front of us, but most of that web application stack had one thing in common. It operated and computed on structured data.
We had to transform the messy, unreliable real world of unstructured information into structured data that computers could process and understand. Believing in the internet and computer revolution meant believing in a world of clean, structured, and processable data. This was taken for granted as cloud computing and the first SaaS applications were born. The world of cloud computing was one of data processing.
This seemed like magic to the casual end-user, just consuming a web application in the browser without installation, database, or maintenance. SaaS was born. The fuel for most of these applications was data, and most of this data was unstructured, so each new application was born with an ecosystem of data extraction, conversion, cleaning, and manual maintenance.
This is costly, manual, and messy work. The clean cloud revolution has a long tail of dirty, manual, and costly data conversion. Where data existed, to begin with, the revolution was quick, but where data was analogue or non-existent, it was slow, which is why we are still lagging in digitising most business processes.
I got personally involved in this as a student when, by accident, I got involved in a project at the Danish Ministry of Science, Technology and Innovation. At the time, Jon Bosak and Tim Bray had created XML, and for the first time, we had a data-centric way of exchanging data between web applications; it’s fair to say that without XML, we would not have had a modern web stack.
At the time, the ministry was digitising the Danish government supply chain. I was part of a small group of pirates who had devised the idea to do this over the Internet (without charging suppliers to do digital business) rather than ancient EDI or similar protocols, but we needed a data exchange layer.
My Tradeshift co-founder Mikkel Hippe1 worked with Jon Bosak and others on an XML variant for business called Universal Business Language (UBL2). The beauty of XML and UBL was that they acted as a meta-layer, a universal language between applications that define all terms and data structures the same way. So no matter if you convert data from applications a, b, c, they all map to the same universal data structure and can all talk together.
This was also when Tradeshift was born, connecting companies with cloud computing and a universal language to exchange information, making it cheaper and simpler than ever before to do global business. It worked exceptionally well; in 10 years, we captured more trade than the GDP of Denmark, connected millions of companies in hundreds of countries, and did it at a fraction of the cost of previous generations of B2B by using the best technologies at the time.
Yet every single connection between buyer and supplier had to be done by convincing suppliers to join a cloud, mapping between different companies’ data standards, testing connections, and then making sure it worked, just so the large customer could get structured data into their ERP system and automate their business processes.
For every supplier connected, there was probably $10-20,000 of costs and manual labour in setting up an integrated link for large customers to automate their business processes. Furthermore, most systems were designed to be as restrictive as possible so no bad data could enter, potentially polluting the structured data and automated business processes, causing exceptions and rework further down in the supply chain.
This was not exclusive to Tradeshift; we just experienced it at a much larger scale due to the network effects of a supply chain, but the same goes for any modern SaaS applications. If it is true that structured data is oil, then manual data processing and mapping by people is the pollution we don’t see.
To understand how radical LLMs are, we must consider the last generation of data standardisation. UBL is called Universal Business Language, and we had to painstakingly define every word in every thinkable business process, or computers would not be able to process the data.
But what if we didn’t have to?
LLMs make it as easy to compute on unstructured data as structured data since they are predictive models that don’t care if the language is XML, ancient Greek, or lost languages3. They only care whether the data, or similar examples of it, were present in their training.
To show how powerful this is, I took a random PDF invoice from the internet and asked ChatGPT to read it, tell me what it was, and then transform it into a UBL XML invoice. This took seconds and cost cents. Contacting the supplier, integrating systems, or testing anything is unnecessary since the receiving application can understand and act on it without translation.
The next generation of applications will be able to use unstructured data as input as easily as structured data. This is as radical as saying that we can have cars that drive without needing oil wells, tankers, combustion engines, or smokestacks. The consequences will be enormous; just look at the massive data integration and transformation industry behind every cloud application today.
This is the true industrial revolution of LLMs: the whole supply chain of SaaS software is forever changed.
I know that many will argue that LLMs are still expensive compared to regular cloud computing, but is that the case if we take into account the whole supply chain of data and manual labour, or are we missing the potential because we can not imagine a world without all these other layers of the software stack?
Further, just like the original industrial revolution, I think most people are underestimating the economies of scale as this revolution accelerates; as Seqouia points out LLM infrastructure is being overbuilt to the tune of $200bn4 a year (just like the overinvestment in railways from the 1870ies was crucial for Americas later industrial base or the overinvestment in fibre infrastructure formed the foundation of modern cloud computing from the 2000’s).
This creates a massive market opportunity for LLM-based applications as costs get commoditised, combined with the fact that models are getting more efficient5. Making it even cheaper to build LLM-based applications and investments in legacy applications risky.
Why spend all the effort on data capture, integration, and capture from 3rd party systems when you can build an application that can take any input and solve your business problem instead?
This takes me to my main point: With the next generation of Software, we have moved away from process-centric computing to focus on problem and solution-based computing. The bigger question will be what this means for system integrators, IT departments, business applications, and the whole ecosystem. I doubt many applications or services that resemble the world before this industrial revolution will stick around.