The AI Engineering Revolution: How Foundation Models Are Transforming Business

The Scale of AI Post-2020

If you could use only one word to describe AI post-2020, it would be scale. The AI models behind applications like ChatGPT, Google's Gemini, and Midjourney have reached such proportions that they consume a nontrivial portion of the world's electricity, and we're approaching the limits of publicly available internet data to train them.

The scaling up of AI models has two major consequences. First, AI models are becoming more powerful and capable of more tasks, enabling more applications. More people and teams leverage AI to increase productivity, create economic value, and improve quality of life.

Second, training large language models (LLMs) requires data, compute resources, and specialized talent that only a few organizations can afford. This has led to the emergence of model as a service: models developed by these few organizations are made available for others to use as a service. Anyone who wishes to leverage AI to build applications can now do so without having to invest up front in building a model.

The Rise of AI Engineering

Foundation models emerged from large language models, which originated as language models in the 1950s. While applications like ChatGPT and GitHub's Copilot may seem to have come out of nowhere, they are the culmination of decades of technology advancements.

What Is a Language Model?

A language model encodes statistical information about one or more languages. Intuitively, this information tells us how likely a word is to appear in a given context. For example, given the context "My favorite color is __", a language model that encodes English should predict "blue" more often than "car".

The basic unit of a language model is a token — which can be a character, a word, or a part of a word. You can think of a language model as a completion machine: given a text (prompt), it tries to complete that text. As simple as it sounds, completion is incredibly powerful. Many tasks — including translation, summarization, coding, and solving math problems — can be framed as completion tasks.

The Power of Self-Supervision

What made language models the center of the scaling revolution? The answer is self-supervision. Traditional ML requires labeled data, which is expensive and slow to obtain. Language models can learn from text sequences without any labeling. Because text is everywhere — books, blog posts, articles, comments — it's possible to construct massive training datasets, allowing language models to scale into LLMs.

From Language Models to Foundation Models

Language models are limited to text. But as humans, we perceive the world through vision, hearing, touch, and more. For this reason, language models are being extended to incorporate more data modalities. GPT-4V and Claude can understand both images and text. Some models even understand videos, 3D assets, and protein structures.

While many people still call these models "LLMs," they're better characterized as foundation models. The word "foundation" signifies both the importance of these models and the fact that they can be built upon for different needs.

Foundation models also mark the transition from task-specific models to general-purpose models. Previously, a model trained for sentiment analysis wouldn't be able to do translation. Today, a single foundation model can do both — and much more.

Foundation Model Use Cases That Matter

The number of potential applications seems endless. Whatever use case you think of, there's probably an AI for that. Here are the categories driving the most value across consumer and enterprise applications:

Coding

In multiple surveys, coding is hands down the most popular AI use case. GitHub Copilot's annual recurring revenue crossed $100 million just two years after launch. McKinsey found that AI helps developers be twice as productive for documentation, and 25–50% more productive for code generation and refactoring.

Image and Video Production

Thanks to its probabilistic nature, AI excels at creative tasks. Midjourney generated $200 million in annual recurring revenue at just one-and-a-half years old. AI is now common for profile pictures, promotional images, and ad variations customized by season and location.

Writing and Content

An MIT study found that among professionals exposed to ChatGPT, average time on writing tasks decreased by 40% and output quality rose by 18%. AI is particularly good at SEO, email drafting, ad copywriting, and product descriptions.

Customer Support and Conversational Bots

The most popular enterprise bots are customer support agents. They save costs while improving experience by responding faster than human agents. AI copilots can guide customers through complex tasks like filing insurance claims and navigating corporate policies.

Information Aggregation and Workflow Automation

According to Salesforce, 74% of generative AI users use it to distill complex ideas and summarize information. AI agents that can plan and use tools have the potential to make every person vastly more productive by automating repetitive tasks like lead management, invoicing, data entry, and reporting.

Planning AI Applications for Your Business

Given the seemingly limitless potential of AI, it's tempting to jump into building applications. But if you're doing this for a living, it's worth stepping back. It's easy to build a cool demo with foundation models. It's hard to create a profitable product.

Assess the Risk Spectrum

1. Existential threat: If competitors with AI can make you obsolete, incorporating AI must be top priority. Gartner found 7% of executives cited business continuity as their reason for embracing AI.
2. Missed opportunity: AI can make user acquisition cheaper, increase retention, improve support, and accelerate sales and market research.
3. Strategic positioning: Many companies have failed by waiting too long to adopt transformational technology (Kodak, Blockbuster, BlackBerry).

The Last Mile Challenge

Initial success with foundation models can be misleading. It might take a weekend to build a demo but months to build a product. LinkedIn shared that it took one month to achieve 80% of the experience they wanted — and four more months to surpass 95%. Much of that time was spent dealing with hallucinations and product edge cases.

"The journey from 0 to 60 is easy, whereas progressing from 60 to 100 becomes exceedingly challenging." — UltraChat, Ding et al. (2023)

AI Investment Momentum

The New AI Engineering Stack

There are three layers to any AI application stack:

1. Application Development: Prompt engineering, evaluation, and AI interfaces. This layer has seen the most action and is still rapidly evolving.
2. Model Development: Modeling, training, fine-tuning, dataset engineering, and inference optimization. The pressure for efficient inference has never been higher.
3. Infrastructure: Model serving, compute management, and monitoring. Core needs remain the same even as models and applications change.

A key shift: with foundation models, differentiation no longer comes from model quality alone (since many teams use the same models). It must be gained through the application development process — better prompts, better evaluation, better interfaces, and better data.

Key Takeaways