When Stack Overflow introduced our Industry Guide to AI back in January 2024, the world was at an inflection point. It was clear that this new wave of technology powered by Generative AI was poised to have a profound impact across nearly every industry, but very few companies had figured out how to build and scale their GenAI efforts. Most were not ready to put this new technology into production, or allow their employees and customers to freely interact with it.

Today, most major organizations have explored or adopted GenAI in some capacity. Software development can be enhanced by AI code suggestions and the automated creation of documentation and tests. Marketers can write and refine more copy, design and illustrate campaigns, and even transform ideas and images into working web code. Video and audio production are enhanced by systems that can automatically clean, edit, enhance, and even create new material on the fly. The list goes on.

In our updated guide, we reviewed and refreshed many sections to reflect how the tools and technologies that underpin this new era have evolved. We also updated our guide to implementation to give readers a sense of where we sit in the hype cycle. Some of the foundational elements of GenAI have changed significantly. For example, the style of retrieval augmented generation we highlighted last year is now known as “naive RAG,” because best practices have evolved to a far more elaborate set of tools and techniques to improve on the basic approach.

We have also seen new modalities appear. Previously, all models were built to provide an immediate answer to a user’s query. Now, there are models that can be trained to receive a query, think through a response, iteratively work out an answer, and provide the user with a much more nuanced reply. On the subject of AI innovation, while some hackers had created AI agents a year ago, almost all interaction with GenAI was in the form of a chatbot that could respond with text or images. We now have systems that can take a user prompt and, with the user’s permission, utilize their computer to carry out instructions—everything from a basic web search to building a chart in a spreadsheet to tweaking and then running a software program.

Needless to say, 2024 was a year of rapid experimentation and chance in the AI space.

And while capabilities and adoption have greatly expanded, a major challenge to continued progress also seems to have emerged: model performance. As has been widely reported, the largest AI labs are no longer seeing large improvements in the performance of their models, despite increasing the size, cost, and complexity of the training. Incremental gains are still occurring, but the scaling laws that predicted the improvements from GPT 1-4 are not materializing.

Ilya Sutskever, co-founder and former chief scientist of OpenAI, told reporters that gains made by simply increasing the amount of training data and size of the models have plateaued.

“The 2010s were the age of scaling, now we’re back in the age of wonder and discovery once again. Everyone is looking for the next thing,” Sutskever said. “Scaling the right thing matters more now than ever.”

Not everyone agrees that the industry has reached a plateau, so it’s best to keep an open mind. While we’ll continue to add updates to this guide, the picture will likely come into better focus in 2025, as major updates to foundation models from the leading AI labs are already beginning to feel overdue.

Given that this is one of the central topics being debated in the GenAI industry, our goal is to focus on providing as much information and context as we can to Sutskever’s question: If simply scaling is no longer enough, what kind of data or techniques will have the biggest impact on pushing AI models to new levels of intelligence and capability? How should organizations think about generating, cleaning, annotating, and preparing their internal data to take best advantage of this?