A Conversation with OpenAI’s Chief Product Officer: Thinking about Products in the AI Era

Over the past year, OpenAI has been making progress in both products and models. Today, I’m going to share with you all an interview with Kevin Weil (OpenAI’s Chief Product Officer) conducted by Lenny, a well-known podcast host. Through this interview, you can learn about the product philosophy that OpenAI follows internally.

Generally speaking, it can be summarized into several major points.

Model Maximalism

What is “Model Maximalism”?

Build products around what the model can “almost do”, because the model will eventually catch up and achieve astonishing results.

This has already become a consensus in the era of large language models, and major companies are actively embracing this philosophy. It’s expected that while models will continue to improve over time, product development will lead the way.

Rather than expending a great deal of energy to build complex logic or workflows to address the current model’s shortcomings, it is better to trust in the model’s ability to evolve rapidly. It is possible that in a month, you won’t need to bother with this aspect at all.

Kevin mentioned that during the internal development of GPT-4o, the team had been obsessively experimenting with it for months. Everyone was deeply convinced of its capabilities and potential, which ultimately led to the launch that went viral on social media.

When developing a product like Deep Research, which requires the model to engage in in-depth thinking and information integration, they start designing the corresponding Evals (evaluation system) while conceptualizing the product form. Then, they watch the model continuously improve on these evaluation metrics.

The AI model you are using today is the worst one you have ever used in your life.

When we can truly understand this statement, we won’t overly dwell on the current limitations. We should boldly build applications that slightly exceed the model’s boundaries. It may not be long before, in just a few months, more powerful models emerge, transforming the currently barely functional product into an excellent one.

Iterative Deployment

What is “Iterative Deployment”?

Simply put, it means rapid deployment, public iteration, and learning about the true capabilities of the model together with users and the entire community.

Kevin mentioned that even within their team, they don’t fully trust the quarterly roadmap they’ve set. This is because technological changes are happening so rapidly that plans often can’t keep up.

The value of planning lies in thinking and aligning directions, rather than rigidly adhering to the plan.

Sam Altman encourages moving forward fast and is okay with the mistakes that come from acting quickly. If a new feature doesn’t work out well after it’s launched, just roll it back fast or fix it later. Don’t get stuck in some approval process.

For example, the obsequious persona of ChatGPT – 4o was retracted a few days ago

The core of this strategy lies in…

No one can fully predict all the potential, quirks, and optimal application scenarios of a new model.

Rather than building behind closed doors, it’s better to bring the product to market as early as possible—letting real users interact with it in real-world scenarios. This way, we can “evolve together” with society as a whole, and explore the true boundaries of the model.

Evals

If we need to build around the model’s capability boundaries and also achieve rapid iteration, how can we know whether the model is really up to the task? The answer is, of course, an evaluation system!

Kevin said, “We need to know whether the model’s accuracy in a certain task is 60%, 95%, or 99.5%.” This accuracy directly determines how the product should be designed. If a model can only achieve 60% accuracy in a task, the product design must take error tolerance and manual intervention into account. If it can achieve an accuracy of over 95%, a more automated process can be constructed.

During the development of Deep Research, the product team started designing Evals from the very beginning. They defined what a “stunning response” is and translated this into quantifiable evaluation metrics. Then, the research and engineering teams could fine – tune and optimize the model based on these Evals, continuously improving the model’s performance in key use cases.

Insights into the Frontiers of AI

In addition to the above three methodologies, Kevin also revealed OpenAI’s numerous thoughts on the evolution of AI technology and future opportunities:
• Models are becoming increasingly intelligent, faster, cheaper, and more secure. This multi-dimensional, exponential evolution speed is comparable to or even surpasses Moore’s Law.
• OpenAI focuses on building powerful foundational models and APIs. However, they are acutely aware that the vast majority of data and knowledge in the world resides within specific industries and enterprises. This means that there are huge opportunities to transform existing products and services in every vertical field using AI (especially through fine-tuning). OpenAI can’t and doesn’t want to do it all by itself.
• Kevin said that when solving complex problems internally, they often use model combinations. Break down big problems into small tasks and use the model most suitable for each task (it could be a fine-tuned one, a faster one, or one with stronger reasoning capabilities). Don’t always think that one model can solve all problems.

Conclusion

As a tech optimist, Kevin firmly believes that technology will ultimately drive societal progress. In the face of the transformations brought by AI, his advice is not to be anxious, but to focus on cultivating timeless core abilities: staying curious, independent, confident, and most importantly—learning how to think. That, perhaps, is the best preparation for the future.