From AI Agent Harnesses to Context Engineering: The New Software Discipline
Lance Martin of LangChain joins us on High Signal
This week on High Signal, Lance Martin of LangChain joins High Signal to outline a new playbook for software engineering in the AI era.
Lance has been a friend of mine (Duncan’s) for the last decade and actually helped recruit me to Uber. He joined LangChain as part of the founding team two and a half years ago and has had a ground floor view of the rapid changes in software engineering. He’s built a prominent voice in the community, teaching classes and building the top open deep research agent in his spare time.
In the episode, Lance explains how the exponential improvement of foundation models is forcing a complete rethink of how software is built — why top products from Claude Code to Manus are in a constant state of re-architecture simply to keep up.
We dig into why the old rules of ML engineering no longer apply, and how Rich Sutton’s “bitter lesson” dictates that simple, adaptable systems are the only ones that will survive. The conversation provides a framework for leaders on the critical new disciplines of context engineering to manage cost and reliability, the architectural power of the “agent harness” to expand capabilities without adding complexity, and why the most effective evaluation of these new systems is shifting away from static benchmarks and towards a dynamic model of in-app user feedback.
Find us on Spotify, Apple Podcasts, and YouTube, or the full show notes online.
In this episode, Lance shares:
AI Engineering Works at a New Abstraction Layer: The ML landscape has fundamentally shifted from a world where every organization trained its own specialized models (like in the self-driving era) to one defined by a few large foundation model providers. Most users now operate at a higher level of abstraction, focusing on prompt engineering, context management, and building agents rather than model architecture and training.
The Bitter Lesson Demands Constant Re-Architecture: In the age of LLMs, applications are built on an exponentially improving primitive. This dictates that structures and assumptions baked into an architecture today will be made obsolete by tomorrow’s models, forcing continuous, aggressive re-architecture (e.g., one major agent product rebuilt five times in eight months) to avoid bottlenecking future performance.
Start Simple and Build for Verifiable Evaluation: Lessons from traditional ML still apply, emphasizing that simplicity is essential: use a simple prompt, then a workflow, and only move to an agent if the problem is truly open-ended. Evaluation remains critical, and systems should be designed around “Verifier’s Law,” meaning tasks are easier to solve if their successful completion is easily verifiable.
Context Engineering: Reduce, Offload, and Isolate. Managing the LLM’s context window is vital for controlling costs, improving latency, and maintaining output quality, as performance can degrade (context rot) even with very large context windows. Strategies include Reduction (pruning/summarizing old messages), Offloading (saving data to a file system, or using Bash/CLI tools to expand the action space instead of binding numerous tools), and Isolation (using sub-agents for token-heavy tasks).
Evaluation Must Be Dynamic and Component-Driven. Static benchmarks are quickly saturated by rapidly improving models. Effective evaluation now relies on aggressive “dogfooding,” capturing in-app user feedback, inspecting raw execution traces, and rolling new failure cases into dynamic eval sets. Additionally, system quality is improved by setting up separate evaluations for sub-components, such as the retrieval step in a RAG system.
🎬 Watch the clip where Lance talks about the whole new world of building agentic software where state-of-the-art products are re-architecting their system five times in 6 months, due to increasing model capabilities:
Where to find Lance
Lance on X / Twitter: https://x.com/RLanceMartin
Lance on LinkedIn: https://www.linkedin.com/in/lance-martin-64a33b5/
Lance on Github: https://github.com/rlancemartin
Lance’s blog: https://rlancemartin.github.io
Catch Up on High Signal
Missed our conversation with Anu Bharadwaj (President, Atlassian) on Why Most AI Agents Fail (and What It Takes to Reach Production)? Check it out here.
The concept of context engineering as a distinct discipline is facinating. The strategies of reduction, offloading, and isolation make sense for managing costs and latency. What striked me most is how the bitter lesson applies here, that simple systems will outlast complex ones as models improve. The constant rearchitecture cycle you mentioned really captures where software engieering is heading in the AI era.