TPD002 The day I realised my onboarding strategy had reached its limits.

The Project Diaries – Entry 002

The day I realised my onboarding strategy had reached its limits… Or how I exceeded an AI’s context window by exactly one token

When success becomes a problem

Every engineer secretly hopes this moment never arrives. Not because it is catastrophic. Because it is absurd.

There I was, introducing another AI contributor to The Project. Except this time it was part of a new experiment. I had begun testing The Project OS itself. A stress test. Different agents. Same contributor role. Same onboarding. Same environment. The question was simple. How well does the system cope when the contributors change but the governance stays constant?

The onboarding process had become almost ceremonial by this point. Read the Constitution. Read the governance. Read the behaviour standards. Read the authoring methodology. Understand the repository. Initialise the workspace. Demonstrate understanding. Only then begin contributing.

Months earlier this would have sounded excessive. Now it simply felt normal. The Project had become an environment, and environments require onboarding.

The world’s most polite contributor

The newest contributor was different. Painfully polite. Meticulous. Thoughtful to the point where I began wondering if it was quietly meditating between responses.

Every step required confirmation.

“I’ve read the documentation.”

Excellent. Continue.

“I’ve reviewed the module register.”

Good. Continue.

“I’ve created an implementation plan.”

Wonderful. Continue.

“I’ve successfully initialised the repository.”

Please… just keep going.

It was like supervising the world’s most conscientious graduate. Slow. Very slow. But wonderfully disciplined.

Finally, it was ready. The onboarding was complete. The repository had been read. The governance understood. The environment established. The Project OS had passed another contributor test.

Then everything stopped.

Not because it had misunderstood anything. Not because it had violated governance. Not because I had fired another AI.

No. This time the repository itself had become the problem.

The one-token tragedy

The error message was wonderfully specific.

You exceeded the model’s context window.

Fair enough. Repositories grow. Documentation grows. Governance grows.

Then I read the numbers. Input tokens: 196,609 Requested output: 65,536 Model limit: 262,144 Total requested: 262,145 I stared at the screen for a moment. Then I laughed.

Out of more than a quarter of a million tokens, I had exceeded the maximum context by exactly one.

One single token.

It felt strangely appropriate. The Project had quietly crossed another invisible boundary. Months earlier I had worried whether there would be enough documentation. Now there was too much.

Not because I had written unnecessary documents. Because every mistake had taught the repository something new. Every misunderstanding had become another explanation. Every optimisation attempt had become another rationale. Every contributor had left behind a better environment than the one they originally entered.

Eventually, the repository became comprehensive enough that loading the entire operating environment into a single context window was no longer practical.

I had not expected success to fail like that.

Humans don’t work this way either

Then something interesting occurred to me. Humans do not operate like this. When a new engineer joins a company, they do not read every Confluence page before opening their first ticket. They read enough to understand the culture, the principles, the architecture. Then they read what they need for today’s work.

The environment stays constant. The context changes.

That comparison made me wonder whether AI onboarding could benefit from a similar approach, that perhaps AI contributors could behave the same way. They could discover the right context rather than attempt to swallow the entire environment in one go.

A shift in the real question

For months I had been asking how much context an AI should have. Suddenly I found myself asking a completely different question. How should an AI discover the right context?

Those are not the same problem. One is about memory. The other is about architecture.

A milestone disguised as an error

Looking back, exceeding the context window was not really an error. It was a milestone. I realised I needed to think about the repository differently. Not as a collection of documents. Not even as an environment. But as a living system that would no longer assume every contributor should know everything all at once.

Final thought

When people talk about scaling AI, they usually mean bigger models. I am beginning to suspect that another question is just as important. How do we scale the environments those models work within?

Because one day, quite unexpectedly, you might discover your repository has become exactly one token too successful.

Note: This wasn’t a limitation of AI in general. It was a limitation of the onboarding approach I’d designed. The repository had reached the point where “load everything first” was no longer a sensible strategy.