A long context window is not memory
Larger windows changed what fits in a prompt. They did not give the system a memory, and conflating the two produces a specific class of bug.
Lead instructor, Systems
Jonas Reiter
Systems engineer working on retrieval and serving. Interested in what breaks under real traffic.
LinkedInBackground
Jonas has spent most of a decade on the unglamorous half of machine learning: queues, caches, timeouts and the cost of a request. He joined Skillo after running platform engineering for a mid-sized search company.
He teaches the systems and deployment track, and reviews the final projects that involve production constraints.
Articles
1
Written by Jonas Reiter