Write the evaluation harness before the model
Teams that build the test suite first ship slower for two weeks and faster for the following year. Here is the order we teach.
Lead instructor, Evaluation
Mara Okonjo
Machine learning engineer. Builds evaluation harnesses for teams that ship models to production.
LinkedInBackground
Mara spent eight years on inference infrastructure before moving into teaching. She has run evaluation for recommendation systems at consumer scale, and she is unsentimental about the gap between a benchmark score and a system that behaves in front of users.
She writes the evaluation programme and marks every assignment on it herself.
Articles
1
Written by Mara Okonjo