Yasser Sheikh · MSc BEng AMIChemE RITTech

Writing · Apr 2026

Parse, don't validate, especially in a data-science repo.

Data-science codebases have a reputation for being the untyped swamp of software engineering, and the reputation is largely earned. I also think it gets the economics exactly backwards. When a web app has a type error, you usually get an exception and a stack trace within seconds. When an ML pipeline has one, the model trains happily on nonsense and produces predictions that are confidently and silently wrong. In my field those predictions are recommendations for a running chemical plant. Data science needs types more than product code does, not less.

Parse at the boundary

The single highest-leverage habit I know is to parse external data into a typed structure at the moment it enters your code. An API response, a sensor feed, a dataframe from someone else's pipeline: it gets converted into adataclass, a TypedDict or a schema at the boundary, and from that point on the type checker knows what it is. The alternative, waving it through asdict[str, Any] after a quick validation, means every function downstream is guessing. A dict[str, Any] crossing your codebase is a unit error waiting for production, and kilopascals look exactly like bars inside a float.

A suppression is a deferred bug report

Every type checker has escape hatches, and every escape hatch eventually becomes load-bearing. In the codebases I look after the rule is to fix the error, not silence it, and above all never to switch a whole file's checking off. A file-level ignore does not remove type errors; it removes your ability to see the next one. Where a suppression is genuinely unavoidable, it names the specific rule on the specific line, so that it reads as a decision someone made rather than a habit everyone caught.

Types buy courage

The compile-time bug-catching is real, but it is not the main prize. The main prize is that types make refactoring cheap. In a well-typed codebase you can rename a concept, split a module or change a schema, and the checker walks you through every consequence until the change is complete. I have led large refactors of live model-serving code in which the type checker effectively served as the test suite for the plumbing, which left the actual tests free to say something about behaviour. Untyped code does not rot because it is untyped. It rots because everyone becomes too frightened to touch it.

Notebooks are not exempt

Exploration is real work and notebooks are the right tool for it; some of my best modelling decisions started as a scruffy cell comparing five kernels. But "it's only a notebook" is how untyped prototype code ends up in production with# TODO: clean up fossilised above it. The discipline that works for me is to keep notebooks in a consistent shape, load then clean then explore then model, and to promote anything that survives into typed, tested library code before anything downstream is allowed to depend on it. The rule is not that notebooks must be tidy. The rule is that nothing important is allowed to live there.