David vs Goliath - the power of small models

With the trend to larger and larger models, we might forget that a specialized, domain-specific small model might be a better solution to a given problem, not only on raw performance but also when accounting for all factors like run-time requirements and costs.

At Chimnie, we needed specialised parsing of postal addresses in the UK. After using off-the-shelf larger models, we decided to train our own transformer model from scratch, specialized in UK addresses. It turned out to be more performant while requiring a fraction of the resources of other more general models. And it was not hard to do!

In this 5 minute demo I’ll show how we built the dataset, the training pipeline and the final working model.