Guidelines for data science teams — a summary of Daniel Molnar’s talks

Daniel Molnar is one of those people whose talks I rewatch regularly. So far, I have seen two of them: “Data Janitor 101” and “Data Janitor returns.

In my opinion, his talks may be summarized as applying the good practices of Agile teams and “lean startups” to data science. He recommends striving for fast and good enough. He also strongly advises against over-engineering and using fancy tools.

I like that he encourages people to care about data quality and to choose KPIs that are challenging. After all, what is the point of measuring something just for the sake of feeling good?

In the data engineering area, he advises that doing ETLs in batches and building a data warehouse is enough, because making the ETL real-time or using streams is not worth the effort.

He also recommends an iterative approach to machine learning. First, deploy fast a good enough model. Later, build the model offline and redeploy each quarter. It looks like a suggestion to avoid the hype and focus on solving real problems. If you have been reading my blog for some time, you know that I love this approach.

Subscribe to the newsletter and join the free email course.

For me, the most essential idea from his talks is the “Friday 17:00 test”. In short, it tells you whether your data product (dashboard, report, recommendation, prediction, etc.) is actionable. Daniel Molnar suggests asking a question:

What can a person do with this result if he/she gets it at 5p.m. on Friday?

If the answer is “nothing, at least until Monday,” maybe you need to rethink that product.

Remember to share on social media!
If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.

If you want to contact me, send me a message on LinkedIn or Twitter.

Bartosz Mikulski
Bartosz Mikulski * MLOps Engineer / data engineer * conference speaker * co-founder of Software Craft Poznan & Poznan Scala User Group

Subscribe to the newsletter and get access to my free email course on building trustworthy data pipelines.