All Stories

Four books to boost your programmer career

I quit my dream job because of a book
Bartosz Mikulski
06 Jan 2020

What is the difference between data lake, data warehouse, and data mart

We can easily distinguish between them by focusing on three qualities: data structure (schema), data quality, and ownership.
Bartosz Mikulski
18 Dec 2019

Three biggest traps to avoid while setting Spark executor memory

What happens when you set the executor memory of a Spark worker which uses YARN as the cluster resource manager? Does it get exactly the amount of memory you requested?...
Bartosz Mikulski
16 Dec 2019

How to run Airflow DAGs for a specified date in the past?

Have you created a new Airflow DAG, but now you have to run it using every data snapshot created during the last six months? Don’t worry. You don’t need to...
Bartosz Mikulski
11 Dec 2019

What do you need to know about storing passwords in AWS?

How to use the AWS Secrets Manager
Bartosz Mikulski
09 Dec 2019

Apache Spark: should we use RDD, Dataset, or DataFrame?

Is there a difference between Dataset and DataFrame? Why do we even have both?
Bartosz Mikulski
04 Dec 2019

What a data engineer can learn from The Unicorn Project?

Have you ever seen a novel about developers? Reading such a book seems to be a massive waste of time, doesn’t it? After all, the internet is full of stories...
Bartosz Mikulski
02 Dec 2019

AI in production: Roobits Events360

What would you do if you were writing an application which had to process one billion events per day?
Bartosz Mikulski
18 Nov 2019

AI in production: Carta Healthcare

Bartosz Mikulski
11 Nov 2019

Using Exponentially Weighted Moving Average for anomaly detection

Bartosz Mikulski
05 Nov 2019

Using Boltzmann distribution as the exploration policy in TensorFlow-agent reinforcement learning models

There is a whole spectrum of exploration strategies between random and greedy policies.
Bartosz Mikulski
04 Nov 2019

Data engineering principles according to Gatis Seja

Lessons learnt from Gatis Seja's presentation about data engineering principles
Bartosz Mikulski
15 Oct 2019