mikulskibartosz.name
About me
Newsletter
Bartosz Mikulski
Building trustworthy data pipelines because AI cannot learn from dirty data
All Stories
Three biggest traps to avoid while setting Spark executor memory
What happens when you set the executor memory of a Spark worker which uses YARN as the cluster resource manager? Does it get exactly the amount of memory you requested?...
How to run Airflow DAGs for a specified date in the past?
Have you created a new Airflow DAG, but now you have to run it using every data snapshot created during the last six months? Don’t worry. You don’t need to...
What do you need to know about storing passwords in AWS?
How to use the AWS Secrets Manager
Apache Spark: should we use RDD, Dataset, or DataFrame?
Is there a difference between Dataset and DataFrame? Why do we even have both?
What a data engineer can learn from The Unicorn Project?
Have you ever seen a novel about developers? Reading such a book seems to be a massive waste of time, doesn’t it? After all, the internet is full of stories...
AI in production: Roobits Events360
What would you do if you were writing an application which had to process one billion events per day?
AI in production: Carta Healthcare
Using Exponentially Weighted Moving Average for anomaly detection
Using Boltzmann distribution as the exploration policy in TensorFlow-agent reinforcement learning models
There is a whole spectrum of exploration strategies between random and greedy policies.
Data engineering principles according to Gatis Seja
Lessons learnt from Gatis Seja's presentation about data engineering principles
How to remove outliers from Seaborn boxplot charts
Hide outliers when displaying boxplot in Seaborn
Python memory management in Jupyter Notebook
How to avoid memory leaks in Jupyter Notebook
« Prev
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Next »
About me
|
Newsletter