mikulskibartosz.name
Start here
About me
mlops.today
Bartosz Mikulski
Building trustworthy data pipelines because AI cannot learn from dirty data
All Stories
How to make sure that you did not leave an EMR cluster running
How to get notifications about running EMR cluster
How to automatically remove files from S3 using lifecycle rules defined in Terraform
How to define S3 lifecycle rules using Terraform
How to retry a Python function call
How to retry a Python function call in case of an error
Send a Slack message from an Airflow DAG
How to use the SlackAPIPostOperator to send a templated message to a Slack channel
How to delay an Airflow DAG until a given hour using the DateTimeSensor
How to use the DateTimeSensor in Airflow
How to run PySpark code using the Airflow SSHOperator
How to submit a PySpark job using SSHOperator in Airflow
How to add a manual step to an Airflow DAG using the JiraOperator
How can you add a human action to an Airflow DAG?
How Data Mechanics can reduce your Apache Spark costs by 70%
Stop wasting time and money tuning Apache Spark parameters
Conditionally pick an Airflow DAG branch using an SQL query
How to use the BranchSQLOperator to choose a DAG branch to execute
How to trigger an Airflow DAG from another DAG
How to trigger another DAG from an Airflow DAG
Why does the ExternalTaskSensor get stuck?
How to fix the stuck ExternalTaskSensor
How to render an Airflow template for testing
How to generate the code of an Airflow task from a template and a given execution date
« Prev
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Next »