mikulskibartosz.name
Career Coaching for Data Professionals
Speaker
Newsletter
Bartosz Mikulski
Building trustworthy data pipelines because AI cannot learn from dirty data
All Stories
What is s3:TestEvent, and why does it break my event processing?
S3 sends s3:TestEvent to SQS after setting up the bucket notifications
Making OFFSET LIMIT queries in AWS Athena
How to use OFFSET in AWS Athena queries
How to get an alert if an AWS lambda does not get invoked during the last 24 hours
How to get a notification when AWS Lambda stops begin used
How to set Airflow variables while creating a dev environment
How to use command-line to set Airflow variables
How to check when an Athena table was updated
How to track the time when an Athena table was updated
How to run an Airflow DAG in a loop
How to keep running a DAG indefinitely
How to use xcom_pull to get a variable from another DAG
Get an XCOM variable from another DAG
What to do when Airflow BashOperator fails with TemplateNotFound error
How to fix TemplateNotFound error when using Airflow BashOperator
Copy directories in S3 using s3-dist-cp
How to copy files in S3 and preserve the directory structure
How to select a random sample of rows using Athena
How to use a window function to select random rows from Athena
Use HttpSensor to pause an Airflow DAG until a website is available
Pause an Airflow DAG until an HTTP endpoint returns 200 OK
How to add an EMR step in Airflow and wait until it finishes running
How to use AwsHook and EmrStepSensor to add an EMR step and wait until it finishes running
« Prev
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Next »