All Stories

How to emulate temporary tables in Athena

Use CTAS to create a temporary table in Athena
Bartosz Mikulski
28 Dec 2020

How to enable S3 bucket versioning using Terraform

How to configure Define S3 bucket versioning in Terraform
Bartosz Mikulski
27 Dec 2020

How to get a notification when a new file is uploaded to an S3 bucket

Get a Slack notification when a file is uploaded to an S3 bucket
Bartosz Mikulski
26 Dec 2020

Get an XCom value in the Airflow on_failure_callback function

How to get the task instance in the on_failure_callback to get access to XCom
Bartosz Mikulski
25 Dec 2020

Add the row insertion time to a MySQL table

Automatically add the insertion and update time in MySQL
Bartosz Mikulski
24 Dec 2020

Best practices about partitioning data in S3 by date

How to partition data in S3 by date in a way that makes your life easier
Bartosz Mikulski
23 Dec 2020

How to write to a SQL database using JDBC in PySpark

How to use JDBC driver in PySpark to write a DataFrame to a SQL database
Bartosz Mikulski
22 Dec 2020

How to add dependencies as jar files or Python scripts to PySpark

How to add a jar file or a Python file as a Pyspark dependency
Bartosz Mikulski
21 Dec 2020

How to reset the consumer offset in Apache Kafka topic

How to use kafka-consumer-groups.sh to reset topic offsets
Bartosz Mikulski
20 Dec 2020

How to purge a Kafka topic

How to remove all messages from a Kafka topic
Bartosz Mikulski
19 Dec 2020

Get the last day of the month in Redshift

How to use the last_day function in Redshift
Bartosz Mikulski
18 Dec 2020

How to make an unconditional join in Redshift

LEFT OUTER JOIN ON 1=1 in Redshift
Bartosz Mikulski
17 Dec 2020
About me | Newsletter