Apache

Tutorials for Apache’s prolific suite of big data products such as Apache Spark, Apache Kafka, Apache Airflow, and other critical technologies for data engineers.

Spark

Join and Aggregate PySpark DataFrames

Perform SQL-like joins and aggregations on your PySpark DataFrames.

Spark

Working with PySpark RDDs

Working with Spark's original data structure API: Resilient Distributed Datasets.

Apache

Manage Data Pipelines with Apache Airflow

Use Apache Airflow to build and monitor better data pipelines.

Spark

Structured Streaming in PySpark

Become familiar with building a structured stream in PySpark using the Databricks interface.

Apache

Becoming Familiar with Apache Kafka and Message Queues

An overview of how Kafka works, as well as equivalent message brokers.

Spark

Cleaning PySpark DataFrames

Easy DataFrame cleaning techniques ranging from dropping rows to selecting important data.

Spark

Transforming PySpark DataFrames

Apply transformations to PySpark DataFrames such as creating new columns, filtering rows, or modifying string & number values.

Spark

Learning Apache Spark with PySpark & Databricks

Get started with Apache Spark in part 1 of our series, where we leverage Databricks and PySpark.