What we’re about
🌟 Welcome to the Vancouver Apache Airflow Meetup! 🚀
Are you passionate about orchestrating data workflows, automating tasks, and streamlining data pipelines? Join us in the heart of Los Angeles, where we gather to explore the exciting world of Apache Airflow.
🧠Who Are We?
We are a community of data engineers, developers, data scientists, and cloud enthusiasts who share a common interest in Apache Airflow. Our goal is to create a vibrant platform for learning, networking, and collaboration.
🔍 What Do We Do?
- Discover the latest trends and best practices in data orchestration.
- Share real-world use cases and success stories.
- Dive into hands-on workshops and tutorials.
- Connect with like-minded professionals in the field.
- Foster a supportive and inclusive environment for all skill levels.
🙋‍♀️ Who Should Join?
Whether you're a seasoned Apache Airflow pro or just getting started, this meetup is for you! Our diverse group welcomes beginners, experts, and everyone in between who is eager to expand their knowledge and skills in data orchestration.
đź’ˇ Why Join Us?
By joining the Vancouver Apache Airflow Meetup, you'll gain valuable insights, make new connections, and stay at the forefront of this dynamic field. Plus, you'll be part of a vibrant community passionate about driving innovation in data automation.
Ready to embark on this journey with us? Join our meetup group today and be part of the exciting Apache Airflow discussions in Vancouver!
This group is brought to you by Astronomer
Upcoming events (1)
See all- Vancouver Airflow Meetup at Amazon!AWS Offices, Vancouver, BC
Join Airflow experts from Stripe, AWS, and Astronomer for an evening of talks, networking, and delicious food and drinks at Amazon's Offices!
Join our three speakers as they delve into the exciting new features of the upcoming Airflow 3 release and share best practices for safely testing your Airflow pipelines—all while you enjoy delicious food and drinks!
WHAT WE'LL COVER:
What's Coming in Airflow 3
- Speaker: Kenten Danas, Developer Relations Manager at Astronomer
- Apache Airflow has become the de facto standard for data orchestration, with growing adoption in machine learning and AI use cases over recent years. Nearly four years have passed since the release of Airflow 2, and as a community, we've reached consensus that it’s time for a major, foundational update: Airflow 3—now just around the corner. In this talk, we’ll introduce the vision behind Airflow 3, highlighting key industry trends and how Airflow will adapt to meet these needs. We’ll cover the architectural enhancements designed to support emerging use cases and distributed data infrastructure models. Attendees will gain insight into the major new features, the intended outcomes of the release, and the foundational concepts of Airflow 3 that will continue evolving in subsequent 3.x releases.
Stress-Free Airflow development: From Dev to Prod at Stripe
- Speaker: Nick Bilozerov, Sr. Software Engineer at Stripe
- At Stripe, compliance with regulations is of utmost importance, and ensuring the integrity of production data is crucial. To address this challenge, Stripe developed a powerful system called User Scope Mode (USM), which allows users to safely and efficiently test new or existing Airflow pipelines without the risk of corrupting production data. USM takes care of automatically overwriting the necessary configurations for Airflow pipelines, enabling users to test their production-ready pipelines locally with ease. This approach empowers Stripe’s teams to iterate and refine their workflows without the burden of manual setup or the fear of disrupting live operations.
In this talk, we’ll dive into the inner workings of USM and explore how it has transformed Stripe’s development and testing processes. Discover how this system seamlessly integrates with Airflow, allowing users to validate their pipelines with confidence and agility, all while maintaining the highest standards of compliance and data integrity.
Beyond SLAs: A Deep Dive into Airflow 3's New Deadline Alerts
- Speaker: Dennis Ferruzzi, Software Developer at Amazon & Apache Airflow Committer
- Do you have a DAG that needs to be done by a certain time? Have you tried to use Airflow's existing SLA feature and found it restrictive or complicated? In this talk we'll take a look at the all-new Deadline Alerts feature coming in Airflow 3 which replaces and revolutionizes SLA. We'll discuss how Deadline Alerts work and how they're different than the previous SLA feature. Then we'll look at some examples of workflows you can build with the new feature. Finally, we'll walk through some future usecases of using Deadlines for Tasks and even Assets.
AGENDA
- 5:00 PM-5:30 PM: Arrivals, Eat, Drink and Network!
- 5:30 PM-7:00 PM: Presentations!
- 7:00 PM- 8:00 PM: Eat more, drink more, and network!
