FoundationDB Meetup [In-Person and Online]

Join us for the FoundationDB Meetup in San Jose, CA!
This is a monthly meetup series in Adobe's office in San Jose. We will feature an interesting roster of speakers who use or work on FoundationDB.

Agenda:

• Social Hour - 30 mins
• Talk: Designing a control plane for FDB fleet management - 45 mins
• Talk: Spark connector for FDB - 15 mins

Talk Title: Designing a control plane for FDB fleet management
Speaker: Senthil Kumar Ramamoorthy (Senior Engineer, Snowflake)
Speaker Bio:
With over two decades of experience, Senthil has made significant contributions to the development and optimization of advanced storage technologies. His expertise spans snapshots, replication, name-space management, space accounting, garbage collection, cache, and performance optimization across NFS and CIFS protocols. He is also an inventor with multiple patents related to storage efficiency and optimization.
Currently, he is an engineer at Snowflake, where he led the development of the snapshot-based backup and restore feature. He is also the lead engineer for designing and implementing the control plane for FDB fleet management and is a committer for FoundationDB. Senthil brings deep expertise in both the architectural and practical aspects of storage technologies and is passionate about optimizing storage performance for modern distributed systems.
Talk Summary:
In this talk, Senthil will delve into how FoundationDB is managed at scale at Snowflake, with a focus on the control plane for fleet management. He will discuss key aspects such as provisioning, scaling, autoscaling, and load balancing across large fleets of FDB clusters. The session will also cover the challenges of managing distributed systems at scale, upcoming enhancements in fleet management, and lessons learned from real-world deployments.

Talk Title: Spark connector for FDB
Speaker: Akanksha Nagpal (Senior Engineer, Adobe)
Speaker Bio: Akanksha is a Software Engineer with extensive experience in designing and building large-scale distributed systems within the Adobe Experience Platform (AEP). She has led the development of systems for efficient big data ingestion into Adobe's Identity Graph, including building highly scalable data pipelines with Apache Spark and Scala, handling over 500,000 messages per second. Akanksha has also played key role in exploring FoundationDB's architecture and encryption capabilities, focusing on its potential for Customer Managed Keys (CMK) within Adobe's infrastructure. Passionate about innovation in data processing, she is dedicated to contributing to the broader engineering community.
Talk Summary:
This talk will delve into how Adobe developed a custom Spark connector for FoundationDB to enhance data processing capabilities, addressing limitations of existing solutions. Akanksha will discuss the motivation behind this development, how the connector integrates Spark with FoundationDB for large-scale data operations, and its role in powering customer insights and reporting in AEP. She will also share key lessons learned, operational insights for managing the connector in production, and Adobe's future roadmap, including planned enhancements and potential open-source contributions.

Disclaimer - The meeting will be streamed, recorded, and uploaded to YouTube or other online streaming sites used by this group.