2024 RocksDB EOY Meetup at Meta HQ

We’re excited for our EOY 2024 RocksDB meetup!

If you are interested in presenting a talk, please fill out the form here: https://shorturl.at/tPDZU

Date: Tuesday, December 10th from 3:30-8 pm
Where: Building 15 in MPK Classic Campus , Conference Room: Son of Ping Pong
For remote attendees, here is the zoom link.

Agenda:
3:30 - 4:10 pm - Check-in/social time
4:10 pm: Kick-off & Welcome
4:15-6:30: RocksDB Talks by Industry, Academia, and internal Meta engineers. See below for talks (order TBD)
6:30 - 8 pm: Social Hour & Dinner

Title: RocksDB stress/crash testing
Speaker: Hui Xiao (Meta)
Summary: This talk is to share internals about RocksDB integration test - stress/crash testing, including its architecture, testing coverage and major bugs found

Title: Multi-Column-Family Iterator in RocksDB
Speaker: Jay Huh (Meta)
Summary: Multi-Column-Family Iterator is a new feature launched in 2024. Available in two variants, CoalescingIterator and AttributeGroupIterator, it provides a convenient API for users to iterate across keys from multiple column families, automatically resolving key & column-name conflicts and providing consistent view across the column families.

Title: Multi-dimensional filtering in RocksDB
Speaker: Peter Dillinger (Meta)
Summary: RocksDB has experimental support for splitting keys into segments and constructing filters on various key segments. Such filters are especially useful for range queries involving key segments that are correlated with write time. We are looking to expand and polish this new generalized framework for filtering.

Title: Sally - A RocksDB Storage Management Plane
Speaker: Dhanabal Ekambaram, Sushil Patil (Meta)
Summary: Come learn how Meta manages millions of RocksDB instances in production. Sally(Storage Abstraction Layer LibrarY) is a data and storage management plane that manages the performance and durability needs of customers through intent based storage profiles abstracting away details of local vs remote storage backends. Sally supports complete RocksDB lifecycle management including operations like provisioning, operating, cloning, deletion, backups, and restores.

Title: Decoupled-RocksDB and Its Automatic Tuning with LLMs
Institution: Arizona State University
Speaker: Zhichao Cao
Summary: In this talk, ASU presents Decoupled-RocksDB, a new RocksDB architecture designed for disaggregated (cloud) infrastructure. Decoupled-RocksDB introduces a lightweight compaction service managed by the execution plane, which can run on any node near disaggregated storage. A control plane dynamically schedules compaction jobs to optimal locations (either local nodes or dedicated compaction nodes), adapting to resource availability, job size, and overall performance impact. Additionally, we offload immutable memtables to disaggregated memory (e.g., RDMA-enabled) to support scalable remote flush and enhance read performance through hybrid read delegations. Finally, we’ve proposed and developed an automatic tuning platform for RocksDB powered by large language models (LLMs), enabling rapid, cost-efficient, and interpretable tuning for diverse workloads and deployment scenarios.

Title: How Timeplus leverages RocksDB to do billions of keys streaming join and aggregation
Speaker: Ken Chen
Summary: Large scale keys (at billions scale) join and aggregation in streaming processing is difficult. It is even more difficult to do data "mutability" (the data of the same key changed, need revise the previous join / aggregation result) streaming join and aggregation at this scale and when changes are frequent. We can't hold all of the aggregated states and / or joined data in-memory since they are too large. In this talk, we will share real-world large scale keys streaming aggregation / join use cases and how Timeplus leverages RocksDB to build `Mutable Stream` and on-disk `HybridHashTable / HybridAggregator` to solve the challenge at far lower memory footprint (10 ~ 20X memory reduction) but still achieve the good performance.

Title: JunoDB - PayPal's Distributed Key-value Store
Speaker: Yaping Shi
Summary: JunoDB is a highly scalable and available distributed key-value store that powers PayPal’s diverse range of applications, processing over 350 billion requests daily. Its architecture includes a lightweight client library, a proxy layer that simplifies client interactions while ensuring load balancing and connection scalability, and a sharded storage layer leveraging RocksDB for persistent storage. With a cloud-agnostic design, JunoDB can operate across on-premises and multi-cloud environments.
There will be swag for in-person attendees(limited to first come first serve)

Note: Please check back on this event as we will update with more details as the date approaches.

Hope to see you there!

----

This event will take place in Building 11 at Meta’s Menlo Park Campus. We will not have a shuttle service and there should be plenty of available parking. Please note that if you decide to take Lyft or Uber to the event there is no pickup or drop-off allowed from the main campus and you will need to get dropped off at another building location where you can walk (recommended drop-off at Building 41 or 58).