sharechat.com, as data

We use ScyllaDB as our storage backend, which provides high throughput and low latency access to our pre-aggregated tiles. ScyllaDB's architecture is particularly well-suited for our use case as it can handle high write loads from our Flink jobs while simultaneously serving read requests from our Feature Service.### **4. Feature Service**The Feature Service is our serving layer that handles real-time feature requests from the recommendation system. When a feature request arrives, this service:* Checks the local cache if the same request has been already processed recently. Returns the result immediately if it did, if not:* Determines which tiles are required to compute the requested feature* Retrieves the relevant tiles from ScyllaDB* Performs final aggregations across tiles to compute the exact feature values* Returns the computed features to the calling service and caches the computed result# **Tiles optimizations**The initial database schema and tiling configuration led to scalability problems. Original schema mapped each entity into its own partition, with timestamp and feature name being ordered clustering columns.Tiles were computed for segments of one minute, 30 minutes and one day. The most popular requested aggregation ranges were 1 hour, 1 day, 7 days or 30 days, and the number of tiles required to be fetched were 70 per feature on average.If we do the math, it becomes obvious why this approach didn’t scale well. At the moment of testing, the system has around 8K rps for fetching the feed, with around 2K candidates being ranked:### **Step 1: Make db schema more “compact”**