Homec4science

Implement a more compact, general database-backed key-value cache

Authored by epriestley <git@epriestley.com> on Dec 21 2012, 23:17.

Description

Implement a more compact, general database-backed key-value cache

Summary:
See discussion in D4204. Facebook currently has a 314MB remarkup cache with a 55MB index, which is slow to access. Under the theory that this is an index size/quality problem (the current index is on a potentially-384-byte field, with many keys sharing prefixes), provide a more general index with fancy new features:

  • It implements PhutilKeyValueCache, so it can be a component in cache stacks and supports TTL.
  • It has a 12-byte hash-based key.
  • It automatically compresses large blocks of data (most of what we store is highly-compressible HTML).

Test Plan:

  • Basics:
    • Loaded /paste/, saw caches generate and save.
    • Reloaded /paste/, saw the page hit cache.
  • GC:
    • Ran GC daemon, saw nothing.
    • Set maximum lifetime to 1 second, ran GC daemon, saw it collect the entire cache.
  • Deflate:
    • Selected row formats from the database, saw a mixture of 'raw' and 'deflate' storage.
    • Used profiler to verify that 'deflate' is fast (12 calls @ 220us on my paste list).
  • Ran unit tests

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4259

Details

Committed
epriestley <git@epriestley.com>Dec 21 2012, 23:17
Pushed
aubortJan 31 2017, 17:16
Parents
rPH62bc3373e5a6: Fix error with inline comments on images
Branches
Unknown
Tags
Unknown

Event Timeline

epriestley <git@epriestley.com> committed rPHaae5f9efd3d3: Implement a more compact, general database-backed key-value cache (authored by epriestley <git@epriestley.com>).Dec 21 2012, 23:17