add cache_index_and_filter_blocks

[lhztop]

add cache_index_and_filter_blocks options, which is import when bloom filter

[twmht]

@lhztop

thanks for the PR.

it's great.

Can you add some test cases as well?

[lhztop]

Ok. now it's in test_options.py

[gcp]

Any reason this hasn't been pulled in?

Looks like it also makes it possible to expose bottommost_compression which would be nice to be able to configure (FB recommends LZ4 top, ZSTD bottom).

[twmht]

@gcp

yup. some test cases failed, I am going to verify.

[oersted]

Any news on this? According to facebook/rocksdb#3216 this might be a solution to a memory leak issue I have.

[twmht]

@oersted

yup. I am currently migrating all of the existing features to pybind11 branch.

[twmht]

@oersted

so if we set cache_index_and_filter_blocks to true, then it can solve the memory leak problem?

[oersted]

@twmht Actually it looks like it was an unrelated garbage collection issue, I was creating too many new small objects in my merge operation that weren't getting collected in time. Thanks anyways.

[iFA88]

Take some attention to https://github.com/facebook/rocksdb/blob/6.3.fb/include/rocksdb/table.h#L61-L71

I have several databases with different sizes: 30gb 250gb 400gb 1.2tb. I doesn't have any memory issue.

[twmht]

@oersted

I will add merge operations to pybind11 branch in a few days. after that You can try the branch to test if it has memory leak.