leveldb/db
mjwiacek e84b5bdb5a This CL fixes a bug encountered when reading records from leveldb files that have been split, as in a [] input task split.
Detailed description:

Suppose an input split is generated between two leveldb record blocks and the preceding block ends with null padding.

A reader that previously read at least 1 record within the first block (before encountering the padding) upon trying to read the next record, will successfully and correctly read the next logical record from the subsequent block, but will return a last record offset pointing to the padding in the first block.

When this happened in a [], it resulted in duplicate records being handled at what appeared to be different offsets that were separated by only a few bytes.

This behavior is only observed when at least 1 record was read from the first block before encountering the padding. If the initial offset for a reader was within the padding, the correct record offset would be reported, namely the offset within the second block.

The tests failed to catch this scenario/bug, because each read test only read a single record with an initial offset. This CL adds an explicit test case for this scenario, and modifies the test structure to read all remaining records in the test case after an initial offset is specified.  Thus an initial offset that jumps to record #3, with 5 total records in the test file, will result in reading 2 records, and validating the offset of each of them in order to pass successfully.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=115338487
2016-03-31 15:53:34 -07:00
..
autocompact_test.cc LevelDB 1.13 2013-08-21 11:12:47 -07:00
builder.cc A number of fixes: 2011-10-31 17:22:06 +00:00
builder.h A number of fixes: 2011-10-31 17:22:06 +00:00
c_test.c Small fixes. 2012-10-16 16:17:53 -07:00
c.cc Small fixes. 2012-10-16 16:17:53 -07:00
corruption_test.cc LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
db_bench.cc Only compiling TrimSpace on linux. 2015-12-09 10:35:07 -08:00
db_impl.cc Merge pull request #284 from ideawu/master 2016-01-12 11:30:32 -08:00
db_impl.h LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
db_iter.cc Release LevelDB 1.14 2013-09-19 13:49:19 -07:00
db_iter.h LevelDB 1.13 2013-08-21 11:12:47 -07:00
db_test.cc Add "approximate-memory-usage" property to leveldb::DB::GetProperty 2015-12-09 10:34:58 -08:00
dbformat_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
dbformat.cc Release leveldb 1.10 2013-05-14 17:03:07 -07:00
dbformat.h Release 1.18 2014-09-16 14:19:52 -07:00
dumpfile.cc Release 1.18 2014-09-16 14:19:52 -07:00
fault_injection_test.cc LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
filename_test.cc Release LevelDB 1.14 2013-09-19 13:49:19 -07:00
filename.cc Release LevelDB 1.14 2013-09-19 13:49:19 -07:00
filename.h Release LevelDB 1.14 2013-09-19 13:49:19 -07:00
leveldbutil.cc Putting build artifacts in subdirectory. 2016-01-29 16:10:00 -08:00
log_format.h Release 1.18 2014-09-16 14:19:52 -07:00
log_reader.cc This CL fixes a bug encountered when reading records from leveldb files that have been split, as in a [] input task split. 2016-03-31 15:53:34 -07:00
log_reader.h Suppress error reporting after seeking but before a valid First or Full record is encountered. 2015-12-09 10:34:57 -08:00
log_test.cc This CL fixes a bug encountered when reading records from leveldb files that have been split, as in a [] input task split. 2016-03-31 15:53:34 -07:00
log_writer.cc LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
log_writer.h LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
memtable.cc A number of fixes: 2011-10-31 17:22:06 +00:00
memtable.h Resolve race when getting approximate-memory-usage property 2015-12-09 11:27:50 -08:00
recovery_test.cc LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
repair.cc Release 1.18 2014-09-16 14:19:52 -07:00
skiplist_test.cc Fix signed/unsigned mismatch on VC++ builds 2016-02-19 13:59:19 -08:00
skiplist.h Move header guard below copyright banner. 2014-12-11 08:04:40 -08:00
snapshot.h Clean up layering of storage/leveldb/... 2014-12-11 08:02:45 -08:00
table_cache.cc Release LevelDB 1.14 2013-09-19 13:49:19 -07:00
table_cache.h Added bloom filter support. 2012-04-17 08:36:46 -07:00
version_edit_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
version_edit.cc A number of fixes: 2011-10-31 17:22:06 +00:00
version_edit.h A number of fixes: 2011-10-31 17:22:06 +00:00
version_set_test.cc A number of fixes: 2011-10-31 17:22:06 +00:00
version_set.cc Will not reuse manifest if reuse_logs options is false. 2015-08-11 14:59:48 -07:00
version_set.h LevelDB now attempts to reuse the preceding MANIFEST and log file when re-opened. 2015-08-11 14:56:39 -07:00
write_batch_internal.h Clean up layering of storage/leveldb/... 2014-12-11 08:02:45 -08:00
write_batch_test.cc added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00
write_batch.cc added group commit; drastically speeds up mult-threaded synchronous write workloads 2012-03-08 16:23:21 -08:00