1e0ac8ab8f
Summary: My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed... What's done: - Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr. - WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk. - Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info. - Durability testing completely reworked. What's not done (and should be when we decide how): - Old WAL file purging. - Config refactor (naming and organization). Will do when we discuss what we want. - Changelog and new feature documentation (both depending on the point above). - Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment). - Moving the implementation of WAL stuff to .cpp where possible. - Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into. - True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature. Reviewers: buda, mislav.bradac, teon.banek, dgleich Reviewed By: dgleich Subscribers: mtomic, pullbot Differential Revision: https://phabricator.memgraph.io/D958
36 lines
1.5 KiB
CMake
36 lines
1.5 KiB
CMake
include_directories(SYSTEM ${GTEST_INCLUDE_DIR})
|
|
|
|
add_executable(mg_recovery_check
|
|
mg_recovery_check.cpp
|
|
${memgraph_src_dir}/communication/bolt/v1/decoder/decoded_value.cpp
|
|
${memgraph_src_dir}/data_structures/concurrent/skiplist_gc.cpp
|
|
${memgraph_src_dir}/database/graph_db.cpp
|
|
${memgraph_src_dir}/database/graph_db_accessor.cpp
|
|
${memgraph_src_dir}/durability/recovery.cpp
|
|
${memgraph_src_dir}/durability/snapshooter.cpp
|
|
${memgraph_src_dir}/durability/wal.cpp
|
|
${memgraph_src_dir}/query/typed_value.cpp
|
|
${memgraph_src_dir}/storage/edge_accessor.cpp
|
|
${memgraph_src_dir}/storage/locking/record_lock.cpp
|
|
${memgraph_src_dir}/storage/property_value.cpp
|
|
${memgraph_src_dir}/storage/record_accessor.cpp
|
|
${memgraph_src_dir}/storage/vertex_accessor.cpp
|
|
${memgraph_src_dir}/transactions/transaction.cpp
|
|
)
|
|
|
|
target_link_libraries(mg_recovery_check stdc++fs Threads::Threads fmt glog
|
|
gflags cppitertools)
|
|
target_link_libraries(mg_recovery_check gtest gtest_main)
|
|
|
|
# Copy CSV data to CMake build dir
|
|
configure_file(csv/comment_nodes.csv csv/comment_nodes.csv COPYONLY)
|
|
configure_file(csv/forum_nodes.csv csv/forum_nodes.csv COPYONLY)
|
|
configure_file(csv/relationships.csv csv/relationships.csv COPYONLY)
|
|
# Copy the actual runner to CMake build dir
|
|
configure_file(test_mg_import_csv test_mg_import_csv COPYONLY)
|
|
|
|
add_test(NAME test_mg_import_csv
|
|
COMMAND test_mg_import_csv
|
|
--mg-import-csv ../src/mg_import_csv
|
|
--mg-recovery-check ./mg_recovery_check)
|