Commit Graph

49 Commits

Author SHA1 Message Date
florijan
1ee6d6d05e Make WAL recovery linear
Summary:
This is a proposal on how the WAL recovery process can be implemented so
that Deltas aren't accumulated, but instead applied in the same order
they are written to the WAL.

I *believe* that the only additional requirement on the system are
atomic transaction Begin/Commit/Abort. By atomic I mean that they are
present in the WAL in exactly the same ordering like in the transaciton
engine, to ensure the same commitability of original and recovery
transactions.

This could be a requirement for HA recovery. It is desirable that WAL
and HA log become the same thing, and the recovery process too.

Reviewers: mtomic, dgleich, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1068
2017-12-22 10:02:09 +01:00
florijan
9191800fc6 Remove tx::Transaction undefined methods
Summary: Remove two tx::Transaction methods that are not defined and never used.

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1072
2017-12-21 12:02:39 +01:00
florijan
3ae45e0d19 Add master/worker flags, main functions and coordination
Reviewers: dgleich, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1060
2017-12-19 16:05:57 +01:00
Dominik Gleich
0f1e8aa763 Add rpc server local thread
Summary: Update tests for new rpc server

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1057
2017-12-14 13:37:32 +01:00
Dominik Gleich
a8488ac497 Rpc client locking
Summary:
Rpc client wasn't thread safe and required a lock before each rpc call.
The locking functionality is now incorporated in Rpc client.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1056
2017-12-13 17:37:32 +01:00
Dominik Gleich
19e96c98b6 Generalize cereal message macro
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1053
2017-12-13 12:53:15 +01:00
florijan
4982d45e27 Add RPC to distributed tx::Engine
Reviewers: mislav.bradac, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1035
2017-12-11 11:05:01 +01:00
florijan
ce3638b25e Prepare transactional engine for distributed
Summary:
The current idea is that the same MG binary can be used for single-node,
distributed master and distributed worker. The transactional engine in
the single-node and distributed master is the same: it determines the
transactional time and exposes all the "global" functionalities. In the
distributed worker the "global" functions must contact the master.

Reviewers: dgleich, mislav.bradac, buda

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1013
2017-12-01 09:17:44 +01:00
Dominik Gleich
9c24faa4d6 Expose faster edge_accessor constructor
Summary:
Fix nullptr exceptions

Inline edge_type and from/to in edge accessor

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1010
2017-11-29 12:40:11 +01:00
florijan
3bd5a882cf Remove Update and Reconstruct from GraphDbAccessor
Summary: Code simplification made possible by making `locks_` `mutable` in `tx::Transaction`.

Reviewers: dgleich, buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1015
2017-11-29 10:02:01 +01:00
Dominik Gleich
a7f9255c17 Remove redundant transaction from index creation
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1011
2017-11-27 16:47:12 +01:00
florijan
23241e76b9 Prepare tx::Engine for distributed
Summary:
This diff contains step 1:
- Remove clog exposure from tx::engine
- Reduce and cleanup tx::Engine API

All current functionality is kept, but the API is reduced. This is very
desirable because every function in tx::Engine will need to be
considered and implemented in both Master and Worker situations. The
less we have, the better.

Next step is exactly that: seeing how each of these functions behaves in
a distributed system and implementing accordingly.

Reviewers: dgleich, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1008
2017-11-24 14:51:29 +01:00
Marko Budiselic
3c9e143c95 Fix SegFault within executor state
Reviewers: mislav.bradac, dgleich, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D995
2017-11-20 11:27:08 +01:00
florijan
1e0ac8ab8f Write-ahead log
Summary:
My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed...

What's done:
- Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr.
- WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk.
- Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info.
- Durability testing completely reworked.

What's not done (and should be when we decide how):
- Old WAL file purging.
- Config refactor (naming and organization). Will do when we discuss what we want.
- Changelog and new feature documentation (both depending on the point above).
- Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment).
- Moving the implementation of WAL stuff to .cpp where possible.
- Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into.
- True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature.

Reviewers: buda, mislav.bradac, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D958
2017-11-13 09:51:39 +01:00
Dominik Gleich
f50d87f736 Take care of warnings/errors created by cppcheck.
Summary:
Warnings I ignored:

Creates a new stacktrace object and then dumps it
	102.570384 (102.495075) E[1]: [src/utils/exceptions.hpp:116]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.
	102.570390 (102.495081) E[1]: [src/utils/exceptions.hpp:127]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.

Used all over the codebase without explicit cast
102.570412 (102.495103) E[1]: [src/utils/stacktrace.hpp:14]: (style) Class 'Line' has a constructor with 1 argument that is not explicit.

Not really used anywhere before initialized:
102.570526 (102.495217) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::preds' is not initialized in the constructor.
102.570530 (102.495221) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::succs' is not initialized in the constructor.

Implicit conversions between types are used all over the codebase:
102.570548 (102.495239) E[1]: [src/storage/property_value.hpp:41]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570552 (102.495243) E[1]: [src/storage/property_value.hpp:42]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570557 (102.495248) E[1]: [src/storage/property_value.hpp:43]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570561 (102.495252) E[1]: [src/storage/property_value.hpp:44]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570566 (102.495257) E[1]: [src/storage/property_value.hpp:47]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570570 (102.495261) E[1]: [src/storage/property_value.hpp:50]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570578 (102.495269) E[1]: [src/storage/property_value.hpp:53]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570582 (102.495273) E[1]: [src/storage/property_value.hpp:57]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570591 (102.495282) E[1]: [src/query/typed_value.hpp:80]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570596 (102.495287) E[1]: [src/query/typed_value.hpp:81]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570601 (102.495292) E[1]: [src/query/typed_value.hpp:82]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570605 (102.495296) E[1]: [src/query/typed_value.hpp:83]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570609 (102.495300) E[1]: [src/query/typed_value.hpp:89]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570614 (102.495305) E[1]: [src/query/typed_value.hpp:92]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570618 (102.495309) E[1]: [src/query/typed_value.hpp:95]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570623 (102.495314) E[1]: [src/query/typed_value.hpp:98]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570627 (102.495318) E[1]: [src/query/typed_value.hpp:102]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570632 (102.495323) E[1]: [src/query/typed_value.hpp:105]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570636 (102.495327) E[1]: [src/query/typed_value.hpp:108]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570641 (102.495332) E[1]: [src/query/typed_value.hpp:109]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570645 (102.495336) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:88]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570650 (102.495341) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:89]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570654 (102.495345) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:90]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570659 (102.495350) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:91]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570663 (102.495354) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:94]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570668 (102.495359) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:97]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570672 (102.495363) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:100]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570677 (102.495368) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:104]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570681 (102.495372) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:107]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570690 (102.495381) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:110]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570694 (102.495385) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:113]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.

CypherParser:

102.570767 (102.495458) E[1]: [src/query/frontend/opencypher/generated/CypherParser.h:69]: (style) Class 'CypherParser' has a constructor with 1 argument that is not explicit.
102.570772 (102.495463) E[1]: [src/query/frontend/opencypher/generated/CypherLexer.h:40]: (style) Class 'CypherLexer' has a constructor with 1 argument that is not explicit.
102.570776 (102.495467) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:86]: (style) The scope of the variable '_la' can be reduced.
102.570781 (102.495472) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:311]: (style) The scope of the variable '_la' can be reduced.
102.570785 (102.495476) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:402]: (style) The scope of the variable '_la' can be reduced.
102.570789 (102.495480) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:497]: (style) The scope of the variable '_la' can be reduced.
102.570797 (102.495488) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:778]: (style) The scope of the variable '_la' can be reduced.
102.570802 (102.495493) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:895]: (style) The scope of the variable '_la' can be reduced.
102.570806 (102.495497) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:991]: (style) The scope of the variable '_la' can be reduced.
102.570811 (102.495502) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1190]: (style) The scope of the variable '_la' can be reduced.
102.570815 (102.495506) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1274]: (style) The scope of the variable '_la' can be reduced.
102.570820 (102.495511) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1393]: (style) The scope of the variable '_la' can be reduced.
102.570824 (102.495515) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1570]: (style) The scope of the variable '_la' can be reduced.
102.570829 (102.495520) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1695]: (style) The scope of the variable '_la' can be reduced.
102.570834 (102.495525) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1800]: (style) The scope of the variable '_la' can be reduced.
102.570839 (102.495530) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1903]: (style) The scope of the variable '_la' can be reduced.
102.570843 (102.495534) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2019]: (style) The scope of the variable '_la' can be reduced.
102.570848 (102.495539) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2228]: (style) The scope of the variable '_la' can be reduced.
102.570852 (102.495543) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2542]: (style) The scope of the variable '_la' can be reduced.
102.570857 (102.495548) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2797]: (style) The scope of the variable '_la' can be reduced.
102.570861 (102.495552) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2966]: (style) The scope of the variable '_la' can be reduced.
102.570866 (102.495557) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3067]: (style) The scope of the variable '_la' can be reduced.
102.570870 (102.495561) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3289]: (style) The scope of the variable '_la' can be reduced.
102.570875 (102.495566) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3295]: (style) The scope of the variable 'alt' can be reduced.
102.570879 (102.495570) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3419]: (style) The scope of the variable '_la' can be reduced.
102.570884 (102.495575) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3596]: (style) The scope of the variable '_la' can be reduced.
102.570888 (102.495579) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3688]: (style) The scope of the variable '_la' can be reduced.
102.570893 (102.495584) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3963]: (style) The scope of the variable '_la' can be reduced.
102.570897 (102.495588) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4452]: (style) The scope of the variable '_la' can be reduced.
102.570902 (102.495593) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4586]: (style) The scope of the variable '_la' can be reduced.
102.570906 (102.495597) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4813]: (style) The scope of the variable '_la' can be reduced.
102.570911 (102.495602) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4943]: (style) The scope of the variable '_la' can be reduced.
102.570918 (102.495609) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5026]: (style) The scope of the variable '_la' can be reduced.
102.570923 (102.495614) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5569]: (style) The scope of the variable '_la' can be reduced.
102.570928 (102.495619) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5664]: (style) The scope of the variable '_la' can be reduced.
102.570932 (102.495623) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5755]: (style) The scope of the variable '_la' can be reduced.
102.570937 (102.495628) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5888]: (style) The scope of the variable '_la' can be reduced.
102.570941 (102.495632) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6045]: (style) The scope of the variable '_la' can be reduced.
102.570946 (102.495637) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6142]: (style) The scope of the variable '_la' can be reduced.
102.570950 (102.495641) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6347]: (style) The scope of the variable '_la' can be reduced.
102.570955 (102.495646) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6523]: (style) The scope of the variable '_la' can be reduced.
102.570959 (102.495650) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6614]: (style) The scope of the variable '_la' can be reduced.
102.570964 (102.495655) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6899]: (style) The scope of the variable '_la' can be reduced.
102.570968 (102.495659) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6992]: (style) The scope of the variable '_la' can be reduced.
102.570973 (102.495664) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7147]: (style) The scope of the variable '_la' can be reduced.
102.570977 (102.495668) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7680]: (style) The scope of the variable '_la' can be reduced.
102.570982 (102.495673) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7759]: (style) The scope of the variable '_la' can be reduced.
102.570986 (102.495677) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7938]: (style) The scope of the variable '_la' can be reduced.
102.570991 (102.495682) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8126]: (style) The scope of the variable '_la' can be reduced.
102.570995 (102.495686) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8220]: (style) The scope of the variable '_la' can be reduced.
102.571000 (102.495691) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8313]: (style) The scope of the variable '_la' can be reduced.
102.571004 (102.495695) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8491]: (style) The scope of the variable '_la' can be reduced.
102.571009 (102.495700) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8703]: (style) The scope of the variable '_la' can be reduced.
102.571013 (102.495704) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8783]: (style) The scope of the variable '_la' can be reduced.
102.571018 (102.495709) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8914]: (style) The scope of the variable '_la' can be reduced.
102.571022 (102.495713) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9119]: (style) The scope of the variable '_la' can be reduced.
102.571027 (102.495718) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9220]: (style) The scope of the variable '_la' can be reduced.
102.571034 (102.495725) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9414]: (style) The scope of the variable '_la' can be reduced.
102.571039 (102.495730) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9660]: (style) The scope of the variable '_la' can be reduced.
102.571043 (102.495734) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10008]: (style) The scope of the variable '_la' can be reduced.
102.571048 (102.495739) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10158]: (style) The scope of the variable '_la' can be reduced.
102.571052 (102.495743) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10250]: (style) The scope of the variable '_la' can be reduced.
102.571057 (102.495748) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10370]: (style) The scope of the variable '_la' can be reduced.
102.571061 (102.495752) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10637]: (style) The scope of the variable '_la' can be reduced.
102.571065 (102.495756) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10749]: (style) The scope of the variable '_la' can be reduced.
102.571070 (102.495761) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10815]: (style) The scope of the variable '_la' can be reduced.
102.571075 (102.495766) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10881]: (style) The scope of the variable '_la' can be reduced.

We know that we represented it correctly in memory:
102.571079 (102.495770) E[1]: [src/communication/bolt/v1/decoder/decoder.hpp:252]: (portability) Casting between integer* and double* which have an incompatible binary data representation.

Cont assigned but not used after:
102.571101 (102.495792) E[1]: [src/query/frontend/ast/ast.hpp:1008]: (style) Variable 'cont' is assigned a value that is never used.

Reviewers: teon.banek, buda, mferencevic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D967
2017-11-10 10:28:02 +01:00
florijan
fc240fbb67 Transaction command_id type reduced from 8B to 2B
Summary: This reduces the size of a single `mvcc::Record<T>` from 40B to 24B.

Reviewers: buda, mislav.bradac, dgleich

Reviewed By: mislav.bradac, dgleich

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D920
2017-10-25 10:42:57 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Mislav Bradac
11b2b83c96 Make gc more aggressive and optimise is_not_visible function
Reviewers: florijan, mferencevic

Reviewed By: florijan

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D849
2017-10-06 10:07:55 +02:00
Mislav Bradac
7dedb88449 Minor cleanup
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D840
2017-09-27 17:25:00 +02:00
Mislav Bradac
3140f175fc Implement record lock deadlock breaker
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D823
2017-09-27 16:11:05 +02:00
florijan
a9381df09e Edges data structure now supports multiple edge filtering (implicit OR)
Summary: - modified all utils/algorithm functions to be inline and in the utils namespace

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D830
2017-09-26 13:46:18 +02:00
Mislav Bradac
0eceefb2d4 Newer bits on head chunk in DynamicBitset
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D813
2017-09-20 16:56:58 +02:00
Mislav Bradac
404ffdc4ea Fix bug in lock store
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D594
2017-07-29 17:48:34 +02:00
Mislav Bradac
1c6ef47994 Fix bug in transactions/engine
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D560
2017-07-17 14:36:14 +02:00
Mislav Bradac
50cd53e5c9 Migrate timer to use walltime, instead of cputime
Summary:
 - Move some stuff to poc
 - Use steady_clock instead of system_clock

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D555
2017-07-17 13:42:44 +02:00
Mislav Bradac
4a500b99fa Add a way to stop query execution
Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D552
2017-07-14 14:36:44 +02:00
Mislav Bradac
5c810b6579 Make some members private in CommitLog
Reviewers: buda, florijan

Reviewed By: buda, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D544
2017-07-12 14:58:34 +02:00
Mislav Bradac
cbe6648eb8 Migrate command line args to gflgs in tests
Summary: Some other minor cleanups

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D523
2017-07-06 13:54:12 +02:00
florijan
9e6260d0c3 tx::Engine - commit log encapsulation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D460
2017-06-13 12:32:48 +02:00
florijan
afbb940a05 GC bugfixes, MVCC and transaction refactoring
Summary:
- GC changed to evaluate old records w.r.t. the oldest transaction's id AND snapshot, as opposed to only id
- MVCC hints exp+aborted race condition prevented
- minor MVCC refactors and cleanups
- minor Transaction refactors and cleanups

Reviewers: buda, dgleich

Reviewed By: buda, dgleich

Subscribers: dtomicevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D434
2017-06-12 10:46:12 +02:00
Dominik Gleich
842901ecd2 LabelProperty index.
Summary:
Add return values.

After merge.

Inital working version. Still missing comments.

Update documentation.

Add checking for previous vlist and value equality.

After merge.

Remove functor, add boolean ffunction.

Build index.

More functionality. Start implementing tests.

Add tests.

Reviewers: matej.gradicek, mislav.bradac, mferencevic, buda, florijan

Reviewed By: mislav.bradac, buda, florijan

Subscribers: lion, florijan, teon.banek, buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D355
2017-05-16 16:33:20 +02:00
Florijan Stamenkovic
1333bfeb39 Mvcc - unit test infrastructure setup
Summary:
Gradicek's Mvcc test have seen the following changes:
- provided a test infrastructure (fixture and macros) to facilitate testing and increase readability
- split tests into multi-transaction update, VersionList::find and general Mvcc testing
- multi-transaction update tests have been refactored (i *think* nothing got deleted, but it was a mess so I don't guarantee)
- changed all multithreaded tests to be single-threaded because multiple threads were not necessary
- changed transaction naming from T5, T8, T10 to T1, T2... for consistency with actual transaction indices

What still needs to be done:
- Gleich and Gradicek need to review the infrastructure (possible improvements)
- multi-transaction update tests need to be addressed by Gradicek (see "TODO gradicek" in code, discuss with Flor)
- the wiki/draw.io documentation needs to be updated. it is not imperative that all the tests be drawn in draw.io, only the general infrastructure explained. perhaps only a few examples drawn. Gradicek discuss with Flor
- Gleich see the "TODO Gleich" lines in the diff and discuss with flor

Suggested workflow:
- review this diff, hopefully land (before resolving all the TODOs)
- discard D169
- Gradicek and Gleich address the TODOs
- Flor reviews the results (in following diffs)

Reviewers: dgleich, matej.gradicek, buda

Reviewed By: matej.gradicek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D348
2017-05-05 14:46:18 +02:00
Teon Banek
3b9d13f1e1 Fix errors introduced in D298
Reviewers: florijan, mislav.bradac, mferencevic, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D326
2017-04-28 11:11:32 +02:00
Marko Budiselic
e6191eadb4 The rest of core engine exceptions are extended from the BasicException (because of the error handling in the communication stack). Maximum value of the MVCC command id is calculated in the compile time.
Summary:
The rest of core engine exceptions are extended from the BasicException (because of the error handling in the communication stack). Maximum value of the MVCC command id is calculated in the compile time.

I've put the tasks to implement concurrent tests in the qa repo in the backlog. Hopefully they will be implemented within this sprint.

Reviewers: mislav.bradac, mferencevic, florijan

Reviewed By: mislav.bradac

Subscribers: dgleich, pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D298
2017-04-27 13:17:40 -04:00
Dominik Gleich
e8fd479bbc Fix memory leak. Refactor mvcc. Revert commit function to old and introduce new one.
Summary:
Update tests.

Documentation:
https://phabricator.memgraph.io/w/memgraph_implementation/mvcc/

Reviewers: mislav.bradac, florijan, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D227
2017-04-10 12:16:03 +02:00
Dominik Gleich
087c56315e Fix engine.
Summary:
Test engine update.

https://docs.google.com/document/d/1rnFsCFock-K3ZbuvMiCYWhGS_m7-DZV37JtxWRClD_8/edit

Reviewers: mislav.bradac, dtomicevic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D167
2017-03-24 12:37:01 +01:00
Dominik Gleich
f505c76189 Fix lock store bug.
Summary: Lock store was not locking properly. It created a lock object which was destructed in the end of function scope which caused bits to be set to UNLOCKED.

Reviewers: matej.gradicek, buda

Reviewed By: matej.gradicek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D154
2017-03-22 15:40:37 +01:00
Dominik Gleich
57f1b40d8d Refactor transaction inside db_accessor. Also introduce advance_command.
Summary:
Modification to MVCC - CHECK!

Refactor transaction_read

Add automatic commit/abort functionality to graph_db_accessor.

Fix failing test in graph_db_accessor.

Reviewers: buda, dtomicevic, florijan

Reviewed By: buda, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D87
2017-03-07 09:42:04 +01:00
Dominik Gleich
82414b9111 Fix naming of asserts.runtime_assert & assert -> debug_assert
Summary:
Merge branch 'dev' into rename_assert

Fix new files.

Remove cassert.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D66
2017-02-24 11:41:55 +01:00
Marko Budiselic
d4e3c4cd10 clang format has been run on all hpp and cpp files under src and tests 2017-02-18 11:54:37 +01:00
Marko Budiselic
e7f5bd4c21 antlr integration, *.hpp and *.cpp inside src dir, cleanup
Summary: antlr integration, *.hpp and *.cpp inside src dir, cleanup

Test Plan: manual

Reviewers: mislav.bradac, dgleich, florijan

Reviewed By: florijan

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D49
2017-02-17 16:20:31 +01:00
Kruno Tomola Fabro
2a59ed8906 Minor refactorings:
Importes now use logger.

Refactored order of constrution of objects in Db.

Moved index creation/removing from Db to Indexes.

Completed Garbage class.
Cleaner now calls garbage.clean() for databases.

Renamed List to ConcurrentList which better names it.
2016-09-12 20:13:04 +01:00
Kruno Tomola Fabro
443d0e0098 Changed TransactionId to TransactionRead. 2016-09-09 20:56:53 +01:00
Kruno Tomola Fabro
4443cb8b7b Finished creating part of snapshot. 2016-09-08 13:25:52 +01:00
Kruno Tomola Fabro
b2ce3d58a4 Added cleaner.
Added multithreading to dbms.
Skiplist now supports emplace insert.
2016-08-30 00:45:07 +01:00
Kruno Tomola Fabro
8a89f6601d EdgeType indexes added.
Implemented untested UniqueOrderedIndex.

Introduced TypeGroupEdge/Vertex into database.

Added Index capabilityes to PropertyFamily.
Added method for adding index.
Added method for removing index.
2016-08-25 15:29:45 +01:00
Marko Budiselic
869da8dcda All files for the Release are now isolated 2016-08-10 09:39:02 +01:00
Marko Budiselic
b94cae12d1 some new header + source files (vertices, vertex_accessor, properties), first version of Index works, skiplist<T> + concurrent_set + concurrent_map (still has to be finished) 2016-07-05 04:01:22 +01:00
Marko Budiselic
52625e39a3 The folder structure was changed, all source code is now within the src dir. 2016-05-08 11:33:07 +02:00