Commit Graph

1675 Commits

Author SHA1 Message Date
florijan
87f8c4e256 Fix durability test flakyness (hopefully)
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1098
2018-01-10 14:24:40 +01:00
florijan
3cf81380b8 Implement vertex/edge serialization
Summary:
Serialization of vertices and edges for distributed. Based on Boost
serialization. Threrefore moved TypedValue serialization from AST to
utils.

Reviewers: buda, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1088
2018-01-10 13:32:18 +01:00
florijan
4a0345e1c5 Prepare counter for distributed
Reviewers: dgleich, teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D1090
2018-01-10 13:28:41 +01:00
Marko Budiselic
b736c37199 Add customers folder at the Doxygen exclude patterns
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D1094
2018-01-08 13:24:37 +00:00
Marko Culinovic
7a06040a76 Setup RAFT test infrastructure
Summary:
`JailService` exposes jail API and knows how to
start and stop binaries. Every test should be
defined as python module with exposed
`run` method. Script `master.py` is used
for running tests and takes test module
name as argument. Machine IP addresses are
defined in environment variables. To run test
locally use `local_runner` script.

Reviewers: mislav.bradac, mferencevic, mtomic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1065
2018-01-05 09:59:42 +01:00
florijan
99000c6ec1 Remove PropertyValueStore templatization
Summary:
A PropertyValueStore is not a generic data structure, but only ever used
to store properties in a Vertex/Edge. It has behaviours specific to it.
So, the templatization was not necessary.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1089
2018-01-02 16:18:55 +01:00
Matej Ferencevic
82bede9a97 Implement measurements collection in tests
Summary:
Implement measurements collection in QA

Implement measurements collection in LDBC

Implement measurements collection in stress tests

Reviewers: mislav.bradac, buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1087
2018-01-01 17:49:37 +01:00
Matej Ferencevic
859641cb0c Changed macro benchmark summary format
Reviewers: buda, mtomic, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: florijan, pullbot

Differential Revision: https://phabricator.memgraph.io/D972
2017-12-28 16:35:45 +01:00
Dominik Gleich
503381549e Change gid bit size
Summary:
Change gid methods

Rename GidGenerator and add tests

Fix tools broken by gid changes

Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1044
2017-12-28 11:04:52 +01:00
Mislav Bradac
8952df06c1 Temporary workaround for raft network problem
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1085
2017-12-27 15:16:14 +01:00
Mislav Bradac
d3623585e7 Migrate cereal to boost_serialization
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1077
2017-12-27 13:44:46 +01:00
florijan
7b3d298741 Fix interpreter plan ownership
Summary:
If there was no plan caching, the CachedPlan would not survive
`Interpreter::operator()`, as it was not owned by the
`Interpreter::Result`. If there was caching, it could hapen that the
cache got invalidated while that plan was being interpreted (by
another thread) without that interpretation retaining ownership.

Also simplified code around this.

Reviewers: mislav.bradac, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1084
2017-12-27 13:33:13 +01:00
Teon Banek
8526e79619 Remove Union from Upcoming Features
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1083
2017-12-22 15:24:05 +01:00
florijan
9b878c91eb Make interpretation pullable
Reviewers: mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1079
2017-12-22 15:19:05 +01:00
Dominik Gleich
caaab41108 Add new snapshot format support
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1082
2017-12-22 15:16:53 +01:00
Dominik Gleich
b094e7473b Add virtual destructor
Summary:
Not having a virtual destructor caused tests
to fail (cypher_main_visitor, interpreter) sporadically
since unfreed memory was re-used incorrectly.
Also Valgrind complained constantly.

Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1081
2017-12-22 15:02:57 +01:00
florijan
1ee6d6d05e Make WAL recovery linear
Summary:
This is a proposal on how the WAL recovery process can be implemented so
that Deltas aren't accumulated, but instead applied in the same order
they are written to the WAL.

I *believe* that the only additional requirement on the system are
atomic transaction Begin/Commit/Abort. By atomic I mean that they are
present in the WAL in exactly the same ordering like in the transaciton
engine, to ensure the same commitability of original and recovery
transactions.

This could be a requirement for HA recovery. It is desirable that WAL
and HA log become the same thing, and the recovery process too.

Reviewers: mtomic, dgleich, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1068
2017-12-22 10:02:09 +01:00
Teon Banek
9cacf970cc Install memgraph in subdirectory of lib
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1076
2017-12-21 16:53:55 +01:00
Marin Tomic
6861a580c7 Add #pragma once to cereal_optional.hpp
Reviewers: mferencevic

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D1080
2017-12-21 16:50:13 +01:00
Teon Banek
6294bf19ec Add documentation for mg_import_csv
Reviewers: mtomic, buda, florijan, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1073
2017-12-21 14:19:30 +01:00
Teon Banek
86b6f32ce6 Move load_construct_data from ast.cpp to ast.hpp
Reviewers: florijan, mislav.bradac, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1078
2017-12-21 13:08:16 +01:00
florijan
9191800fc6 Remove tx::Transaction undefined methods
Summary: Remove two tx::Transaction methods that are not defined and never used.

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1072
2017-12-21 12:02:39 +01:00
Marin Tomic
e999207b2f Remove reactors
Summary: kill reactors

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1075
2017-12-20 17:56:57 +01:00
Teon Banek
6d4113d9db Require minimum 1.62 version of Boost
Reviewers: mislav.bradac, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1070
2017-12-20 17:46:37 +01:00
Matej Ferencevic
14bbd726a6 Increase build './setup' timeout
Reviewers: teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1069
2017-12-20 17:15:02 +01:00
Dominik Gleich
1556d78d15 Update snapshot format
Summary:
Set vertex/edge generator id from recovery

Add tests

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1061
2017-12-20 16:57:42 +01:00
Marin Tomic
20aa8d563a Implement RaftNetworkInterface via RPC
Summary:
Add missing fields in RequestVote RPC

Unnest Raft RPC requests and responses

Make RPC request and reply serializable

Refactor Raft network interface

Implement RPC retrying with backoff period

Add cerealization functions for optional

Add RPC network interface for raft

Add some comments and documentation

Add manual test for RPC network interface

Remove raft reactor experiments

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot, mculinovic

Differential Revision: https://phabricator.memgraph.io/D1066
2017-12-20 16:48:46 +01:00
florijan
312016ca31 Support both locally and system installed libboost
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1071
2017-12-20 16:13:03 +01:00
Teon Banek
5f7837d613 Serialize Ast classes
Summary:
Although the first solution used cereal, the final implementation uses
boost. Since the cereal is still used in the codebase, compilation has
been modified to support multithreaded cereal.

In addition to serializing Ast classes, the following also needed to be
serialized:

  * GraphDbTypes
  * Symbol
  * TypedValue

TypedValue is treated specially, by inlining the serialization code in
the Ast class, concretely PrimitiveLiteral.

Another special case was the Function Ast class, which now stores a
function name which is resolved to a concrete std::function on
construction.

Tests have been added for serialized Ast in
tests/unit/cypher_main_visitor

Reviewers: mferencevic, mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1067
2017-12-20 13:28:34 +01:00
florijan
3ae45e0d19 Add master/worker flags, main functions and coordination
Reviewers: dgleich, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1060
2017-12-19 16:05:57 +01:00
florijan
37722e54b3 Add RPC to concurrent ID mapper
Summary:
The distributed ID mapper is not yet utilised in GraphDb as those
changes are in D1060. Depending on landing order it will be added.

Reviewers: dgleich, mislav.bradac

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1064
2017-12-19 09:11:16 +01:00
Dominik Gleich
0eb30db8db Remove executor not run option
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1062
2017-12-18 14:36:15 +01:00
Teon Banek
f0469ba634 Bump Memgraph version to 0.9
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D1063
2017-12-18 10:37:31 +01:00
Marin Tomic
5b7cb4602c Add simple client facing API for RaftMember
Summary:
Implement AddComand method on RaftMember

Move RPCType out of rpc request and reply

Add unit test for AddCommand

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot, mculinovic

Differential Revision: https://phabricator.memgraph.io/D1042
2017-12-18 07:32:39 +01:00
Matej Ferencevic
49bea9b0d0 Report clang-format errors
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1059
2017-12-15 11:52:19 +01:00
Matej Ferencevic
a355bf4e77 Remove obsolete scripts
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1047
2017-12-15 10:48:38 +01:00
Dominik Gleich
47dab02c2d Update changelog
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1058
2017-12-15 10:36:32 +01:00
florijan
b42169a0cb Fix RPC pimp iwyu
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1055
2017-12-14 16:34:07 +01:00
Marko Culinovic
0cc1ac92b0 Add TED Talk example to quick start
Reviewers: teon.banek, florijan, mtomic, buda

Reviewed By: teon.banek, buda

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1022
2017-12-14 13:48:07 +01:00
Dominik Gleich
0f1e8aa763 Add rpc server local thread
Summary: Update tests for new rpc server

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1057
2017-12-14 13:37:32 +01:00
florijan
52b3941c8a Fix tx::WorkerEngine flakyness. Change Endpoint::port type.
Reviewers: mislav.bradac, dgleich

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1054
2017-12-14 10:36:24 +01:00
Matija Santl
2758b35c08 Fix semicolon in column name
Summary:
Special casing the last token when adding it to the named expression list.
Added a test to check this case.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1052
2017-12-14 09:44:21 +01:00
Dominik Gleich
a8488ac497 Rpc client locking
Summary:
Rpc client wasn't thread safe and required a lock before each rpc call.
The locking functionality is now incorporated in Rpc client.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1056
2017-12-13 17:37:32 +01:00
Dominik Gleich
581404a7b6 Make index creation fully parallel
Summary:
Testing on following operations:

UNWIND RANGE(0,1000000) AS _ CREATE (n: Node {a: 1.0, b:2.0});
CREATE INDEX ON :Node(a);
CREATE INDEX ON :Node(b);

Before:
~32sec

After:
~18sec

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1041
2017-12-13 16:50:56 +01:00
Matija Santl
4da8312abe Fix query returning a map
Summary: Add curvy braces handling in `QueryStripper` and an accompanying test for it.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1051
2017-12-13 15:38:20 +01:00
Mislav Bradac
27370b99ff Fix Pattern::Accept
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1048
2017-12-13 13:15:22 +01:00
Mislav Bradac
cf7d9070d8 Fix flakyness in rpc/messaging tests
Summary: In this diff I just wanted to fix tests' flakyness. We can discuss if we want to always pass endpoint as an argument and never pass address:port pair explicitly. However if we decide that, I will do that change in another diff.

Reviewers: dgleich, florijan

Reviewed By: dgleich, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1049
2017-12-13 13:15:02 +01:00
Dominik Gleich
19e96c98b6 Generalize cereal message macro
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1053
2017-12-13 12:53:15 +01:00
Matija Santl
9ad7d82a54 Implement UNION query combinator
Summary:
Union query combinator implementation consists of:
 * adjustments to the AST and `cypher_main_visitor`
 * enabling `QueryStripper` to parse multiple `return` statements (not stopping after first)
 * symbol generation for union results
 * union logical operator
 * query plan generator adjustments

Reviewers: teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D1038
2017-12-13 09:54:00 +01:00
Matej Ferencevic
dae79413bf Increase cppcheck timeout
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1046
2017-12-12 20:14:30 +01:00