Commit Graph

917 Commits

Author SHA1 Message Date
florijan
c4327b26f4 Extract tx::SingleNodeEngine from tx::MasterEngine
Summary:
No logic changes, just split `tx::MasterEngine` into
`tx::SingleNodeEngine` and `tx::MasterEngine`. This gives better
responsibility separation and is more appropriate now there is no
Start/Shutdown.

Reviewers: dgleich, teon.banek, buda

Reviewed By: dgleich, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1099
2018-01-11 15:44:42 +01:00
Dominik Gleich
007a7f1a6d Change network design from Start/Shutdown to constructor/destructor
Summary:
Make ServerT start on constructor

Remove shutdown from MasterCoordinator

Distributed system remove Shutdown

Rcp server start and shutdown removed

Reviewers: florijan, mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1097
2018-01-10 14:58:57 +01:00
florijan
3cf81380b8 Implement vertex/edge serialization
Summary:
Serialization of vertices and edges for distributed. Based on Boost
serialization. Threrefore moved TypedValue serialization from AST to
utils.

Reviewers: buda, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1088
2018-01-10 13:32:18 +01:00
florijan
4a0345e1c5 Prepare counter for distributed
Reviewers: dgleich, teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D1090
2018-01-10 13:28:41 +01:00
florijan
99000c6ec1 Remove PropertyValueStore templatization
Summary:
A PropertyValueStore is not a generic data structure, but only ever used
to store properties in a Vertex/Edge. It has behaviours specific to it.
So, the templatization was not necessary.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1089
2018-01-02 16:18:55 +01:00
Dominik Gleich
503381549e Change gid bit size
Summary:
Change gid methods

Rename GidGenerator and add tests

Fix tools broken by gid changes

Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1044
2017-12-28 11:04:52 +01:00
Mislav Bradac
8952df06c1 Temporary workaround for raft network problem
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1085
2017-12-27 15:16:14 +01:00
Mislav Bradac
d3623585e7 Migrate cereal to boost_serialization
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1077
2017-12-27 13:44:46 +01:00
florijan
7b3d298741 Fix interpreter plan ownership
Summary:
If there was no plan caching, the CachedPlan would not survive
`Interpreter::operator()`, as it was not owned by the
`Interpreter::Result`. If there was caching, it could hapen that the
cache got invalidated while that plan was being interpreted (by
another thread) without that interpretation retaining ownership.

Also simplified code around this.

Reviewers: mislav.bradac, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1084
2017-12-27 13:33:13 +01:00
florijan
9b878c91eb Make interpretation pullable
Reviewers: mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1079
2017-12-22 15:19:05 +01:00
Dominik Gleich
b094e7473b Add virtual destructor
Summary:
Not having a virtual destructor caused tests
to fail (cypher_main_visitor, interpreter) sporadically
since unfreed memory was re-used incorrectly.
Also Valgrind complained constantly.

Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1081
2017-12-22 15:02:57 +01:00
florijan
1ee6d6d05e Make WAL recovery linear
Summary:
This is a proposal on how the WAL recovery process can be implemented so
that Deltas aren't accumulated, but instead applied in the same order
they are written to the WAL.

I *believe* that the only additional requirement on the system are
atomic transaction Begin/Commit/Abort. By atomic I mean that they are
present in the WAL in exactly the same ordering like in the transaciton
engine, to ensure the same commitability of original and recovery
transactions.

This could be a requirement for HA recovery. It is desirable that WAL
and HA log become the same thing, and the recovery process too.

Reviewers: mtomic, dgleich, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1068
2017-12-22 10:02:09 +01:00
Teon Banek
9cacf970cc Install memgraph in subdirectory of lib
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1076
2017-12-21 16:53:55 +01:00
Marin Tomic
6861a580c7 Add #pragma once to cereal_optional.hpp
Reviewers: mferencevic

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D1080
2017-12-21 16:50:13 +01:00
Teon Banek
86b6f32ce6 Move load_construct_data from ast.cpp to ast.hpp
Reviewers: florijan, mislav.bradac, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1078
2017-12-21 13:08:16 +01:00
florijan
9191800fc6 Remove tx::Transaction undefined methods
Summary: Remove two tx::Transaction methods that are not defined and never used.

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1072
2017-12-21 12:02:39 +01:00
Marin Tomic
e999207b2f Remove reactors
Summary: kill reactors

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1075
2017-12-20 17:56:57 +01:00
Dominik Gleich
1556d78d15 Update snapshot format
Summary:
Set vertex/edge generator id from recovery

Add tests

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1061
2017-12-20 16:57:42 +01:00
Marin Tomic
20aa8d563a Implement RaftNetworkInterface via RPC
Summary:
Add missing fields in RequestVote RPC

Unnest Raft RPC requests and responses

Make RPC request and reply serializable

Refactor Raft network interface

Implement RPC retrying with backoff period

Add cerealization functions for optional

Add RPC network interface for raft

Add some comments and documentation

Add manual test for RPC network interface

Remove raft reactor experiments

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot, mculinovic

Differential Revision: https://phabricator.memgraph.io/D1066
2017-12-20 16:48:46 +01:00
florijan
312016ca31 Support both locally and system installed libboost
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1071
2017-12-20 16:13:03 +01:00
Teon Banek
5f7837d613 Serialize Ast classes
Summary:
Although the first solution used cereal, the final implementation uses
boost. Since the cereal is still used in the codebase, compilation has
been modified to support multithreaded cereal.

In addition to serializing Ast classes, the following also needed to be
serialized:

  * GraphDbTypes
  * Symbol
  * TypedValue

TypedValue is treated specially, by inlining the serialization code in
the Ast class, concretely PrimitiveLiteral.

Another special case was the Function Ast class, which now stores a
function name which is resolved to a concrete std::function on
construction.

Tests have been added for serialized Ast in
tests/unit/cypher_main_visitor

Reviewers: mferencevic, mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1067
2017-12-20 13:28:34 +01:00
florijan
3ae45e0d19 Add master/worker flags, main functions and coordination
Reviewers: dgleich, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1060
2017-12-19 16:05:57 +01:00
florijan
37722e54b3 Add RPC to concurrent ID mapper
Summary:
The distributed ID mapper is not yet utilised in GraphDb as those
changes are in D1060. Depending on landing order it will be added.

Reviewers: dgleich, mislav.bradac

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1064
2017-12-19 09:11:16 +01:00
Dominik Gleich
0eb30db8db Remove executor not run option
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1062
2017-12-18 14:36:15 +01:00
Marin Tomic
5b7cb4602c Add simple client facing API for RaftMember
Summary:
Implement AddComand method on RaftMember

Move RPCType out of rpc request and reply

Add unit test for AddCommand

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot, mculinovic

Differential Revision: https://phabricator.memgraph.io/D1042
2017-12-18 07:32:39 +01:00
florijan
b42169a0cb Fix RPC pimp iwyu
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1055
2017-12-14 16:34:07 +01:00
Dominik Gleich
0f1e8aa763 Add rpc server local thread
Summary: Update tests for new rpc server

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1057
2017-12-14 13:37:32 +01:00
florijan
52b3941c8a Fix tx::WorkerEngine flakyness. Change Endpoint::port type.
Reviewers: mislav.bradac, dgleich

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1054
2017-12-14 10:36:24 +01:00
Matija Santl
2758b35c08 Fix semicolon in column name
Summary:
Special casing the last token when adding it to the named expression list.
Added a test to check this case.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1052
2017-12-14 09:44:21 +01:00
Dominik Gleich
a8488ac497 Rpc client locking
Summary:
Rpc client wasn't thread safe and required a lock before each rpc call.
The locking functionality is now incorporated in Rpc client.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1056
2017-12-13 17:37:32 +01:00
Dominik Gleich
581404a7b6 Make index creation fully parallel
Summary:
Testing on following operations:

UNWIND RANGE(0,1000000) AS _ CREATE (n: Node {a: 1.0, b:2.0});
CREATE INDEX ON :Node(a);
CREATE INDEX ON :Node(b);

Before:
~32sec

After:
~18sec

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1041
2017-12-13 16:50:56 +01:00
Matija Santl
4da8312abe Fix query returning a map
Summary: Add curvy braces handling in `QueryStripper` and an accompanying test for it.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1051
2017-12-13 15:38:20 +01:00
Mislav Bradac
27370b99ff Fix Pattern::Accept
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1048
2017-12-13 13:15:22 +01:00
Mislav Bradac
cf7d9070d8 Fix flakyness in rpc/messaging tests
Summary: In this diff I just wanted to fix tests' flakyness. We can discuss if we want to always pass endpoint as an argument and never pass address:port pair explicitly. However if we decide that, I will do that change in another diff.

Reviewers: dgleich, florijan

Reviewed By: dgleich, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1049
2017-12-13 13:15:02 +01:00
Dominik Gleich
19e96c98b6 Generalize cereal message macro
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1053
2017-12-13 12:53:15 +01:00
Matija Santl
9ad7d82a54 Implement UNION query combinator
Summary:
Union query combinator implementation consists of:
 * adjustments to the AST and `cypher_main_visitor`
 * enabling `QueryStripper` to parse multiple `return` statements (not stopping after first)
 * symbol generation for union results
 * union logical operator
 * query plan generator adjustments

Reviewers: teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D1038
2017-12-13 09:54:00 +01:00
Marko Culinovic
52a9be3cc8 Add examples to installation
Summary:
Examples are added in release/examples directory. Each example
must have its own directory with populate.cyp file inside it.
This file contains graph creation queries written in OpenCypher.
When memgraph is built, database snapshots for each example are
created in release/examples/build directory. During memgraph
installation these snapshots are copied to share/memgraph/examples.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1036
2017-12-12 14:59:54 +01:00
Mislav Bradac
eb272f0b67 Assert on endpoint failures
Summary: .

Reviewers: mferencevic, florijan

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1045
2017-12-12 14:27:24 +01:00
Teon Banek
0d40f6a759 Merge memgraph_link_target into memgraph executable target
Summary:
Memgraph's executable target is unnecessarily complicated by having a
name which depends on the commit hash and build type. It is now renamed
to simply 'memgraph', while the executable output uses the encoding of
commit hash and build type as the name. This simplifies the dependency
handling in cmake, where now the full path to the executable can be
easily obtained by using the generator expression:
`$<TARGET_FILE:memgraph>`.

In addition to the above simplification, memgraph_link_target is now
replaced with a post build step on memgraph target itself. This
guarantees that the symlink always exists after building the executable,
as well as simplify the make invocation. The invocation now boils down
to `make memgraph` instead of `make memgraph_link_target`.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1043
2017-12-11 16:11:19 +01:00
Marin Tomic
6b4a2cafc5 Fix OnRequestVote RPC callback
Summary: RPC recipient should update its term even if it is rejecting the request.

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1039
2017-12-11 12:53:17 +01:00
florijan
4982d45e27 Add RPC to distributed tx::Engine
Reviewers: mislav.bradac, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1035
2017-12-11 11:05:01 +01:00
Dominik Gleich
eb1adccf40 Fix compilation
Reviewers: teon.banek, mislav.bradac, mferencevic

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D1040
2017-12-11 10:40:50 +01:00
Marin Tomic
0bf692f8ec Refactor Raft leader election to use multiple threads
Summary:
Implement log replication

Rebase and fix src/CMakeLists.txt

Some style fixes

Changed shared_ptr to unique_ptr for RaftPeerState

Change Id and Leader to const

Move implementation to separate class

Fix raft_experiments.cpp

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1033
2017-12-08 16:30:57 +01:00
florijan
f5c0455af4 Prepare record accessor for distributed
Summary:
What's done:
- `RecordAccessor` can represent remote data
- `GraphDbAccessor` manages remote data
- Cleanup: different `EdgeAccessor lazyness (@dgleich: take a look), unused methods, documentation...
- `TODO` placeholders for remote implementation

What's not done:
- RPC and data transfer
- how exactly remote errors are handled
- not sure if any MVCC Record info for remote data should be tracked
- WAL and RPC Deltas properly handled (Gleich working on extracting `Wal::Op`)

This implementation should not break single-node execution, and should provide good abstractions and placeholders for distributed. Once that's satisfied, it should land.

Reviewers: dgleich, buda, mislav.bradac

Reviewed By: dgleich

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D1030
2017-12-08 14:12:18 +01:00
Dominik Gleich
03db948d7e Refactor operations into StateDeltas
Summary: Operations are moved and renamed from WAL to a separate file in preparation for HA and distributed storage.

Reviewers: florijan, mtomic, mislav.bradac

Reviewed By: florijan

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D1034
2017-12-07 14:23:41 +01:00
Mislav Bradac
60f4db2b9f Add first version of message passing and rpc
Reviewers: mtomic, buda

Reviewed By: mtomic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1002
2017-12-05 14:51:18 +01:00
Dominik Gleich
3ddbcad0d9 Refactor global ids and prepare for distributed
Summary:
Change ids to global ids

Fix tests

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1019
2017-12-05 13:05:55 +01:00
florijan
e218bc1c69 Improve utils::RandomString
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1023
2017-12-05 10:10:36 +01:00
Teon Banek
a17261038c Include tools CMakeLists in the top level CMakeLists
Summary:
Split main CMakeLists into src/CMakeLists

The main CMakeLists duty is to make all the required libraries and
variables visible to all of the other sub-CMakeLists. After doing that,
it should include those sub-CMakeLists according to configuration
options.

This should make global configurations easier to reuse without polluting
the global space with locally related configurations. It is a necessary
step for including other projects like 'tools' in the release
installation.

Building tools is automatically disabled, but can be enabled by setting
the TOOLS option to ON when running cmake. This should allow on demand
building as well as combined installation of Memgraph and its tools.

Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1018
2017-12-04 15:18:18 +01:00
florijan
d1dbf22cd1 Prepare ConcurrentIdMapper for distributed
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1017
2017-12-04 14:06:05 +01:00
florijan
e26456d5ad Fix durability test flakyness
Summary:
It occurred that part of the durability flakyness test might be that the
same durability directory is used always. If the test is run
simultaneously on a single system, there will be interference.

This might not actually fix all the flakyness :(

I also made the `utils::RandomString` function since that's now used in
multiple places, tested it etc.

Reviewers: buda, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1020
2017-12-04 13:46:56 +01:00
Teon Banek
a799351eb0 Correctly inspect property filters during planning
Summary:
This change generates multiple PropertyFilters for expressions such as
`n.prop1 = m.prop2`. When choosing one PropertyFilter, we want to also
remove the other one, because they represent the same original
expression.  Therefore, the removal is no longer based on FilterInfo
equality, but on the original expression equality. Additionally,
FilterInfo and PropertyFilter equality operators have been removed to
avoid any pretense they do what you expect or want.

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1021
2017-12-04 09:57:41 +01:00
florijan
ce3638b25e Prepare transactional engine for distributed
Summary:
The current idea is that the same MG binary can be used for single-node,
distributed master and distributed worker. The transactional engine in
the single-node and distributed master is the same: it determines the
transactional time and exposes all the "global" functionalities. In the
distributed worker the "global" functions must contact the master.

Reviewers: dgleich, mislav.bradac, buda

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1013
2017-12-01 09:17:44 +01:00
florijan
2fbe967465 Remove UniqueObjectStore
Summary: Because it will never be used, we already have replacements for it.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1016
2017-11-30 13:04:37 +01:00
Dominik Gleich
9c24faa4d6 Expose faster edge_accessor constructor
Summary:
Fix nullptr exceptions

Inline edge_type and from/to in edge accessor

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1010
2017-11-29 12:40:11 +01:00
florijan
3bd5a882cf Remove Update and Reconstruct from GraphDbAccessor
Summary: Code simplification made possible by making `locks_` `mutable` in `tx::Transaction`.

Reviewers: dgleich, buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1015
2017-11-29 10:02:01 +01:00
Dominik Gleich
2b2de245d1 Allow concurrent index creation
Summary: Update tests

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1012
2017-11-28 16:44:12 +01:00
Dominik Gleich
4f55d5d78b Fix test deadlock
Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1014
2017-11-28 16:43:21 +01:00
florijan
d5bcf9a7d7 Remove old WAL files after snapshot
Summary: Once a snapshot is successfully written, delete WAL files which are no longer necessary for recovery. Note that this prohibits recovering the WAL from any except the last snapshot.

Reviewers: buda, mislav.bradac, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1000
2017-11-28 09:45:54 +01:00
Dominik Gleich
a7f9255c17 Remove redundant transaction from index creation
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1011
2017-11-27 16:47:12 +01:00
florijan
23241e76b9 Prepare tx::Engine for distributed
Summary:
This diff contains step 1:
- Remove clog exposure from tx::engine
- Reduce and cleanup tx::Engine API

All current functionality is kept, but the API is reduced. This is very
desirable because every function in tx::Engine will need to be
considered and implemented in both Master and Worker situations. The
less we have, the better.

Next step is exactly that: seeing how each of these functions behaves in
a distributed system and implementing accordingly.

Reviewers: dgleich, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1008
2017-11-24 14:51:29 +01:00
florijan
61e28bffd0 Fix return collection containing both aggregation and group-by
Summary:
Referring to the TCK failure on:
```
MATCH (a {name: 'Andres'})<-[:FATHER]-(child)
RETURN {foo: a.name='Andres', kids: collect(child.name)}
```

In the planner we'd only treat a list|map as a group_by if it contained
no aggregations. That's changed so that if a map contains both aggregations
and non-aggregations, then non-aggregations are treated as individual
group_by expressions.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: buda, pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1004
2017-11-24 14:43:48 +01:00
Dominik Gleich
67538aceeb Migrate labels/properties/edgetypes to ids
Summary:
In preparation for distributed storage we need to have labels/properties/edgetypes uniquely identifiable by their ids, which will be global in near future.
The old design has to be abandoned because it's not possible to keep track of global labels/properties/edgetypes while they are local pointers.

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D993
2017-11-23 17:12:37 +01:00
florijan
5ef2d20133 Add single query manual test
Summary:
Practical for debugging trivial queries:
```
~ gdb --args ./tests/manual/single_query "RETURN 2 + 1"

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1003
2017-11-23 14:51:46 +01:00
florijan
968aa4926a Add parallel customers/Otto test
Summary:
Looking for connected components in a random graph. This test performs the following:
- Generates a random graph that is NOT sequential in memory (otherwise itertion over edges is 2 or more times faster).
- Connectivity by iterating over all the edges.
- Ditto over vertices.
- Ditto over vertices in parallel.

Not done:
- Edge filtering based on XY. I could/should add that to see how it affects perf.
- Getting component info out from union-find.

Local results are encouraging. Iterating over the graph is the bottleneck. Still, I get connectivity of 10M vertices/edges in <7sec (parallel over vertices). Will test on 250M remote now.

Locally obtained results (20M/20M, 2 threads)
```
I1115 14:57:55.136875   357 otto_parallel.cpp:50] Generating 2000000 vertices...
I1115 14:58:19.057734   357 otto_parallel.cpp:74] Generated 2000000 vertices in 23.9208 seconds.
I1115 14:58:19.919221   357 otto_parallel.cpp:82] Generating 2000000 edges...
I1115 14:58:39.519951   357 otto_parallel.cpp:93] Generated 2000000 edges in 19.3398 seconds.
I1115 14:58:39.520349   357 otto_parallel.cpp:196] Running Edge iteration...
I1115 14:58:43.857264   357 otto_parallel.cpp:199]      Done in 4.33691 seconds, result: 3999860270398
I1115 14:58:43.857316   357 otto_parallel.cpp:196] Running Vertex iteration...
I1115 14:58:49.498181   357 otto_parallel.cpp:199]      Done in 5.64087 seconds, result: 4000090070787
I1115 14:58:49.498208   357 otto_parallel.cpp:196] Running Connected components - Edges...
I1115 14:58:54.232530   357 otto_parallel.cpp:199]      Done in 4.73433 seconds, result: 323935
I1115 14:58:54.232570   357 otto_parallel.cpp:196] Running Connected components - Vertices...
I1115 14:59:00.412395   357 otto_parallel.cpp:199]      Done in 6.17983 seconds, result: 323935
I1115 14:59:00.412422   357 otto_parallel.cpp:196] Running Parallel connected components - Vertices...
I1115 14:59:04.662087   357 otto_parallel.cpp:199]      Done in 4.24967 seconds, result: 323935
I1115 14:59:04.662116   357 otto_parallel.cpp:196] Running Expansion...
I1115 14:59:13.913015   357 otto_parallel.cpp:199]      Done in 9.25091 seconds, result: 323935
```

Reviewers: buda, mislav.bradac, dgleich, teon.banek

Reviewed By: buda, teon.banek

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D982
2017-11-23 09:20:53 +01:00
Teon Banek
5a41478789 Add maintainer scripts for DEB package
Summary:
Add postinst script for DEB package

The script creates a 'memgraph' group and sets permission on installed
'/var/*/memgraph' directories. Only the group is created, while
'memgraph' user is not. It seems more sane only to require group
membership for using memgraph.

Add conffiles for DEB package

This allows for `dpkg` to detect changes in configuration files and
present them to the user. Therefore, we don't need to care whether the
configuration merges are handled correctly nor if we accidentally
overwrite them.

Add postrm script for DEB packaging

The script is only used so that `dpkg --purge` removes '/var/*/memgraph'
directories, even if they contain something.

Add email, longer description and license file to DEB packaging, as well
as a systemd service.

Provide a logrotate configuration and support it in memgraph.

Use DEB package for Docker installation

This way, the whole installation process and testing should go through
DEB.

Generate release archives in Apollo with standard names

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D989
2017-11-22 16:41:25 +01:00
florijan
597934203e Support explicit int->bool and bool->int
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D999
2017-11-22 12:11:16 +01:00
florijan
8bbf1af525 Cleanup durability config, docs, CHANGELOG
Reviewers: teon.banek, buda, mislav.bradac, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D992
2017-11-21 10:17:13 +01:00
Matija Santl
df4cbdc5b2 Add error messsage for large integer literals
Summary:
Fixing https://app.asana.com/0/170237629387822/481366792497820/f

Test plan:
Started two builds, one with the fix and the second one without the fix.
Connected to each of them using `neo4j` client.

Logs received from the build wihtout the fix:
```
neo4j> Create (n: BigInteger{id:12345678912345678912345}) return n;
<interactive>:0:0: error:
```
```
I1120 13:29:09.551208 30482 executing.hpp:69] [Run] 'Create (n: BigInteger{id:12345678912345678912345}) return n'
W1120 13:29:09.552387 30482 executing.hpp:145] Error message:
```

Logs received from the build with the fix:
```
neo4j> Create (n: BigInteger{id:12345678912345678912345}) return n;
<interactive>:0:0: error: Integer literal exceeds 64 bits
```
```
I1120 13:29:07.940943 30453 executing.hpp:69] [Run] 'Create (n: BigInteger{id:12345678912345678912345}) return n'
W1120 13:29:07.942919 30453 executing.hpp:146] Error message: Integer literal exceeds 64 bits
```

Reviewers: mislav.bradac, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D997
2017-11-20 14:33:40 +01:00
Marko Budiselic
3c9e143c95 Fix SegFault within executor state
Reviewers: mislav.bradac, dgleich, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D995
2017-11-20 11:27:08 +01:00
Mislav Bradac
651806c14f Remove FindChannel function
Reviewers: buda, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D990
2017-11-17 10:11:17 +01:00
florijan
885d19401f Move WAL implementation to .cpp
Summary: No implementation changes, just code migration, rebase, import cleanup.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D983
2017-11-16 09:12:34 +01:00
Mislav Bradac
2d6675df63 Fix FindChannel implementation
Summary:
Change queue implementation in distributed reactor

Fix FindChannel implementations

Reviewers: mtomic, buda, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D986
2017-11-15 17:55:33 +01:00
Mislav Bradac
e703e955a5 Fix bug in InListOperator
Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D981
2017-11-14 15:39:10 +01:00
florijan
463e86653d Vertex and Edge distributed storage support
Summary: Vertex and Edge now use Address for storing connections to other Edges and Vertices, to support distributed storage.

Reviewers: mislav.bradac, dgleich, buda

Reviewed By: mislav.bradac, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D977
2017-11-14 15:18:06 +01:00
Mislav Bradac
5c0f378394 Add thread safe queue
Reviewers: dgleich, florijan

Reviewed By: dgleich

Differential Revision: https://phabricator.memgraph.io/D979
2017-11-13 16:44:49 +01:00
Marin Tomic
41f868319d Raft election
Summary: Implemented leader election part of raft protocol

Reviewers: mislav.bradac, buda, teon.banek, dgleich

Reviewed By: mislav.bradac

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D966
2017-11-13 12:47:12 +01:00
florijan
1e0ac8ab8f Write-ahead log
Summary:
My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed...

What's done:
- Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr.
- WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk.
- Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info.
- Durability testing completely reworked.

What's not done (and should be when we decide how):
- Old WAL file purging.
- Config refactor (naming and organization). Will do when we discuss what we want.
- Changelog and new feature documentation (both depending on the point above).
- Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment).
- Moving the implementation of WAL stuff to .cpp where possible.
- Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into.
- True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature.

Reviewers: buda, mislav.bradac, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D958
2017-11-13 09:51:39 +01:00
Teon Banek
93bc9accd8 Raise NotYetImplemented when encountering UNION
Summary:
We need to explicitly say that UNION clause isn't supported, otherwise
it gets silently ignored. So for example, `CREATE () UNION CREATE ()`
would create 2 nodes without a hitch. On the other hand,
`RETURN 1 UNION RETURN 2` would complain that there is more than 1
RETURN in the query, which was misleading.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D976
2017-11-10 15:10:02 +01:00
Teon Banek
796e4f50e5 Add named path symbols to new symbols of a match
Summary:
Previously, named path symbols remained untracked as `new_symbols` during planning. This meant that
operator `Optional` would be left unaware of those symbols, and therefore not reset them to `Null`
if optional matching failed.

Test Optional operator will be aware of path symbols

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D974
2017-11-10 15:09:47 +01:00
Teon Banek
de7a788311 Set switch statement warnings as errors
Summary:
The warnings lifted to errors should help us track down cases which
haven't been covered in a switch. Obviously, using the default case will
not trigger these errors, so default should only be used when we
actually want to cover more cases.

Reviewers: florijan, dgleich, mferencevic

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D975
2017-11-10 13:12:02 +01:00
Dominik Gleich
f50d87f736 Take care of warnings/errors created by cppcheck.
Summary:
Warnings I ignored:

Creates a new stacktrace object and then dumps it
	102.570384 (102.495075) E[1]: [src/utils/exceptions.hpp:116]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.
	102.570390 (102.495081) E[1]: [src/utils/exceptions.hpp:127]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.

Used all over the codebase without explicit cast
102.570412 (102.495103) E[1]: [src/utils/stacktrace.hpp:14]: (style) Class 'Line' has a constructor with 1 argument that is not explicit.

Not really used anywhere before initialized:
102.570526 (102.495217) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::preds' is not initialized in the constructor.
102.570530 (102.495221) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::succs' is not initialized in the constructor.

Implicit conversions between types are used all over the codebase:
102.570548 (102.495239) E[1]: [src/storage/property_value.hpp:41]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570552 (102.495243) E[1]: [src/storage/property_value.hpp:42]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570557 (102.495248) E[1]: [src/storage/property_value.hpp:43]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570561 (102.495252) E[1]: [src/storage/property_value.hpp:44]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570566 (102.495257) E[1]: [src/storage/property_value.hpp:47]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570570 (102.495261) E[1]: [src/storage/property_value.hpp:50]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570578 (102.495269) E[1]: [src/storage/property_value.hpp:53]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570582 (102.495273) E[1]: [src/storage/property_value.hpp:57]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570591 (102.495282) E[1]: [src/query/typed_value.hpp:80]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570596 (102.495287) E[1]: [src/query/typed_value.hpp:81]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570601 (102.495292) E[1]: [src/query/typed_value.hpp:82]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570605 (102.495296) E[1]: [src/query/typed_value.hpp:83]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570609 (102.495300) E[1]: [src/query/typed_value.hpp:89]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570614 (102.495305) E[1]: [src/query/typed_value.hpp:92]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570618 (102.495309) E[1]: [src/query/typed_value.hpp:95]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570623 (102.495314) E[1]: [src/query/typed_value.hpp:98]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570627 (102.495318) E[1]: [src/query/typed_value.hpp:102]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570632 (102.495323) E[1]: [src/query/typed_value.hpp:105]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570636 (102.495327) E[1]: [src/query/typed_value.hpp:108]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570641 (102.495332) E[1]: [src/query/typed_value.hpp:109]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570645 (102.495336) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:88]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570650 (102.495341) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:89]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570654 (102.495345) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:90]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570659 (102.495350) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:91]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570663 (102.495354) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:94]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570668 (102.495359) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:97]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570672 (102.495363) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:100]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570677 (102.495368) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:104]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570681 (102.495372) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:107]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570690 (102.495381) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:110]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570694 (102.495385) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:113]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.

CypherParser:

102.570767 (102.495458) E[1]: [src/query/frontend/opencypher/generated/CypherParser.h:69]: (style) Class 'CypherParser' has a constructor with 1 argument that is not explicit.
102.570772 (102.495463) E[1]: [src/query/frontend/opencypher/generated/CypherLexer.h:40]: (style) Class 'CypherLexer' has a constructor with 1 argument that is not explicit.
102.570776 (102.495467) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:86]: (style) The scope of the variable '_la' can be reduced.
102.570781 (102.495472) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:311]: (style) The scope of the variable '_la' can be reduced.
102.570785 (102.495476) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:402]: (style) The scope of the variable '_la' can be reduced.
102.570789 (102.495480) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:497]: (style) The scope of the variable '_la' can be reduced.
102.570797 (102.495488) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:778]: (style) The scope of the variable '_la' can be reduced.
102.570802 (102.495493) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:895]: (style) The scope of the variable '_la' can be reduced.
102.570806 (102.495497) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:991]: (style) The scope of the variable '_la' can be reduced.
102.570811 (102.495502) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1190]: (style) The scope of the variable '_la' can be reduced.
102.570815 (102.495506) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1274]: (style) The scope of the variable '_la' can be reduced.
102.570820 (102.495511) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1393]: (style) The scope of the variable '_la' can be reduced.
102.570824 (102.495515) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1570]: (style) The scope of the variable '_la' can be reduced.
102.570829 (102.495520) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1695]: (style) The scope of the variable '_la' can be reduced.
102.570834 (102.495525) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1800]: (style) The scope of the variable '_la' can be reduced.
102.570839 (102.495530) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1903]: (style) The scope of the variable '_la' can be reduced.
102.570843 (102.495534) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2019]: (style) The scope of the variable '_la' can be reduced.
102.570848 (102.495539) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2228]: (style) The scope of the variable '_la' can be reduced.
102.570852 (102.495543) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2542]: (style) The scope of the variable '_la' can be reduced.
102.570857 (102.495548) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2797]: (style) The scope of the variable '_la' can be reduced.
102.570861 (102.495552) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2966]: (style) The scope of the variable '_la' can be reduced.
102.570866 (102.495557) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3067]: (style) The scope of the variable '_la' can be reduced.
102.570870 (102.495561) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3289]: (style) The scope of the variable '_la' can be reduced.
102.570875 (102.495566) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3295]: (style) The scope of the variable 'alt' can be reduced.
102.570879 (102.495570) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3419]: (style) The scope of the variable '_la' can be reduced.
102.570884 (102.495575) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3596]: (style) The scope of the variable '_la' can be reduced.
102.570888 (102.495579) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3688]: (style) The scope of the variable '_la' can be reduced.
102.570893 (102.495584) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3963]: (style) The scope of the variable '_la' can be reduced.
102.570897 (102.495588) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4452]: (style) The scope of the variable '_la' can be reduced.
102.570902 (102.495593) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4586]: (style) The scope of the variable '_la' can be reduced.
102.570906 (102.495597) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4813]: (style) The scope of the variable '_la' can be reduced.
102.570911 (102.495602) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4943]: (style) The scope of the variable '_la' can be reduced.
102.570918 (102.495609) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5026]: (style) The scope of the variable '_la' can be reduced.
102.570923 (102.495614) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5569]: (style) The scope of the variable '_la' can be reduced.
102.570928 (102.495619) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5664]: (style) The scope of the variable '_la' can be reduced.
102.570932 (102.495623) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5755]: (style) The scope of the variable '_la' can be reduced.
102.570937 (102.495628) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5888]: (style) The scope of the variable '_la' can be reduced.
102.570941 (102.495632) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6045]: (style) The scope of the variable '_la' can be reduced.
102.570946 (102.495637) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6142]: (style) The scope of the variable '_la' can be reduced.
102.570950 (102.495641) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6347]: (style) The scope of the variable '_la' can be reduced.
102.570955 (102.495646) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6523]: (style) The scope of the variable '_la' can be reduced.
102.570959 (102.495650) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6614]: (style) The scope of the variable '_la' can be reduced.
102.570964 (102.495655) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6899]: (style) The scope of the variable '_la' can be reduced.
102.570968 (102.495659) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6992]: (style) The scope of the variable '_la' can be reduced.
102.570973 (102.495664) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7147]: (style) The scope of the variable '_la' can be reduced.
102.570977 (102.495668) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7680]: (style) The scope of the variable '_la' can be reduced.
102.570982 (102.495673) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7759]: (style) The scope of the variable '_la' can be reduced.
102.570986 (102.495677) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7938]: (style) The scope of the variable '_la' can be reduced.
102.570991 (102.495682) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8126]: (style) The scope of the variable '_la' can be reduced.
102.570995 (102.495686) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8220]: (style) The scope of the variable '_la' can be reduced.
102.571000 (102.495691) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8313]: (style) The scope of the variable '_la' can be reduced.
102.571004 (102.495695) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8491]: (style) The scope of the variable '_la' can be reduced.
102.571009 (102.495700) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8703]: (style) The scope of the variable '_la' can be reduced.
102.571013 (102.495704) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8783]: (style) The scope of the variable '_la' can be reduced.
102.571018 (102.495709) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8914]: (style) The scope of the variable '_la' can be reduced.
102.571022 (102.495713) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9119]: (style) The scope of the variable '_la' can be reduced.
102.571027 (102.495718) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9220]: (style) The scope of the variable '_la' can be reduced.
102.571034 (102.495725) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9414]: (style) The scope of the variable '_la' can be reduced.
102.571039 (102.495730) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9660]: (style) The scope of the variable '_la' can be reduced.
102.571043 (102.495734) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10008]: (style) The scope of the variable '_la' can be reduced.
102.571048 (102.495739) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10158]: (style) The scope of the variable '_la' can be reduced.
102.571052 (102.495743) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10250]: (style) The scope of the variable '_la' can be reduced.
102.571057 (102.495748) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10370]: (style) The scope of the variable '_la' can be reduced.
102.571061 (102.495752) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10637]: (style) The scope of the variable '_la' can be reduced.
102.571065 (102.495756) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10749]: (style) The scope of the variable '_la' can be reduced.
102.571070 (102.495761) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10815]: (style) The scope of the variable '_la' can be reduced.
102.571075 (102.495766) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10881]: (style) The scope of the variable '_la' can be reduced.

We know that we represented it correctly in memory:
102.571079 (102.495770) E[1]: [src/communication/bolt/v1/decoder/decoder.hpp:252]: (portability) Casting between integer* and double* which have an incompatible binary data representation.

Cont assigned but not used after:
102.571101 (102.495792) E[1]: [src/query/frontend/ast/ast.hpp:1008]: (style) Variable 'cont' is assigned a value that is never used.

Reviewers: teon.banek, buda, mferencevic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D967
2017-11-10 10:28:02 +01:00
Teon Banek
7fabdda938 Fix an error where null AND false returned null
Summary:
Tests have been updated to catch this error and other behaviour. Other
than this change, `AND` should behave as before.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D970
2017-11-09 16:46:25 +01:00
Teon Banek
4b8076f41a Fix issues with lambda symbols and returning *
Summary:
Correctly propagate user declared symbols for lambda.
Unbind lambda symbols after inlining filters.
Also update unit tests.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D971
2017-11-09 16:46:06 +01:00
Marin Tomic
dddfe52a45 Add Watchdog utility
Summary: see documentation

Reviewers: mislav.bradac, buda, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: teon.banek, dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D951
2017-11-09 11:23:11 +01:00
Teon Banek
b36386cfe9 Prevent double termination signals causing crashes
Summary:
Use sigaction to register signal handlers.
This is preferred over `signal` function, according to `man 3p signal`.
Add global sig_atomic_t flag when shutting down.
Block other signal handlers when shutting down.

Reviewers: mislav.bradac, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D943
2017-11-09 10:41:16 +01:00
florijan
ab2f7c3288 Ring-buffer - heap array instead of vec
Summary: I like this better, it's clearer.

Reviewers: dgleich, mislav.bradac

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D961
2017-11-08 12:49:22 +01:00
Marin Tomic
0930e8c457 Fix name uniqueness check in System.Spawn
Summary: Old code would start reactor thread first, and then try to insert it in unordered_map. If it failed to insert it, the thread handler would get destructed with thread still running, which raises an exception.

Reviewers: buda, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D956
2017-11-03 15:14:37 +01:00
Dominik Gleich
09ad52958f POC - Typed value assignment for different types.
Summary: TypedValue assignment operator=(TypedValue &other) is inefficient for types which already have a defined assignment operator, or can be assigned trivially, without using TypedValue to pass the value for assignment.

Reviewers: mislav.bradac, florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D934
2017-11-02 14:21:22 +01:00
florijan
f2aad117cf RingBuffer - buffer on heap instead of stack
Summary: RingBuffer - buffer in vector

Reviewers: mislav.bradac, buda, dgleich

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D952
2017-11-02 12:07:23 +01:00
Mislav Bradac
b39231f7df Add tests for string utils
Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D941
2017-10-30 14:15:51 +01:00
Mislav Bradac
00f76d1391 Add nodes and relationships functions
Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D942
2017-10-30 12:38:12 +01:00
Teon Banek
55456b4214 Remove Dbms
Summary:
Remove name from GraphDb.
Take GraphDb in query test macros instead of accessor.
Add is_accepting_transactions flag to GraphDb.

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D940
2017-10-30 12:33:29 +01:00
florijan
be9c875fa9 Durability - snapshot summary refactor
Summary:
- Removed durability::Summary because it was wired into reader and stopped me from recovering WAL files.
- Refactored and renamed BufferedFile(Reader/Writer) to HashedFile(Reader/Writer).
- Vertex and edge counts in the snapshot are now hashed.

Breaking snapshot compatibility again (hashing), but since the previous version was not released, and we are not caching snapshots, the previous version does not need to be supported.

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek, mislav.bradac

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D932
2017-10-27 13:51:02 +02:00
Mislav Bradac
8d1e463e6b Remove unused capture
Reviewers: dgleich

Reviewed By: dgleich

Differential Revision: https://phabricator.memgraph.io/D939
2017-10-27 13:36:23 +02:00
Mislav Bradac
e6d3edf9a9 Fix distributed reactors
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D935
2017-10-27 11:26:04 +02:00
Teon Banek
df72861b90 Make csv_to_snapshot more user friendly
Summary:
Time csv_to_snapshot conversion and log it.
Check if writing csv_to_snapshot failed.
Extract LoadConfig from memgraph_bolt to config.hpp.
Read memgraph config in csv_to_snapshot for snapshot_directory.
Rename csv_to_snapshot to mg_import_csv.
Add tests for tools.
Run tools tests in apollo.

Reviewers: mislav.bradac, florijan, mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D931
2017-10-26 09:37:56 +02:00
Mislav Bradac
fe3d752904 Revise reactors code
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D923
2017-10-25 15:36:18 +02:00
florijan
9f7ef8e0e9 utils::string - split functions extended
Summary:
- RSplit added
- split limits added
- tests

Reviewers: teon.banek, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D930
2017-10-25 15:26:31 +02:00
florijan
fc240fbb67 Transaction command_id type reduced from 8B to 2B
Summary: This reduces the size of a single `mvcc::Record<T>` from 40B to 24B.

Reviewers: buda, mislav.bradac, dgleich

Reviewed By: mislav.bradac, dgleich

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D920
2017-10-25 10:42:57 +02:00
Teon Banek
db34cb2a40 Refactor collecting filters during planning
Summary:
Move QueryParts and Filters to a new file.
Reorganize FilterInfo struct.
Remove label filter if we do indexed scan by label.
Remove property filter used in indexed scan.

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D915
2017-10-24 14:13:14 +02:00
Dominik Gleich
184996d925 Change allocator.
Summary:
There seems to be a gain all over memgraph.
I strongly suggest including this in the codebase.

Link to project:
https://github.com/r-lyeh/ltalloc

Reviewers: teon.banek, mferencevic, mislav.bradac, florijan

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D914
2017-10-24 10:20:15 +02:00
florijan
b65fcc8f90 Bolt: BaseEncoder split into PrimitiveEncoder and BaseEncoder
Summary:
This is an attempt at solving circular dependencies happening in WAL implementation. The cycle is:

BaseEncoder -> GraphDbAccessor -> GraphDb -> WAL -> BaseEncoder.

The cycle will be broken by this diff because the WAL only needs primitive encoding and will be able to use `PrimitiveEncoder` only.

This fix is not ideal, since the BaseEncoder -> GraphDbAccessor dependency is very unnatural. This could probably be fixes properly with a refactor of GraphDb/GraphDbAccessor that is in the post, but that takes time and this fix is not very dirty, more of an added complication.

Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D925
2017-10-24 09:47:00 +02:00
florijan
add801a80a Ring buffer added
Summary: Locked version. There are some benchmarks, it seems the lock won't be the bottleneck in the WAL (DB ops causing WAL delta insertions into it will be slower, flushing the WAL be slower).

Reviewers: buda, mislav.bradac, dgleich

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D919
2017-10-24 08:53:35 +02:00
florijan
4460dd79f9 Durability - snapshot magic num, version and tx snapshot added
Summary:
New snapshot structure:
- magic number
- snapshot version (old-version recovery not yet implemented)
- transaction snapshot (will be used in the WAL)
- the rest is as before (indices, vertices, edges)

Not backward compatible with the old snapshotting.

Does not improve error handling (user feedback). A task for that has been added.

Reviewers: buda, mislav.bradac, mferencevic, teon.banek

Reviewed By: teon.banek

Subscribers: teon.banek, dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D912
2017-10-23 15:51:35 +02:00
Mislav Bradac
19a44a7d94 Close session on timeouts
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D897
2017-10-17 14:34:31 +02:00
florijan
14fabe2125 Removed one function from version_list
Summary: It wasn't used in MG, only in tests.

Reviewers: buda, dgleich, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D909
2017-10-17 11:21:56 +02:00
Mislav Bradac
ebe3c74a84 Cleanup skiplist insert_here method
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D907
2017-10-17 11:01:44 +02:00
Teon Banek
8da3839a1d Convert variable start planning to iterative process
Summary: This change increases the planning time, but should reduce memory consumption.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D901
2017-10-16 10:12:09 +02:00
Teon Banek
96d9846b25 Remove traces of custom assert
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D905
2017-10-13 09:34:13 +02:00
Dominik Gleich
ebc7b2b1b9 Const map/set iteration.
Summary: In the current state, it was not possible to iterate, or even access a const map, or const set structure because of an incorrect implementation of "ConstAccessors".

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D902
2017-10-12 16:35:39 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Mislav Bradac
e078c9f7cd Refactor Interpreter
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D896
2017-10-11 11:06:25 +02:00
Dominik Gleich
cce6db3442 Const iterator works.
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D886
2017-10-11 11:01:24 +02:00
Mislav Bradac
f10380a861 Begin refactor of network layer
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D855
2017-10-10 14:55:13 +02:00
Mislav Bradac
31798eb957 Handle index creation correctly
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D891
2017-10-09 19:22:39 +02:00
Mislav Bradac
bac99cc267 Return TransientError when possible
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D889
2017-10-09 17:58:54 +02:00
Mislav Bradac
93b9592c63 Add malloc_trim
Summary: Lets run daily tonight with malloc_trim and see what is memory usage

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D888
2017-10-09 15:48:19 +02:00
Mislav Bradac
2fd43019d3 Change scheduler to first wait then execute
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D884
2017-10-09 15:40:00 +02:00
Matej Ferencevic
45a02b56f8 Fixed bolt client endless loop when server dies.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D879
2017-10-08 13:31:05 +02:00
Matej Ferencevic
37b2caae63 Implemented hidden flags.
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D861
2017-10-07 15:08:48 +02:00
Mislav Bradac
e11216f058 Remove malloc_trim
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D876
2017-10-06 19:13:01 +02:00
florijan
880a563cf3 Generate snapshot added
Summary:
A cool new graph generator. It's really cool!

It is currently being used for generating the Ravelin graph with the following config:
```
{
  "indexes": ["Node.id"],
  "nodes": [
    {
      "count": 40000000,
      "labels": ["Node"],
      "properties": {
              "id" : {"type": "counter", "param" : "Node.id"},
              "fradulent" : {"type" : "bernoulli", "param" : 0.0005}
      }
    }
  ],
  "edges": [
    {
      "count": 80000000,
      "from": "Node",
      "to": "Node",
      "type": "Edge"
    }
  ]
}
```

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D872
2017-10-06 15:53:02 +02:00
Teon Banek
8bf0a9cf92 Fill the alpha.conf with all settings and document them
Summary:
This should be the basis for default configuration that is to be shipped
in release builds/packages.

Reviewers: mislav.bradac, florijan, buda, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D851
2017-10-06 15:15:19 +02:00
Mislav Bradac
da3e46c2d6 Fix race condition in BuildIndex
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D871
2017-10-06 15:13:26 +02:00
Mislav Bradac
c6f1920f8b Remove EdgeType index - not used in interpreter
Reviewers: teon.banek, florijan

Reviewed By: teon.banek, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D870
2017-10-06 15:02:34 +02:00
Mislav Bradac
c3a211fd4e Add message to HintedAbortError
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D868
2017-10-06 14:39:50 +02:00
Teon Banek
4c25123d83 Make docker installation equivalent to regular install
Summary:
This puts the whole installation and packaging under a single point of
entry. (Docker, DEB, RPM, etc.)

Rename alpha.dockerfile to beta.dockerfile
Use Debian Stretch for docker
Remove building old hardcoded compiler
Rename build_interpreter to build_memgraph
Remove unused config-file

Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D857
2017-10-06 13:23:34 +02:00
Mislav Bradac
11b2b83c96 Make gc more aggressive and optimise is_not_visible function
Reviewers: florijan, mferencevic

Reviewed By: florijan

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D849
2017-10-06 10:07:55 +02:00
florijan
d0fb4d7242 BFS - lower bound support
Summary: This is not a very important functionality, but it turned out simple to do, so let's add it to have a consistent query support.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D862
2017-10-06 09:33:20 +02:00
Mislav Bradac
1f01ef6cd1 Rerun antlr on original query to generate appropriate error message
Reviewers: buda, florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D858
2017-10-05 13:35:39 +02:00
florijan
76fe8bfadf Variable expansion consolidaton
Summary:
- Removed BreadthFirstAtom, using EdgeAtom only with a Type enum.
- Both variable expansions (breadth and depth first) now have mandatory inner node and edge Identifiers.
- Both variable expansions use inline property filtering and support inline lambdas.
- BFS and variable expansion now have the same planning process.
- Planner modified in the following ways:
	- Variable expansions support inline property filtering (two filters added to all_filters, one for inline, one for post-expand).
	- Asserting against existing_edge since we don't support that anymore.
	- Edge and node symbols bound after variable expansion to disallow post-expand filters to get inlined.
	- Some things simplified due to different handling.
- BreadthFirstExpand logical operator merged into ExpandVariable. Two Cursor classes remain and are dynamically chosen from.

As part of planned planner refactor we should ensure that a filter is applied only once. The current implementation is very suboptimal for property filtering in variable expansions.

@buda: we will start refactoring this these days. This current planner logic is too dense and complex. It is becoming technical debt. Most of the time I spent working on this has been spent figuring the planning out, and I still needed Teon's help at times. Implementing the correct and optimal version of query execution (avoiding multiple potentially expensive filterings) was out of reach also due to tech debt.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D852
2017-10-05 13:12:39 +02:00
Matej Ferencevic
d249ff11a5 Misc glog and gflags fixes.
Summary:
Log files aren't created by default anymore.
All logs are reported to stderr by default.
Normalized flag names.
Removed unnecessary flags from gflags.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D856
2017-10-04 17:29:53 +02:00
florijan
0370e83b4d mvcc::Hints reduction
Summary: It's a sketch. I can tidy it up if you like this approach.

Reviewers: mislav.bradac, mferencevic

Reviewed By: mislav.bradac

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D841
2017-10-03 16:56:06 +02:00
Matej Ferencevic
a40248ab65 Removed filename strings from release binary.
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D839
2017-10-03 11:06:42 +02:00
florijan
5d2ee8f031 AST - pattern atom properties changed from map to unordered_map
Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D844
2017-10-02 14:24:56 +02:00
Matej Ferencevic
30573d4fd5 Removed chdir from main binary.
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D842
2017-10-02 14:00:08 +02:00
Matej Ferencevic
b76a5f8c79 Changed 'maybe_unused' to 'gnu::unused'.
Reviewers: teon.banek, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D843
2017-10-02 13:59:35 +02:00
Mislav Bradac
7dedb88449 Minor cleanup
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D840
2017-09-27 17:25:00 +02:00
florijan
03c78c1277 BFS syntax changes
Summary:
- The new BFS syntax implemented as proposed.
- AST BreadthFirstAtom now uses EdgeAtom members: has_range_{true}, upper_bound_, lower_bound_
- Edges data structure now handles all the edge filtering (single or multiple edges), to ease planning. Additional edge filtering (additional Filter op in the plan) is removed. AST EdgeTypeTest is no longer used and is removed.

Current state is stable but there are things left to do:
- BFS property filtering.
- BFS lower_bound_ support.
- Support for lambdas in variable length expansion. This includes obligatory (even if not user_defined) inner_node and inner_edge symbols for easier handling.
- Code-sharing between BFS and variable length expansions.

I'll add asana tasks (and probably start working on them immediately) when/if this lands.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D836
2017-09-27 16:25:19 +02:00
Mislav Bradac
3140f175fc Implement record lock deadlock breaker
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D823
2017-09-27 16:11:05 +02:00
Mislav Bradac
afff458afa Small cleanup of some mvcc classes
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D835
2017-09-27 15:16:30 +02:00
Mislav Bradac
56017e598e Abort all transactions on SIGTERM
Reviewers: buda, teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D818
2017-09-27 10:56:44 +02:00
Mislav Bradac
4e4ed063fc Fix memory leak in InsertEdge
Summary:
Few days ago I put a task to reduce number of exceptions in memgraph codebase :)
I think we should use exceptions only in modules where we are using  smart pointers.

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D834
2017-09-27 10:55:11 +02:00
Teon Banek
8c9ff2d79a Add version.hpp.in and display it on '--version' flag
Reviewers: mferencevic, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D831
2017-09-27 09:04:44 +02:00
Teon Banek
eb821aa615 Remove ast-cache flag
Summary:
AST caching should be well tested by now.
We should consider removing `Context.is_query_cached_` member as well as the
implementation and tests for `CypherMainVisitor`.

Reviewers: mislav.bradac, mferencevic, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D833
2017-09-27 09:04:31 +02:00
Teon Banek
b85ce7dfce Remove some unused variables and lambda captures
Reviewers: mferencevic, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D832
2017-09-27 09:04:15 +02:00
Teon Banek
9d4ae833cc Setup basic installation for CPackDeb
Summary:
Remove traces of plan_compiler in cmake.
Add memgraph executable to install target.
Fix indentation in tests/CMakeLists.txt.
Install alpha.conf to /etc/memgraph/config.
Rename executable to 'memgraph' and install /var dirs.

Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D829
2017-09-26 14:54:13 +02:00
florijan
a9381df09e Edges data structure now supports multiple edge filtering (implicit OR)
Summary: - modified all utils/algorithm functions to be inline and in the utils namespace

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D830
2017-09-26 13:46:18 +02:00
florijan
d9503d6b65 Flags cleanup and QueryEngine removal
Summary:
I started with cleaning flags up (removing unused ones, documenting undocumented ones). There were some flags to remove in `QueryEngine`. Seeing how we never use hardcoded queries (AFAIK last Mislav's testing also indicated they aren't faster then interpretation), when removing those unused flags the `QueryEngine` becomes obsolete. That means that a bunch of other stuff becomes obsolete, along with the hardcoded queries. So I removed it all (this has been discussed and approved on the daily).

Some flags that were previously undocumented in `docs/user_technical/installation` are now documented. The following flags are NOT documented and in my opinion should not be displayed when starting `./memgraph --help` (@mferencevic):
```
query_vertex_count_to_expand_existsing (from rule_based_planner.cpp)
query_max_plans (rule_based_planner.cpp)
```

If you think that another organization is needed w.r.t. flag visibility, comment.

@teon.banek: I had to remove some stuff from CMakeLists to make it buildable. Please review what I removed and clean up if necessary if/when this lands. If the needed changes are minor, you can also comment.

Reviewers: buda, mislav.bradac, teon.banek, mferencevic

Reviewed By: buda, mislav.bradac

Subscribers: pullbot, mferencevic, teon.banek

Differential Revision: https://phabricator.memgraph.io/D825
2017-09-22 17:03:45 +02:00