Commit Graph

706 Commits

Author SHA1 Message Date
Teon Banek
e0474a8e92 Replace boost with capnp in RPC
Summary:
Converts the RPC stack to use Cap'n Proto for serialization instead of
boost. There are still some traces of boost in other places in the code,
but most of it is removed. A future diff should cleanup boost for good.

The RPC API is now changed to be more flexible with regards to how
serialize data. This makes the simplest cases a bit more verbose, but
allows complex serialization code to be correctly written instead of
relying on hacks. (For reference, look for the old serialization of
`PullRpc` which had a nasty pointer hacks to inject accessors in
`TypedValue`.)

Since RPC messages were uselessly modeled via inheritance of Message
base class, that class is now removed. Furthermore, that approach
doesn't really work with Cap'n Proto. Instead, each message type is
required to have some type information. This can be automated, so
`define-rpc` has been added to LCP, which hopefully simplifies defining
new RPC request and response messages.

Specify Cap'n Proto schema ID in cmake

This preserves Cap'n Proto generated typeIds across multiple generations
of capnp schemas through LCP. It is imperative that typeId stays the
same to ensure that different compilations of Memgraph may communicate
via RPC in a distributed cluster.

Use CLOS for meta information on C++ types in LCP

Since some structure slots and functions have started to repeat
themselves, it makes sense to model C++ meta information via Common Lisp
Object System.

Depends on D1391

Reviewers: buda, dgleich, mferencevic, mtomic, mculinovic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1407
2018-06-04 10:45:12 +02:00
Teon Banek
e56ed0acce Add generating Capnp schema in LCP
Summary:
Add additional structs and functions for handling C++ meta information.
Add capnp-file and capnp-id arguments to lcp:process-file.
Generate cpp along with hpp and capnp in lcp.
Wrap LogicalOperator base class in lcp:define-class.
Modify logical operators for capnp serialization.
Add query/common.capnp.

Reviewers: mculinovic, buda, mtomic, msantl, ipaljak, dgleich, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1391
2018-05-30 16:14:54 +02:00
Teon Banek
c7b6cae526 Extract io/network into mg-io library
Reviewers: buda, dgleich, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1411
2018-05-30 14:58:41 +02:00
Dominik Gleich
c294127065 Auto generate vertex/edge ids
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1406
2018-05-25 16:03:12 +02:00
Dominik Gleich
63f90a453d Expose ID
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1400
2018-05-23 15:04:50 +02:00
Matija Santl
f872c93ad1 Add command id to remote produce
Summary:
Command id is necessary in remote produce to identify an ongoing pull
because a transaction can have multiple commands that all belong under
the same plan and tx id.

Reviewers: teon.banek, mtomic, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1386
2018-05-16 10:20:39 +02:00
Marin Tomic
91e38f6413 Distributed BFS
Summary: depends on D1387

Reviewers: msantl, teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1341
2018-05-15 17:38:51 +02:00
Marin Tomic
0a6b8cdf4f Remove AS_IS from GraphView
Summary:
Removing `AS_IS` from GraphView because it doesn't seem like it is necessary for query execution and it also has weird semantics (you might get a mix of old and new records). `Unwind`, `OrderBy` and `PullRemoteOrderBy` now use `OLD` graph view.

Remove AS_IS from GraphView

Fix query_cost_estimator tests

Fix query_expression_evaluator tests

Fix query_plan_match_filter_return tests

Fix query_plan_create_set_remove_delete tests

Fix query_plan_accumulate_aggregate tests

Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1390
2018-05-15 13:10:19 +02:00
Teon Banek
30506f44f5 Use LCP to generate LogicalOperator boost serialization
Summary:
Since we are moving from boost to Capnp for serialization, it makes
sense to keep all of the LogicalOperator classes in LCP format. This
will make it easier to generate Capnp code.

Depends on D1361

Reviewers: buda, mferencevic, msantl, dgleich, ipaljak, mculinovic, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1362
2018-05-10 09:56:40 +02:00
Marko Culinovic
49d69560e6 Serialize cpp types with capnp
Summary: Serialize shared_ptr, unique_ptr, optional and vector.

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D1364
2018-05-04 15:02:38 +02:00
Marko Culinovic
4b9970ccf8 Save and load pointers in ast capnp serialization
Summary:
Pointers in AstTreeStorage can point to same nodes. These nodes
should be serialized(saved) only once when serializing Ast tree.
Same goes for deserializing(loading).
When deserializing nodes, one should not use storage Create method,
because it will add node which isn't completely loaded to storage vector.
Therefore, one should use new Deserialize method which doesn't
add node to storage vector. Inserting node into storage vector
is defered until all node data is loaded.

Pass pointer to saved uids set, instead of storage

Delete unused set

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1344
2018-05-04 09:50:34 +02:00
Marko Culinovic
8795501890 Serialize ast using capnproto
Reviewers: teon.banek, florijan, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1306
2018-05-03 14:12:38 +02:00
Teon Banek
ab574bf84f Benchmark Cap'n Proto and boost serialization
Reviewers: florijan, mferencevic, buda, mculinovic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1273
2018-05-02 12:38:18 +02:00
Marko Budiselic
c76170a9db Clean utils folder (namespaces, function names)
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1359
2018-04-22 09:44:32 +02:00
Matija Santl
5c7d3a908f Add two state dijkstra for wsp implementation
Summary:
Added a new markdown file for concepts and a new test to test the edge
case with upper bound.

Reviewers: teon.banek, mtomic, dgleich, buda, ipaljak

Reviewed By: teon.banek, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1366
2018-04-20 16:10:42 +02:00
Matija Santl
0bcf2edeae Two phase commit on cursor destruction
Summary:
When commiting/aborting a transaction in tx master engine, make a two
phase commit to all workers so they can stop all futures and clear
transactional cache.

Reviewers: dgleich, florijan

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1320
2018-04-03 16:20:00 +02:00
florijan
a88c598822 Use wait-in-destruct future everywhere
Summary:
Before we used `utils::Future` only where it's created by our `ThreadPool`.
I suggest in this diff that we use it everywhere, it's a bit more defensive and
should not have any downsides.

Reviewers: msantl, teon.banek

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1316
2018-03-26 14:15:11 +02:00
Matija Santl
29ba055b64 Add custom VLOGs for distributed memgraph
Summary:
Add different priority VLOGs for distributed memgraph.

For level 3 you'll get logs for dispatching/consuming plans.
For level 4 you'll get logs for tx start/commit/abort, remote produce, remote
pull, remote result consume,
For level 5 there will be a log for each request/response made by the RPC
client.

Master log snippet P9
Worker log snippet P10

Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1296
2018-03-26 09:24:39 +02:00
florijan
ac8c96ccc2 Tidyup distributed stuff naming
Summary:
Remove "produce_" and "Produce" as prefix from all distributed stuff.
It's not removed in src/query/ stuff (operators).

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1315
2018-03-23 16:32:29 +01:00
florijan
95f1b6fe56 Fix remote plan invalidation
Summary:
Ensures that query plans are invalidated on the remote only when it's
guaranteed they will never be used on the master. This is done by
invalidating remote caches in the `CachedPlan` destructor.

There are two unplesant side-effects. First, an RPC call is made in an
object destructor. This is somewhat ugly, but not that different then
making an RPC call that must succeed in any other function. Note that
this does NOT slow down any query execution because the relevant
destructor is called by the skiplist garbage collector. The second ugly
side-effect is that in the unit test now we need to sleep to ensure the
skiplist GC destructs a cached plan before checking that it's
invalidated on the remote worker.

We might want to redesign this at some point.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1302
2018-03-16 11:18:16 +01:00
florijan
03f1547a8d Invalidate distributed plan caches
Summary:
 - Remove caches on workers as a result of plan expiration or race
   during insertion.
 - Extract caching functionality into a class.
 - Minor refactor of Interpreter::operator()
 - New RPC and test for it.
 - Rename ConsumePlanRes to DispatchPlanRes for consistency, remove
   return value as it's always true and never used.
 - Interpreter is now constructed with a `GraphDb` reference. At the
   moment only for reaching the `distributed::PlanDispatcher`, but in
   the future we should probably use that primarily for planning.

I added a function to `PlanConsumer` that is only used for testing.
I prefer not doing this, but I felt this needed testing. I can remove
it now if you like.

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1292
2018-03-13 14:59:51 +01:00
Teon Banek
226992f420 Check values can be used for range indexed ScanAll
Summary: Test indexing ScanAll with invalid property values

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1293
2018-03-13 10:23:30 +01:00
florijan
67092ae4d7 Fix ScanAll cursor family w.r.t. empty collections
Reviewers: teon.banek, dgleich, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1291
2018-03-12 11:18:07 +01:00
florijan
42ca81eb01 Use custom future that waits on destruct
Reviewers: teon.banek, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1286
2018-03-09 10:16:17 +01:00
florijan
848749cf25 Fix and rename REPL
Summary: Fix and rename REPL

Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D1284
2018-03-08 10:58:43 +01:00
Teon Banek
9c95d4c381 Allow replacing Accumulate with Synchronize in all cases
Summary:
This fixes a bug where the planning would raise NotYetImplemented error,
due to preventing plan splitting while the operator is already on
master. For example, `MATCH (a), (b) CREATE (a)-[e:r]->(b) RETURN e`
would split the plan after Cartesian and before Create. Thus, the rest
of the plan would be on master, including the Accumulate before Produce.
We still can (and must) replace Accumulate with Synchronize but this
would fail due to unneeded check.

Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1272
2018-03-05 15:35:36 +01:00
florijan
43c0e91057 Enable transaction killer on worker
Summary:
I have not thoroughly thought this through, especially the worker
destruction (is it legit to abort all running tx?), but it's tested to
abort during remote pull, what we need.

Also I improved error handling for vertex deletion failure during
remote pull (@dgleich).

Reviewers: teon.banek, msantl, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1263
2018-03-05 12:45:42 +01:00
Matija Santl
418db646c1 Abort from remote pulls if db says so
Summary:
If the `db` says we need to abort we should abort. There are two
places (in both `PullRemote` and `PullRemoteOrderBy`) where the check is made.
The first one is on the very beginning of the `Pull` method and the second one
is in the loop that checks/waits for remote results.

Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1265
2018-03-02 14:23:02 +01:00
Teon Banek
f963c54f77 Use 10us sleep while waiting on RPC
Summary:
Usually a single RPC call takes 50us, which is a lot lower than our
millisecond sleep. Calling sleep with less than a ms duration may not
be really supported (due to timer granularity), but this change should
cause some sub-ms sleep.

Reviewers: florijan, msantl, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1258
2018-03-01 14:50:36 +01:00
Dominik Gleich
99375a4b47 Vertex removal using rpcs
Summary:
Remove vertex remote

Add tests

Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1230
2018-02-28 11:35:44 +01:00
Teon Banek
f1aece8352 Visit PullRemote's input only if it has any
Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1239
2018-02-26 13:18:10 +01:00
Teon Banek
a44d9a0fac Always plan Synchronize if the plan needs it
Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1241
2018-02-26 13:16:08 +01:00
Teon Banek
30d2cfb9db Plan PullRemoteOrderBy
Summary:
During distributed execution, OrderBy is split across workers and the
master gets to merge those results via PullRemoteOrderBy. Since this
operator may be an input to almost any other operator, virtual accessors
to `input` have been added in LogicalOperator.

Depends on D1221

Reviewers: florijan, msantl, buda

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1232
2018-02-26 11:15:46 +01:00
Matija Santl
b9d61a0127 Implement distributed aware OrderBy operator
Summary:
Extracted `TypedValueVectorCompare` and `RemotePuller` from operators
so it can be reused.
The new `PullRemoteOrerBy` operator pulls one result from each worker and one
from master, relies on the fact that workers/master returned sorted results,
returns the next one in order, and pulls the source of that result to get the
next one.

Depends on D1215 that (at the moment) is still in review.

Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1221
2018-02-23 17:44:21 +01:00
florijan
cbc9420c17 Support distributed query::plan::CreateExpand
Summary:
- The expansion vertex gets created on the origin's worker
- The edge automatically gets created wherever necessary
- Vertex creation logic reuse
- End to end test

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1227
2018-02-23 17:34:01 +01:00
florijan
80b86f0314 Add test/manual/distributed_console
Summary:
- Add the said test
- Add `workerid` function
- Add random wait (enabled with flag) to `rpc::Client::Call` for network
  latency simulation

Reviewers: buda, mferencevic, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1222
2018-02-23 15:23:26 +01:00
Teon Banek
452ce6a30c Use std::async for fetching remote edges in Expand
Summary:
This may improve the query execution workload, because the Expand will
yield local results while it waits for remote ones. Note that we rely on
the fact that walking the graph produces results without any predetermined
order. Therefore, we can yield paths as we see fit. Enforcing the order
is done through OrderBy operator, and this change shouldn't affect that.

Reviewers: florijan, msantl, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1215
2018-02-22 17:11:53 +01:00
Teon Banek
c7eaf4711a Pass a flag for distributed CreateNode
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1218
2018-02-22 14:25:46 +01:00
florijan
ae95e9480c Use RemoteCreateVertex in CreateNode operator
Summary:
- Add `database::GraphDb::GetWorkerIds()`
- Change `CreateNode` constructor API
- Make `CreateNode` distribute nodes uniformly over workers

Did not yet modify `CreateExpand`, coming in the next diff.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1216
2018-02-21 12:17:43 +01:00
Teon Banek
2055176139 Plan basic distributed Cartesian without any checks
Summary:
This is still very much in progress.  No advanced checks are done to
prevent planning unimplemented things.  Basic Cartesian product should
work, for example `MATCH (a), (b) CREATE (a)-[:r]->(c)-[:r]->(b)`. But
anything more advanced may lead to undefined behaviour of the planner
and therefore execution. Use at your own risk!

Add ModifiedSymbols method to LogicalOperator

For planning Cartesian, we need information on which symbols are filled
by operator sub-trees.  Currently, this is used to set symbols which
should be transferred over network. Later, they should be used to detect
whether filter expressions use symbols modified from Cartesian branches.
Then we will be able to ensure correct dependency of filters and their
behaviour.

Prepare DistributedPlan for multiple worker plans

Since Cartesian branches need to be split and handled by each worker, we
now dispatch multiple plans to workers.

Reviewers: florijan, msantl, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1208
2018-02-20 13:02:08 +01:00
Teon Banek
8856682c7f Always plan Synchronize for write queries
Reviewers: florijan, msantl

Reviewed By: florijan, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1179
2018-02-20 09:55:42 +01:00
Matija Santl
4225d5c871 Fix Cartesian and Synchronize operator serialization
Summary: ooops

Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1212
2018-02-20 09:03:25 +01:00
florijan
9721ccf61c Cleanup per-transaction caches in distributed
Summary:
On the master cleanups are hooked directly into the transaction engine.
This is beneficial because the master might have bigger caches and we
want to clear them as soon as possible.

On the workers there is a periodic RPC call to the master about living
transactions, which takes care of releasing local caches. This is
suboptimal because long transactions will prevent cache GC (like with
data GC). It is however fairly simple.

Note that all cleanup is not done automatically and `RemotePull` has
been reduced accordingly. @msantl, please verify correctness and
consider if the code can be additionally simplified.

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot, msantl

Differential Revision: https://phabricator.memgraph.io/D1202
2018-02-16 15:30:05 +01:00
Matija Santl
84e7aec27b Implement Cartesian Cursor
Summary:
Pulls left op cursor and keeps the result, and then for each pull of
the right op cursor, adds all the left op results to produce a cartesian
product.

Reviewers: teon.banek, florijan

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1201
2018-02-16 13:43:12 +01:00
Teon Banek
b8acefb1e3 Add Cartesian operator stub
Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1199
2018-02-15 12:45:14 +01:00
Matija Santl
9eb01f363f Add sleep in remote pull operator
Summary:
Added a hidden flag, a int32 value, that represents the number of
milliseconds the thread that pulls remote results should sleep in case when the
local results are exhausted but there are still some remote results that are
pending.

Reviewers: teon.banek, florijan

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1198
2018-02-15 10:21:34 +01:00
florijan
796946ad1b Implement sync operator
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1192
2018-02-14 13:03:18 +01:00
Teon Banek
0616eb0aa4 Remove pull_local flag from PullRemote
Summary:
The same behaviour can now be achieved by passing a nullptr for input
operator.

Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1195
2018-02-14 12:56:44 +01:00
Matija Santl
b1a8e48dc4 Add ExpandWeightedShortestPathCursor
Summary: Implement Dijkstras algorithm to find weighted shortest path.

Reviewers: teon.banek, florijan, buda

Reviewed By: teon.banek

Subscribers: pullbot, mtomic

Differential Revision: https://phabricator.memgraph.io/D1182
2018-02-14 09:32:41 +01:00
Teon Banek
4326847ab3 Add SINGLE function to openCypher
Reviewers: florijan, msantl, buda

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1193
2018-02-13 17:30:44 +01:00
Teon Banek
e8608f4050 Request correct remote symbols for non-associative aggregation
Reviewers: florijan, msantl, mculinovic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1191
2018-02-13 15:39:09 +01:00
Teon Banek
c1e4676316 Add optional total_weight variable to wShortest grammar
Reviewers: florijan, msantl, buda

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1187
2018-02-12 13:23:17 +01:00
Teon Banek
db407142ce Add wShortest expansion to openCypher
Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1181
2018-02-09 11:02:41 +01:00
Matija Santl
013ee34f90 Fix RemotePull infinite loop when all workers don't have results
Reviewers: buda, teon.banek, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1184
2018-02-08 16:29:34 +01:00
florijan
45da645a4b Fix remote pull
Summary:
- Extracted TypedValue reconstruction into utils (will need that soon)
- Added more error handling in RemotePull

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1183
2018-02-08 15:00:40 +01:00
Teon Banek
fa55c130ba Add REDUCE function to openCypher
Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1175
2018-02-07 12:48:20 +01:00
Teon Banek
d05d73f13a Fix distributed plan caching and dispatching race condition
Summary:
Distributed plan is now dispatched before inserting into a cache. This
prevents a race condition when another thread would take the cached plan
and assume it was dispatched, thus crashing during execution.

The fix introduces a possibility that the same plan may be distributed
and dispatched twice (with different plan IDs). This shouldn't raise an
issue during execution, because only one plan will be cached and thus
executed. Obviously, we still need to invalidate the unused plan from
worker's caches.

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1178
2018-02-07 12:48:04 +01:00
Matija Santl
a66351c3f4 Make RemotePull operator async
Reviewers: teon.banek, florijan

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1168
2018-02-06 13:42:11 +01:00
Teon Banek
abe419984a Plan Unwind and write operations in distributed
Summary:
This is a basic take on planning distributed writes. Main logic is in handling
the Accumulate operator, which requires Synchronize operation if the results
are used after modification.

Tests for planning distributed writes have been added.

Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1172
2018-02-06 10:13:53 +01:00
Teon Banek
583f6f1794 Add Synchronize operator stub
Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1166
2018-02-02 11:21:15 +01:00
Teon Banek
c2d0348333 Use make_unique instead of new when planning operators
Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1160
2018-02-01 13:05:28 +01:00
Teon Banek
27a90a5c25 Support distributed CreateIndex plan
Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1162
2018-02-01 13:05:11 +01:00
florijan
e5035cf477 Support graph elements in remote pull rpc
Reviewers: teon.banek, dgleich, msantl, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1157
2018-02-01 10:53:41 +01:00
Teon Banek
03e571ea3a Remember only symbols from group by during aggregation
Summary:
This avoids the unnecessary work of storing symbols which can never be
read after an aggregation is complete. With regards to distributed, a
major benefit is gained in reducing what is transferred over the
network. Hopefully, this doesn't break some obscure case where we
actually needed to remember all used symbols.

Reviewers: florijan, mislav.bradac, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1158
2018-01-31 16:16:05 +01:00
Teon Banek
471d63ad86 Don't distribute single execution queries
Summary:
Queries such as `RETURN 1` should only be run on a single machine. This
change assumes that a query should only be distributed if it contains at
least one `ScanAll` operator, i.e. a `MATCH` clause.

Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1154
2018-01-30 11:07:26 +01:00
florijan
3f30a6fad9 Add async remote pull RPC
Summary:
Remote pulls can now be async. Note that
`RemotePullRpcClients::RemotePull` takes references to data structures
which should not be temporary in the caller. Still, maybe safer to make
copies?

Changed `RpcWorkerClients` API to make that possible.

Reviewers: dgleich, msantl, teon.banek

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1150
2018-01-30 10:32:20 +01:00
Teon Banek
d41ffb5039 Add planning distributed execution of AVG
Reviewers: florijan, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1151
2018-01-29 16:24:51 +01:00
Teon Banek
760c6246d8 Make and dispatch worker plans on distributed master
Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1144
2018-01-29 10:55:57 +01:00
Matija Santl
a73a4c3762 Implement PullRemote logical operator
Summary: PullRemoteCursor will pull all clients in a RoundRobin fashion until all clients are exhausted and there are no more results to return.

Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1147
2018-01-26 17:12:45 +01:00
florijan
ccefae4002 Add distributed execution RPC
Summary:
NOTE: This diff is still in progress. Many TODOs, lacking documentation
etc. But the main logic is there (some could be different), and it tests
out OK.

Reviewers: teon.banek, msantl, buda

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1138
2018-01-25 14:50:28 +01:00
Teon Banek
dbc9c995c9 Plan distributed execution of basic read operators
Summary:
Remove AdvanceCommand operator declaration.
Add query/plan/distributed source files.
Add hacked cloning of LogicalOperator via serialization.
Add virtual Default methods to CompositeVisitor.

Use BOOST_CLASS_EXPORT_KEY in ast.hpp and operator.hpp.

This is needed in a single binary to correctly serialize polymorphic
classes. The previous implementation worked because the memgraph_lib was
linked with test binaries, but nobody was actually serializing things
inside the single memgraph binary itself.

Print PullRemote symbols in tests/manual/query_planner

Add names to implicitly created aggregation symbols

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1137
2018-01-25 14:20:51 +01:00
Teon Banek
1d6ac3d23d Add ProduceRemote and PullRemote operator stubs
Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1124
2018-01-22 15:08:32 +01:00
Teon Banek
252018ab22 Serialize query plan operators
Summary:
With the added support for serialization, we should be able to transfer
plans across distributed workers.

The planner tests has been extended to test serialization. Operators
should be mostly tested, but the expression they contain aren't
completely. The quick solution is to use typeid for rudimentary
expression equality testing.  The more involved solution of comparing
the expression tree for equality is the correct choice. It should be
done in the near future.

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1122
2018-01-22 12:48:05 +01:00
Teon Banek
2a8c64882f Serialize SymbolTable
Summary:
Other than the plan operators and the frame, we will need to pass the
generated symbol table to distributed workers.

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1123
2018-01-22 12:47:51 +01:00
Teon Banek
93de41e717 Throw RecordDeletedError if updating a deleted record
Summary:
Previously, we would have a `DCHECK` which crashes the application. This
was evident when testing a queries, such as:

    MATCH (n) DELETE n SET n.prop = 42

Since the argument to update clauses is evaluated during execution, it
makes it very difficult to prevent such errors during semantic analysis.
For example:

    MATCH (n)--(m) WITH collect(n) as ns, m
    DETACH DELETE ns[m.prop] SET head(ns).prop = 42

Test query updates on deleted graph elements

Reviewers: florijan, dgleich

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1114
2018-01-17 14:30:57 +01:00
florijan
813d37e939 Migrate db::types to storage::
Reviewers: teon.banek, dgleich

Reviewed By: teon.banek, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1110
2018-01-17 10:35:12 +01:00
florijan
6fc6a27288 Refactor GraphDb
Summary:
GraphDb is refactored to become an API exposing different parts
necessary for the database to function. These different parts can have
different implementations in SingleNode or distributed Master/Server
GraphDb implementations.

Interally GraphDb is implemented using two class heirarchies. One
contains all the members and correct wiring for each situation. The
other takes care of initialization and shutdown. This architecture is
practical because it can guarantee that the initialization of the
object structure is complete, before initializing state.

Reviewers: buda, mislav.bradac, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1093
2018-01-12 16:47:24 +01:00
florijan
3cf81380b8 Implement vertex/edge serialization
Summary:
Serialization of vertices and edges for distributed. Based on Boost
serialization. Threrefore moved TypedValue serialization from AST to
utils.

Reviewers: buda, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1088
2018-01-10 13:32:18 +01:00
Mislav Bradac
d3623585e7 Migrate cereal to boost_serialization
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1077
2017-12-27 13:44:46 +01:00
florijan
7b3d298741 Fix interpreter plan ownership
Summary:
If there was no plan caching, the CachedPlan would not survive
`Interpreter::operator()`, as it was not owned by the
`Interpreter::Result`. If there was caching, it could hapen that the
cache got invalidated while that plan was being interpreted (by
another thread) without that interpretation retaining ownership.

Also simplified code around this.

Reviewers: mislav.bradac, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1084
2017-12-27 13:33:13 +01:00
florijan
9b878c91eb Make interpretation pullable
Reviewers: mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1079
2017-12-22 15:19:05 +01:00
Teon Banek
86b6f32ce6 Move load_construct_data from ast.cpp to ast.hpp
Reviewers: florijan, mislav.bradac, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1078
2017-12-21 13:08:16 +01:00
Teon Banek
5f7837d613 Serialize Ast classes
Summary:
Although the first solution used cereal, the final implementation uses
boost. Since the cereal is still used in the codebase, compilation has
been modified to support multithreaded cereal.

In addition to serializing Ast classes, the following also needed to be
serialized:

  * GraphDbTypes
  * Symbol
  * TypedValue

TypedValue is treated specially, by inlining the serialization code in
the Ast class, concretely PrimitiveLiteral.

Another special case was the Function Ast class, which now stores a
function name which is resolved to a concrete std::function on
construction.

Tests have been added for serialized Ast in
tests/unit/cypher_main_visitor

Reviewers: mferencevic, mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1067
2017-12-20 13:28:34 +01:00
Matija Santl
2758b35c08 Fix semicolon in column name
Summary:
Special casing the last token when adding it to the named expression list.
Added a test to check this case.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1052
2017-12-14 09:44:21 +01:00
Matija Santl
4da8312abe Fix query returning a map
Summary: Add curvy braces handling in `QueryStripper` and an accompanying test for it.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1051
2017-12-13 15:38:20 +01:00
Mislav Bradac
27370b99ff Fix Pattern::Accept
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1048
2017-12-13 13:15:22 +01:00
Matija Santl
9ad7d82a54 Implement UNION query combinator
Summary:
Union query combinator implementation consists of:
 * adjustments to the AST and `cypher_main_visitor`
 * enabling `QueryStripper` to parse multiple `return` statements (not stopping after first)
 * symbol generation for union results
 * union logical operator
 * query plan generator adjustments

Reviewers: teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D1038
2017-12-13 09:54:00 +01:00
florijan
f5c0455af4 Prepare record accessor for distributed
Summary:
What's done:
- `RecordAccessor` can represent remote data
- `GraphDbAccessor` manages remote data
- Cleanup: different `EdgeAccessor lazyness (@dgleich: take a look), unused methods, documentation...
- `TODO` placeholders for remote implementation

What's not done:
- RPC and data transfer
- how exactly remote errors are handled
- not sure if any MVCC Record info for remote data should be tracked
- WAL and RPC Deltas properly handled (Gleich working on extracting `Wal::Op`)

This implementation should not break single-node execution, and should provide good abstractions and placeholders for distributed. Once that's satisfied, it should land.

Reviewers: dgleich, buda, mislav.bradac

Reviewed By: dgleich

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D1030
2017-12-08 14:12:18 +01:00
Dominik Gleich
3ddbcad0d9 Refactor global ids and prepare for distributed
Summary:
Change ids to global ids

Fix tests

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1019
2017-12-05 13:05:55 +01:00
Teon Banek
a799351eb0 Correctly inspect property filters during planning
Summary:
This change generates multiple PropertyFilters for expressions such as
`n.prop1 = m.prop2`. When choosing one PropertyFilter, we want to also
remove the other one, because they represent the same original
expression.  Therefore, the removal is no longer based on FilterInfo
equality, but on the original expression equality. Additionally,
FilterInfo and PropertyFilter equality operators have been removed to
avoid any pretense they do what you expect or want.

Reviewers: florijan, msantl

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1021
2017-12-04 09:57:41 +01:00
Dominik Gleich
2b2de245d1 Allow concurrent index creation
Summary: Update tests

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1012
2017-11-28 16:44:12 +01:00
florijan
61e28bffd0 Fix return collection containing both aggregation and group-by
Summary:
Referring to the TCK failure on:
```
MATCH (a {name: 'Andres'})<-[:FATHER]-(child)
RETURN {foo: a.name='Andres', kids: collect(child.name)}
```

In the planner we'd only treat a list|map as a group_by if it contained
no aggregations. That's changed so that if a map contains both aggregations
and non-aggregations, then non-aggregations are treated as individual
group_by expressions.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: buda, pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1004
2017-11-24 14:43:48 +01:00
Dominik Gleich
67538aceeb Migrate labels/properties/edgetypes to ids
Summary:
In preparation for distributed storage we need to have labels/properties/edgetypes uniquely identifiable by their ids, which will be global in near future.
The old design has to be abandoned because it's not possible to keep track of global labels/properties/edgetypes while they are local pointers.

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D993
2017-11-23 17:12:37 +01:00
florijan
5ef2d20133 Add single query manual test
Summary:
Practical for debugging trivial queries:
```
~ gdb --args ./tests/manual/single_query "RETURN 2 + 1"

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D1003
2017-11-23 14:51:46 +01:00
florijan
597934203e Support explicit int->bool and bool->int
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D999
2017-11-22 12:11:16 +01:00
Matija Santl
df4cbdc5b2 Add error messsage for large integer literals
Summary:
Fixing https://app.asana.com/0/170237629387822/481366792497820/f

Test plan:
Started two builds, one with the fix and the second one without the fix.
Connected to each of them using `neo4j` client.

Logs received from the build wihtout the fix:
```
neo4j> Create (n: BigInteger{id:12345678912345678912345}) return n;
<interactive>:0:0: error:
```
```
I1120 13:29:09.551208 30482 executing.hpp:69] [Run] 'Create (n: BigInteger{id:12345678912345678912345}) return n'
W1120 13:29:09.552387 30482 executing.hpp:145] Error message:
```

Logs received from the build with the fix:
```
neo4j> Create (n: BigInteger{id:12345678912345678912345}) return n;
<interactive>:0:0: error: Integer literal exceeds 64 bits
```
```
I1120 13:29:07.940943 30453 executing.hpp:69] [Run] 'Create (n: BigInteger{id:12345678912345678912345}) return n'
W1120 13:29:07.942919 30453 executing.hpp:146] Error message: Integer literal exceeds 64 bits
```

Reviewers: mislav.bradac, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D997
2017-11-20 14:33:40 +01:00
Mislav Bradac
e703e955a5 Fix bug in InListOperator
Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D981
2017-11-14 15:39:10 +01:00
florijan
1e0ac8ab8f Write-ahead log
Summary:
My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed...

What's done:
- Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr.
- WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk.
- Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info.
- Durability testing completely reworked.

What's not done (and should be when we decide how):
- Old WAL file purging.
- Config refactor (naming and organization). Will do when we discuss what we want.
- Changelog and new feature documentation (both depending on the point above).
- Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment).
- Moving the implementation of WAL stuff to .cpp where possible.
- Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into.
- True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature.

Reviewers: buda, mislav.bradac, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D958
2017-11-13 09:51:39 +01:00
Teon Banek
93bc9accd8 Raise NotYetImplemented when encountering UNION
Summary:
We need to explicitly say that UNION clause isn't supported, otherwise
it gets silently ignored. So for example, `CREATE () UNION CREATE ()`
would create 2 nodes without a hitch. On the other hand,
`RETURN 1 UNION RETURN 2` would complain that there is more than 1
RETURN in the query, which was misleading.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D976
2017-11-10 15:10:02 +01:00
Teon Banek
796e4f50e5 Add named path symbols to new symbols of a match
Summary:
Previously, named path symbols remained untracked as `new_symbols` during planning. This meant that
operator `Optional` would be left unaware of those symbols, and therefore not reset them to `Null`
if optional matching failed.

Test Optional operator will be aware of path symbols

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D974
2017-11-10 15:09:47 +01:00
Dominik Gleich
f50d87f736 Take care of warnings/errors created by cppcheck.
Summary:
Warnings I ignored:

Creates a new stacktrace object and then dumps it
	102.570384 (102.495075) E[1]: [src/utils/exceptions.hpp:116]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.
	102.570390 (102.495081) E[1]: [src/utils/exceptions.hpp:127]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.

Used all over the codebase without explicit cast
102.570412 (102.495103) E[1]: [src/utils/stacktrace.hpp:14]: (style) Class 'Line' has a constructor with 1 argument that is not explicit.

Not really used anywhere before initialized:
102.570526 (102.495217) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::preds' is not initialized in the constructor.
102.570530 (102.495221) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::succs' is not initialized in the constructor.

Implicit conversions between types are used all over the codebase:
102.570548 (102.495239) E[1]: [src/storage/property_value.hpp:41]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570552 (102.495243) E[1]: [src/storage/property_value.hpp:42]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570557 (102.495248) E[1]: [src/storage/property_value.hpp:43]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570561 (102.495252) E[1]: [src/storage/property_value.hpp:44]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570566 (102.495257) E[1]: [src/storage/property_value.hpp:47]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570570 (102.495261) E[1]: [src/storage/property_value.hpp:50]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570578 (102.495269) E[1]: [src/storage/property_value.hpp:53]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570582 (102.495273) E[1]: [src/storage/property_value.hpp:57]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570591 (102.495282) E[1]: [src/query/typed_value.hpp:80]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570596 (102.495287) E[1]: [src/query/typed_value.hpp:81]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570601 (102.495292) E[1]: [src/query/typed_value.hpp:82]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570605 (102.495296) E[1]: [src/query/typed_value.hpp:83]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570609 (102.495300) E[1]: [src/query/typed_value.hpp:89]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570614 (102.495305) E[1]: [src/query/typed_value.hpp:92]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570618 (102.495309) E[1]: [src/query/typed_value.hpp:95]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570623 (102.495314) E[1]: [src/query/typed_value.hpp:98]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570627 (102.495318) E[1]: [src/query/typed_value.hpp:102]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570632 (102.495323) E[1]: [src/query/typed_value.hpp:105]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570636 (102.495327) E[1]: [src/query/typed_value.hpp:108]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570641 (102.495332) E[1]: [src/query/typed_value.hpp:109]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570645 (102.495336) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:88]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570650 (102.495341) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:89]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570654 (102.495345) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:90]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570659 (102.495350) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:91]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570663 (102.495354) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:94]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570668 (102.495359) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:97]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570672 (102.495363) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:100]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570677 (102.495368) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:104]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570681 (102.495372) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:107]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570690 (102.495381) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:110]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570694 (102.495385) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:113]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.

CypherParser:

102.570767 (102.495458) E[1]: [src/query/frontend/opencypher/generated/CypherParser.h:69]: (style) Class 'CypherParser' has a constructor with 1 argument that is not explicit.
102.570772 (102.495463) E[1]: [src/query/frontend/opencypher/generated/CypherLexer.h:40]: (style) Class 'CypherLexer' has a constructor with 1 argument that is not explicit.
102.570776 (102.495467) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:86]: (style) The scope of the variable '_la' can be reduced.
102.570781 (102.495472) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:311]: (style) The scope of the variable '_la' can be reduced.
102.570785 (102.495476) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:402]: (style) The scope of the variable '_la' can be reduced.
102.570789 (102.495480) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:497]: (style) The scope of the variable '_la' can be reduced.
102.570797 (102.495488) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:778]: (style) The scope of the variable '_la' can be reduced.
102.570802 (102.495493) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:895]: (style) The scope of the variable '_la' can be reduced.
102.570806 (102.495497) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:991]: (style) The scope of the variable '_la' can be reduced.
102.570811 (102.495502) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1190]: (style) The scope of the variable '_la' can be reduced.
102.570815 (102.495506) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1274]: (style) The scope of the variable '_la' can be reduced.
102.570820 (102.495511) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1393]: (style) The scope of the variable '_la' can be reduced.
102.570824 (102.495515) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1570]: (style) The scope of the variable '_la' can be reduced.
102.570829 (102.495520) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1695]: (style) The scope of the variable '_la' can be reduced.
102.570834 (102.495525) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1800]: (style) The scope of the variable '_la' can be reduced.
102.570839 (102.495530) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1903]: (style) The scope of the variable '_la' can be reduced.
102.570843 (102.495534) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2019]: (style) The scope of the variable '_la' can be reduced.
102.570848 (102.495539) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2228]: (style) The scope of the variable '_la' can be reduced.
102.570852 (102.495543) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2542]: (style) The scope of the variable '_la' can be reduced.
102.570857 (102.495548) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2797]: (style) The scope of the variable '_la' can be reduced.
102.570861 (102.495552) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2966]: (style) The scope of the variable '_la' can be reduced.
102.570866 (102.495557) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3067]: (style) The scope of the variable '_la' can be reduced.
102.570870 (102.495561) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3289]: (style) The scope of the variable '_la' can be reduced.
102.570875 (102.495566) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3295]: (style) The scope of the variable 'alt' can be reduced.
102.570879 (102.495570) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3419]: (style) The scope of the variable '_la' can be reduced.
102.570884 (102.495575) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3596]: (style) The scope of the variable '_la' can be reduced.
102.570888 (102.495579) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3688]: (style) The scope of the variable '_la' can be reduced.
102.570893 (102.495584) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3963]: (style) The scope of the variable '_la' can be reduced.
102.570897 (102.495588) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4452]: (style) The scope of the variable '_la' can be reduced.
102.570902 (102.495593) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4586]: (style) The scope of the variable '_la' can be reduced.
102.570906 (102.495597) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4813]: (style) The scope of the variable '_la' can be reduced.
102.570911 (102.495602) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4943]: (style) The scope of the variable '_la' can be reduced.
102.570918 (102.495609) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5026]: (style) The scope of the variable '_la' can be reduced.
102.570923 (102.495614) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5569]: (style) The scope of the variable '_la' can be reduced.
102.570928 (102.495619) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5664]: (style) The scope of the variable '_la' can be reduced.
102.570932 (102.495623) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5755]: (style) The scope of the variable '_la' can be reduced.
102.570937 (102.495628) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5888]: (style) The scope of the variable '_la' can be reduced.
102.570941 (102.495632) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6045]: (style) The scope of the variable '_la' can be reduced.
102.570946 (102.495637) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6142]: (style) The scope of the variable '_la' can be reduced.
102.570950 (102.495641) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6347]: (style) The scope of the variable '_la' can be reduced.
102.570955 (102.495646) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6523]: (style) The scope of the variable '_la' can be reduced.
102.570959 (102.495650) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6614]: (style) The scope of the variable '_la' can be reduced.
102.570964 (102.495655) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6899]: (style) The scope of the variable '_la' can be reduced.
102.570968 (102.495659) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6992]: (style) The scope of the variable '_la' can be reduced.
102.570973 (102.495664) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7147]: (style) The scope of the variable '_la' can be reduced.
102.570977 (102.495668) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7680]: (style) The scope of the variable '_la' can be reduced.
102.570982 (102.495673) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7759]: (style) The scope of the variable '_la' can be reduced.
102.570986 (102.495677) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7938]: (style) The scope of the variable '_la' can be reduced.
102.570991 (102.495682) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8126]: (style) The scope of the variable '_la' can be reduced.
102.570995 (102.495686) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8220]: (style) The scope of the variable '_la' can be reduced.
102.571000 (102.495691) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8313]: (style) The scope of the variable '_la' can be reduced.
102.571004 (102.495695) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8491]: (style) The scope of the variable '_la' can be reduced.
102.571009 (102.495700) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8703]: (style) The scope of the variable '_la' can be reduced.
102.571013 (102.495704) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8783]: (style) The scope of the variable '_la' can be reduced.
102.571018 (102.495709) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8914]: (style) The scope of the variable '_la' can be reduced.
102.571022 (102.495713) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9119]: (style) The scope of the variable '_la' can be reduced.
102.571027 (102.495718) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9220]: (style) The scope of the variable '_la' can be reduced.
102.571034 (102.495725) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9414]: (style) The scope of the variable '_la' can be reduced.
102.571039 (102.495730) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9660]: (style) The scope of the variable '_la' can be reduced.
102.571043 (102.495734) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10008]: (style) The scope of the variable '_la' can be reduced.
102.571048 (102.495739) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10158]: (style) The scope of the variable '_la' can be reduced.
102.571052 (102.495743) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10250]: (style) The scope of the variable '_la' can be reduced.
102.571057 (102.495748) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10370]: (style) The scope of the variable '_la' can be reduced.
102.571061 (102.495752) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10637]: (style) The scope of the variable '_la' can be reduced.
102.571065 (102.495756) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10749]: (style) The scope of the variable '_la' can be reduced.
102.571070 (102.495761) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10815]: (style) The scope of the variable '_la' can be reduced.
102.571075 (102.495766) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10881]: (style) The scope of the variable '_la' can be reduced.

We know that we represented it correctly in memory:
102.571079 (102.495770) E[1]: [src/communication/bolt/v1/decoder/decoder.hpp:252]: (portability) Casting between integer* and double* which have an incompatible binary data representation.

Cont assigned but not used after:
102.571101 (102.495792) E[1]: [src/query/frontend/ast/ast.hpp:1008]: (style) Variable 'cont' is assigned a value that is never used.

Reviewers: teon.banek, buda, mferencevic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D967
2017-11-10 10:28:02 +01:00
Teon Banek
7fabdda938 Fix an error where null AND false returned null
Summary:
Tests have been updated to catch this error and other behaviour. Other
than this change, `AND` should behave as before.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D970
2017-11-09 16:46:25 +01:00
Teon Banek
4b8076f41a Fix issues with lambda symbols and returning *
Summary:
Correctly propagate user declared symbols for lambda.
Unbind lambda symbols after inlining filters.
Also update unit tests.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D971
2017-11-09 16:46:06 +01:00
Dominik Gleich
09ad52958f POC - Typed value assignment for different types.
Summary: TypedValue assignment operator=(TypedValue &other) is inefficient for types which already have a defined assignment operator, or can be assigned trivially, without using TypedValue to pass the value for assignment.

Reviewers: mislav.bradac, florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D934
2017-11-02 14:21:22 +01:00
Mislav Bradac
00f76d1391 Add nodes and relationships functions
Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D942
2017-10-30 12:38:12 +01:00
Teon Banek
55456b4214 Remove Dbms
Summary:
Remove name from GraphDb.
Take GraphDb in query test macros instead of accessor.
Add is_accepting_transactions flag to GraphDb.

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D940
2017-10-30 12:33:29 +01:00
Teon Banek
db34cb2a40 Refactor collecting filters during planning
Summary:
Move QueryParts and Filters to a new file.
Reorganize FilterInfo struct.
Remove label filter if we do indexed scan by label.
Remove property filter used in indexed scan.

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D915
2017-10-24 14:13:14 +02:00
Teon Banek
8da3839a1d Convert variable start planning to iterative process
Summary: This change increases the planning time, but should reduce memory consumption.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D901
2017-10-16 10:12:09 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Mislav Bradac
e078c9f7cd Refactor Interpreter
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D896
2017-10-11 11:06:25 +02:00
Mislav Bradac
31798eb957 Handle index creation correctly
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D891
2017-10-09 19:22:39 +02:00
Mislav Bradac
bac99cc267 Return TransientError when possible
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D889
2017-10-09 17:58:54 +02:00
Matej Ferencevic
37b2caae63 Implemented hidden flags.
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D861
2017-10-07 15:08:48 +02:00
Mislav Bradac
c3a211fd4e Add message to HintedAbortError
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D868
2017-10-06 14:39:50 +02:00
florijan
d0fb4d7242 BFS - lower bound support
Summary: This is not a very important functionality, but it turned out simple to do, so let's add it to have a consistent query support.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D862
2017-10-06 09:33:20 +02:00
Mislav Bradac
1f01ef6cd1 Rerun antlr on original query to generate appropriate error message
Reviewers: buda, florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D858
2017-10-05 13:35:39 +02:00
florijan
76fe8bfadf Variable expansion consolidaton
Summary:
- Removed BreadthFirstAtom, using EdgeAtom only with a Type enum.
- Both variable expansions (breadth and depth first) now have mandatory inner node and edge Identifiers.
- Both variable expansions use inline property filtering and support inline lambdas.
- BFS and variable expansion now have the same planning process.
- Planner modified in the following ways:
	- Variable expansions support inline property filtering (two filters added to all_filters, one for inline, one for post-expand).
	- Asserting against existing_edge since we don't support that anymore.
	- Edge and node symbols bound after variable expansion to disallow post-expand filters to get inlined.
	- Some things simplified due to different handling.
- BreadthFirstExpand logical operator merged into ExpandVariable. Two Cursor classes remain and are dynamically chosen from.

As part of planned planner refactor we should ensure that a filter is applied only once. The current implementation is very suboptimal for property filtering in variable expansions.

@buda: we will start refactoring this these days. This current planner logic is too dense and complex. It is becoming technical debt. Most of the time I spent working on this has been spent figuring the planning out, and I still needed Teon's help at times. Implementing the correct and optimal version of query execution (avoiding multiple potentially expensive filterings) was out of reach also due to tech debt.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D852
2017-10-05 13:12:39 +02:00
florijan
5d2ee8f031 AST - pattern atom properties changed from map to unordered_map
Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D844
2017-10-02 14:24:56 +02:00
florijan
03c78c1277 BFS syntax changes
Summary:
- The new BFS syntax implemented as proposed.
- AST BreadthFirstAtom now uses EdgeAtom members: has_range_{true}, upper_bound_, lower_bound_
- Edges data structure now handles all the edge filtering (single or multiple edges), to ease planning. Additional edge filtering (additional Filter op in the plan) is removed. AST EdgeTypeTest is no longer used and is removed.

Current state is stable but there are things left to do:
- BFS property filtering.
- BFS lower_bound_ support.
- Support for lambdas in variable length expansion. This includes obligatory (even if not user_defined) inner_node and inner_edge symbols for easier handling.
- Code-sharing between BFS and variable length expansions.

I'll add asana tasks (and probably start working on them immediately) when/if this lands.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D836
2017-09-27 16:25:19 +02:00
Mislav Bradac
3140f175fc Implement record lock deadlock breaker
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D823
2017-09-27 16:11:05 +02:00
Teon Banek
eb821aa615 Remove ast-cache flag
Summary:
AST caching should be well tested by now.
We should consider removing `Context.is_query_cached_` member as well as the
implementation and tests for `CypherMainVisitor`.

Reviewers: mislav.bradac, mferencevic, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D833
2017-09-27 09:04:31 +02:00
Teon Banek
9d4ae833cc Setup basic installation for CPackDeb
Summary:
Remove traces of plan_compiler in cmake.
Add memgraph executable to install target.
Fix indentation in tests/CMakeLists.txt.
Install alpha.conf to /etc/memgraph/config.
Rename executable to 'memgraph' and install /var dirs.

Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D829
2017-09-26 14:54:13 +02:00
florijan
a9381df09e Edges data structure now supports multiple edge filtering (implicit OR)
Summary: - modified all utils/algorithm functions to be inline and in the utils namespace

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D830
2017-09-26 13:46:18 +02:00
florijan
d9503d6b65 Flags cleanup and QueryEngine removal
Summary:
I started with cleaning flags up (removing unused ones, documenting undocumented ones). There were some flags to remove in `QueryEngine`. Seeing how we never use hardcoded queries (AFAIK last Mislav's testing also indicated they aren't faster then interpretation), when removing those unused flags the `QueryEngine` becomes obsolete. That means that a bunch of other stuff becomes obsolete, along with the hardcoded queries. So I removed it all (this has been discussed and approved on the daily).

Some flags that were previously undocumented in `docs/user_technical/installation` are now documented. The following flags are NOT documented and in my opinion should not be displayed when starting `./memgraph --help` (@mferencevic):
```
query_vertex_count_to_expand_existsing (from rule_based_planner.cpp)
query_max_plans (rule_based_planner.cpp)
```

If you think that another organization is needed w.r.t. flag visibility, comment.

@teon.banek: I had to remove some stuff from CMakeLists to make it buildable. Please review what I removed and clean up if necessary if/when this lands. If the needed changes are minor, you can also comment.

Reviewers: buda, mislav.bradac, teon.banek, mferencevic

Reviewed By: buda, mislav.bradac

Subscribers: pullbot, mferencevic, teon.banek

Differential Revision: https://phabricator.memgraph.io/D825
2017-09-22 17:03:45 +02:00
Matej Ferencevic
383175d85f Bolt - path enconding and decoding
Summary:
Added path encoding and decoding in our Bolt layer. Returning paths to the Neo client seems to work fine.
Not sure how to test that the decoded works fine (our client?).
Tests are not written, that suite is complicated. Leaving that to mferencevic.
Also reduced DecodedValue a bit with macros. Less code, less error prone.

Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D810
2017-09-21 13:55:41 +02:00
Teon Banek
41a027a721 Remove unused variables and function
Reviewers: florijan, mislav.bradac, mferencevic, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D816
2017-09-20 17:04:57 +02:00
Teon Banek
4a602a445a Split MATCH ... WHERE filtering on AND
Summary:
Test planner splits MATCH ... WHERE
Remove distinction between FilterAnd and AndOperator

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D814
2017-09-20 17:04:30 +02:00
Teon Banek
182a9241cf Use symbol names for header if missing token position
Summary:
This fixes a bug, where streaming would try to get the name of the symbol from
an invalid token position. For example, `MATCH (n) WITH n RETURN *`

Reviewers: florijan, mislav.bradac

Reviewed By: florijan, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D811
2017-09-20 15:38:53 +02:00
Teon Banek
08eaaba77f Add caching logical plans
Summary: Add const to MakeCursor method

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D791
2017-09-20 13:31:26 +02:00
florijan
d6a885cce6 Traversal API removed
Summary: Removed the traversal API, been waiting for named path to land because that data structure has replaced traversal's Path. I left the wiki page of the API, just put a warning that it's not used.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D809
2017-09-20 10:30:45 +02:00
florijan
9edc472eaf Resolving TODOs in the operator implementations
Summary:
Three TODOs resolved.

1. around line 897 - we currently don't support expansion into existing variable length edges (there is a TODO in symbol_generator.cpp:213), so this should not be done at the moment.

2. around line 1025 - This TODO was on review and nobody commented, so I'm removing it. Should have done that when the diff landed.

3. around line 1560 - This does not seem possible. Edge-uniqueness checks happen within a single `[OPTIONAL ] MATCH`. If it is OPTIONAL (the case interesting here), then the uniqueness check also gets planned under the optional branch. So, if an optional fails, the uniqueness check will get skipped, as opposed to getting executed over a Null. I added an edge-case test to verify this (and checked with the planner test).

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D807
2017-09-19 09:48:28 +02:00
florijan
347611edfd Named path support
Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D794
2017-09-19 09:45:50 +02:00
florijan
48e446688f IndexInfo function added
Summary:
- Keys() functions in the indices can't be const because ConcurrentMap doesn't provide const accessors (and they are broken in skiplist) :D
- no cucumber tests because many tests create indices so it's hard to say what's inside and what not

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D797
2017-09-16 14:09:35 +02:00
Teon Banek
b31478dfd8 Look into parameter value when estimating plan cost
Summary: Test cost estimator considers parameter values

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D798
2017-09-15 16:14:57 +02:00
florijan
59b9b7af21 CounterSet function added
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D795
2017-09-14 18:18:20 +02:00
Teon Banek
d116e0deb1 Generate ParameterLookup instead of PrimitiveLiteral
Summary:
This is done when the generated AST will be cached.
Remove LiteralsPlugger.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D788
2017-09-13 17:49:34 +02:00
florijan
bfbec8d550 Counter function added
Reviewers: buda, mferencevic, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D787
2017-09-13 17:09:12 +02:00
Teon Banek
a83bea0b74 Add ParameterLookup to AST
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D782
2017-09-13 14:28:48 +02:00
florijan
682fced81b query::plan - Ops use vector instead of list
Summary:
Replaced std::list with std::vector in all plan operators. Performance increase in harness tests is not visible. Defined a custom test:

```
unwind range(0, 1000000) as x
create ({a: tointeger(rand() * 100), b: tointeger(rand() * 100), c: tointeger(rand() * 100), d: tointeger(rand() * 10), e: tointeger(rand() * 10), f: tointeger(rand() * 10)});

match (n) return min(n.a), max(n.b), sum(n.c), n.d, n.e, n.f

match (n) with distinct n.a AS a, n.b AS b, n.c AS c, n.d AS d, n.e AS e, n.f AS f return count(*)
```

In that test performance gains are 9.8% on the aggregation query (mean 0.83s vs 092s) and 34% (mean 2.15s vs 3.25s) on the distinct query. Doubt we'll see much on any of the LDBC tests because they don't stress those operators nearly as much.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D778
2017-09-12 14:38:32 +02:00
Teon Banek
4601f6c368 Add filtering BFS by edge type
Summary:
Antlr grammar has been updated to support putting edge types after the BFS
symbol. Planner collects edge type filters for BFS and inlines them in the
operator by joining the filter with the user input BFS filter itself. This
requires no change from the standpoint of the operator. On the other hand, in
order to use the faster lookup by a single edge type, `ExpandBreadthFirst`
operator now accept an optional edge type. The edge type is passed from the
planner only if the user is filtering by a single type.

Unit tests as well as tck have been updated.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D777
2017-09-12 11:29:38 +02:00
florijan
14064926b6 plan::OrderBy - use vector instead of list
Summary: Reduces latency on LDBC query 9 from 7.9sec to 6.8sec (14%). That query has 650k rows in ORDER BY, 3 ordering elements and 10ish values get returned (both of them are now accumulated into vectors).

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D775
2017-09-11 14:11:33 +02:00
Teon Banek
35f726dfd2 Use EdgeType in Expand and ExpandVariable
Summary:
Add function First to utils.
Insert EdgeType into Expand during planning.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D769
2017-09-11 11:16:04 +02:00
Mislav Bradac
74d0a426ab Use \u for utf16 and \U for utf32
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D773
2017-09-09 16:16:30 +02:00
Teon Banek
92b9bbd4bd Remove const requirement from DbAccessor in planning
Summary: The constness of the DbAccessor interferes with caching the results.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D771
2017-09-09 14:06:32 +02:00
Teon Banek
0aee18544d Add caching VerticesCount during planning and estimation
Summary:
Benchmark planning and estimating indexed ScanAll. According to the benchmark,
caching speeds up the whole process of planning and estimation by a factor of
2. Most of the performance gain is in the `CostEstimator` itself, due to plenty
of calls to `VerticesCount` when estimating all of the generated plans.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D765
2017-09-09 12:26:48 +02:00
florijan
2c4966edfe Vertex edges data structure introduced
Summary:
- The `Edges` data structure now handles common ops, including providing an iterator over edges whose "other" vertex is know.
- This should improve performance on dense_expand tests in the harness without other side-effects.
- query::plan::Expand operator modified not to check for existing-node stuff since that now gets handled by the `Edges` data structure.
- `Edges::Iterator` implemented only for const iterators since that suffices for now. Can implement non-const if the need arrises.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D763
2017-09-08 13:21:21 +02:00
Teon Banek
c8d34c58ef Generate indexed ScanAll to make Expand existing
Summary: Test generating ScanAll before expand to use existing

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: buda, lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D746
2017-09-08 09:50:18 +02:00
Matej Ferencevic
6a1f083370 Added parameters to query logging.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D757
2017-09-06 16:48:09 +02:00
florijan
4975743f35 Assert function added
Summary: `assert` function added. Useful to us for asserting DB state in harness tests. Potentially useful to the client for breaking out of a query as soon as a predicate fails, as opposed to collecting result and checking them client-side.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D750
2017-09-05 13:57:56 +02:00
Matej Ferencevic
70d9f3f6f1 Refactored harness and added PostgreSQL support.
Summary:
Moved Neo4j config to config dir.

Neo4j and PostgreSQL are now downloaded to libs.

Renamed metadata flags in memgraph.

Changed apollo generate for new harness.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D741
2017-09-05 09:45:41 +02:00
Mislav Bradac
c56621682e Parse utf16 surrogate codepoints correctly
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D745
2017-09-04 16:03:45 +02:00
Teon Banek
d584a7b18b Test ExpandVariable reverses on Direction::BOTH
Summary: Add `is_flipped` to `Expansion`

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D744
2017-09-04 15:35:17 +02:00
Matej Ferencevic
65c6b50ade Removed query logging in release mode.
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D742
2017-09-04 13:37:20 +02:00
florijan
f68bac922f Map indexing added
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D739
2017-09-04 09:12:46 +02:00
florijan
75ccdc8dca TypedValue refactor
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D738
2017-09-02 14:25:46 +02:00
florijan
0a5e9256f1 ExpandBreadthFirst can also have optional input
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D735
2017-09-01 14:14:23 +02:00
florijan
7f40607dd3 TypedValue test and implementation improvements
Summary: Some TypedValue arithmetic ops threw exceptions for the wrong reasons, the op applicability type checks were wrong. Also the tests for that behavior were wrong. These are the fixes.

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D724
2017-09-01 11:21:32 +02:00
Teon Banek
52709ad04c Inline filter inside ExpandVariable
Summary:
Reorder class definition in ast.hpp.
Test inlining filters in ExpandVariable.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D726
2017-08-30 16:23:16 +02:00
Teon Banek
fe6d64066b Plan Filter at the end of With clause
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D727
2017-08-30 15:21:28 +02:00
Mislav Bradac
9a39a125b2 Remove rand from dba
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D721
2017-08-29 12:28:32 +02:00
Teon Banek
7fc821ad25 Setup basic interactive planner testing
Summary:
Add basic InteractiveDbAccessor.
Add printing logical operator tree.
Cache some InteractiveDbAccessor vertex counts.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D707
2017-08-28 10:25:17 +02:00
florijan
ca8fb55ac5 Collect Map added
Summary:
Tests are on the way. Please first comment if you're OK with this implementation, some points are discussable.

What works now:
```
bash:MEMGRAPH_ROOT/build/>./tests/manual/console 10
MG>MATCH (n) RETURN COLLECT("age_" + n.age, n.height)

+-----------------------------------------------------------------------------------------------------------------------------------+
| COLLECT("age_" + n.age, n.height)                                                                                                 |
+-----------------------------------------------------------------------------------------------------------------------------------+
| {age_10: 176, age_13: 180, age_24: 172, age_25: 179, age_32: 123, age_33: 186, age_37: 147, age_43: 162, age_49: 126, age_6: 170} |
+-----------------------------------------------------------------------------------------------------------------------------------+
```

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D695
2017-08-25 12:51:18 +02:00
Teon Banek
fdc389f1eb Templatize CostEstimator on DbAccessor
Summary:
This allows for inserting dummy DbAccessor in tests. Unfortunate side
effect of this change is that the whole implementation had to be moved
from cpp to hpp.

Also templatize remaining RuleBasedPlanner implementation

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D704
2017-08-24 14:27:14 +02:00
florijan
2954276ca8 Property storage now supports Map
Summary:
Added:
- map support in PropertyValue
- conversion of map TypedValue to PropertyValue if appropriate flag is set (undocumented because it's private)
- ordering of map PropertyValue in LabelPropertyIndex
- issue raised regarding list and value property modifications in storage (currently unsupported)

Maybe I missed some feature or whatever?

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D692
2017-08-24 10:29:49 +02:00
Teon Banek
4ccffbfd9b Split planners in headers and templatize PlanningContext
Summary:
This change should allow for passing a different PlanningContext and/or
GraphDbAccessor. In turn, we can write tests which pass a dummy context
for decoupled testing of the planning process (from the rest of the
system).

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D700
2017-08-24 09:23:20 +02:00
Teon Banek
591d086013 Map symbols to expansions to speed up variable planning
Summary:
Test variable planning BFS.
Add more tests for variably planning ExpandVariable.
Don't recreate the whole matching when varying expansions.
Use explicit constructors in private planner classes.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D691
2017-08-23 11:39:45 +02:00
florijan
1a619c54e9 Rand() function added
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D690
2017-08-22 15:18:41 +02:00
Mislav Bradac
8792b8f931 Implement CASE construct
Summary: Missing: documentation

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D685
2017-08-21 13:30:41 +02:00
Teon Banek
ac9f6170d6 Support reversing ExpandVariable
Summary:
This is needed in cases when the planner decides to start expanding from
the other end.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D681
2017-08-21 11:59:31 +02:00
Mislav Bradac
5a69f46f1f Fix bug in planner VaryMatchingStart iterator
Reviewers: teon.banek, florijan, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D680
2017-08-18 17:22:35 +02:00
Mislav Bradac
0435f5c851 Fix warnings in planner
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D678
2017-08-18 14:36:49 +02:00
Mislav Bradac
3a365a2808 Migrate unique_ptr from InitEdges to optional
Reviewers: buda, florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D674
2017-08-17 17:06:30 +02:00
Mislav Bradac
6db9e38e1e Fix bug in named expression
Summary: Fix stripped named expression keyword case bug

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D670
2017-08-17 14:51:12 +02:00
Teon Banek
9e85ccc892 Iterate matching variants and limit expanding
Summary:
Matching variants are generated iteratively and limited. Additionally,
When generating expansions, if there is nothing to continue on the
expanded nodes, simply append all the remaining expansions to the
currently generated one. This should speed up queries with large number
of `MATCH` clauses.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D612
2017-08-16 11:34:04 +02:00
florijan
0703724295 GraphDbAccessor::BuildIndex - deadlock bugfix
Reviewers: buda, mislav.bradac, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D658
2017-08-11 10:34:18 +02:00
florijan
1d112e1141 GraphDbAccessor - style change
Summary: Not strictly neccessary, but it's been itching me. It took an hour.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D648
2017-08-09 16:09:08 +02:00
florijan
47c1cd6e3d Degree(Vertex) function added
Summary:
- added only one function for getting the total (in + out) vertex degree, it's required for the Ravelin use-case
- specific `degree_in` and `degree_out` functions can be added as necessary
- also fixed random_graph_generator bug (needed it for testing)

Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D652
2017-08-09 15:06:10 +02:00
florijan
b8957c999d Map type now supported
Summary:
- MapLiteral added
- PropertyLookup on maps added

This is the basic implementation, missing are:
- unit tests
- feature and TCK tests
- documentation
- changelog

That stuff is coming. Please review the implementation (Mislav).

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D640
2017-08-08 14:34:26 +02:00
Teon Banek
0b8d71ee8f Plan BreadthFirstExpand
Summary:
Test planning BreadthFirstExpand

Add bfs tests to memgraph qa

Allow pointers in `print-operator-tree` for gdb

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D618
2017-08-07 10:44:10 +02:00
Mislav Bradac
cc8e9f2996 Fix of stupid, stupid bug
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: florijan, pullbot

Differential Revision: https://phabricator.memgraph.io/D635
2017-08-04 15:03:59 +02:00
florijan
bce4407ebe CostEstimator - adding new ScanAll and Expand ops
Summary:
Now all ScanAll and Expand ops are covered by the cost estimator. For ScanAll with indices cost estimation is pretty good, for new Expand ops it is tragically bad (Expand to the power of expansion depth, plus arbitrary filtering). Static cost estimation is wrong wrong wrong.

Currently cost estimation of even trivial plans that use indices is wrong because the planner leaves filtering expressions that are implicitly handled by the index in the operator tree, IIRC. Tasking Teon to revise this, even though I'm not sure how bad an influence this has on cost estimation and it's use in plan choosing.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D633
2017-08-04 14:32:22 +02:00
florijan
6c22caa80e Query::Plan::Expand with preceeding optional bug fix
Summary:
This diff contains a bug fix for the expansion operators that are currently on dev.
More importantly, it proposes end-to-end testing for edge-cases for which it's a
pain to write single-phase tests. In my opinion this is OK, you're all reviewers so
you can comment.

The test relies on left-to-right query execution. We need this guarantee in tests
like this. I propose renaming "RuleBasedPlanner" to "LeftToRightPlanner" to make
this explicit. As Teon is not here at the moment, will make this a task/discussion.

Reviewers: buda, mislav.bradac, teon.banek, lion

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D626
2017-08-04 12:16:26 +02:00
florijan
0f73c2451b Query::Plan::ExpandVariable - graph state bugfix
Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D631
2017-08-04 12:12:44 +02:00
Mislav Bradac
d750970381 Lock when accessing parser to prevent antlr bugs
Reviewers: buda, mferencevic

Reviewed By: buda, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D625
2017-08-03 16:38:08 +02:00
florijan
e45ae4c4b6 Query::Plan::ExpandBreadthFirst added
Summary:
Not complete (but review can start):
- implementation should be done
- still need to finish tests
- documentation missing

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D616
2017-08-03 16:22:38 +02:00
Mislav Bradac
6bc9deba5f Replace map with bitset in stripped query
Summary:
Implement trie and use it in stripper

Make it nicer

Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D614
2017-08-02 22:17:36 +02:00
Teon Banek
4b72118894 Generate symbols for BFS
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D608
2017-07-30 12:25:30 +02:00
Teon Banek
c84d8f6bd7 Add BreadthFirstAtom to Ast
Summary: Add BFS to Cypher grammar

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D605
2017-07-29 23:53:31 +02:00
Teon Banek
160a6866dc Break the query part on Unwind
Summary:
`UNWIND` can come before `MATCH`, so it needs to break query parts. If
it didn't, a query part would incorrectly grab all the matches and plan
them incorrectly. A test for such a case has been added.

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D598
2017-07-29 18:14:06 +02:00
florijan
04c2ab3ba9 Plan::ExpandVariable - bounds are now expressions
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D596
2017-07-29 14:33:16 +02:00
Teon Banek
c08bb6e00c Catch LexingException in our REPL
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D595
2017-07-28 16:59:26 +02:00
Teon Banek
9e157eb495 Generate All to filter variable length paths
Summary:
Use All to filter variable length paths.
Test filtered variable length paths in qa

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D592
2017-07-26 13:03:54 +02:00
Teon Banek
9ae1a9a585 Plan variable length expand
Summary: Handle reordering expansions with variable path length

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D579
2017-07-26 12:36:47 +02:00
Teon Banek
b33aae42ab Add ALL function to openCypher
Summary:
Add All expression to Ast
Evaluate All expression
Visit All and generate symbols
Handle All when collecting context during planning

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D587
2017-07-25 16:32:45 +02:00
Teon Banek
da7bfe05b7 Collect symbols in variable length path
Summary:
Add EdgeList symbol type and check for redeclaration

The result of variable path is a list of edges, so the symbol type has
been added. In the future, we need to extend the type checker and the
type structure to have a generic list type.

We also currently do not support reusing an already bound symbol for a
variable path, so the SymbolGenerator will raise a redeclaration error.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot, lion

Differential Revision: https://phabricator.memgraph.io/D574
2017-07-25 12:39:14 +02:00
Teon Banek
20f2c885c8 Test grammar for variable length with filters
Summary: Update Cypher grammar to pickup properties after range

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D578
2017-07-21 11:51:55 +02:00
florijan
7328f5ec7f Query::Plan::VariableExpand added
Summary:
Variable expansion logical operator added. Some functionalities are missing:
- taking into account optional matching when expanding into existing symbol
- accepting Expression bounds (current implementation takes size_t)

Also, a TODO is added for handling optional matching in the uniqueness operator (with an Asana task)

All this will be done in the following diff, this is already substantial.

Also, please consider if we want to have all those `VLOG`s in the code. Not very pretty. And I think that `VLOG` is not compiled-away in release build, will put an asana task.

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D569
2017-07-20 14:26:41 +02:00
Teon Banek
b4d2d1ff81 Support variable length path in CypherMainVisitor
Summary:
Allow expressions for variable length path bounds

Replace test which expected a syntax exception

Since we now allow variable length to have an arbitrary expression, the
test case is obsolete. It was replaced with something that excepts an
expression which wasn't allowed before.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D568
2017-07-20 13:14:50 +02:00
Mislav Bradac
e8a465e4b5 Add query parameters support
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D570
2017-07-19 18:44:59 +02:00
florijan
a7d8b7124e GraphDbAccessor - index range API
Summary:
- GraphDbAccessor - index range API added
- index api tests refactored
- skiplist minor cleanup.

Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D533
2017-07-19 11:55:40 +02:00
Teon Banek
16d94c8aaf Make the message mandatory for NotYetImplemented exception
Reviewers: florijan, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D565
2017-07-18 16:40:49 +02:00
Teon Banek
1ad3b7f906 Remove unused time_second from query/engine.hpp
Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D562
2017-07-17 16:56:13 +02:00
Teon Banek
104fe22f53 Plan ScanAll by using a range index
Summary:
Add optional bounds to PropertyFilter and collect them

Relation operators (e.g. `<`, `>` ...) should be used to produce
scanning the index by a range of values. For that reason, PropertyFilter
is extended to store either the equality expression or range bounds.
The `AnalyzeFilter` function is extended to look for those operators and
see if their top level expression contains a property lookup. If it
does, a filter with a bound is generated.

Test for property comparison preventing index use

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D529
2017-07-17 16:34:07 +02:00
Teon Banek
3d8d0efea5 Convert CartesianProduct to iterable
Summary:
This change allows for lazy calculation of Cartesian product, by using
an iterator. Using lazy evaluation, we can easily limit the number of
generated products and therefore the number of generated query plans.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D554
2017-07-17 16:30:06 +02:00
Mislav Bradac
4dabf31005 Remove unused function time_second
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D561
2017-07-17 15:55:43 +02:00
Mislav Bradac
50cd53e5c9 Migrate timer to use walltime, instead of cputime
Summary:
 - Move some stuff to poc
 - Use steady_clock instead of system_clock

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D555
2017-07-17 13:42:44 +02:00
Mislav Bradac
4a500b99fa Add a way to stop query execution
Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D552
2017-07-14 14:36:44 +02:00
Teon Banek
387c0621cb Use division to detect unsigned wrap-around
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D550
2017-07-13 15:17:19 +02:00
Teon Banek
99e27cb7e9 Add limiting number of generated plans for a query
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D548
2017-07-13 14:27:15 +02:00
Mislav Bradac
26144ee8fd Fix bugs in MatchReal (StrippedQuery)
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D546
2017-07-12 16:50:53 +02:00
Mislav Bradac
6068a95a0e Move dbms to database
Summary:
There was only two files in dbms directory so I moved them to database
directory.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D540
2017-07-12 12:44:11 +02:00
Teon Banek
88153595ce Make GraphDbAccessor mandatory for planning
Reviewers: florijan, mislav.bradac, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D527
2017-07-10 10:13:12 +02:00
Teon Banek
d9a724f0cd Plan scanning by label & property index for node atoms
Summary:
Add Filters class for storing additional info

Add FindOr to utils/algorithm.hpp

Use all collected labels when scanning by them

Collect label filters inside WHERE

Document the Filters class

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot, lion

Differential Revision: https://phabricator.memgraph.io/D515
2017-07-07 11:30:54 +02:00
Teon Banek
9fbaf69441 Plan CreateIndex operator
Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D525
2017-07-06 14:05:15 +02:00
Mislav Bradac
d608d523c5 Add CreateIndex conversion to ast
Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D522
2017-07-06 13:32:37 +02:00
Teon Banek
1f7a146b95 Replace utils::Border with utils::Bound
Summary:
The new Bound class does not have comparison operators defined. The
reason being, we want to support having values which we may not want to
compare. For example, having an Expression which should first be
evaluated and then compared.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D520
2017-07-04 10:00:38 +02:00
Teon Banek
1841ff9616 Add operators for getting nodes by label-property index
Summary:
Add ScanAllByLabelPropertyRange operator
This operator uses the label + property indexing feature to iterate over
the vertices. The property value of each vertex is checked whether it is
inside the given range of values. The range is inclusive from both
sides. If the value isn't in range, the vertex is filtered out.

This manual filtering should be replaced by a database API when it
becomes available.

Add ScanAllByLabelPropertyValue operator

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D503
2017-07-03 11:03:46 +02:00
Teon Banek
f5db9d65b0 Add CreateIndex operator
Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D516
2017-07-03 11:03:15 +02:00
Mislav Bradac
3119ae5343 Supstitute named expressions in ast cache
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D510
2017-06-26 16:33:17 +02:00
Mislav Bradac
3a922de963 Implementation of new lexer
Summary: WORK IN PROGRESS

Reviewers: buda, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D496
2017-06-21 16:16:17 +02:00
Dominik Gleich
e2f3aba332 Use GLogger instead of broken memgraph Logger.
Summary:
http://rpg.ifi.uzh.ch/docs/glog.html

Second phase before tests complete.

Delete logging test.

Finish relase loging.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D500
2017-06-21 15:33:24 +02:00
Mislav Bradac
b6ecc5631f Disallow keywords as symbolic names
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D497
2017-06-20 11:58:14 +02:00
Mislav Bradac
6403599a5d Generate plan_compiler_flags only if needed
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D485
2017-06-16 15:58:32 +02:00
Mislav Bradac
1862b04ac2 Fix bug in StrippedQuery
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D482
2017-06-16 15:04:48 +02:00
Mislav Bradac
5403b99e18 Remove logger from StrippedQuery
Summary:
benchmark/query/stripped.cpp outputs garbage because of some race
condition in logger. Sometime even crashes, just be sure this
should be removed.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D477
2017-06-16 12:13:23 +02:00
Teon Banek
7d00c4eac0 Add query_cost_planner gflag
Reviewers: mislav.bradac, buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D479
2017-06-16 10:20:02 +02:00
Mislav Bradac
67b859cf13 Add AST cache
Reviewers: buda, teon.banek, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D468
2017-06-14 18:59:31 +02:00
Mislav Bradac
c56eb4310e Remember token positions for literals
Summary: Remove yaml

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D466
2017-06-13 17:11:27 +02:00
Teon Banek
69cfd197d8 Don't raise TypedValueException during query execution
Summary:
Handle TypedValueExceptions in query/plan/operator.cpp

Raise QueryRuntimeException during expression evaluation

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D457
2017-06-13 14:46:56 +02:00
Mislav Bradac
f7d540829a Add deep clone to Ast
Summary: Prepare your aspergillums for blessing this magnificent diff.

Reviewers: buda, teon.banek, florijan

Reviewed By: teon.banek

Subscribers: lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D458
2017-06-13 12:22:25 +02:00
Teon Banek
3b88d7429f Visit edge properties during symbol generation
Summary:
This is an obvious bug, caused by an oversight. A test has been added
for this case.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D455
2017-06-12 15:27:15 +02:00
Dominik Gleich
0fb29a7ce0 Add release logging without core dump.
Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D447
2017-06-09 11:26:19 +02:00
Dominik Gleich
1057d253ca Rename config flags.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D446
2017-06-09 10:52:26 +02:00
Mislav Bradac
fea9031605 Refactor stripper
Summary: Fix tests

Reviewers: buda, florijan, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D435
2017-06-07 18:47:09 +02:00
Dominik Gleich
65507da9eb Load configs from flagfiles.
Summary: Migrate configto gflags.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D433
2017-06-07 15:38:17 +02:00
Dominik Gleich
ebdee4e509 Skiplist garbage collector rework.
Summary:
Drawing:
https://drive.google.com/open?id=0B-W7PQZqMD9hcG04b0lKaGZGOWM

Reviewers: mislav.bradac, buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D395
2017-06-07 10:47:02 +02:00
Teon Banek
7278bdff94 Check for aggregation in all Expression types
Summary:
This fixes an issue when aggregations and/or group by expressions
weren't picked up from certain operators. In addition to that, we would
segfault in cases when the `has_aggregation_` is empty. For example,
function calls without arguments: `RETURN PI()`.

Test aggregations inside some operators

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D425
2017-06-06 15:05:43 +02:00
Teon Banek
b2c523b93a Generate multiple plans and use the least costly one
Summary:
This is a simple integration of multiple plan generation and cost
estimation. Each plan produced by VariableStartPlanner is cost estimated
and the best is used for execution.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D424
2017-06-06 14:03:41 +02:00
florijan
509d5db67a Query - plan - cost estimator basic. REPL mods
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: teon.banek, buda

Subscribers: pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D399
2017-06-02 12:44:17 +02:00
Teon Banek
f0422c0e11 Generate multiple plans depending on starting node
Summary:
Permute query parts.

Permute matching only by selecting the starting node.

Flip the expansion when expanding from the other node.

Split planner into rule_based_planner and variable_start_planner

Use symbol hash when collecting expansion nodes

Multiple node atoms may point to the same symbol, and we could generate
multiple starting positions per atom which are the same. Using symbol
hash and equality prevents generating those redundant plans.

Correctly permute optional and merge matchings

Test VariableStartPlanner

Reviewers: florijan, mislav.bradac, buda, lion

Reviewed By: florijan, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D417
2017-06-01 16:38:21 +02:00
Teon Banek
1356fd5391 Use the label index with fewest vertices for ScanAll
Reviewers: florijan, mislav.bradac

Reviewed By: florijan, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D415
2017-05-31 15:41:21 +02:00
Teon Banek
d706a91b1b Don't make an Expansion of single node if it is expanded
Summary:
We would redundantly generate an Expansion for the first node if it was part of
an expand. For example, the pattern `(n) -[r]- (m)` would generate
`Expansion{n}` and `Expansion{n, r, m}`, when only the latter is enough. This
change corrects that behaviour by dropping the first Expansion.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D412
2017-05-31 14:07:56 +02:00
Teon Banek
62f6a58c32 Generalize MakeLogicalPlan with regards to planner
Summary:
This change modifies the planning API to be more general, in order to support
picking different planning strategies. The current planning strategy has been
named RuleBasedPlanner.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D411
2017-05-31 14:07:41 +02:00
Mislav Bradac
2005056cd5 Measure times in hardcoded queries
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D410
2017-05-30 19:47:35 +02:00
Mislav Bradac
34b6afdf92 Measure antlr time
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D409
2017-05-30 17:29:55 +02:00
florijan
7403338f38 Query - Aggregation with no input, SQL-style handling.
Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D400
2017-05-30 16:01:35 +02:00
Teon Banek
e631eb4eb2 Rename directions from LEFT/RIGHT to IN/OUT
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D405
2017-05-30 13:14:34 +02:00
Teon Banek
f9bcd6d760 Replace Atoms with Symbols in Expand operator
Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D404
2017-05-30 12:10:17 +02:00
Teon Banek
d72d3061eb Reconstruct vertices iterable in-place
Summary:
Since the vertices iterable used in ScanAllCursor may be lazily
generated, it needs to be recreated, instead of simply calling
`begin()`. In our current implementation, we use cppitertools which do
not have move assignment implemented. Because of that, a hackish
in-place destruction and construction is used to reset the iterable.

Reviewers: florijan, mislav.bradac, dgleich, buda

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D401
2017-05-29 15:55:47 +02:00
Teon Banek
74b082f050 Preprocess Ast to QueryParts and plan ScanAllByLabel
Summary:
Mention the non-existent function name in semantic error. Don't merge optional
matches into one Matching, because it is an error to treat multiple optional
matches as a single optional match. Document new structures and functions. Add
not so smart ScanAllByLabel generation.

Reviewers: mislav.bradac, buda, florijan, lion

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D394
2017-05-26 15:57:45 +02:00
florijan
04670f7328 Query - separate timing for planning and execution
Reviewers: mislav.bradac, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D393
2017-05-23 17:03:46 +02:00
Teon Banek
d016472c3a Add ScanAllByLabel operator.
Summary:
Replace NodeAtom with Symbol inside ScanAll. Move ScanAllCursor outside of
ScanAll class and make it generic with regards to vertices it produces.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D387
2017-05-22 09:28:42 +02:00
Mislav Bradac
839d63284b Implement STARTS WITH, ENDS WITH, CONTAINS
Reviewers: buda, teon.banek, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D389
2017-05-19 19:28:06 +02:00
Mislav Bradac
f9cd87bb46 Implement collect aggregation
Reviewers: teon.banek, florijan

Reviewed By: teon.banek, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D386
2017-05-19 17:24:12 +02:00
Teon Banek
28fbeb8e9d Raise if MATCH is after OPTIONAL MATCH
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D383
2017-05-19 16:27:21 +02:00
Mislav Bradac
6a027ea3b2 Throw on key appearing twice in map literal
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D388
2017-05-19 16:17:40 +02:00
florijan
97638d3327 Query - grammar changed to support '1 IN [[1, 2]][0]'
Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D381
2017-05-19 09:09:12 +02:00
Teon Banek
663d78feaa Handle expanding from Null vertex
Summary:
This fixes a bug when the MATCH clause would follow an OPTIONAL MATCH.
In case when the optional part would fail to generate results, expanding
would cause an error.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D372
2017-05-18 09:04:16 +02:00
Teon Banek
8380d3694f Raise semantic error when redeclaring node in MERGE
Summary:
openCypher expects MERGE to behave like CREATE. As such, it shouldn't be
allowed to refer to declared nodes, while providing labels and
properties.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D373
2017-05-17 15:08:38 +02:00
Teon Banek
30a4f40093 Skip Null in SET and REMOVE clauses
Summary:
openCypher expects removing/setting properties and labels on Null
vertices/edges does not produce an error. Instead, Nulls are simply
skipped.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D375
2017-05-17 15:07:03 +02:00
Teon Banek
8cdf87e109 Bind UNWIND symbols during planning
Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D371
2017-05-17 10:37:12 +02:00
Teon Banek
23ab2b41a3 Refactor ExpressionEvaluator to classic visitor
Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D368
2017-05-16 15:33:47 +02:00
Teon Banek
15d7226515 Add classic and hierarchical visiting
Summary:
Merge utils/visitor directory into single file.
Rename Visitor to HierarchicalVisitor.
Add regular Visitor.
Split HierarchicalVisitor into LeafVisitor and CompositeVisitor.
Add more documentation on visitor pattern.
Make PostVisit and Visit return bool.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D364
2017-05-16 12:35:09 +02:00
Teon Banek
87e5dc0dfb Plan '*' in RETURN and WITH
Summary:
Make Symbol members read only.
Check WITH/RETURN * in SymbolGenerator.
Test semantic checks for WITH/RETURN *.
Sort expanded user identifiers by name.
Test planning WITH/RETURN *.

Reviewers: buda, florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D357
2017-05-12 13:05:40 +01:00
Mislav Bradac
f82bda6c0c Implement Range function
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D358
2017-05-10 17:12:15 +02:00
Mislav Bradac
a236d704a3 Join filters with FilterAnd (short-circuit)
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D356
2017-05-08 15:49:49 +02:00
Teon Banek
be81751db3 Allow filter symbols in Match to be out of binding order
Summary:
Check symbols in property maps after visiting Match.
Plan Filters as soon as possible.
Take AstTreeStorage in MakeLogicalPlan instead of Query.
Plan generic Filter instead of specialized operators.
Remove traces of EdgeFilter and NodeFilter.

Reviewers: buda, mislav.bradac, florijan

Reviewed By: mislav.bradac, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D344
2017-05-08 12:03:38 +02:00
florijan
0866fdcb0c Query:: COUNT(*) added to logical planning and execution
Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D353
2017-05-08 11:19:57 +02:00
Mislav Bradac
871b81656b Implement InListOperator
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D354
2017-05-07 17:52:08 +02:00
Mislav Bradac
5ae55499bc Convert escaped labels to AST
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D352
2017-05-06 15:34:02 +02:00
Mislav Bradac
b76685cba7 Fix bug
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D351
2017-05-05 20:10:32 +02:00
Mislav Bradac
3d026b106d Handle asterisk in AST conversion
Summary: TODO: Handle COUNT_ASTERISK aggregation in operator (will do in next diff)

Reviewers: teon.banek, florijan, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D350
2017-05-05 20:01:56 +02:00
Mislav Bradac
cfb987a90c Evaluate epressions EdgeTypeTest and LabelsTest
Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D345
2017-05-05 13:25:23 +02:00
Mislav Bradac
72d4a67c4a Convert labels test from antlr to AST
Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D346
2017-05-05 12:13:55 +02:00
Mislav Bradac
2e9c0e4cb9 Implement evaluation of list indexing and slicing
Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D342
2017-05-04 15:34:33 +02:00
Mislav Bradac
d2f9affb4c Add labels and edge type test nodes to ast
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D343
2017-05-04 14:55:22 +02:00
florijan
0968224a3d Query::Delete - can now delete Null (naturally occuring in optional match)
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D339
2017-05-04 10:35:20 +02:00
Mislav Bradac
cb7310fb6a Add list indexing and slicing conversion to AST
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D341
2017-05-03 19:40:34 +02:00
Teon Banek
d06f80e3f3 Plan Distinct
Summary:
Support WITH/RETURN DISTINCT in test macros.
Test planning Distinct.
Implement OutputSymbols in Distinct operator.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D340
2017-05-03 16:35:48 +02:00
florijan
0a33a555a6 GraphDdAccessor and KeyIndex API upgrades. Test refactoring
Summary:
The GraphDbAccessor and KeyIndex APIs can now also return records for the current transaction+command graph state. This is necessary to correctly implement MERGE. The new logic is has increased the MVCC+Accessor related chaos and should be revised when refactoring MVCC (as planned).

Previous index testing was separated into VertexIndex and EdgeIndex testing. This is inappropriate since most of the logic is exaclty the same. Also it was not clearly defined what gets tested via the GraphDbAccessor API, and what directly through the KeyIndex API. This has also been refactored, but it needs additional work (Gleich).

Reviewers: buda, dgleich

Reviewed By: buda, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D306
2017-05-03 15:42:04 +02:00
florijan
355f62ce71 Query::Plan - Distinct added
Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D336
2017-05-03 15:27:24 +02:00
Teon Banek
cbd3998899 Plan Unwind operator
Summary:
Support ListLiteral in test macros.
Test planning Unwind.
Support UNWIND in test macros.
Test SymbolGenerator for UNWIND clause.
Use namespace in QueryPlan Unwind test.

Reviewers: mislav.bradac, buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D331
2017-05-03 13:54:33 +02:00
Teon Banek
eef6fb1981 Convert antlr Unwind to AST
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D330
2017-05-03 12:32:18 +02:00
florijan
ffc977dbfc Query::Plan::Unwind added
Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D328
2017-05-03 08:38:25 +02:00
florijan
dc5ef33c8f Query::ExpresionEval - Null.prop is now Null
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D329
2017-05-03 08:36:56 +02:00
florijan
2e5eccf197 Query::AST::Literal refactor. Repl and TypedValue mods.
Summary:
- Query::AST::Literal refactor (LiteralBase introduced, ListLiteral added)
- Repl now prints out list TypedValues properly
- TypedValue to string conversion refactors

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D327
2017-04-28 14:58:16 +02:00
Teon Banek
9653c703dc Plan optional matching
Summary:
Support OPTIONAL MATCH in test macros.
Test planning Optional.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D322
2017-04-28 14:03:08 +02:00
Mislav Bradac
1c51ce77ef Even more awesome functions
Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D319
2017-04-27 14:11:12 +02:00
florijan
2710485742 Query::Plan::Optional added, MergeCursor slight mod
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D321
2017-04-27 12:06:02 +02:00
Teon Banek
8b8aee9b38 Add optional flag to Match clause
Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D320
2017-04-27 11:16:30 +02:00
Teon Banek
f0aaca4a1a Plan Skip and Limit after OrderBy
Summary:
Revive the OutputSymbols method.
Use OutputSymbols to stream results in Interpret.

Reviewers: mislav.bradac, buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D318
2017-04-26 16:27:41 +02:00
Teon Banek
8fa574026e Plan Merge operator
Summary:
Check symbols in Merge.
Support MERGE macro in query tests.
Test SymbolGenerator with MERGE.
Test planning Merge.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D317
2017-04-26 14:47:10 +02:00
Mislav Bradac
0fa4555cad Implement opencypher functions
Summary: Implement opencypher functions

Reviewers: florijan, buda, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D299
2017-04-26 13:27:42 +02:00
florijan
ecca97ef72 Query::Plan::Merge added
Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D316
2017-04-25 16:24:24 +02:00
Teon Banek
788d4a37d0 Convert antlr merge to AST
Summary: Convert antlr merge to AST and add Merge node to AST

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D315
2017-04-25 16:04:05 +02:00
florijan
cc00101cbd Query::StateSwitch added
Summary:
- an enum in query/common.hpp for flagging what kind of switching should be done
- changes in expression evaluator
- changes in logical operators
- modification in RecordAccessor::SwitchOld to support operator functionality
- tests

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D314
2017-04-25 15:54:30 +02:00
florijan
774029cfcf Query::Plan - switch from 'cycle' to 'existing' naming where appropriate
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D312
2017-04-24 15:20:11 +02:00
Teon Banek
e53e232e49 Plan OrderBy
Summary:
Support OrderBy in test macros.
Test planning OrderBy.
Handle symbol visibility for ORDER BY and WHERE.
Add Hash struct to Symbol.
Collect used symbols in ORDER BY and WHERE.

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D307
2017-04-24 15:18:01 +02:00
Mislav Bradac
fe36835519 Add db_accessor to ExpressionEvaluator
Summary: Add db_accessor to ExpressionEvaluator

Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D310
2017-04-24 14:38:38 +02:00
florijan
d29a1f5353 Query::Plan::Operator - ACCEPT macro, Cursor::Reset, const correctness
Summary:
Query::Plan::Operator - ACCEPT_WITH_INPUT macro for default accept implementation
Query::Plan::Cursor Reset() function added (NOT tested), will be needed for MERGE
Query::Plan - logical const correctness and cleanup

Reviewers: teon.banek, buda

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D311
2017-04-24 13:00:27 +02:00