Commit Graph

295 Commits

Author SHA1 Message Date
florijan
ca8fb55ac5 Collect Map added
Summary:
Tests are on the way. Please first comment if you're OK with this implementation, some points are discussable.

What works now:
```
bash:MEMGRAPH_ROOT/build/>./tests/manual/console 10
MG>MATCH (n) RETURN COLLECT("age_" + n.age, n.height)

+-----------------------------------------------------------------------------------------------------------------------------------+
| COLLECT("age_" + n.age, n.height)                                                                                                 |
+-----------------------------------------------------------------------------------------------------------------------------------+
| {age_10: 176, age_13: 180, age_24: 172, age_25: 179, age_32: 123, age_33: 186, age_37: 147, age_43: 162, age_49: 126, age_6: 170} |
+-----------------------------------------------------------------------------------------------------------------------------------+
```

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D695
2017-08-25 12:51:18 +02:00
Teon Banek
fdc389f1eb Templatize CostEstimator on DbAccessor
Summary:
This allows for inserting dummy DbAccessor in tests. Unfortunate side
effect of this change is that the whole implementation had to be moved
from cpp to hpp.

Also templatize remaining RuleBasedPlanner implementation

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D704
2017-08-24 14:27:14 +02:00
florijan
2954276ca8 Property storage now supports Map
Summary:
Added:
- map support in PropertyValue
- conversion of map TypedValue to PropertyValue if appropriate flag is set (undocumented because it's private)
- ordering of map PropertyValue in LabelPropertyIndex
- issue raised regarding list and value property modifications in storage (currently unsupported)

Maybe I missed some feature or whatever?

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D692
2017-08-24 10:29:49 +02:00
Teon Banek
4ccffbfd9b Split planners in headers and templatize PlanningContext
Summary:
This change should allow for passing a different PlanningContext and/or
GraphDbAccessor. In turn, we can write tests which pass a dummy context
for decoupled testing of the planning process (from the rest of the
system).

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D700
2017-08-24 09:23:20 +02:00
Teon Banek
591d086013 Map symbols to expansions to speed up variable planning
Summary:
Test variable planning BFS.
Add more tests for variably planning ExpandVariable.
Don't recreate the whole matching when varying expansions.
Use explicit constructors in private planner classes.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D691
2017-08-23 11:39:45 +02:00
florijan
1a619c54e9 Rand() function added
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D690
2017-08-22 15:18:41 +02:00
Mislav Bradac
8792b8f931 Implement CASE construct
Summary: Missing: documentation

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D685
2017-08-21 13:30:41 +02:00
Teon Banek
ac9f6170d6 Support reversing ExpandVariable
Summary:
This is needed in cases when the planner decides to start expanding from
the other end.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D681
2017-08-21 11:59:31 +02:00
Mislav Bradac
5a69f46f1f Fix bug in planner VaryMatchingStart iterator
Reviewers: teon.banek, florijan, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D680
2017-08-18 17:22:35 +02:00
Mislav Bradac
0435f5c851 Fix warnings in planner
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D678
2017-08-18 14:36:49 +02:00
Mislav Bradac
3a365a2808 Migrate unique_ptr from InitEdges to optional
Reviewers: buda, florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D674
2017-08-17 17:06:30 +02:00
Mislav Bradac
6db9e38e1e Fix bug in named expression
Summary: Fix stripped named expression keyword case bug

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D670
2017-08-17 14:51:12 +02:00
Teon Banek
9e85ccc892 Iterate matching variants and limit expanding
Summary:
Matching variants are generated iteratively and limited. Additionally,
When generating expansions, if there is nothing to continue on the
expanded nodes, simply append all the remaining expansions to the
currently generated one. This should speed up queries with large number
of `MATCH` clauses.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D612
2017-08-16 11:34:04 +02:00
florijan
0703724295 GraphDbAccessor::BuildIndex - deadlock bugfix
Reviewers: buda, mislav.bradac, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D658
2017-08-11 10:34:18 +02:00
florijan
1d112e1141 GraphDbAccessor - style change
Summary: Not strictly neccessary, but it's been itching me. It took an hour.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D648
2017-08-09 16:09:08 +02:00
florijan
47c1cd6e3d Degree(Vertex) function added
Summary:
- added only one function for getting the total (in + out) vertex degree, it's required for the Ravelin use-case
- specific `degree_in` and `degree_out` functions can be added as necessary
- also fixed random_graph_generator bug (needed it for testing)

Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D652
2017-08-09 15:06:10 +02:00
florijan
b8957c999d Map type now supported
Summary:
- MapLiteral added
- PropertyLookup on maps added

This is the basic implementation, missing are:
- unit tests
- feature and TCK tests
- documentation
- changelog

That stuff is coming. Please review the implementation (Mislav).

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D640
2017-08-08 14:34:26 +02:00
Teon Banek
0b8d71ee8f Plan BreadthFirstExpand
Summary:
Test planning BreadthFirstExpand

Add bfs tests to memgraph qa

Allow pointers in `print-operator-tree` for gdb

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D618
2017-08-07 10:44:10 +02:00
Mislav Bradac
cc8e9f2996 Fix of stupid, stupid bug
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: florijan, pullbot

Differential Revision: https://phabricator.memgraph.io/D635
2017-08-04 15:03:59 +02:00
florijan
bce4407ebe CostEstimator - adding new ScanAll and Expand ops
Summary:
Now all ScanAll and Expand ops are covered by the cost estimator. For ScanAll with indices cost estimation is pretty good, for new Expand ops it is tragically bad (Expand to the power of expansion depth, plus arbitrary filtering). Static cost estimation is wrong wrong wrong.

Currently cost estimation of even trivial plans that use indices is wrong because the planner leaves filtering expressions that are implicitly handled by the index in the operator tree, IIRC. Tasking Teon to revise this, even though I'm not sure how bad an influence this has on cost estimation and it's use in plan choosing.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D633
2017-08-04 14:32:22 +02:00
florijan
6c22caa80e Query::Plan::Expand with preceeding optional bug fix
Summary:
This diff contains a bug fix for the expansion operators that are currently on dev.
More importantly, it proposes end-to-end testing for edge-cases for which it's a
pain to write single-phase tests. In my opinion this is OK, you're all reviewers so
you can comment.

The test relies on left-to-right query execution. We need this guarantee in tests
like this. I propose renaming "RuleBasedPlanner" to "LeftToRightPlanner" to make
this explicit. As Teon is not here at the moment, will make this a task/discussion.

Reviewers: buda, mislav.bradac, teon.banek, lion

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D626
2017-08-04 12:16:26 +02:00
florijan
0f73c2451b Query::Plan::ExpandVariable - graph state bugfix
Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D631
2017-08-04 12:12:44 +02:00
Mislav Bradac
d750970381 Lock when accessing parser to prevent antlr bugs
Reviewers: buda, mferencevic

Reviewed By: buda, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D625
2017-08-03 16:38:08 +02:00
florijan
e45ae4c4b6 Query::Plan::ExpandBreadthFirst added
Summary:
Not complete (but review can start):
- implementation should be done
- still need to finish tests
- documentation missing

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D616
2017-08-03 16:22:38 +02:00
Mislav Bradac
6bc9deba5f Replace map with bitset in stripped query
Summary:
Implement trie and use it in stripper

Make it nicer

Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D614
2017-08-02 22:17:36 +02:00
Teon Banek
4b72118894 Generate symbols for BFS
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D608
2017-07-30 12:25:30 +02:00
Teon Banek
c84d8f6bd7 Add BreadthFirstAtom to Ast
Summary: Add BFS to Cypher grammar

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D605
2017-07-29 23:53:31 +02:00
Teon Banek
160a6866dc Break the query part on Unwind
Summary:
`UNWIND` can come before `MATCH`, so it needs to break query parts. If
it didn't, a query part would incorrectly grab all the matches and plan
them incorrectly. A test for such a case has been added.

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D598
2017-07-29 18:14:06 +02:00
florijan
04c2ab3ba9 Plan::ExpandVariable - bounds are now expressions
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D596
2017-07-29 14:33:16 +02:00
Teon Banek
c08bb6e00c Catch LexingException in our REPL
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D595
2017-07-28 16:59:26 +02:00
Teon Banek
9e157eb495 Generate All to filter variable length paths
Summary:
Use All to filter variable length paths.
Test filtered variable length paths in qa

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D592
2017-07-26 13:03:54 +02:00
Teon Banek
9ae1a9a585 Plan variable length expand
Summary: Handle reordering expansions with variable path length

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D579
2017-07-26 12:36:47 +02:00
Teon Banek
b33aae42ab Add ALL function to openCypher
Summary:
Add All expression to Ast
Evaluate All expression
Visit All and generate symbols
Handle All when collecting context during planning

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D587
2017-07-25 16:32:45 +02:00
Teon Banek
da7bfe05b7 Collect symbols in variable length path
Summary:
Add EdgeList symbol type and check for redeclaration

The result of variable path is a list of edges, so the symbol type has
been added. In the future, we need to extend the type checker and the
type structure to have a generic list type.

We also currently do not support reusing an already bound symbol for a
variable path, so the SymbolGenerator will raise a redeclaration error.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot, lion

Differential Revision: https://phabricator.memgraph.io/D574
2017-07-25 12:39:14 +02:00
Teon Banek
20f2c885c8 Test grammar for variable length with filters
Summary: Update Cypher grammar to pickup properties after range

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D578
2017-07-21 11:51:55 +02:00
florijan
7328f5ec7f Query::Plan::VariableExpand added
Summary:
Variable expansion logical operator added. Some functionalities are missing:
- taking into account optional matching when expanding into existing symbol
- accepting Expression bounds (current implementation takes size_t)

Also, a TODO is added for handling optional matching in the uniqueness operator (with an Asana task)

All this will be done in the following diff, this is already substantial.

Also, please consider if we want to have all those `VLOG`s in the code. Not very pretty. And I think that `VLOG` is not compiled-away in release build, will put an asana task.

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D569
2017-07-20 14:26:41 +02:00
Teon Banek
b4d2d1ff81 Support variable length path in CypherMainVisitor
Summary:
Allow expressions for variable length path bounds

Replace test which expected a syntax exception

Since we now allow variable length to have an arbitrary expression, the
test case is obsolete. It was replaced with something that excepts an
expression which wasn't allowed before.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D568
2017-07-20 13:14:50 +02:00
Mislav Bradac
e8a465e4b5 Add query parameters support
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D570
2017-07-19 18:44:59 +02:00
florijan
a7d8b7124e GraphDbAccessor - index range API
Summary:
- GraphDbAccessor - index range API added
- index api tests refactored
- skiplist minor cleanup.

Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D533
2017-07-19 11:55:40 +02:00
Teon Banek
16d94c8aaf Make the message mandatory for NotYetImplemented exception
Reviewers: florijan, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D565
2017-07-18 16:40:49 +02:00
Teon Banek
1ad3b7f906 Remove unused time_second from query/engine.hpp
Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D562
2017-07-17 16:56:13 +02:00
Teon Banek
104fe22f53 Plan ScanAll by using a range index
Summary:
Add optional bounds to PropertyFilter and collect them

Relation operators (e.g. `<`, `>` ...) should be used to produce
scanning the index by a range of values. For that reason, PropertyFilter
is extended to store either the equality expression or range bounds.
The `AnalyzeFilter` function is extended to look for those operators and
see if their top level expression contains a property lookup. If it
does, a filter with a bound is generated.

Test for property comparison preventing index use

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D529
2017-07-17 16:34:07 +02:00
Teon Banek
3d8d0efea5 Convert CartesianProduct to iterable
Summary:
This change allows for lazy calculation of Cartesian product, by using
an iterator. Using lazy evaluation, we can easily limit the number of
generated products and therefore the number of generated query plans.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D554
2017-07-17 16:30:06 +02:00
Mislav Bradac
4dabf31005 Remove unused function time_second
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D561
2017-07-17 15:55:43 +02:00
Mislav Bradac
50cd53e5c9 Migrate timer to use walltime, instead of cputime
Summary:
 - Move some stuff to poc
 - Use steady_clock instead of system_clock

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D555
2017-07-17 13:42:44 +02:00
Mislav Bradac
4a500b99fa Add a way to stop query execution
Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D552
2017-07-14 14:36:44 +02:00
Teon Banek
387c0621cb Use division to detect unsigned wrap-around
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D550
2017-07-13 15:17:19 +02:00
Teon Banek
99e27cb7e9 Add limiting number of generated plans for a query
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D548
2017-07-13 14:27:15 +02:00
Mislav Bradac
26144ee8fd Fix bugs in MatchReal (StrippedQuery)
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D546
2017-07-12 16:50:53 +02:00
Mislav Bradac
6068a95a0e Move dbms to database
Summary:
There was only two files in dbms directory so I moved them to database
directory.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D540
2017-07-12 12:44:11 +02:00