Commit Graph

562 Commits

Author SHA1 Message Date
Mislav Bradac
74d0a426ab Use \u for utf16 and \U for utf32
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D773
2017-09-09 16:16:30 +02:00
Teon Banek
92b9bbd4bd Remove const requirement from DbAccessor in planning
Summary: The constness of the DbAccessor interferes with caching the results.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D771
2017-09-09 14:06:32 +02:00
Teon Banek
0aee18544d Add caching VerticesCount during planning and estimation
Summary:
Benchmark planning and estimating indexed ScanAll. According to the benchmark,
caching speeds up the whole process of planning and estimation by a factor of
2. Most of the performance gain is in the `CostEstimator` itself, due to plenty
of calls to `VerticesCount` when estimating all of the generated plans.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D765
2017-09-09 12:26:48 +02:00
Teon Banek
f0b1f24006 unit.recovery: Read vertex IDs into int64_t, not int
Summary:
This fixes the problem when the test would fail on some PCs...
Notably, mine...

Reviewers: florijan, mferencevic, mislav.bradac, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D770
2017-09-09 09:24:42 +02:00
Matej Ferencevic
541a0dae24 Addded query error message to harness client.
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D766
2017-09-08 15:30:58 +02:00
florijan
2c4966edfe Vertex edges data structure introduced
Summary:
- The `Edges` data structure now handles common ops, including providing an iterator over edges whose "other" vertex is know.
- This should improve performance on dense_expand tests in the harness without other side-effects.
- query::plan::Expand operator modified not to check for existing-node stuff since that now gets handled by the `Edges` data structure.
- `Edges::Iterator` implemented only for const iterators since that suffices for now. Can implement non-const if the need arrises.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D763
2017-09-08 13:21:21 +02:00
florijan
9f8361dd27 Edge storage benchmark added
Summary: For the time being we'll go with vector and a custom iterator for know-destination-vertex lookups, but this benchmark might be handy to keep for future work.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D762
2017-09-08 13:12:20 +02:00
florijan
6edf2cc5ab BFS feature test that uses edges added
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D768
2017-09-08 13:03:57 +02:00
Matej Ferencevic
21782901c1 Added 3 iterations to macro benchmark.
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D759
2017-09-08 11:23:25 +02:00
Teon Banek
c8d34c58ef Generate indexed ScanAll to make Expand existing
Summary: Test generating ScanAll before expand to use existing

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: buda, lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D746
2017-09-08 09:50:18 +02:00
florijan
63966c15a6 LDBC Query1 converted
Summary: Ugly formatting and bad variable naming due to 1024 limit on neo4j client.

Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D748
2017-09-08 09:30:22 +02:00
Mislav Bradac
38c83d80cb Rename Executioner to Executor
Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D764
2017-09-07 15:13:17 +02:00
florijan
be16409da2 Harness MATCH tests refactored
Summary:
1. Test setup rewritten to take cca 8 seconds. Note that edges are created by using:
`MATCH (a) WITH a MATCH (b) WITH b WHERE rand() < X CREATE (a)-[:ET]->(b)`
Where `X` is a threshold calculated so the desired edge count is the expectation. This seems the only feasable way of generating a large number of edges since query execution does not depend on edge count, but on vertex count.

2. Using the new `assert` function to verify graph state. I recommend doing that in all the harness tests (I don't think we currently have something better).

3. All tests rewritten to take around 200ms per iteration.

4. Test are using SKIP to avoid sending data to the client, but ensure that appropriate operations get executed. This currently seems like the best way of removing unwanted side-effects.

Harness will cost us our sanity. And it doesn't even provide good quality regression testing we really need :(

Reviewers: buda, mislav.bradac, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D752
2017-09-07 12:05:05 +02:00
Teon Banek
1693530d92 ldbc: Separate running short reads from updates
Reviewers: buda

Reviewed By: buda

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D758
2017-09-07 10:45:09 +02:00
Matej Ferencevic
f848394e5d Fixed bolt chunked encoder buffer for big data.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D756
2017-09-06 15:36:24 +02:00
Matej Ferencevic
642c2f07bb Added PostgreSQL support to harness client.
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D753
2017-09-06 15:22:25 +02:00
florijan
cd0d8eb543 Harness - dense expand group added, more tests on bench
Summary: It turns out trivial if I use unwind for vertex creation, MATCH for edge creation and UNWIND for test duration. It took hours to converge to this :(

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D747
2017-09-06 14:58:50 +02:00
Matej Ferencevic
8cf69738bf Fixed bolt decoder chunked buffer implementation.
Summary:
Bolt client now uses new chunked decoder.

Fixed bolt session to use new chunked decoder.

Fixed chunked decoder buffer test.

Fixed bolt session test.

Removed debug message from client.

Fixed bolt encoder to comply with specification.

Reviewers: buda, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D751
2017-09-06 13:19:42 +02:00
florijan
4975743f35 Assert function added
Summary: `assert` function added. Useful to us for asserting DB state in harness tests. Potentially useful to the client for breaking out of a query as soon as a predicate fails, as opposed to collecting result and checking them client-side.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D750
2017-09-05 13:57:56 +02:00
Matej Ferencevic
70d9f3f6f1 Refactored harness and added PostgreSQL support.
Summary:
Moved Neo4j config to config dir.

Neo4j and PostgreSQL are now downloaded to libs.

Renamed metadata flags in memgraph.

Changed apollo generate for new harness.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D741
2017-09-05 09:45:41 +02:00
Marko Budiselic
b30e3252e2 A python script that visualizes latency results from LDBC.
Summary:
  * extension of run_benchmark script
  * tools/plot_latency.py

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D743
2017-09-04 16:36:42 +02:00
Mislav Bradac
c56621682e Parse utf16 surrogate codepoints correctly
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D745
2017-09-04 16:03:45 +02:00
Teon Banek
d584a7b18b Test ExpandVariable reverses on Direction::BOTH
Summary: Add `is_flipped` to `Expansion`

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D744
2017-09-04 15:35:17 +02:00
florijan
f68bac922f Map indexing added
Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D739
2017-09-04 09:12:46 +02:00
florijan
21550d3bb1 LDBC queries checked
Summary:
- checked existing queries and added new ones
- minor changes to LDBC setup

Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D737
2017-09-02 10:36:02 +02:00
Teon Banek
77e574fcc5 Cleanup some errors in ldbc setup scripts
Summary:
Don't require setup_system to run as root, nor apt-get
Implement command_fail for ldbc/setup_dependencies
ldbc.setup_dataset: Find Java on ArchLinux

Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D729
2017-09-01 14:13:34 +02:00
Teon Banek
26297ca641 ldbc: Extract update queries
Summary:
Convert update queries to oC supported by MG
Ignore ve2 in ldbc

Reviewers: buda, mislav.bradac, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D733
2017-09-01 14:13:04 +02:00
Mislav Bradac
2111612fc8 Run Neo on ramdisk in harness
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D734
2017-09-01 12:04:42 +02:00
florijan
7f40607dd3 TypedValue test and implementation improvements
Summary: Some TypedValue arithmetic ops threw exceptions for the wrong reasons, the op applicability type checks were wrong. Also the tests for that behavior were wrong. These are the fixes.

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D724
2017-09-01 11:21:32 +02:00
Mislav Bradac
bba5d134c0 Refactor harness
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D732
2017-08-31 11:23:11 +02:00
Teon Banek
1e87f888ae Correctly transform exists to is not null check
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D731
2017-08-31 09:01:21 +02:00
Teon Banek
52709ad04c Inline filter inside ExpandVariable
Summary:
Reorder class definition in ast.hpp.
Test inlining filters in ExpandVariable.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D726
2017-08-30 16:23:16 +02:00
Mislav Bradac
e68f7ea536 Use median instead of avg in harness
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D725
2017-08-30 15:28:30 +02:00
Teon Banek
fe6d64066b Plan Filter at the end of With clause
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D727
2017-08-30 15:21:28 +02:00
Mislav Bradac
9a39a125b2 Remove rand from dba
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D721
2017-08-29 12:28:32 +02:00
Mislav Bradac
3e793fb8ac Add single_create group to harness
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D714
2017-08-28 11:45:12 +02:00
Teon Banek
7fc821ad25 Setup basic interactive planner testing
Summary:
Add basic InteractiveDbAccessor.
Add printing logical operator tree.
Cache some InteractiveDbAccessor vertex counts.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D707
2017-08-28 10:25:17 +02:00
Mislav Bradac
409c9200e2 Remove unnecessary requirements
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D717
2017-08-28 10:19:14 +02:00
Marko Budiselic
a8e0792609 LDBC
Summary: Add LDBC helper scripts.

Reviewers: teon.banek, mferencevic, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D563
2017-08-28 09:34:22 +02:00
Mislav Bradac
7628084ccf Add no-strict option to harness
Reviewers: mferencevic

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D716
2017-08-25 15:07:21 +02:00
florijan
ca8fb55ac5 Collect Map added
Summary:
Tests are on the way. Please first comment if you're OK with this implementation, some points are discussable.

What works now:
```
bash:MEMGRAPH_ROOT/build/>./tests/manual/console 10
MG>MATCH (n) RETURN COLLECT("age_" + n.age, n.height)

+-----------------------------------------------------------------------------------------------------------------------------------+
| COLLECT("age_" + n.age, n.height)                                                                                                 |
+-----------------------------------------------------------------------------------------------------------------------------------+
| {age_10: 176, age_13: 180, age_24: 172, age_25: 179, age_32: 123, age_33: 186, age_37: 147, age_43: 162, age_49: 126, age_6: 170} |
+-----------------------------------------------------------------------------------------------------------------------------------+
```

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D695
2017-08-25 12:51:18 +02:00
Matej Ferencevic
0914c5a941 Migrate harness to use our bolt client
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D711
2017-08-25 11:10:15 +02:00
Mislav Bradac
19bec4acc8 Fix expansion benchmark
Reviewers: florijan, mferencevic, teon.banek, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D708
2017-08-25 10:09:21 +02:00
Matej Ferencevic
5a2086b18f Libraries are now downloaded from our local cache server.
Summary:
Improved libs cleanup script.

The root init script now downloads everything.

Removed obsolete init scripts.

Reviewers: buda, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D710
2017-08-25 09:58:39 +02:00
Matej Ferencevic
c507e74384 First version of bolt cpp client.
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D689
2017-08-24 17:44:44 +02:00
Matej Ferencevic
9d1265f41f Initial version of macro benchmark summary script.
Summary:
Changed harness output to be compatible with script.

Changed cppcheck message formatting.

Changed qa message formatting.

Changed macro benchmark summary icons.

Fixed newlines in status generators.

Added group name to harness summary.

Macro benchmark summary now handles different files.

Added more output options to summary script.

Diff builds now build parent for performance comparison.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D706
2017-08-24 15:11:36 +02:00
Teon Banek
fdc389f1eb Templatize CostEstimator on DbAccessor
Summary:
This allows for inserting dummy DbAccessor in tests. Unfortunate side
effect of this change is that the whole implementation had to be moved
from cpp to hpp.

Also templatize remaining RuleBasedPlanner implementation

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D704
2017-08-24 14:27:14 +02:00
florijan
2954276ca8 Property storage now supports Map
Summary:
Added:
- map support in PropertyValue
- conversion of map TypedValue to PropertyValue if appropriate flag is set (undocumented because it's private)
- ordering of map PropertyValue in LabelPropertyIndex
- issue raised regarding list and value property modifications in storage (currently unsupported)

Maybe I missed some feature or whatever?

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D692
2017-08-24 10:29:49 +02:00
Teon Banek
591d086013 Map symbols to expansions to speed up variable planning
Summary:
Test variable planning BFS.
Add more tests for variably planning ExpandVariable.
Don't recreate the whole matching when varying expansions.
Use explicit constructors in private planner classes.

Reviewers: mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D691
2017-08-23 11:39:45 +02:00
florijan
b23cb43bd4 Benchmark Expand - disabled Match testing
Summary: Disabling it because it seems that (singleton + fixture + multiple_benchmarks) cause occasional crashes of the test. The more important test is left enabled.

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D696
2017-08-23 11:26:48 +02:00