Commit Graph

1434 Commits

Author SHA1 Message Date
Matej Ferencevic
04ceb8d4b1 Implement valueType openCypher function
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2818
2020-09-08 16:49:53 +02:00
Matej Ferencevic
d2ff465f8e Fix Cypher ID function Null handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2815
2020-09-04 13:18:08 +02:00
Matej Ferencevic
dd9180da32 Add request streaming support to the RPC client
Summary:
This change only adds streaming support to the client request. The client
response, server request and server response are still handled only when all of
the data is received.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2807
2020-08-14 16:19:48 +02:00
Matej Ferencevic
aaf0c1ca08 Improve SLK
Summary:
SLK now correctly handles different CPU architectures (BIG/little endian).
Also, more string encoding functions have been added.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2806
2020-08-14 15:27:20 +02:00
Matej Ferencevic
1513a455de Merge durability and storage
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2803
2020-08-14 15:08:43 +02:00
Matej Ferencevic
857de23687 Split storage durability implementation into multiple files
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2760
2020-08-14 13:36:52 +02:00
Matej Ferencevic
28590aea53 Fix variable expand between same symbol
Summary:
This diff fixes the variable expand operator to work correctly then the start
and destination nodes use the same symbol or when the destination symbol is an
existing symbol.

Previously, the variable expand operator produced paths that were both
completely wrong (they shouldn't have been produced) and nonexistent (they
didn't even exist in the storage). Invalid data was produced because of a
wrong equality check that was introduced in D1703.

This issue was reported externally and the supplied test case was:
```
CREATE (p1:Person {id: 1})-[:KNOWS]->(:Person {id: 2})-[:KNOWS]->(:Person {id: 3})-[:KNOWS]->(:Person {id: 4})-[:KNOWS]->(p1);
MATCH path = (pers:Person {id: 3})-[:KNOWS*2]->(pers) RETURN path;
```

Also, tests have been added so the behavior remains correct.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2793
2020-07-01 00:17:25 +02:00
jseljan
a7b672ebbf Add TCK tests relating predicate functions
Summary:
Implement TCK regression-like tests relating 'any', 'all', and 'none'
predicate functions

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2791
2020-06-29 12:56:59 +02:00
jseljan
9831f0396a Fix 'all' and 'single' functions; update their unit and TCK tests
Summary:
Semantics of 'all' and 'single' were updated to be
consistent with that of 'any' and 'none'

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2789
2020-06-24 13:43:25 +02:00
jseljan
8bfebfca9d Add 'none' function implementation, its unit and TCK tests
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2788
2020-06-23 13:06:06 +02:00
jseljan
21cee1eaec Fix any function implementation
Summary: Change any function's handling of Null elements in a list

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2787
2020-06-19 15:40:57 +02:00
jseljan
098333f735 Add prototype implementation of the any function
Summary: Add any function prototype - no tests

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2785
2020-06-10 17:01:24 +02:00
Matej Ferencevic
3dd393f8eb Fix multi-command transaction handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2784
2020-06-10 12:25:41 +02:00
Matej Ferencevic
6628d20e5a Make DUMP DATABASE work correctly in explicit transactions
Summary:
`DUMP DATABASE` used a separate transaction to read database data. That
wouldn't be an issue if the query was correctly disallowed in multicommand
transactions. Because it was allowed the output wasn't transactionally correct.
Instead of disabling `DUMP DATABASE` in multicommand transactions this change
fixes it so that it works properly in multicommand transactions.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2781
2020-06-08 15:56:16 +02:00
Matej Ferencevic
b923d2bc36 Integrate property store
Reviewers: buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2764
2020-06-01 14:11:11 +02:00
Tonko Sabolcec
3932376301 Return properties as a list in SHOW CONSTRAINT INFO for unique constraints
Summary:
Before this change properties were joined by ", " and returned as a single string,
which was ambiguous for properties that contain ", ". This diff solves this
problem by returning properties as a list type.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2778
2020-05-29 12:59:33 +02:00
Tonko Sabolcec
59bc9d8989 Fix label/edge/property name escaping in DUMP DATABASE
Summary:
This diff fixes the issue for label name (and edge type/property)
with spaces and special characters to avoid possible OpenCypher injections.
Consider an example where label name is 'hello :world'. `DUMP DATABASE`
used to return query which creates a node (u:hello :world) - i.e. node
that contains two labels 'hello' and 'world'. This fix escapes names to
create the following node with exactly one label as expected:
```
(u:`hello :world`)
```

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2774
2020-05-21 14:46:51 +02:00
Matej Ferencevic
f047f55020 Implement IsPropertyEqual for property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2768
2020-05-20 15:41:13 +02:00
Lovro Lugovic
0c42bedf2f Add support for rescanning query modules
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2765
2020-05-13 21:33:28 +02:00
Matej Ferencevic
4d2eda398f Improve property store performance
Summary:
This diff improves the performance of `PropertyStore` with two main
techniques:

First:
`PropertyValue` has a very expensive constructor and destructor. The
`PropertyValue` was previously passed as a return value from many functions
wrapped in a `std::optional`. That caused the `PropertyValue`
constructor/destructor to be called for each intermediary value that was passed
between functions. This diff changes the functions to return a `bool` value
that imitates the `std::optional` "emptyness" flag and the `PropertyValue` is
modified using a pointer to it so that its constructor/destructor is called
only once.

Second:
The `PropertyStore` buffer was previously iterated through at least twice.
First to determine the exact position of the encoded property and then to
actually decode the property. This diff combines the two passes into a single
pass so that the property is immediately loaded if it is found.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2766
2020-05-05 10:55:00 +02:00
Marko Budiselic
f35225f26b Optimize IN list execution
Summary:
Use Unwind + ScanAllByLabelPropertyValue logical operator to
accelerate the execution of queries like the following one:
`MATCH (n:Label) WHERE n.property IN [] ...`

Reviewers: llugovic, mferencevic

Reviewed By: llugovic, mferencevic

Subscribers: llugovic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2758
2020-04-21 11:38:38 +02:00
Matej Ferencevic
d04993df67 Improve storage unit test
Summary: The tests are extracted from D2570.

Reviewers: llugovic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2745
2020-04-01 15:26:44 +02:00
Matej Ferencevic
7e35798401 Remove leftover raft unit test
Reviewers: llugovic, buda

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2744
2020-04-01 13:48:02 +02:00
Matej Ferencevic
a1b5bdd88f Implement concurrent tests for storage indices
Summary: The same test is written both for label indices and label+property indices.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2741
2020-04-01 10:43:05 +02:00
Teon Banek
b7738c64b3 Make py::Object conversion to PyObject * explicit
Summary:
This fixes an issue in Py(Vertex|Edge)GetProperty and prevents any
further issues of that type at the cost of additional typing effort.

Reviewers: ipaljak, llugovic

Reviewed By: ipaljak, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2735
2020-03-24 12:23:13 +01:00
Teon Banek
2b8f068ca9 Correctly delimit arguments when printing signature
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2732
2020-03-19 14:19:53 +01:00
Teon Banek
008efaf243 Implement pretty printing CallProcedure operator
Reviewers: mferencevic, llugovic, ipaljak

Reviewed By: mferencevic, llugovic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2726
2020-03-18 10:32:16 +01:00
Teon Banek
d63eb191f9 Fix a planning issue when CALL preceded filtering
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2722
2020-03-16 15:06:39 +01:00
Matej Ferencevic
f48ad62647 Implement storage lock
Summary:
The storage now uses a file in the data directory (`.lock`) to determine
whether there is another instance of the storage running with the same data
directory. That helps notify the user/administrator that the system is running
in an unsupported configuration.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2719
2020-03-13 19:27:01 +01:00
Lovro Lugovic
dd5d29fa1d Implement conversion of mgp_vertex to/from mgp.Vertex
Summary:
- Fix AssertPickleAndCopyAreNotSupported
- Test Vertex and Edge conversions

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2717
2020-03-11 16:13:24 +01:00
Matej Ferencevic
5695774251 Improve CSV importer
Summary:
The importer now supports all of the flags that the modern Neo4j CSV importer
supports.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2709
2020-03-10 15:51:29 +01:00
Tonko Sabolcec
7ea2d1b638 Add concurrent test for unique constraints
Summary:
This diff contains simple tests for unique constraints which tries to
change property values or labels in multiple threads at the same time.
During testing, a bug has been encountered in unique constraints, i.e.
one guard lock on vertices was missing.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2711
2020-03-10 12:52:58 +01:00
Tonko Sabolcec
6f83fff171 Implement durability functionality for unique constraints
Summary:
This diff contains a necessary functionality to save and restore unique
constraint operations. The previous snapshot/WAL version is backward
compatible. Integration tests for migration from older snapshot and WAL
versions are also included.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2680
2020-03-06 14:14:54 +01:00
Matej Ferencevic
8363991575 Implement new CSV parser
Summary:
The new CSV parser in `mg_import_csv` behaves the same when importing a CSV
file as the standard Python CSV importer. Tests are added for all CSV field
edge-cases.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2702
2020-03-06 11:32:54 +01:00
Teon Banek
1d30a2e5cb Convert mgp_value to 'mgp' Python types, not '_mgp' ones
Summary:
When invoking a Python registered procedures we want to convert
`mgp_value` types to user facing 'mgp' types.

Depends on D2706

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2707
2020-03-06 11:23:08 +01:00
Teon Banek
456267249e Convert mgp_list to PyTuple instead of PyList
Summary: It's much safer to give users immutable types.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2705
2020-03-06 11:22:18 +01:00
Teon Banek
74d9dd0b0a Implement mgp.Path
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2700
2020-03-04 12:51:06 +01:00
Matej Ferencevic
a456d6cdc0 Implement mg_import_csv
Summary:
This diff restores (and fixes) the old mg_import_csv implementation. The
importer now supports the new storage engine.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2690
2020-03-03 14:33:43 +01:00
Tonko Sabolcec
14c4c6d060 Update query dump with unique constraints
Summary:
Now that unique constraint feature is added to Memgraph database,
we should update `DUMP DATABASE` with list of existing unique constraints.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2698
2020-03-02 13:50:35 +01:00
Tonko Sabolcec
eed83a210e Implement unique constraint functionality in query module
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2681
2020-03-02 13:00:04 +01:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
2020-03-02 12:45:18 +01:00
Matej Ferencevic
bfbace8168 Remove old HA implementation
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2686
2020-02-25 17:19:35 +01:00
Teon Banek
e7f363ecbe Add Python class for mgp_edge
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2679
2020-02-25 10:26:08 +01:00
Matej Ferencevic
47ce444c02 Fix auth Cypher visitor
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2675
2020-02-17 17:41:08 +01:00
Matej Ferencevic
e8903ef477 Expose auth user/role name regex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2673
2020-02-17 17:40:53 +01:00
Matej Ferencevic
dfc546183d Add community to Apollo build
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2671
2020-02-17 13:27:20 +01:00
Lovro Lugovic
5953f07be3 Add conversion of py::Object to mgp_value
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2667
2020-02-14 16:08:20 +01:00
Tonko Sabolcec
95c9755e13 Support property set for unique constraints
Summary:
Before this change, unique constraints supported only pairs of label
and a single property. With this change, unique constraints can be
created for label and set of properties.
Better tests for unique constraints in general are also included in
this diff.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2653
2020-02-13 12:41:35 +01:00
Teon Banek
0a7de969f3 Add converting mgp_value to py::Object
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2665
2020-02-12 09:30:14 +01:00
Tonko Sabolcec
e2e7823ec4 Add basic support for unique constraints in storage
Summary:
This diff contains a basic implementation of unique constraints consistent with
the MVCC storage.
Stale records in the unique constraints are collected by the garbage collector.
Tests for checking correctness of unique constraints and violations are included.
Note: currently we only support a pair of label and a single property. Support for
multiple properties will be added later.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: buda, ipaljak, pullbot

Differential Revision: https://phabricator.memgraph.io/D2608
2020-02-10 12:45:28 +01:00