Commit Graph

1426 Commits

Author SHA1 Message Date
jseljan
9831f0396a Fix 'all' and 'single' functions; update their unit and TCK tests
Summary:
Semantics of 'all' and 'single' were updated to be
consistent with that of 'any' and 'none'

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2789
2020-06-24 13:43:25 +02:00
jseljan
8bfebfca9d Add 'none' function implementation, its unit and TCK tests
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2788
2020-06-23 13:06:06 +02:00
jseljan
21cee1eaec Fix any function implementation
Summary: Change any function's handling of Null elements in a list

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2787
2020-06-19 15:40:57 +02:00
jseljan
098333f735 Add prototype implementation of the any function
Summary: Add any function prototype - no tests

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2785
2020-06-10 17:01:24 +02:00
Matej Ferencevic
3dd393f8eb Fix multi-command transaction handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2784
2020-06-10 12:25:41 +02:00
Matej Ferencevic
6628d20e5a Make DUMP DATABASE work correctly in explicit transactions
Summary:
`DUMP DATABASE` used a separate transaction to read database data. That
wouldn't be an issue if the query was correctly disallowed in multicommand
transactions. Because it was allowed the output wasn't transactionally correct.
Instead of disabling `DUMP DATABASE` in multicommand transactions this change
fixes it so that it works properly in multicommand transactions.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2781
2020-06-08 15:56:16 +02:00
Matej Ferencevic
b923d2bc36 Integrate property store
Reviewers: buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2764
2020-06-01 14:11:11 +02:00
Tonko Sabolcec
3932376301 Return properties as a list in SHOW CONSTRAINT INFO for unique constraints
Summary:
Before this change properties were joined by ", " and returned as a single string,
which was ambiguous for properties that contain ", ". This diff solves this
problem by returning properties as a list type.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2778
2020-05-29 12:59:33 +02:00
Tonko Sabolcec
59bc9d8989 Fix label/edge/property name escaping in DUMP DATABASE
Summary:
This diff fixes the issue for label name (and edge type/property)
with spaces and special characters to avoid possible OpenCypher injections.
Consider an example where label name is 'hello :world'. `DUMP DATABASE`
used to return query which creates a node (u:hello :world) - i.e. node
that contains two labels 'hello' and 'world'. This fix escapes names to
create the following node with exactly one label as expected:
```
(u:`hello :world`)
```

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2774
2020-05-21 14:46:51 +02:00
Matej Ferencevic
f047f55020 Implement IsPropertyEqual for property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2768
2020-05-20 15:41:13 +02:00
Lovro Lugovic
0c42bedf2f Add support for rescanning query modules
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2765
2020-05-13 21:33:28 +02:00
Matej Ferencevic
4d2eda398f Improve property store performance
Summary:
This diff improves the performance of `PropertyStore` with two main
techniques:

First:
`PropertyValue` has a very expensive constructor and destructor. The
`PropertyValue` was previously passed as a return value from many functions
wrapped in a `std::optional`. That caused the `PropertyValue`
constructor/destructor to be called for each intermediary value that was passed
between functions. This diff changes the functions to return a `bool` value
that imitates the `std::optional` "emptyness" flag and the `PropertyValue` is
modified using a pointer to it so that its constructor/destructor is called
only once.

Second:
The `PropertyStore` buffer was previously iterated through at least twice.
First to determine the exact position of the encoded property and then to
actually decode the property. This diff combines the two passes into a single
pass so that the property is immediately loaded if it is found.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2766
2020-05-05 10:55:00 +02:00
Marko Budiselic
f35225f26b Optimize IN list execution
Summary:
Use Unwind + ScanAllByLabelPropertyValue logical operator to
accelerate the execution of queries like the following one:
`MATCH (n:Label) WHERE n.property IN [] ...`

Reviewers: llugovic, mferencevic

Reviewed By: llugovic, mferencevic

Subscribers: llugovic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2758
2020-04-21 11:38:38 +02:00
Matej Ferencevic
d04993df67 Improve storage unit test
Summary: The tests are extracted from D2570.

Reviewers: llugovic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2745
2020-04-01 15:26:44 +02:00
Matej Ferencevic
7e35798401 Remove leftover raft unit test
Reviewers: llugovic, buda

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2744
2020-04-01 13:48:02 +02:00
Matej Ferencevic
a1b5bdd88f Implement concurrent tests for storage indices
Summary: The same test is written both for label indices and label+property indices.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2741
2020-04-01 10:43:05 +02:00
Teon Banek
b7738c64b3 Make py::Object conversion to PyObject * explicit
Summary:
This fixes an issue in Py(Vertex|Edge)GetProperty and prevents any
further issues of that type at the cost of additional typing effort.

Reviewers: ipaljak, llugovic

Reviewed By: ipaljak, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2735
2020-03-24 12:23:13 +01:00
Teon Banek
2b8f068ca9 Correctly delimit arguments when printing signature
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2732
2020-03-19 14:19:53 +01:00
Teon Banek
008efaf243 Implement pretty printing CallProcedure operator
Reviewers: mferencevic, llugovic, ipaljak

Reviewed By: mferencevic, llugovic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2726
2020-03-18 10:32:16 +01:00
Teon Banek
d63eb191f9 Fix a planning issue when CALL preceded filtering
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2722
2020-03-16 15:06:39 +01:00
Matej Ferencevic
f48ad62647 Implement storage lock
Summary:
The storage now uses a file in the data directory (`.lock`) to determine
whether there is another instance of the storage running with the same data
directory. That helps notify the user/administrator that the system is running
in an unsupported configuration.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2719
2020-03-13 19:27:01 +01:00
Lovro Lugovic
dd5d29fa1d Implement conversion of mgp_vertex to/from mgp.Vertex
Summary:
- Fix AssertPickleAndCopyAreNotSupported
- Test Vertex and Edge conversions

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2717
2020-03-11 16:13:24 +01:00
Matej Ferencevic
5695774251 Improve CSV importer
Summary:
The importer now supports all of the flags that the modern Neo4j CSV importer
supports.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2709
2020-03-10 15:51:29 +01:00
Tonko Sabolcec
7ea2d1b638 Add concurrent test for unique constraints
Summary:
This diff contains simple tests for unique constraints which tries to
change property values or labels in multiple threads at the same time.
During testing, a bug has been encountered in unique constraints, i.e.
one guard lock on vertices was missing.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2711
2020-03-10 12:52:58 +01:00
Tonko Sabolcec
6f83fff171 Implement durability functionality for unique constraints
Summary:
This diff contains a necessary functionality to save and restore unique
constraint operations. The previous snapshot/WAL version is backward
compatible. Integration tests for migration from older snapshot and WAL
versions are also included.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2680
2020-03-06 14:14:54 +01:00
Matej Ferencevic
8363991575 Implement new CSV parser
Summary:
The new CSV parser in `mg_import_csv` behaves the same when importing a CSV
file as the standard Python CSV importer. Tests are added for all CSV field
edge-cases.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2702
2020-03-06 11:32:54 +01:00
Teon Banek
1d30a2e5cb Convert mgp_value to 'mgp' Python types, not '_mgp' ones
Summary:
When invoking a Python registered procedures we want to convert
`mgp_value` types to user facing 'mgp' types.

Depends on D2706

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2707
2020-03-06 11:23:08 +01:00
Teon Banek
456267249e Convert mgp_list to PyTuple instead of PyList
Summary: It's much safer to give users immutable types.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2705
2020-03-06 11:22:18 +01:00
Teon Banek
74d9dd0b0a Implement mgp.Path
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2700
2020-03-04 12:51:06 +01:00
Matej Ferencevic
a456d6cdc0 Implement mg_import_csv
Summary:
This diff restores (and fixes) the old mg_import_csv implementation. The
importer now supports the new storage engine.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2690
2020-03-03 14:33:43 +01:00
Tonko Sabolcec
14c4c6d060 Update query dump with unique constraints
Summary:
Now that unique constraint feature is added to Memgraph database,
we should update `DUMP DATABASE` with list of existing unique constraints.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2698
2020-03-02 13:50:35 +01:00
Tonko Sabolcec
eed83a210e Implement unique constraint functionality in query module
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2681
2020-03-02 13:00:04 +01:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
2020-03-02 12:45:18 +01:00
Matej Ferencevic
bfbace8168 Remove old HA implementation
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2686
2020-02-25 17:19:35 +01:00
Teon Banek
e7f363ecbe Add Python class for mgp_edge
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2679
2020-02-25 10:26:08 +01:00
Matej Ferencevic
47ce444c02 Fix auth Cypher visitor
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2675
2020-02-17 17:41:08 +01:00
Matej Ferencevic
e8903ef477 Expose auth user/role name regex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2673
2020-02-17 17:40:53 +01:00
Matej Ferencevic
dfc546183d Add community to Apollo build
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2671
2020-02-17 13:27:20 +01:00
Lovro Lugovic
5953f07be3 Add conversion of py::Object to mgp_value
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2667
2020-02-14 16:08:20 +01:00
Tonko Sabolcec
95c9755e13 Support property set for unique constraints
Summary:
Before this change, unique constraints supported only pairs of label
and a single property. With this change, unique constraints can be
created for label and set of properties.
Better tests for unique constraints in general are also included in
this diff.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2653
2020-02-13 12:41:35 +01:00
Teon Banek
0a7de969f3 Add converting mgp_value to py::Object
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2665
2020-02-12 09:30:14 +01:00
Tonko Sabolcec
e2e7823ec4 Add basic support for unique constraints in storage
Summary:
This diff contains a basic implementation of unique constraints consistent with
the MVCC storage.
Stale records in the unique constraints are collected by the garbage collector.
Tests for checking correctness of unique constraints and violations are included.
Note: currently we only support a pair of label and a single property. Support for
multiple properties will be added later.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: buda, ipaljak, pullbot

Differential Revision: https://phabricator.memgraph.io/D2608
2020-02-10 12:45:28 +01:00
Matej Ferencevic
7128320b36 Implement MG_ENTERPRISE CMake flag
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2658
2020-02-06 13:39:30 +01:00
Matej Ferencevic
3e20afde6e Increase stress test query timeout
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2657
2020-02-05 14:52:13 +01:00
Matej Ferencevic
6c422f3208 Remove global include directories in CMake
Summary:
All external libraries now automatically include their include directories. It
is necessary only to link to the external library using
`target_link_libraries(target library)` and the include directory of the
library will be automatically available for the binary.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2654
2020-02-03 13:18:30 +01:00
Matej Ferencevic
c96421424f Reorganize unit CMakeLists
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2651
2020-01-31 11:18:36 +01:00
Matej Ferencevic
84a6ab75cb Fix race condition in long running stress test
Summary:
The long running stress test had a subtle race condition which caused the test
to fail with an error message like "Runner X edges creation failed because of:
Can't serialize due to concurrent operations.". This situation was caused
because some workers could complete their initialization (initial vertex and
edge creation) before other workers. The workers that completed their
initialization would then proceed to execute the test. In the test they could
execute queries that make global updates on the graph that could interfere with
the concurrently running initialization queries of other workers.

This diff makes the runners wait until all initialization queries are fully
executed before they execute global operations on the graph.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2647
2020-01-29 10:19:53 +01:00
Matej Ferencevic
de48548164 Move RPC to root source directory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2641
2020-01-27 11:50:24 +01:00
Matej Ferencevic
fd81ebdfe3 Move KVStore to root source directory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2640
2020-01-24 16:01:51 +01:00
Matej Ferencevic
0c7313bb5f Remove unused datastructures
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2639
2020-01-24 15:11:02 +01:00