Commit Graph

1020 Commits

Author SHA1 Message Date
antonio2368
42f6118c00 Add safe file deletion utility FileRetainer (#38)
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
antonio2368
b10255a12f Add initial support for multiple clients (#31)
* Add tests for multiple clients
* Use variant for RPC server and clients
* Using synchronized list for replication clients, extracted variant access to a function
* Set MAIN as default, add unregister function, add a name for replication clients
* Use the regular list for clients
* Use test fixture so storage directory is cleaned
* Use seq_cst for replication_state

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
Marko Budiselić
c68ed8d94e Add implementation of synchronous replication (#7)
This implements the initial version of synchronous replication.
Currently, only one replica is supported and that isn't configurable.

To run the main instance use the following command:
```
./memgraph \
    --main \
    --data-directory main-data \
    --storage-properties-on-edges \
    --storage-wal-enabled \
    --storage-snapshot-interval-sec 300
```

To run the replica instance use the following command:
```
./memgraph \
    --replica \
    --data-directory replica-data \
    --storage-properties-on-edges \
    --bolt-port 7688
```

You can then write/read data to Bolt port 7687 (the main instance) and also you
can read the data from the replica instance using Bolt port 7688.

NOTE: The main instance *must* be started without any data and the replica
*must* be started before any data is added to the main instance.

* Add basic synchronous replication test
* Using RWLock for replication stuff

Co-authored-by: Matej Ferencevic <matej.ferencevic@memgraph.io>
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
Marko Budiselić
d5b02eafb1 Add syntactic support for replication queries - prototype (#8)
Extend parser to handle replication queries - first implementation.

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2021-01-21 15:49:32 +01:00
antonio2368
4e7c569071 Migrate code to C++20 (#44) 2020-11-17 11:03:08 +01:00
Marko Budiselić
958bc870b3
Migrate to toolchain-v2 (#33)
* GCC_VERSION=10.2.0
* BINUTILS_VERSION=2.35.1
* GDB_VERSION=10.1 (except centos-7 8.3)
* CMAKE_VERSION=3.18.4
* CPPCHECK_VERSION=2.2
* LLVM_VERSION=11.0.0
* SWIG_VERSION=4.0.2
2020-11-12 20:18:11 +01:00
antonio2368
0bcc1d67bc
Add support for Bolt v4(.1) (#10)
* Added handshake support
* Add support for v4 hello and goodbye
* Add support for pulling n results
* Add support for transactions
* Add pull n for the dump
* Add support for NOOP
* Add support for multiple queries
* Update bolt session to support qid
* Update drivers test with multiple versions and go
* Extract failure handling into a function
* Use unique ptr instead of optional for query execution
* Destroy stream before query execution

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2020-10-16 12:49:33 +02:00
Josip Seljan
f7f861ca71
Add plan for label+property indexed node where property is not null (#2)
Replace ScanAll + Filter with a ScanAll variant performing label and property
lookup.

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2020-10-01 13:22:21 +02:00
Matej Ferencevic
0dcfdb9b89 Add GitHub workflows 2020-09-21 12:22:40 +02:00
Matej Ferencevic
04ceb8d4b1 Implement valueType openCypher function
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2818
2020-09-08 16:49:53 +02:00
Matej Ferencevic
d2ff465f8e Fix Cypher ID function Null handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2815
2020-09-04 13:18:08 +02:00
Matej Ferencevic
dd9180da32 Add request streaming support to the RPC client
Summary:
This change only adds streaming support to the client request. The client
response, server request and server response are still handled only when all of
the data is received.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2807
2020-08-14 16:19:48 +02:00
Matej Ferencevic
aaf0c1ca08 Improve SLK
Summary:
SLK now correctly handles different CPU architectures (BIG/little endian).
Also, more string encoding functions have been added.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2806
2020-08-14 15:27:20 +02:00
Matej Ferencevic
1513a455de Merge durability and storage
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2803
2020-08-14 15:08:43 +02:00
Matej Ferencevic
857de23687 Split storage durability implementation into multiple files
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2760
2020-08-14 13:36:52 +02:00
Matej Ferencevic
28590aea53 Fix variable expand between same symbol
Summary:
This diff fixes the variable expand operator to work correctly then the start
and destination nodes use the same symbol or when the destination symbol is an
existing symbol.

Previously, the variable expand operator produced paths that were both
completely wrong (they shouldn't have been produced) and nonexistent (they
didn't even exist in the storage). Invalid data was produced because of a
wrong equality check that was introduced in D1703.

This issue was reported externally and the supplied test case was:
```
CREATE (p1:Person {id: 1})-[:KNOWS]->(:Person {id: 2})-[:KNOWS]->(:Person {id: 3})-[:KNOWS]->(:Person {id: 4})-[:KNOWS]->(p1);
MATCH path = (pers:Person {id: 3})-[:KNOWS*2]->(pers) RETURN path;
```

Also, tests have been added so the behavior remains correct.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2793
2020-07-01 00:17:25 +02:00
jseljan
9831f0396a Fix 'all' and 'single' functions; update their unit and TCK tests
Summary:
Semantics of 'all' and 'single' were updated to be
consistent with that of 'any' and 'none'

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2789
2020-06-24 13:43:25 +02:00
jseljan
8bfebfca9d Add 'none' function implementation, its unit and TCK tests
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2788
2020-06-23 13:06:06 +02:00
jseljan
21cee1eaec Fix any function implementation
Summary: Change any function's handling of Null elements in a list

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2787
2020-06-19 15:40:57 +02:00
jseljan
098333f735 Add prototype implementation of the any function
Summary: Add any function prototype - no tests

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2785
2020-06-10 17:01:24 +02:00
Matej Ferencevic
3dd393f8eb Fix multi-command transaction handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2784
2020-06-10 12:25:41 +02:00
Matej Ferencevic
6628d20e5a Make DUMP DATABASE work correctly in explicit transactions
Summary:
`DUMP DATABASE` used a separate transaction to read database data. That
wouldn't be an issue if the query was correctly disallowed in multicommand
transactions. Because it was allowed the output wasn't transactionally correct.
Instead of disabling `DUMP DATABASE` in multicommand transactions this change
fixes it so that it works properly in multicommand transactions.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2781
2020-06-08 15:56:16 +02:00
Matej Ferencevic
b923d2bc36 Integrate property store
Reviewers: buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2764
2020-06-01 14:11:11 +02:00
Tonko Sabolcec
3932376301 Return properties as a list in SHOW CONSTRAINT INFO for unique constraints
Summary:
Before this change properties were joined by ", " and returned as a single string,
which was ambiguous for properties that contain ", ". This diff solves this
problem by returning properties as a list type.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2778
2020-05-29 12:59:33 +02:00
Tonko Sabolcec
59bc9d8989 Fix label/edge/property name escaping in DUMP DATABASE
Summary:
This diff fixes the issue for label name (and edge type/property)
with spaces and special characters to avoid possible OpenCypher injections.
Consider an example where label name is 'hello :world'. `DUMP DATABASE`
used to return query which creates a node (u:hello :world) - i.e. node
that contains two labels 'hello' and 'world'. This fix escapes names to
create the following node with exactly one label as expected:
```
(u:`hello :world`)
```

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2774
2020-05-21 14:46:51 +02:00
Matej Ferencevic
f047f55020 Implement IsPropertyEqual for property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2768
2020-05-20 15:41:13 +02:00
Lovro Lugovic
0c42bedf2f Add support for rescanning query modules
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2765
2020-05-13 21:33:28 +02:00
Marko Budiselic
f35225f26b Optimize IN list execution
Summary:
Use Unwind + ScanAllByLabelPropertyValue logical operator to
accelerate the execution of queries like the following one:
`MATCH (n:Label) WHERE n.property IN [] ...`

Reviewers: llugovic, mferencevic

Reviewed By: llugovic, mferencevic

Subscribers: llugovic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2758
2020-04-21 11:38:38 +02:00
Matej Ferencevic
d04993df67 Improve storage unit test
Summary: The tests are extracted from D2570.

Reviewers: llugovic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2745
2020-04-01 15:26:44 +02:00
Matej Ferencevic
7e35798401 Remove leftover raft unit test
Reviewers: llugovic, buda

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2744
2020-04-01 13:48:02 +02:00
Teon Banek
b7738c64b3 Make py::Object conversion to PyObject * explicit
Summary:
This fixes an issue in Py(Vertex|Edge)GetProperty and prevents any
further issues of that type at the cost of additional typing effort.

Reviewers: ipaljak, llugovic

Reviewed By: ipaljak, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2735
2020-03-24 12:23:13 +01:00
Teon Banek
2b8f068ca9 Correctly delimit arguments when printing signature
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2732
2020-03-19 14:19:53 +01:00
Teon Banek
008efaf243 Implement pretty printing CallProcedure operator
Reviewers: mferencevic, llugovic, ipaljak

Reviewed By: mferencevic, llugovic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2726
2020-03-18 10:32:16 +01:00
Teon Banek
d63eb191f9 Fix a planning issue when CALL preceded filtering
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2722
2020-03-16 15:06:39 +01:00
Lovro Lugovic
dd5d29fa1d Implement conversion of mgp_vertex to/from mgp.Vertex
Summary:
- Fix AssertPickleAndCopyAreNotSupported
- Test Vertex and Edge conversions

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2717
2020-03-11 16:13:24 +01:00
Tonko Sabolcec
6f83fff171 Implement durability functionality for unique constraints
Summary:
This diff contains a necessary functionality to save and restore unique
constraint operations. The previous snapshot/WAL version is backward
compatible. Integration tests for migration from older snapshot and WAL
versions are also included.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2680
2020-03-06 14:14:54 +01:00
Teon Banek
1d30a2e5cb Convert mgp_value to 'mgp' Python types, not '_mgp' ones
Summary:
When invoking a Python registered procedures we want to convert
`mgp_value` types to user facing 'mgp' types.

Depends on D2706

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2707
2020-03-06 11:23:08 +01:00
Teon Banek
456267249e Convert mgp_list to PyTuple instead of PyList
Summary: It's much safer to give users immutable types.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2705
2020-03-06 11:22:18 +01:00
Teon Banek
74d9dd0b0a Implement mgp.Path
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2700
2020-03-04 12:51:06 +01:00
Tonko Sabolcec
14c4c6d060 Update query dump with unique constraints
Summary:
Now that unique constraint feature is added to Memgraph database,
we should update `DUMP DATABASE` with list of existing unique constraints.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2698
2020-03-02 13:50:35 +01:00
Tonko Sabolcec
eed83a210e Implement unique constraint functionality in query module
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2681
2020-03-02 13:00:04 +01:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
2020-03-02 12:45:18 +01:00
Matej Ferencevic
bfbace8168 Remove old HA implementation
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2686
2020-02-25 17:19:35 +01:00
Teon Banek
e7f363ecbe Add Python class for mgp_edge
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2679
2020-02-25 10:26:08 +01:00
Matej Ferencevic
47ce444c02 Fix auth Cypher visitor
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2675
2020-02-17 17:41:08 +01:00
Matej Ferencevic
e8903ef477 Expose auth user/role name regex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2673
2020-02-17 17:40:53 +01:00
Lovro Lugovic
5953f07be3 Add conversion of py::Object to mgp_value
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2667
2020-02-14 16:08:20 +01:00
Tonko Sabolcec
95c9755e13 Support property set for unique constraints
Summary:
Before this change, unique constraints supported only pairs of label
and a single property. With this change, unique constraints can be
created for label and set of properties.
Better tests for unique constraints in general are also included in
this diff.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2653
2020-02-13 12:41:35 +01:00
Teon Banek
0a7de969f3 Add converting mgp_value to py::Object
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2665
2020-02-12 09:30:14 +01:00
Tonko Sabolcec
e2e7823ec4 Add basic support for unique constraints in storage
Summary:
This diff contains a basic implementation of unique constraints consistent with
the MVCC storage.
Stale records in the unique constraints are collected by the garbage collector.
Tests for checking correctness of unique constraints and violations are included.
Note: currently we only support a pair of label and a single property. Support for
multiple properties will be added later.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: buda, ipaljak, pullbot

Differential Revision: https://phabricator.memgraph.io/D2608
2020-02-10 12:45:28 +01:00