Commit Graph

1820 Commits

Author SHA1 Message Date
antonio2368
0bcc1d67bc
Add support for Bolt v4(.1) (#10)
* Added handshake support
* Add support for v4 hello and goodbye
* Add support for pulling n results
* Add support for transactions
* Add pull n for the dump
* Add support for NOOP
* Add support for multiple queries
* Update bolt session to support qid
* Update drivers test with multiple versions and go
* Extract failure handling into a function
* Use unique ptr instead of optional for query execution
* Destroy stream before query execution

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2020-10-16 12:49:33 +02:00
Josip Seljan
f7f861ca71
Add plan for label+property indexed node where property is not null (#2)
Replace ScanAll + Filter with a ScanAll variant performing label and property
lookup.

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2020-10-01 13:22:21 +02:00
Matej Ferencevic
04ceb8d4b1 Implement valueType openCypher function
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2818
2020-09-08 16:49:53 +02:00
Matej Ferencevic
fb7025a716 Clean-up awesome memgraph functions error handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2817
2020-09-08 13:16:07 +02:00
Matej Ferencevic
d2ff465f8e Fix Cypher ID function Null handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2815
2020-09-04 13:18:08 +02:00
Matej Ferencevic
c12a87e9ca Allow inheritance of storage durability decoder
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2810
2020-08-17 11:12:59 +02:00
Matej Ferencevic
63de0b5db4 Allow inheritance of storage durability encoder
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2809
2020-08-17 10:23:39 +02:00
Matej Ferencevic
dd9180da32 Add request streaming support to the RPC client
Summary:
This change only adds streaming support to the client request. The client
response, server request and server response are still handled only when all of
the data is received.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2807
2020-08-14 16:19:48 +02:00
Matej Ferencevic
aaf0c1ca08 Improve SLK
Summary:
SLK now correctly handles different CPU architectures (BIG/little endian).
Also, more string encoding functions have been added.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2806
2020-08-14 15:27:20 +02:00
Matej Ferencevic
1513a455de Merge durability and storage
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2803
2020-08-14 15:08:43 +02:00
Matej Ferencevic
4f9e9aeafe Move snapshot/WAL encoding/decoding into their respective files
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2761
2020-08-14 14:17:45 +02:00
Matej Ferencevic
857de23687 Split storage durability implementation into multiple files
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2760
2020-08-14 13:36:52 +02:00
Matej Ferencevic
df27b8477d Remove unused UnwrapDegreeResult function
Reviewers: jseljan

Reviewed By: jseljan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2797
2020-07-01 13:05:55 +02:00
Matej Ferencevic
28590aea53 Fix variable expand between same symbol
Summary:
This diff fixes the variable expand operator to work correctly then the start
and destination nodes use the same symbol or when the destination symbol is an
existing symbol.

Previously, the variable expand operator produced paths that were both
completely wrong (they shouldn't have been produced) and nonexistent (they
didn't even exist in the storage). Invalid data was produced because of a
wrong equality check that was introduced in D1703.

This issue was reported externally and the supplied test case was:
```
CREATE (p1:Person {id: 1})-[:KNOWS]->(:Person {id: 2})-[:KNOWS]->(:Person {id: 3})-[:KNOWS]->(:Person {id: 4})-[:KNOWS]->(p1);
MATCH path = (pers:Person {id: 3})-[:KNOWS*2]->(pers) RETURN path;
```

Also, tests have been added so the behavior remains correct.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2793
2020-07-01 00:17:25 +02:00
Matej Ferencevic
0a2f2bfc90 Improve expansion performance
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2794
2020-06-29 16:37:11 +02:00
jseljan
9831f0396a Fix 'all' and 'single' functions; update their unit and TCK tests
Summary:
Semantics of 'all' and 'single' were updated to be
consistent with that of 'any' and 'none'

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2789
2020-06-24 13:43:25 +02:00
jseljan
8bfebfca9d Add 'none' function implementation, its unit and TCK tests
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2788
2020-06-23 13:06:06 +02:00
jseljan
21cee1eaec Fix any function implementation
Summary: Change any function's handling of Null elements in a list

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2787
2020-06-19 15:40:57 +02:00
jseljan
098333f735 Add prototype implementation of the any function
Summary: Add any function prototype - no tests

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2785
2020-06-10 17:01:24 +02:00
Matej Ferencevic
3dd393f8eb Fix multi-command transaction handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2784
2020-06-10 12:25:41 +02:00
Matej Ferencevic
6628d20e5a Make DUMP DATABASE work correctly in explicit transactions
Summary:
`DUMP DATABASE` used a separate transaction to read database data. That
wouldn't be an issue if the query was correctly disallowed in multicommand
transactions. Because it was allowed the output wasn't transactionally correct.
Instead of disabling `DUMP DATABASE` in multicommand transactions this change
fixes it so that it works properly in multicommand transactions.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2781
2020-06-08 15:56:16 +02:00
Matej Ferencevic
b923d2bc36 Integrate property store
Reviewers: buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2764
2020-06-01 14:11:11 +02:00
Tonko Sabolcec
3932376301 Return properties as a list in SHOW CONSTRAINT INFO for unique constraints
Summary:
Before this change properties were joined by ", " and returned as a single string,
which was ambiguous for properties that contain ", ". This diff solves this
problem by returning properties as a list type.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2778
2020-05-29 12:59:33 +02:00
Matej Ferencevic
1a8ae7f9c2 Suppress empty query modules directory errors
Reviewers: llugovic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2771
2020-05-25 10:09:40 +02:00
Matej Ferencevic
b510ac1049 Fix WeightedShortestPath integer overflows
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2775
2020-05-25 10:09:19 +02:00
Tonko Sabolcec
59bc9d8989 Fix label/edge/property name escaping in DUMP DATABASE
Summary:
This diff fixes the issue for label name (and edge type/property)
with spaces and special characters to avoid possible OpenCypher injections.
Consider an example where label name is 'hello :world'. `DUMP DATABASE`
used to return query which creates a node (u:hello :world) - i.e. node
that contains two labels 'hello' and 'world'. This fix escapes names to
create the following node with exactly one label as expected:
```
(u:`hello :world`)
```

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2774
2020-05-21 14:46:51 +02:00
Matej Ferencevic
b6dfb44b4c Fix various integer overflows in query
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2773
2020-05-21 14:30:01 +02:00
Matej Ferencevic
81ad5110ff Fix LIMIT and SKIP overflow bugs
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2772
2020-05-21 10:39:01 +02:00
Matej Ferencevic
f92dee6e7b Fix COUNT overflow bug
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2769
2020-05-20 16:57:17 +02:00
Matej Ferencevic
f047f55020 Implement IsPropertyEqual for property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2768
2020-05-20 15:41:13 +02:00
Lovro Lugovic
0c42bedf2f Add support for rescanning query modules
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2765
2020-05-13 21:33:28 +02:00
Matej Ferencevic
7d9f741ceb Add missing optional check to property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2767
2020-05-06 10:07:58 +02:00
Matej Ferencevic
4d2eda398f Improve property store performance
Summary:
This diff improves the performance of `PropertyStore` with two main
techniques:

First:
`PropertyValue` has a very expensive constructor and destructor. The
`PropertyValue` was previously passed as a return value from many functions
wrapped in a `std::optional`. That caused the `PropertyValue`
constructor/destructor to be called for each intermediary value that was passed
between functions. This diff changes the functions to return a `bool` value
that imitates the `std::optional` "emptyness" flag and the `PropertyValue` is
modified using a pointer to it so that its constructor/destructor is called
only once.

Second:
The `PropertyStore` buffer was previously iterated through at least twice.
First to determine the exact position of the encoded property and then to
actually decode the property. This diff combines the two passes into a single
pass so that the property is immediately loaded if it is found.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2766
2020-05-05 10:55:00 +02:00
Lovro Lugovic
317f7f118c Make Python modules reloadable
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2763
2020-04-27 17:29:58 +02:00
Marko Budiselic
f35225f26b Optimize IN list execution
Summary:
Use Unwind + ScanAllByLabelPropertyValue logical operator to
accelerate the execution of queries like the following one:
`MATCH (n:Label) WHERE n.property IN [] ...`

Reviewers: llugovic, mferencevic

Reviewed By: llugovic, mferencevic

Subscribers: llugovic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2758
2020-04-21 11:38:38 +02:00
Marko Budiselic
e04b96a0ac Reduce the frame stack size
Reviewers: llugovic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2762
2020-04-19 20:27:20 +02:00
Matej Ferencevic
e9ea8693fa Fix CentOS compilation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2755
2020-04-06 16:01:35 +02:00
Matej Ferencevic
39deba3b9e Fix wrong implicit cast in query dumper
Summary: All integers are signed in openCypher.

Reviewers: buda, llugovic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2746
2020-04-01 18:35:23 +02:00
Matej Ferencevic
1d2bb2cda6 Remove leftover traces of HA from query
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2743
2020-04-01 13:47:47 +02:00
Lovro Lugovic
8fb3a53b78 Expose query timeout checking to modules
Summary:
- Add the `mgp_must_abort(const mgp_graph *graph)` C API.
- Add the `ProcCtx.must_abort()` Python API.

The usage is very simple -- the function returns a boolean indicating whether
the procedure should abort.

Reviewers: mferencevic, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2742
2020-04-01 13:30:03 +02:00
Matej Ferencevic
3cd89e1fe4 Remove internal mgp.py file from Python traceback
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2740
2020-03-31 15:49:06 +02:00
Matej Ferencevic
90c83de7c2 Fix Python memory leaks when exceptions are thrown
Summary:
The Python `ExceptionInfo` data has the potential to extend lifetime of Python
objects that were on the Python frame while the execution was thrown. This
lifetime extension is very dangerous when executing Python procedures because
we use a custom internal memory allocator that is destroyed immediately when
the procedure is done with its execution.

This diff only keeps the formatted Python traceback (as a string) to avoid any
lifetime extension for the Python objects so that the memory allocator can
safely be destroyed after the procedure has finished its execution.

Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2739
2020-03-31 14:24:05 +02:00
Matej Ferencevic
7cd96dc2f5 Fix Python objects init
Summary:
`PyObject_Init` shouldn't be used in conjunction with `PyObject_New`.

The official Python documentation doesn't say this explicitly and many examples
on the web suggest that they must be used together. When they are used together
everything works when Memgraph is built against the release Python binary, but
doesn't work when built against the debug Python binary.

An implicit example that `PyObject_Init` shouldn't be used with `PyObject_New`
can be found in the Python source, in the file `Include/objimpl.h`:
```
   This example code implements an object constructor with a custom
   allocator, where PyObject_New is inlined, and shows the important
   distinction between two steps (at least):
       1) the actual allocation of the object storage;
       2) the initialization of the Python specific fields
      in this storage with PyObject_{Init, InitVar}.

   PyObject *
   YourObject_New(...)
   {
       PyObject *op;

       op = (PyObject *) Your_Allocator(_PyObject_SIZE(YourTypeStruct));
       if (op == NULL)
       return PyErr_NoMemory();

       PyObject_Init(op, &YourTypeStruct);

       op->ob_field = value;
       ...
       return op;
   }

   Note that in C++, the use of the new operator usually implies that
   the 1st step is performed automatically for you, so in a C++ class
   constructor you would start directly with PyObject_Init/InitVar
```
It explains that `PyObject_New` is actually equal to `malloc` +
`PyObject_Init`.

Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2737
2020-03-30 12:40:42 +02:00
Matej Ferencevic
41551bcd36 Fix Python struct formatting
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2738
2020-03-28 20:13:02 +01:00
Teon Banek
b7738c64b3 Make py::Object conversion to PyObject * explicit
Summary:
This fixes an issue in Py(Vertex|Edge)GetProperty and prevents any
further issues of that type at the cost of additional typing effort.

Reviewers: ipaljak, llugovic

Reviewed By: ipaljak, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2735
2020-03-24 12:23:13 +01:00
Matej Ferencevic
a8b81fdcde Unify C/Python query module logging
Summary:
Make logging for operations with C/Python query modules (loading, closing,
reloading) consistent.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2734
2020-03-20 16:02:29 +01:00
Matej Ferencevic
9f006c1b57 Fix storage GC indices/constraints subtle race condition
Summary:
In order for this race condition to cause damage, an index/constraint must be
created/dropped at the exact moment that the GC is cleaning
indices/constraints.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2733
2020-03-20 12:41:20 +01:00
Teon Banek
2b8f068ca9 Correctly delimit arguments when printing signature
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2732
2020-03-19 14:19:53 +01:00
Teon Banek
d4c2798551 Save and restore the exception in WithMgpModule
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2731
2020-03-19 14:19:50 +01:00
Ivan Paljak
44917fdfa6 Use storage::EdgeAccessor::Gid() as __hash__ for mgp.Edge
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2729
2020-03-18 15:13:13 +01:00