Commit Graph

3024 Commits

Author SHA1 Message Date
Matej Ferencevic
4a53a31fcf Add FootballTransfers tutorial data
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2779
2020-07-01 09:24:35 +02:00
Matej Ferencevic
28590aea53 Fix variable expand between same symbol
Summary:
This diff fixes the variable expand operator to work correctly then the start
and destination nodes use the same symbol or when the destination symbol is an
existing symbol.

Previously, the variable expand operator produced paths that were both
completely wrong (they shouldn't have been produced) and nonexistent (they
didn't even exist in the storage). Invalid data was produced because of a
wrong equality check that was introduced in D1703.

This issue was reported externally and the supplied test case was:
```
CREATE (p1:Person {id: 1})-[:KNOWS]->(:Person {id: 2})-[:KNOWS]->(:Person {id: 3})-[:KNOWS]->(:Person {id: 4})-[:KNOWS]->(p1);
MATCH path = (pers:Person {id: 3})-[:KNOWS*2]->(pers) RETURN path;
```

Also, tests have been added so the behavior remains correct.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2793
2020-07-01 00:17:25 +02:00
Matej Ferencevic
0a2f2bfc90 Improve expansion performance
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2794
2020-06-29 16:37:11 +02:00
jseljan
a7b672ebbf Add TCK tests relating predicate functions
Summary:
Implement TCK regression-like tests relating 'any', 'all', and 'none'
predicate functions

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2791
2020-06-29 12:56:59 +02:00
jseljan
9831f0396a Fix 'all' and 'single' functions; update their unit and TCK tests
Summary:
Semantics of 'all' and 'single' were updated to be
consistent with that of 'any' and 'none'

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2789
2020-06-24 13:43:25 +02:00
jseljan
8bfebfca9d Add 'none' function implementation, its unit and TCK tests
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2788
2020-06-23 13:06:06 +02:00
jseljan
21cee1eaec Fix any function implementation
Summary: Change any function's handling of Null elements in a list

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2787
2020-06-19 15:40:57 +02:00
jseljan
098333f735 Add prototype implementation of the any function
Summary: Add any function prototype - no tests

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2785
2020-06-10 17:01:24 +02:00
Matej Ferencevic
3dd393f8eb Fix multi-command transaction handling
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2784
2020-06-10 12:25:41 +02:00
Matej Ferencevic
6628d20e5a Make DUMP DATABASE work correctly in explicit transactions
Summary:
`DUMP DATABASE` used a separate transaction to read database data. That
wouldn't be an issue if the query was correctly disallowed in multicommand
transactions. Because it was allowed the output wasn't transactionally correct.
Instead of disabling `DUMP DATABASE` in multicommand transactions this change
fixes it so that it works properly in multicommand transactions.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2781
2020-06-08 15:56:16 +02:00
Matej Ferencevic
b923d2bc36 Integrate property store
Reviewers: buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2764
2020-06-01 14:11:11 +02:00
Tonko Sabolcec
3932376301 Return properties as a list in SHOW CONSTRAINT INFO for unique constraints
Summary:
Before this change properties were joined by ", " and returned as a single string,
which was ambiguous for properties that contain ", ". This diff solves this
problem by returning properties as a list type.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2778
2020-05-29 12:59:33 +02:00
Marko Budiselic
d36da12b48 Update enterprise license
Summary:
In case something has to be updated in `release/LICENSE_ENTERPRISE.md`
please put your comments in the document marked with `UPCOMING`
inside this https://drive.google.com/drive/folders/1GbnryzrjkmNDVigac4d1x__JWbiqlxMD
folder because the content has to be valid from the legal standpoint
(it requires some legal review).

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2770
2020-05-27 19:13:30 +02:00
Matej Ferencevic
a149f5bd81 Improve environment setup script
Summary:
The following improvements are in this diff:
 - Verify that run dependencies are correct
 - Setup gdbinit and install pahole
 - Add curl to CMake to enable HTTPS
 - Use HTTPS for LLVM download
 - Fix Debian 9 and Ubuntu 18.04 dependencies
 - Fix CentOS 7 sha256sum verification
 - Add support for Debian 10

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2776
2020-05-25 11:05:01 +02:00
Matej Ferencevic
1a8ae7f9c2 Suppress empty query modules directory errors
Reviewers: llugovic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2771
2020-05-25 10:09:40 +02:00
Matej Ferencevic
b510ac1049 Fix WeightedShortestPath integer overflows
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2775
2020-05-25 10:09:19 +02:00
Tonko Sabolcec
59bc9d8989 Fix label/edge/property name escaping in DUMP DATABASE
Summary:
This diff fixes the issue for label name (and edge type/property)
with spaces and special characters to avoid possible OpenCypher injections.
Consider an example where label name is 'hello :world'. `DUMP DATABASE`
used to return query which creates a node (u:hello :world) - i.e. node
that contains two labels 'hello' and 'world'. This fix escapes names to
create the following node with exactly one label as expected:
```
(u:`hello :world`)
```

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2774
2020-05-21 14:46:51 +02:00
Matej Ferencevic
b6dfb44b4c Fix various integer overflows in query
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2773
2020-05-21 14:30:01 +02:00
Matej Ferencevic
81ad5110ff Fix LIMIT and SKIP overflow bugs
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2772
2020-05-21 10:39:01 +02:00
Matej Ferencevic
f92dee6e7b Fix COUNT overflow bug
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2769
2020-05-20 16:57:17 +02:00
Matej Ferencevic
f047f55020 Implement IsPropertyEqual for property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2768
2020-05-20 15:41:13 +02:00
Lovro Lugovic
0c42bedf2f Add support for rescanning query modules
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2765
2020-05-13 21:33:28 +02:00
Matej Ferencevic
7d9f741ceb Add missing optional check to property store
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2767
2020-05-06 10:07:58 +02:00
Matej Ferencevic
4d2eda398f Improve property store performance
Summary:
This diff improves the performance of `PropertyStore` with two main
techniques:

First:
`PropertyValue` has a very expensive constructor and destructor. The
`PropertyValue` was previously passed as a return value from many functions
wrapped in a `std::optional`. That caused the `PropertyValue`
constructor/destructor to be called for each intermediary value that was passed
between functions. This diff changes the functions to return a `bool` value
that imitates the `std::optional` "emptyness" flag and the `PropertyValue` is
modified using a pointer to it so that its constructor/destructor is called
only once.

Second:
The `PropertyStore` buffer was previously iterated through at least twice.
First to determine the exact position of the encoded property and then to
actually decode the property. This diff combines the two passes into a single
pass so that the property is immediately loaded if it is found.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2766
2020-05-05 10:55:00 +02:00
Lovro Lugovic
317f7f118c Make Python modules reloadable
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2763
2020-04-27 17:29:58 +02:00
Marko Budiselic
f35225f26b Optimize IN list execution
Summary:
Use Unwind + ScanAllByLabelPropertyValue logical operator to
accelerate the execution of queries like the following one:
`MATCH (n:Label) WHERE n.property IN [] ...`

Reviewers: llugovic, mferencevic

Reviewed By: llugovic, mferencevic

Subscribers: llugovic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2758
2020-04-21 11:38:38 +02:00
Marko Budiselic
e04b96a0ac Reduce the frame stack size
Reviewers: llugovic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2762
2020-04-19 20:27:20 +02:00
Matej Ferencevic
21be41995c Fix CMake RPM typo and specify all package config files
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2757
2020-04-06 19:12:30 +02:00
Matej Ferencevic
6c69351ea1 Fix RPM packaging
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2756
2020-04-06 16:01:48 +02:00
Matej Ferencevic
e9ea8693fa Fix CentOS compilation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2755
2020-04-06 16:01:35 +02:00
Matej Ferencevic
9d3f75a34d Don't link with libreadline in community release
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2754
2020-04-06 10:17:02 +02:00
Marko Budiselic
590fc5c9a8 Add NetworkX to Dockerfiles
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2753
2020-04-04 15:41:59 +02:00
Matej Ferencevic
1d5952bb9c Fix RPM package installation
Summary: `chattr` must be after `chown`.

Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D2752
2020-04-03 18:50:52 +02:00
Matej Ferencevic
24eb021840 Fix wrong function invocation in get_version script
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2751
2020-04-03 17:38:12 +02:00
Matej Ferencevic
ac3560ec8c Add changelog for version 1.0
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D2750
2020-04-03 14:09:02 +02:00
Matej Ferencevic
974ce1e9e9 Fix Debian package installation
Summary: `chattr` must be after `chown`.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2748
2020-04-02 19:46:28 +02:00
Matej Ferencevic
66e644551f Clean-up packaged examples
Summary:
The snapshot files now have identical names each time they are generated. Also,
all unnecessary directories (like `auth`) are removed from the examples
automatically.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2747
2020-04-02 19:45:32 +02:00
Matej Ferencevic
df72723664 Install Python query modules
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2749
2020-04-02 19:45:12 +02:00
Dino Santl
96bb7a87e3 Add graph analyzer python module.
Summary:
In new commit time limit is introduced.

Also, subgraph query is a little bit different because of memory leak. If we use subgraph in a same way as in wcc.py module then we cannot have Exceptions - reason is memory leak in parameters (node and edge list). Edge and node lists are now list of integers (database id).

P.S. if you know how avoid this please let me know.

Reviewers: mferencevic, llugovic, tsabolcec, tlastre, buda

Reviewed By: mferencevic, llugovic, tlastre, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2736
2020-04-02 15:26:20 +02:00
Matej Ferencevic
39deba3b9e Fix wrong implicit cast in query dumper
Summary: All integers are signed in openCypher.

Reviewers: buda, llugovic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2746
2020-04-01 18:35:23 +02:00
Matej Ferencevic
d04993df67 Improve storage unit test
Summary: The tests are extracted from D2570.

Reviewers: llugovic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2745
2020-04-01 15:26:44 +02:00
Matej Ferencevic
7e35798401 Remove leftover raft unit test
Reviewers: llugovic, buda

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2744
2020-04-01 13:48:02 +02:00
Matej Ferencevic
1d2bb2cda6 Remove leftover traces of HA from query
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2743
2020-04-01 13:47:47 +02:00
Lovro Lugovic
8fb3a53b78 Expose query timeout checking to modules
Summary:
- Add the `mgp_must_abort(const mgp_graph *graph)` C API.
- Add the `ProcCtx.must_abort()` Python API.

The usage is very simple -- the function returns a boolean indicating whether
the procedure should abort.

Reviewers: mferencevic, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2742
2020-04-01 13:30:03 +02:00
Matej Ferencevic
a1b5bdd88f Implement concurrent tests for storage indices
Summary: The same test is written both for label indices and label+property indices.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2741
2020-04-01 10:43:05 +02:00
Matej Ferencevic
3cd89e1fe4 Remove internal mgp.py file from Python traceback
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2740
2020-03-31 15:49:06 +02:00
Matej Ferencevic
90c83de7c2 Fix Python memory leaks when exceptions are thrown
Summary:
The Python `ExceptionInfo` data has the potential to extend lifetime of Python
objects that were on the Python frame while the execution was thrown. This
lifetime extension is very dangerous when executing Python procedures because
we use a custom internal memory allocator that is destroyed immediately when
the procedure is done with its execution.

This diff only keeps the formatted Python traceback (as a string) to avoid any
lifetime extension for the Python objects so that the memory allocator can
safely be destroyed after the procedure has finished its execution.

Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2739
2020-03-31 14:24:05 +02:00
Matej Ferencevic
7cd96dc2f5 Fix Python objects init
Summary:
`PyObject_Init` shouldn't be used in conjunction with `PyObject_New`.

The official Python documentation doesn't say this explicitly and many examples
on the web suggest that they must be used together. When they are used together
everything works when Memgraph is built against the release Python binary, but
doesn't work when built against the debug Python binary.

An implicit example that `PyObject_Init` shouldn't be used with `PyObject_New`
can be found in the Python source, in the file `Include/objimpl.h`:
```
   This example code implements an object constructor with a custom
   allocator, where PyObject_New is inlined, and shows the important
   distinction between two steps (at least):
       1) the actual allocation of the object storage;
       2) the initialization of the Python specific fields
      in this storage with PyObject_{Init, InitVar}.

   PyObject *
   YourObject_New(...)
   {
       PyObject *op;

       op = (PyObject *) Your_Allocator(_PyObject_SIZE(YourTypeStruct));
       if (op == NULL)
       return PyErr_NoMemory();

       PyObject_Init(op, &YourTypeStruct);

       op->ob_field = value;
       ...
       return op;
   }

   Note that in C++, the use of the new operator usually implies that
   the 1st step is performed automatically for you, so in a C++ class
   constructor you would start directly with PyObject_Init/InitVar
```
It explains that `PyObject_New` is actually equal to `malloc` +
`PyObject_Init`.

Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2737
2020-03-30 12:40:42 +02:00
Matej Ferencevic
41551bcd36 Fix Python struct formatting
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2738
2020-03-28 20:13:02 +01:00
Lovro Lugovic
e5b3414335 Implement the PageRank module
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2730
2020-03-25 13:58:50 +01:00