Commit Graph

1765 Commits

Author SHA1 Message Date
Lovro Lugovic
dd5d29fa1d Implement conversion of mgp_vertex to/from mgp.Vertex
Summary:
- Fix AssertPickleAndCopyAreNotSupported
- Test Vertex and Edge conversions

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2717
2020-03-11 16:13:24 +01:00
Teon Banek
963d779d48 Install and setup loading Python Query Modules
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D2715
2020-03-11 10:24:53 +01:00
Matej Ferencevic
5695774251 Improve CSV importer
Summary:
The importer now supports all of the flags that the modern Neo4j CSV importer
supports.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2709
2020-03-10 15:51:29 +01:00
Ivan Paljak
181f937c15 Add Python class for mgp_properties_iterator
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2712
2020-03-10 14:34:32 +01:00
Tonko Sabolcec
7ea2d1b638 Add concurrent test for unique constraints
Summary:
This diff contains simple tests for unique constraints which tries to
change property values or labels in multiple threads at the same time.
During testing, a bug has been encountered in unique constraints, i.e.
one guard lock on vertices was missing.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2711
2020-03-10 12:52:58 +01:00
Teon Banek
c4a1a6c0b4 Extract py::AppendToSysPath function
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2714
2020-03-10 11:06:21 +01:00
Teon Banek
d0411510fa Add GetExecutablePath to utils/file
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2713
2020-03-10 11:06:18 +01:00
Teon Banek
fbdcad1106 Implement calling Python procedures
Summary:
You should now be able to invoke query procedures written in Python. To
test the example you can run memgraph with PYTHONPATH set to `include`.
For example, assuming you are in the root of the repo, run this command.

    PYTHONPATH=$PWD/include ./build/memgraph --query-modules-directory=./query_modules/

Alternatively, you can set a symlink inside the ./query_modules to point
to `include/mgp.py`, so there's no need to set PYTHONPATH. For example,
assuming you are in the root of the repo, run the following.

    cd ./query_modules
    ln -s ../include/mgp.py
    cd ..
    ./build/memgraph --query-modules-directory=./query_modules/

Depends on D207

Reviewers: mferencevic, ipaljak, dsantl

Reviewed By: ipaljak

Subscribers: buda, tlastre, pullbot

Differential Revision: https://phabricator.memgraph.io/D2708
2020-03-06 16:25:51 +01:00
Tonko Sabolcec
6f83fff171 Implement durability functionality for unique constraints
Summary:
This diff contains a necessary functionality to save and restore unique
constraint operations. The previous snapshot/WAL version is backward
compatible. Integration tests for migration from older snapshot and WAL
versions are also included.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2680
2020-03-06 14:14:54 +01:00
Matej Ferencevic
8363991575 Implement new CSV parser
Summary:
The new CSV parser in `mg_import_csv` behaves the same when importing a CSV
file as the standard Python CSV importer. Tests are added for all CSV field
edge-cases.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2702
2020-03-06 11:32:54 +01:00
Teon Banek
1d30a2e5cb Convert mgp_value to 'mgp' Python types, not '_mgp' ones
Summary:
When invoking a Python registered procedures we want to convert
`mgp_value` types to user facing 'mgp' types.

Depends on D2706

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2707
2020-03-06 11:23:08 +01:00
Teon Banek
31a4c55e76 Add HasAttr, GetAttr, and SetAttr to py::Object
Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2706
2020-03-06 11:22:39 +01:00
Teon Banek
456267249e Convert mgp_list to PyTuple instead of PyList
Summary: It's much safer to give users immutable types.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2705
2020-03-06 11:22:18 +01:00
Matej Ferencevic
000d6dba55 Fix utils::SkipList const iterator bug
Summary:
This issue was already fixed in D2119 for the iterator, but I missed to fix the
const iterator in that diff...

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2703
2020-03-05 11:17:26 +01:00
Teon Banek
6e9404c89a Correctly implement mgp_value move constructor
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2704
2020-03-05 10:00:52 +01:00
Teon Banek
74d9dd0b0a Implement mgp.Path
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2700
2020-03-04 12:51:06 +01:00
Matej Ferencevic
a456d6cdc0 Implement mg_import_csv
Summary:
This diff restores (and fixes) the old mg_import_csv implementation. The
importer now supports the new storage engine.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2690
2020-03-03 14:33:43 +01:00
Ivan Paljak
ec0caad44c Fix memory leaks in Py*Iter* methods
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2699
2020-03-02 14:40:59 +01:00
Tonko Sabolcec
14c4c6d060 Update query dump with unique constraints
Summary:
Now that unique constraint feature is added to Memgraph database,
we should update `DUMP DATABASE` with list of existing unique constraints.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2698
2020-03-02 13:50:35 +01:00
Teon Banek
b922ca75d0 Add mgp_path_equal to C API
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2697
2020-03-02 13:19:22 +01:00
Teon Banek
f5a94d6e29 Make MakePy<name> less error prone
Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2696
2020-03-02 13:19:19 +01:00
Tonko Sabolcec
eed83a210e Implement unique constraint functionality in query module
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2681
2020-03-02 13:00:04 +01:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
2020-03-02 12:45:18 +01:00
Teon Banek
64da28ca83 Fix some memory leaks in PyObjectToMgpValue
Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2695
2020-02-28 10:57:39 +01:00
Teon Banek
4049b9d10a Add mgp_path_copy to C API
Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2694
2020-02-28 10:57:36 +01:00
Ivan Paljak
bb4e747fd3 Finalize PyEdgeFromVertex and PyEdgeToVertex implementations
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2693
2020-02-28 09:36:47 +01:00
Ivan Paljak
b6b6403b06 Implement PyEdgesIterator and integrate with PyVertex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2692
2020-02-27 15:57:02 +01:00
Teon Banek
ad892f2db3 Support typing annotations in Python Query Modules
Reviewers: mferencevic, ipaljak, tlastre

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2689
2020-02-27 13:18:57 +01:00
Ivan Paljak
32e56684db Add Python class for mgp_vertex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2688
2020-02-26 14:01:43 +01:00
Matej Ferencevic
bfbace8168 Remove old HA implementation
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2686
2020-02-25 17:19:35 +01:00
Matej Ferencevic
4e5a91e7fb Merge memgraph_init with main file
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2684
2020-02-25 12:25:25 +01:00
Matej Ferencevic
2562070b06 Use std::string_view in storage NameTo... functions
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2683
2020-02-25 12:15:31 +01:00
Teon Banek
e7f363ecbe Add Python class for mgp_edge
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2679
2020-02-25 10:26:08 +01:00
Teon Banek
34db077cbd Register Python procedures
Summary:
With this diff you should now be able to register `example.py` and read
procedures found there. The procedures will be listed through `CALL
mg.procedures() YIELD *` query, but invoking them will raise
`NotYetImplemented`.

If you wish to test this, you will need to run the Memgraph executable
with PYTHONPATH set to the `include` directory where `mgp.py` is found.
Additionally, you need to pass `--query-modules-directory` flag to
Memgraph, such that it points to where it will find the `example.py`.

For example, when running from the root directory of Memgraph repo, the
shell invocation below should do the trick (assuming `./build/memgraph`
is where is the executable). Make sure that `./query_modules/` does not
have `example.so` built, as that may interfere with loading
`example.py`.

    PYTHONPATH=$PWD/include ./build/memgraph --query-modules-directory=./query_modules/

Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, ipaljak, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2678
2020-02-19 14:35:22 +01:00
Matej Ferencevic
47ce444c02 Fix auth Cypher visitor
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2675
2020-02-17 17:41:08 +01:00
Matej Ferencevic
e8903ef477 Expose auth user/role name regex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2673
2020-02-17 17:40:53 +01:00
Lovro Lugovic
5953f07be3 Add conversion of py::Object to mgp_value
Reviewers: teon.banek, ipaljak, mferencevic

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2667
2020-02-14 16:08:20 +01:00
Teon Banek
ad740e4ae2 Add _mgp.Graph and _mgp.VerticesIterator to embedded Python
Reviewers: llugovic, ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2669
2020-02-14 10:00:55 +01:00
Tonko Sabolcec
95c9755e13 Support property set for unique constraints
Summary:
Before this change, unique constraints supported only pairs of label
and a single property. With this change, unique constraints can be
created for label and set of properties.
Better tests for unique constraints in general are also included in
this diff.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2653
2020-02-13 12:41:35 +01:00
Teon Banek
6a7b983a58 Normalize the exception in py::FetchError
Reviewers: llugovic, ipaljak

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2668
2020-02-12 16:32:24 +01:00
Matej Ferencevic
ffbe5b449d Implement version names for each Memgraph version
Summary:
With this diff, each build of Memgraph has a version that uniquely identifies
it. The given version uniquely identifies both official release builds and
development builds. Enterprise/community builds are also differentiated in the
version.  Also, support for custom suffixes is added to support custom builds
for customers.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2662
2020-02-12 10:05:40 +01:00
Teon Banek
0a7de969f3 Add converting mgp_value to py::Object
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2665
2020-02-12 09:30:14 +01:00
Ivan Paljak
2aa960403a Refactor query module to support differently loaded modules
Summary:
`query::procedure::Module` is now an interface class through which
users interact with loaded modules. At the moment, the interface is implemented
by three concrete classes: `query::procedure::BuiltinModule`,
`query::procedure::SharedLibraryModule` and `query::procedure::PythonModule`.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2664
2020-02-11 16:56:26 +01:00
Teon Banek
d17d1497d1 Add missing includes to py.hpp
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2661
2020-02-10 14:37:05 +01:00
Tonko Sabolcec
e2e7823ec4 Add basic support for unique constraints in storage
Summary:
This diff contains a basic implementation of unique constraints consistent with
the MVCC storage.
Stale records in the unique constraints are collected by the garbage collector.
Tests for checking correctness of unique constraints and violations are included.
Note: currently we only support a pair of label and a single property. Support for
multiple properties will be added later.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: buda, ipaljak, pullbot

Differential Revision: https://phabricator.memgraph.io/D2608
2020-02-10 12:45:28 +01:00
Teon Banek
ab53cbb0c3 Embed Python and import modules into global interpreter
Summary:
This only imports the `.py` files into the global interpreter. These
modules are not exposed to query execution. A later diff will add that
support.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2660
2020-02-10 11:06:08 +01:00
Matej Ferencevic
90286b8166 Install license, systemd and examples from release
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2659
2020-02-07 10:54:07 +01:00
Matej Ferencevic
7128320b36 Implement MG_ENTERPRISE CMake flag
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2658
2020-02-06 13:39:30 +01:00
Matej Ferencevic
87511460f1 Remove unused CMake options
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2656
2020-02-05 14:52:22 +01:00
Matej Ferencevic
a3229881bf Move Antlr generation to query
Summary:
The antlr openCypher parser generation is moved to query. Also, header files
have been added to the list of generated files so that if any header file is
deleted CMake will know that it has to regenerate it.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2652
2020-01-31 16:14:05 +01:00