Commit Graph

2607 Commits

Author SHA1 Message Date
Ivan Paljak
ae0be9032e Implement Louvain community detection algorithm
Summary: Louvain implementation according to original paper.

Reviewers: dsantl

Reviewed By: dsantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2556
2019-11-21 15:06:30 +01:00
Matej Ferencevic
29590f1112 Implement edge iteration with known destination
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2553
2019-11-21 11:58:12 +01:00
Teon Banek
019b6cddaa Use a custom value printer for procedure signature
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2557
2019-11-21 11:17:00 +01:00
Teon Banek
5354a11a00 Add registering multiple procedures in query modules
Reviewers: mferencevic, dsantl, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2551
2019-11-21 10:13:09 +01:00
Matej Ferencevic
8be2b4af63 Remove output formatters from vertex/edge accessors
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2555
2019-11-20 20:12:19 +01:00
Matej Ferencevic
d5f02c5ef8 Fix label+property index iteration in v2
Summary:
Now when iterating over a label+property index the index verifies that the
bounds meet the criteria imposed by openCypher.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2552
2019-11-18 14:54:45 +01:00
Teon Banek
1dc97a37f8 Cache globally allocated mgp_type instances
Summary:
This simplifies the C API and reduces total allocations done when
constructing a type.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2550
2019-11-15 10:44:55 +01:00
Matej Ferencevic
d17d463884 Add NONEXISTENT_OBJECT error to storage v2
Summary:
Previously, when accessing the labels/properties/edges of a vertex/edge that
was just created the NEW view would correctly display the change, but the OLD
view would be invalid and would crash the database. With this change the OLD
view of a freshly created vertex/edge won't cause a crash, but will instead
report an error.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2549
2019-11-15 10:01:22 +01:00
Matej Ferencevic
40ae31e9a0 Remove manual/repl test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2548
2019-11-13 16:32:45 +01:00
Teon Banek
edaa03a960 Expose explicit Label index creation through Cypher
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2547
2019-11-13 16:28:14 +01:00
Teon Banek
9c095501d8 query/procedure: Add type system for Cypher
Summary:
The type system is modelled after "CIP2015-09-16"
https://github.com/opencypher/openCypher/blob/master/cip/1.accepted/CIP2015-09-16-public-type-system-type-annotation.adoc

This is needed for registering procedures and their signatures. The
users will be able to specify what a custom procedure accepts and
returns. All of this needs to be available for inspection during
runtime. Therefore, this diff implements printing types as a user
presentable string. In the future, we will probably want to add type
checking through these types, because openCypher requires type checking
on values passed in and returned from custom procedures.

Reviewers: mferencevic, ipaljak, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2544
2019-11-13 14:24:21 +01:00
Teon Banek
b4df6ba0a9 Correctly prepare Explain and Profile
Summary: Also test Explain and Profile through Intepreter.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2546
2019-11-13 12:03:11 +01:00
Matej Ferencevic
329818b1b0 Make the Cypher dumper a function
Summary:
The dumper is now a function and doesn't have to worry about any state. The
function streams the Cypher queries directly to the client. This diff also
makes the dumper work with storage v2.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2545
2019-11-12 16:01:29 +01:00
Matej Ferencevic
e4edb2be99 Finalize storage v2 index API
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2543
2019-11-12 12:11:38 +01:00
Matej Ferencevic
205477b28a Implement indices and constraints info for v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2542
2019-11-12 11:06:53 +01:00
Lovro Lugovic
8735082bbd Don't use OutputTable within query handlers
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2541
2019-11-11 14:12:38 +01:00
Lovro Lugovic
905b3ee2df Don't unconditionally start a transaction on Prepare
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2532
2019-11-11 13:40:06 +01:00
Matej Ferencevic
a850644f5d Fix atomic memory ordering in durability
Summary:
The atomic memory order should be `acquire` for `load` operations, `release`
for `store` operations and `acq_rel` for any RMW (read-modify-write) operations
(like `fetch_add`).

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2540
2019-11-11 12:57:04 +01:00
Matej Ferencevic
e350e2b7a4 Make the database dumper use only Vertices
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2539
2019-11-08 14:36:12 +01:00
Lovro Lugovic
cdb9c08047 Merge Interpret and Prepare
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2502
2019-11-08 10:25:51 +01:00
Lovro Lugovic
bf03c57181 Remove Results from the Interpreter
Summary:
- Add the `AnyStream` wrapper
- Remove the `Results` struct and store a function (handler) instead

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2497
2019-11-08 09:22:30 +01:00
Teon Banek
283a91cc60 Integrate loading openCypher module procedures
Summary:
All mgp_* symbols are exported from Memgraph executable, no other
symbols should be visible.

The primary C API header, mg_procedure.h, is now part of the
installation. Also, added a shippable query module example.

Directory `query_modules` is meant to contain sources of modules we
write and ship as part of the installation. Currently, there's only an
example module, but there may be potentially more. Some modules could
only be installed as part of the enterprise release.

For Memgraph to load custom procedures, it needs to be started with a
flag pointing to a directory with compiled shared libraries implementing
those procedures.

Reviewers: mferencevic, ipaljak, llugovic, dsantl, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2538
2019-11-07 15:23:02 +01:00
Matej Ferencevic
9eb1f1d5cd Implement ClearProperties for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2537
2019-11-07 13:07:26 +01:00
Teon Banek
df43ccea95 Add CallProcedure LogicalOperator
Summary: Depends on D2534

Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2535
2019-11-06 15:46:00 +01:00
Teon Banek
5567e6ec96 Add utils::LimitedMemoryResource
Reviewers: llugovic, mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2534
2019-11-06 15:45:56 +01:00
Matej Ferencevic
a835939a6e Implement buffering for utils file wrappers
Summary:
This change increases the throughput of the storage v2 durability 20x. With
this change, the storage v2 durability is 3x faster than the storage v1
durability in both recovery and snapshotting (before the change v2 durability
is slower than v1 durability).

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2529
2019-11-05 12:30:09 +01:00
Matej Ferencevic
47d6196b32 Improve binary coverage script
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2530
2019-11-04 17:02:03 +01:00
Matej Ferencevic
73134ab6b7 Integrate all storage v2 flags into main binary
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2528
2019-11-04 17:01:56 +01:00
Teon Banek
32069c77a0 Add CallProcedure clause to Cypher
Summary:
This adds support for basic invocation to CALL clause of openCypher. The
accepted CIP has a lot more features that are avaiable with CALL clause.

https://github.com/opencypher/openCypher/blob/master/cip/1.accepted/CIP2015-06-24-call-procedures.adoc#appendix-procedure-naming-conventions

Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2523
2019-11-04 16:21:29 +01:00
Matej Ferencevic
4248b140d4 Backup existing durability files
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2526
2019-11-04 10:38:13 +01:00
Matej Ferencevic
42516afce8 Remove Kafka integration implementation and tests
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2525
2019-11-04 09:56:07 +01:00
Matej Ferencevic
875a4a8629 Implement WAL loading for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2521
2019-10-31 17:12:37 +01:00
Teon Banek
b5482a8169 Add .clang-tidy for tests and update the config
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2524
2019-10-31 16:07:09 +01:00
Teon Banek
043ae87a83 query/procedure: Implement graph related API
Reviewers: mferencevic, llugovic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2522
2019-10-30 15:11:53 +01:00
Lovro Lugovic
bd998dc618 Refactor query parsing in the Interpreter
Summary: Depends on D2482

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2499
2019-10-30 10:32:22 +01:00
Lovro Lugovic
b0cc564f8d Make ResultStreamFaker a normal class
Summary:
Store accumulated results as `communication::bolt::Value`s instead of
`TypedValue`s.

Add additional overloads for `Result` and `Summary` which accept `TypedValue`s
but internally perform conversions.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2514
2019-10-30 10:22:21 +01:00
Matej Ferencevic
13ba9cc23e Implement WAL writing for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2515
2019-10-29 16:18:12 +01:00
Teon Banek
4879686ef4 query/procedure: Implement mgp_list & mgp_map API
Reviewers: mferencevic, llugovic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2518
2019-10-29 15:52:05 +01:00
Teon Banek
a72743757c query/procedure: Implement mgp_result API
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2516
2019-10-29 11:02:30 +01:00
Matej Ferencevic
2e0586e182 Implement single function for WAL loading
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2513
2019-10-28 15:42:49 +01:00
Teon Banek
4c6eb0746e query/procedure: Implement mgp_value C API
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2498
2019-10-28 15:24:14 +01:00
Lovro Lugovic
3750fbc093 Merge Interpreter and TransactionEngine
Summary:
Depends on D2471

- Add pointer to storage to `InterpreterContext`
- Rename `operator()` to `Prepare`
- Use `Interpret` instead of `operator()` (`Interpret` will be removed soon)
- Remove the `in_explicit_transaction` parameter
- Remove the memory resource parameter from `Interpret`
- Remove the storage accessor parameter from `Interpret`
- Fix up tests (remove the `Interpreter` from `database_transaction_timeout`)

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2482
2019-10-25 16:11:10 +02:00
Matej Ferencevic
98855889d5 Implement WalFile for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2510
2019-10-25 16:08:17 +02:00
Lovro Lugovic
d2fac02b74 Pull out InterpreterContext
Summary: Make `InterpreterContext` a top level instead of a nested struct

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2512
2019-10-25 15:58:47 +02:00
Lovro Lugovic
4c25719c45 Extract state shared between interpreters into InterpreterContext
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2471
2019-10-25 14:12:14 +02:00
Teon Banek
35180f11d3 Update .clang-tidy to not warn trailing return type
Summary:
This is primarily an update which disables the
`modernize-use-trailing-return-type` suggestion introduced in
clang-tidy-9. Old clang-tidy versions should ignore removed checks which
don't exist in the first place, so this should change should be
compatible with any version.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2511
2019-10-25 11:16:50 +02:00
Matej Ferencevic
bafc0a9d16 Reduce memory blowup on each Pull
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2509
2019-10-23 10:01:17 +02:00
Matej Ferencevic
3bed1cc3ea Correctly pass config into AllVerticesIterable
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2500
2019-10-21 14:15:32 +02:00
Teon Banek
e566126832 query/procedure: Implement memory management C API
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2489
2019-10-18 14:35:04 +02:00
Teon Banek
469102aa5e Pass mgp_memory to custom Module.main_fn
Summary: Depends on D2495

Reviewers: mferencevic, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2496
2019-10-18 13:38:01 +02:00