Commit Graph

2623 Commits

Author SHA1 Message Date
Teon Banek
dd30522b2a Update mgp_result_new_record documentation
Reviewers: mferencevic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2575
2019-11-28 14:58:46 +01:00
Teon Banek
e31331aae4 Rework CallProcedure to validate result sets
Summary:
This diff renames `__reload__` procedure to be `mg.reload` accepting a
module name. The main CallCustomProcedure function is now split into
multiple parts, so that there's more control over finding a procedure,
type checking its arguments and finally checking the returned result
set.

Depends on D2572

Reviewers: mferencevic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2573
2019-11-28 14:58:43 +01:00
Teon Banek
d71f1bfa35 Type check procedure arguments in CALL clause
Summary: Depends on D2571

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2572
2019-11-28 14:58:36 +01:00
Teon Banek
e1399582b8 Implement SatisfiesType on CypherType classes
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2571
2019-11-28 14:58:34 +01:00
Matej Ferencevic
c5def558e8 Cache vertex accessor for all vertices iterable
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2569
2019-11-27 10:55:12 +01:00
Teon Banek
df6ce53c67 query/procedure: Expose vertex ID through C API
Reviewers: mferencevic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2568
2019-11-26 17:06:31 +01:00
Ivan Paljak
03ecc58715 Make the Louvain code comply with our conventions
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2566
2019-11-26 10:06:19 +01:00
Matej Ferencevic
9e04183be7 Implement query execution timeout
Summary:
The query execution now has a timeout for each Cypher query it executes. The
timeout is implemented using TSC and will work only when TSC is available (same
as PROFILE). TSC is used to mitigate the performance impact of reading the
current time constantly.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2562
2019-11-25 17:45:40 +01:00
Teon Banek
673ebf428c Add functions for treating character strings as byte strings
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2565
2019-11-25 17:03:12 +01:00
Ivan Paljak
7467d52d5b Add cmake files and a dummy louvain query module
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2561
2019-11-25 16:38:56 +01:00
Matej Ferencevic
fc146a71e0 Migrate tests to storage v2 part 3
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2560
2019-11-25 16:21:04 +01:00
Matej Ferencevic
8a94b16e79 Fix parameters for PROFILE queries
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2563
2019-11-25 11:05:47 +01:00
Matej Ferencevic
42a4f310ff Reduce number of copies in query::Parameters
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2564
2019-11-25 11:00:55 +01:00
Teon Banek
3c615759b6 Type check values used for indexed lookup in range
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2558
2019-11-22 10:28:01 +01:00
Matej Ferencevic
f61a8c3358 Migrate tests to storage v2 part 2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2559
2019-11-22 10:21:30 +01:00
Matej Ferencevic
9cc10e131d Migrate tests to storage v2 part 1
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2554
2019-11-21 19:00:32 +01:00
Ivan Paljak
ae0be9032e Implement Louvain community detection algorithm
Summary: Louvain implementation according to original paper.

Reviewers: dsantl

Reviewed By: dsantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2556
2019-11-21 15:06:30 +01:00
Matej Ferencevic
29590f1112 Implement edge iteration with known destination
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2553
2019-11-21 11:58:12 +01:00
Teon Banek
019b6cddaa Use a custom value printer for procedure signature
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2557
2019-11-21 11:17:00 +01:00
Teon Banek
5354a11a00 Add registering multiple procedures in query modules
Reviewers: mferencevic, dsantl, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2551
2019-11-21 10:13:09 +01:00
Matej Ferencevic
8be2b4af63 Remove output formatters from vertex/edge accessors
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2555
2019-11-20 20:12:19 +01:00
Matej Ferencevic
d5f02c5ef8 Fix label+property index iteration in v2
Summary:
Now when iterating over a label+property index the index verifies that the
bounds meet the criteria imposed by openCypher.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2552
2019-11-18 14:54:45 +01:00
Teon Banek
1dc97a37f8 Cache globally allocated mgp_type instances
Summary:
This simplifies the C API and reduces total allocations done when
constructing a type.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2550
2019-11-15 10:44:55 +01:00
Matej Ferencevic
d17d463884 Add NONEXISTENT_OBJECT error to storage v2
Summary:
Previously, when accessing the labels/properties/edges of a vertex/edge that
was just created the NEW view would correctly display the change, but the OLD
view would be invalid and would crash the database. With this change the OLD
view of a freshly created vertex/edge won't cause a crash, but will instead
report an error.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2549
2019-11-15 10:01:22 +01:00
Matej Ferencevic
40ae31e9a0 Remove manual/repl test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2548
2019-11-13 16:32:45 +01:00
Teon Banek
edaa03a960 Expose explicit Label index creation through Cypher
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2547
2019-11-13 16:28:14 +01:00
Teon Banek
9c095501d8 query/procedure: Add type system for Cypher
Summary:
The type system is modelled after "CIP2015-09-16"
https://github.com/opencypher/openCypher/blob/master/cip/1.accepted/CIP2015-09-16-public-type-system-type-annotation.adoc

This is needed for registering procedures and their signatures. The
users will be able to specify what a custom procedure accepts and
returns. All of this needs to be available for inspection during
runtime. Therefore, this diff implements printing types as a user
presentable string. In the future, we will probably want to add type
checking through these types, because openCypher requires type checking
on values passed in and returned from custom procedures.

Reviewers: mferencevic, ipaljak, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2544
2019-11-13 14:24:21 +01:00
Teon Banek
b4df6ba0a9 Correctly prepare Explain and Profile
Summary: Also test Explain and Profile through Intepreter.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2546
2019-11-13 12:03:11 +01:00
Matej Ferencevic
329818b1b0 Make the Cypher dumper a function
Summary:
The dumper is now a function and doesn't have to worry about any state. The
function streams the Cypher queries directly to the client. This diff also
makes the dumper work with storage v2.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2545
2019-11-12 16:01:29 +01:00
Matej Ferencevic
e4edb2be99 Finalize storage v2 index API
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2543
2019-11-12 12:11:38 +01:00
Matej Ferencevic
205477b28a Implement indices and constraints info for v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2542
2019-11-12 11:06:53 +01:00
Lovro Lugovic
8735082bbd Don't use OutputTable within query handlers
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2541
2019-11-11 14:12:38 +01:00
Lovro Lugovic
905b3ee2df Don't unconditionally start a transaction on Prepare
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2532
2019-11-11 13:40:06 +01:00
Matej Ferencevic
a850644f5d Fix atomic memory ordering in durability
Summary:
The atomic memory order should be `acquire` for `load` operations, `release`
for `store` operations and `acq_rel` for any RMW (read-modify-write) operations
(like `fetch_add`).

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2540
2019-11-11 12:57:04 +01:00
Matej Ferencevic
e350e2b7a4 Make the database dumper use only Vertices
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2539
2019-11-08 14:36:12 +01:00
Lovro Lugovic
cdb9c08047 Merge Interpret and Prepare
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2502
2019-11-08 10:25:51 +01:00
Lovro Lugovic
bf03c57181 Remove Results from the Interpreter
Summary:
- Add the `AnyStream` wrapper
- Remove the `Results` struct and store a function (handler) instead

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2497
2019-11-08 09:22:30 +01:00
Teon Banek
283a91cc60 Integrate loading openCypher module procedures
Summary:
All mgp_* symbols are exported from Memgraph executable, no other
symbols should be visible.

The primary C API header, mg_procedure.h, is now part of the
installation. Also, added a shippable query module example.

Directory `query_modules` is meant to contain sources of modules we
write and ship as part of the installation. Currently, there's only an
example module, but there may be potentially more. Some modules could
only be installed as part of the enterprise release.

For Memgraph to load custom procedures, it needs to be started with a
flag pointing to a directory with compiled shared libraries implementing
those procedures.

Reviewers: mferencevic, ipaljak, llugovic, dsantl, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2538
2019-11-07 15:23:02 +01:00
Matej Ferencevic
9eb1f1d5cd Implement ClearProperties for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2537
2019-11-07 13:07:26 +01:00
Teon Banek
df43ccea95 Add CallProcedure LogicalOperator
Summary: Depends on D2534

Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2535
2019-11-06 15:46:00 +01:00
Teon Banek
5567e6ec96 Add utils::LimitedMemoryResource
Reviewers: llugovic, mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2534
2019-11-06 15:45:56 +01:00
Matej Ferencevic
a835939a6e Implement buffering for utils file wrappers
Summary:
This change increases the throughput of the storage v2 durability 20x. With
this change, the storage v2 durability is 3x faster than the storage v1
durability in both recovery and snapshotting (before the change v2 durability
is slower than v1 durability).

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2529
2019-11-05 12:30:09 +01:00
Matej Ferencevic
47d6196b32 Improve binary coverage script
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2530
2019-11-04 17:02:03 +01:00
Matej Ferencevic
73134ab6b7 Integrate all storage v2 flags into main binary
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2528
2019-11-04 17:01:56 +01:00
Teon Banek
32069c77a0 Add CallProcedure clause to Cypher
Summary:
This adds support for basic invocation to CALL clause of openCypher. The
accepted CIP has a lot more features that are avaiable with CALL clause.

https://github.com/opencypher/openCypher/blob/master/cip/1.accepted/CIP2015-06-24-call-procedures.adoc#appendix-procedure-naming-conventions

Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2523
2019-11-04 16:21:29 +01:00
Matej Ferencevic
4248b140d4 Backup existing durability files
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2526
2019-11-04 10:38:13 +01:00
Matej Ferencevic
42516afce8 Remove Kafka integration implementation and tests
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2525
2019-11-04 09:56:07 +01:00
Matej Ferencevic
875a4a8629 Implement WAL loading for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2521
2019-10-31 17:12:37 +01:00
Teon Banek
b5482a8169 Add .clang-tidy for tests and update the config
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2524
2019-10-31 16:07:09 +01:00
Teon Banek
043ae87a83 query/procedure: Implement graph related API
Reviewers: mferencevic, llugovic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2522
2019-10-30 15:11:53 +01:00