Commit Graph

785 Commits

Author SHA1 Message Date
Teon Banek
97b20a9384 Use non-template Value invocation
Reviewers: mtomic, mferencevic

Reviewed By: mtomic, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2317
2019-08-22 16:20:31 +02:00
Matej Ferencevic
529b7109f7 Use new SkipList for query caches
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2305
2019-08-22 14:10:41 +02:00
Teon Banek
f7575eb112 Add utilities for type checking awesome MG functions
Reviewers: mtomic, msantl

Reviewed By: mtomic, msantl

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2259
2019-08-12 16:24:24 +02:00
Matej Ferencevic
111dd8bf19 Remove distributed
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2213
2019-07-17 15:23:42 +02:00
Matej Ferencevic
d5106c7233 Handle zero increment in counters
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2208
2019-07-16 12:35:35 +02:00
Teon Banek
c4c6febbc4 Move Parsing utils from query/common to frontend/parsing
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2195
2019-07-11 12:12:59 +02:00
Teon Banek
a0e1175634 Remove unused includes from operator.cpp
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2194
2019-07-10 15:15:53 +02:00
Teon Banek
5bc0be50c5 Remove GraphDbAccessor parameter from MakeCursor
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2189
2019-07-09 14:22:55 +02:00
Teon Banek
775930ef4e Use stack for initial memory of MonotonicBufferResource in Pull
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2139
2019-06-26 10:26:53 +02:00
Teon Banek
e9dcb1dcf2 Use a different allocator for per-pull allocations
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2137
2019-06-26 10:26:53 +02:00
Teon Banek
5adb4aafe0 Use execution memory for Frame allocations
Summary:
Micro benchmarks show some minor variations compared to the previous
commit. Smaller cases are a bit worse while larger data cases are a bit
better.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2136
2019-06-24 15:05:44 +02:00
Teon Banek
6086257204 Use per_pull and whole execution allocators in Cursors
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2134
2019-06-24 15:05:43 +02:00
Teon Banek
65ab2574bc Use memory from EvaluationContext in awesome functions
Reviewers: mferencevic, msantl, mtomic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2133
2019-06-18 14:13:23 +02:00
Teon Banek
3fd14e2d5f Use custom allocator in Evaluator through context
Summary:
Micro benchmarks show improvements in performance of MapLiteral from 5%
to 40% depending on the size of the input. On the other hand, a sequence
of AdditionOperators behaves the same with both allocation schemes.

Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2132
2019-06-17 10:53:25 +02:00
Teon Banek
57d967786c Make TypedValue constructor explicit for non-primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2128
2019-06-12 14:55:58 +02:00
Teon Banek
b3bc4d6809 Make TypedValue constructor explicit for primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2127
2019-06-12 14:55:57 +02:00
Teon Banek
6d3e3ac4aa Remove TypedValue::Null
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2121
2019-06-12 09:55:18 +02:00
Teon Banek
df90184b50 Use utils::Allocator in query::Path
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2117
2019-06-12 09:55:18 +02:00
Teon Banek
96505b7fb4 Use Allocator for map in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2116
2019-06-12 09:55:17 +02:00
Lovro Lugovic
59af45f94e LCP: Fix up LCP warnings
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2090
2019-06-12 09:37:47 +02:00
Tonko Sabolcec
eb45ed09df Dump cleanup
Summary:
`dump()` awesome Memgraph function is removed.
`mg_dump` executable now uses "dump database" command instead of "return dump()"
and dumps database to standard output instead of a file.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2122
2019-06-10 17:01:31 +02:00
Tonko Sabolcec
9cc2224ac3 Handle dump database query
Summary:
Stream queries to the output table.
For effective output streaming, a simple operator, `OutputTableStream` is implemented which fetches and
produces a single row on each Pull.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2099
2019-06-10 13:22:14 +02:00
Teon Banek
fa62ea1920 Use ValueMap instead of Value<>
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2114
2019-06-10 10:35:04 +02:00
Teon Banek
6bb72d14a1 Use Allocator for string in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2104
2019-06-10 10:35:04 +02:00
Teon Banek
38c6625e2c utils: Support std::basic_string with allocators
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2102
2019-06-03 17:07:49 +02:00
Teon Banek
c0dc37fe34 Use ValueString instead of Value<>
Reviewers: mtomic, llugovic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2101
2019-06-03 17:07:49 +02:00
Teon Banek
bb2dbc290f Use Allocator for vector in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2086
2019-06-03 17:07:49 +02:00
Teon Banek
0152ac2dfe Add MemoryResource to TypedValue
Summary:
This diff only introduces a MemoryResource member to TypedValue and correctly
propagates through various constructors and assignments. At the moment,
MemoryResource is not used to actually allocate anything.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2085
2019-06-03 17:07:48 +02:00
Tonko Sabolcec
1bcb3cbd86 Add grammar for dump query
Summary:
Defined Antlr4 grammar for `DUMP DATABASE` query.
Defined DUMP permission in auth module.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2077
2019-06-03 10:44:18 +02:00
Matija Santl
87423eed06 Add constraint in HA
Summary:
HA should now support constraints in the same way the SM version does.
I only tested this thing manually, but I plan to add a new integration test for
this also.

Reviewers: ipaljak, vkasljevic, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2083
2019-05-29 14:55:04 +02:00
Tonko Sabolcec
d0f05c6cc5 Create simple dump executable
Summary:
Implemented a simple dump executable which runs `RETURN dump()` on the database using Bolt protocol.
The `dump()` function returns a list of openCypher queries needed for database reconstruction (note: in the future we should use more appropriate method that supports streaming of data).

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D2067
2019-05-29 10:30:41 +02:00
Teon Banek
f6264ab2ae Use ValueList instead of Value<>
Summary:
This change will make the transition to allocator backed TypedValue
smoother.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2084
2019-05-28 09:41:31 +02:00
Teon Banek
64a05bc972 Use CHECK in TypedValueVectorCompare|Equal
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2066
2019-05-28 09:41:31 +02:00
Matija Santl
e4fd49a530 Remove unique index from query execution
Summary: Remove leftover unique index code from query execution.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2078
2019-05-28 08:16:35 +02:00
Matija Santl
f2cc41fa59 Remove unique from label-prop index
Summary: Remove unique feature from label-property index

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: tsabolcec, pullbot

Differential Revision: https://phabricator.memgraph.io/D2045
2019-05-23 12:39:41 +02:00
Matej Ferencevic
3c48e612ef Make move operations noexcept
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1991
2019-05-21 10:46:25 +02:00
Teon Banek
ee2fa47b29 Remove inheriting TotalOrdering in TypedValue
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2070
2019-05-21 09:51:18 +02:00
Matija Santl
5d5dfbb6f7 Fix how HA handles leader change during commit
Summary:
During it's leadership, one peer can receive RPC messages from other peers that his reign is over.
The problem is when this happens during a transaction commit.

This is handled in the following way.
If we're the current leader and we want to commit a transaction, we need to make sure the Raft Log is replicated before we can tell the client that the transaction is committed.
During that wait, we can only notice that the replication takes too long, and we report that with `LOG(WARNING)` messages.

If we change the Raft mode during the wait, our Raft implementation will internally commit this transaction, but won't be able to acquire the Raft lock because the `db.Reset` has been called.
This is why there is an manual lock acquire. If we pick up that the `db.Reset` has been called, we throw an `UnexpectedLeaderChangeException` exception to the client.

Another thing with long running transactions, if someone decides to kill a `memgraph_ha` instance during the commit, the transaction will have `abort` hint set. This will cause the `src/query/operator.cpp` to throw a `HintedAbortError`. We need to catch this during the shutdown, because the `memgraph_ha` isn't dead from the user perspective, and the transaction wasn't aborted because it took too long, but we can differentiate between those two.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1956
2019-05-20 16:39:44 +02:00
Matija Santl
37c68f0508 Add multiple properties unique constraint
Summary:
Unique constraint now support multiple properties

Depends on D2043

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2044
2019-05-20 15:18:05 +02:00
Teon Banek
4f4837392e Use MemoryResource in UnwindCursor
Summary:
Micro benchmarks show no change compared to global new & delete. This is
to be expected, because Unwind relies only on `std::vector` which ought
to reserve the memory in reasonable chunks.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2064
2019-05-17 15:28:54 +02:00
Teon Banek
880f23620d Move UnwindCursor from LCP to CPP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2062
2019-05-17 15:28:54 +02:00
Teon Banek
8459fd90fc Use MemoryResource in OrderByCursor
Summary:
Micro benchmarks show an improvement to performance of about 10%
compared to global new & delete.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2061
2019-05-17 15:28:53 +02:00
Teon Banek
abbb57c868 Move OrderByCursor from LCP to CPP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2060
2019-05-17 15:28:53 +02:00
Teon Banek
0075eee58b Use MemoryResource in AggregationCursor
Summary: Micro benchmarks show some improvement, but unfortunately not much.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2058
2019-05-17 15:28:53 +02:00
Matej Ferencevic
d4b2d76a35 Reimplement counter openCypher function
Reviewers: teon.banek, msantl

Reviewed By: teon.banek, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2047
2019-05-16 11:09:02 +02:00
Teon Banek
98a853a95c Move AggregateCursor from LCP to CPP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2056
2019-05-16 09:58:15 +02:00
Teon Banek
1cc71c6ce8 Use MemoryResource in AccumulateCursor
Summary:
Micro benchmarks show that MonotonicBufferResource improves performance
by a factor of 1.5.

Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2048
2019-05-15 16:13:35 +02:00
Teon Banek
78830b6ed8 Use MemoryResource in STShortestPathCursor
Summary:
Benchmarks show minor improvements. Perhaps it makes sense at some later date
to use another allocator for things lasting only in a single `Pull`.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2018
2019-05-15 16:13:23 +02:00
Teon Banek
801bdb3a91 Use MemoryResource for members of ExpandVariableCursor
Summary:
Unfortunately, the written micro benchmark only reports minor
improvements compared to default allocator. The results are in some
cases even a tiny bit worse.

Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2039
2019-05-15 16:07:27 +02:00
Matija Santl
d70792f1ce Remove existence constraint
Summary: Removing existence constraint

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2043
2019-05-15 09:48:51 +02:00
Teon Banek
7a586b3686 Allocate Cursor through utils::MemoryResource
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2026
2019-05-13 11:36:40 +02:00
Teon Banek
2909ef63d2 Use utils::MonotonicBufferResource in query execution
Reviewers: mferencevic, mtomic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2016
2019-05-13 11:36:19 +02:00
Lovro Lugovic
017dec8c0e LCP: Refactor the type representation
Summary:
# Summary

## Concepts and terminology

A **namestring for <object>** is a Lisp string that names the C++ language element
`<object>` (such as a namespace, a variable, a class, etc.). Therefore we have a
"namestring for a namespace", a "namestring for a variable", etc.

A **designator for a namestring for <object>** is a Lisp object that denotes a
"namestring for <object>". These are symbols and strings themselves.

A **typestring** is a Lisp string that represents a C++ type, i.e. its
declaration.

A **typestring designator** is a Lisp object that denotes a typestring.

A typestring (and the corresponding C++ type) is said to be **supported** if it
can be parsed using `parse-cpp-type-declaration`. This concept is important and
should form the base of our design because we can't really hope to ever support
all of C++'s type declarations.

A typestring (and the corresponding C++ type) that is not supported is
**unsupported**.

A **processed typestring** is a typestring that is either fully qualified or
unqualified.

A C++ type is said to be **known** if LCP knows extra information about the type,
rather than just what kind of type it is, in which namespace it lives, etc. For
now, the only known types are those that are defined within LCP itself using
`define-class` & co.

A C++ type is **unknown** if it is not known.

**Typestring resolution** is the process of resolving a (processed) typestring
into an instance of `cpp-type` or `unsupported-cpp-type`.

**Resolving accessors** are accessors which perform typestring resolution.

## Changes

Explicitly introduce the concept of supported and known types. `cpp-type` models
supported types while `unsupported-cpp-type` models unsupported types.
Subclasses of `cpp-type` model known types. `general-cpp-type` is either a
`cpp-type` or an `unsupported-cpp-type`.

Add various type queries.

Fix `define-rpc`'s `:initarg` (remove it from the `cpp-member` struct).

Introduce namestrings and their designators in `names.lisp`.

Introduce typestrings and their designators.

Introduce **processed typestrings**. Our DSL's macros (`define-class` & co.)
convert all of the given typestrings into processed typestrings because we don't
attempt to support partially qualified names and relative name lookup yet. A
warning is signalled when a partially qualified name is treated as a fully
qualified name.

The slots of `cpp-type`, `cpp-class`, `cpp-member` and `cpp-capnp-opts` now
store processed typestrings which are lazily resolved into their corresponding
C++ types.

The only thing that instances of `unsupported-cpp-type` are good for is getting
the typestring that was used to construct them. Most of LCP's functions only
work with known C++ types, i.e. `cpp-type` instances. The only function so far
that works for both of them is `cpp-type-decl`.

Since "unsupportedness" is now explicitly part of LCP, client code is expected
to manually check whether a returned type is unsupported or not (or face
receiving an error otherwise), unless a function is documented to return only
`cpp-type` instances.

A similar thing goes for "knowness". Client code is expected to manually check
whether a returned type is known or not, unless a function is documented to
return only (instances of `cpp-type` subclasses) known types.

## TODO

Resolution still has to be done for the following slots of the
following structures:

-   `slk-opts`
    -   `save-args`
    -   `load-args`

-   `clone-opts`
    -   `args`
    -   `return-type`

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1962
2019-05-10 16:18:31 +02:00
Teon Banek
394039a05e Use and bench custom allocator in Distinct
Summary:
According to the written benchmark, using MonotonicBufferResource yields
significant improvements to performance of Distinct. The setup fills the
database with vertices depending on the benchmark state. No edges are
created. Then we run DISTINCT on that. Since each vertex is unique, we
will store everything in the `DistinctCursor::seen_rows_`, which is
backed by a MemoryResource. This setup, on my machine, yields 10 times
better performance when run with MonotonicBufferResource.

Reviewers: mferencevic, mtomic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1894
2019-05-09 15:45:49 +02:00
Tonko Sabolcec
4d8f8be1ab Add function that dumps vertices and edges of the graph.
Summary: Fix lint errors

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1996
2019-05-08 10:52:52 +02:00
Matej Ferencevic
5c244c1ad4 Remove Cap'n Proto
Summary:
There will be a lot of leftover files, execute the following commands inside
`src/` to remove them:
```
git clean -xf
rm -r rpc/ storage/single_node_ha/rpc/
```

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2011
2019-05-08 10:51:10 +02:00
Teon Banek
8e6317436d Add missing comma to stripped_lexer_constants
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2010
2019-05-07 09:39:05 +02:00
Teon Banek
bc46de7b33 Move DistinctCursor to operator.cpp
Summary: Depends on D1980

Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1986
2019-04-29 10:35:44 +02:00
Teon Banek
e3fbe56588 Accept utils::MemoryResource in MakeCursor
Summary:
This API change is needed in order to propagate the memory allocation
scheme for the execution of LogicalOperator::Cursor

Depends on D1990

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1980
2019-04-29 10:34:54 +02:00
Matej Ferencevic
8b7df83473 Abort SHOW RAFT INFO queries
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1983
2019-04-25 17:38:40 +02:00
Teon Banek
e128a9f80f Move distributed query stuff to new folder
Summary:
The new distributed directory is inside the query, and mirrors the query
structure. This groups all of the distributed (query) source code
together, which should make the potential directory extraction easier.

Reviewers: mferencevic, llugovic, mtomic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1923
2019-04-24 09:34:53 +02:00
Matej Ferencevic
9291a5fc4d Migrate to C++17
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1974
2019-04-23 14:46:44 +02:00
Matej Ferencevic
1e7bdcec4d Initial version of HA client
Reviewers: msantl, ipaljak

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1958
2019-04-18 16:48:30 +02:00
Matej Ferencevic
45f2a06a8b Fix various compiler warnings and remove unused test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1963
2019-04-17 15:48:52 +02:00
Matej Ferencevic
026c796e06 Move GraphDbAccessor to stack in SN and HA
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1944
2019-04-17 11:29:43 +02:00
Marin Tomic
c269e2f468 Add syntax for node key constraints
Reviewers: msantl, teon.banek, llugovic, vkasljevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1953
2019-04-11 10:23:37 +02:00
Matej Ferencevic
63aeed0624 Add SHOW RAFT INFO query to HA
Reviewers: msantl, mtomic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1946
2019-04-08 13:55:55 +02:00
Marin Tomic
dc231fe4e7 Fix bug in SymbolTable
Summary:
This is a bugfix for D1836. It made `SymbolTable` return references to vector
elements, which then get invalidated and weird stuff happens.

This made a `DCHECK` in `rule_based_planner.hpp` trigger, and it was noticed by
@ipaljak 2 months later. All `DCHECK`s in `rule_based_planner.hpp` are now
changed to `CHECK`s.

Also, hash function for `Symbol` was wrong, because it also took
`user_declared` field into consideration, and `==` operator doesn't do that.

Reviewers: ipaljak, teon.banek, mferencevic, msantl

Reviewed By: msantl

Subscribers: pullbot, ipaljak

Differential Revision: https://phabricator.memgraph.io/D1938
2019-04-02 16:13:48 +02:00
Marin Tomic
3965d2341b Add grammar for unique constraints
Summary:
This should be the same as Neo4J grammar. Please check:
https://neo4j.com/docs/cypher-refcard/current/

Reviewers: teon.banek, msantl, llugovic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1936
2019-03-28 16:48:16 +01:00
Teon Banek
49b552408b Use label+property index with regex matching
Reviewers: mtomic, llugovic, msantl

Reviewed By: mtomic

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D1890
2019-03-19 16:15:32 +01:00
Teon Banek
12799f12ef Fill the EvaluationContext correctly for ProfileQuery
Reviewers: mtomic, llugovic, msantl

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1919
2019-03-15 13:07:44 +01:00
Matija Santl
1de34d8b92 Add proper storage stats for HA
Summary:
`SHOW STORAGE STATS` when executed in a Raft cluster should return
stats for each member of the cluster.

`StorageStats` starts a RPC server on each member of the cluster that answers
about its local storage stats.

The query can be invoked only on the current leader, the leader sends a request
to each peer and shows the results it gets. If some peers don't answer within 1
second, stats for those peers won't be shown.

The new output can be seen here: P27

Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1907
2019-03-07 15:00:40 +01:00
Teon Banek
5084123de3 Add regex matching to openCypher
Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D1880
2019-02-27 12:54:50 +01:00
Ivan Paljak
ed28ed873d Remove Kafka, auth and audit log from Memgraph HA
Summary: Removing Kafka, auth and audit log features from HA instance for the time being.

Reviewers: mferencevic, msantl

Reviewed By: mferencevic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1882
2019-02-27 10:25:26 +01:00
Vinko Kasljevic
bbb69a1c00 Edges iterator implementation
Summary:
`EdgesIterable` is used for iterating over Edges in distributed memgraph. Because of lru cache there is a possibility of data getting evicted as someone iterates over it.
To prevent that `EdgesIterable` will lock that data and release it when it's deconstructed.

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1868
2019-02-26 09:44:22 +01:00
Teon Banek
4d1d9fb15a Add syntax for managing data constraints
Reviewers: mtomic, mferencevic, buda, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1879
2019-02-25 14:31:29 +01:00
Teon Banek
6bba5f4cd0 Add clauses for showing DB information
Summary:
Also add STATS Privilege and Permission.
Update tests and changelog.

Reviewers: mtomic, mferencevic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1873
2019-02-19 13:32:43 +01:00
Vinko Kasljevic
127a67ab13 Remove DistributedGraphDb
Summary: DistributedGraphDb is no longer needed.

Reviewers: msantl, teon.banek, ipaljak

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1857
2019-02-14 09:38:32 +01:00
Matej Ferencevic
bb052be002 Remove serialization from utils
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1852
2019-02-13 15:41:40 +01:00
Matija Santl
1b11e109fa Add storageInfo awesome memgraph function
Summary:
Added a new awesome function called `storageInfo`. This function
returns a list of key-value pairs containing the following (useful?)
information:
 * number of vertices
 * number of edges
 * average degree
 * memory usage
 * disk usage

The current implementation is in `awesome_memgraph_functions` but it will end up
as a separate clause for better access control.

Reviewers: teon.banek, mtomic, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D1850
2019-02-11 16:25:01 +01:00
Teon Banek
e461a08340 Move creating indexed lookup to a rewrite pass
Summary:
RuleBasedPlanner now generates only the regular ScanAll operations, and
Filter operations are appended as soon as possible. The newly added
Rewrite step, takes this operator tree and replaces viable Filter &
ScanAll operators with appropriate ScanAllBy<Index> operator. This
change ought to simplify the behaviour of DistributedPlanner when that
stage is moved before the indexed lookup rewrite.

Showing unoptimized plan in interactive planner is also supported.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1839
2019-02-06 14:46:19 +01:00
Teon Banek
8a508a1cb6 Remove unused helper closure variable
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1845
2019-02-06 14:06:52 +01:00
Marin Tomic
c03ca5f8f0 Remove UID tracking from AstStorage
Summary:
All AST nodes had a member `uid_` that was used as a key in
`SymbolTable`. It is renamed to `symbol_pos_` and it appears only in
`Identifier`, `NamedExpression` and `Aggregation`, since only those types were
used in `SymbolTable`. SymbolGenerator is now responsible for creating symbols
in `SymbolTable` and assigning positions to AST nodes.

Cloning and serialization code is now simpler since there is no need to track
UIDs.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1836
2019-02-05 14:19:48 +01:00
Marin Tomic
7542f5b0ba Include JSON serialized logical plan in summary for EXPLAIN
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1831
2019-01-25 17:01:17 +01:00
Teon Banek
a871212675 Replace dynamic_casts in query with utils::Downcast
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1830
2019-01-25 15:31:14 +01:00
Teon Banek
f91428f23f Generate virtual GetTypeInfo with LCP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1827
2019-01-25 14:12:50 +01:00
Teon Banek
743d82b78d Use plan rewrite hook to generate DistributedPlan
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1826
2019-01-23 10:48:03 +01:00
Teon Banek
abc9f0b0e7 Add hooks for post processing generated plans
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1824
2019-01-22 16:45:05 +01:00
Marin Tomic
9d36f775f2 Remove GraphView from variable expansion operators
Summary:
Variable expansions cannot appear in merge patterns or after updates,
so they can only be planned with GraphState::OLD. Because of that, it makes
sense to remove GraphView parameter from them to reduce confusion.

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1825
2019-01-22 15:35:05 +01:00
Marin Tomic
914f40411a Implement plan cloning with LCP
Summary:
Implement proper plan cloning using LCP instead of hacking it with
serialization.

depends on D1815

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1816
2019-01-22 14:52:06 +01:00
Marin Tomic
f9c63987c6 Clone AST using LCP
Summary:
use newly added LCP functionality to get rid of manually written
`Clone` functions in AST.

depends on D1808

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1815
2019-01-18 13:24:40 +01:00
Marin Tomic
903263d896 Fix line comment stripping in query stripper
Summary:
When there was an empty line comment starting right at the end of the
query, stripper wouldn't properly change state from `IN_LINE_COMMENT` to `OUT`
and it would return the wrong length (0) from `MatchWhitespaceAndComments`.
Because of that, the two slashes would be interpreted as division operators.

Query "RETURN 5;//" would be changed by stripper into "RETURN 5; / /" which
obviously can't be parsed;

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1819
2019-01-18 13:24:02 +01:00
Lovro Lugovic
c86cf57171 Fix IndexInMulticommandTxException error message
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1817
2019-01-17 13:58:40 +01:00
Lovro Lugovic
5f13b92664 Output profiling data within the summary in JSON format
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1801
2019-01-17 13:54:06 +01:00
Teon Banek
d7422a16d2 Cleanup query/context.hpp
Summary:
Rename Context to ExecutionContext and make it struct
Move ParsingContext to cypher_main_visitor.hpp

Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1810
2019-01-17 10:21:34 +01:00
Lovro Lugovic
12c8b3f75f Properly handle the caching of Cypher queries within metaqueries
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1800
2019-01-17 09:25:54 +01:00
Lovro Lugovic
2133895db1 Fix pretty printing bug
Summary:
Identifiers would be printed correctly if they were printed as part of a larger
`Expression` (so the visitor did the printing). However, if `PrintObject` was
called with an `Identifier *` then it would delegate to the template overload
and just print the pointer.

Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1802
2019-01-17 09:15:49 +01:00
Teon Banek
b90375c3ae Remove GraphDbAccessor and storage types from Ast
Summary:
This diff removes the need for a database when parsing a query and
creating an Ast. Instead of storing storage::{Label,Property,EdgeType}
in Ast nodes, we store the name and an index into all of the names. This
allows for easy creation of a map from {Label,Property,EdgeType} index
into the concrete storage type. Obviously, this comes with a performance
penalty during execution, but it should be minor. The upside is that the
query/frontend minimally depends on storage (PropertyValue), which makes
writing tests easier as well as running them a lot faster (there is no
database setup). This is most noticeable in the ast_serialization test
which took a long time due to start up of a distributed database.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1774
2019-01-16 09:47:42 +01:00
Teon Banek
150331e701 Remove unused variables
Reviewers: mferencevic, llugovic

Reviewed By: mferencevic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1803
2019-01-15 15:30:34 +01:00
Lovro Lugovic
2730f2d35f Add query profiling: Tree
Reviewers: mtomic, teon.banek, mferencevic

Reviewed By: mtomic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1772
2019-01-15 12:13:40 +01:00
Marin Tomic
8c7b87d8b0 Cleanup BFS subcursors using transactional cache clean-up
Summary:
Before I wrongly assumed `Shutdown` will always be called on Cursors and
removed BFS subcursors there. Now it is done using transactional cache clean-up
mechanism. There's a separate clean-up for `BfsSubcursorStorage` (for actual
subcursors) and `BfsRpcServer` (for database accessors created by the server).

I've changed `BfsRpcServer` to have a `GraphDbAccessor` per transaction,
instead of per `BfsSubcursor`. Mainly because there is no reliable way to check
if the transaction tied to the accessor has expired as it is not safe to call
`transaction_id` method (since `GraphDbAccessor` is holding only a reference to
`Transaction` object).

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1792
2019-01-14 17:31:20 +01:00
Matija Santl
f53913e053 Add automated test for Raft
Summary:
Created a new integration test for Raft protocol.

The tests iterates through the Raft cluster and does the following:
* kill machine `X`
* execute a query
* bring `X` back to life

The first step is to insert a vertex in the cluster, and last step is to check
if the cluster has all the data.

I also edited some of the raft core files because this test surafaced some bugs.

The `tester` binary is a hacked version of the HA client and so are the parts in
the code that refuse to execute a query is the machine is not in `Leader` mode.o
Those parts will go away once we have a proper HA client.

I've run the `runner.py` for a while (215 times)
```
while ./runner.py &> log.txt; do echo -n "."; done
```
and it didn't break.

Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1788
2019-01-14 13:41:36 +01:00
Marin Petricevic
664622f68e add 'sample' awesome memgraph function
Summary: This simple function is required by the Tensorflow integration so that Memgraph can always return regular matrices of desired size.

Reviewers: teon.banek, mtomic, dsantl

Reviewed By: mtomic, dsantl

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1783
2019-01-14 13:23:16 +01:00
Teon Banek
a6b296c73c Pretty print edge_types for Expand operators
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1791
2019-01-10 13:18:02 +01:00
Matej Ferencevic
3209788cd4 Implement new spin lock
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1786
2019-01-08 09:15:07 +01:00
Teon Banek
72ee3fda85 Replace NodeAtom and EdgeAtom with CreationInfo
Summary:
This (almost) removes the dependency of operators on NodeAtom and
EdgeAtom. Only EdgeAtom::Direction is needed. The change was done as the
initial step of removing dependency on storage from Ast. Additionally,
it makes sense for LogicalOperator to only depend on Expression classes.

Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1779
2018-12-20 13:06:33 +01:00
Matija Santl
2b3d19e508 Reset TransactionEngine internal state on Abort
Summary:
With neo4j java driver 1.7.0. they don't send `ROLLBACK`. This causes
unexpected nested transaction errors. This diff should fix that.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1780
2018-12-19 16:34:33 +01:00
Teon Banek
b1e21489e9 Serialize logical operators using SLK
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1770
2018-12-13 10:02:20 +01:00
Teon Banek
8ed1e3509d Serialize Ast via SLK
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1764
2018-12-12 14:40:32 +01:00
Teon Banek
26d23959d3 Serialize C++ part of query folder via SLK
Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1761
2018-12-10 14:29:28 +01:00
Matej Ferencevic
e70ac03607 Remove inheritance from storage types
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1757
2018-12-07 11:03:22 +01:00
Teon Banek
7638b09867 Generate SLK serialization from LCP
Summary:
Classes marked with `:serialize (:slk)` will now generate SLK
serialization code. This diff also changes how the `:serialize` option
is parsed, so that multiple different serialization backends are
supported.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1755
2018-12-05 14:58:39 +01:00
Matej Ferencevic
809e1779b1 Fix BFS subcursor cleanup
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1748
2018-11-30 22:02:38 +01:00
Teon Banek
7d01ba5178 Use generic :dont-save mechanism in LCP
Summary:
It doesn't make sense to tie skipping a member for serialization to a
particular serialization implementation.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1743
2018-11-22 10:01:38 +01:00
Teon Banek
14c9044909 Convert symbol.hpp to LCP
Summary:
With quite frequent changes of serialization backend, it's getting
really painful updating the C++ implementation. This diff defines the
Symbol class via LCP, so that the C++ serialization is generated instead
of written by hand.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1741
2018-11-22 09:52:18 +01:00
Teon Banek
58b450a2ea Remove template from ExpandUniquenessFilter
Summary:
Instantiation with VertexAccessor was never used, so the template
needlessly complicated the rest of the codebase.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1736
2018-11-19 13:03:39 +01:00
Teon Banek
8ed1fbbbe1 Generate TypeInfo object for all LCP defined classes
Summary:
TypeInfo will be needed for upcoming serialization via SLK. This diff
changes the already defined TypeInfo by removing the reliance on
capnp::typeId calls. The struct itself is now in utils.

Hopefully, this shouldn't break our RPC stack due to new ID generation.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1735
2018-11-16 13:27:58 +01:00
Teon Banek
a272fa2e6b Add basic support for saving classes with SLK
Summary:
This is the initial work on transferring our serialization code from
Cap'n Proto to SaveLoadKit. The commit contains tests for generated
code, but still requires work on supporting some features. Most notably,
generating and storing type IDs for derived classes so that they can be
loaded from a base pointer. Naturally, loading implementation hasn't
even been started yet.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1733
2018-11-15 13:26:38 +01:00
Teon Banek
5df4d55ec1 Extract QueryVisitor from HierarchicalTreeVisitor
Summary:
This change makes HierarchicalTreeVisitor visit only Cypher related AST
nodes. QueryVisitor can be used to differentiate between various query
types we have. The next step is to either rename HierarchicalTreeVisitor
to something like CypherQueryVisitor, or perhaps extract Clause visiting
from it.

Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1710
2018-11-12 12:53:25 +01:00
Lovro Lugovic
06d4568950 Add ExpressionPrettyPrinter
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1719
2018-11-07 13:06:14 +01:00
Vinko Kasljevic
7ba8228c46 Refactor storage file structure
Summary:
- Create types folder in storage/common
- Move locking and kvstore to storage/common
- Add storage/distributed/rpc folder

Reviewers: teon.banek, ipaljak, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1713
2018-11-06 18:17:31 +01:00
Teon Banek
218384786d Replace TreeVisitor with ExpressionVisitor
Summary:
This is the initial step in making the visitors more granular in what
they visit.

Also add ignoring multiple inheritance in LCP and update LCP docs.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1704
2018-10-30 09:24:45 +01:00
Marin Tomic
50b2646765 Compose ExpandCommon instead of inheriting from it
Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1703
2018-10-29 16:51:52 +01:00
Lovro Lugovic
d9db38dca6 Implement hierarchical Accept for AuthQuery and StreamQuery
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1708
2018-10-29 13:57:37 +01:00
Lovro Lugovic
06ae2ffecc Remove the Coalesce function
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1676
2018-10-29 11:37:20 +01:00
Teon Banek
66dcd536f0 Support serializing LCP enum vector
Summary:
Generate conversion functions from C++ enum to Cap'n Proto and vice
verse. This should reduce the generated code size and hopefully improve
readability.

Use conversion functions for enums in default serialization generation.
`capnp-save-enum` and `capnp-load-enum` should now be used exclusively
to serialize non LCP defined enumerations. They now always require
`enum-values` argument.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1696
2018-10-26 16:05:47 +02:00
Matija Santl
f6f58d843b Drop index
Reviewers: teon.banek, mferencevic, vkasljevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1659
2018-10-26 13:51:22 +02:00
Marin Tomic
1f07879489 Make explain check case insensitive
Summary: I wrongly assumed that stripper lowercases keywords.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1697
2018-10-26 10:45:36 +02:00
Marin Tomic
e4b661fe4a Cache required privileges for query execution
Summary: this should reduce parsing time for very simple queries.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1698
2018-10-26 10:40:13 +02:00
Marin Tomic
4f2a69b973 Remove debug logging from stream handling in interpreter
Reviewers: msantl, teon.banek

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1699
2018-10-26 10:39:04 +02:00
Matija Santl
800db5058e Add blocking transactions for index creation
Summary:
Blocking transaction has the ability to stop the transaction engine from
starting new transactions (regular or blocking) and to wait all other active
transactions to finish (to become non active, committed or aborted). One thing
that blocking transactions support is defining the parent transaction which
does not need to end in order for the blocking one to start. This is because of
a use case where we start nested transactions.

One could thing we should build indexes inside those blocking transactions. This
is true and I wanted to implement this, but this would require some digging in
the interpreter which I didn't want to do in this change.

Reviewers: mferencevic, vkasljevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1695
2018-10-24 16:31:50 +02:00
Teon Banek
910fc7c4d1 Add DROP INDEX to Cypher
Reviewers: mtomic, llugovic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1690
2018-10-23 09:17:44 +02:00
Marin Tomic
eee8b57daf Separate query types in AST and interpreter
Summary:
`Query` is now an abstract class which has `CypherQuery`,
`ExplainQuery`, `IndexQuery`, `AuthQuery` and `StreamQuery` as derived
classes. Only `CypherQuery` is forwarded to planner and the rest of the
queries are handled directly in the interpreter. This enabled us to
remove auth, explain and stream operators, clean up `Context` class and
remove coupling between `Results` class and plan cache. This should make
it easier to add similar functionality because no logical operator
boilerplate is needed. It should also be easier to separate community
and enterprise features for open source.

Remove Explain logical operator
Separate IndexQuery in AST
Handle index creation in interpreter
Remove CreateIndex operator and ast nodes
Remove plan cache reference from Results
Move auth queries out of operator tree
Remove auth from context
Fix tests, separate stream queries
Remove in_explicit_transaction and streams from context

Reviewers: teon.banek, mferencevic, msantl

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1664
2018-10-22 16:43:42 +02:00
Lovro Lugovic
60ec94fb30 Fix total_weight not being returned with RETURN *
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1665
2018-10-18 15:03:46 +02:00
Lovro Lugovic
76504f7b73 Fix anonymous variable capture
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1673
2018-10-18 14:58:12 +02:00
Teon Banek
0829b2bb90 Copy accumulated elements in Synchronize
Summary:
The row elements must not be moved, because the following pulls may
still use them.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1672
2018-10-18 10:03:25 +02:00
Matija Santl
fb6284cb99 Add unique constraint to label property index
Summary:
LabelPropertyIndex now has the ability to enforce unique constraint.

This doesn't lock the tx engine.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, vkasljevic, buda

Differential Revision: https://phabricator.memgraph.io/D1660
2018-10-16 14:20:47 +02:00
Marin Tomic
285e02d5ec Remove root tracking from AST storage
Summary: Up till now, `AstStorage` also took care of tracking the root of the `Query` and loading of cloning of `Query` nodes would change that root. This felt out of place because sometimes `AstStorage` is used only for storing expressions, and we don't even have an entire query in the storage. This diff removes that feature from `AstStorage`. Now its only functionality is owning AST nodes and assigning unique IDs to them.

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1646
2018-10-16 10:22:21 +02:00
Teon Banek
1b6a6c15e5 Add syntax for creating unique indexes
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1653
2018-10-12 12:50:58 +02:00
Matej Ferencevic
b527b2b4e4 Refactor PropertyValue
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1654
2018-10-12 11:10:29 +02:00
Lovro Lugovic
4b5c0d3426 Implement coalesce as a special operator
Reviewers: teon.banek, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1652
2018-10-11 16:25:26 +02:00
Matija Santl
cbe100ef5a Fix Edges filter by vertex
Summary:
This diff fixes the storage part of the bug that happen for the
following queries if all nodes end up on the non-master machine.
```
CREATE (a:T{c: True})-[:X{x: 2.5}]->(:A:B)-[:Y]->()-[:Z{r: 1}]->(a)

MATCH (:T{c: True})-[a:X{x: 2.5}]->(node:A:B)-[:Y]->()-[:Z{r: 1}]->() RETURN a AS node, node AS a
```

The current state of the code doesn't work completely, it's still missing the
query execution fix.

Reviewers: teon.banek, ipaljak, vkasljevic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1629
2018-10-11 15:29:31 +02:00
Lovro Lugovic
467e46c302 Add inDegree and outDegree functions
Reviewers: teon.banek, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1651
2018-10-11 15:05:39 +02:00
Matej Ferencevic
6525451489 Remove distributed logic from single node
Summary:
Start removing `is_remote` from `Address`
Remove `GlobalAddress`
Remove `GlobalizedAddress`
Remove bitmasks from `Address`
Remove `is_local` from `Address`
Remove `is_local` from `RecordAccessor`
Remove `worker_id` from `Address`
Remove `worker_id` from `GidGenerator`
Unfriend `IndexRpcServer` from `Storage`
Remove `LocalizedAddressIfPossible`
Make member private
Remove `worker_id` from `Storage`
Copy function to ease removal of distributed logic
Remove `worker_id` from `WriteAheadLog`
Remove `worker_id` from `GraphDb`
Remove `worker_id` from durability
Remove nonexistant function
Remove `gid` from `Address`
Remove usage of `Address`
Remove `Address`
Remove `VertexAddress` and `EdgeAddress`
Fix Id test
Remove `cypher_id` from `VersionList`
Remove `cypher_id` from durability
Remove `cypher_id` member from `VersionList`
Remove `cypher_id` from database
Fix recovery (revert D1142)
Remove unnecessary functions from `GraphDbAccessor`
Revert `InsertEdge` implementation to the way it was in/before D1142
Remove leftover `VertexAddress` from `Edge`
Remove `PostCreateIndex` and `PopulateIndexFromBuildIndex`
Split durability paths into single node and distributed
Fix `TransactionIdFromWalFilename` implementation
Fix tests
Remove `cypher_id` from `snapshooter` and `durability` test

Reviewers: msantl, teon.banek

Reviewed By: msantl

Subscribers: msantl, pullbot

Differential Revision: https://phabricator.memgraph.io/D1647
2018-10-11 13:08:22 +02:00
Marin Tomic
695129f401 Separate query types in openCypher grammars
Summary: depends on D1640

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1643
2018-10-10 15:08:48 +02:00
Teon Banek
ca2106bb91 Handle transitive dependencies for Cartesian
Reviewers: mtomic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1641
2018-10-10 12:55:05 +02:00
Marin Tomic
ce6085fa06 Fix pointer tracking in AST serialization
Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1640
2018-10-10 11:24:39 +02:00
Lovro Lugovic
5bdb04a693 Add move constructor and assignment operator to TypedValue
Summary:
`heaptrack` shows a miniscule decrease in memory usage during query
execution.

Running the below query on the TEDTalk database 100 times gives the
following results:

- number of allocations: from 642647 to 642589
- bytes allocated in total: from 48.79 MiB to 48.78 MiB

```
MATCH (t:Tag)<-[:HasTag]-(n:Talk)
RETURN t.name AS Tag, COUNT(n) AS TalksCount
ORDER BY TalksCount DESC, Tag LIMIT 20;
```

Regarding `TypedValue`'s destructor: we're allowed to manually destruct the
union memebers that we constructed using placement-new. However, it is undefined
behavior to call the destructor after an object's lifetime has ended. Calling
`TypedValue`'s own destructor within its assignment operator counts as ending
its lifetime, which means that the next call to its destructor will invoke
undefined behavior.

See the C++ Draft N4140, clauses 12.4/15 and 3.8/1.3.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1630
2018-10-09 14:49:09 +02:00
Matej Ferencevic
ade2593b51 Separate distributed from single node GraphDb
Summary:
To clean the working directory after this diff you should execute:

```
rm src/database/counters_rpc_messages.capnp
rm src/database/counters_rpc_messages.hpp
rm src/database/serialization.capnp
rm src/database/serialization.hpp
```

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1636
2018-10-05 13:32:05 +02:00
Marin Tomic
343fe54f2c Move 'explain' to Cypher.g4 and add it to keyword list
Summary: Move explain to Cypher.g4 so MemgraphCypher contains only enterprise features. Also added `EXPLAIN` to keyword list because query `MATCH (explain) RETURN explain` wasn't working.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1632
2018-10-05 10:09:50 +02:00