Commit Graph

2486 Commits

Author SHA1 Message Date
Matej Ferencevic
0833a74fd8 Refactor storage delta in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2171
2019-07-03 15:22:38 +02:00
Matej Ferencevic
db72ef05f8 Add support for vertex properties to storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2163
2019-07-03 11:07:57 +02:00
Matija Santl
542d65544b Fix replication log usage in HA
Summary:
Instead of calling the wanted method, this diff takes copies of both `is_replicated(x)` and `is_active(x)` results so that they can't change during the method execution.

The problem with the previous implementation was that the information about a
log could change during two consecutive queries as ilustrated in the example
below.
```
      Thread 1                                  Thread2
rlog->set_active(1);
if (rlog->is_replicated(1)) return true;
                                          rlog->set_replicated(1);
if (rlog->is_active(1)) return false;
throw InvalidLogReplicationLookup();
```

The snippet above would throw as we don't have any information about the log
`1`, because the `set_replicated` call would "shadow" the active bit.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D2165
2019-07-03 09:38:40 +02:00
Matej Ferencevic
83a99aaf7d Implement vertex deletion in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2157
2019-07-01 16:54:34 +02:00
Matej Ferencevic
9e649794fc Rework delta creation in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2159
2019-07-01 15:47:43 +02:00
Marin Tomic
c7a8be31cb [StorageV2] Fix move constructor for delta
Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D2158
2019-07-01 14:43:15 +02:00
Marin Tomic
71c76cfb73 [StorageV2] Make delta list double linked
Summary: this is necessary for garbage collection

Reviewers: teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2156
2019-07-01 14:16:20 +02:00
Marin Tomic
a7fabe6109 Fix PrepareForWrite function in storage V2
Summary:
We forgot to update `modified_vertices` in the case when the vertex
has an empty version chain. It didn't manifest before because it was impossible
for a vertex to have an empty version chain without garbage collection.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2154
2019-07-01 13:02:51 +02:00
Matej Ferencevic
f2b23828ab Implement PropertyValue for storage v2
Reviewers: teon.banek, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2149
2019-07-01 10:56:33 +02:00
Marin Tomic
fabfa77bd9 Fix CreateVertex function in storage v2
Summary:
We forgot to add the newly created vertex into `modified_vertices`
list.

Reviewers: teon.banek, mferencevic

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D2153
2019-06-28 15:43:57 +02:00
Matej Ferencevic
8169513f57 Remove unnecessary constructors from storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2152
2019-06-28 14:43:35 +02:00
Matej Ferencevic
cf9bb1f6e2 Remove the need for a default constructor when using the SkipList
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2151
2019-06-28 13:06:59 +02:00
Marin Tomic
6ce8fae54a Implement commit log for storage v2
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2148
2019-06-27 13:52:51 +02:00
Matej Ferencevic
54e1e4e1ff Initial version of storage v2
Summary:
Initial implementation of new storage engine.  It implements snapshot isolation
for transactions.  All changes in the database are stored as deltas instead of
making full copies.  Currently, the storage supports full transaction
functionality (commit, abort, command advancement).  Also, support has been
implemented only for vertices that have only labels.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2138
2019-06-27 10:52:38 +02:00
Matej Ferencevic
70c14b2048 Fix utils::SkipList atomic memory order
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2120
2019-06-26 14:18:13 +02:00
Teon Banek
775930ef4e Use stack for initial memory of MonotonicBufferResource in Pull
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2139
2019-06-26 10:26:53 +02:00
Teon Banek
e9dcb1dcf2 Use a different allocator for per-pull allocations
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2137
2019-06-26 10:26:53 +02:00
Teon Banek
5adb4aafe0 Use execution memory for Frame allocations
Summary:
Micro benchmarks show some minor variations compared to the previous
commit. Smaller cases are a bit worse while larger data cases are a bit
better.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2136
2019-06-24 15:05:44 +02:00
Teon Banek
6086257204 Use per_pull and whole execution allocators in Cursors
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2134
2019-06-24 15:05:43 +02:00
Tonko Sabolcec
1253319710 Bugfix: use indices when dumping edges
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2142
2019-06-21 16:06:23 +02:00
Tonko Sabolcec
7edf3e6935 Switch dump executable to libmgclient
Summary:
Dump executable used to use C++ bolt api for connecting to the server.
However, this is currently not efficient enough because it fetches entire data at once.
We switched to C api libmgclient which supports streaming of data.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2129
2019-06-19 12:28:12 +02:00
Tonko Sabolcec
804d123a6b Dump double properties with full precision
Summary: Sets full precision to avoid precision loss when dumping double values.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2125
2019-06-19 11:44:05 +02:00
Teon Banek
65ab2574bc Use memory from EvaluationContext in awesome functions
Reviewers: mferencevic, msantl, mtomic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2133
2019-06-18 14:13:23 +02:00
Teon Banek
3fd14e2d5f Use custom allocator in Evaluator through context
Summary:
Micro benchmarks show improvements in performance of MapLiteral from 5%
to 40% depending on the size of the input. On the other hand, a sequence
of AdditionOperators behaves the same with both allocation schemes.

Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2132
2019-06-17 10:53:25 +02:00
Teon Banek
57d967786c Make TypedValue constructor explicit for non-primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2128
2019-06-12 14:55:58 +02:00
Teon Banek
b3bc4d6809 Make TypedValue constructor explicit for primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2127
2019-06-12 14:55:57 +02:00
Teon Banek
6d3e3ac4aa Remove TypedValue::Null
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2121
2019-06-12 09:55:18 +02:00
Teon Banek
df90184b50 Use utils::Allocator in query::Path
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2117
2019-06-12 09:55:18 +02:00
Teon Banek
96505b7fb4 Use Allocator for map in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2116
2019-06-12 09:55:17 +02:00
Lovro Lugovic
0b2240e6e4 LCP: Split up PROCESS-FILE
Summary:
Split up `process-file` so that looking at the generated code for an LCP form is
easier from the REPL.

`process-lcp`, `generate-hpp` and `generate-cpp` now perform the generation of
C++ code, but take a list of "C++ elements" (the results of LCP forms) as input
and write their output to streams. They do no reading/evaluating of LCP forms of
their own.

`read-lcp` and `read-lcp-file` are used to read and evaluate a stream of LCP
forms. The latter is a specialized version for file streams which also reports
the position of the form within the file when an error happens.

`process-lcp-string` and `process-lcp-file` are convenient wrappers around the
main functionality that take a string (file) and output to strings (files).
Using `read-lcp` and `read-lcp-file` they process LCP forms and pass them off to
`process-lcp` for code generation.

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2097
2019-06-12 09:38:41 +02:00
Lovro Lugovic
21d05e9360 LCP: Add resolution for SLK-OPTS and CLONE-OPTS
Summary: Depends on D2094

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2095
2019-06-12 09:38:05 +02:00
Lovro Lugovic
dbd226d05d LCP: Minor improvements
Summary:
- Add a unified `generate_lcp` target
- Simplify SLK-ERROR and CLONE-ERROR
- Rework WITH-VARS
- Expect CPP-CLASS, not CPP-TYPE
- Fix CPP-TYPE-REFERENCE-P
- Add CPP-GENSYM
- Add util.lisp to LCP's source files in CMake
- Make the CMake variable `lcp_src_files` a cache (persistent) variable
- Add `lcp_src_files` as a dependency to `test_lcp`
- Rename `lcp_compile` to `compile-lcp`
- Fetch docstrings for CPP-NAME-* functions at run-time
- Remove existing C++ entities on redefinition

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2094
2019-06-12 09:38:02 +02:00
Lovro Lugovic
0732264fd5 LCP: Make :PUBLIC, :PROTECTED and :PRIVATE options aggregating
Summary: Depends on D2092

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2093
2019-06-12 09:37:59 +02:00
Lovro Lugovic
2736f31dcf LCP: Rework DEFINE-CLASS and DEFINE-ENUM
Summary: Depends on D2091

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2092
2019-06-12 09:37:55 +02:00
Lovro Lugovic
b8514ccf04 LCP: Update documentation
Summary: Depends on D2090

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2091
2019-06-12 09:37:52 +02:00
Lovro Lugovic
59af45f94e LCP: Fix up LCP warnings
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2090
2019-06-12 09:37:47 +02:00
Matej Ferencevic
18698ac7e6 Disable distributed test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2126
2019-06-11 12:17:12 +02:00
Matej Ferencevic
1e3b0ea727 Setup mgclient library
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2124
2019-06-11 10:12:52 +02:00
Tonko Sabolcec
eb45ed09df Dump cleanup
Summary:
`dump()` awesome Memgraph function is removed.
`mg_dump` executable now uses "dump database" command instead of "return dump()"
and dumps database to standard output instead of a file.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2122
2019-06-10 17:01:31 +02:00
Tonko Sabolcec
485592eb48 Add support for dumping unqiue constraints
Summary: Dumps unique constraints, tests included

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2112
2019-06-10 16:50:46 +02:00
Matej Ferencevic
8194fbd6f7 Fix utils::SkipList iterator bug
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2119
2019-06-10 15:32:47 +02:00
Tonko Sabolcec
9cc2224ac3 Handle dump database query
Summary:
Stream queries to the output table.
For effective output streaming, a simple operator, `OutputTableStream` is implemented which fetches and
produces a single row on each Pull.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2099
2019-06-10 13:22:14 +02:00
Teon Banek
fa62ea1920 Use ValueMap instead of Value<>
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2114
2019-06-10 10:35:04 +02:00
Teon Banek
6bb72d14a1 Use Allocator for string in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2104
2019-06-10 10:35:04 +02:00
Teon Banek
16555eeb9a Add special construct for std::pair in Allocator
Summary:
This is unfortunately needed in the C++17 standard, so that the
allocator is correctly propagated to elements of pair which respect the
"Uses Allocator" protocol. C++20 standard resolves this issue, but we
still have a long way before it is released and implemented by the
compiler and standard library vendors.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2107
2019-06-06 13:44:56 +02:00
Ivan Paljak
716a80edf0 Include HA client in index integration test
Summary:
The test now uses `ha_client`. Logging is also modified to output 1-indexed
worker ids.

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2113
2019-06-05 16:07:30 +02:00
Matija Santl
c9ba638b9f Fix includes for constraints
Summary: Forward declare the needed struct and include in the correct file.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2110
2019-06-04 16:47:23 +02:00
Ivan Paljak
9d932797ab Include HA client in HA basic integration test
Summary:
- Included HA client
- Fixed log messages to be 1-indexed
- Added id properties to created nodes for easier debugging
- Create and check steps are now executed 20 times each

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2111
2019-06-04 16:12:51 +02:00
Ivan Paljak
45413c0612 Improve large log entries integration test
Summary:
- Server ids are now 1-indexed in logs
- All created nodes have distinct is properties which helps with debugging

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2109
2019-06-04 15:21:05 +02:00
Ivan Paljak
c775688aa7 Add HA integration test for leader election
Summary:
This tests checks the correctness of a leader election process when its
decoupled from log replication. In other words, in this test we do not change
the state of the database, i. e., the Raft log remains empty.

The test proceeds as follows for clusters of size 3 and 5:
  1. Start a random subset of workers in the cluster
  2. Check if the leader has been elected
  3. Kill all living workers
  4. GOTO 1 and repeat 10 times

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2105
2019-06-04 12:48:35 +02:00