Commit Graph

1261 Commits

Author SHA1 Message Date
Marin Tomic
e7f897984f Fix wrong settings in gc bench
Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2207
2019-07-15 17:53:03 +02:00
Teon Banek
bf4f69f2e5 storage/v2: Reduce the (In|Out)Edges tuple to only EdgeAccessor
Summary:
The first 2 tuple elements are redundant as they are available through
EdgeAccessor and they needlessly complicate the usage of the API.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2200
2019-07-12 15:30:24 +02:00
Teon Banek
c4c6febbc4 Move Parsing utils from query/common to frontend/parsing
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2195
2019-07-11 12:12:59 +02:00
Marin Tomic
8414479abe [StorageV2] Implement GC
Summary:
Here are some numbers from the benchmark:

```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 8
Config: NoGc, Time: 25.9836
Config: OnFinishGc, Time: 49.012
Config: 100msPeriodicGc, Time: 45.9856
Config: 1000msPeriodicGc, Time: 40.3094
```

```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 7
Config: NoGc, Time: 20.4256
Config: OnFinishGc, Time: 39.6669
Config: 100msPeriodicGc, Time: 30.7956
Config: 1000msPeriodicGc, Time: 35.128
```

It is not that bad if there is a core dedicated to doing garbage collection.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2168
2019-07-10 16:12:08 +02:00
Teon Banek
5bc0be50c5 Remove GraphDbAccessor parameter from MakeCursor
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2189
2019-07-09 14:22:55 +02:00
Teon Banek
3a34d6fcdd Support all power of 2 alignments in MonotonicBufferResource
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2188
2019-07-09 14:22:05 +02:00
Ivan Paljak
2936312612 Add memgraph ha normal operation long running test
Summary:
The functionality of the test is the same as for single node Memgraph.
Locally, it seems to work fine. I'll update the apollo related files when I feel
a bit more certain that everything works locally.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2135
2019-07-09 13:59:56 +02:00
Teon Banek
76d8020169 Implement a PoolResource for allocations
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2147
2019-07-09 09:31:16 +02:00
Teon Banek
a417ef36f1 Use MemoryResource in SkipList
Summary:
This is a preparation step in case we want to have a custom allocator in
SkipList. For example, pool based allocator for SkipListNode.
Introduction of MemoryResource and removal of `calloc` has reduced the
performance a bit according to micro benchmarks. This performance hit is
not visible on benchmarks which do more concurrent operations.

Reviewers: mferencevic, mtomic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2140
2019-07-09 09:31:15 +02:00
Matej Ferencevic
b1f7bbf051 Implement properties for edges in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2187
2019-07-08 16:55:47 +02:00
Matej Ferencevic
5b8e7ff432 Implement edges in storage v2
Summary:
This change implements full edges support in storage v2. Edges can be created
and deleted. Support for detach-deleting vertices is added and regular vertex
deletion verifies existance of edges.

Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2180
2019-07-08 16:06:57 +02:00
Matej Ferencevic
a2c5b5eac6 Temporarily disable LDBC test
Reviewers: mtomic, buda

Reviewed By: buda

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2178
2019-07-05 14:04:38 +02:00
Matej Ferencevic
069000a92a Fix CommitLog namespace in storage v2
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2179
2019-07-05 13:53:32 +02:00
Matej Ferencevic
26f14443c2 Move vertex deletion to storage accessor
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2174
2019-07-04 14:47:14 +02:00
Matej Ferencevic
6e6dff81e0 Split storage and vertex accessor into cpp in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2172
2019-07-04 12:56:06 +02:00
Matej Ferencevic
9e223cdabd Use only local workers for builds on Apollo
Reviewers: mtomic, buda

Reviewed By: mtomic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2170
2019-07-03 22:30:55 +02:00
Matej Ferencevic
db72ef05f8 Add support for vertex properties to storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2163
2019-07-03 11:07:57 +02:00
Matej Ferencevic
83a99aaf7d Implement vertex deletion in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2157
2019-07-01 16:54:34 +02:00
Matej Ferencevic
f2b23828ab Implement PropertyValue for storage v2
Reviewers: teon.banek, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2149
2019-07-01 10:56:33 +02:00
Marin Tomic
6ce8fae54a Implement commit log for storage v2
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2148
2019-06-27 13:52:51 +02:00
Matej Ferencevic
54e1e4e1ff Initial version of storage v2
Summary:
Initial implementation of new storage engine.  It implements snapshot isolation
for transactions.  All changes in the database are stored as deltas instead of
making full copies.  Currently, the storage supports full transaction
functionality (commit, abort, command advancement).  Also, support has been
implemented only for vertices that have only labels.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2138
2019-06-27 10:52:38 +02:00
Teon Banek
775930ef4e Use stack for initial memory of MonotonicBufferResource in Pull
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2139
2019-06-26 10:26:53 +02:00
Teon Banek
5adb4aafe0 Use execution memory for Frame allocations
Summary:
Micro benchmarks show some minor variations compared to the previous
commit. Smaller cases are a bit worse while larger data cases are a bit
better.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2136
2019-06-24 15:05:44 +02:00
Teon Banek
6086257204 Use per_pull and whole execution allocators in Cursors
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2134
2019-06-24 15:05:43 +02:00
Tonko Sabolcec
1253319710 Bugfix: use indices when dumping edges
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2142
2019-06-21 16:06:23 +02:00
Teon Banek
3fd14e2d5f Use custom allocator in Evaluator through context
Summary:
Micro benchmarks show improvements in performance of MapLiteral from 5%
to 40% depending on the size of the input. On the other hand, a sequence
of AdditionOperators behaves the same with both allocation schemes.

Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2132
2019-06-17 10:53:25 +02:00
Teon Banek
57d967786c Make TypedValue constructor explicit for non-primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2128
2019-06-12 14:55:58 +02:00
Teon Banek
b3bc4d6809 Make TypedValue constructor explicit for primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2127
2019-06-12 14:55:57 +02:00
Teon Banek
6d3e3ac4aa Remove TypedValue::Null
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2121
2019-06-12 09:55:18 +02:00
Teon Banek
df90184b50 Use utils::Allocator in query::Path
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2117
2019-06-12 09:55:18 +02:00
Teon Banek
96505b7fb4 Use Allocator for map in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2116
2019-06-12 09:55:17 +02:00
Lovro Lugovic
dbd226d05d LCP: Minor improvements
Summary:
- Add a unified `generate_lcp` target
- Simplify SLK-ERROR and CLONE-ERROR
- Rework WITH-VARS
- Expect CPP-CLASS, not CPP-TYPE
- Fix CPP-TYPE-REFERENCE-P
- Add CPP-GENSYM
- Add util.lisp to LCP's source files in CMake
- Make the CMake variable `lcp_src_files` a cache (persistent) variable
- Add `lcp_src_files` as a dependency to `test_lcp`
- Rename `lcp_compile` to `compile-lcp`
- Fetch docstrings for CPP-NAME-* functions at run-time
- Remove existing C++ entities on redefinition

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2094
2019-06-12 09:38:02 +02:00
Matej Ferencevic
18698ac7e6 Disable distributed test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2126
2019-06-11 12:17:12 +02:00
Tonko Sabolcec
485592eb48 Add support for dumping unqiue constraints
Summary: Dumps unique constraints, tests included

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2112
2019-06-10 16:50:46 +02:00
Tonko Sabolcec
9cc2224ac3 Handle dump database query
Summary:
Stream queries to the output table.
For effective output streaming, a simple operator, `OutputTableStream` is implemented which fetches and
produces a single row on each Pull.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2099
2019-06-10 13:22:14 +02:00
Teon Banek
fa62ea1920 Use ValueMap instead of Value<>
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2114
2019-06-10 10:35:04 +02:00
Teon Banek
6bb72d14a1 Use Allocator for string in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2104
2019-06-10 10:35:04 +02:00
Teon Banek
16555eeb9a Add special construct for std::pair in Allocator
Summary:
This is unfortunately needed in the C++17 standard, so that the
allocator is correctly propagated to elements of pair which respect the
"Uses Allocator" protocol. C++20 standard resolves this issue, but we
still have a long way before it is released and implemented by the
compiler and standard library vendors.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2107
2019-06-06 13:44:56 +02:00
Ivan Paljak
716a80edf0 Include HA client in index integration test
Summary:
The test now uses `ha_client`. Logging is also modified to output 1-indexed
worker ids.

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2113
2019-06-05 16:07:30 +02:00
Ivan Paljak
9d932797ab Include HA client in HA basic integration test
Summary:
- Included HA client
- Fixed log messages to be 1-indexed
- Added id properties to created nodes for easier debugging
- Create and check steps are now executed 20 times each

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2111
2019-06-04 16:12:51 +02:00
Ivan Paljak
45413c0612 Improve large log entries integration test
Summary:
- Server ids are now 1-indexed in logs
- All created nodes have distinct is properties which helps with debugging

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2109
2019-06-04 15:21:05 +02:00
Ivan Paljak
c775688aa7 Add HA integration test for leader election
Summary:
This tests checks the correctness of a leader election process when its
decoupled from log replication. In other words, in this test we do not change
the state of the database, i. e., the Raft log remains empty.

The test proceeds as follows for clusters of size 3 and 5:
  1. Start a random subset of workers in the cluster
  2. Check if the leader has been elected
  3. Kill all living workers
  4. GOTO 1 and repeat 10 times

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2105
2019-06-04 12:48:35 +02:00
Teon Banek
07188fff7f Add utils::Substr which can use different allocator
Reviewers: mtomic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2103
2019-06-03 17:07:50 +02:00
Teon Banek
38c6625e2c utils: Support std::basic_string with allocators
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2102
2019-06-03 17:07:49 +02:00
Teon Banek
c0dc37fe34 Use ValueString instead of Value<>
Reviewers: mtomic, llugovic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2101
2019-06-03 17:07:49 +02:00
Teon Banek
bb2dbc290f Use Allocator for vector in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2086
2019-06-03 17:07:49 +02:00
Teon Banek
0152ac2dfe Add MemoryResource to TypedValue
Summary:
This diff only introduces a MemoryResource member to TypedValue and correctly
propagates through various constructors and assignments. At the moment,
MemoryResource is not used to actually allocate anything.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2085
2019-06-03 17:07:48 +02:00
Tonko Sabolcec
1bcb3cbd86 Add grammar for dump query
Summary:
Defined Antlr4 grammar for `DUMP DATABASE` query.
Defined DUMP permission in auth module.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2077
2019-06-03 10:44:18 +02:00
Ivan Paljak
24ae8a759b Update constants in large log entries test to prevent test timeouts
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2100
2019-05-29 15:40:46 +02:00
Matija Santl
87423eed06 Add constraint in HA
Summary:
HA should now support constraints in the same way the SM version does.
I only tested this thing manually, but I plan to add a new integration test for
this also.

Reviewers: ipaljak, vkasljevic, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2083
2019-05-29 14:55:04 +02:00