Matej Ferencevic
3079522f47
Make storage v2 config hierarchical
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2434
2019-09-23 16:03:53 +02:00
Matej Ferencevic
9d39ecab7b
Implement encoder/decoder for durability v2
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2393
2019-09-18 16:32:32 +02:00
Matej Ferencevic
9652f8e265
Add clear method to SkipList
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2385
2019-09-13 15:24:14 +02:00
Matej Ferencevic
55d8b98aeb
Prepare storage v2 config for more options
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2382
2019-09-13 12:42:23 +02:00
Matej Ferencevic
51fc4df87a
Add utils::InputFile wrapper
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2377
2019-09-12 16:30:11 +02:00
Teon Banek
7bd45f8714
Make query execution work with storage_v2
...
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2203
2019-09-12 10:22:00 +02:00
Matej Ferencevic
68f19df305
Migrate LDAP integration to auth module
...
Reviewers: teon.banek, buda
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2368
2019-09-11 17:15:08 +02:00
Matej Ferencevic
73379bc57e
Add functions for listing indices and constraints
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2371
2019-09-11 15:45:14 +02:00
Ivan Paljak
deee3b8ab7
Remove Raft's dependency on transaction id
...
Reviewers: buda, mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2338
2019-09-10 18:02:05 +02:00
Teon Banek
1000a5e544
Use BasicResult instead of optional for v2 Commit
...
Summary: Depends on D2365
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2366
2019-09-10 13:32:54 +02:00
Teon Banek
f02617f2bd
Replace query::GraphView with storage::View
...
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2356
2019-09-06 14:58:27 +02:00
Teon Banek
2e51ef9f66
Use storage/v2/view in glue/communication
...
Summary:
Switching to Storage V2 API will require passing storage::View when
serializing VertexAccessor and EdgeAccessor, so this is just the first
step in adapting the code.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2352
2019-09-05 10:04:44 +02:00
Teon Banek
7213bec886
Make Gid a full on class wrapper around uint64_t
...
Summary:
This makes Gid the same as the one in storage/v2. Before they can be
merge into one implementation, we probably want to have a similar
transition for remaining ID types.
Depends on D2346
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2347
2019-09-03 15:14:21 +02:00
Teon Banek
cdfaf0d4a6
Move gid::Gid to storage/common/types
...
Summary:
It never made sense that a global ID is its own namespace in the storage
directory tree.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2346
2019-09-03 14:57:44 +02:00
Ivan Paljak
58be850d5c
Fix HA log compaction test
...
Summary: The test now checks if the cluster is alive at the end of the test.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2343
2019-09-03 08:45:53 +02:00
Matej Ferencevic
60367a287e
Implement wrappers for STL containers with our allocator
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2342
2019-09-02 17:04:15 +02:00
Teon Banek
c8340ad120
Merge old and new PropertyValue implementations
...
Summary:
This effectively replaces the old PropertyValue implementation from the
one in storage/v2
Depends on D2333
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2335
2019-09-02 16:43:50 +02:00
Teon Banek
32ae2b4d23
Correctly release PoolResource memory
...
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2341
2019-09-02 15:49:57 +02:00
Teon Banek
ed802073f6
Make PropertyValue constructors explicit
...
Summary: Depends on D2332
Reviewers: ipaljak, mferencevic
Reviewed By: ipaljak
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2333
2019-09-02 12:31:53 +02:00
Matej Ferencevic
d3b141f3ba
Allocate initial execution memory
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2316
2019-09-02 11:40:38 +02:00
Teon Banek
9f4a7dcddf
Remove PropertyValue::Null
...
Reviewers: ipaljak, mferencevic
Reviewed By: ipaljak
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2332
2019-08-29 09:02:51 +02:00
Teon Banek
023538c19c
Add PoolResource benchmarking in execution and skiplist
...
Summary:
With a pool allocator, lookups in STL set and map are up to 50% faster.
This is probably due to contiguous memory of pooled objects, i.e. nodes
of those containers. In some cases, the lookup outperforms the SkipList.
Insertions are also faster, though not as dramatically, up to 30%. This
does make a significant difference when the STL containers are used in a
single thread as they outperform the SkipList significantly.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2326
2019-08-27 09:32:01 +02:00
Teon Banek
1fa9d7752c
Fix a bug in Pool::Deallocate
...
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2325
2019-08-27 09:32:00 +02:00
Marin Tomic
7e741b8d25
Implement vertex count estimation
...
Reviewers: teon.banek, mferencevic
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2306
2019-08-22 17:34:00 +02:00
Teon Banek
97b20a9384
Use non-template Value invocation
...
Reviewers: mtomic, mferencevic
Reviewed By: mtomic, mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2317
2019-08-22 16:20:31 +02:00
Marin Tomic
21d461e77d
Skip deleted vertices when building index
...
Summary:
There's a possible race condition where we add a deleted vertex into
index and garbage collection removes it from main storage before indices are
cleaned-up.
Reviewers: mferencevic, teon.banek
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2314
2019-08-22 10:25:44 +02:00
Teon Banek
4f93632fb8
storage/v2: Add (In|Out)Degree to VertexAccessor
...
Reviewers: mtomic, mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2309
2019-08-21 12:58:00 +02:00
Marin Tomic
1ab0b8e0ff
Implement existence constraints
...
Reviewers: teon.banek, mferencevic
Reviewed By: teon.banek, mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2307
2019-08-20 16:27:36 +02:00
Teon Banek
ce9e2614fa
Invert the result of SetProperty and add documentation
...
Reviewers: mtomic, mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2297
2019-08-13 16:21:33 +02:00
Matija Santl
b684e697fd
Adjust raft constants for long running tests
...
Summary:
Based on the recent builds (https://apollo.memgraph.io/runs/642045/ ,
https://apollo.memgraph.io/runs/642700/ https://apollo.memgraph.io/runs/640896/
https://apollo.memgraph.io/runs/640538/ ) `ha` stress tests fail.
Every tests that failed, has a re-election in progress. I'm bumping election
timeout constatns in order to give the cluster more time to do what it needs to
do before going in the state where it fails.
Reviewers: ipaljak
Reviewed By: ipaljak
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2292
2019-08-13 10:40:24 +02:00
Teon Banek
f7575eb112
Add utilities for type checking awesome MG functions
...
Reviewers: mtomic, msantl
Reviewed By: mtomic, msantl
Subscribers: buda, pullbot
Differential Revision: https://phabricator.memgraph.io/D2259
2019-08-12 16:24:24 +02:00
Matej Ferencevic
fb6dc83ef4
Replace bswap with proper endian functions
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2290
2019-08-07 15:45:03 +02:00
Marin Tomic
35587cddbd
[StorageV2] Implement label and label-property indices
...
Reviewers: mferencevic, teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2225
2019-07-26 12:23:18 +02:00
Teon Banek
867f30d9b3
Add VerticesIterable to storage/v2
...
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2229
2019-07-23 13:04:16 +02:00
Marin Tomic
5a3e98badd
Implement Synchronized<T> utility
...
Reviewers: mferencevic, teon.banek
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2222
2019-07-22 14:55:21 +02:00
Matej Ferencevic
d7d0963434
Implement explicit types in storage v2
...
Reviewers: teon.banek, mtomic
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2224
2019-07-22 14:16:02 +02:00
Matej Ferencevic
e9a7623288
Implement name to id mapper in storage v2
...
Reviewers: teon.banek, mtomic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2217
2019-07-19 11:24:27 +02:00
Ivan Paljak
fa4c303af6
Add HA normal operation long running test to daily build
...
Summary: Currently set to run for 2 hours.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2206
2019-07-18 16:53:07 +02:00
Matej Ferencevic
111dd8bf19
Remove distributed
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2213
2019-07-17 15:23:42 +02:00
Teon Banek
59b3f84eb9
Set [[nodiscard]] attribute on storage::Result
...
Reviewers: mtomic, mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2214
2019-07-17 15:11:29 +02:00
Matej Ferencevic
4e1c7c9e34
Rename Return to Value in storage v2 Result
...
Reviewers: mtomic, teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2199
2019-07-17 12:33:39 +02:00
Matej Ferencevic
b365037e23
Cleanup locks
...
Summary: Move RWLock and replace exceptions with `CHECK`s
Reviewers: mtomic, teon.banek
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2205
2019-07-17 10:33:44 +02:00
Ivan Paljak
4acad5795b
Expose the status of transaction within Raft
...
Summary:
For proper client interaction, we need to expose the (term_id, log_index)
pair for the transaction that's about to be replicated and we need to be able
to retrieve the status of a transaction defined by that pair. Transaction
status can be one of the following:
1) REPLICATED (self-explanatory)
2) WAITING (waiting for replication)
3) ABORTED (self-explanatory)
4) INVALID (received request with either invalid term_id or invalid log_index)
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2201
2019-07-16 13:53:16 +02:00
Matej Ferencevic
d5106c7233
Handle zero increment in counters
...
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2208
2019-07-16 12:35:35 +02:00
Marin Tomic
e7f897984f
Fix wrong settings in gc bench
...
Reviewers: mferencevic, teon.banek
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2207
2019-07-15 17:53:03 +02:00
Teon Banek
bf4f69f2e5
storage/v2: Reduce the (In|Out)Edges tuple to only EdgeAccessor
...
Summary:
The first 2 tuple elements are redundant as they are available through
EdgeAccessor and they needlessly complicate the usage of the API.
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2200
2019-07-12 15:30:24 +02:00
Teon Banek
c4c6febbc4
Move Parsing utils from query/common to frontend/parsing
...
Reviewers: mtomic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2195
2019-07-11 12:12:59 +02:00
Marin Tomic
8414479abe
[StorageV2] Implement GC
...
Summary:
Here are some numbers from the benchmark:
```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 8
Config: NoGc, Time: 25.9836
Config: OnFinishGc, Time: 49.012
Config: 100msPeriodicGc, Time: 45.9856
Config: 1000msPeriodicGc, Time: 40.3094
```
```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 7
Config: NoGc, Time: 20.4256
Config: OnFinishGc, Time: 39.6669
Config: 100msPeriodicGc, Time: 30.7956
Config: 1000msPeriodicGc, Time: 35.128
```
It is not that bad if there is a core dedicated to doing garbage collection.
Reviewers: mferencevic, teon.banek
Reviewed By: mferencevic, teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2168
2019-07-10 16:12:08 +02:00
Teon Banek
5bc0be50c5
Remove GraphDbAccessor parameter from MakeCursor
...
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2189
2019-07-09 14:22:55 +02:00
Teon Banek
3a34d6fcdd
Support all power of 2 alignments in MonotonicBufferResource
...
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2188
2019-07-09 14:22:05 +02:00
