Summary: For example, the aggregate element produced for `COUNT(*)` has its `value` set to `NULL`.
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2463
Summary:
Switching to Storage V2 API will require passing storage::View when
serializing VertexAccessor and EdgeAccessor, so this is just the first
step in adapting the code.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2352
Summary:
This makes Gid the same as the one in storage/v2. Before they can be
merge into one implementation, we probably want to have a similar
transition for remaining ID types.
Depends on D2346
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2347
Summary:
It never made sense that a global ID is its own namespace in the storage
directory tree.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2346
Summary: The test now checks if the cluster is alive at the end of the test.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2343
Summary:
This effectively replaces the old PropertyValue implementation from the
one in storage/v2
Depends on D2333
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2335
Summary:
With a pool allocator, lookups in STL set and map are up to 50% faster.
This is probably due to contiguous memory of pooled objects, i.e. nodes
of those containers. In some cases, the lookup outperforms the SkipList.
Insertions are also faster, though not as dramatically, up to 30%. This
does make a significant difference when the STL containers are used in a
single thread as they outperform the SkipList significantly.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2326
Summary:
There's a possible race condition where we add a deleted vertex into
index and garbage collection removes it from main storage before indices are
cleaned-up.
Reviewers: mferencevic, teon.banek
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2314
Summary: Currently set to run for 2 hours.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2206
Summary:
For proper client interaction, we need to expose the (term_id, log_index)
pair for the transaction that's about to be replicated and we need to be able
to retrieve the status of a transaction defined by that pair. Transaction
status can be one of the following:
1) REPLICATED (self-explanatory)
2) WAITING (waiting for replication)
3) ABORTED (self-explanatory)
4) INVALID (received request with either invalid term_id or invalid log_index)
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2201
Summary:
The first 2 tuple elements are redundant as they are available through
EdgeAccessor and they needlessly complicate the usage of the API.
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2200
Summary:
The functionality of the test is the same as for single node Memgraph.
Locally, it seems to work fine. I'll update the apollo related files when I feel
a bit more certain that everything works locally.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2135
Summary:
This is a preparation step in case we want to have a custom allocator in
SkipList. For example, pool based allocator for SkipListNode.
Introduction of MemoryResource and removal of `calloc` has reduced the
performance a bit according to micro benchmarks. This performance hit is
not visible on benchmarks which do more concurrent operations.
Reviewers: mferencevic, mtomic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2140
Summary:
This change implements full edges support in storage v2. Edges can be created
and deleted. Support for detach-deleting vertices is added and regular vertex
deletion verifies existance of edges.
Reviewers: mtomic, teon.banek
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2180
Summary:
Initial implementation of new storage engine. It implements snapshot isolation
for transactions. All changes in the database are stored as deltas instead of
making full copies. Currently, the storage supports full transaction
functionality (commit, abort, command advancement). Also, support has been
implemented only for vertices that have only labels.
Reviewers: teon.banek, mtomic
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2138
Summary:
Micro benchmarks show some minor variations compared to the previous
commit. Smaller cases are a bit worse while larger data cases are a bit
better.
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2136
Summary:
Micro benchmarks show improvements in performance of MapLiteral from 5%
to 40% depending on the size of the input. On the other hand, a sequence
of AdditionOperators behaves the same with both allocation schemes.
Reviewers: mtomic, mferencevic, msantl
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2132
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.
Reviewers: mtomic, llugovic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2121
Summary:
Stream queries to the output table.
For effective output streaming, a simple operator, `OutputTableStream` is implemented which fetches and
produces a single row on each Pull.
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2099
Summary:
This is unfortunately needed in the C++17 standard, so that the
allocator is correctly propagated to elements of pair which respect the
"Uses Allocator" protocol. C++20 standard resolves this issue, but we
still have a long way before it is released and implemented by the
compiler and standard library vendors.
Reviewers: mtomic, llugovic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2107
Summary:
The test now uses `ha_client`. Logging is also modified to output 1-indexed
worker ids.
Reviewers: msantl
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2113
Summary:
- Included HA client
- Fixed log messages to be 1-indexed
- Added id properties to created nodes for easier debugging
- Create and check steps are now executed 20 times each
Reviewers: msantl
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2111
Summary:
- Server ids are now 1-indexed in logs
- All created nodes have distinct is properties which helps with debugging
Reviewers: msantl
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2109
Summary:
This tests checks the correctness of a leader election process when its
decoupled from log replication. In other words, in this test we do not change
the state of the database, i. e., the Raft log remains empty.
The test proceeds as follows for clusters of size 3 and 5:
1. Start a random subset of workers in the cluster
2. Check if the leader has been elected
3. Kill all living workers
4. GOTO 1 and repeat 10 times
Reviewers: msantl
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2105
Summary:
This diff only introduces a MemoryResource member to TypedValue and correctly
propagates through various constructors and assignments. At the moment,
MemoryResource is not used to actually allocate anything.
Reviewers: mtomic, llugovic, mferencevic, msantl
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2085
Summary:
HA should now support constraints in the same way the SM version does.
I only tested this thing manually, but I plan to add a new integration test for
this also.
Reviewers: ipaljak, vkasljevic, mferencevic
Reviewed By: ipaljak, mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2083
Summary:
At the moment, this test will fail. There are currently two issues with it:
- Cap'n Proto limit exception might occur
- `db.Reset()` method can hang (deadlock)
The first issue should be resolved by migration to SLK and we are currently
working on the second one. The test will land after those are fixed.
Reviewers: msantl, mferencevic
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2006
Summary:
Edge creation in tests used to happen in different
transaction than vertex creation which caused unexpected
behaviour. In this change, both vertices and edges are
created in the same transaction.
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2089
Summary:
Preparing unique constraint code to be implemented in HA. To do so, I'm
moving everything to `stograge/common/` folder. I also added a new header,
`gid.hpp` which does a `ifdef` include of the correct `gid.hpp` based on the
product.
Reviewers: ipaljak, mferencevic, vkasljevic, teon.banek
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2079
Summary:
During it's leadership, one peer can receive RPC messages from other peers that his reign is over.
The problem is when this happens during a transaction commit.
This is handled in the following way.
If we're the current leader and we want to commit a transaction, we need to make sure the Raft Log is replicated before we can tell the client that the transaction is committed.
During that wait, we can only notice that the replication takes too long, and we report that with `LOG(WARNING)` messages.
If we change the Raft mode during the wait, our Raft implementation will internally commit this transaction, but won't be able to acquire the Raft lock because the `db.Reset` has been called.
This is why there is an manual lock acquire. If we pick up that the `db.Reset` has been called, we throw an `UnexpectedLeaderChangeException` exception to the client.
Another thing with long running transactions, if someone decides to kill a `memgraph_ha` instance during the commit, the transaction will have `abort` hint set. This will cause the `src/query/operator.cpp` to throw a `HintedAbortError`. We need to catch this during the shutdown, because the `memgraph_ha` isn't dead from the user perspective, and the transaction wasn't aborted because it took too long, but we can differentiate between those two.
Reviewers: mferencevic, ipaljak
Reviewed By: mferencevic, ipaljak
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1956
Summary:
Micro benchmarks show no change compared to global new & delete. This is
to be expected, because Unwind relies only on `std::vector` which ought
to reserve the memory in reasonable chunks.
Reviewers: mtomic, llugovic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2064
Summary:
Micro benchmarks show an improvement to performance of about 10%
compared to global new & delete.
Reviewers: mtomic, llugovic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2061
Summary:
Micro benchmarks show that MonotonicBufferResource improves performance
by a factor of 1.5.
Reviewers: mtomic, mferencevic, llugovic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2048
Summary:
Benchmarks show minor improvements. Perhaps it makes sense at some later date
to use another allocator for things lasting only in a single `Pull`.
Reviewers: mferencevic, mtomic, llugovic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2018
Summary:
Unfortunately, the written micro benchmark only reports minor
improvements compared to default allocator. The results are in some
cases even a tiny bit worse.
Reviewers: mtomic, mferencevic, llugovic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2039
Summary:
This change introduces dumping of indices keys. During the dump process,
an internal label is assigned to each vertex and index on vertex's
internal property id is created for faster matching during edge creation.
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: msantl, pullbot
Differential Revision: https://phabricator.memgraph.io/D2046
Summary:
Prior to this change, a huge query was returned by DumpGenerator that
dumped the entire graph. This change split the single query to multiple
queries, each dumping a single vertex/edge. For easier vertex matching
when dumping edge, an internal property id is assigned to each vertex and
removed after the whole graph is dumped.
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2038
Summary:
According to the written benchmark, using MonotonicBufferResource yields
significant improvements to performance of Distinct. The setup fills the
database with vertices depending on the benchmark state. No edges are
created. Then we run DISTINCT on that. Since each vertex is unique, we
will store everything in the `DistinctCursor::seen_rows_`, which is
backed by a MemoryResource. This setup, on my machine, yields 10 times
better performance when run with MonotonicBufferResource.
Reviewers: mferencevic, mtomic, msantl
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1894
Summary:
There will be a lot of leftover files, execute the following commands inside
`src/` to remove them:
```
git clean -xf
rm -r rpc/ storage/single_node_ha/rpc/
```
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D2011
Summary:
This API change is needed in order to propagate the memory allocation
scheme for the execution of LogicalOperator::Cursor
Depends on D1990
Reviewers: mtomic, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1980
Summary:
The new distributed directory is inside the query, and mirrors the query
structure. This groups all of the distributed (query) source code
together, which should make the potential directory extraction easier.
Reviewers: mferencevic, llugovic, mtomic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1923
Summary:
The tests `RelationshipPatternNoDetails` and `PatternPartBraces` in
`memgraph__unit__cypher_main_visitor` checked for the names of the anonymous
identifiers and therefore implicitly relied on the order of the traversal of the
tree.
This "bug" surfaced when Memgraph was compiled with GCC (tested on >= 6.3.0).
Reviewers: mtomic, teon.banek, mferencevic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1945
Summary: New tutorial for backpacking through europe
Reviewers: dsantl, msantl, buda
Reviewed By: dsantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1954
Summary:
Test to check that the recovery works even if the snapshot is corrupted
in distributed.
Depends on D1930
Reviewers: vkasljevic, mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1950
