Commit Graph

2486 Commits

Author SHA1 Message Date
Teon Banek
07188fff7f Add utils::Substr which can use different allocator
Reviewers: mtomic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2103
2019-06-03 17:07:50 +02:00
Teon Banek
38c6625e2c utils: Support std::basic_string with allocators
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2102
2019-06-03 17:07:49 +02:00
Teon Banek
c0dc37fe34 Use ValueString instead of Value<>
Reviewers: mtomic, llugovic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2101
2019-06-03 17:07:49 +02:00
Teon Banek
bb2dbc290f Use Allocator for vector in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2086
2019-06-03 17:07:49 +02:00
Teon Banek
0152ac2dfe Add MemoryResource to TypedValue
Summary:
This diff only introduces a MemoryResource member to TypedValue and correctly
propagates through various constructors and assignments. At the moment,
MemoryResource is not used to actually allocate anything.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2085
2019-06-03 17:07:48 +02:00
Tonko Sabolcec
1bcb3cbd86 Add grammar for dump query
Summary:
Defined Antlr4 grammar for `DUMP DATABASE` query.
Defined DUMP permission in auth module.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2077
2019-06-03 10:44:18 +02:00
Ivan Paljak
d6de4f4295 Exit early on request vote if already voted
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2098
2019-05-29 17:01:22 +02:00
Ivan Paljak
24ae8a759b Update constants in large log entries test to prevent test timeouts
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2100
2019-05-29 15:40:46 +02:00
Matija Santl
87423eed06 Add constraint in HA
Summary:
HA should now support constraints in the same way the SM version does.
I only tested this thing manually, but I plan to add a new integration test for
this also.

Reviewers: ipaljak, vkasljevic, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2083
2019-05-29 14:55:04 +02:00
Ivan Paljak
d7acf75a78 Add an integration test which replicates large log entries
Summary:
At the moment, this test will fail. There are currently two issues with it:

  - Cap'n Proto limit exception might occur
  - `db.Reset()` method can hang (deadlock)

The first issue should be resolved by migration to SLK and we are currently
working on the second one. The test will land after those are fixed.

Reviewers: msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2006
2019-05-29 11:00:55 +02:00
Tonko Sabolcec
d0f05c6cc5 Create simple dump executable
Summary:
Implemented a simple dump executable which runs `RETURN dump()` on the database using Bolt protocol.
The `dump()` function returns a list of openCypher queries needed for database reconstruction (note: in the future we should use more appropriate method that supports streaming of data).

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D2067
2019-05-29 10:30:41 +02:00
Ivan Paljak
c21d04ce8d Keep last raft log entry metadata in memory
Reviewers: msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2096
2019-05-28 18:33:31 +02:00
Teon Banek
f6264ab2ae Use ValueList instead of Value<>
Summary:
This change will make the transition to allocator backed TypedValue
smoother.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2084
2019-05-28 09:41:31 +02:00
Teon Banek
18cf877a47 Respect std::uses_allocator trait in utils::Allocator
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2081
2019-05-28 09:41:31 +02:00
Teon Banek
64a05bc972 Use CHECK in TypedValueVectorCompare|Equal
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2066
2019-05-28 09:41:31 +02:00
Matija Santl
e4fd49a530 Remove unique index from query execution
Summary: Remove leftover unique index code from query execution.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2078
2019-05-28 08:16:35 +02:00
Tonko Sabolcec
b57e50865e Bugfix in database dump tests
Summary:
Edge creation in tests used to happen in different
transaction than vertex creation which caused unexpected
behaviour. In this change, both vertices and edges are
created in the same transaction.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2089
2019-05-27 15:43:35 +02:00
Matej Ferencevic
42bf81021e Fix multiple raft issues
Summary:
Fix condition variable notifications
Fix vote requested invalid memory access (size off by one)
Fix blocking wait in RaftPeer while candidate
Don't copy large log entries when only the term is needed

Reviewers: ipaljak, msantl

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2088
2019-05-27 11:03:24 +02:00
dlozic
81a7a7b600 Add Raft term updates integration test
Reviewers: ipaljak, msantl, mferencevic

Reviewed By: ipaljak, msantl, mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2002
2019-05-27 10:33:40 +02:00
Teon Banek
88778fc70f Explicitly forbid propagation of utils::Allocator
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2080
2019-05-24 15:15:00 +02:00
Matej Ferencevic
0638179a8f Fix raft voting
Reviewers: ipaljak, msantl

Reviewed By: ipaljak, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2082
2019-05-24 12:21:37 +02:00
Matija Santl
4ec3f62f4e Move constraints to common
Summary:
Preparing unique constraint code to be implemented in HA. To do so, I'm
moving everything to `stograge/common/` folder. I also added a new header,
`gid.hpp` which does a `ifdef` include of the correct `gid.hpp` based on the
product.

Reviewers: ipaljak, mferencevic, vkasljevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2079
2019-05-23 16:36:14 +02:00
Matija Santl
e1fdb85a34 Keep SnapshotMetadata in memory
Summary:
Instead of asking rocksdb for snapshot metadata, keep it in memory for
faster access.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2075
2019-05-23 14:26:12 +02:00
Matija Santl
f2cc41fa59 Remove unique from label-prop index
Summary: Remove unique feature from label-property index

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: tsabolcec, pullbot

Differential Revision: https://phabricator.memgraph.io/D2045
2019-05-23 12:39:41 +02:00
Vinko Kasljevic
1f93edaa88 Add checks to GlobalAddress in order to prevent misuse
Reviewers: msantl, ipaljak

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2074
2019-05-21 15:37:03 +02:00
Matej Ferencevic
3c48e612ef Make move operations noexcept
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1991
2019-05-21 10:46:25 +02:00
Teon Banek
ee2fa47b29 Remove inheriting TotalOrdering in TypedValue
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2070
2019-05-21 09:51:18 +02:00
Teon Banek
7e7b6c7535 Add "Code Review Guidelines" to docs/dev
Reviewers: mferencevic, msantl, dsantl, buda

Reviewed By: mferencevic, msantl

Differential Revision: https://phabricator.memgraph.io/D2065
2019-05-21 09:51:16 +02:00
Matija Santl
5d5dfbb6f7 Fix how HA handles leader change during commit
Summary:
During it's leadership, one peer can receive RPC messages from other peers that his reign is over.
The problem is when this happens during a transaction commit.

This is handled in the following way.
If we're the current leader and we want to commit a transaction, we need to make sure the Raft Log is replicated before we can tell the client that the transaction is committed.
During that wait, we can only notice that the replication takes too long, and we report that with `LOG(WARNING)` messages.

If we change the Raft mode during the wait, our Raft implementation will internally commit this transaction, but won't be able to acquire the Raft lock because the `db.Reset` has been called.
This is why there is an manual lock acquire. If we pick up that the `db.Reset` has been called, we throw an `UnexpectedLeaderChangeException` exception to the client.

Another thing with long running transactions, if someone decides to kill a `memgraph_ha` instance during the commit, the transaction will have `abort` hint set. This will cause the `src/query/operator.cpp` to throw a `HintedAbortError`. We need to catch this during the shutdown, because the `memgraph_ha` isn't dead from the user perspective, and the transaction wasn't aborted because it took too long, but we can differentiate between those two.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1956
2019-05-20 16:39:44 +02:00
Matija Santl
37c68f0508 Add multiple properties unique constraint
Summary:
Unique constraint now support multiple properties

Depends on D2043

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2044
2019-05-20 15:18:05 +02:00
Matej Ferencevic
1e79313538 Make HA heartbeat independent of other RPCs
Reviewers: msantl, ipaljak

Reviewed By: msantl, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2073
2019-05-20 15:04:15 +02:00
Matej Ferencevic
6c49e6de02 Add SSL support to HA RPC
Reviewers: msantl, teon.banek

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2055
2019-05-20 12:53:52 +02:00
Marin Tomic
4e927a52e1 Change ExpandUniquenessFilter to EdgeUniquenessFilter in planning docs
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D2069
2019-05-20 11:14:17 +02:00
Teon Banek
4f4837392e Use MemoryResource in UnwindCursor
Summary:
Micro benchmarks show no change compared to global new & delete. This is
to be expected, because Unwind relies only on `std::vector` which ought
to reserve the memory in reasonable chunks.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2064
2019-05-17 15:28:54 +02:00
Teon Banek
880f23620d Move UnwindCursor from LCP to CPP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2062
2019-05-17 15:28:54 +02:00
Teon Banek
8459fd90fc Use MemoryResource in OrderByCursor
Summary:
Micro benchmarks show an improvement to performance of about 10%
compared to global new & delete.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2061
2019-05-17 15:28:53 +02:00
Teon Banek
abbb57c868 Move OrderByCursor from LCP to CPP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2060
2019-05-17 15:28:53 +02:00
Teon Banek
0075eee58b Use MemoryResource in AggregationCursor
Summary: Micro benchmarks show some improvement, but unfortunately not much.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2058
2019-05-17 15:28:53 +02:00
Matej Ferencevic
6c84092023 Add GetLeaderId to HAClient
Reviewers: msantl, dlozic, ipaljak

Reviewed By: msantl, dlozic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2059
2019-05-16 15:22:51 +02:00
Ivan Paljak
137e020b22 Improve sync performance in Raft
Summary: We should now exchange around O(log(n)) messages to get a follower that is n transactions behind back in sync.

Reviewers: msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2051
2019-05-16 13:53:13 +02:00
Matej Ferencevic
d4b2d76a35 Reimplement counter openCypher function
Reviewers: teon.banek, msantl

Reviewed By: teon.banek, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2047
2019-05-16 11:09:02 +02:00
Teon Banek
98a853a95c Move AggregateCursor from LCP to CPP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2056
2019-05-16 09:58:15 +02:00
Teon Banek
f9471f341d Fix move assign of MonotonicBufferResource
Summary:
I was a bit stupid and thought I can use `std::swap`, while `std::swap`
is implemented in terms of move itself.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2054
2019-05-15 17:20:32 +02:00
Matej Ferencevic
6082d31843 Fix communication context SSL leaks
Summary:
A `valgrind` run of a modified client that connects multiple times to the
database now shows no memory leaks, previously the context created with
`SSL_CTX_new` was leaked every time.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek, mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2052
2019-05-15 16:47:54 +02:00
Matija Santl
52adbc6a8b Revert "Introduce shutdown mode in Raft"
Summary: This reverts commit 08b9197c79.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2053
2019-05-15 16:39:32 +02:00
Teon Banek
1cc71c6ce8 Use MemoryResource in AccumulateCursor
Summary:
Micro benchmarks show that MonotonicBufferResource improves performance
by a factor of 1.5.

Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2048
2019-05-15 16:13:35 +02:00
Teon Banek
78830b6ed8 Use MemoryResource in STShortestPathCursor
Summary:
Benchmarks show minor improvements. Perhaps it makes sense at some later date
to use another allocator for things lasting only in a single `Pull`.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2018
2019-05-15 16:13:23 +02:00
Teon Banek
801bdb3a91 Use MemoryResource for members of ExpandVariableCursor
Summary:
Unfortunately, the written micro benchmark only reports minor
improvements compared to default allocator. The results are in some
cases even a tiny bit worse.

Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2039
2019-05-15 16:07:27 +02:00
Tonko Sabolcec
b09b21b832 Dump indices in CypherDumpGenerator
Summary:
This change introduces dumping of indices keys. During the dump process,
an internal label is assigned to each vertex and index on vertex's
internal property id is created for faster matching during edge creation.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: msantl, pullbot

Differential Revision: https://phabricator.memgraph.io/D2046
2019-05-15 11:05:16 +02:00
Matija Santl
d70792f1ce Remove existence constraint
Summary: Removing existence constraint

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2043
2019-05-15 09:48:51 +02:00