Commit Graph

1240 Commits

Author SHA1 Message Date
Teon Banek
775930ef4e Use stack for initial memory of MonotonicBufferResource in Pull
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2139
2019-06-26 10:26:53 +02:00
Teon Banek
5adb4aafe0 Use execution memory for Frame allocations
Summary:
Micro benchmarks show some minor variations compared to the previous
commit. Smaller cases are a bit worse while larger data cases are a bit
better.

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2136
2019-06-24 15:05:44 +02:00
Teon Banek
6086257204 Use per_pull and whole execution allocators in Cursors
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2134
2019-06-24 15:05:43 +02:00
Tonko Sabolcec
1253319710 Bugfix: use indices when dumping edges
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2142
2019-06-21 16:06:23 +02:00
Teon Banek
3fd14e2d5f Use custom allocator in Evaluator through context
Summary:
Micro benchmarks show improvements in performance of MapLiteral from 5%
to 40% depending on the size of the input. On the other hand, a sequence
of AdditionOperators behaves the same with both allocation schemes.

Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2132
2019-06-17 10:53:25 +02:00
Teon Banek
57d967786c Make TypedValue constructor explicit for non-primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2128
2019-06-12 14:55:58 +02:00
Teon Banek
b3bc4d6809 Make TypedValue constructor explicit for primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2127
2019-06-12 14:55:57 +02:00
Teon Banek
6d3e3ac4aa Remove TypedValue::Null
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2121
2019-06-12 09:55:18 +02:00
Teon Banek
df90184b50 Use utils::Allocator in query::Path
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2117
2019-06-12 09:55:18 +02:00
Teon Banek
96505b7fb4 Use Allocator for map in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2116
2019-06-12 09:55:17 +02:00
Lovro Lugovic
dbd226d05d LCP: Minor improvements
Summary:
- Add a unified `generate_lcp` target
- Simplify SLK-ERROR and CLONE-ERROR
- Rework WITH-VARS
- Expect CPP-CLASS, not CPP-TYPE
- Fix CPP-TYPE-REFERENCE-P
- Add CPP-GENSYM
- Add util.lisp to LCP's source files in CMake
- Make the CMake variable `lcp_src_files` a cache (persistent) variable
- Add `lcp_src_files` as a dependency to `test_lcp`
- Rename `lcp_compile` to `compile-lcp`
- Fetch docstrings for CPP-NAME-* functions at run-time
- Remove existing C++ entities on redefinition

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2094
2019-06-12 09:38:02 +02:00
Matej Ferencevic
18698ac7e6 Disable distributed test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2126
2019-06-11 12:17:12 +02:00
Tonko Sabolcec
485592eb48 Add support for dumping unqiue constraints
Summary: Dumps unique constraints, tests included

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2112
2019-06-10 16:50:46 +02:00
Tonko Sabolcec
9cc2224ac3 Handle dump database query
Summary:
Stream queries to the output table.
For effective output streaming, a simple operator, `OutputTableStream` is implemented which fetches and
produces a single row on each Pull.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2099
2019-06-10 13:22:14 +02:00
Teon Banek
fa62ea1920 Use ValueMap instead of Value<>
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2114
2019-06-10 10:35:04 +02:00
Teon Banek
6bb72d14a1 Use Allocator for string in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2104
2019-06-10 10:35:04 +02:00
Teon Banek
16555eeb9a Add special construct for std::pair in Allocator
Summary:
This is unfortunately needed in the C++17 standard, so that the
allocator is correctly propagated to elements of pair which respect the
"Uses Allocator" protocol. C++20 standard resolves this issue, but we
still have a long way before it is released and implemented by the
compiler and standard library vendors.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2107
2019-06-06 13:44:56 +02:00
Ivan Paljak
716a80edf0 Include HA client in index integration test
Summary:
The test now uses `ha_client`. Logging is also modified to output 1-indexed
worker ids.

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2113
2019-06-05 16:07:30 +02:00
Ivan Paljak
9d932797ab Include HA client in HA basic integration test
Summary:
- Included HA client
- Fixed log messages to be 1-indexed
- Added id properties to created nodes for easier debugging
- Create and check steps are now executed 20 times each

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2111
2019-06-04 16:12:51 +02:00
Ivan Paljak
45413c0612 Improve large log entries integration test
Summary:
- Server ids are now 1-indexed in logs
- All created nodes have distinct is properties which helps with debugging

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2109
2019-06-04 15:21:05 +02:00
Ivan Paljak
c775688aa7 Add HA integration test for leader election
Summary:
This tests checks the correctness of a leader election process when its
decoupled from log replication. In other words, in this test we do not change
the state of the database, i. e., the Raft log remains empty.

The test proceeds as follows for clusters of size 3 and 5:
  1. Start a random subset of workers in the cluster
  2. Check if the leader has been elected
  3. Kill all living workers
  4. GOTO 1 and repeat 10 times

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2105
2019-06-04 12:48:35 +02:00
Teon Banek
07188fff7f Add utils::Substr which can use different allocator
Reviewers: mtomic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2103
2019-06-03 17:07:50 +02:00
Teon Banek
38c6625e2c utils: Support std::basic_string with allocators
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2102
2019-06-03 17:07:49 +02:00
Teon Banek
c0dc37fe34 Use ValueString instead of Value<>
Reviewers: mtomic, llugovic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2101
2019-06-03 17:07:49 +02:00
Teon Banek
bb2dbc290f Use Allocator for vector in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2086
2019-06-03 17:07:49 +02:00
Teon Banek
0152ac2dfe Add MemoryResource to TypedValue
Summary:
This diff only introduces a MemoryResource member to TypedValue and correctly
propagates through various constructors and assignments. At the moment,
MemoryResource is not used to actually allocate anything.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2085
2019-06-03 17:07:48 +02:00
Tonko Sabolcec
1bcb3cbd86 Add grammar for dump query
Summary:
Defined Antlr4 grammar for `DUMP DATABASE` query.
Defined DUMP permission in auth module.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2077
2019-06-03 10:44:18 +02:00
Ivan Paljak
24ae8a759b Update constants in large log entries test to prevent test timeouts
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2100
2019-05-29 15:40:46 +02:00
Matija Santl
87423eed06 Add constraint in HA
Summary:
HA should now support constraints in the same way the SM version does.
I only tested this thing manually, but I plan to add a new integration test for
this also.

Reviewers: ipaljak, vkasljevic, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2083
2019-05-29 14:55:04 +02:00
Ivan Paljak
d7acf75a78 Add an integration test which replicates large log entries
Summary:
At the moment, this test will fail. There are currently two issues with it:

  - Cap'n Proto limit exception might occur
  - `db.Reset()` method can hang (deadlock)

The first issue should be resolved by migration to SLK and we are currently
working on the second one. The test will land after those are fixed.

Reviewers: msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2006
2019-05-29 11:00:55 +02:00
Teon Banek
f6264ab2ae Use ValueList instead of Value<>
Summary:
This change will make the transition to allocator backed TypedValue
smoother.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2084
2019-05-28 09:41:31 +02:00
Teon Banek
18cf877a47 Respect std::uses_allocator trait in utils::Allocator
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2081
2019-05-28 09:41:31 +02:00
Matija Santl
e4fd49a530 Remove unique index from query execution
Summary: Remove leftover unique index code from query execution.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2078
2019-05-28 08:16:35 +02:00
Tonko Sabolcec
b57e50865e Bugfix in database dump tests
Summary:
Edge creation in tests used to happen in different
transaction than vertex creation which caused unexpected
behaviour. In this change, both vertices and edges are
created in the same transaction.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2089
2019-05-27 15:43:35 +02:00
dlozic
81a7a7b600 Add Raft term updates integration test
Reviewers: ipaljak, msantl, mferencevic

Reviewed By: ipaljak, msantl, mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2002
2019-05-27 10:33:40 +02:00
Matija Santl
4ec3f62f4e Move constraints to common
Summary:
Preparing unique constraint code to be implemented in HA. To do so, I'm
moving everything to `stograge/common/` folder. I also added a new header,
`gid.hpp` which does a `ifdef` include of the correct `gid.hpp` based on the
product.

Reviewers: ipaljak, mferencevic, vkasljevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2079
2019-05-23 16:36:14 +02:00
Matija Santl
f2cc41fa59 Remove unique from label-prop index
Summary: Remove unique feature from label-property index

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: tsabolcec, pullbot

Differential Revision: https://phabricator.memgraph.io/D2045
2019-05-23 12:39:41 +02:00
Matija Santl
5d5dfbb6f7 Fix how HA handles leader change during commit
Summary:
During it's leadership, one peer can receive RPC messages from other peers that his reign is over.
The problem is when this happens during a transaction commit.

This is handled in the following way.
If we're the current leader and we want to commit a transaction, we need to make sure the Raft Log is replicated before we can tell the client that the transaction is committed.
During that wait, we can only notice that the replication takes too long, and we report that with `LOG(WARNING)` messages.

If we change the Raft mode during the wait, our Raft implementation will internally commit this transaction, but won't be able to acquire the Raft lock because the `db.Reset` has been called.
This is why there is an manual lock acquire. If we pick up that the `db.Reset` has been called, we throw an `UnexpectedLeaderChangeException` exception to the client.

Another thing with long running transactions, if someone decides to kill a `memgraph_ha` instance during the commit, the transaction will have `abort` hint set. This will cause the `src/query/operator.cpp` to throw a `HintedAbortError`. We need to catch this during the shutdown, because the `memgraph_ha` isn't dead from the user perspective, and the transaction wasn't aborted because it took too long, but we can differentiate between those two.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1956
2019-05-20 16:39:44 +02:00
Matija Santl
37c68f0508 Add multiple properties unique constraint
Summary:
Unique constraint now support multiple properties

Depends on D2043

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2044
2019-05-20 15:18:05 +02:00
Matej Ferencevic
6c49e6de02 Add SSL support to HA RPC
Reviewers: msantl, teon.banek

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2055
2019-05-20 12:53:52 +02:00
Teon Banek
4f4837392e Use MemoryResource in UnwindCursor
Summary:
Micro benchmarks show no change compared to global new & delete. This is
to be expected, because Unwind relies only on `std::vector` which ought
to reserve the memory in reasonable chunks.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2064
2019-05-17 15:28:54 +02:00
Teon Banek
8459fd90fc Use MemoryResource in OrderByCursor
Summary:
Micro benchmarks show an improvement to performance of about 10%
compared to global new & delete.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2061
2019-05-17 15:28:53 +02:00
Teon Banek
0075eee58b Use MemoryResource in AggregationCursor
Summary: Micro benchmarks show some improvement, but unfortunately not much.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2058
2019-05-17 15:28:53 +02:00
Matej Ferencevic
d4b2d76a35 Reimplement counter openCypher function
Reviewers: teon.banek, msantl

Reviewed By: teon.banek, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2047
2019-05-16 11:09:02 +02:00
Teon Banek
1cc71c6ce8 Use MemoryResource in AccumulateCursor
Summary:
Micro benchmarks show that MonotonicBufferResource improves performance
by a factor of 1.5.

Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2048
2019-05-15 16:13:35 +02:00
Teon Banek
78830b6ed8 Use MemoryResource in STShortestPathCursor
Summary:
Benchmarks show minor improvements. Perhaps it makes sense at some later date
to use another allocator for things lasting only in a single `Pull`.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2018
2019-05-15 16:13:23 +02:00
Teon Banek
801bdb3a91 Use MemoryResource for members of ExpandVariableCursor
Summary:
Unfortunately, the written micro benchmark only reports minor
improvements compared to default allocator. The results are in some
cases even a tiny bit worse.

Reviewers: mtomic, mferencevic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2039
2019-05-15 16:07:27 +02:00
Tonko Sabolcec
b09b21b832 Dump indices in CypherDumpGenerator
Summary:
This change introduces dumping of indices keys. During the dump process,
an internal label is assigned to each vertex and index on vertex's
internal property id is created for faster matching during edge creation.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: msantl, pullbot

Differential Revision: https://phabricator.memgraph.io/D2046
2019-05-15 11:05:16 +02:00
Matija Santl
d70792f1ce Remove existence constraint
Summary: Removing existence constraint

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2043
2019-05-15 09:48:51 +02:00
Matej Ferencevic
d37460105a Implement HA Bolt proxy server
Reviewers: msantl, ipaljak, teon.banek

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2030
2019-05-14 17:20:21 +02:00