Commit Graph

2410 Commits

Author SHA1 Message Date
Teon Banek
6d3e3ac4aa Remove TypedValue::Null
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2121
2019-06-12 09:55:18 +02:00
Teon Banek
df90184b50 Use utils::Allocator in query::Path
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2117
2019-06-12 09:55:18 +02:00
Teon Banek
96505b7fb4 Use Allocator for map in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2116
2019-06-12 09:55:17 +02:00
Lovro Lugovic
0b2240e6e4 LCP: Split up PROCESS-FILE
Summary:
Split up `process-file` so that looking at the generated code for an LCP form is
easier from the REPL.

`process-lcp`, `generate-hpp` and `generate-cpp` now perform the generation of
C++ code, but take a list of "C++ elements" (the results of LCP forms) as input
and write their output to streams. They do no reading/evaluating of LCP forms of
their own.

`read-lcp` and `read-lcp-file` are used to read and evaluate a stream of LCP
forms. The latter is a specialized version for file streams which also reports
the position of the form within the file when an error happens.

`process-lcp-string` and `process-lcp-file` are convenient wrappers around the
main functionality that take a string (file) and output to strings (files).
Using `read-lcp` and `read-lcp-file` they process LCP forms and pass them off to
`process-lcp` for code generation.

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2097
2019-06-12 09:38:41 +02:00
Lovro Lugovic
21d05e9360 LCP: Add resolution for SLK-OPTS and CLONE-OPTS
Summary: Depends on D2094

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2095
2019-06-12 09:38:05 +02:00
Lovro Lugovic
dbd226d05d LCP: Minor improvements
Summary:
- Add a unified `generate_lcp` target
- Simplify SLK-ERROR and CLONE-ERROR
- Rework WITH-VARS
- Expect CPP-CLASS, not CPP-TYPE
- Fix CPP-TYPE-REFERENCE-P
- Add CPP-GENSYM
- Add util.lisp to LCP's source files in CMake
- Make the CMake variable `lcp_src_files` a cache (persistent) variable
- Add `lcp_src_files` as a dependency to `test_lcp`
- Rename `lcp_compile` to `compile-lcp`
- Fetch docstrings for CPP-NAME-* functions at run-time
- Remove existing C++ entities on redefinition

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2094
2019-06-12 09:38:02 +02:00
Lovro Lugovic
0732264fd5 LCP: Make :PUBLIC, :PROTECTED and :PRIVATE options aggregating
Summary: Depends on D2092

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2093
2019-06-12 09:37:59 +02:00
Lovro Lugovic
2736f31dcf LCP: Rework DEFINE-CLASS and DEFINE-ENUM
Summary: Depends on D2091

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2092
2019-06-12 09:37:55 +02:00
Lovro Lugovic
b8514ccf04 LCP: Update documentation
Summary: Depends on D2090

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2091
2019-06-12 09:37:52 +02:00
Lovro Lugovic
59af45f94e LCP: Fix up LCP warnings
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2090
2019-06-12 09:37:47 +02:00
Matej Ferencevic
18698ac7e6 Disable distributed test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2126
2019-06-11 12:17:12 +02:00
Matej Ferencevic
1e3b0ea727 Setup mgclient library
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2124
2019-06-11 10:12:52 +02:00
Tonko Sabolcec
eb45ed09df Dump cleanup
Summary:
`dump()` awesome Memgraph function is removed.
`mg_dump` executable now uses "dump database" command instead of "return dump()"
and dumps database to standard output instead of a file.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2122
2019-06-10 17:01:31 +02:00
Tonko Sabolcec
485592eb48 Add support for dumping unqiue constraints
Summary: Dumps unique constraints, tests included

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2112
2019-06-10 16:50:46 +02:00
Matej Ferencevic
8194fbd6f7 Fix utils::SkipList iterator bug
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2119
2019-06-10 15:32:47 +02:00
Tonko Sabolcec
9cc2224ac3 Handle dump database query
Summary:
Stream queries to the output table.
For effective output streaming, a simple operator, `OutputTableStream` is implemented which fetches and
produces a single row on each Pull.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2099
2019-06-10 13:22:14 +02:00
Teon Banek
fa62ea1920 Use ValueMap instead of Value<>
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2114
2019-06-10 10:35:04 +02:00
Teon Banek
6bb72d14a1 Use Allocator for string in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2104
2019-06-10 10:35:04 +02:00
Teon Banek
16555eeb9a Add special construct for std::pair in Allocator
Summary:
This is unfortunately needed in the C++17 standard, so that the
allocator is correctly propagated to elements of pair which respect the
"Uses Allocator" protocol. C++20 standard resolves this issue, but we
still have a long way before it is released and implemented by the
compiler and standard library vendors.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2107
2019-06-06 13:44:56 +02:00
Ivan Paljak
716a80edf0 Include HA client in index integration test
Summary:
The test now uses `ha_client`. Logging is also modified to output 1-indexed
worker ids.

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2113
2019-06-05 16:07:30 +02:00
Matija Santl
c9ba638b9f Fix includes for constraints
Summary: Forward declare the needed struct and include in the correct file.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2110
2019-06-04 16:47:23 +02:00
Ivan Paljak
9d932797ab Include HA client in HA basic integration test
Summary:
- Included HA client
- Fixed log messages to be 1-indexed
- Added id properties to created nodes for easier debugging
- Create and check steps are now executed 20 times each

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2111
2019-06-04 16:12:51 +02:00
Ivan Paljak
45413c0612 Improve large log entries integration test
Summary:
- Server ids are now 1-indexed in logs
- All created nodes have distinct is properties which helps with debugging

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2109
2019-06-04 15:21:05 +02:00
Ivan Paljak
c775688aa7 Add HA integration test for leader election
Summary:
This tests checks the correctness of a leader election process when its
decoupled from log replication. In other words, in this test we do not change
the state of the database, i. e., the Raft log remains empty.

The test proceeds as follows for clusters of size 3 and 5:
  1. Start a random subset of workers in the cluster
  2. Check if the leader has been elected
  3. Kill all living workers
  4. GOTO 1 and repeat 10 times

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2105
2019-06-04 12:48:35 +02:00
Teon Banek
07188fff7f Add utils::Substr which can use different allocator
Reviewers: mtomic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2103
2019-06-03 17:07:50 +02:00
Teon Banek
38c6625e2c utils: Support std::basic_string with allocators
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2102
2019-06-03 17:07:49 +02:00
Teon Banek
c0dc37fe34 Use ValueString instead of Value<>
Reviewers: mtomic, llugovic, msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2101
2019-06-03 17:07:49 +02:00
Teon Banek
bb2dbc290f Use Allocator for vector in TypedValue
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2086
2019-06-03 17:07:49 +02:00
Teon Banek
0152ac2dfe Add MemoryResource to TypedValue
Summary:
This diff only introduces a MemoryResource member to TypedValue and correctly
propagates through various constructors and assignments. At the moment,
MemoryResource is not used to actually allocate anything.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2085
2019-06-03 17:07:48 +02:00
Tonko Sabolcec
1bcb3cbd86 Add grammar for dump query
Summary:
Defined Antlr4 grammar for `DUMP DATABASE` query.
Defined DUMP permission in auth module.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2077
2019-06-03 10:44:18 +02:00
Ivan Paljak
d6de4f4295 Exit early on request vote if already voted
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2098
2019-05-29 17:01:22 +02:00
Ivan Paljak
24ae8a759b Update constants in large log entries test to prevent test timeouts
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2100
2019-05-29 15:40:46 +02:00
Matija Santl
87423eed06 Add constraint in HA
Summary:
HA should now support constraints in the same way the SM version does.
I only tested this thing manually, but I plan to add a new integration test for
this also.

Reviewers: ipaljak, vkasljevic, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2083
2019-05-29 14:55:04 +02:00
Ivan Paljak
d7acf75a78 Add an integration test which replicates large log entries
Summary:
At the moment, this test will fail. There are currently two issues with it:

  - Cap'n Proto limit exception might occur
  - `db.Reset()` method can hang (deadlock)

The first issue should be resolved by migration to SLK and we are currently
working on the second one. The test will land after those are fixed.

Reviewers: msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2006
2019-05-29 11:00:55 +02:00
Tonko Sabolcec
d0f05c6cc5 Create simple dump executable
Summary:
Implemented a simple dump executable which runs `RETURN dump()` on the database using Bolt protocol.
The `dump()` function returns a list of openCypher queries needed for database reconstruction (note: in the future we should use more appropriate method that supports streaming of data).

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D2067
2019-05-29 10:30:41 +02:00
Ivan Paljak
c21d04ce8d Keep last raft log entry metadata in memory
Reviewers: msantl, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2096
2019-05-28 18:33:31 +02:00
Teon Banek
f6264ab2ae Use ValueList instead of Value<>
Summary:
This change will make the transition to allocator backed TypedValue
smoother.

Reviewers: mtomic, llugovic, mferencevic, msantl

Reviewed By: mtomic, llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2084
2019-05-28 09:41:31 +02:00
Teon Banek
18cf877a47 Respect std::uses_allocator trait in utils::Allocator
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2081
2019-05-28 09:41:31 +02:00
Teon Banek
64a05bc972 Use CHECK in TypedValueVectorCompare|Equal
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2066
2019-05-28 09:41:31 +02:00
Matija Santl
e4fd49a530 Remove unique index from query execution
Summary: Remove leftover unique index code from query execution.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2078
2019-05-28 08:16:35 +02:00
Tonko Sabolcec
b57e50865e Bugfix in database dump tests
Summary:
Edge creation in tests used to happen in different
transaction than vertex creation which caused unexpected
behaviour. In this change, both vertices and edges are
created in the same transaction.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2089
2019-05-27 15:43:35 +02:00
Matej Ferencevic
42bf81021e Fix multiple raft issues
Summary:
Fix condition variable notifications
Fix vote requested invalid memory access (size off by one)
Fix blocking wait in RaftPeer while candidate
Don't copy large log entries when only the term is needed

Reviewers: ipaljak, msantl

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2088
2019-05-27 11:03:24 +02:00
dlozic
81a7a7b600 Add Raft term updates integration test
Reviewers: ipaljak, msantl, mferencevic

Reviewed By: ipaljak, msantl, mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D2002
2019-05-27 10:33:40 +02:00
Teon Banek
88778fc70f Explicitly forbid propagation of utils::Allocator
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2080
2019-05-24 15:15:00 +02:00
Matej Ferencevic
0638179a8f Fix raft voting
Reviewers: ipaljak, msantl

Reviewed By: ipaljak, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2082
2019-05-24 12:21:37 +02:00
Matija Santl
4ec3f62f4e Move constraints to common
Summary:
Preparing unique constraint code to be implemented in HA. To do so, I'm
moving everything to `stograge/common/` folder. I also added a new header,
`gid.hpp` which does a `ifdef` include of the correct `gid.hpp` based on the
product.

Reviewers: ipaljak, mferencevic, vkasljevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2079
2019-05-23 16:36:14 +02:00
Matija Santl
e1fdb85a34 Keep SnapshotMetadata in memory
Summary:
Instead of asking rocksdb for snapshot metadata, keep it in memory for
faster access.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2075
2019-05-23 14:26:12 +02:00
Matija Santl
f2cc41fa59 Remove unique from label-prop index
Summary: Remove unique feature from label-property index

Reviewers: ipaljak, mferencevic, vkasljevic

Reviewed By: mferencevic

Subscribers: tsabolcec, pullbot

Differential Revision: https://phabricator.memgraph.io/D2045
2019-05-23 12:39:41 +02:00
Vinko Kasljevic
1f93edaa88 Add checks to GlobalAddress in order to prevent misuse
Reviewers: msantl, ipaljak

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2074
2019-05-21 15:37:03 +02:00
Matej Ferencevic
3c48e612ef Make move operations noexcept
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1991
2019-05-21 10:46:25 +02:00