Commit Graph

2517 Commits

Author SHA1 Message Date
Teon Banek
7d01ba5178 Use generic :dont-save mechanism in LCP
Summary:
It doesn't make sense to tie skipping a member for serialization to a
particular serialization implementation.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1743
2018-11-22 10:01:38 +01:00
Teon Banek
14c9044909 Convert symbol.hpp to LCP
Summary:
With quite frequent changes of serialization backend, it's getting
really painful updating the C++ implementation. This diff defines the
Symbol class via LCP, so that the C++ serialization is generated instead
of written by hand.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1741
2018-11-22 09:52:18 +01:00
Matija Santl
dd6fe013dc Parse raft config
Summary:
Added command line parameters to specify rpc flags, raft and
coordination config files and current server id.

Reviewers: ipaljak, teon.banek

Reviewed By: ipaljak, teon.banek

Subscribers: pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1742
2018-11-21 17:34:13 +01:00
Teon Banek
6dbe054bdc Remove including query in durability
Reviewers: msantl, ipaljak, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1740
2018-11-21 15:27:51 +01:00
Teon Banek
a75008f94b Support SLK serialization of enums in LCP
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1737
2018-11-21 13:28:41 +01:00
Ivan Paljak
73da1e4463 Add Raft skeleton
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1732
2018-11-19 13:21:58 +01:00
Teon Banek
58b450a2ea Remove template from ExpandUniquenessFilter
Summary:
Instantiation with VertexAccessor was never used, so the template
needlessly complicated the rest of the codebase.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1736
2018-11-19 13:03:39 +01:00
Teon Banek
8ed1fbbbe1 Generate TypeInfo object for all LCP defined classes
Summary:
TypeInfo will be needed for upcoming serialization via SLK. This diff
changes the already defined TypeInfo by removing the reliance on
capnp::typeId calls. The struct itself is now in utils.

Hopefully, this shouldn't break our RPC stack due to new ID generation.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1735
2018-11-16 13:27:58 +01:00
Teon Banek
a272fa2e6b Add basic support for saving classes with SLK
Summary:
This is the initial work on transferring our serialization code from
Cap'n Proto to SaveLoadKit. The commit contains tests for generated
code, but still requires work on supporting some features. Most notably,
generating and storing type IDs for derived classes so that they can be
loaded from a base pointer. Naturally, loading implementation hasn't
even been started yet.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1733
2018-11-15 13:26:38 +01:00
Vinko Kasljevic
b77d186f58 Add storage stat for single node
Summary:
GraphDb now has GetStorageStat method that returns live view to
object containing vertex_count, edge_count and avg_degree().
Stat is updated on each garbage collection run and represents number of
objects that are visible to any transaction.

Reviewers: teon.banek, msantl, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1731
2018-11-15 11:53:03 +01:00
Dino Santl
d3634e9a39 Add SmallVector and tests
Summary: SmallVector data structure introduction.  This is the first step. In this revision are SmallVector implementation and tests. The second step will be swapping std::vector with SmallVector in the current codebase.

Reviewers: teon.banek, buda, ipaljak, mtomic

Reviewed By: teon.banek, ipaljak

Subscribers: ipaljak, pullbot

Differential Revision: https://phabricator.memgraph.io/D1730
2018-11-14 15:04:12 +01:00
Ivan Paljak
7f44b895b4 Add Raft RPC messages
Summary:
Basic RPC messages for Raft protocol. They will most likely be updated as we
move along with the implementation.

Reviewers: msantl, teon.banek, mferencevic

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1726
2018-11-12 14:24:23 +01:00
Matija Santl
54fa46541e Serialize StateDelta for HA
Summary:
Since we need to send `StateDelta`s over the wire in HA, we need to be
able to serialize those bad boys.
This diff hopefully does this the right way.

Reviewers: teon.banek, mferencevic, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1725
2018-11-12 13:36:11 +01:00
Matej Ferencevic
6d21e58b09 Add ClientPool and ThreadPool tests to RPC benchmark
Summary: Add script for plotting throughput

Reviewers: teon.banek, mculinovic, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1721
2018-11-12 13:05:37 +01:00
Teon Banek
5df4d55ec1 Extract QueryVisitor from HierarchicalTreeVisitor
Summary:
This change makes HierarchicalTreeVisitor visit only Cypher related AST
nodes. QueryVisitor can be used to differentiate between various query
types we have. The next step is to either rename HierarchicalTreeVisitor
to something like CypherQueryVisitor, or perhaps extract Clause visiting
from it.

Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1710
2018-11-12 12:53:25 +01:00
Matej Ferencevic
bbe095d41a Improve serialization benchmark
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1729
2018-11-12 12:20:02 +01:00
Matej Ferencevic
2c354e82f3 Add support for PropertyValue to SLK
Summary:
Improve tests
Add exceptions

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1727
2018-11-12 11:20:15 +01:00
Ivan Paljak
45098dd80b Update Raft feature spec (no-op in reads)
Reviewers: msantl, teon.banek, buda, mtomic, mferencevic

Reviewed By: msantl

Differential Revision: https://phabricator.memgraph.io/D1728
2018-11-12 10:49:22 +01:00
Marin Petricevic
20dcb05c50 Add Tensorflow op feature spec
Reviewers: buda, teon.banek, mferencevic, msantl

Reviewed By: buda, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1691
2018-11-08 11:17:20 +01:00
Matej Ferencevic
4ad9cfe1f4 Fix distributed tests exit code
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1499
2018-11-07 22:46:55 +01:00
Matej Ferencevic
cc8f199a5f Reduce distributed tests flakiness
Summary:
The single-binary distributed tests didn't wait for the whole cluster to be up
and running before they started running their tests. Now all tests ensure that
all workers are registered to the master and all other workers before starting
any tests.

Reviewers: mculinovic

Reviewed By: mculinovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1724
2018-11-07 19:27:28 +01:00
Lovro Lugovic
06d4568950 Add ExpressionPrettyPrinter
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1719
2018-11-07 13:06:14 +01:00
Vinko Kasljevic
7ba8228c46 Refactor storage file structure
Summary:
- Create types folder in storage/common
- Move locking and kvstore to storage/common
- Add storage/distributed/rpc folder

Reviewers: teon.banek, ipaljak, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1713
2018-11-06 18:17:31 +01:00
Matej Ferencevic
f923e7f7cb Remove wrongly committed files
Summary: Add file to .gitignore

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1722
2018-11-06 16:49:48 +01:00
Matej Ferencevic
31d31cf4b9 Fix parallel macro benchmarks
Reviewers: mculinovic

Reviewed By: mculinovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1723
2018-11-06 16:49:12 +01:00
Matej Ferencevic
2cbe97d7c2 Implement RPC benchmark
Reviewers: teon.banek, mculinovic, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1720
2018-11-05 10:45:43 +01:00
Teon Banek
08c14bfafc Split C++ meta information types to a new file
Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1716
2018-11-05 09:37:09 +01:00
Teon Banek
cf19bbad8f Add basic indentation to vim-lcp plugin
Reviewers: mtomic, mferencevic, msantl, buda

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1718
2018-10-31 15:37:42 +01:00
Matija Santl
4f0a7df4bb Fix unique constraint recovery
Summary:
See https://app.asana.com/0/743890251333732/888297761596047/f for more details.

# BEFORE
```
Note: Google Test filter = *UniqueConstraintRecovery*
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Durability
[ RUN       ] Durability.UniqueConstraintRecovery
unknown file: Failure
C++ exception with description "Index couldn't be created due to constraint
violation!" thrown in the test body.
[  FAILED   ] Durability.UniqueConstraintRecovery (3 ms)
[----------] 1 test from Durability (3 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (3 ms total)
[  PASSED   ] 0 tests.
[  FAILED   ] 1 test, listed below:
[  FAILED   ] Durability.UniqueConstraintRecovery

 1 FAILED TEST
 ```

# AFTER

 ```
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Durability
[ RUN       ] Durability.UniqueConstraintRecovery
[       OK  ] Durability.UniqueConstraintRecovery (4 ms)
[----------] 1 test from Durability (4 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (4 ms total)
[  PASSED   ] 1 test.
```

Reviewers: ipaljak, vkasljevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1714
2018-10-31 15:29:06 +01:00
Matej Ferencevic
0493470f98 Add support for std::pair in SLK
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1717
2018-10-31 15:21:36 +01:00
Matej Ferencevic
878b5c9f2b Improve analyze_rpc_calls script
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1715
2018-10-31 14:30:07 +01:00
Matej Ferencevic
60fb4901f5 Initial version of custom serialization
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1711
2018-10-31 12:00:43 +01:00
Matej Ferencevic
95ad542f19 Remove coalesce benchmark
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1712
2018-10-30 21:15:22 +01:00
Matej Ferencevic
ef53bd8eb7 Improve coverage script
Summary:
The code coverage debug script now supports all Memgraph targets (the main
binaries, unit tests and tools).

Reviewers: mculinovic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1709
2018-10-30 16:23:24 +01:00
Matija Santl
43cb506f06 Prepare codebase for high availability
Summary:
Initial for fork (in form of a c/p) form the current single node
version. Once we finish HA we plan to re-link the files to the single node
versions if they don't change.

Reviewers: ipaljak, buda, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1705
2018-10-30 10:58:50 +01:00
Matej Ferencevic
25adfdb90c Lower DeferredDeleter memory allocations
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1688
2018-10-30 09:31:39 +01:00
Teon Banek
218384786d Replace TreeVisitor with ExpressionVisitor
Summary:
This is the initial step in making the visitors more granular in what
they visit.

Also add ignoring multiple inheritance in LCP and update LCP docs.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1704
2018-10-30 09:24:45 +01:00
Marin Tomic
50b2646765 Compose ExpandCommon instead of inheriting from it
Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1703
2018-10-29 16:51:52 +01:00
Lovro Lugovic
d9db38dca6 Implement hierarchical Accept for AuthQuery and StreamQuery
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1708
2018-10-29 13:57:37 +01:00
Lovro Lugovic
06ae2ffecc Remove the Coalesce function
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1676
2018-10-29 11:37:20 +01:00
Dino Santl
d87664d5db Movie tutorial database added
Summary: Queries for filling database with movie data.

Reviewers: ipaljak, teon.banek, mferencevic

Reviewed By: ipaljak

Subscribers: mculinovic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1693
2018-10-29 11:02:41 +01:00
Teon Banek
7892f69920 Add changelog for version 0.14
Reviewers: mferencevic, buda, msantl

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D1706
2018-10-29 10:32:23 +01:00
Ivan Paljak
598ce81754 Add HA feature spec
Reviewers: msantl, buda, teon.banek, mferencevic, mtomic, vkasljevic

Reviewed By: msantl

Differential Revision: https://phabricator.memgraph.io/D1692
2018-10-29 10:23:33 +01:00
Teon Banek
66dcd536f0 Support serializing LCP enum vector
Summary:
Generate conversion functions from C++ enum to Cap'n Proto and vice
verse. This should reduce the generated code size and hopefully improve
readability.

Use conversion functions for enums in default serialization generation.
`capnp-save-enum` and `capnp-load-enum` should now be used exclusively
to serialize non LCP defined enumerations. They now always require
`enum-values` argument.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1696
2018-10-26 16:05:47 +02:00
Matija Santl
f6f58d843b Drop index
Reviewers: teon.banek, mferencevic, vkasljevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1659
2018-10-26 13:51:22 +02:00
Marin Tomic
1f07879489 Make explain check case insensitive
Summary: I wrongly assumed that stripper lowercases keywords.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1697
2018-10-26 10:45:36 +02:00
Marin Tomic
e4b661fe4a Cache required privileges for query execution
Summary: this should reduce parsing time for very simple queries.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1698
2018-10-26 10:40:13 +02:00
Marin Tomic
4f2a69b973 Remove debug logging from stream handling in interpreter
Reviewers: msantl, teon.banek

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1699
2018-10-26 10:39:04 +02:00
Matej Ferencevic
69b666c487 Fix integration distributed test flakiness
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1702
2018-10-26 10:17:23 +02:00
Teon Banek
30e18026a5 Regenerate LCP files if the LCP itself changed
Summary:
Unfortunately, CMake macros cannot access variables defined in the file
where macro is defined. This means that we have to repeat the values by
hand in order to track the correct dependency of LCP.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1701
2018-10-26 10:06:04 +02:00