Commit Graph

185 Commits

Author SHA1 Message Date
Matej Ferencevic
cde2875a50 Fix linter script
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1988
2019-04-25 17:21:23 +02:00
Matej Ferencevic
13309a0323 Reduce cppcheck warnings
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1985
2019-04-25 17:20:18 +02:00
Teon Banek
4e56673f0d Add basic support for arc lint with clang-tidy
Reviewers: mferencevic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1975
2019-04-23 15:51:36 +02:00
Matej Ferencevic
9291a5fc4d Migrate to C++17
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1974
2019-04-23 14:46:44 +02:00
Matej Ferencevic
03226c4b7e Migrate to Clang 8
Summary:
Add environment setup script
Migrate to new toolchain
Fix tiny issues in codebase
Remove unnecessary test

Reviewers: teon.banek, msantl, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1960
2019-04-19 12:34:50 +02:00
Matej Ferencevic
026c796e06 Move GraphDbAccessor to stack in SN and HA
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1944
2019-04-17 11:29:43 +02:00
Vinko Kasljevic
21bd8b492f Add recovery and GraphDbAccessor interface for UniqueLabelPropertyConstraint
Reviewers: msantl, ipaljak

Reviewed By: msantl

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1925
2019-03-27 12:33:56 +01:00
Vinko Kasljevic
6cc8d8f5b1 Add wal for ExistenceConstraints
Summary: Depends on D1797

Reviewers: msantl, ipaljak, mferencevic

Reviewed By: msantl, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1862
2019-03-06 08:52:00 +01:00
Matej Ferencevic
ce3a7d6fc2 Improve mg_import_csv test
Summary: Check that the import works without a specified field type.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1877
2019-02-22 10:07:56 +01:00
Marin Tomic
d819c7b48c Make analyze_rpc_calls script work with new TypeInfo
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1750
2018-11-27 15:02:53 +01:00
Teon Banek
8ed1fbbbe1 Generate TypeInfo object for all LCP defined classes
Summary:
TypeInfo will be needed for upcoming serialization via SLK. This diff
changes the already defined TypeInfo by removing the reliance on
capnp::typeId calls. The struct itself is now in utils.

Hopefully, this shouldn't break our RPC stack due to new ID generation.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1735
2018-11-16 13:27:58 +01:00
Matej Ferencevic
6d21e58b09 Add ClientPool and ThreadPool tests to RPC benchmark
Summary: Add script for plotting throughput

Reviewers: teon.banek, mculinovic, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1721
2018-11-12 13:05:37 +01:00
Teon Banek
cf19bbad8f Add basic indentation to vim-lcp plugin
Reviewers: mtomic, mferencevic, msantl, buda

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1718
2018-10-31 15:37:42 +01:00
Matej Ferencevic
878b5c9f2b Improve analyze_rpc_calls script
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1715
2018-10-31 14:30:07 +01:00
Matej Ferencevic
ef53bd8eb7 Improve coverage script
Summary:
The code coverage debug script now supports all Memgraph targets (the main
binaries, unit tests and tools).

Reviewers: mculinovic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1709
2018-10-30 16:23:24 +01:00
Matej Ferencevic
7f2ad99ce7 Integrate mg_client test with Apollo
Reviewers: teon.banek, buda, mculinovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1685
2018-10-19 15:31:57 +02:00
Matej Ferencevic
0013cbb173 Polish Bolt client and mg_client
Reviewers: mculinovic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1681
2018-10-19 13:55:51 +02:00
Marko Culinovic
d8293f1b6b Fix newline bug in mg_import_csv
Summary:
Handle special case when delimiter is the last character in line.
This means that another empty column needs to be added.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1678
2018-10-19 10:03:45 +02:00
Marko Culinovic
93267531be Improve mg_client
Summary:
 - Set query time to the milisecond precision.
 - Print 'Bye' when `EOF` is received in interactive mode.
 - In non-interactive mode output exit on first failed query and
   print the query along with the error message.
 - Don't quote stings when they are alone

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1670
2018-10-18 10:09:26 +02:00
Matej Ferencevic
f1bae7be44 Reduce mg_import_csv memory usage
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1663
2018-10-16 14:22:14 +02:00
Marko Culinovic
c553a309d2 New client features: non-interactive shell, shell commands, csv format
Summary:
If the `STDIN_FILENO` is a `TTY`, then an interactive command prompt
is shown. Otherwise, client runs in non-interactive mode.
Shell commands start with a colon sign, and end at the end of
line.
Csv and tabular output formats are supported.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1633
2018-10-16 09:04:34 +02:00
Lovro Lugovic
a6a621e08b Add LCP tests
Summary: Create an ASDF system for LCP. Add LCP tests.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1645
2018-10-15 11:52:20 +02:00
Matej Ferencevic
6525451489 Remove distributed logic from single node
Summary:
Start removing `is_remote` from `Address`
Remove `GlobalAddress`
Remove `GlobalizedAddress`
Remove bitmasks from `Address`
Remove `is_local` from `Address`
Remove `is_local` from `RecordAccessor`
Remove `worker_id` from `Address`
Remove `worker_id` from `GidGenerator`
Unfriend `IndexRpcServer` from `Storage`
Remove `LocalizedAddressIfPossible`
Make member private
Remove `worker_id` from `Storage`
Copy function to ease removal of distributed logic
Remove `worker_id` from `WriteAheadLog`
Remove `worker_id` from `GraphDb`
Remove `worker_id` from durability
Remove nonexistant function
Remove `gid` from `Address`
Remove usage of `Address`
Remove `Address`
Remove `VertexAddress` and `EdgeAddress`
Fix Id test
Remove `cypher_id` from `VersionList`
Remove `cypher_id` from durability
Remove `cypher_id` member from `VersionList`
Remove `cypher_id` from database
Fix recovery (revert D1142)
Remove unnecessary functions from `GraphDbAccessor`
Revert `InsertEdge` implementation to the way it was in/before D1142
Remove leftover `VertexAddress` from `Edge`
Remove `PostCreateIndex` and `PopulateIndexFromBuildIndex`
Split durability paths into single node and distributed
Fix `TransactionIdFromWalFilename` implementation
Fix tests
Remove `cypher_id` from `snapshooter` and `durability` test

Reviewers: msantl, teon.banek

Reviewed By: msantl

Subscribers: msantl, pullbot

Differential Revision: https://phabricator.memgraph.io/D1647
2018-10-11 13:08:22 +02:00
Matej Ferencevic
4e5fe37dd6 Remove virtual and pimpl from single node
Summary:
This diff removes: `SingleNodeRecoveryTransactions`, `TypemapPack`
It also removes virtual and/or pimpl from: `SingleNodeCounters`,
`StorageGcSingleNode`, `SingleNodeConcurrentIdMapper`,
accessors (revert D1510), transaction engine, `GraphDbAccessor`, `GraphDb`

Reviewers: msantl, teon.banek

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1639
2018-10-09 11:48:30 +02:00
Matej Ferencevic
ade2593b51 Separate distributed from single node GraphDb
Summary:
To clean the working directory after this diff you should execute:

```
rm src/database/counters_rpc_messages.capnp
rm src/database/counters_rpc_messages.hpp
rm src/database/serialization.capnp
rm src/database/serialization.hpp
```

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1636
2018-10-05 13:32:05 +02:00
Matej Ferencevic
75950664a7 Separate distributed from single node storage
Summary:
This diff splits single node and distributed storage from each other.
Currently all of the storage code is copied into two directories (one single
node, one distributed).  The logic used in the storage implementation isn't
touched, it will be refactored in following diffs.

To clean the working directory after this diff you should execute:
```
rm database/state_delta.capnp
rm database/state_delta.hpp
rm storage/concurrent_id_mapper_rpc_messages.capnp
rm storage/concurrent_id_mapper_rpc_messages.hpp
```

Reviewers: teon.banek, buda, msantl

Reviewed By: teon.banek, msantl

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1625
2018-10-05 09:19:33 +02:00
Marko Culinovic
2bd0822155 Implement console client for memgraph
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1616
2018-10-02 13:18:54 +02:00
Teon Banek
1fd9a72e10 Generate Load functions from LCP as top level
Summary: Depends on D1596

Reviewers: mtomic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1601
2018-09-28 10:34:20 +02:00
Teon Banek
a5926b4e0f Generate Save functions from LCP as top level
Summary:
This should allow us to more easily decouple the code which should be
open sourced. Unfortunately, the downside of this approach is that we
cannot rely on virtual calls to dispatch the serialization to correct
type. Another downside is that members need to be publicly accessible
for serialization.

Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1596
2018-09-28 10:26:56 +02:00
Matija Santl
0eb0b913a8 Support dynamic worker addition
Summary:
With this diff we should be able to support dynamic worker addition.
This is ofcourse a minimal effort maximal impact approach.

This diff introduces new RPC calls when a worker registers.

The `DynamicWorkerAddition` doesn't use `GraphDbAccessor` to get indices because
they create WAL entries.

Reviewers: vkasljevic, ipaljak, buda

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1594
2018-09-14 10:53:55 +02:00
Marko Budiselic
d9153921b8 Add distributed Id Cypher function
Reviewers: msantl, vkasljevic, teon.banek, ipaljak

Reviewed By: msantl, teon.banek

Subscribers: pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1477
2018-09-03 13:04:03 +02:00
Ivan Paljak
146c35ec4a Add magic number to WAL and fix version consistency check
Summary: Recovery file is version inconsistent iff it starts with a correct magic number, has at least one integer written after that and that integer differs from kVersion.

Reviewers: mferencevic, ipaljak, vkasljevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1565
2018-08-29 16:53:10 +02:00
Teon Banek
bb679a4b1d Move distributed operators to its own file
Summary:
This is the first step in separating the implementation of distributed
features in operators. Following steps are:

  * decoupling distributed visitors
  * injecting distributed details in operator state
  * minor cleanup or anything else that was overlooked

Reviewers: mtomic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1563
2018-08-28 14:47:14 +02:00
Matej Ferencevic
e7cde4b4ef Make analyze_rpc_calls compatible with Cap'n Proto
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1561
2018-08-24 15:40:00 +02:00
Marko Culinovic
dab95af366 Extract stats to static lib
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1546
2018-08-23 10:33:50 +02:00
Teon Banek
da9dc10373 Add a LCP syntax highlighter for vim
Reviewers: buda, msantl, mtomic, mferencevic, ipaljak, mculinovic, vkasljevic, mpetricevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1548
2018-08-22 11:06:25 +02:00
Teon Banek
da3630f8a9 Add lcp-mode for Emacs
Reviewers: mtomic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1549
2018-08-21 17:22:24 +02:00
Teon Banek
8a43ac461c Add documentation of LCP features
Reviewers: mtomic, msantl, buda, ipaljak, vkasljevic, mferencevic, mculinovic, mpetricevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1527
2018-08-20 14:07:46 +02:00
Matej Ferencevic
705c43a816 Add memgraph coverage script
Reviewers: mculinovic, msantl

Reviewed By: mculinovic

Differential Revision: https://phabricator.memgraph.io/D1536
2018-08-08 15:35:09 +02:00
Teon Banek
eeb03b132f Remove dependency on distributed from durability
Reviewers: msantl, vkasljevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1521
2018-08-02 09:40:34 +02:00
Marko Culinovic
b59edf1640 Add support for node label, rel type and quote
Summary:
New flags are introduced.
1) `node-label` flag can be specified multiple times to add
extra labels to all nodes
2) `relationship-type` overwrites relationship types from file
with given flag value
3) `quote` is used to specify quotation character

Added tests for new flags.

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1514
2018-07-31 13:28:58 +02:00
Teon Banek
9f460914ed Separate distributed implementation of GraphDbAccessor
Summary:
GraphDbAccessor is now constructed only through GraphDb. This allows the
concrete GraphDb to instantiate a concrete GraphDbAccessor. This allows
us to use virtual calls, so that the implementation may be kept
separate. The major downside of doing things this way is heap allocation
of GraphDbAccessor. In case it turns out to be a real performance
issues, another solution with pointer to static implementation may be
used.

InsertVertexIntoRemote is now a non-member function, which reduces
coupling. It made no sense for it to be member function because it used
only the public parts of GraphDbAccessor.

Reviewers: msantl, mtomic, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1504
2018-07-26 09:16:39 +02:00
Matej Ferencevic
e28fd2025d Rename bolt::DecodedValue to bolt::Value
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1506
2018-07-24 15:59:20 +02:00
Dominik Gleich
0681040395 Durability distributed wal
Summary:
Recover only snapshot

Return recovery info on worker recovery

Update tests

Start wal recovery

Single node wal split

Keep track of wal possible to recover

Fix comment

Wal tx intersection

Merge branch 'master' into sync_wal_tx

Reviewers: buda, ipaljak, dgleich, vkasljevic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1489
2018-07-24 08:55:13 +02:00
Matej Ferencevic
653ab6121c Increase code coverage timeout
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1487
2018-07-11 14:50:14 +02:00
Teon Banek
d7a9c5bab8 Extract TypedValue/DecodedValue conversion to higher component
Summary:
This is the first step in cutting the crazy dependencies of
communication module to the whole database. Includes have been
reorganized and conversion between DecodedValue and other Memgraph types
(TypedValue and PropertyValue) has been extracted to a higher level
component called `communication/conversion`. Encoder, like Decoder, now
relies only on DecodedValue. Hopefully the conversion operations will
not significantly slow down streaming Bolt data.

Additionally, Bolt ID is now wrapped in a class. Our storage model uses
*unsigned* int64, while Bolt expects *signed* int64. The implicit
conversions may lead to encode/decode errors, so the wrapper should
enforce some type safety to prevent such errors.

Reviewers: mferencevic, buda, msantl, mtomic

Reviewed By: mferencevic, mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1453
2018-07-11 12:51:31 +02:00
Marko Budiselic
888c6a4bca Add support for the id function
Reviewers: dgleich, teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: dgleich, mferencevic, teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1462
2018-07-04 22:14:13 +02:00
Dominik Gleich
b49eabc432 Fix warnings
Reviewers: teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1470
2018-07-04 16:39:58 +02:00
Matej Ferencevic
7924a94e77 Add overwrite flag to mg_import_csv
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1442
2018-06-21 15:04:41 +02:00
Teon Banek
2fbf2c7ff4 Handle dependent branches in basic Cartesian
Summary:
This change should correctly plan Cartesian which have dependent Filter
or Expand operators. Tests have been added for those cases. Other cases
are not yet supported and should throw an exception.

Reviewers: msantl, mtomic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1426
2018-06-19 13:06:34 +02:00