Commit Graph

70 Commits

Author SHA1 Message Date
Dominik Gleich
24857cc1cf Support distributed (label, property) indexes
Summary:
Call workers buildindex

Merge branch 'master' into setup_distributed_index

Use ExecuteOnWorkers api

Merge branch 'master' into setup_distributed_index

Improve test

Merge branch 'master' into setup_distributed_index

Finish test

Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1134
2018-01-25 18:08:04 +01:00
florijan
ccefae4002 Add distributed execution RPC
Summary:
NOTE: This diff is still in progress. Many TODOs, lacking documentation
etc. But the main logic is there (some could be different), and it tests
out OK.

Reviewers: teon.banek, msantl, buda

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1138
2018-01-25 14:50:28 +01:00
Matej Ferencevic
fc20ddcd25 RPC refactor
Summary:
Start removal of old logic
Remove more obsolete classes
Move Message class to RPC
Remove client logic from system
Remove messaging namespace
Move protocol from messaging to rpc
Move System from messaging to rpc
Remove unnecessary namespace
Remove System from RPC Client
Split Client and Server into separate files
Start implementing new client logic
First semi-working state
Changed network protocol layout
Rewrite client
Fix client receive bug
Cleanup code of debug lines
Migrate to accessors
Migrate back to binary boost archives
Remove debug logging from server
Disable timeout test
Reduce message_id from uint64_t to uint32_t
Add multiple workers to server
Fix compiler warnings
Apply clang-format

Reviewers: teon.banek, florijan, dgleich, buda, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1129
2018-01-24 15:27:40 +01:00
Matija Santl
31aded2dae Add test for PlanDispatcher and PlanConsumer
Summary:
Added test for `PlanDispatcher` and `PlanConsumer`.
This diff also contains a fix for the async rpc call on all clients.h

Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: pullbot, dgleich

Differential Revision: https://phabricator.memgraph.io/D1135
2018-01-24 12:38:52 +01:00
Matija Santl
62323965e3 Implement distributed plan dispatcher/consumer methods
Summary:
Implementations of `DistributePlanRpc`.
I'll add tests afterwards #promise

Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1131
2018-01-24 10:50:27 +01:00
Dominik Gleich
ca32538f63 Fix set of clients
Reviewers: florijan

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D1133
2018-01-23 13:08:52 +01:00
Dominik Gleich
5ad39a8735 Fix ExecuteOnWorkers
Reviewers: florijan

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D1132
2018-01-23 11:29:39 +01:00
Dominik Gleich
5a698444bf Add ExecuteOnWorkers
Reviewers: florijan

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D1130
2018-01-23 10:56:52 +01:00
Matija Santl
3bee31d8f3 Add PlanDispatcher and PlanConsumer stubs
Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1126
2018-01-23 10:15:24 +01:00
Dominik Gleich
ca9fac8adc Expose worker ids
Summary: Add worker ids expose to rpc_worker_clients

Reviewers: msantl

Reviewed By: msantl

Differential Revision: https://phabricator.memgraph.io/D1128
2018-01-22 17:09:12 +01:00
florijan
912d178391 Extract RpcWorkerClients
Summary: Extracting the RPC client-per-worker functionality we'll commonly need.

Reviewers: teon.banek, msantl, dgleich

Reviewed By: dgleich

Differential Revision: https://phabricator.memgraph.io/D1127
2018-01-22 16:27:05 +01:00
florijan
35197d6c4b Add distributed record TODOs
Summary:
It seems that RecordAccessor &co are ready for read-only distributed
execution. In read-only there is no command advancement and the implied
cache invalidation, `SwitchOld` and `SwitchNew` perform default
switching and `Reconstruct` uses the `RemoteCache` which is implemented.

I just added a few TODOs for proper CRUD.

Reviewers: dgleich

Reviewed By: dgleich

Differential Revision: https://phabricator.memgraph.io/D1125
2018-01-22 14:55:45 +01:00
florijan
e1e4a70714 Implement graph element rpc
Summary:
- End to end distributed GraphDb testing
- Refactors as necessary
- Basic RemoteCache for storing remote data
- RemoteDataRpc

As we are on a tight schedule, please let's focus on the essentials:
functionality and proper testing.

Reviewers: dgleich, teon.banek, buda

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1121
2018-01-22 14:20:41 +01:00
florijan
813d37e939 Migrate db::types to storage::
Reviewers: teon.banek, dgleich

Reviewed By: teon.banek, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1110
2018-01-17 10:35:12 +01:00
Dominik Gleich
5418dfb19e Rename NetworkEndpoint
Summary:
Rename redunant port str

Add endpoint << operator

Migrate everything to endpoint

Reviewers: mferencevic, florijan

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1100
2018-01-15 15:47:37 +01:00
florijan
6fc6a27288 Refactor GraphDb
Summary:
GraphDb is refactored to become an API exposing different parts
necessary for the database to function. These different parts can have
different implementations in SingleNode or distributed Master/Server
GraphDb implementations.

Interally GraphDb is implemented using two class heirarchies. One
contains all the members and correct wiring for each situation. The
other takes care of initialization and shutdown. This architecture is
practical because it can guarantee that the initialization of the
object structure is complete, before initializing state.

Reviewers: buda, mislav.bradac, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1093
2018-01-12 16:47:24 +01:00
Dominik Gleich
007a7f1a6d Change network design from Start/Shutdown to constructor/destructor
Summary:
Make ServerT start on constructor

Remove shutdown from MasterCoordinator

Distributed system remove Shutdown

Rcp server start and shutdown removed

Reviewers: florijan, mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1097
2018-01-10 14:58:57 +01:00
florijan
3cf81380b8 Implement vertex/edge serialization
Summary:
Serialization of vertices and edges for distributed. Based on Boost
serialization. Threrefore moved TypedValue serialization from AST to
utils.

Reviewers: buda, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1088
2018-01-10 13:32:18 +01:00
Mislav Bradac
d3623585e7 Migrate cereal to boost_serialization
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1077
2017-12-27 13:44:46 +01:00
florijan
3ae45e0d19 Add master/worker flags, main functions and coordination
Reviewers: dgleich, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1060
2017-12-19 16:05:57 +01:00