Commit Graph

20 Commits

Author SHA1 Message Date
Dominik Gleich
d9f25cc668 Write committed/aborted op to wal
Summary:
Wal on workers didn't contain committed transactions ids, this is needed for
distributed recovery so that the master may decide which transactions are
present on all the workers.

Reviewers: buda, msantl

Reviewed By: buda

Subscribers: pullbot, msantl, buda

Differential Revision: https://phabricator.memgraph.io/D1440
2018-07-05 12:43:18 +02:00
Dominik Gleich
5b7947cc18 Revert "Remote vertex with gid creation"
Summary: This reverts commit 7d161319f0.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1410
2018-05-30 14:44:50 +02:00
Dominik Gleich
dd04aa7631 Revert "Remote edge with gid creation"
Summary: This reverts commit adda7d1200.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1409
2018-05-30 13:41:30 +02:00
Dominik Gleich
adda7d1200 Remote edge with gid creation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1402
2018-05-25 09:57:55 +02:00
Dominik Gleich
7d161319f0 Remote vertex with gid creation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1401
2018-05-23 10:10:14 +02:00
Marin Tomic
632663dca5 Fix address handling in remote edge creation
Summary: Global address of `from` was stored in Edge record when created via CreateEdge RPC.

Reviewers: buda, msantl, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1387
2018-05-07 13:20:51 +02:00
florijan
ac8c96ccc2 Tidyup distributed stuff naming
Summary:
Remove "produce_" and "Produce" as prefix from all distributed stuff.
It's not removed in src/query/ stuff (operators).

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1315
2018-03-23 16:32:29 +01:00
florijan
a39602093d Rename Find[Vertex|Edge][Checked]
Summary:
Find[Vertex|Edge] -> Find[Vertex|Edge]Optional
Find[Vertex|Edge]Checked -> Find[Vertex|Edge]

In some places change old code that finds-optional and immediately checks
to use the checked functionality.

It seems that in all the src/ stuff optional finds are no loger used,
only in tests, but there they are used extensively so I don't feel those
functions should be removed.

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1309
2018-03-21 11:15:59 +01:00
florijan
7dbee00f2f Fix updating a newly created edge
Reviewers: msantl, teon.banek, dgleich

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1299
2018-03-14 12:49:24 +01:00
Dominik Gleich
d5b9a11e87 DetachRemoveVertex remote
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1264
2018-03-02 10:51:26 +01:00
florijan
e61bb8ab1e Support distributed edge removal
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1256
2018-02-28 14:35:40 +01:00
Dominik Gleich
99375a4b47 Vertex removal using rpcs
Summary:
Remove vertex remote

Add tests

Reviewers: florijan, teon.banek

Reviewed By: florijan

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1230
2018-02-28 11:35:44 +01:00
Dominik Gleich
ef6cfc4c0e Refactor state deltas call
Summary:
Add label test

Index gets updated after remote update

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1246
2018-02-26 15:44:50 +01:00
florijan
dc21511e6e Fix remote edge create
Summary:
Difficult to test properly as the problem was an implicit conversion of
`gid::Gid` to `mvcc::VersionList<> *`. The only proper way to defend
against this would be to make `gid::Gid` a type.

The `CHECK` added in this diff does not help, but should be there, so...

Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1244
2018-02-26 14:00:45 +01:00
Dominik Gleich
753aa07cdf Tests and fix remote update already deleted bug
Summary:
Updating a record locally while there is an remote update waiting to be applied caused
the operation to return as already deleted, instead of applying it

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1226
2018-02-22 16:36:22 +01:00
florijan
0c40c67ac2 Implement remote create (storage, RPC, not operator)
Summary:
Implementation of remote vertex and edge creation. This diff addresses
the creation API (`GraphDbAccessor::InsertEdge`,
`GraphDbAccessor::InsertRemoteVertex`) and the necessary RPC and
`RemoteCache` stuff.

What is missing for full remote creation support are
`query::plan::operator` changes that are expected to minor. Pushing this
diff as it's large enough, operator and end to end tests in the next.

Also, the naming of existing structures and files is confusing (update
refering to both updates and created, `results` used too often etc.). I
will address this too, but feel free to comment on bad naming.

Reviewers: dgleich, teon.banek, msantl

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1210
2018-02-21 09:17:48 +01:00
florijan
9721ccf61c Cleanup per-transaction caches in distributed
Summary:
On the master cleanups are hooked directly into the transaction engine.
This is beneficial because the master might have bigger caches and we
want to clear them as soon as possible.

On the workers there is a periodic RPC call to the master about living
transactions, which takes care of releasing local caches. This is
suboptimal because long transactions will prevent cache GC (like with
data GC). It is however fairly simple.

Note that all cleanup is not done automatically and `RemotePull` has
been reduced accordingly. @msantl, please verify correctness and
consider if the code can be additionally simplified.

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot, msantl

Differential Revision: https://phabricator.memgraph.io/D1202
2018-02-16 15:30:05 +01:00
florijan
81e2e8f64f Add remote updates RPC
Summary:
Updates are supported, insertions and removals not in this diff. The
test is a bit overdesigned, it happens.

Reviewers: teon.banek, dgleich, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1176
2018-02-07 15:29:57 +01:00
florijan
f84b0b0787 Fix address serialization
Summary:
Defined a new test based on reported bug, for multiple remote expansion.
Fixed the bug. Introduced minor refactors in distributed unit testing.

Reviewers: mculinovic, dgleich

Reviewed By: mculinovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1173
2018-02-06 11:01:06 +01:00
florijan
f808252142 Prepare RecordAccessor for distributed, part one
Summary:
This diff consolidates local and remote update handling. It ensures and
tests that updates for remote elements are visible locally (on the
updating worker).

The next part will be accumulating remote updates and applying them on
the owner.

Also extracted a common testing fixture.

Reviewers: dgleich, buda, mtomic

Reviewed By: mtomic

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1169
2018-02-05 09:48:50 +01:00