Commit Graph

357 Commits

Author SHA1 Message Date
Matej Ferencevic
e4edb2be99 Finalize storage v2 index API
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2543
2019-11-12 12:11:38 +01:00
Matej Ferencevic
a850644f5d Fix atomic memory ordering in durability
Summary:
The atomic memory order should be `acquire` for `load` operations, `release`
for `store` operations and `acq_rel` for any RMW (read-modify-write) operations
(like `fetch_add`).

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2540
2019-11-11 12:57:04 +01:00
Matej Ferencevic
9eb1f1d5cd Implement ClearProperties for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2537
2019-11-07 13:07:26 +01:00
Matej Ferencevic
73134ab6b7 Integrate all storage v2 flags into main binary
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2528
2019-11-04 17:01:56 +01:00
Matej Ferencevic
4248b140d4 Backup existing durability files
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2526
2019-11-04 10:38:13 +01:00
Matej Ferencevic
875a4a8629 Implement WAL loading for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2521
2019-10-31 17:12:37 +01:00
Matej Ferencevic
13ba9cc23e Implement WAL writing for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2515
2019-10-29 16:18:12 +01:00
Matej Ferencevic
2e0586e182 Implement single function for WAL loading
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2513
2019-10-28 15:42:49 +01:00
Matej Ferencevic
98855889d5 Implement WalFile for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2510
2019-10-25 16:08:17 +02:00
Matej Ferencevic
3bed1cc3ea Correctly pass config into AllVerticesIterable
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2500
2019-10-21 14:15:32 +02:00
Matej Ferencevic
842f0f3bf6 Add missing check to snapshot recovery
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2490
2019-10-16 13:18:52 +02:00
Matej Ferencevic
3adedc0679 Implement snapshots for storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2400
2019-10-01 13:42:37 +02:00
Matej Ferencevic
3756534490 Don't crash the database when using utils::InputFile
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2460
2019-10-01 10:15:42 +02:00
Matej Ferencevic
8c7c593680 Clean-up of storage v2 delta
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2451
2019-09-27 10:47:28 +02:00
Matej Ferencevic
9ad49698e9 Add support for disabling properties on edges
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2447
2019-09-25 10:17:19 +02:00
Matej Ferencevic
8b6ae08682 Create/drop label indices explicitly in storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2432
2019-09-24 13:12:50 +02:00
Matej Ferencevic
3079522f47 Make storage v2 config hierarchical
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2434
2019-09-23 16:03:53 +02:00
Matej Ferencevic
9d39ecab7b Implement encoder/decoder for durability v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2393
2019-09-18 16:32:32 +02:00
Matej Ferencevic
55d8b98aeb Prepare storage v2 config for more options
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2382
2019-09-13 12:42:23 +02:00
Matej Ferencevic
b74141aafe Fix accessor locking in storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2376
2019-09-12 13:11:12 +02:00
Teon Banek
7bd45f8714 Make query execution work with storage_v2
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2203
2019-09-12 10:22:00 +02:00
Matej Ferencevic
73379bc57e Add functions for listing indices and constraints
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2371
2019-09-11 15:45:14 +02:00
Ivan Paljak
deee3b8ab7 Remove Raft's dependency on transaction id
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2338
2019-09-10 18:02:05 +02:00
Teon Banek
1000a5e544 Use BasicResult instead of optional for v2 Commit
Summary: Depends on D2365

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2366
2019-09-10 13:32:54 +02:00
Teon Banek
f02617f2bd Replace query::GraphView with storage::View
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2356
2019-09-06 14:58:27 +02:00
Teon Banek
9a94205a07 Document exceptions and other details in storage v2 API
Summary:
Most instances of `@throw std::bad_alloc` are left unexplained as these
functions perform general heap allocations are it's obvious from the
function name that it will do so. Basically anything with `Create`, `Make` or
`Build` implies allocations. Additionally, which parts exactly perform
allocations are an implementation detail. Functions which do unexpected
heap allocations have the reason stated in the documentation, these
functions typically have exactly one spot which could raise such an
exception.

Some functions are marked as `noexcept`, these are usually "special
functions" such as constructors and operators. This could potentially
improve performance because STL may use API overloads that work faster
with `noexcept` stuff. Remaining non-throwing functions aren't marked as
`noexcept` as that wasn't our practice nor is common in our codebase. On
the other hand, if we continue enforcing the documentation of thrown
exceptions, perhaps we should start using `noexcept`.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2350
2019-09-05 10:04:40 +02:00
Teon Banek
02722a1ed5 Add default constructor and std::hash to v2/id_types
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2349
2019-09-03 16:13:52 +02:00
Teon Banek
7213bec886 Make Gid a full on class wrapper around uint64_t
Summary:
This makes Gid the same as the one in storage/v2. Before they can be
merge into one implementation, we probably want to have a similar
transition for remaining ID types.

Depends on D2346

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2347
2019-09-03 15:14:21 +02:00
Teon Banek
cdfaf0d4a6 Move gid::Gid to storage/common/types
Summary:
It never made sense that a global ID is its own namespace in the storage
directory tree.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2346
2019-09-03 14:57:44 +02:00
Matej Ferencevic
a4a74934f4 Improve memory allocations in execution and storage
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2340
2019-09-02 17:03:58 +02:00
Teon Banek
c8340ad120 Merge old and new PropertyValue implementations
Summary:
This effectively replaces the old PropertyValue implementation from the
one in storage/v2

Depends on D2333

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2335
2019-09-02 16:43:50 +02:00
Teon Banek
ed802073f6 Make PropertyValue constructors explicit
Summary: Depends on D2332

Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2333
2019-09-02 12:31:53 +02:00
Teon Banek
9f4a7dcddf Remove PropertyValue::Null
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2332
2019-08-29 09:02:51 +02:00
Marin Tomic
7e741b8d25 Implement vertex count estimation
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2306
2019-08-22 17:34:00 +02:00
Teon Banek
97b20a9384 Use non-template Value invocation
Reviewers: mtomic, mferencevic

Reviewed By: mtomic, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2317
2019-08-22 16:20:31 +02:00
Matej Ferencevic
a9d2f4842f Reduce GC CPU usage in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2313
2019-08-22 14:09:56 +02:00
Marin Tomic
21d461e77d Skip deleted vertices when building index
Summary:
There's a possible race condition where we add a deleted vertex into
index and garbage collection removes it from main storage before indices are
cleaned-up.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2314
2019-08-22 10:25:44 +02:00
Teon Banek
4f93632fb8 storage/v2: Add (In|Out)Degree to VertexAccessor
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2309
2019-08-21 12:58:00 +02:00
Marin Tomic
1ab0b8e0ff Implement existence constraints
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2307
2019-08-20 16:27:36 +02:00
Teon Banek
b8e964a82d Remove is_transaction_starter_ from Storage::Accessor
Reviewers: mferencevic, mtomic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2308
2019-08-20 12:43:05 +02:00
Matej Ferencevic
d263d0887f Fix bug in storage v2 GC
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2304
2019-08-19 13:21:35 +02:00
Teon Banek
ce9e2614fa Invert the result of SetProperty and add documentation
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2297
2019-08-13 16:21:33 +02:00
Teon Banek
34f80722c2 Return Error::DELETED_OBJECT from CreateEdge instead of aborting
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2295
2019-08-09 12:38:20 +02:00
Teon Banek
e18cb7b4c7 Make Result constructors implicit
Reviewers: mferencevic, mtomic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2293
2019-08-08 15:47:35 +02:00
Teon Banek
5bae9c6457 Move storage::Result to utils
Summary:
Utils now contain a BasicResult implementation which supports different
types of error. This will make it useful for other parts of the code as
well as during the transition from old to new storage.

Reviewers: mtomic, msantl, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2291
2019-08-08 14:37:06 +02:00
Teon Banek
1a20c557b8 Update storage API docs with thrown exceptions
Summary:
The documentation includes `std` exceptions like `std::bad_alloc` or
`std::system_error`, for which there's probably nothing we can do. This
may seem unnecessary, but it will be really helpful when writing the C
API for interfacing with custom modules and plugins, as well as when
switching to storage v2 API.

In general, we should start updating the documentation of functions
which may throw exceptions. This ought to be enforced in code review, so
that the implementation and documentation are kept in sync.

Reviewers: mferencevic, mtomic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2288
2019-08-07 15:58:38 +02:00
Teon Banek
134525851b Expose Id mappings from accessor-less Storage
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2285
2019-08-06 14:30:21 +02:00
Teon Banek
ed9f82bcf4 Add a common VerticesIterable wrapper
Summary:
This ought to simplify the code which needs to work with any kind of
vertex iteration, be it through an index store or regular.

Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2258
2019-08-06 13:30:31 +02:00
Matej Ferencevic
bad1f8b525 Remove copy constructor/assignment from SkipList accessors
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2230
2019-08-06 11:18:31 +02:00
Teon Banek
fc493e36ff Add lvalue and rvalue getters to storage::Result
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2242
2019-07-26 15:47:14 +02:00