Commit Graph

2841 Commits

Author SHA1 Message Date
Teon Banek
3a7bd6236e Set -Werror=return-stack-address in CMakeLists
Summary:
This is a clang only flag, it promotes warnings to errors when returning
an address to something on the stack

Reviewers: mferencevic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2394
2019-09-18 10:04:34 +02:00
Teon Banek
04c037e0fd Add PluginRegistry for openCypher custom procedures
Summary:
This diff implements a mechanism for registering plugins which provide
custom procedures for openCypher. Although the `Plugin` struct already
stores some function pointers, these are not set in stone w.r.t. to
requirements and signatures.

For example, in the future, we may want to allow a single plugin to
register multiple custom procedures instead of just one.

Reviewers: ipaljak, dsantl, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2386
2019-09-17 12:46:45 +02:00
Matej Ferencevic
fe2196b403 Add more file utils
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2384
2019-09-13 15:24:52 +02:00
Matej Ferencevic
9652f8e265 Add clear method to SkipList
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2385
2019-09-13 15:24:14 +02:00
Matej Ferencevic
55d8b98aeb Prepare storage v2 config for more options
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2382
2019-09-13 12:42:23 +02:00
Matej Ferencevic
51fc4df87a Add utils::InputFile wrapper
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2377
2019-09-12 16:30:11 +02:00
Matej Ferencevic
b74141aafe Fix accessor locking in storage v2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2376
2019-09-12 13:11:12 +02:00
Teon Banek
7bd45f8714 Make query execution work with storage_v2
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2203
2019-09-12 10:22:00 +02:00
Matej Ferencevic
68f19df305 Migrate LDAP integration to auth module
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2368
2019-09-11 17:15:08 +02:00
Matej Ferencevic
73379bc57e Add functions for listing indices and constraints
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2371
2019-09-11 15:45:14 +02:00
Teon Banek
6d9933373e Add missing include to scoped_profile.hpp
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2370
2019-09-11 12:46:37 +02:00
Ivan Paljak
deee3b8ab7 Remove Raft's dependency on transaction id
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2338
2019-09-10 18:02:05 +02:00
Teon Banek
85b01f8497 Add query/db_accessor.hpp which wraps storage implementation
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2361
2019-09-10 15:43:07 +02:00
Matej Ferencevic
13daf1efd6 Implement auth module
Summary:
This diff adds support for an auth module. The module is used to provide
authentication and authorization (only user to role mappings). The module can
be written in any language and uses a simple protocol to communicate with
Memgraph.

Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2359
2019-09-10 14:46:06 +02:00
Teon Banek
1000a5e544 Use BasicResult instead of optional for v2 Commit
Summary: Depends on D2365

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2366
2019-09-10 13:32:54 +02:00
Teon Banek
51d8d16114 Add utils::BasicResult specialization for void
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2365
2019-09-10 13:32:54 +02:00
Teon Banek
f02617f2bd Replace query::GraphView with storage::View
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2356
2019-09-06 14:58:27 +02:00
Teon Banek
ec22026fed Add glue::ToBolt(Vertex|Edge|Path|Value) for V2 storage
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2355
2019-09-06 10:51:10 +02:00
Teon Banek
2e51ef9f66 Use storage/v2/view in glue/communication
Summary:
Switching to Storage V2 API will require passing storage::View when
serializing VertexAccessor and EdgeAccessor, so this is just the first
step in adapting the code.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2352
2019-09-05 10:04:44 +02:00
Teon Banek
9a94205a07 Document exceptions and other details in storage v2 API
Summary:
Most instances of `@throw std::bad_alloc` are left unexplained as these
functions perform general heap allocations are it's obvious from the
function name that it will do so. Basically anything with `Create`, `Make` or
`Build` implies allocations. Additionally, which parts exactly perform
allocations are an implementation detail. Functions which do unexpected
heap allocations have the reason stated in the documentation, these
functions typically have exactly one spot which could raise such an
exception.

Some functions are marked as `noexcept`, these are usually "special
functions" such as constructors and operators. This could potentially
improve performance because STL may use API overloads that work faster
with `noexcept` stuff. Remaining non-throwing functions aren't marked as
`noexcept` as that wasn't our practice nor is common in our codebase. On
the other hand, if we continue enforcing the documentation of thrown
exceptions, perhaps we should start using `noexcept`.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2350
2019-09-05 10:04:40 +02:00
Teon Banek
02722a1ed5 Add default constructor and std::hash to v2/id_types
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2349
2019-09-03 16:13:52 +02:00
Teon Banek
7213bec886 Make Gid a full on class wrapper around uint64_t
Summary:
This makes Gid the same as the one in storage/v2. Before they can be
merge into one implementation, we probably want to have a similar
transition for remaining ID types.

Depends on D2346

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2347
2019-09-03 15:14:21 +02:00
Teon Banek
cdfaf0d4a6 Move gid::Gid to storage/common/types
Summary:
It never made sense that a global ID is its own namespace in the storage
directory tree.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2346
2019-09-03 14:57:44 +02:00
Ivan Paljak
58be850d5c Fix HA log compaction test
Summary: The test now checks if the cluster is alive at the end of the test.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2343
2019-09-03 08:45:53 +02:00
Ivan Paljak
1dccf02e1b Fix failed assertion in HA snapshot recovery
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2345
2019-09-03 08:10:33 +02:00
Teon Banek
726ae5da6a Check for aggregations when planning RegexMatch
Reviewers: mferencevic, ipaljak, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2344
2019-09-02 17:18:47 +02:00
Matej Ferencevic
60367a287e Implement wrappers for STL containers with our allocator
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2342
2019-09-02 17:04:15 +02:00
Matej Ferencevic
a4a74934f4 Improve memory allocations in execution and storage
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2340
2019-09-02 17:03:58 +02:00
Teon Banek
c8340ad120 Merge old and new PropertyValue implementations
Summary:
This effectively replaces the old PropertyValue implementation from the
one in storage/v2

Depends on D2333

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2335
2019-09-02 16:43:50 +02:00
Teon Banek
32ae2b4d23 Correctly release PoolResource memory
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2341
2019-09-02 15:49:57 +02:00
Teon Banek
ed802073f6 Make PropertyValue constructors explicit
Summary: Depends on D2332

Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2333
2019-09-02 12:31:53 +02:00
Matej Ferencevic
d3b141f3ba Allocate initial execution memory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2316
2019-09-02 11:40:38 +02:00
Teon Banek
9f4a7dcddf Remove PropertyValue::Null
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2332
2019-08-29 09:02:51 +02:00
Teon Banek
02fb644bdc Resolve unused variables
Reviewers: ipaljak, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2330
2019-08-27 16:10:00 +02:00
Teon Banek
a878a11e70 Add FunctionContext to simplify awesome function signature
Reviewers: ipaljak, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2327
2019-08-27 13:49:40 +02:00
Teon Banek
023538c19c Add PoolResource benchmarking in execution and skiplist
Summary:
With a pool allocator, lookups in STL set and map are up to 50% faster.
This is probably due to contiguous memory of pooled objects, i.e. nodes
of those containers. In some cases, the lookup outperforms the SkipList.
Insertions are also faster, though not as dramatically, up to 30%. This
does make a significant difference when the STL containers are used in a
single thread as they outperform the SkipList significantly.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2326
2019-08-27 09:32:01 +02:00
Teon Banek
1fa9d7752c Fix a bug in Pool::Deallocate
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2325
2019-08-27 09:32:00 +02:00
Teon Banek
bb832de608 Prepend MonotonicBufferResource::Buffer to an allocation
Summary:
This is a different scheme for setting up a bookkeeping object while
still supporting arbitrary allocation alignment requests. The previous
scheme was simpler as it always allocated a power of 2 bytes, but the
trade-off was increased memory usage. This should waste less memory.

Reviewers: mtomic, mferencevic, ipaljak

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2321
2019-08-23 14:18:14 +02:00
Marin Tomic
7e741b8d25 Implement vertex count estimation
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2306
2019-08-22 17:34:00 +02:00
Teon Banek
97b20a9384 Use non-template Value invocation
Reviewers: mtomic, mferencevic

Reviewed By: mtomic, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2317
2019-08-22 16:20:31 +02:00
Matej Ferencevic
529b7109f7 Use new SkipList for query caches
Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2305
2019-08-22 14:10:41 +02:00
Matej Ferencevic
a9d2f4842f Reduce GC CPU usage in storage v2
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2313
2019-08-22 14:09:56 +02:00
Marin Tomic
21d461e77d Skip deleted vertices when building index
Summary:
There's a possible race condition where we add a deleted vertex into
index and garbage collection removes it from main storage before indices are
cleaned-up.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2314
2019-08-22 10:25:44 +02:00
Teon Banek
4f93632fb8 storage/v2: Add (In|Out)Degree to VertexAccessor
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2309
2019-08-21 12:58:00 +02:00
Marin Tomic
1ab0b8e0ff Implement existence constraints
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2307
2019-08-20 16:27:36 +02:00
Teon Banek
b8e964a82d Remove is_transaction_starter_ from Storage::Accessor
Reviewers: mferencevic, mtomic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2308
2019-08-20 12:43:05 +02:00
Matej Ferencevic
d263d0887f Fix bug in storage v2 GC
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2304
2019-08-19 13:21:35 +02:00
Teon Banek
ce9e2614fa Invert the result of SetProperty and add documentation
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2297
2019-08-13 16:21:33 +02:00
Matija Santl
b684e697fd Adjust raft constants for long running tests
Summary:
Based on the recent builds (https://apollo.memgraph.io/runs/642045/,
https://apollo.memgraph.io/runs/642700/ https://apollo.memgraph.io/runs/640896/
https://apollo.memgraph.io/runs/640538/) `ha` stress tests fail.

Every tests that failed, has a re-election in progress. I'm bumping election
timeout constatns in order to give the cluster more time to do what it needs to
do before going in the state where it fails.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2292
2019-08-13 10:40:24 +02:00
Teon Banek
f7575eb112 Add utilities for type checking awesome MG functions
Reviewers: mtomic, msantl

Reviewed By: mtomic, msantl

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2259
2019-08-12 16:24:24 +02:00