Commit Graph

37 Commits

Author SHA1 Message Date
Teon Banek
d7a9c5bab8 Extract TypedValue/DecodedValue conversion to higher component
Summary:
This is the first step in cutting the crazy dependencies of
communication module to the whole database. Includes have been
reorganized and conversion between DecodedValue and other Memgraph types
(TypedValue and PropertyValue) has been extracted to a higher level
component called `communication/conversion`. Encoder, like Decoder, now
relies only on DecodedValue. Hopefully the conversion operations will
not significantly slow down streaming Bolt data.

Additionally, Bolt ID is now wrapped in a class. Our storage model uses
*unsigned* int64, while Bolt expects *signed* int64. The implicit
conversions may lead to encode/decode errors, so the wrapper should
enforce some type safety to prevent such errors.

Reviewers: mferencevic, buda, msantl, mtomic

Reviewed By: mferencevic, mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1453
2018-07-11 12:51:31 +02:00
Matej Ferencevic
8cdd6a906a Increase durability large test timeout
Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1454
2018-06-27 12:09:37 +02:00
Matej Ferencevic
1d448d40ca Implement SSL support for servers and clients
Summary:
This diff implements OpenSSL support in the network stack.
Currently SSL support is only enabled for Bolt connections,
support for RPC connections will be added in another diff.

Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1328
2018-06-20 17:56:47 +02:00
Ivan Paljak
035540c598 POD serialization, rocksdb integration and Gleich's optimization
Reviewers: buda, dgleich, mferencevic, msantl, teon.banek

Reviewed By: buda, dgleich, teon.banek

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1399
2018-06-12 13:54:41 +02:00
Marko Budiselic
44474b55e3 Add durability stress test
Reviewers: dgleich, mferencevic

Reviewed By: dgleich, mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1371
2018-04-22 08:32:44 +02:00
Matej Ferencevic
c4a0da6054 Refactor network clients to use a single client implementation
Summary: All network clients (Bolt & RPC) now use a wrapper.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1324
2018-03-28 10:09:58 +02:00
Matej Ferencevic
b2f3bf9709 Refactor Apollo build projects
Summary: Fix LDBC run

Reviewers: buda, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1317
2018-03-27 13:47:18 +02:00
Matej Ferencevic
2a68543f3e Change Apollo slave groups
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1238
2018-02-25 23:02:31 +01:00
Dominik Gleich
5418dfb19e Rename NetworkEndpoint
Summary:
Rename redunant port str

Add endpoint << operator

Migrate everything to endpoint

Reviewers: mferencevic, florijan

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1100
2018-01-15 15:47:37 +01:00
Matej Ferencevic
499ad3ba15 Add initial version of Apollo config files
Summary:
Improve Apollo config files

Add name to apollo_build

Remove old generate script from build

Add build_release symlink to release build

Rename 'args' to 'arguments'

Add run definition for cppcheck

Host doxygen documentation

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1095
2018-01-10 14:46:10 +01:00
Matej Ferencevic
82bede9a97 Implement measurements collection in tests
Summary:
Implement measurements collection in QA

Implement measurements collection in LDBC

Implement measurements collection in stress tests

Reviewers: mislav.bradac, buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1087
2018-01-01 17:49:37 +01:00
Mislav Bradac
73c01cfb73 Reuse sessions in bipartite
Reviewers: florijan, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1028
2017-12-05 15:55:35 +01:00
Dominik Gleich
73c1206e81 Remove extension from Python executables
Reviewers: mferencevic, buda

Reviewed By: buda

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D988
2017-11-27 14:39:51 +01:00
florijan
8bbf1af525 Cleanup durability config, docs, CHANGELOG
Reviewers: teon.banek, buda, mislav.bradac, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D992
2017-11-21 10:17:13 +01:00
florijan
5525af558b Long running test - RemoveEdges never called
Summary: RemoveEdges is an extremely slow operation because it iterates over all the vertices to find the appropriate edge. It kind of messes up the DB usage. This diff stops it ever getting called, but does not delete the function. We might want it to happen **very rarely**, but it's probably best never to call it.

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D910
2017-10-17 11:59:39 +02:00
Teon Banek
06b0111ddc Remove unused stuff from CMakeLists
Summary: In the process, make experimental/distributed compilable.

Reviewers: mislav.bradac, buda, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D906
2017-10-16 13:45:18 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Matej Ferencevic
def612622e Decreased long running test size.
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D880
2017-10-08 14:19:53 +02:00
Teon Banek
4c25123d83 Make docker installation equivalent to regular install
Summary:
This puts the whole installation and packaging under a single point of
entry. (Docker, DEB, RPM, etc.)

Rename alpha.dockerfile to beta.dockerfile
Use Debian Stretch for docker
Remove building old hardcoded compiler
Rename build_interpreter to build_memgraph
Remove unused config-file

Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D857
2017-10-06 13:23:34 +02:00
Matej Ferencevic
e6c3d42e02 Changed long running stress parameters.
Summary:
Increased query execution timeout.

Enabled global queries by default.

Implemented faster RandomElement for vertices and edges.

Changed long running verify message format.

Changed vertex and edge count to be per worker.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D864
2017-10-05 22:10:39 +02:00
florijan
8e1626d4fb LongRunning - batched edge creation improved. Verification before loop
Summary:
Batch creation now does not iterate over V**2 vertex pairs, but over V vertices and generates E/V random edges for each one. Assuming there is an index over :V(id), this is much more efficient and supports larger vertex counts (Ferenc: modify the test config as desired).

An additional requirement is that E/V (for each worker) is a whole number.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D860
2017-10-05 13:51:54 +02:00
Matej Ferencevic
d249ff11a5 Misc glog and gflags fixes.
Summary:
Log files aren't created by default anymore.
All logs are reported to stderr by default.
Normalized flag names.
Removed unnecessary flags from gflags.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D856
2017-10-04 17:29:53 +02:00
Matej Ferencevic
73052dcff4 Changed long running large stress parameters.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D853
2017-10-04 09:57:42 +02:00
Matej Ferencevic
2603209849 Replaced Python long running test with C++ version
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D802
2017-09-18 15:06:22 +02:00
Matej Ferencevic
9d63dfa9ca Added slave groups to Apollo generate.
Summary: Added large stress to daily release.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D786
2017-09-13 17:02:41 +02:00
Matej Ferencevic
5a2086b18f Libraries are now downloaded from our local cache server.
Summary:
Improved libs cleanup script.

The root init script now downloads everything.

Removed obsolete init scripts.

Reviewers: buda, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D710
2017-08-25 09:58:39 +02:00
Matej Ferencevic
1eec3186c4 Stress tests now spawn only one memgraph.
Summary:
Added logging to bipartite and create_match tests.

Renamed thread-count to worker-count.

Changed logging in memgraph.

Reviewers: buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D667
2017-08-18 14:48:54 +02:00
Matej Ferencevic
26c4651897 Removed global state from long_running stress test.
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D637
2017-08-17 10:06:16 +02:00
florijan
31ee8a148f Long running test speedup
Summary: Index added to long running test to speed up edge creation and vertex deletion.

Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D621
2017-08-03 14:48:55 +02:00
Matej Ferencevic
2ba3df942b Added continuous_integration script to stress tests.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D620
2017-08-02 12:06:59 +02:00
Mislav Bradac
68e78c417b NeoRunner + Harness bugfix
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D588
2017-07-29 17:46:24 +02:00
Mislav Bradac
fee59a7ec2 Add first parallel benchmark
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D557
2017-07-17 19:29:29 +02:00
Marko Budiselic
5d9373a22a Stress test small fix.
Summary: Update of tests/stress/create_match.py to use the right functions from tests/stress/common.py.

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D547
2017-07-12 18:22:12 +02:00
Marko Budiselic
5b1fde2a98 Credit Card Fraud Detection Gist
Summary: Add Credit Card Fraud Detection as a TCK feature.

Reviewers: matej.gradicek, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D514
2017-06-29 16:27:49 +02:00
florijan
a726ac0023 Tests - long-running test added
Summary:
Tests - long-running test added

 - common functionality refactor
 - long running tests WIP

tests/stress/common: minor refactor, long_running WIP

tests/stress/long_running - stable single-threaded, unstable concurrent

tests/stress/long_running - added error summary output

test/stress/long_running - add/remove vertex/edge stable multithreaded

test/stress/long_running - added vertex label testing

tests/stress/long_running - time limit added

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D489
2017-06-21 12:10:36 +02:00
Marko Budiselic
28173eaa3e Bipartite stress test.
Summary: Bipartite stress test. Common methods useful in the process of testing with python Bolt driver.

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: florijan, pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D423
2017-06-12 10:47:37 +02:00
Marko Budiselic
1438ff97c1 First stress test version.
Summary: Stress test: create vertices.

Reviewers: mislav.bradac, mferencevic, florijan

Reviewed By: mislav.bradac

Subscribers: teon.banek, pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D402
2017-05-31 15:31:33 +02:00