Commit Graph

64 Commits

Author SHA1 Message Date
Marko Budiselić
b42e47b0be
Reduce the size of TypedValue (#560)
- Reduce the size of TypedValue
- Fix double allocation
- Add `Graph` to `TypedValue` unit tests
- Fix allocator usage in `TypedValue`
- Add graph projection to `long_running.cpp` stress test
2022-09-20 14:21:34 +02:00
János Benjamin Antal
0bc298c3ad
Fix handling of the ROUTE Bolt message (#475)
The fields of ROUTE message were not read from the input buffer, thus the
input buffer got corrupted. Sending a new message to the server would result
reading the remaining fields from the buffer, which means reading some values
instead of message signature. Because of this unmet expectation, Memgraph closed
the connection. With this fix, the fields of the ROUTE message are properly
read and ignored.
2022-08-26 13:19:27 +02:00
Marko Budiselić
80e0e439b7
Add parser stress test (#463) 2022-07-26 20:54:56 +02:00
jbajic
12b4ec1589 Add memgraph namespace 2022-03-14 15:47:41 +01:00
Antonio Andelic
bd21bc82b7
Add license to cpp/hpp/py test files (#283) 2021-10-26 08:53:56 +02:00
antonio2368
3f3c55a4aa
Format all the memgraph and test source files (#97) 2021-02-18 15:32:43 +01:00
Marko Budiselić
90d4ebdb1e
Polish CI scripts (#83)
* Add Jepsen to the release workflow
* Increase the long-running stress timeout to 16 minutes
* Remove ha_client code
2021-01-26 18:57:23 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72)
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
Marko Budiselić
573b6cb045
Clean and upgrade openCypher and stress tests (#27)
* Add initial behave test improvements
* Make HTML human-readable
* Remove tutorial tests
* Migrate to neo4j-driver==4.1.1
2020-11-24 13:09:14 +01:00
antonio2368
0a7d4278b1
Fix older version of SSL (#24)
* Add manual locking callback for SSL

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2020-10-20 12:55:13 +02:00
Matej Ferencevic
e0ffc533b9
Remove leftover Apollo files (#1) 2020-09-21 14:58:52 +02:00
Matej Ferencevic
dfc546183d Add community to Apollo build
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2671
2020-02-17 13:27:20 +01:00
Matej Ferencevic
3e20afde6e Increase stress test query timeout
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2657
2020-02-05 14:52:13 +01:00
Matej Ferencevic
84a6ab75cb Fix race condition in long running stress test
Summary:
The long running stress test had a subtle race condition which caused the test
to fail with an error message like "Runner X edges creation failed because of:
Can't serialize due to concurrent operations.". This situation was caused
because some workers could complete their initialization (initial vertex and
edge creation) before other workers. The workers that completed their
initialization would then proceed to execute the test. In the test they could
execute queries that make global updates on the graph that could interfere with
the concurrently running initialization queries of other workers.

This diff makes the runners wait until all initialization queries are fully
executed before they execute global operations on the graph.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2647
2020-01-29 10:19:53 +01:00
Matej Ferencevic
4776bea221 Temporarily disable HA build and tests
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2619
2020-01-13 17:04:27 +01:00
Matej Ferencevic
b3d1cd8257 Improve long running stress test
Summary:
The test is now more strict than before. Each verification step doesn't just
verify that object counts are correct, it now also verifies that all object IDs
are correct. The continuous integration script is improved to have a more
deterministic startup.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2606
2019-12-23 14:18:52 +01:00
Matej Ferencevic
eb38b4f373 Rename Bolt flags
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2587
2019-12-09 11:24:09 +01:00
Matej Ferencevic
8a0abd6dbd Make memgraph with storage v2 the main binary
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2567
2019-12-06 10:51:16 +01:00
Matija Santl
b684e697fd Adjust raft constants for long running tests
Summary:
Based on the recent builds (https://apollo.memgraph.io/runs/642045/,
https://apollo.memgraph.io/runs/642700/ https://apollo.memgraph.io/runs/640896/
https://apollo.memgraph.io/runs/640538/) `ha` stress tests fail.

Every tests that failed, has a re-election in progress. I'm bumping election
timeout constatns in order to give the cluster more time to do what it needs to
do before going in the state where it fails.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2292
2019-08-13 10:40:24 +02:00
Ivan Paljak
fa4c303af6 Add HA normal operation long running test to daily build
Summary: Currently set to run for 2 hours.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2206
2019-07-18 16:53:07 +02:00
Ivan Paljak
2936312612 Add memgraph ha normal operation long running test
Summary:
The functionality of the test is the same as for single node Memgraph.
Locally, it seems to work fine. I'll update the apollo related files when I feel
a bit more certain that everything works locally.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2135
2019-07-09 13:59:56 +02:00
Matej Ferencevic
9e223cdabd Use only local workers for builds on Apollo
Reviewers: mtomic, buda

Reviewed By: mtomic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2170
2019-07-03 22:30:55 +02:00
Matej Ferencevic
d4b2d76a35 Reimplement counter openCypher function
Reviewers: teon.banek, msantl

Reviewed By: teon.banek, msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2047
2019-05-16 11:09:02 +02:00
Matej Ferencevic
0013cbb173 Polish Bolt client and mg_client
Reviewers: mculinovic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1681
2018-10-19 13:55:51 +02:00
Matej Ferencevic
75950664a7 Separate distributed from single node storage
Summary:
This diff splits single node and distributed storage from each other.
Currently all of the storage code is copied into two directories (one single
node, one distributed).  The logic used in the storage implementation isn't
touched, it will be refactored in following diffs.

To clean the working directory after this diff you should execute:
```
rm database/state_delta.capnp
rm database/state_delta.hpp
rm storage/concurrent_id_mapper_rpc_messages.capnp
rm storage/concurrent_id_mapper_rpc_messages.hpp
```

Reviewers: teon.banek, buda, msantl

Reviewed By: teon.banek, msantl

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1625
2018-10-05 09:19:33 +02:00
Teon Banek
d78022470c Explicitly list all test files in CMakeLists
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1619
2018-09-28 10:18:04 +02:00
Matej Ferencevic
e28fd2025d Rename bolt::DecodedValue to bolt::Value
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1506
2018-07-24 15:59:20 +02:00
Teon Banek
d7a9c5bab8 Extract TypedValue/DecodedValue conversion to higher component
Summary:
This is the first step in cutting the crazy dependencies of
communication module to the whole database. Includes have been
reorganized and conversion between DecodedValue and other Memgraph types
(TypedValue and PropertyValue) has been extracted to a higher level
component called `communication/conversion`. Encoder, like Decoder, now
relies only on DecodedValue. Hopefully the conversion operations will
not significantly slow down streaming Bolt data.

Additionally, Bolt ID is now wrapped in a class. Our storage model uses
*unsigned* int64, while Bolt expects *signed* int64. The implicit
conversions may lead to encode/decode errors, so the wrapper should
enforce some type safety to prevent such errors.

Reviewers: mferencevic, buda, msantl, mtomic

Reviewed By: mferencevic, mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1453
2018-07-11 12:51:31 +02:00
Matej Ferencevic
8cdd6a906a Increase durability large test timeout
Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1454
2018-06-27 12:09:37 +02:00
Matej Ferencevic
1d448d40ca Implement SSL support for servers and clients
Summary:
This diff implements OpenSSL support in the network stack.
Currently SSL support is only enabled for Bolt connections,
support for RPC connections will be added in another diff.

Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1328
2018-06-20 17:56:47 +02:00
Ivan Paljak
035540c598 POD serialization, rocksdb integration and Gleich's optimization
Reviewers: buda, dgleich, mferencevic, msantl, teon.banek

Reviewed By: buda, dgleich, teon.banek

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1399
2018-06-12 13:54:41 +02:00
Marko Budiselic
44474b55e3 Add durability stress test
Reviewers: dgleich, mferencevic

Reviewed By: dgleich, mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1371
2018-04-22 08:32:44 +02:00
Matej Ferencevic
c4a0da6054 Refactor network clients to use a single client implementation
Summary: All network clients (Bolt & RPC) now use a wrapper.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1324
2018-03-28 10:09:58 +02:00
Matej Ferencevic
b2f3bf9709 Refactor Apollo build projects
Summary: Fix LDBC run

Reviewers: buda, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1317
2018-03-27 13:47:18 +02:00
Matej Ferencevic
2a68543f3e Change Apollo slave groups
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1238
2018-02-25 23:02:31 +01:00
Dominik Gleich
5418dfb19e Rename NetworkEndpoint
Summary:
Rename redunant port str

Add endpoint << operator

Migrate everything to endpoint

Reviewers: mferencevic, florijan

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1100
2018-01-15 15:47:37 +01:00
Matej Ferencevic
499ad3ba15 Add initial version of Apollo config files
Summary:
Improve Apollo config files

Add name to apollo_build

Remove old generate script from build

Add build_release symlink to release build

Rename 'args' to 'arguments'

Add run definition for cppcheck

Host doxygen documentation

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1095
2018-01-10 14:46:10 +01:00
Matej Ferencevic
82bede9a97 Implement measurements collection in tests
Summary:
Implement measurements collection in QA

Implement measurements collection in LDBC

Implement measurements collection in stress tests

Reviewers: mislav.bradac, buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1087
2018-01-01 17:49:37 +01:00
Mislav Bradac
73c01cfb73 Reuse sessions in bipartite
Reviewers: florijan, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1028
2017-12-05 15:55:35 +01:00
Dominik Gleich
73c1206e81 Remove extension from Python executables
Reviewers: mferencevic, buda

Reviewed By: buda

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D988
2017-11-27 14:39:51 +01:00
florijan
8bbf1af525 Cleanup durability config, docs, CHANGELOG
Reviewers: teon.banek, buda, mislav.bradac, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D992
2017-11-21 10:17:13 +01:00
florijan
5525af558b Long running test - RemoveEdges never called
Summary: RemoveEdges is an extremely slow operation because it iterates over all the vertices to find the appropriate edge. It kind of messes up the DB usage. This diff stops it ever getting called, but does not delete the function. We might want it to happen **very rarely**, but it's probably best never to call it.

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D910
2017-10-17 11:59:39 +02:00
Teon Banek
06b0111ddc Remove unused stuff from CMakeLists
Summary: In the process, make experimental/distributed compilable.

Reviewers: mislav.bradac, buda, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D906
2017-10-16 13:45:18 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Matej Ferencevic
def612622e Decreased long running test size.
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D880
2017-10-08 14:19:53 +02:00
Teon Banek
4c25123d83 Make docker installation equivalent to regular install
Summary:
This puts the whole installation and packaging under a single point of
entry. (Docker, DEB, RPM, etc.)

Rename alpha.dockerfile to beta.dockerfile
Use Debian Stretch for docker
Remove building old hardcoded compiler
Rename build_interpreter to build_memgraph
Remove unused config-file

Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D857
2017-10-06 13:23:34 +02:00
Matej Ferencevic
e6c3d42e02 Changed long running stress parameters.
Summary:
Increased query execution timeout.

Enabled global queries by default.

Implemented faster RandomElement for vertices and edges.

Changed long running verify message format.

Changed vertex and edge count to be per worker.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D864
2017-10-05 22:10:39 +02:00
florijan
8e1626d4fb LongRunning - batched edge creation improved. Verification before loop
Summary:
Batch creation now does not iterate over V**2 vertex pairs, but over V vertices and generates E/V random edges for each one. Assuming there is an index over :V(id), this is much more efficient and supports larger vertex counts (Ferenc: modify the test config as desired).

An additional requirement is that E/V (for each worker) is a whole number.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D860
2017-10-05 13:51:54 +02:00
Matej Ferencevic
d249ff11a5 Misc glog and gflags fixes.
Summary:
Log files aren't created by default anymore.
All logs are reported to stderr by default.
Normalized flag names.
Removed unnecessary flags from gflags.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D856
2017-10-04 17:29:53 +02:00
Matej Ferencevic
73052dcff4 Changed long running large stress parameters.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D853
2017-10-04 09:57:42 +02:00