Kostas Kyrimis
ed0b67dfdb
Fix compilation issues of the merge
2022-11-28 13:38:12 +02:00
Kostas Kyrimis
0f34c49e21
Merge branch 'project-pineapples' into T1159-MG-Add-memgraph-functions
2022-11-28 13:18:51 +02:00
Tyler Neely
a8dc6fd41e
Change the namespace of ShardRequestManager to query::v2 instead of msgs
2022-11-28 09:43:56 +00:00
jbajic
36a7abb170
Add remove benchmark
2022-11-27 22:50:36 +01:00
jbajic
41b06a0a37
Extract common functionalities
2022-11-27 22:28:23 +01:00
jbajic
cceab46a7c
Add find benchmark
2022-11-27 21:59:18 +01:00
jbajic
243fa5e4b2
Add insert benchmark
2022-11-27 21:21:55 +01:00
Kostas Kyrimis
6f4996de0e
Fix broken merge and address GH comments
2022-11-25 16:20:38 +02:00
Kostas Kyrimis
01d5953bb6
Merge branch 'project-pineapples' into T1159-MG-Add-memgraph-functions
2022-11-25 15:47:44 +02:00
jbajic
c5138c8d58
Add b+ tree
2022-11-24 17:45:43 +01:00
jbajic
8fe1b8d7fc
Add std::map and skiplist benchamrk
2022-11-24 17:44:41 +01:00
Kostas Kyrimis
7a3caa320c
WiP
2022-11-24 15:29:51 +02:00
Kostas Kyrimis
07a8ac0db8
Merge branch 'project-pineapples' into T0919-MG-implement-get-properties-storage
2022-11-24 15:29:05 +02:00
jbajic
7aa68164f0
Merge branch 'project-pineapples' into T0995-MG-implement-top-error-handling-storage
2022-11-23 21:55:08 +01:00
jbajic
b3eec92525
Merge branch 'project-pineapples' into T0995-MG-implement-top-error-handling-storage
2022-11-23 16:11:58 +01:00
jeremy
e77843f2ec
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-11-23 14:47:29 +01:00
Kostas Kyrimis
2ff81ebf04
Address missed GH comments && fix broken merge
2022-11-23 15:16:14 +02:00
Kostas Kyrimis
407418e8f5
Merge branch 'project-pineapples' into T1159-MG-Add-memgraph-functions
2022-11-23 14:53:53 +02:00
Kostas Kyrimis
6d544e4fc0
Address GH comments
2022-11-23 14:51:38 +02:00
jbajic
9b19dd57d3
Remove storage include
2022-11-23 13:19:25 +01:00
Jeremy B
aa146d28f8
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-11-23 12:26:58 +01:00
János Benjamin Antal
1cd10ab409
Merge branch 'project-pineapples' into T1116-MG-update-delete-vertex-labels
2022-11-23 12:15:34 +01:00
Jure Bajic
e565fc6e3a
Merge branch 'project-pineapples' into T0995-MG-implement-top-error-handling-storage
2022-11-23 02:33:07 -08:00
Jeremy B
84800cff76
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-11-23 10:57:05 +01:00
jbajic
9fade5ebac
Fix tests
2022-11-23 10:37:33 +01:00
Jure Bajic
75606dfeb0
Merge branch 'project-pineapples' into T1116-MG-update-delete-vertex-labels
2022-11-23 00:34:27 -08:00
János Benjamin Antal
9c36d8928a
Merge branch 'project-pineapples' into MG-remove-unused-storage-files
2022-11-23 09:19:14 +01:00
Kostas Kyrimis
9ec72bd969
Add GetProperties shard handler and tests
2022-11-22 19:44:22 +02:00
Tyler Neely
c0a103e851
Do not advance the clock with every message, as this prevents messages of a certain request depth from ever completing
2022-11-22 16:00:06 +00:00
jeremy
3a171376d7
OrderBy in Expand has two members to differ vertices Vs edges
2022-11-22 16:47:25 +01:00
jbajic
cd7b33f23f
use comparison between ErrorCode and ShardError
2022-11-22 16:22:42 +01:00
jbajic
d82cfb349e
Use ShardResult
2022-11-22 15:00:32 +01:00
jbajic
5717dfb165
Add ErrorCode to SHARD_ERROR macro
2022-11-22 14:05:04 +01:00
jbajic
2a6dc7bb93
Remove success
2022-11-22 13:49:12 +01:00
Tyler Neely
0b19b62b12
Set the abort_time for raft tests to 1 simulated hour
2022-11-22 11:25:24 +00:00
jeremy
1a67dec302
Update test to use OrderBy and Limit on Expand
2022-11-21 11:45:35 +01:00
Tyler Neely
71dcba331e
Increment simulator time by up to 30ms in ticks
2022-11-21 10:10:45 +00:00
Tyler Neely
04420a84c7
Fix incorrect usage of IncrementServerCountAndWaitForQuiescentState in the shard_rsm.cpp simulation test
2022-11-18 16:54:38 +00:00
Tyler Neely
ce45a548c7
Significantly improve the determinism of the coordinator, UUID generation, the machine manager, the shard manager, and the cluster property test
2022-11-18 16:42:18 +00:00
Tyler Neely
3ad8489735
Run raft sim with random seeds over time, but allow a seed to be easily replayed using the RunWithSeed function
2022-11-18 10:34:21 +00:00
Tyler Neely
6b9a617df0
Streamline simulator tick condition varible notification. Advance time more aggressively
2022-11-18 09:20:15 +00:00
Tyler Neely
f6017697d6
Make raft tests fully deterministic for rng_seeds between 0 and 500 at 1% message loss
2022-11-17 21:32:55 +00:00
Tyler Neely
9c3d683942
Explicitly join test threads before collecting test stats
2022-11-17 21:28:17 +00:00
Tyler Neely
098084314e
Make TestAddress deterministically sortable
2022-11-17 21:22:41 +00:00
Tyler Neely
12880fc71a
Don't advance the simulator handle from server threads themselves
2022-11-17 18:27:12 +00:00
Tyler Neely
80d6776210
Improve simulator determinism
2022-11-17 17:36:46 +00:00
jbajic
ec4804b72a
Move ErrorCode to common
2022-11-17 11:03:03 +01:00
jbajic
1f159b5a80
Fix scheme test
2022-11-16 08:58:06 +01:00
jbajic
1c94c59a24
Fix tests
2022-11-15 19:29:56 +01:00
Kostas Kyrimis
9e81fe791c
Fix clang-tidy warnings, remove commented out code and add HasLabel
...
functions in ShardRequestManager to avoid throwing bad optional on
expression evaluator
2022-11-15 19:20:01 +02:00
Kostas Kyrimis
213583f916
Add expression evaluator && awesome_memgraph_functions unit tests
2022-11-14 20:46:20 +02:00
Kostas Kyrimis
94bc671551
Add awesome memgraph functions e2e tests
2022-11-14 20:46:19 +02:00
Kostas Kyrimis
02ef954e51
Add mg-functions lib and integrated with SE and QE
2022-11-14 20:46:16 +02:00
jbajic
b4c24f4506
Remvoe storage.hpp and storage.cpp files
2022-11-14 10:23:48 +01:00
jbajic
6eabceca4a
Add unit tests for UpdateVertex
2022-11-14 09:02:55 +01:00
Jeremy B
18009c06b6
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-11-09 16:47:06 +01:00
Jure Bajic
db7b2aa59a
Merge branch 'project-pineapples' into T1116-MG-update-delete-vertex-labels
2022-11-09 07:24:26 -08:00
jeremy
6df2db0d19
Remove comment force github workflow
2022-11-09 16:02:59 +01:00
jeremy
968584a8fc
Add comment force github workflow
2022-11-09 16:02:25 +01:00
jbajic
8636788ab2
Fix typos
2022-11-09 16:00:25 +01:00
jbajic
23f1536eac
Add tests
2022-11-09 15:42:09 +01:00
jeremy
33add3ecd0
Force formatting
2022-11-09 15:38:51 +01:00
jbajic
a2735c8953
Add label add/remove in UpdateVertex req
2022-11-09 12:10:46 +01:00
jeremy
2087877df2
Add more checks in test
2022-11-09 08:48:51 +01:00
Jeremy B
efa4378fb4
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-09 08:26:25 +01:00
jeremy
cad0e80d00
Update test
2022-11-08 17:42:31 +01:00
Jeremy B
e442963466
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-08 17:00:30 +01:00
János Benjamin Antal
32fe4d94d9
Merge branch 'project-pineapples' into T1145-MG-handle-insertions-of-vertices-that-already-exist-in-a-better-way
2022-11-08 16:35:38 +01:00
jeremy
d7ce7cea13
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-11-08 15:02:56 +01:00
jeremy
61b9457718
Remove split-files logic from test code
2022-11-08 11:35:54 +01:00
János Benjamin Antal
159b30ba5f
Merge branch 'project-pineapples' into T1148-MG-fix-expand-one-source-vertex
2022-11-08 08:20:25 +01:00
gvolfing
b67e5b9a6c
Merge branch 'project-pineapples' into T1145-MG-handle-insertions-of-vertices-that-already-exist-in-a-better-way
2022-11-08 07:53:45 +01:00
gvolfing
79756ae6fb
Modify unit test
...
Instead of Creating an exception that is would be only used in this
file, just assert the type of the error the double vertex insertion
operation should yield.
2022-11-08 07:31:01 +01:00
jeremy
5201db46d2
Add assert for split_file
2022-11-07 12:15:57 +01:00
gvolfing
91550128a5
Conform unit test with the new error-handling
2022-11-07 11:46:24 +01:00
jeremy
d5966101ad
Merge branch 'T1079-MG-add-simple-query-to-benchmark_v2' of github.com:memgraph/memgraph into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-07 10:55:48 +01:00
jeremy
b10b1eb239
Correct shard configuration
2022-11-07 10:33:24 +01:00
jeremy
dca94f42bb
Update key type in shard configuration
2022-11-07 10:14:49 +01:00
jeremy
baacc52a65
Add support for split file configuration
2022-11-07 09:54:14 +01:00
jeremy
5273d319e2
Add split file for access control
2022-11-07 09:53:29 +01:00
Jeremy B
23838b50ea
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-05 14:46:33 +01:00
jbajic
4d2036249e
Fix e2e test
2022-11-04 16:37:35 +01:00
Jure Bajic
d23643d4ff
Merge branch 'project-pineapples' into T1148-MG-fix-expand-one-source-vertex
2022-11-04 16:19:55 +01:00
jbajic
e52ce1e363
Add expandone test
2022-11-04 16:18:18 +01:00
jeremy
d17970f6d9
Update default value for --datasets
2022-11-04 16:04:45 +01:00
jeremy
9e72c7cb54
Add extra safe check to in case we call on dataset.py
2022-11-04 15:57:26 +01:00
gvolfing
3d954e7abc
Restructure SchemaResult type and uts usage
...
Rename SchemaResult to ShardOperationResult move it into a separate
header and add a new type to the underlying variant that indicates that
the vertex, the user would like to insert into the skip-list already
exist.
2022-11-04 15:04:25 +01:00
Tyler Neely
8598f6edf4
Fix a race condition that happens when logging from a detached thread in the cluster property test. Improve the ShardManager dtor and log statements
2022-11-04 11:14:39 +00:00
jeremy
e41073bc2c
Update script to need single argument for local dataset
2022-11-04 09:17:09 +01:00
jeremy
14e3e72565
Correct badly written range
2022-11-04 08:52:47 +01:00
Jeremy B
690a389563
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-03 17:43:47 +01:00
Tyler Neely
9203616283
Merge branch 'project-pineapples' into T1122-MG-ShardManager-ThreadPool
2022-11-03 17:01:07 +01:00
Tyler Neely
6138d76690
Remove unit tests for storage v2
2022-11-03 14:21:17 +00:00
Tyler Neely
25fdb1a1f0
Make the high_density_shard_create_scan test run much faster
2022-11-03 13:48:11 +00:00
Tyler Neely
0364311dd0
Log latency histograms in the high_density_shard_create_scan test
2022-11-03 12:44:47 +00:00
Tyler Neely
9235515dab
Reduce high_density_shard_create_scan shard+thread combinations. Log Raft write request demangled names
2022-11-03 12:38:12 +00:00
jeremy
70dc19dfdb
Mgbench: apply filtering on results from client
2022-11-03 11:03:21 +01:00
jeremy
e909e7d2d8
Format
2022-11-02 14:18:04 +01:00
Jeremy B
1d18f1197f
Update tests/mgbench/dataset_creator.py
...
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-11-02 14:14:16 +01:00
Jeremy B
edeebf46ec
Update tests/mgbench/benchmark.py
...
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-11-02 14:13:58 +01:00
Jeremy B
1148fe9aad
Update tests/mgbench/datasets.py
...
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-11-02 14:13:40 +01:00
Tyler Neely
68654b5a19
Merge branch 'project-pineapples' of github.com:memgraph/memgraph into T1122-MG-ShardManager-ThreadPool
2022-11-02 10:55:18 +00:00
Jeremy B
7076788dd5
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-01 18:51:57 +01:00
Jeremy B
96a2ad4b63
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-11-01 18:21:36 +01:00
Tyler Neely
84509fa477
Fix merge conflict with MachineManager constructor
2022-11-01 17:07:14 +00:00
Tyler Neely
19b5fe3caf
Merge branch 'project-pineapples' into tyler_remove_shard_map_from_machine_manager
2022-11-01 17:57:03 +01:00
Tyler Neely
6dd57426f8
Revert broken code suggestion
2022-11-01 15:51:37 +00:00
Tyler Neely
9a1258a708
Update tests/unit/1k_shards_1k_create_scanall.cpp
...
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2022-11-01 16:31:14 +01:00
Tyler Neely
c55ca836a4
Update tests/unit/1k_shards_1k_create_scanall.cpp
...
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2022-11-01 16:31:06 +01:00
Tyler Neely
5674ef4016
Update tests/unit/1k_shards_1k_create_scanall.cpp
...
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2022-11-01 16:30:37 +01:00
Tyler Neely
a6add80fc9
Use static RSM partitioning function for achieving a smooth Shard->ShardWorker distribution
2022-11-01 14:52:38 +00:00
Tyler Neely
5d3eaf6a55
Rename 1k_shards_1k_create_scanall test to high_density_shard_create_scan
2022-11-01 14:02:07 +00:00
Tyler Neely
a13f260236
Parameterize shard worker threads in the MachineConfig and simplify test output
2022-11-01 13:57:57 +00:00
jeremy
1f778ba5f3
Add possibility to give extra tests arg to MGBench
2022-11-01 14:14:55 +01:00
jeremy
787987168c
Make benchmark work with any customer datasets
2022-11-01 12:51:01 +01:00
Tyler Neely
bb1e8aa164
Improve histogram output by adding a pretty table printing function
2022-11-01 10:46:25 +00:00
Tyler Neely
27a1311966
Merge branch 'tyler_1k_shard_unit_test' of github.com:memgraph/memgraph into T1122-MG-ShardManager-ThreadPool
2022-11-01 09:49:15 +00:00
Tyler Neely
119da2d7a7
Update CMakeLists.txt
2022-11-01 09:51:43 +01:00
Tyler Neely
2f77eb96ff
Remove redundant ShardMap copy from MachineManager to avoid race conditions
2022-10-31 16:49:46 +00:00
jeremy
4c5cd1f847
Add possibility to have MgBench working against local file
2022-10-31 16:10:04 +01:00
Tyler Neely
b0aaba6765
Merge branch 'project-pineapples' into tyler_1k_shard_unit_test
2022-10-31 15:47:38 +01:00
jeremy
e1f18f3733
Update location of Pokec datasets on aws
2022-10-31 15:19:34 +01:00
jeremy
c90b38faf0
Update aws address for datasets
2022-10-31 14:49:32 +01:00
jeremy
acbf3c764c
Remove arg from __init__
2022-10-31 13:35:41 +01:00
jeremy
37df41796f
git puMerge branch 'T1079-MG-add-simple-query-to-benchmark_v2' of github.com:memgraph/memgraph into T1079-MG-add-simple-query-to-benchmark_v2
2022-10-31 13:02:12 +01:00
jeremy
f04e1cda4b
Add function
2022-10-31 13:02:05 +01:00
Jeremy B
f28ba89584
Update tests/mgbench/dataset_creator.py
...
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-10-31 13:01:42 +01:00
jeremy
60485311c8
Merge branch 'T1079-MG-add-simple-query-to-benchmark_v2' of github.com:memgraph/memgraph into T1079-MG-add-simple-query-to-benchmark_v2
2022-10-31 12:57:46 +01:00
Jeremy B
36a1c43851
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-10-31 12:57:41 +01:00
Jeremy B
bae8c084b1
Update tests/mgbench/datasets.py
...
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-10-31 12:56:02 +01:00
Jeremy B
2898120eeb
Update tests/mgbench/datasets.py
...
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-10-31 12:55:42 +01:00
jeremy
d62a45752a
Remove unused variable
2022-10-31 12:55:38 +01:00
jeremy
c5ee6ffbc2
Update dataset_creator script
2022-10-31 12:41:28 +01:00
jeremy
5ef08f841a
Update Dataset creation script
2022-10-31 11:56:20 +01:00
Kostas Kyrimis
59c7d81ae8
[ 🍍 < T1086-MG] Test distributed operators e2e ( #607 )
...
* Fix Explain queries
* Add Vertex/Edge accessor support for properties
* Fix projections
* Fix expansions to fetch destination vertex properties
* Fix improper use of ShardMap on bolt and replaced it with the ShardRequestManager
* Add NameToId mappers on ShardRequestManager
* Add e2e tests for operators
* Fix OPTIONAL MATCH
2022-10-31 11:52:20 +02:00
jeremy
03c095e780
Update assert
2022-10-31 10:52:06 +01:00
Jeremy B
b542e49b3e
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-10-31 09:44:34 +01:00
Jeremy B
ddb30f49ea
Update datasets.py
2022-10-31 09:43:22 +01:00
jeremy
46388ad35c
Correct compilation
2022-10-28 13:10:00 +02:00
jeremy
0d5ee49e19
Correct test expectation
2022-10-28 12:55:56 +02:00
Tyler Neely
03cf264b76
Add test for 1k shards, 1k create vertices, and then a scan all
2022-10-28 08:29:27 +00:00
Jeremy B
fa6129dc2b
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-10-26 15:59:15 +02:00
Tyler Neely
eafccaea84
Add a logarithmically bucketing histogram and record response latencies per message type ( #616 )
2022-10-26 15:57:11 +02:00
Jeremy B
aeed7c32f9
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-10-26 11:32:44 +02:00
János Benjamin Antal
f6e78ce6da
Fix TODOs in storage engine ( #614 )
...
Fixed various TODOs that were easy to fix to improve the code quality of the
newly implemented storage.
2022-10-26 10:40:35 +02:00
János Benjamin Antal
3cf79f5bbf
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2
2022-10-25 21:24:20 +02:00
Jeremy B
016b3ee0d2
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3
2022-10-25 10:30:22 +02:00
János Benjamin Antal
332afadf21
Split file parsing ( #600 )
...
Add temporary support for split files. This is only temporary solution until we
get the shard splitting implemented.
2022-10-25 10:27:13 +02:00
Tyler Neely
acc655f4fd
Model-based testing of simulated full cluster ( #584 )
...
This PR adds support for generating randomized workloads that will be executed
against a simulated cluster, as well as against a correctness model. Initially
this just generates ScanAll and CreateVertex requests, and anything that it
creates, it also inserts into a `std::set`, and when we do a ScanAll, it asserts
that we get the same number of requests back. This will become much more
sophisticated over time, but it's already hitting pay-dirt.
2022-10-24 19:54:09 +02:00
jeremy
33c9ccee66
Adapt test
2022-10-24 16:38:28 +02:00
