tursom/memgraph
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,599 Commits 247 Branches 35 Tags 62 MiB
500691318a
Commit Graph

1797 Commits

Author SHA1 Message Date
Jeremy B
efa4378fb4
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-09 08:26:25 +01:00
jeremy
cad0e80d00 Update test 2022-11-08 17:42:31 +01:00
Jeremy B
e442963466
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-08 17:00:30 +01:00
János Benjamin Antal
32fe4d94d9
Merge branch 'project-pineapples' into T1145-MG-handle-insertions-of-vertices-that-already-exist-in-a-better-way 2022-11-08 16:35:38 +01:00
jeremy
d7ce7cea13 Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3 2022-11-08 15:02:56 +01:00
jeremy
61b9457718 Remove split-files logic from test code 2022-11-08 11:35:54 +01:00
János Benjamin Antal
159b30ba5f
Merge branch 'project-pineapples' into T1148-MG-fix-expand-one-source-vertex 2022-11-08 08:20:25 +01:00
gvolfing
b67e5b9a6c Merge branch 'project-pineapples' into T1145-MG-handle-insertions-of-vertices-that-already-exist-in-a-better-way 2022-11-08 07:53:45 +01:00
gvolfing
79756ae6fb Modify unit test 
Instead of Creating an exception that is would be only used in this
file, just assert the type of the error the double vertex insertion
operation should yield.
 2022-11-08 07:31:01 +01:00
jeremy
5201db46d2 Add assert for split_file 2022-11-07 12:15:57 +01:00
gvolfing
91550128a5 Conform unit test with the new error-handling 2022-11-07 11:46:24 +01:00
jeremy
d5966101ad Merge branch 'T1079-MG-add-simple-query-to-benchmark_v2' of github.com:memgraph/memgraph into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-07 10:55:48 +01:00
jeremy
b10b1eb239 Correct shard configuration 2022-11-07 10:33:24 +01:00
jeremy
dca94f42bb Update key type in shard configuration 2022-11-07 10:14:49 +01:00
jeremy
baacc52a65 Add support for split file configuration 2022-11-07 09:54:14 +01:00
jeremy
5273d319e2 Add split file for access control 2022-11-07 09:53:29 +01:00
Jeremy B
23838b50ea
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-05 14:46:33 +01:00
jbajic
4d2036249e Fix e2e test 2022-11-04 16:37:35 +01:00
Jure Bajic
d23643d4ff
Merge branch 'project-pineapples' into T1148-MG-fix-expand-one-source-vertex 2022-11-04 16:19:55 +01:00
jbajic
e52ce1e363 Add expandone test 2022-11-04 16:18:18 +01:00
jeremy
d17970f6d9 Update default value for --datasets 2022-11-04 16:04:45 +01:00
jeremy
9e72c7cb54 Add extra safe check to in case we call on dataset.py 2022-11-04 15:57:26 +01:00
gvolfing
3d954e7abc Restructure SchemaResult type and uts usage 
Rename SchemaResult to ShardOperationResult move it into a separate
header and add a new type to the underlying variant that indicates that
the vertex, the user would like to insert into the skip-list already
exist.
 2022-11-04 15:04:25 +01:00
Tyler Neely
8598f6edf4 Fix a race condition that happens when logging from a detached thread in the cluster property test. Improve the ShardManager dtor and log statements 2022-11-04 11:14:39 +00:00
jeremy
e41073bc2c Update script to need single argument for local dataset 2022-11-04 09:17:09 +01:00
jeremy
14e3e72565 Correct badly written range 2022-11-04 08:52:47 +01:00
Jeremy B
690a389563
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-03 17:43:47 +01:00
Tyler Neely
9203616283
Merge branch 'project-pineapples' into T1122-MG-ShardManager-ThreadPool 2022-11-03 17:01:07 +01:00
Tyler Neely
6138d76690 Remove unit tests for storage v2 2022-11-03 14:21:17 +00:00
Tyler Neely
25fdb1a1f0 Make the high_density_shard_create_scan test run much faster 2022-11-03 13:48:11 +00:00
Tyler Neely
0364311dd0 Log latency histograms in the high_density_shard_create_scan test 2022-11-03 12:44:47 +00:00
Tyler Neely
9235515dab Reduce high_density_shard_create_scan shard+thread combinations. Log Raft write request demangled names 2022-11-03 12:38:12 +00:00
jeremy
70dc19dfdb Mgbench: apply filtering on results from client 2022-11-03 11:03:21 +01:00
jeremy
e909e7d2d8 Format 2022-11-02 14:18:04 +01:00
Jeremy B
1d18f1197f
Update tests/mgbench/dataset_creator.py 
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
 2022-11-02 14:14:16 +01:00
Jeremy B
edeebf46ec
Update tests/mgbench/benchmark.py 
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
 2022-11-02 14:13:58 +01:00
Jeremy B
1148fe9aad
Update tests/mgbench/datasets.py 
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
 2022-11-02 14:13:40 +01:00
Tyler Neely
68654b5a19 Merge branch 'project-pineapples' of github.com:memgraph/memgraph into T1122-MG-ShardManager-ThreadPool 2022-11-02 10:55:18 +00:00
Jeremy B
7076788dd5
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-01 18:51:57 +01:00
Jeremy B
96a2ad4b63
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-11-01 18:21:36 +01:00
Tyler Neely
84509fa477 Fix merge conflict with MachineManager constructor 2022-11-01 17:07:14 +00:00
Tyler Neely
19b5fe3caf
Merge branch 'project-pineapples' into tyler_remove_shard_map_from_machine_manager 2022-11-01 17:57:03 +01:00
Tyler Neely
6dd57426f8 Revert broken code suggestion 2022-11-01 15:51:37 +00:00
Tyler Neely
9a1258a708
Update tests/unit/1k_shards_1k_create_scanall.cpp 
Co-authored-by: Kostas Kyrimis  <kostaskyrim@gmail.com>
 2022-11-01 16:31:14 +01:00
Tyler Neely
c55ca836a4
Update tests/unit/1k_shards_1k_create_scanall.cpp 
Co-authored-by: Kostas Kyrimis  <kostaskyrim@gmail.com>
 2022-11-01 16:31:06 +01:00
Tyler Neely
5674ef4016
Update tests/unit/1k_shards_1k_create_scanall.cpp 
Co-authored-by: Kostas Kyrimis  <kostaskyrim@gmail.com>
 2022-11-01 16:30:37 +01:00
Tyler Neely
a6add80fc9 Use static RSM partitioning function for achieving a smooth Shard->ShardWorker distribution 2022-11-01 14:52:38 +00:00
Tyler Neely
5d3eaf6a55 Rename 1k_shards_1k_create_scanall test to high_density_shard_create_scan 2022-11-01 14:02:07 +00:00
Tyler Neely
a13f260236 Parameterize shard worker threads in the MachineConfig and simplify test output 2022-11-01 13:57:57 +00:00
jeremy
1f778ba5f3 Add possibility to give extra tests arg to MGBench 2022-11-01 14:14:55 +01:00
jeremy
787987168c Make benchmark work with any customer datasets 2022-11-01 12:51:01 +01:00
Tyler Neely
bb1e8aa164 Improve histogram output by adding a pretty table printing function 2022-11-01 10:46:25 +00:00
Tyler Neely
27a1311966 Merge branch 'tyler_1k_shard_unit_test' of github.com:memgraph/memgraph into T1122-MG-ShardManager-ThreadPool 2022-11-01 09:49:15 +00:00
Tyler Neely
119da2d7a7
Update CMakeLists.txt 2022-11-01 09:51:43 +01:00
Tyler Neely
2f77eb96ff Remove redundant ShardMap copy from MachineManager to avoid race conditions 2022-10-31 16:49:46 +00:00
jeremy
4c5cd1f847 Add possibility to have MgBench working against local file 2022-10-31 16:10:04 +01:00
Tyler Neely
b0aaba6765
Merge branch 'project-pineapples' into tyler_1k_shard_unit_test 2022-10-31 15:47:38 +01:00
jeremy
e1f18f3733 Update location of Pokec datasets on aws 2022-10-31 15:19:34 +01:00
jeremy
c90b38faf0 Update aws address for datasets 2022-10-31 14:49:32 +01:00
jeremy
acbf3c764c Remove arg from __init__ 2022-10-31 13:35:41 +01:00
jeremy
37df41796f git puMerge branch 'T1079-MG-add-simple-query-to-benchmark_v2' of github.com:memgraph/memgraph into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-31 13:02:12 +01:00
jeremy
f04e1cda4b Add function 2022-10-31 13:02:05 +01:00
Jeremy B
f28ba89584
Update tests/mgbench/dataset_creator.py 
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
 2022-10-31 13:01:42 +01:00
jeremy
60485311c8 Merge branch 'T1079-MG-add-simple-query-to-benchmark_v2' of github.com:memgraph/memgraph into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-31 12:57:46 +01:00
Jeremy B
36a1c43851
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-31 12:57:41 +01:00
Jeremy B
bae8c084b1
Update tests/mgbench/datasets.py 
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
 2022-10-31 12:56:02 +01:00
Jeremy B
2898120eeb
Update tests/mgbench/datasets.py 
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
 2022-10-31 12:55:42 +01:00
jeremy
d62a45752a Remove unused variable 2022-10-31 12:55:38 +01:00
jeremy
c5ee6ffbc2 Update dataset_creator script 2022-10-31 12:41:28 +01:00
jeremy
5ef08f841a Update Dataset creation script 2022-10-31 11:56:20 +01:00
Kostas Kyrimis
59c7d81ae8
[🍍 < T1086-MG] Test distributed operators e2e (#607) 
* Fix Explain queries
* Add Vertex/Edge accessor support for properties
* Fix projections
* Fix expansions to fetch destination vertex properties
* Fix improper use of ShardMap on bolt and replaced it with the ShardRequestManager 
* Add NameToId mappers on ShardRequestManager
* Add e2e tests for operators
* Fix OPTIONAL MATCH
 2022-10-31 11:52:20 +02:00
jeremy
03c095e780 Update assert 2022-10-31 10:52:06 +01:00
Jeremy B
b542e49b3e
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-31 09:44:34 +01:00
Jeremy B
ddb30f49ea
Update datasets.py 2022-10-31 09:43:22 +01:00
jeremy
46388ad35c Correct compilation 2022-10-28 13:10:00 +02:00
jeremy
0d5ee49e19 Correct test expectation 2022-10-28 12:55:56 +02:00
Tyler Neely
03cf264b76 Add test for 1k shards, 1k create vertices, and then a scan all 2022-10-28 08:29:27 +00:00
Jeremy B
fa6129dc2b
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3 2022-10-26 15:59:15 +02:00
Tyler Neely
eafccaea84
Add a logarithmically bucketing histogram and record response latencies per message type (#616) 2022-10-26 15:57:11 +02:00
Jeremy B
aeed7c32f9
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3 2022-10-26 11:32:44 +02:00
János Benjamin Antal
f6e78ce6da
Fix TODOs in storage engine (#614) 
Fixed various TODOs that were easy to fix to improve the code quality of the
newly implemented storage.
 2022-10-26 10:40:35 +02:00
János Benjamin Antal
3cf79f5bbf
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-25 21:24:20 +02:00
Jeremy B
016b3ee0d2
Merge branch 'project-pineapples' into T1083-MG-limit-and-order-expand-one_v3 2022-10-25 10:30:22 +02:00
János Benjamin Antal
332afadf21
Split file parsing (#600) 
Add temporary support for split files. This is only temporary solution until we
get the shard splitting implemented.
 2022-10-25 10:27:13 +02:00
Tyler Neely
acc655f4fd
Model-based testing of simulated full cluster (#584) 
This PR adds support for generating randomized workloads that will be executed
against a simulated cluster, as well as against a correctness model. Initially
this just generates ScanAll and CreateVertex requests, and anything that it
creates, it also inserts into a `std::set`, and when we do a ScanAll, it asserts
that we get the same number of requests back. This will become much more
sophisticated over time, but it's already hitting pay-dirt.
 2022-10-24 19:54:09 +02:00
jeremy
33c9ccee66 Adapt test 2022-10-24 16:38:28 +02:00
jeremy
386a0c5686 add comment 2022-10-24 15:44:37 +02:00
jeremy
b82e8748ad Attempt impl 2022-10-24 12:03:51 +02:00
jeremy
55e0dbca80 Add limit to ExpandOne 
Add missing pragma
Add test
Merge conflicts
 2022-10-21 16:32:49 +02:00
Marko Budiselić
1c3bb969e9
Decouple interactive planning manual test (#585) 2022-10-21 13:34:13 +02:00
Jure Bajic
e5437080c5
Handle OrderBy in ScanVertices request (#594) 
- Refactor shard_rsm and move function into expr.hpp/expr.cpp and request_helper.hpp/request_helper.cpp
 2022-10-21 12:33:40 +02:00
Tyler Neely
17090dd8ac
Load SPDLOG_LEVEL environment variable in unit tests binaries (#606) 
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
 2022-10-20 18:27:52 +02:00
gvolfing
281cebd386
Add filtering capabilities to ExpandOne request (#581) 2022-10-20 17:51:44 +02:00
János Benjamin Antal
f89a2bbf42
Make ExpandOne work in the query engine (#589) 2022-10-20 11:35:00 +02:00
gvolfing
85b8ce9101
Implement filtering capabilities for ScanAll (#578) 
ScanVerticesrequest was not able to utilize filtering capabilities
before. With these modification it is now able to filter the scanned
vertices based on the filter_expressions field in the
ScanVerticesRequest message type.
 2022-10-19 16:09:00 +02:00
Jure Bajic
6bb40a7f49
Create ExpandOne request (#580) 
Refactor CreateEdge into CreateExpand
 2022-10-19 13:55:46 +02:00
Jeremy B
1707ee648e
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-19 08:47:13 +02:00
gvolfing
07f34838bd
Add e2e test to check ScanAll on new architecture (#603) 
Add testing capabilities similar to what existed before on the older
architecture. With this test specifically, test if we can create
vertices and return them with ScanAll. There is a hard-coded 3s wait in
the test which is needed because the MachineManager need this time to
initialize itself.
 2022-10-18 21:38:56 +02:00
Jeremy B
b6f8b5e130
Merge branch 'project-pineapples' into T1079-MG-add-simple-query-to-benchmark_v2 2022-10-18 15:49:19 +02:00
jeremy
58243f4a26 Rename User->File 
Use parser for argument i.o. simple variable in script
 2022-10-18 15:47:13 +02:00