Commit Graph

10 Commits

Author SHA1 Message Date
Matija Santl
49f906f384 Disable flaky HA benchmark
Summary:
It seems like the HA benchmark is flaky and unrelated builds fail
because of it. Lets disable it for now until we figure out what is happening.

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1814
2019-01-17 12:54:30 +01:00
Matija Santl
aba360968c Change HA benchmark to run for a fixed amount of time
Summary:
In order to get more consistent results, give the benchmark a certain
amount of time it is supposed to run and not the number of queries.

The resluts on my machine are as following:
```
duration 10.0004
executed_writes 25190
write_per_second 2518.91
duration 10.0005
executed_writes 25096
write_per_second 2509.48
duration 10.0004
executed_writes 23068
write_per_second 2306.7
duration 10.0006
executed_writes 26390
write_per_second 2638.84
duration 10.0008
executed_writes 26246
write_per_second 2624.38
duration 10.0006
executed_writes 24752
write_per_second 2475.06
duration 10.0027
executed_writes 24818
write_per_second 2481.14
duration 10.0032
executed_writes 25148
write_per_second 2513.99
duration 10.0009
executed_writes 25075
write_per_second 2507.28
duration 10.0008
executed_writes 25846
write_per_second 2584.4
duration 10.0006
executed_writes 25671
write_per_second 2566.96
duration 10.0025
executed_writes 25983
write_per_second 2597.65
```

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1812
2019-01-17 10:07:44 +01:00
Ivan Paljak
16752af614 Force issuing heartbeats when appending to Raft log
Summary:
Locally run HA feature benchmark:

```
duration: 20.66
executed_writes: 150007
write_per_second: 7527.89
```

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1809
2019-01-16 16:38:45 +01:00
Matija Santl
ac5c6bf0e8 Add multi-threaded benchmark client for HA
Summary:
There are some serious speedups when doing parallel writes.

Results on my machine (4 cores):
```
duration 6.73173
executed_writes 15003
write_per_second 2228.7
```

Reviewers: ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1807
2019-01-16 10:19:33 +01:00
Ivan Paljak
f09c1254f4 Optimize Raft log persistent storage
Summary:
Each `raft::LogEntry` is now persisted under its own key in our `KVStore`. Locally running our HA feature benchmark yields the following results:

```
duration 23.7
executed_writes: 15000
write_per_second: 632.888
```

This represents about 5x increase in throughput.

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1799
2019-01-15 16:06:08 +01:00
Matija Santl
c39a2278ae Add HA benchmark
Summary:
A simple benchmark that starts a HA cluster with 3 machines.
The benchmark issues only `CREATE (:Node)` queries.

Local results (debug build), for this raft config, are:
```
duration 4.26899
executed_writes 300
write_per_second 70.2743
```

Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1798
2019-01-15 13:32:04 +01:00
Matej Ferencevic
4e5fe37dd6 Remove virtual and pimpl from single node
Summary:
This diff removes: `SingleNodeRecoveryTransactions`, `TypemapPack`
It also removes virtual and/or pimpl from: `SingleNodeCounters`,
`StorageGcSingleNode`, `SingleNodeConcurrentIdMapper`,
accessors (revert D1510), transaction engine, `GraphDbAccessor`, `GraphDb`

Reviewers: msantl, teon.banek

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1639
2018-10-09 11:48:30 +02:00
Matej Ferencevic
ade2593b51 Separate distributed from single node GraphDb
Summary:
To clean the working directory after this diff you should execute:

```
rm src/database/counters_rpc_messages.capnp
rm src/database/counters_rpc_messages.hpp
rm src/database/serialization.capnp
rm src/database/serialization.hpp
```

Reviewers: teon.banek, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1636
2018-10-05 13:32:05 +02:00
Matej Ferencevic
75950664a7 Separate distributed from single node storage
Summary:
This diff splits single node and distributed storage from each other.
Currently all of the storage code is copied into two directories (one single
node, one distributed).  The logic used in the storage implementation isn't
touched, it will be refactored in following diffs.

To clean the working directory after this diff you should execute:
```
rm database/state_delta.capnp
rm database/state_delta.hpp
rm storage/concurrent_id_mapper_rpc_messages.capnp
rm storage/concurrent_id_mapper_rpc_messages.hpp
```

Reviewers: teon.banek, buda, msantl

Reviewed By: teon.banek, msantl

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1625
2018-10-05 09:19:33 +02:00
Matija Santl
57b84f2da3 Add kafka benchmark
Summary:
In order to add kafka benchmark, `memgraph_bolt.cpp` has been split.
Now we have `memgraph_init.cpp/hpp` files with common memgraph startup code.
Kafka benchmark implements a new `main` function that doesn't start a bolt
server, it just creates and starts a stream. Then it waits for the stream to
start consuming and measures the time it took to import the given number of
entries.

This benchmark is in a new folder, `feature_benchmark`, and so should any new
bechmark that measures performance of memgraphs features.

Reviewers: mferencevic, teon.banek, ipaljak, vkasljevic

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1552
2018-08-29 16:35:31 +02:00