Commit Graph

440 Commits

Author SHA1 Message Date
Jeremy B
65a7ba01da
Add information on show replicas to express how up-to-date a replica is (#412)
* Add test

* Add implementation and adapted test

* Update workloads.yaml to have a timeout > 0

* Update tests (failing due to merging of "add replica state")
2022-06-23 10:22:57 +02:00
Kostas Kyrimis
7a2bbd4bb3
Remove ref from std::string_view (#215) 2022-06-21 17:18:43 +03:00
Jeremy B
589e0e098b
Forbid two replicas to point to the same ip port (#406) 2022-06-20 17:10:20 +03:00
Marko Budiselić
599c0a641f
Add replica state to SHOW REPLICAS (#379) 2022-06-20 13:28:42 +03:00
Marko Budiselić
8e3ab1ad0f
Add frequent replica ping (#380) 2022-05-20 19:29:17 -07:00
János Benjamin Antal
537855a0b2
Fix usages of constexpr (#367)
* Fix usages of constexpr
2022-03-31 13:52:43 +02:00
jbajic
60ad05acff Address review comments 2022-03-14 15:47:41 +01:00
jbajic
695bb343f1 Resolve clang-tidy error 2022-03-14 15:47:41 +01:00
jbajic
12b4ec1589 Add memgraph namespace 2022-03-14 15:47:41 +01:00
Antonio Andelic
69eca9b043
Procedures for handling modules (#330) 2022-02-11 11:29:41 +01:00
Antonio Andelic
265b203b00
Build for ARM64 (#340) 2022-02-03 13:03:35 +01:00
Antonio Andelic
bb1308acc7
Use libs from toolchain (#326) 2022-01-21 10:22:36 +01:00
Jure Bajic
16709dff6c
Add stats and notifications in results metadata (#285) 2021-11-15 13:51:13 +01:00
Antonio Andelic
e9a937ad6d
Update license headers and change date (#270) 2021-10-13 10:06:07 +02:00
Antonio Andelic
4e7ea34ae9
Add message with link utility (#243) 2021-10-07 14:51:30 +02:00
Jure Bajic
9e678f8cbe Add new memgraph licenses and move third party licenses (#248) 2021-10-03 12:26:48 +02:00
Kostas Kyrimis
d04e23805d
Address temporal type epic comments (#249) 2021-10-02 18:48:50 +02:00
Kostas Kyrimis
da68f86fc9 Add support for printing out in the ast temporal data values (#227) 2021-10-02 14:54:48 +02:00
antonio2368
5da32c1bff Add TemporalData to PropertyValue (#174) 2021-10-02 14:53:53 +02:00
antonio2368
b94e50bf1c
Use from_timestamp for WAL retention (#240) 2021-09-24 12:14:54 +02:00
János Benjamin Antal
2afc1b99f6
New MGP API (#219) 2021-09-09 10:44:47 +02:00
antonio2368
e51954fc94
Update toolchain to v3 (#189)
* Make memgraph buildable with new toolchain

* Use toolchain v3 in workflows
2021-07-08 14:20:48 +02:00
antonio2368
3b336e3e0b
Add CREATE SNAPSHOT query (#182) 2021-06-30 12:31:30 +02:00
antonio2368
cbf826e0c3
Load WAL on replica using transactions (#95) 2021-06-16 13:22:48 +02:00
antonio2368
8cd9f696cf
Multiple isolation levels (#171)
This PR introduces READ COMMITTED and READ UNCOMMITTED isolation levels.
The isolation level can be set with a config or with a query for different scopes.
2021-06-14 15:47:57 +02:00
antonio2368
b459639968 Other predefined variables (#143) 2021-05-26 14:13:14 +02:00
antonio2368
11c0dde11c Support removed vertices in the triggers (#140) 2021-05-26 14:13:14 +02:00
antonio2368
2f3fa656d9 Predefined variables (#138)
* Add support for injecting predefined variables in any scope
2021-05-26 14:13:14 +02:00
János Benjamin Antal
06f761bdf9
Add logs for loading snapshot and WAL files (#121)
* Add logs for loading snapshot and WAL files
2021-03-26 15:02:35 +01:00
antonio2368
cc56ac3dd8 Expose query for cleaning memory and add memory limit flag (#100)
* Add memory flag, add additional meminfo utilities

* Add free memory query
2021-03-24 15:10:28 +01:00
antonio2368
dee885d69c Add exception enablers and blockers (#99)
* Throw OOMException while creating vertices and edges

* Throw on indices creation

* Throw on setting a property

* Throw oom exception while recovering

* Throw exception when query engine asks for extra memory

* Block out of memor exception during skip list GC
2021-03-24 14:53:15 +01:00
antonio2368
16715d5005
Enable replication in community (#104)
* Enable replication in community
2021-03-10 10:36:38 +01:00
antonio2368
f807b495ab
Update commit log after recovery (#96)
* Define additional commit log constructor which takes an oldest active id

* Delay commit log construction until the recovery process is finished

* Add test for commit log with initial id

* Silence the macro redefinition warning
2021-02-19 11:00:10 +01:00
antonio2368
3f3c55a4aa
Format all the memgraph and test source files (#97) 2021-02-18 15:32:43 +01:00
Marko Budiselić
fae407d3fe
Add implementation of average number of equals estimate in SkipList (#9)
* Implement average number of equals estimate in SkipList
2021-02-10 14:38:54 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72)
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
antonio2368
c0bd59bb09 Fix recovery process (#78)
* Use correct last loaded timestamp for WAL
2021-01-21 15:56:21 +01:00
antonio2368
a84ffe86c1 Add LOCK DATA DIRECTORY query (#68) 2021-01-21 15:56:21 +01:00
antonio2368
10c8256ec9 Fix epoch id handling (#73) 2021-01-21 15:56:21 +01:00
antonio2368
b7bbd026de Add large Jepsen test (#67)
* Fix WAL recovery step
2021-01-21 15:56:21 +01:00
antonio2368
adc355a22a Small replication fixes (#59)
* Set state to invalid after exception

* Add proper locking

* Start background replicating only if in valid state

* Freeze transaction timestamp on replica

* Timeout fixes

* Fix Jepsen run script

* Disable perf checker and enable nemesis

* Add documentation for some chunks of code

* Decrease timeout so main doesn't hang on network partitions too long
2021-01-21 15:56:21 +01:00
Josip Seljan
a0fb3fc463 Implement ReplicationQueryHandler class (#52)
* Refactor io::network::Endpoint class
* Add ParseSocketOrIpAddress static method to Endpoint class
* Implement ReplQueryHandler methods
* Add implementation of SetReplicationRole to ReplQueryHandler
* Fix PrepareReplicationQuery (create PullPlanVector)

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2021-01-21 15:56:21 +01:00
antonio2368
f7b764607d Improve error handling (#56) 2021-01-21 15:49:32 +01:00
antonio2368
200ce5f45e Add configs and support for semi-sync and SSL (#55)
* Add config for replication client/server
* Add SSL to replication
* Add semi-sync replication
* Expose necessary information about replication
* Thread pool fix
* Set BasicResult value type to void
2021-01-21 15:49:32 +01:00
antonio2368
a0705746cb Add epoch id and refactor replication client/server (#51) 2021-01-21 15:49:32 +01:00
antonio2368
7e9175052a Define communication process (#49)
* Add basic communication process using commit timestamp
* Add file number to req
* Add proper recovery handling
* Allow loading of WALs with same seq num
* Allow always desired commit timestamp
* Set replica timestamp for operation
* Mark non-transactional timestamp as finished
2021-01-21 15:49:32 +01:00
antonio2368
03cc568e39 Add support for async replication (#41)
* Add thread pool
* Define async replication
* Expose replication state
* Rename TransactionHandler to ReplicaStream
2021-01-21 15:49:32 +01:00
antonio2368
bc0c944910 Add replica recovery process (#40)
* Add file transfer over RPC
* Snapshot transfer implementation
* Allow snapshot creation only for MAIN instances
* Replica and main can have replication clients
* Use only snapshots and WALs that are from the Main storage
* Add flush lock and expose buffer
* Add fstat for file size and TryFlushing method
* Use lseek for size

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
antonio2368
b10255a12f Add initial support for multiple clients (#31)
* Add tests for multiple clients
* Use variant for RPC server and clients
* Using synchronized list for replication clients, extracted variant access to a function
* Set MAIN as default, add unregister function, add a name for replication clients
* Use the regular list for clients
* Use test fixture so storage directory is cleaned
* Use seq_cst for replication_state

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
Marko Budiselić
c68ed8d94e Add implementation of synchronous replication (#7)
This implements the initial version of synchronous replication.
Currently, only one replica is supported and that isn't configurable.

To run the main instance use the following command:
```
./memgraph \
    --main \
    --data-directory main-data \
    --storage-properties-on-edges \
    --storage-wal-enabled \
    --storage-snapshot-interval-sec 300
```

To run the replica instance use the following command:
```
./memgraph \
    --replica \
    --data-directory replica-data \
    --storage-properties-on-edges \
    --bolt-port 7688
```

You can then write/read data to Bolt port 7687 (the main instance) and also you
can read the data from the replica instance using Bolt port 7688.

NOTE: The main instance *must* be started without any data and the replica
*must* be started before any data is added to the main instance.

* Add basic synchronous replication test
* Using RWLock for replication stuff

Co-authored-by: Matej Ferencevic <matej.ferencevic@memgraph.io>
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00