Fixes root cause of a cascade of failures in replication code:
- Replica handling of deleting an edge is now corrected. Now tolerant of multiple edges of the same relationship type.
- Improved robustness: correct exception handling around failed stream of current WAL file. This now means a REPLICA failure will no longer prevent transactions on MAIN from performing WAL writes.
- Slightly better diagnostic messages, not user friendly but helps get developer to correct root cause quicker.
- Proactively remove vertex+edges during Abort rather than defer to GC to do that work, this included fixing constraints and indexes to be safe.
Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Single (instance level) connection to a replica (messages from all databases get multiplexed through it)
ReplicationClient split in two: ReplicationClient and ReplicationStorageClient
New ReplicationClient, moved under replication, handles the raw connection, owned by MainRoleData
ReplicationStorageClient handles the storage <-> replica state machine and holds to a stream
Removed epoch and storage from *Clients
rpc::Stream proactively aborts on error and sets itself to a defunct state
Removed HandleRpcFailure, instead we simply log the error and let the FrequentCheck handle re-connection
replica_state is now a synced variable
ReplicaStorageClient state machine bugfixes
Single FrequentCheck that goes through DBMS
Moved ReplicationState under DbmsHandler
Moved some replication startup logic under the DbmsHandler's constructor
Removed InMemoryReplicationClient
CreateReplicationClient has been removed from Storage
Simplified GetRecoverySteps and made safer
---------
Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
There might be a performance impect of updating the metadata store on
bulk operations. Hence this flag which is disabling the collection by
default. If the queries to obtain the information are called with this
flag disabled, the database will throw an exception.
* Interpreter transaction ID decoupled from storage transaction ID
* Transactional scope for indices, statistics and constraints
* Storage::Accessor now has 2 modes (unique and shared)
* Introduced ResourceLock to fix pthread mutex problems
* Split InfoQuery in two: non-transactional SystemInfoQuery and transactional DatabaseInfoQuery
* Replicable and durable statistics
* Bumped WAL/Snapshot versions
* Initial implementation of the Lamport clock
---------
Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Concurrent access to the same query module had a race condition on the
pointer that was used to handle the custom memory management. With this
commit, a mapping has been added to keep information about what
thread used the pointer to handle the memory resources. This should be
fine since the respected query executions are running on a dedicated
thread. Access to the mapping itself is threadsafe. A simple RAII
wrapper for the mapping container has also been added for simpler
client-side use.
Add supernode vertex cache to account for long delta chains and modifications in the same module being independent of scanning of the nodes in the next iteration of the pulling mechanism.
* Decouple BoltSession and communication::bolt::Session
* Add CREATE/USE/DROP DATABASE
* Add SHOW DATABASES
* Cover WebSocket session
* Simple session safety implemented via RWLock
* Storage symlinks for backward. compatibility
* Extend the audit log with the DB info
* Add auth part
* Add tenant recovery
* update profile query to use poolresource
* Optimize update of indexes
* Add ignore empty strings to load csv
* Add operator changes to support handling of nulls
* Store chunks in memory pools ordered
* Use same max block per chunks number
* Remove redundant return statement
* add hacky cached solution
* change map to set
* remove memory
* Add match load csv invalid behaviour commit
* Accept input on LOAD CSV
* Ommit changes not tied to the PR
* Add tests for match + load csv
* Add gqlalchemy installation for e2e tests
* Modify setup script to update packages
* Revert gqlalchemy to 1.3.3
* Revert gqlalchemy to 1.3.3
* Address PR review comments
* Ommit semicolon
---------
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
* Fix init file startup in community edition
* Add possibility to build binary without MG_ENTERPRISE
* Added trace spdlog for when init file is not present
* Add gqlalchemy and unit tests
* Add init data files which correspond to the right directory by the github actions
* Parallelize edge recovery
* Load vertex labels and properties parallel
* Add parallel connectivity loading
* Add batches information to snapshot
* Introduce `items_per_batch` and `recovery_thread_count` flags
* Make possible to load snapshots with old version
* Add vertex batches to `RecoveryInfo`
* Extend durability integration tests with v15 test cases
* Add `std::vector` based `InitProperties`
* Use `InitProperties` in snapshot loading
* Added check if there is invalid reference to the underlying edge
* Added fix and e2e tests
* Isolation levels tracking based on from_vertex_
* Added explicit transaction test + edge accessor changes based on the vertex_edge
* Autocommit on tests, initialize deleted by checking out_edges
Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
* [T1006-MG < T1017-MG] Add LBA checks to all read procedures in C API (#515)
* Initial Impl
* NextPermittedEdge introduced
* revert moving constructor to cpp
* edge from and edge to methods expanded with lba check
* minor fix
* added check to path expand procedure
* Added integration tests for read query procedures
* additional check
* changed iterator type to reference
* comments from pr
Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
* [T1006-MG < T1018-MG] Add LBA checks to all update procedures in C API (#516)
* Initial Impl
* NextPermittedEdge introduced
* revert moving constructor to cpp
* edge from and edge to methods expanded with lba check
* minor fix
* extended update methods
* added check to path expand procedure
* Added integration tests for read query procedures
* Added integration tests for update query modules
* additional check
* changed iterator type to reference
* fixed bug in Update property for node; fixed 2 e2e tests
* replaced enum
Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
* [T1006-MG < T1019-MG] Add LBA checks to all Create and Delete procedures in C API (#517)
* Initial Impl
* NextPermittedEdge introduced
* revert moving constructor to cpp
* edge from and edge to methods expanded with lba check
* minor fix
* extended update methods
* initial implementation
* added check to path expand procedure
* Added integration tests for read query procedures
* Added integration tests for update query modules
* Added unit tests for creation of vertex, adding and removing vertex label
* additional check
* changed iterator type to reference
* Added unit tests for create edge
* Corrected query module in create edge
* fixed bug in Update property for node; fixed 2 e2e tests
* fixed merge errors
* Expanded FineGrainedAuthChecker with HasGlobalPermissionOnVertices and HasGlobalPermissionOnEdges
* Removed two wrong checks; Added two global checks
* return null added
* introduced new mgp_error value
* fixed endless loop
* replaced enum
* intermediate
* tests updated
* PermissionDeniedError -> AuthorizationError rename
* rename in enum permission_denied error -> authorization error
* mgp_vertex_remove_label check improved
* quotes changed; order of imports fixed
* string constant introduced
* import fixed
* yaml format
Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
