Commit Graph

171 Commits

Author SHA1 Message Date
Antonio Filipovic
a84f570c6d
Use extent hooks for per query memory limit (#1340) 2023-10-25 16:01:59 +02:00
andrejtonev
22d8ef75e0
Updated telemetry client-side (#1337) 2023-10-16 14:16:00 +02:00
Gareth Andrew Lloyd
d278a33f31
Decouple pure replication state from storage [part 1] (#1325)
A major refactor to decouple replication state from storage.
ATM it is still owned by storage but a following part should fix that.
2023-10-10 11:44:19 +01:00
Gareth Andrew Lloyd
3cc2bc2791
Refactor interpreter to support multiple distributed clocks (Part 1) (#1281)
* Interpreter transaction ID decoupled from storage transaction ID
* Transactional scope for indices, statistics and constraints
* Storage::Accessor now has 2 modes (unique and shared)
* Introduced ResourceLock to fix pthread mutex problems
* Split InfoQuery in two: non-transactional SystemInfoQuery and transactional DatabaseInfoQuery
* Replicable and durable statistics
* Bumped WAL/Snapshot versions
* Initial implementation of the Lamport clock

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
2023-10-05 16:58:39 +02:00
andrejtonev
bce48361ca
Decoupling Interpreter from Storage (#1186)
Unique/global InterpreterContext that is Storage agnostic (has a reference to the DbmsHandler instead)

* InterpreterContext is no longer the owner of Storage
* New Database structure that handles Storage, Triggers, Streams
* Renamed SessinContextHandler to DbmsHandler and simplified the multi-tenant logic
* Added Gatekeeper and updated handlers to use it

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2023-09-20 13:13:54 +02:00
Andi
4b3ba908c7
Code improvements on disk storage (#1153)
* Improvements based on a code review

---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-08-26 14:16:12 +02:00
andrejtonev
e8850549d2
Add multi-tenancy v1 (#952)
* Decouple BoltSession and communication::bolt::Session
* Add CREATE/USE/DROP DATABASE
* Add SHOW DATABASES
* Cover WebSocket session
* Simple session safety implemented via RWLock
* Storage symlinks for backward. compatibility
* Extend the audit log with the DB info
* Add auth part
* Add tenant recovery
2023-08-01 18:49:11 +02:00
Gareth Andrew Lloyd
8ebab84324
Add handling of partial results on timeout (#1046) 2023-07-30 10:48:11 +02:00
Marko Budiselić
9d056e7649
Add experimental/v1 of ON_DISK_TRANSACTIONAL storage (#850)
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-06-29 11:44:55 +02:00
Antonio Filipovic
d917c3f0fd
Fix slow IN LIST evaluation (#901) 2023-05-29 17:52:20 +02:00
andrejtonev
d842adbed3
Handle user-defined metadata and expose it with SHOW TRANSACTIONS(#945) 2023-05-29 11:40:14 +02:00
Josipmrden
651b6f3a5a Expose system metrics over HTTP Endpoint (#940) 2023-05-18 05:10:57 +00:00
Antonio Filipovic
208705f296
Reduce memory consumption on return from python procedures (#932) 2023-05-16 10:33:09 +02:00
Antonio Filipovic
2c6a55775d
Fix max block size bug on LOAD CSV(#877) 2023-04-19 16:10:20 +02:00
Antonio Filipovic
a586f2f98d
Change EvalContext and QueryExecution to use PoolResource on LOAD CSV (#825)
* Change PullPlan to use specific PoolResource for LOAD CSV
2023-04-04 16:54:08 +02:00
Josipmrden
398503da7a
Add index statistics for better query planning (#812) 2023-03-30 15:34:34 +02:00
Andi
029be10f1d
Add queries to show or terminate active transactions (#790) 2023-03-27 15:46:00 +02:00
Boris Taševski
fb4641a6be
Fix logic in fine grained permissions (#551) 2022-09-14 12:39:23 +02:00
niko4299
201f75e809
Add MG_ENTERPRISE and license checks (#547) 2022-09-14 01:10:28 +02:00
Boris Taševski
05f120b7d4
[E129-MG < T1004-MG] Expand cypher with more granular label permissions (#500)
* Added enum for more granular access control; Expanded functionality of fine grained access checker; Propagated changes to Edit, Deny and Revoke permissions methods in interpreter

* Introduced Merge method for merging two colle with permissions

* e2e tests implementation started

* Expanded cypher to support fine grained permissions

* ast.lcp::AuthQuery removed labels, added support for label permissions

* promoted label permissions to vector

* removed unnecesary enum value

* expanded glue/auth with LabelPrivilegeToLabelPermission

* added const

* extended Grant Deny and Revoke Privileges with new label privileges

* extended Edit Grant Deny and Revoke Privileges to properly use new model

* Fixed unit tests

* FineGrainedAccessChecker Grant and Deny methods reworked

* Revoke cypher slightly reworked; Revoke for labels works without label permissions

* EditPermission's label_permission lambda now takes two parameters

* constants naming enforced; replaced asterisks with string constant

* removed faulty test addition

* Naming fixes; FineGrainedAccessChecker unit tests introduced

* unnecessary includes removed; minor code improvements

* minor fix

* Access checker reworked; denies and grant merged into single permission object; Created global_permission that applies to all non-created permissions. Grant, Deny and Revoke reworked; Merge method reworked

* Fixed wrong check;

* Fix after merge; renamed constants; removed unused constant

* Fix after merge; workloads.yaml for lbaprocedures e2e tests updated with new grammar

* Fixes after merge

* Fixes after merge

* fixed Revoke that was not fixed after the merge

* updated cypher main visitor tests

* PR review changes; Naming and const fixed, replaced double tertiary with lambda

* unwrapping the iterator fix

* merge 1003 minor fix

* minor spelling fixes

* Introduced visitPrivilegesList because of the doubled code

* const added

* string const to enum

* redundant braces

* added const

* minor code improvement

* e2e tests expanded

* if -> switch

* enum class inherits uint8_t now

* LabelPrililege::EDIT -> LabelPrivilege::UPDATE

* LabelPermission -> EntityPermission; LabelPrivilege -> EntityPrivilege

* EntityPrivilege -> FineGrainedPrivilege; EntityPermission -> FineGrainedPermission
2022-08-22 14:11:43 +02:00
niko4299
e15576f56c
[E129-MG <-T0982-MG] implement edge type filtering (#489)
* GRANT, REVOKE, DENY and access_checker DONE

* Added AccessChecker to ExecutionContext

* grammar expanded; (#462)

* current

* T0954 mg expand user and role to hold permissions on labels (#465)

* added FineGrainedAccessPermissions class to model

* expanded user and role with fine grained access permissions

* fixed grammar

* [E129 < T0953-MG] GRANT, DENY, REVOKE added in interpreter and mainVisitor (#464)

* GRANT, DENY, REVOKE added in interpreter and mainVisitor

* Commented labelPermissons

* remove labelsPermission adding

* Fixed

* Removed extra lambda

* fixed

* [E129<-T0955-MG] Expand ExecutionContext with label related information (#467)

* added

* Added FineGrainedAccessChecker to Context

* fixed

* Added filtering

* testing

* Added edge filtering to storage, need to add filtering in simple Expand in operator.cpp

* Removed storage changes

* MATCH filtering working

* EdgeTypeFiltering working, just need to test everything again

* Removed FineGrainedAccessChecker

* Removed Expand Path

* Fix

* Tested FineGrainedAccessHandler, need to test AuthChecker

* Added integration test for lba

* Fixed merge conflicts

* PR fix

* fixed

* PR fix

* Fix test

* removed .vscode, .cache, .githooks

* githooks

* added tests

* fixed build

* Changed ast.lcp and User pointer to value in context.hpp

* Fixed test

* Remove denies on grant all

* AuthChecker

* Pr fix, auth_checker still not fixed

* Create mg-glue and extract UserBasedAuthChecker from AuthChecker

* Build fixed, need to fix test

* e2e tests

* e2e test working

* Added unit test, e2e and FineGrainedChecker

* Mege E129, auth_checker tests

* Fixed test

* e2e fix

Co-authored-by: Boris Taševski <36607228+BorisTasevski@users.noreply.github.com>
Co-authored-by: josipmrden <josip.mrden@external-basf.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
2022-08-16 15:57:23 +02:00
Boris Taševski
480df4ed69
Merge old Label Based Auth Epic branch into new one because of commits with bad checks on the old epic branch (#478)
* grammar expanded; (#462)

* T0954 mg expand user and role to hold permissions on labels (#465)

* added FineGrainedAccessPermissions class to model

* expanded user and role with fine grained access permissions

* fixed grammar

* [E129 < T0953-MG] GRANT, DENY, REVOKE added in interpreter and mainVisitor (#464)

* GRANT, DENY, REVOKE added in interpreter and mainVisitor

* Commented labelPermissons

* remove labelsPermission adding

* Removed extra lambda

* [E129<-T0955-MG] Expand ExecutionContext with label related information (#467)

* Added FineGrainedAccessChecker to Context

* fixed failing tests for label based authorization (#480)

* Marked FineGrainedAccessChecker ctor explicit; Introduced change to clang-tidy; (#483)

Co-authored-by: niko4299 <51059248+niko4299@users.noreply.github.com>
2022-08-02 12:51:22 +02:00
Marko Budiselić
74d3663821
Upgrade Antrl to 4.10.1 and remove antlr_lock (#441) 2022-07-26 08:31:38 +02:00
Jeremy B
b737e53456
Remove sync with timeout (#423)
* Remove timout when registering a sync replica

* Simplify jepsen configuration file

* Remove timeout from jepsen configuration

* Add unit test

* Remove TimeoutDispatcher
2022-07-05 09:40:50 +02:00
Jeremy B
65a7ba01da
Add information on show replicas to express how up-to-date a replica is (#412)
* Add test

* Add implementation and adapted test

* Update workloads.yaml to have a timeout > 0

* Update tests (failing due to merging of "add replica state")
2022-06-23 10:22:57 +02:00
Marko Budiselić
599c0a641f
Add replica state to SHOW REPLICAS (#379) 2022-06-20 13:28:42 +03:00
Marko Budiselić
8e3ab1ad0f
Add frequent replica ping (#380) 2022-05-20 19:29:17 -07:00
János Benjamin Antal
537855a0b2
Fix usages of constexpr (#367)
* Fix usages of constexpr
2022-03-31 13:52:43 +02:00
jbajic
60ad05acff Address review comments 2022-03-14 15:47:41 +01:00
jbajic
12b4ec1589 Add memgraph namespace 2022-03-14 15:47:41 +01:00
Antonio Andelic
d9bb4e2e46
Use default values instantly for batch size/interval (#306) 2021-11-30 16:29:51 +01:00
Antonio Andelic
0e4719018a Introduce pulsar dependency (#286) 2021-11-16 15:47:40 +01:00
Antonio Andelic
0ebd52aac3 Support for different consumers (#280) 2021-11-16 15:40:27 +01:00
Jure Bajic
16709dff6c
Add stats and notifications in results metadata (#285) 2021-11-15 13:51:13 +01:00
Jure Bajic
9e678f8cbe Add new memgraph licenses and move third party licenses (#248) 2021-10-03 12:26:48 +02:00
antonio2368
d58a1cbb58
License key introduction and removing community edition package (#232) 2021-09-29 19:14:39 +02:00
János Benjamin Antal
09cfca35f8
Add privilege check in triggers and streams (#200) 2021-07-22 16:22:08 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer (#172)
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00
antonio2368
715162e205
Disable LOAD CSV with config (#180) 2021-06-30 11:19:13 +02:00
antonio2368
e016c74e4b
Trigger restore after query modules load (#183) 2021-06-30 10:01:51 +02:00
antonio2368
8cd9f696cf
Multiple isolation levels (#171)
This PR introduces READ COMMITTED and READ UNCOMMITTED isolation levels.
The isolation level can be set with a config or with a query for different scopes.
2021-06-14 15:47:57 +02:00
antonio2368
560eb04f67
Small trigger fixes (#158)
* Fix warning message

* Update version

* Run query callbacks only on pull

* Use warn level for failure of loading a trigger
2021-05-26 19:57:08 +02:00
János Benjamin Antal
1abee1ed3a Use single thread for after triggers (#154) 2021-05-26 14:36:41 +02:00
antonio2368
62a628c51f Add trigger queries and trigger store (#145)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:36:41 +02:00
antonio2368
883f9c7ed3 Trigger event types (#144)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:13:14 +02:00
antonio2368
11c0dde11c Support removed vertices in the triggers (#140) 2021-05-26 14:13:14 +02:00
antonio2368
2f3fa656d9 Predefined variables (#138)
* Add support for injecting predefined variables in any scope
2021-05-26 14:13:14 +02:00
antonio2368
7bf40eb5d2 Add support for after commit triggers (#136) 2021-05-26 14:13:14 +02:00
antonio2368
7e44434cdf Initial trigger definition (#133)
* Pull out cypher query parsing logic

* Define trigger structure

* Run triggers before commit

* Use skip list for saving triggers
2021-05-26 14:13:14 +02:00
antonio2368
999b3ef79f
Fix memory tracking issues (#150) 2021-05-18 18:42:19 +02:00