Commit Graph

149 Commits

Author SHA1 Message Date
Josip Mrden
0a66feccff Merge branch 'master' into E129-MG-label-based-authorization 2022-09-06 11:14:27 +02:00
Boris Taševski
d008a2ad8d
[E129-MG < T1007-MG] Expand Cursors with LBA checks ()
* [T1007-MG < T0997-MG] Authorization on paths ()

* Added read authorization in paths operators

* [T1007-MG < T1016-MG] Added authorization in create and delete operators ()

* Added authorization in RemoveNodeCursor, RemoveExpandCursor, CreateNodeCursor, CreateExpandCursor,MergeCursor

* [T1007-MG < T1014-MG] Add authorization to read operators ()

Added label based access control to read operators (ScanAll).

* [T1007-MG < T1015-MG] Add authorization to update operators (SetProperty, SetProperties, RemoveProperty) ()

Added label based authorization to update operators

Co-authored-by: niko4299 <51059248+niko4299@users.noreply.github.com>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
2022-09-02 17:12:07 +02:00
Josipmrden
7478300762
[E129-MG < T997-MG] Show label privileges ()
Added showing of label privileges functionality to fine grained access control.
2022-08-31 12:14:16 +02:00
niko4299
e15576f56c
[E129-MG <-T0982-MG] implement edge type filtering ()
* GRANT, REVOKE, DENY and access_checker DONE

* Added AccessChecker to ExecutionContext

* grammar expanded; ()

* current

* T0954 mg expand user and role to hold permissions on labels ()

* added FineGrainedAccessPermissions class to model

* expanded user and role with fine grained access permissions

* fixed grammar

* [E129 < T0953-MG] GRANT, DENY, REVOKE added in interpreter and mainVisitor ()

* GRANT, DENY, REVOKE added in interpreter and mainVisitor

* Commented labelPermissons

* remove labelsPermission adding

* Fixed

* Removed extra lambda

* fixed

* [E129<-T0955-MG] Expand ExecutionContext with label related information ()

* added

* Added FineGrainedAccessChecker to Context

* fixed

* Added filtering

* testing

* Added edge filtering to storage, need to add filtering in simple Expand in operator.cpp

* Removed storage changes

* MATCH filtering working

* EdgeTypeFiltering working, just need to test everything again

* Removed FineGrainedAccessChecker

* Removed Expand Path

* Fix

* Tested FineGrainedAccessHandler, need to test AuthChecker

* Added integration test for lba

* Fixed merge conflicts

* PR fix

* fixed

* PR fix

* Fix test

* removed .vscode, .cache, .githooks

* githooks

* added tests

* fixed build

* Changed ast.lcp and User pointer to value in context.hpp

* Fixed test

* Remove denies on grant all

* AuthChecker

* Pr fix, auth_checker still not fixed

* Create mg-glue and extract UserBasedAuthChecker from AuthChecker

* Build fixed, need to fix test

* e2e tests

* e2e test working

* Added unit test, e2e and FineGrainedChecker

* Mege E129, auth_checker tests

* Fixed test

* e2e fix

Co-authored-by: Boris Taševski <36607228+BorisTasevski@users.noreply.github.com>
Co-authored-by: josipmrden <josip.mrden@external-basf.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
2022-08-16 15:57:23 +02:00
Jure Bajic
531db2d47c
Fix WebSocket test ()
* Fix websocket unit tests hanging
* Rename websocket to monitoring unit test
2022-08-08 14:49:48 +02:00
Jeremy B
f629de7e60
Save replication settings ()
* Storage takes care of the saving of setting when a new replica is added

* Restore replicas at startup

* Modify interactive_mg_runner + memgraph to support that data-directory can be configured in CONTEXT

* Extend e2e test

* Correct typo

* Add flag to config to specify when replication should be stored (true by default when starting Memgraph)

* Remove un-necessary "--" in yaml file

* Make sure Memgraph stops if a replica can't be restored.

* Add UT covering the parsing  of ReplicaStatus to/from json

* Add assert in e2e script to check that a port is free before using it

* Add test covering crash on Jepsen

* Make sure applciaiton crashes if it starts on corrupted replications' info

Starting with a non-reponsive replica is allowed.

* Add temporary startup flag: this is needed so jepsen do not automatically restore replica on startup of main. This will be removed in T0835
2022-07-07 13:30:28 +02:00
Josip Matak
4abaf27765
Memgraph magic functions ()
* Extend mgp_module with include adding functions

* Add return type to the function API

* Change Cypher grammar

* Add Python support for functions

* Implement error handling

* E2e tests for functions

* Write cpp e2e functions

* Create mg.functions() procedure

* Implement case insensitivity for user-defined Magic Functions.
2022-04-21 15:45:31 +02:00
Jure Bajic
bd2c30fddc Websocket unit tests () 2022-02-17 10:35:56 +01:00
Antonio Andelic
bb1308acc7
Use libs from toolchain () 2022-01-21 10:22:36 +01:00
Antonio Andelic
8716b8e992
Update to newest spdlog and fmt () 2021-12-07 11:05:33 +01:00
Antonio Andelic
0e4719018a Introduce pulsar dependency () 2021-11-16 15:47:40 +01:00
antonio2368
4e604de9d7 Add support for parsing temporal types () 2021-10-02 14:54:45 +02:00
antonio2368
e628b5ba6e Add temporal types to TypedValue () 2021-10-02 14:53:53 +02:00
antonio2368
5da32c1bff Add TemporalData to PropertyValue () 2021-10-02 14:53:53 +02:00
Benjamin Antal
482798295e Add write procedures 2021-10-02 13:17:41 +02:00
antonio2368
d58a1cbb58
License key introduction and removing community edition package () 2021-09-29 19:14:39 +02:00
antonio2368
9f855676cb
Add jemalloc purge () 2021-09-21 14:43:27 +02:00
Kostas Kyrimis
d80ff745eb Introduce mgp_trans api ()
* Added public interface for registering mgp_trans and extended modules accordingly

* Added test for mgp_trans

* Added mg.transformations() to the module registry

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
2021-07-07 15:57:36 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer ()
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00
Kostas Kyrimis
4004e94ca1 Introduce messages C-API () 2021-07-07 15:38:51 +02:00
János Benjamin Antal
36afc6c5f3 Add Kafka consumer ()
* Add CMake config for Kafka integration

* Add Consumer

* Add simple unit test for consumer

* Add explicit offset handling and test for it
2021-07-07 15:38:51 +02:00
antonio2368
15911b64dc
Use timers for query timeout thread ()
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-06-16 16:03:58 +02:00
antonio2368
8cd9f696cf
Multiple isolation levels ()
This PR introduces READ COMMITTED and READ UNCOMMITTED isolation levels.
The isolation level can be set with a config or with a query for different scopes.
2021-06-14 15:47:57 +02:00
antonio2368
62a628c51f Add trigger queries and trigger store ()
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:36:41 +02:00
antonio2368
883f9c7ed3 Trigger event types ()
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:13:14 +02:00
antonio2368
bbed7a2397 Add memory tracker definition ()
* Allow size 0 in MemoryTracker
* Block only exception throwing
* Subtract unsuccessfully allocated size in memory tracker
* Add oom exception enablers and blockers
2021-03-24 14:53:15 +01:00
antonio2368
e8810a4152 Add and use jemalloc lib ()
* Add and use jemalloc lib

* Add autoconf

* Silence macro redefinition warning
2021-03-24 14:53:15 +01:00
Josip Seljan
f950a91732 Add LOAD CSV clause infrastructure ()
* Add LOAD CSV clause infrastructure
* Add LoadCsv operator
* Update csv::Reader class
* Support csv files with and without header

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2021-03-24 12:21:07 +01:00
Josip Seljan
f6d5f576d5 CSV reader class ()
* CSV parsing class
* Basic CSV parsing tests
2021-03-24 12:21:07 +01:00
antonio2368
16715d5005
Enable replication in community ()
* Enable replication in community
2021-03-10 10:36:38 +01:00
antonio2368
f807b495ab
Update commit log after recovery ()
* Define additional commit log constructor which takes an oldest active id

* Delay commit log construction until the recovery process is finished

* Add test for commit log with initial id

* Silence the macro redefinition warning
2021-02-19 11:00:10 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user ()
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
Josip Seljan
8f5b88f24a Fix how query read/write type RWType is determined ()
Co-authored-by: jseljan <josip.seljan@memgraph.io>
2021-01-21 15:56:21 +01:00
antonio2368
03cc568e39 Add support for async replication ()
* Add thread pool
* Define async replication
* Expose replication state
* Rename TransactionHandler to ReplicaStream
2021-01-21 15:49:32 +01:00
antonio2368
42f6118c00 Add safe file deletion utility FileRetainer ()
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
Marko Budiselić
c68ed8d94e Add implementation of synchronous replication ()
This implements the initial version of synchronous replication.
Currently, only one replica is supported and that isn't configurable.

To run the main instance use the following command:
```
./memgraph \
    --main \
    --data-directory main-data \
    --storage-properties-on-edges \
    --storage-wal-enabled \
    --storage-snapshot-interval-sec 300
```

To run the replica instance use the following command:
```
./memgraph \
    --replica \
    --data-directory replica-data \
    --storage-properties-on-edges \
    --bolt-port 7688
```

You can then write/read data to Bolt port 7687 (the main instance) and also you
can read the data from the replica instance using Bolt port 7688.

NOTE: The main instance *must* be started without any data and the replica
*must* be started before any data is added to the main instance.

* Add basic synchronous replication test
* Using RWLock for replication stuff

Co-authored-by: Matej Ferencevic <matej.ferencevic@memgraph.io>
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
Matej Ferencevic
0dcfdb9b89 Add GitHub workflows 2020-09-21 12:22:40 +02:00
Matej Ferencevic
7e35798401 Remove leftover raft unit test
Reviewers: llugovic, buda

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2744
2020-04-01 13:48:02 +02:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
2020-03-02 12:45:18 +01:00
Teon Banek
0a7de969f3 Add converting mgp_value to py::Object
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2665
2020-02-12 09:30:14 +01:00
Matej Ferencevic
7128320b36 Implement MG_ENTERPRISE CMake flag
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2658
2020-02-06 13:39:30 +01:00
Matej Ferencevic
c96421424f Reorganize unit CMakeLists
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2651
2020-01-31 11:18:36 +01:00
Matej Ferencevic
de48548164 Move RPC to root source directory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2641
2020-01-27 11:50:24 +01:00
Matej Ferencevic
fd81ebdfe3 Move KVStore to root source directory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2640
2020-01-24 16:01:51 +01:00
Matej Ferencevic
0c7313bb5f Remove unused datastructures
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2639
2020-01-24 15:11:02 +01:00
Matej Ferencevic
6d10d90d98 Remove unused utils
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2636
2020-01-23 17:22:51 +01:00
Teon Banek
cc2160d397 Remove linking mg-query with mg-auth
Summary: Depends on D2623

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2624
2020-01-16 17:02:37 +01:00
Matej Ferencevic
4776bea221 Temporarily disable HA build and tests
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2619
2020-01-13 17:04:27 +01:00
Matej Ferencevic
5906258de0 Stop building storage v1
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2618
2020-01-13 14:18:12 +01:00
Teon Banek
551c6c5ad6 Add mg-query library and use it in unit tests
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2611
2020-01-08 14:33:35 +01:00