Commit Graph

16 Commits

Author SHA1 Message Date
andrejtonev
dbc6054689
Replication refactor (part 5) (#1378) 2023-11-06 11:50:49 +00:00
andrejtonev
bce48361ca
Decoupling Interpreter from Storage (#1186)
Unique/global InterpreterContext that is Storage agnostic (has a reference to the DbmsHandler instead)

* InterpreterContext is no longer the owner of Storage
* New Database structure that handles Storage, Triggers, Streams
* Renamed SessinContextHandler to DbmsHandler and simplified the multi-tenant logic
* Added Gatekeeper and updated handlers to use it

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2023-09-20 13:13:54 +02:00
Marko Budiselić
9d056e7649
Add experimental/v1 of ON_DISK_TRANSACTIONAL storage (#850)
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-06-29 11:44:55 +02:00
János Benjamin Antal
537855a0b2
Fix usages of constexpr (#367)
* Fix usages of constexpr
2022-03-31 13:52:43 +02:00
jbajic
12b4ec1589 Add memgraph namespace 2022-03-14 15:47:41 +01:00
János Benjamin Antal
6c00d146f2
Add configs for kafka streams (#328) 2022-01-31 17:26:53 +01:00
Antonio Andelic
d9bb4e2e46
Use default values instantly for batch size/interval (#306) 2021-11-30 16:29:51 +01:00
Antonio Andelic
0e4719018a Introduce pulsar dependency (#286) 2021-11-16 15:47:40 +01:00
Antonio Andelic
0ebd52aac3 Support for different consumers (#280) 2021-11-16 15:40:27 +01:00
Antonio Andelic
bd21bc82b7
Add license to cpp/hpp/py test files (#283) 2021-10-26 08:53:56 +02:00
János Benjamin Antal
09cfca35f8
Add privilege check in triggers and streams (#200) 2021-07-22 16:22:08 +02:00
János Benjamin Antal
fb5a2ed4b6 Small improvements for streams (#193)
* Add timeout to CHECK STREAM

* Handle RdKafka::ERR__MAX_POLL_EXCEEDED in consumer

Co-authored-by: Jure Bajic <jbajic@users.noreply.github.com>

Co-authored-by: Jure Bajic <jbajic@users.noreply.github.com>
2021-07-07 16:19:59 +02:00
János Benjamin Antal
ad32db5168 Use transformations in streams and CHECK STREAM (#185)
* Use the correct transformation result type

* Execute the result queries in streams

* Change the result type of parameters to nullable map

* Serialize transformation name

* Fix order of transformation parameters

* Use actual transformation in Streams

* Clear the Python transformations under GIL

* Add CHECK STREAM query

* Handle missing record fields properly
2021-07-07 16:19:59 +02:00
Kostas Kyrimis
a37755ce43 Add remaining query functionality SHOW STREAMS (#184)
* Add query functionality SHOW STREAMS
2021-07-07 16:13:26 +02:00
János Benjamin Antal
ac230d0c2d Add basic queries for managing streams (#179)
* Add CREATE, START, STOP and DROP queries

* Fix definition of port in replica query

* Explicitly stop the consumer before removing

* Fix offset committing in Consumer

* Add tests for basic stream queries

* Remove unnecessary WITH keywords from CREATE query

* Add tests

* Add STREAM privilege

* Disable not working test

The functionality is tested manually, but I couldn't make it work with
the mock kafka cluster.

* Add support for multiple topic names

* Replace skiplist by synchronized map

* Make Consumer::Test const and improve error handling

The improvement in the error handling is mostly done regarding to the
Test function. Instead of trying to revert the assignments, Test just
stores the last commited assignment. When Start or Test is called, they
check for the last commited assignments, and if it is saved, then they
try to restore it. This way:
1. All the failures are returned to the user (failed to save/restore)
2. Failed assignment cannot terminate Memgraph

* Make Test do not block creating/droping other streams
2021-07-07 16:08:22 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer (#172)
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00