Commit Graph

3210 Commits

Author SHA1 Message Date
János Benjamin Antal
ac230d0c2d Add basic queries for managing streams (#179)
* Add CREATE, START, STOP and DROP queries

* Fix definition of port in replica query

* Explicitly stop the consumer before removing

* Fix offset committing in Consumer

* Add tests for basic stream queries

* Remove unnecessary WITH keywords from CREATE query

* Add tests

* Add STREAM privilege

* Disable not working test

The functionality is tested manually, but I couldn't make it work with
the mock kafka cluster.

* Add support for multiple topic names

* Replace skiplist by synchronized map

* Make Consumer::Test const and improve error handling

The improvement in the error handling is mostly done regarding to the
Test function. Instead of trying to revert the assignments, Test just
stores the last commited assignment. When Start or Test is called, they
check for the last commited assignments, and if it is saved, then they
try to restore it. This way:
1. All the failures are returned to the user (failed to save/restore)
2. Failed assignment cannot terminate Memgraph

* Make Test do not block creating/droping other streams
2021-07-07 16:08:22 +02:00
Kostas Kyrimis
d80ff745eb Introduce mgp_trans api (#175)
* Added public interface for registering mgp_trans and extended modules accordingly

* Added test for mgp_trans

* Added mg.transformations() to the module registry

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
2021-07-07 15:57:36 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer (#172)
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00
Kostas Kyrimis
4004e94ca1 Introduce messages C-API (#168) 2021-07-07 15:38:51 +02:00
János Benjamin Antal
36afc6c5f3 Add Kafka consumer (#167)
* Add CMake config for Kafka integration

* Add Consumer

* Add simple unit test for consumer

* Add explicit offset handling and test for it
2021-07-07 15:38:51 +02:00
antonio2368
3c9a46f823
Add Jure to CODEOWNERS (#194) 2021-07-07 12:35:28 +02:00
antonio2368
f994b68ad5
Increase default timeout to 10 minutes (#186) 2021-07-02 09:02:11 +02:00
antonio2368
3b336e3e0b
Add CREATE SNAPSHOT query (#182) 2021-06-30 12:31:30 +02:00
antonio2368
715162e205
Disable LOAD CSV with config (#180) 2021-06-30 11:19:13 +02:00
antonio2368
e016c74e4b
Trigger restore after query modules load (#183) 2021-06-30 10:01:51 +02:00
antonio2368
15911b64dc
Use timers for query timeout thread (#163)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-06-16 16:03:58 +02:00
antonio2368
cbf826e0c3
Load WAL on replica using transactions (#95) 2021-06-16 13:22:48 +02:00
antonio2368
644a3a0b2a
Codeowners update (#173)
* CODEOWNERS update

* clang-tidy changes trigger the diff
2021-06-15 13:21:05 +02:00
antonio2368
8cd9f696cf
Multiple isolation levels (#171)
This PR introduces READ COMMITTED and READ UNCOMMITTED isolation levels.
The isolation level can be set with a config or with a query for different scopes.
2021-06-14 15:47:57 +02:00
antonio2368
90a093bd95
Send py::Object as a reference to callback call (#169)
Sending the py::Object by value caused UB because multiple threads could copy the same object at the same time without the GIL.
By sending the object by reference we eliminate UB and avoid UB.
2021-06-10 10:40:18 +02:00
Marko Budiselić
542a324c96
Fix typing to cypher type impl inside mgp.py (#159) 2021-06-09 19:09:43 +02:00
Josip Seljan
cd03e13443
Upgrade Antlr to v4.9.2 (#161) 2021-06-09 12:50:27 +02:00
antonio2368
be91126134
Run the benchmarks daily (#164) 2021-06-08 08:44:12 +02:00
antonio2368
03cb007339
CHANGELOG redirects to the docs page (#165) 2021-06-07 16:26:15 +02:00
Kostas Kyrimis
524acb17a1
Add global version allocators for C in query modules (#162) 2021-06-07 14:45:05 +02:00
antonio2368
50f6e348dc
Disable google-runtime-references clang-tidy check (#166) 2021-06-07 10:40:56 +02:00
Josip Seljan
839d45b3f8
Remove louvain and connectivity modules (#160) 2021-06-02 11:10:16 +02:00
antonio2368
560eb04f67
Small trigger fixes (#158)
* Fix warning message

* Update version

* Run query callbacks only on pull

* Use warn level for failure of loading a trigger
2021-05-26 19:57:08 +02:00
János Benjamin Antal
a3ecc52429 Use all available predefined variables (#157) 2021-05-26 14:37:11 +02:00
János Benjamin Antal
e8a1d15a55 Add filtering based on registered event types (#155)
* Add filtering to TriggerContextCollector

* Add all predefined variable to ANY triggers

* Make variable names consistent with event types
2021-05-26 14:37:11 +02:00
János Benjamin Antal
1abee1ed3a Use single thread for after triggers (#154) 2021-05-26 14:36:41 +02:00
János Benjamin Antal
5af3d0ff68 Add e2e tests for triggers (#152) 2021-05-26 14:36:41 +02:00
antonio2368
62a628c51f Add trigger queries and trigger store (#145)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:36:41 +02:00
antonio2368
883f9c7ed3 Trigger event types (#144)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:13:14 +02:00
antonio2368
b459639968 Other predefined variables (#143) 2021-05-26 14:13:14 +02:00
antonio2368
11c0dde11c Support removed vertices in the triggers (#140) 2021-05-26 14:13:14 +02:00
antonio2368
2f3fa656d9 Predefined variables (#138)
* Add support for injecting predefined variables in any scope
2021-05-26 14:13:14 +02:00
antonio2368
7bf40eb5d2 Add support for after commit triggers (#136) 2021-05-26 14:13:14 +02:00
antonio2368
7e44434cdf Initial trigger definition (#133)
* Pull out cypher query parsing logic

* Define trigger structure

* Run triggers before commit

* Use skip list for saving triggers
2021-05-26 14:13:14 +02:00
Josip Seljan
2a0b0d969f
Replace mg_client with mgconsole (#153) 2021-05-26 11:59:36 +02:00
antonio2368
13ea35af2d
Fix permission for newer queries (#156) 2021-05-26 10:02:25 +02:00
Kostas Kyrimis
8a99670301
Add init toolchain-v3 script and support for zsh (#149) 2021-05-21 12:00:24 +02:00
Mislav Vuletić
c4555f5448
Fix Arch package help message typo (#119) 2021-05-20 16:04:50 +02:00
antonio2368
999b3ef79f
Fix memory tracking issues (#150) 2021-05-18 18:42:19 +02:00
antonio2368
30413a7b4f
Ignore carriage return at the end of line for a CSV file (#151) 2021-05-18 13:44:29 +02:00
Marko Budiselić
1def0c9104
Fix libs/setup.sh nocached download (#148) 2021-05-13 12:05:36 +02:00
Marko Budiselić
782c377f5d
Add benchmarking GHA Diff job (#147) 2021-05-12 17:17:57 +02:00
Marko Budiselić
cc27a04139
Add bench-graph-client inside tools (#139) 2021-05-07 15:54:09 +02:00
Marko Budiselić
b71345655f
Add balancing of GHA jobs across runners (#146) 2021-05-07 13:37:45 +02:00
antonio2368
ccdd58b336
Fix parsing nested complex types (#142)
* Fix parsing of types for mgp.List
2021-04-27 08:31:18 +02:00
antonio2368
50b6afd73d
Fix clang tidy diff (#141)
* Ignore header files for clang-tidy-diff

* Rename code analysis job
2021-04-21 16:26:13 +02:00
Marko Budiselić
59105f68bd
Add primary and secondary dependency links (#137)
* Add explicit exit in case of git checkout or cherry-pick
2021-04-20 10:54:54 +02:00
antonio2368
8de31092ad
Add back the query memory limit logic (#134) 2021-04-13 10:41:50 +02:00
antonio2368
5c93f81881
Disable failing tests and add logs for replication e2e (#132)
* Disable sequential test

* Remove parent build and benchmark

* Save test data

* Save e2e logs in build folder

* Define different recovery time for each test
2021-04-02 12:29:10 +02:00
János Benjamin Antal
6d4fe5cdd5
Explore and implement the usage of clang-tidy and sanitizers (#125)
* Run clang-tidy on the full code base

* Run clang-tidy on diffs

* Enable ASAN in coverage build

* Add UBSAN to code analysis
2021-04-01 17:08:40 +02:00