Commit Graph

65 Commits

Author SHA1 Message Date
Kostas Kyrimis
6eb52581eb Retry transaction on consumer (#294) 2021-11-16 15:58:36 +01:00
Antonio Andelic
b66cc66503 Add support for Pulsar queries (#292) 2021-11-16 15:52:58 +01:00
Antonio Andelic
0e4719018a Introduce pulsar dependency (#286) 2021-11-16 15:47:40 +01:00
Kostas Kyrimis
10196f3d7d
Add bootstrap servers to create stream (#274) 2021-10-18 10:49:00 +02:00
g-despot
9119c89eba
Add updated startup message (#259) 2021-10-07 15:56:06 +02:00
Antonio Andelic
4e7ea34ae9
Add message with link utility (#243) 2021-10-07 14:51:30 +02:00
Jure Bajic
9e678f8cbe Add new memgraph licenses and move third party licenses (#248) 2021-10-03 12:26:48 +02:00
antonio2368
d58a1cbb58
License key introduction and removing community edition package (#232) 2021-09-29 19:14:39 +02:00
János Benjamin Antal
09cfca35f8
Add privilege check in triggers and streams (#200) 2021-07-22 16:22:08 +02:00
János Benjamin Antal
ad32db5168 Use transformations in streams and CHECK STREAM (#185)
* Use the correct transformation result type

* Execute the result queries in streams

* Change the result type of parameters to nullable map

* Serialize transformation name

* Fix order of transformation parameters

* Use actual transformation in Streams

* Clear the Python transformations under GIL

* Add CHECK STREAM query

* Handle missing record fields properly
2021-07-07 16:19:59 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer (#172)
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00
antonio2368
f994b68ad5
Increase default timeout to 10 minutes (#186) 2021-07-02 09:02:11 +02:00
antonio2368
715162e205
Disable LOAD CSV with config (#180) 2021-06-30 11:19:13 +02:00
antonio2368
e016c74e4b
Trigger restore after query modules load (#183) 2021-06-30 10:01:51 +02:00
antonio2368
8cd9f696cf
Multiple isolation levels (#171)
This PR introduces READ COMMITTED and READ UNCOMMITTED isolation levels.
The isolation level can be set with a config or with a query for different scopes.
2021-06-14 15:47:57 +02:00
antonio2368
62a628c51f Add trigger queries and trigger store (#145)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-05-26 14:36:41 +02:00
antonio2368
9c6bf4b1b8 Updated storage info (#102)
* Refactor readable size function

* Log set memory limit

* Add memory allocation info to storage

* Updated changelog
2021-03-24 15:10:28 +01:00
antonio2368
cc56ac3dd8 Expose query for cleaning memory and add memory limit flag (#100)
* Add memory flag, add additional meminfo utilities

* Add free memory query
2021-03-24 15:10:28 +01:00
antonio2368
bbed7a2397 Add memory tracker definition (#93)
* Allow size 0 in MemoryTracker
* Block only exception throwing
* Subtract unsuccessfully allocated size in memory tracker
* Add oom exception enablers and blockers
2021-03-24 14:53:15 +01:00
antonio2368
593f7a3499
Add query module counters to telemetry (#112) 2021-03-18 11:03:42 +01:00
antonio2368
35d789c56b
Allow multiple folders for query modules (#105)
* Allow multiple query modules directories as arg
2021-03-10 12:18:09 +01:00
antonio2368
a9f5f45b3d
Improve telemetry data (#89)
* Add query type count to telemetry

* Extract telemetry data in different structure

* Add global event counter
2021-03-09 19:55:58 +01:00
antonio2368
3f3c55a4aa
Format all the memgraph and test source files (#97) 2021-02-18 15:32:43 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72)
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
Josip Seljan
a0fb3fc463 Implement ReplicationQueryHandler class (#52)
* Refactor io::network::Endpoint class
* Add ParseSocketOrIpAddress static method to Endpoint class
* Implement ReplQueryHandler methods
* Add implementation of SetReplicationRole to ReplQueryHandler
* Fix PrepareReplicationQuery (create PullPlanVector)

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2021-01-21 15:56:21 +01:00
antonio2368
2e7d3822fe
Disable SSL by default (#25)
* Print the connection type used (with/without SSL)
* Update CHANGELOG

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2020-10-20 15:49:10 +02:00
antonio2368
0a7d4278b1
Fix older version of SSL (#24)
* Add manual locking callback for SSL

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2020-10-20 12:55:13 +02:00
antonio2368
0bcc1d67bc
Add support for Bolt v4(.1) (#10)
* Added handshake support
* Add support for v4 hello and goodbye
* Add support for pulling n results
* Add support for transactions
* Add pull n for the dump
* Add support for NOOP
* Add support for multiple queries
* Update bolt session to support qid
* Update drivers test with multiple versions and go
* Extract failure handling into a function
* Use unique ptr instead of optional for query execution
* Destroy stream before query execution

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2020-10-16 12:49:33 +02:00
Lovro Lugovic
0c42bedf2f Add support for rescanning query modules
Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2765
2020-05-13 21:33:28 +02:00
Matej Ferencevic
e9ea8693fa Fix CentOS compilation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2755
2020-04-06 16:01:35 +02:00
Matej Ferencevic
1d2bb2cda6 Remove leftover traces of HA from query
Reviewers: llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2743
2020-04-01 13:47:47 +02:00
Matej Ferencevic
f48ad62647 Implement storage lock
Summary:
The storage now uses a file in the data directory (`.lock`) to determine
whether there is another instance of the storage running with the same data
directory. That helps notify the user/administrator that the system is running
in an unsupported configuration.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2719
2020-03-13 19:27:01 +01:00
Teon Banek
963d779d48 Install and setup loading Python Query Modules
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D2715
2020-03-11 10:24:53 +01:00
Matej Ferencevic
a456d6cdc0 Implement mg_import_csv
Summary:
This diff restores (and fixes) the old mg_import_csv implementation. The
importer now supports the new storage engine.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2690
2020-03-03 14:33:43 +01:00
Matej Ferencevic
4e5a91e7fb Merge memgraph_init with main file
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2684
2020-02-25 12:25:25 +01:00
Matej Ferencevic
e8903ef477 Expose auth user/role name regex
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2673
2020-02-17 17:40:53 +01:00
Teon Banek
ab53cbb0c3 Embed Python and import modules into global interpreter
Summary:
This only imports the `.py` files into the global interpreter. These
modules are not exposed to query execution. A later diff will add that
support.

Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2660
2020-02-10 11:06:08 +01:00
Matej Ferencevic
7128320b36 Implement MG_ENTERPRISE CMake flag
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2658
2020-02-06 13:39:30 +01:00
Matej Ferencevic
0683ed4134 Remove MG_SINGLE_NODE_V2 define
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2637
2020-01-23 17:24:33 +01:00
Ivan Paljak
3a88668ace Output Memgraph version to stdout on startup
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D2627
2020-01-17 13:04:05 +01:00
Teon Banek
c7f69fd861 Extract AuthQuery handling to an interface class
Reviewers: mferencevic, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2623
2020-01-16 17:02:35 +01:00
Matej Ferencevic
f27682dc26 Make telemetry work with v2 and increase timeout
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2601
2019-12-11 16:27:18 +01:00
Matej Ferencevic
d68106051c Replace hardcoded configuration with generator
Summary:
The configuration file that was used for the Debian/CentOS package was manually
written. This diff adds a configuration file generator that extracts all of the
necessary information about the flags directly from the built binary and uses
that information to generate the configuration file for the packages.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2589
2019-12-09 13:51:10 +01:00
Matej Ferencevic
eb38b4f373 Rename Bolt flags
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2587
2019-12-09 11:24:09 +01:00
Matej Ferencevic
9e04183be7 Implement query execution timeout
Summary:
The query execution now has a timeout for each Cypher query it executes. The
timeout is implemented using TSC and will work only when TSC is available (same
as PROFILE). TSC is used to mitigate the performance impact of reading the
current time constantly.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2562
2019-11-25 17:45:40 +01:00
Teon Banek
283a91cc60 Integrate loading openCypher module procedures
Summary:
All mgp_* symbols are exported from Memgraph executable, no other
symbols should be visible.

The primary C API header, mg_procedure.h, is now part of the
installation. Also, added a shippable query module example.

Directory `query_modules` is meant to contain sources of modules we
write and ship as part of the installation. Currently, there's only an
example module, but there may be potentially more. Some modules could
only be installed as part of the enterprise release.

For Memgraph to load custom procedures, it needs to be started with a
flag pointing to a directory with compiled shared libraries implementing
those procedures.

Reviewers: mferencevic, ipaljak, llugovic, dsantl, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2538
2019-11-07 15:23:02 +01:00
Matej Ferencevic
73134ab6b7 Integrate all storage v2 flags into main binary
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2528
2019-11-04 17:01:56 +01:00
Matej Ferencevic
42516afce8 Remove Kafka integration implementation and tests
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2525
2019-11-04 09:56:07 +01:00
Lovro Lugovic
3750fbc093 Merge Interpreter and TransactionEngine
Summary:
Depends on D2471

- Add pointer to storage to `InterpreterContext`
- Rename `operator()` to `Prepare`
- Use `Interpret` instead of `operator()` (`Interpret` will be removed soon)
- Remove the `in_explicit_transaction` parameter
- Remove the memory resource parameter from `Interpret`
- Remove the storage accessor parameter from `Interpret`
- Fix up tests (remove the `Interpreter` from `database_transaction_timeout`)

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2482
2019-10-25 16:11:10 +02:00
Lovro Lugovic
d2fac02b74 Pull out InterpreterContext
Summary: Make `InterpreterContext` a top level instead of a nested struct

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2512
2019-10-25 15:58:47 +02:00