Commit Graph

303 Commits

Author SHA1 Message Date
antonio2368
4e604de9d7 Add support for parsing temporal types (#187) 2021-10-02 14:54:45 +02:00
antonio2368
5da32c1bff Add TemporalData to PropertyValue (#174) 2021-10-02 14:53:53 +02:00
Benjamin Antal
482798295e Add write procedures 2021-10-02 13:17:41 +02:00
antonio2368
d58a1cbb58
License key introduction and removing community edition package (#232) 2021-09-29 19:14:39 +02:00
antonio2368
9f855676cb
Add jemalloc purge (#239) 2021-09-21 14:43:27 +02:00
János Benjamin Antal
2afc1b99f6
New MGP API (#219) 2021-09-09 10:44:47 +02:00
Jure Bajic
0138d277d4
Improve serialization error message (#208) 2021-08-20 10:47:07 +02:00
antonio2368
e51954fc94
Update toolchain to v3 (#189)
* Make memgraph buildable with new toolchain

* Use toolchain v3 in workflows
2021-07-08 14:20:48 +02:00
Kostas Kyrimis
2e1a717dcb Add telemetry for streams and triggers (#188)
* Add event counters to streams and triggers
2021-07-07 16:19:59 +02:00
János Benjamin Antal
d6a6d280dd Add Streams on top of Kafka Consumer (#172)
* Stop the Consumer grafefully when it is destroyed

* Add Streams

* Add Streams to InterpreterContext

* Remove options to limit processed batches in Consumer

* Add Streams unit tests

* Stop waiting for a full batch if the Consumer stopped

* Add ReadLock functionality to Synchronized

* Use per Consumer-based locking

* Replace shared_mutex with RWLock
2021-07-07 15:57:36 +02:00
János Benjamin Antal
36afc6c5f3 Add Kafka consumer (#167)
* Add CMake config for Kafka integration

* Add Consumer

* Add simple unit test for consumer

* Add explicit offset handling and test for it
2021-07-07 15:38:51 +02:00
antonio2368
15911b64dc
Use timers for query timeout thread (#163)
Co-authored-by: Benjamin Antal <benjamin.antal@memgraph.io>
2021-06-16 16:03:58 +02:00
antonio2368
b459639968 Other predefined variables (#143) 2021-05-26 14:13:14 +02:00
antonio2368
999b3ef79f
Fix memory tracking issues (#150) 2021-05-18 18:42:19 +02:00
antonio2368
30413a7b4f
Ignore carriage return at the end of line for a CSV file (#151) 2021-05-18 13:44:29 +02:00
János Benjamin Antal
6d4fe5cdd5
Explore and implement the usage of clang-tidy and sanitizers (#125)
* Run clang-tidy on the full code base

* Run clang-tidy on diffs

* Enable ASAN in coverage build

* Add UBSAN to code analysis
2021-04-01 17:08:40 +02:00
antonio2368
276e09d7d3
Fix reading rows with empty columns at the end (#127)
* Fix reading rows with empty columns at the end

* Update CHANGELOG for the recovery logs
2021-03-27 09:47:41 +01:00
János Benjamin Antal
06f761bdf9
Add logs for loading snapshot and WAL files (#121)
* Add logs for loading snapshot and WAL files
2021-03-26 15:02:35 +01:00
antonio2368
9c6bf4b1b8 Updated storage info (#102)
* Refactor readable size function

* Log set memory limit

* Add memory allocation info to storage

* Updated changelog
2021-03-24 15:10:28 +01:00
antonio2368
cc56ac3dd8 Expose query for cleaning memory and add memory limit flag (#100)
* Add memory flag, add additional meminfo utilities

* Add free memory query
2021-03-24 15:10:28 +01:00
antonio2368
dee885d69c Add exception enablers and blockers (#99)
* Throw OOMException while creating vertices and edges

* Throw on indices creation

* Throw on setting a property

* Throw oom exception while recovering

* Throw exception when query engine asks for extra memory

* Block out of memor exception during skip list GC
2021-03-24 14:53:15 +01:00
antonio2368
bbed7a2397 Add memory tracker definition (#93)
* Allow size 0 in MemoryTracker
* Block only exception throwing
* Subtract unsuccessfully allocated size in memory tracker
* Add oom exception enablers and blockers
2021-03-24 14:53:15 +01:00
antonio2368
e8810a4152 Add and use jemalloc lib (#90)
* Add and use jemalloc lib

* Add autoconf

* Silence macro redefinition warning
2021-03-24 14:53:15 +01:00
Josip Seljan
25eb2c147a LOAD CSV implementation fixes (#120)
* Change how csv::Reader handles memory resources
* Add multiline quoted string test
2021-03-24 12:21:07 +01:00
antonio2368
d913a67e16 Add parsing improvements (#108)
* Add parsing improvements
2021-03-24 12:21:07 +01:00
Josip Seljan
f950a91732 Add LOAD CSV clause infrastructure (#101)
* Add LOAD CSV clause infrastructure
* Add LoadCsv operator
* Update csv::Reader class
* Support csv files with and without header

Co-authored-by: jseljan <josip.seljan@memgraph.io>
2021-03-24 12:21:07 +01:00
Josip Seljan
f6d5f576d5 CSV reader class (#98)
* CSV parsing class
* Basic CSV parsing tests
2021-03-24 12:21:07 +01:00
antonio2368
a9f5f45b3d
Improve telemetry data (#89)
* Add query type count to telemetry

* Extract telemetry data in different structure

* Add global event counter
2021-03-09 19:55:58 +01:00
antonio2368
f807b495ab
Update commit log after recovery (#96)
* Define additional commit log constructor which takes an oldest active id

* Delay commit log construction until the recovery process is finished

* Add test for commit log with initial id

* Silence the macro redefinition warning
2021-02-19 11:00:10 +01:00
antonio2368
3f3c55a4aa
Format all the memgraph and test source files (#97) 2021-02-18 15:32:43 +01:00
Marko Budiselić
fae407d3fe
Add implementation of average number of equals estimate in SkipList (#9)
* Implement average number of equals estimate in SkipList
2021-02-10 14:38:54 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72)
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
antonio2368
a84ffe86c1 Add LOCK DATA DIRECTORY query (#68) 2021-01-21 15:56:21 +01:00
antonio2368
200ce5f45e Add configs and support for semi-sync and SSL (#55)
* Add config for replication client/server
* Add SSL to replication
* Add semi-sync replication
* Expose necessary information about replication
* Thread pool fix
* Set BasicResult value type to void
2021-01-21 15:49:32 +01:00
antonio2368
76f0d5873b Use unfinished tasks num instead of idle thread num (#45) 2021-01-21 15:49:32 +01:00
antonio2368
03cc568e39 Add support for async replication (#41)
* Add thread pool
* Define async replication
* Expose replication state
* Rename TransactionHandler to ReplicaStream
2021-01-21 15:49:32 +01:00
antonio2368
bc0c944910 Add replica recovery process (#40)
* Add file transfer over RPC
* Snapshot transfer implementation
* Allow snapshot creation only for MAIN instances
* Replica and main can have replication clients
* Use only snapshots and WALs that are from the Main storage
* Add flush lock and expose buffer
* Add fstat for file size and TryFlushing method
* Use lseek for size

Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
antonio2368
42f6118c00 Add safe file deletion utility FileRetainer (#38)
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
2021-01-21 15:49:32 +01:00
antonio2368
4e7c569071 Migrate code to C++20 (#44) 2020-11-17 11:03:08 +01:00
Matej Ferencevic
aaf0c1ca08 Improve SLK
Summary:
SLK now correctly handles different CPU architectures (BIG/little endian).
Also, more string encoding functions have been added.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2806
2020-08-14 15:27:20 +02:00
Matej Ferencevic
f48ad62647 Implement storage lock
Summary:
The storage now uses a file in the data directory (`.lock`) to determine
whether there is another instance of the storage running with the same data
directory. That helps notify the user/administrator that the system is running
in an unsupported configuration.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2719
2020-03-13 19:27:01 +01:00
Matej Ferencevic
5695774251 Improve CSV importer
Summary:
The importer now supports all of the flags that the modern Neo4j CSV importer
supports.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2709
2020-03-10 15:51:29 +01:00
Teon Banek
d0411510fa Add GetExecutablePath to utils/file
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2713
2020-03-10 11:06:18 +01:00
Matej Ferencevic
000d6dba55 Fix utils::SkipList const iterator bug
Summary:
This issue was already fixed in D2119 for the iterator, but I missed to fix the
const iterator in that diff...

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2703
2020-03-05 11:17:26 +01:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
2020-03-02 12:45:18 +01:00
Matej Ferencevic
cfbfeb1dfc Remove TotalOrdering from utils::Timestamp
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2646
2020-01-28 10:40:55 +01:00
Matej Ferencevic
015573b5af Clean-up utils/hashing
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2645
2020-01-28 10:40:43 +01:00
Matej Ferencevic
6d10d90d98 Remove unused utils
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2636
2020-01-23 17:22:51 +01:00
Matej Ferencevic
9cc204793d Remove copy constructor/assignment from utils::SpinLock
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2581
2019-12-04 14:08:58 +01:00
Matej Ferencevic
9e04183be7 Implement query execution timeout
Summary:
The query execution now has a timeout for each Cypher query it executes. The
timeout is implemented using TSC and will work only when TSC is available (same
as PROFILE). TSC is used to mitigate the performance impact of reading the
current time constantly.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2562
2019-11-25 17:45:40 +01:00