tursom/memgraph
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,980 Commits 247 Branches 35 Tags 62 MiB
11c0dde11c
Commit Graph

124 Commits

Author SHA1 Message Date
antonio2368
bbed7a2397 Add memory tracker definition (#93) 
* Allow size 0 in MemoryTracker
* Block only exception throwing
* Subtract unsuccessfully allocated size in memory tracker
* Add oom exception enablers and blockers
 2021-03-24 14:53:15 +01:00
antonio2368
e8810a4152 Add and use jemalloc lib (#90) 
* Add and use jemalloc lib

* Add autoconf

* Silence macro redefinition warning
 2021-03-24 14:53:15 +01:00
Josip Seljan
f950a91732 Add LOAD CSV clause infrastructure (#101) 
* Add LOAD CSV clause infrastructure
* Add LoadCsv operator
* Update csv::Reader class
* Support csv files with and without header

Co-authored-by: jseljan <josip.seljan@memgraph.io>
 2021-03-24 12:21:07 +01:00
Josip Seljan
f6d5f576d5 CSV reader class (#98) 
* CSV parsing class
* Basic CSV parsing tests
 2021-03-24 12:21:07 +01:00
antonio2368
16715d5005
Enable replication in community (#104) 
* Enable replication in community
 2021-03-10 10:36:38 +01:00
antonio2368
f807b495ab
Update commit log after recovery (#96) 
* Define additional commit log constructor which takes an oldest active id

* Delay commit log construction until the recovery process is finished

* Add test for commit log with initial id

* Silence the macro redefinition warning
 2021-02-19 11:00:10 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72) 
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
 2021-01-21 16:30:55 +01:00
Josip Seljan
8f5b88f24a Fix how query read/write type RWType is determined (#71) 
Co-authored-by: jseljan <josip.seljan@memgraph.io>
 2021-01-21 15:56:21 +01:00
antonio2368
03cc568e39 Add support for async replication (#41) 
* Add thread pool
* Define async replication
* Expose replication state
* Rename TransactionHandler to ReplicaStream
 2021-01-21 15:49:32 +01:00
antonio2368
42f6118c00 Add safe file deletion utility FileRetainer (#38) 
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
 2021-01-21 15:49:32 +01:00
Marko Budiselić
c68ed8d94e Add implementation of synchronous replication (#7) 
This implements the initial version of synchronous replication.
Currently, only one replica is supported and that isn't configurable.

To run the main instance use the following command:
```
./memgraph \
    --main \
    --data-directory main-data \
    --storage-properties-on-edges \
    --storage-wal-enabled \
    --storage-snapshot-interval-sec 300
```

To run the replica instance use the following command:
```
./memgraph \
    --replica \
    --data-directory replica-data \
    --storage-properties-on-edges \
    --bolt-port 7688
```

You can then write/read data to Bolt port 7687 (the main instance) and also you
can read the data from the replica instance using Bolt port 7688.

NOTE: The main instance *must* be started without any data and the replica
*must* be started before any data is added to the main instance.

* Add basic synchronous replication test
* Using RWLock for replication stuff

Co-authored-by: Matej Ferencevic <matej.ferencevic@memgraph.io>
Co-authored-by: Antonio Andelic <antonio.andelic@memgraph.io>
 2021-01-21 15:49:32 +01:00
Matej Ferencevic
0dcfdb9b89 Add GitHub workflows 2020-09-21 12:22:40 +02:00
Matej Ferencevic
7e35798401 Remove leftover raft unit test 
Reviewers: llugovic, buda

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2744
 2020-04-01 13:48:02 +02:00
Matej Ferencevic
0e95934719 Remove leftover implementation required by the old storage 
Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek, ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2687
 2020-03-02 12:45:18 +01:00
Teon Banek
0a7de969f3 Add converting mgp_value to py::Object 
Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2665
 2020-02-12 09:30:14 +01:00
Matej Ferencevic
7128320b36 Implement MG_ENTERPRISE CMake flag 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2658
 2020-02-06 13:39:30 +01:00
Matej Ferencevic
c96421424f Reorganize unit CMakeLists 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2651
 2020-01-31 11:18:36 +01:00
Matej Ferencevic
de48548164 Move RPC to root source directory 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2641
 2020-01-27 11:50:24 +01:00
Matej Ferencevic
fd81ebdfe3 Move KVStore to root source directory 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2640
 2020-01-24 16:01:51 +01:00
Matej Ferencevic
0c7313bb5f Remove unused datastructures 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2639
 2020-01-24 15:11:02 +01:00
Matej Ferencevic
6d10d90d98 Remove unused utils 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2636
 2020-01-23 17:22:51 +01:00
Teon Banek
cc2160d397 Remove linking mg-query with mg-auth 
Summary: Depends on D2623

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2624
 2020-01-16 17:02:37 +01:00
Matej Ferencevic
4776bea221 Temporarily disable HA build and tests 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2619
 2020-01-13 17:04:27 +01:00
Matej Ferencevic
5906258de0 Stop building storage v1 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2618
 2020-01-13 14:18:12 +01:00
Teon Banek
551c6c5ad6 Add mg-query library and use it in unit tests 
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2611
 2020-01-08 14:33:35 +01:00
Matej Ferencevic
d968370c3e Implement property store 
Summary:
The property store stores a map of `PropertyId` to `PropertyValue` mappings. It
compresses all of the values in order to use as little memory as possible.

Reviewers: teon.banek, ipaljak

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D2604
 2019-12-23 13:28:46 +01:00
Matej Ferencevic
1aadebbb70 Migrate leftover tests away from old storage 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2590
 2019-12-09 10:27:35 +01:00
Matej Ferencevic
8da71873a7 Remove statsd 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2585
 2019-12-06 16:44:30 +01:00
Matej Ferencevic
fc146a71e0 Migrate tests to storage v2 part 3 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2560
 2019-11-25 16:21:04 +01:00
Matej Ferencevic
f61a8c3358 Migrate tests to storage v2 part 2 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2559
 2019-11-22 10:21:30 +01:00
Matej Ferencevic
9cc10e131d Migrate tests to storage v2 part 1 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2554
 2019-11-21 19:00:32 +01:00
Teon Banek
5354a11a00 Add registering multiple procedures in query modules 
Reviewers: mferencevic, dsantl, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2551
 2019-11-21 10:13:09 +01:00
Teon Banek
9c095501d8 query/procedure: Add type system for Cypher 
Summary:
The type system is modelled after "CIP2015-09-16"
https://github.com/opencypher/openCypher/blob/master/cip/1.accepted/CIP2015-09-16-public-type-system-type-annotation.adoc

This is needed for registering procedures and their signatures. The
users will be able to specify what a custom procedure accepts and
returns. All of this needs to be available for inspection during
runtime. Therefore, this diff implements printing types as a user
presentable string. In the future, we will probably want to add type
checking through these types, because openCypher requires type checking
on values passed in and returned from custom procedures.

Reviewers: mferencevic, ipaljak, dsantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2544
 2019-11-13 14:24:21 +01:00
Matej Ferencevic
329818b1b0 Make the Cypher dumper a function 
Summary:
The dumper is now a function and doesn't have to worry about any state. The
function streams the Cypher queries directly to the client. This diff also
makes the dumper work with storage v2.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2545
 2019-11-12 16:01:29 +01:00
Teon Banek
df43ccea95 Add CallProcedure LogicalOperator 
Summary: Depends on D2534

Reviewers: mferencevic, ipaljak, llugovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2535
 2019-11-06 15:46:00 +01:00
Matej Ferencevic
98855889d5 Implement WalFile for storage v2 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2510
 2019-10-25 16:08:17 +02:00
Matej Ferencevic
3adedc0679 Implement snapshots for storage v2 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2400
 2019-10-01 13:42:37 +02:00
Matej Ferencevic
9d39ecab7b Implement encoder/decoder for durability v2 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2393
 2019-09-18 16:32:32 +02:00
Teon Banek
7bd45f8714 Make query execution work with storage_v2 
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2203
 2019-09-12 10:22:00 +02:00
Ivan Paljak
deee3b8ab7 Remove Raft's dependency on transaction id 
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2338
 2019-09-10 18:02:05 +02:00
Marin Tomic
1ab0b8e0ff Implement existence constraints 
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2307
 2019-08-20 16:27:36 +02:00
Marin Tomic
35587cddbd [StorageV2] Implement label and label-property indices 
Reviewers: mferencevic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2225
 2019-07-26 12:23:18 +02:00
Marin Tomic
5a3e98badd Implement Synchronized<T> utility 
Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2222
 2019-07-22 14:55:21 +02:00
Matej Ferencevic
e9a7623288 Implement name to id mapper in storage v2 
Reviewers: teon.banek, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2217
 2019-07-19 11:24:27 +02:00
Matej Ferencevic
111dd8bf19 Remove distributed 
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2213
 2019-07-17 15:23:42 +02:00
Marin Tomic
8414479abe [StorageV2] Implement GC 
Summary:
Here are some numbers from the benchmark:

```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 8
Config: NoGc, Time: 25.9836
Config: OnFinishGc, Time: 49.012
Config: 100msPeriodicGc, Time: 45.9856
Config: 1000msPeriodicGc, Time: 40.3094
```

```
(TOOLCHAIN) mtomic@poso:~/memgraph/build_release$ tests/benchmark/storage_v2_gc --num-threads 7
Config: NoGc, Time: 20.4256
Config: OnFinishGc, Time: 39.6669
Config: 100msPeriodicGc, Time: 30.7956
Config: 1000msPeriodicGc, Time: 35.128
```

It is not that bad if there is a core dedicated to doing garbage collection.

Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2168
 2019-07-10 16:12:08 +02:00
Matej Ferencevic
5b8e7ff432 Implement edges in storage v2 
Summary:
This change implements full edges support in storage v2. Edges can be created
and deleted. Support for detach-deleting vertices is added and regular vertex
deletion verifies existance of edges.

Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2180
 2019-07-08 16:06:57 +02:00
Matej Ferencevic
6e6dff81e0 Split storage and vertex accessor into cpp in storage v2 
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2172
 2019-07-04 12:56:06 +02:00
Matej Ferencevic
f2b23828ab Implement PropertyValue for storage v2 
Reviewers: teon.banek, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2149
 2019-07-01 10:56:33 +02:00
Marin Tomic
6ce8fae54a Implement commit log for storage v2 
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2148
 2019-06-27 13:52:51 +02:00