Commit Graph

2205 Commits

Author SHA1 Message Date
andrejtonev
5bbed6ef9a
Implement user caching to speed up PullPlan (#1109) 2023-08-01 23:04:35 +02:00
Andi
f0bac53e7b
Improve restore replication role (#1089) 2023-08-01 21:51:52 +02:00
andrejtonev
e8850549d2
Add multi-tenancy v1 (#952)
* Decouple BoltSession and communication::bolt::Session
* Add CREATE/USE/DROP DATABASE
* Add SHOW DATABASES
* Cover WebSocket session
* Simple session safety implemented via RWLock
* Storage symlinks for backward. compatibility
* Extend the audit log with the DB info
* Add auth part
* Add tenant recovery
2023-08-01 18:49:11 +02:00
Andi
60f4ffc6a1
Improve logging if replica cannot recover using curr WAL file (#1086) 2023-08-01 10:33:46 +02:00
gvolfing
210bea83d4
Add GraphQL transpilation compatibility (#1018)
* Add callable mappings feature
* Implement mgps.validate (void procedure)
* Make '_' a valid variable name
2023-07-31 14:48:12 +02:00
Josh Soref
57fe3463f2
Fix a bunch of spelling mistakes (1/n) (#1112) 2023-07-30 14:05:05 +02:00
Antonio Filipovic
285b409927
Improve all shortest paths memory usage (#981)
* Change allocator to PoolResource
2023-07-30 12:58:07 +02:00
Gareth Andrew Lloyd
8ebab84324
Add handling of partial results on timeout (#1046) 2023-07-30 10:48:11 +02:00
imilinovic
3fd9ce4a33
Add mgp::map erase and update (#1103) 2023-07-30 08:36:50 +02:00
Andi
be4eb95a98
Fix Jepsen replication pause (#1082) 2023-07-30 02:36:11 +02:00
Andi
18bd02423a
Fix PropertyStore buffer serialization (#1111) 2023-07-29 19:14:27 +02:00
andrejtonev
58c0c4cebb
Add missing-field-initialization warning flag (#1113) 2023-07-29 17:59:11 +02:00
andrejtonev
110ca3968c
Fix path generation ignores edge's element_id (#1108) 2023-07-29 15:51:51 +02:00
Vlasta
fae039c215
Improve logging for the on-disk storage (#1079) 2023-07-19 22:54:42 +02:00
Marko Budiselić
3b9133fd5a
Improve e2e and replication testing setup (#1061)
* Add `--replication-restore-state-on-startup` with `false` as default

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
2023-07-19 21:18:43 +02:00
Marko Budiselić
9d056e7649
Add experimental/v1 of ON_DISK_TRANSACTIONAL storage (#850)
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-06-29 11:44:55 +02:00
Antonio Filipovic
aa4f68a37d
Add error handling on py batched module init (#1052) 2023-06-28 17:23:42 +02:00
Josipmrden
84721f7e0a
Add vertex degree to index statistics (#1026)
Add graph analysis of vertex degrees when doing ANALYZE GRAPH.
2023-06-27 18:06:20 +02:00
Ante Javor
261aa4f49b
Improve replication logging (#1030) 2023-06-27 17:57:51 +02:00
Gareth Andrew Lloyd
3b781bf525
Add HTTP+GZIP support to LOAD CSV (#1027) 2023-06-26 19:10:48 +02:00
Antonio Filipovic
d573eda8bb
Add python & cpp batching option in procedures
* Add API for batching from the procedure 
* Use PoolResource for batched procedures
2023-06-26 15:46:13 +02:00
Gareth Andrew Lloyd
5b1ba10183
Fix IN_MEMORY_ANALYTICAL storage GC (#1025) 2023-06-23 12:50:03 +02:00
Vlasta
b25e9968ee
Update links inside CSV import tool (#834) 2023-06-22 16:00:22 +02:00
Katarina Supe
68e5610566
Fix replica exception message (#930) 2023-06-22 14:41:59 +02:00
Ante Javor
0ea96663ba
Add check for opening snapshots (#966) 2023-06-22 13:29:49 +02:00
Marko Budiselić
e73eac77a9
Improve libstdc++ dependency on RPM systems (#863) 2023-06-22 10:14:59 +02:00
Josipmrden
b875649270
Add restoring of replication roles upon database startup (#791)
Fix replica node restoration on startup so it is restored as replica and not as main.
2023-06-21 19:08:58 +02:00
Josipmrden
05cc35bf93
Add command NULLIF for identifying nulls in LOAD CSV (#914)
Add NULLIF command which turns all row values corresponding to the string to the nullif character sequence.
2023-06-21 14:50:46 +02:00
Josipmrden
63f8298033
Fix MATCH + LOAD CSV to load CSV only once (#916)
* update profile query to use poolresource
* Optimize update of indexes
* Add ignore empty strings to load csv
* Add operator changes to support handling of nulls
* Store chunks in memory pools ordered
* Use same max block per chunks number
* Remove redundant return statement
* add hacky cached solution
* change map to set
* remove memory
* Add match load csv invalid behaviour commit
* Accept input on LOAD CSV
* Ommit changes not tied to the PR
* Add tests for match + load csv
* Add gqlalchemy installation for e2e tests
* Modify setup script to update packages
* Revert gqlalchemy to 1.3.3
* Revert gqlalchemy to 1.3.3
* Address PR review comments
* Ommit semicolon
---------

Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
2023-06-21 11:13:40 +02:00
Josipmrden
df95775222
Fix init file startup in community edition (#974)
* Fix init file startup in community edition

* Add possibility to build binary without MG_ENTERPRISE

* Added trace spdlog for when init file is not present

* Add gqlalchemy and unit tests

* Add init data files which correspond to the right directory by the github actions
2023-06-20 17:54:50 +02:00
andrejtonev
30ec570bb9
Add Bolt v5 support (#938) 2023-06-12 18:55:15 +02:00
Antonio Filipovic
d917c3f0fd
Fix slow IN LIST evaluation (#901) 2023-05-29 17:52:20 +02:00
andrejtonev
d842adbed3
Handle user-defined metadata and expose it with SHOW TRANSACTIONS(#945) 2023-05-29 11:40:14 +02:00
Josipmrden
651b6f3a5a Expose system metrics over HTTP Endpoint (#940) 2023-05-18 05:10:57 +00:00
Ante Pušić
0d9bd74a8a
Add support for map projection (#892) 2023-05-16 20:05:35 +02:00
andrejtonev
802f8aceda
Add data directory status and (un)lock query (#933) 2023-05-16 18:36:04 +02:00
gvolfing
7ddce539fa
Add return build type command (#894) 2023-05-16 16:02:03 +02:00
gvolfing
c3e4f81026
Include additional info inside storage mode info query (#883) 2023-05-16 14:25:41 +02:00
Antonio Filipovic
208705f296
Reduce memory consumption on return from python procedures (#932) 2023-05-16 10:33:09 +02:00
János Benjamin Antal
3a5f140c2b
Order chunks in utils::Pool to speed up deallocation (#898) 2023-05-02 13:08:20 +02:00
Antonio Filipovic
91017b7f36
Update profile query to use PoolResource for LOAD CSV (#885) 2023-04-26 18:04:13 +02:00
gvolfing
00f8d54249
Parallelize index creation (#882) 2023-04-26 16:28:02 +02:00
János Benjamin Antal
4fcdd52f88
Use correct memory resource (#900) 2023-04-26 10:02:55 +02:00
János Benjamin Antal
6c947947eb Parallelize recovery (#868)
* Parallelize edge recovery

* Load vertex labels and properties parallel

* Add parallel connectivity loading

* Add batches information to snapshot

* Introduce `items_per_batch` and `recovery_thread_count` flags

* Make possible to load snapshots with old version

* Add vertex batches to `RecoveryInfo`

* Extend durability integration tests with v15 test cases

* Add `std::vector` based `InitProperties`

* Use `InitProperties` in snapshot loading
2023-04-25 16:25:25 +02:00
János Benjamin Antal
97e250129e
Change AccumulateCursor to use utils::pmr::deque (#888)
* Increase performance by eliminating unnecessary `TypedValue` copies
2023-04-24 16:22:22 +02:00
Antonio Filipovic
2c6a55775d
Fix max block size bug on LOAD CSV(#877) 2023-04-19 16:10:20 +02:00
Andi
26a0866938
Fix index optimization bug (#860) 2023-04-04 23:43:13 +02:00
Antonio Filipovic
64e837b355
Introduce analytics mode (#772) 2023-04-04 18:46:26 +02:00
Antonio Filipovic
a586f2f98d
Change EvalContext and QueryExecution to use PoolResource on LOAD CSV (#825)
* Change PullPlan to use specific PoolResource for LOAD CSV
2023-04-04 16:54:08 +02:00
Josipmrden
9fc51f74a0
Skip label based auth on user with global visibility on graph (#837) 2023-04-04 11:13:25 +02:00
Josipmrden
128771a6ec
Add SHA-256 password encryption (#839) 2023-04-03 16:29:21 +02:00
Josipmrden
f5a49ed29f Add Cypher subqueries (#794) (#851)
Co-authored-by: Bruno Sačarić <bruno.sacaric@gmail.com>
2023-03-31 13:49:10 +00:00
Josipmrden
398503da7a
Add index statistics for better query planning (#812) 2023-03-30 15:34:34 +02:00
Bruno Sačarić
0819b40202
Fix bug on AllShortest with multiple edges between nodes (#832) 2023-03-29 16:39:41 +02:00
Andi
029be10f1d
Add queries to show or terminate active transactions (#790) 2023-03-27 15:46:00 +02:00
Jure Bajic
c4167bafdd
Add support for Amazon Linux 2 and stop generating C++ using Lisp/LCP (#814) 2023-03-14 19:24:55 +01:00
Ante Pušić
97d45ab1d8
Add Python query module API mock (#757) 2023-03-07 15:41:19 +01:00
Josipmrden
6abd356d01
[master < E214] WHERE Exists feature (#818)
Add WHERE exists() to filter based on neighbouring pattern expressions
2023-03-07 00:28:41 +01:00
Vlasta
99a6c72bba
Change message on incompatible epoch_id error (#786) 2023-03-06 20:01:02 +01:00
Ante Pušić
173f5430aa
Remove noexcept from functions that may throw (#819) 2023-03-06 17:34:34 +01:00
Antonio Filipovic
d79dd69607
Improve performance with props init on node|edge creation (#788) 2023-02-24 15:40:35 +01:00
Antonio Filipovic
862a1afdf1
Improve Visit performance (#774) 2023-02-17 13:09:25 +01:00
Josipmrden
8f70c5f2a5
Fix label-based auth using OLD view instead of NEW when merging nodes (#755) 2023-02-01 13:20:26 +01:00
Andi
04efc7a4a6
Remove torch and igraph from sys cache (#720) 2023-01-31 13:11:59 +01:00
Bruno Sačarić
34dd47ef07
Fix nested FOREACH shadowing bug (#725) 2023-01-25 20:06:05 +01:00
Bruno Sačarić
034b54cb72
Fix bug on all shortest paths with an upper bound (#737) 2023-01-25 15:32:00 +01:00
Josipmrden
8cf51d9f68
Fix bug in query plan to use indexes on optional match and foreach (#736)
* Add fix in query plan to use indexes on optional match and foreach
2023-01-25 12:53:33 +01:00
niko4299
d9eeedb9ee
Adding qid in bolt (#721) 2023-01-18 16:33:03 +01:00
Andi
156e2cd095
On delete triggers invalid edge reference (#717)
* Added check if there is invalid reference to the underlying edge

* Added fix and e2e tests

* Isolation levels tracking based on from_vertex_

* Added explicit transaction test + edge accessor changes based on the vertex_edge

* Autocommit on tests, initialize deleted by checking out_edges

Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2023-01-18 15:05:10 +01:00
Bruno Sačarić
1f2a15e7c8
Fix MATCH not allowed on replica (#709) 2022-12-23 14:47:12 +01:00
Antonio Filipovic
e5e37bc14a
Fix LOAD CSV large memory usage (#712) 2022-12-22 19:38:48 +01:00
Andi
310e305cfb
Fix python module reloading (#706) 2022-12-12 21:11:13 +01:00
Marko Budiselić
9d6a23b6bd
Add init-file and init-data-file capabilities (#696) 2022-12-09 18:50:33 +01:00
Andi
f2d5ab61c4
Fix Python submodules reloading (#653) 2022-12-09 14:30:41 +01:00
Andi
0f77c85824
Fix cursor exhaustion by adding EmptyResult operator (#667) 2022-12-09 11:44:07 +01:00
niko4299
d6d4153fb7
Fix graph projection bug (#697) 2022-12-08 13:45:20 +01:00
Tyler Neely
7d6a5e5b9c
Add support for -h to show help in addition to --help (#682) 2022-12-07 16:51:32 +01:00
niko4299
3e11f38548
Add aggregation distinct (#654) (#665) 2022-12-03 13:48:44 +02:00
Ante Javor
11300960de
Add mixed workload and Neo4j client to mgbench (#566)
* Fix bolt bug inside the C++ client
* Add tail latency stats
* Add hot run option
* Add query caching
* Add jcmd memory tracking
2022-11-28 08:47:22 +01:00
Bruno Sačarić
58e6097664
Fix ALLSHORTEST combined with id function (#636) 2022-11-04 19:36:03 +01:00
Jure Bajic
ff21c0705c
Add multiple license support (#618)
Make license info available through LicenseChecker
Add LicenseInfoSender
Move license library from utils
Rename telemetry_lib to mg-telemetry
2022-11-04 15:23:43 +01:00
Antonio Filipovic
3daab6ce97
Fix bug in the C API in-edges iterator (#613) 2022-10-25 19:18:44 +02:00
Antonio Filipovic
417cf4b30b
Fix bug related to EdgeType and Label getters in query modules (#582)
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2022-10-06 21:21:11 +02:00
Bruno Sačarić
5261d82063
Fix passing user's fine_grained_access_handler instead of role's (#579)
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
2022-09-30 18:27:47 +02:00
Marko Budiselić
b42e47b0be
Reduce the size of TypedValue (#560)
- Reduce the size of TypedValue
- Fix double allocation
- Add `Graph` to `TypedValue` unit tests
- Fix allocator usage in `TypedValue`
- Add graph projection to `long_running.cpp` stress test
2022-09-20 14:21:34 +02:00
Boris Taševski
a0b8871b36
Fix cland tidy errors and other warning (#555) 2022-09-15 15:51:35 +02:00
Marko Budiselic
bb6cf35441 Merge master cpp module API 2022-09-15 11:29:52 +02:00
Ante Pušić
5bc301d21d
Add C++ query modules API (#546)
Co-authored-by: Ante Pusic <ante.pusic@memgraph.com>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.com>
2022-09-15 11:26:26 +02:00
Boris Taševski
43e0520bc8
Merge master (#554) 2022-09-15 07:25:36 +02:00
Bruno Sačarić
2c8e45e889
Add run_id to the query summary (#548) 2022-09-14 20:21:06 +02:00
Boris Taševski
0876a8848d
Merge master to epic and fix differences (#552) 2022-09-14 18:36:21 +02:00
Boris Taševski
fb4641a6be
Fix logic in fine grained permissions (#551) 2022-09-14 12:39:23 +02:00
niko4299
201f75e809
Add MG_ENTERPRISE and license checks (#547) 2022-09-14 01:10:28 +02:00
niko4299
dc8dad9794
Add authorization in SetLabels, RemoveLabels, Allshortestpath cursor (#537) 2022-09-13 17:14:23 +02:00
Boris Taševski
aa02745915
[E129-MG < T1030-MG] Tech debts (#540)
* renamed parameters (#539)

* added variable declarations in ifs; minor code improvements; (#541)

* dba parameter removed (#543)

* Accept -> Has rename; HasGlobalPermissionOnVertices/Edges -> HasGlobalPrivilegeOnVertices/Edges (#545)

* replaced passing dba from reference to pointer
2022-09-13 11:37:17 +02:00
Josipmrden
b2d5a8eeca
[E129-MG < T1040-MG] Add exceptions in LBA cursors (#536)
Exceptions added in update and create delete operators instead of logging
2022-09-12 14:04:40 +02:00
Boris Taševski
c09b175c76
[E129-MG < T1006-MG] Expand C API with LBA checks (#527)
* [T1006-MG < T1017-MG] Add LBA checks to all read procedures in C API (#515)

* Initial Impl

* NextPermittedEdge introduced

* revert moving constructor to cpp

* edge from and edge to methods expanded with lba check

* minor fix

* added check to path expand procedure

* Added integration tests for read query procedures

* additional check

* changed iterator type to reference

* comments from pr

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>

* [T1006-MG < T1018-MG] Add LBA checks to all update procedures in C API (#516)

* Initial Impl

* NextPermittedEdge introduced

* revert moving constructor to cpp

* edge from and edge to methods expanded with lba check

* minor fix

* extended update methods

* added check to path expand procedure

* Added integration tests for read query procedures

* Added integration tests for update query modules

* additional check

* changed iterator type to reference

* fixed bug in Update property for node; fixed 2 e2e tests

* replaced enum

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>

* [T1006-MG < T1019-MG] Add LBA checks to all Create and Delete procedures in C API (#517)

* Initial Impl

* NextPermittedEdge introduced

* revert moving constructor to cpp

* edge from and edge to methods expanded with lba check

* minor fix

* extended update methods

* initial implementation

* added check to path expand procedure

* Added integration tests for read query procedures

* Added integration tests for update query modules

* Added unit tests for creation of vertex, adding and removing vertex label

* additional check

* changed iterator type to reference

* Added unit tests for create edge

* Corrected query module in create edge

* fixed bug in Update property for node; fixed 2 e2e tests

* fixed merge errors

* Expanded FineGrainedAuthChecker with HasGlobalPermissionOnVertices and HasGlobalPermissionOnEdges

* Removed two wrong checks; Added two global checks

* return null added

* introduced new mgp_error value

* fixed endless loop

* replaced enum

* intermediate

* tests updated

* PermissionDeniedError -> AuthorizationError rename

* rename in enum permission_denied error -> authorization error

* mgp_vertex_remove_label check improved

* quotes changed; order of imports fixed

* string constant introduced

* import fixed

* yaml format

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>

Co-authored-by: Josip Mrden <josip.mrden@memgraph.io>
2022-09-08 17:48:34 +02:00
Kostas Kyrimis
f1fe77adfb
Graph project feature implementation (#508) (#535) 2022-09-07 16:00:49 +03:00
Josip Mrden
35f8978560 Merge branch 'master' into E129-MG-label-based-authorization 2022-09-07 09:28:32 +02:00
Josip Matak
9e8fb2516b
Add all shortest path algorithm (#409) 2022-09-06 16:21:32 +02:00