Commit Graph

1298 Commits

Author SHA1 Message Date
Ante Pušić
989bb97514
Extend Cypher queries with the index hinting feature (#1345) 2023-10-27 14:26:19 +02:00
Ante Pušić
3158a16ffd
Add filtering details to EXPLAIN and PROFILE (#1265) 2023-10-25 21:36:20 +02:00
Josipmrden
e617ff9b59
Provide textual information for inefficient plans with notifications (#1343) 2023-10-24 22:20:05 +02:00
Josipmrden
be16ca7362
Add cartesian and hash join operators (#1193) 2023-10-24 21:54:42 +02:00
imilinovic
1f118e7521
Add renaming of edge types (#1364) 2023-10-24 17:12:09 +02:00
Matija Pintarić
97ed912ab6
Implement map key exists in mgp (#1336) 2023-10-23 15:29:41 +02:00
andrejtonev
26e31ca06f
Fix SHOW CONFIG to show the run-time flag status (#1278) 2023-10-23 10:18:07 +02:00
Marko Budiselić
9524a51576
Add v5 toolchain (#608)
* clang 17.0.2
* gcc 13.2
* upgrade libs
* tmp disable gpg check, tmp disable fblibs
2023-10-16 19:01:39 +02:00
andrejtonev
22d8ef75e0
Updated telemetry client-side (#1337) 2023-10-16 14:16:00 +02:00
Andi
de9280b334
Refactor disk storage (#1347) 2023-10-16 09:11:07 +02:00
Andi
1a3c5af797
Improve expansions on disk (#1335)
* Improve disk expansions
2023-10-11 10:18:50 +02:00
Gareth Andrew Lloyd
d278a33f31
Decouple pure replication state from storage [part 1] (#1325)
A major refactor to decouple replication state from storage.
ATM it is still owned by storage but a following part should fix that.
2023-10-10 11:44:19 +01:00
Gareth Andrew Lloyd
3cc2bc2791
Refactor interpreter to support multiple distributed clocks (Part 1) (#1281)
* Interpreter transaction ID decoupled from storage transaction ID
* Transactional scope for indices, statistics and constraints
* Storage::Accessor now has 2 modes (unique and shared)
* Introduced ResourceLock to fix pthread mutex problems
* Split InfoQuery in two: non-transactional SystemInfoQuery and transactional DatabaseInfoQuery
* Replicable and durable statistics
* Bumped WAL/Snapshot versions
* Initial implementation of the Lamport clock

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
2023-10-05 16:58:39 +02:00
Gareth Andrew Lloyd
d71b6a5007
Refactor replication client/server (#1311) 2023-09-29 11:21:42 +01:00
andrejtonev
bce48361ca
Decoupling Interpreter from Storage (#1186)
Unique/global InterpreterContext that is Storage agnostic (has a reference to the DbmsHandler instead)

* InterpreterContext is no longer the owner of Storage
* New Database structure that handles Storage, Triggers, Streams
* Renamed SessinContextHandler to DbmsHandler and simplified the multi-tenant logic
* Added Gatekeeper and updated handlers to use it

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2023-09-20 13:13:54 +02:00
imilinovic
404cdf05d3
Add path pop to mgp API (#1249) 2023-09-19 12:37:55 +02:00
andrejtonev
5e5f4ffc5d
Add more runtime configurable settings (#1183)
server name, query timeout settings, log.level, log.to_stderr
2023-09-11 11:30:54 -04:00
Ante Pušić
060b9d1c16
[master < T1204] Add detailed operator info to PROFILE and EXPLAIN (#1204) 2023-09-11 14:34:27 +02:00
Ante Pušić
29a505cb38
Property lookup caching (#1168) 2023-09-11 13:03:54 +02:00
Josipmrden
58546a9fe1
Add detach delete in bulk (#1078) 2023-09-10 18:53:03 +02:00
Josipmrden
07dea328d8
[master < T1110] Add merge optimization to expand dynamically during runtime (#1110) 2023-09-08 17:12:25 +02:00
Gareth Andrew Lloyd
bd1852f407
Reduce flake SnapshotFallback test (#1237)
Fixed the wait period, this should ensure at least one snapshot was 
made. Also cleaned up the checking around this. And also better 
corruption.
2023-09-08 14:21:35 +01:00
imilinovic
9c51dbbb01
Implement changing from and to vertices in relationships (#1221) 2023-09-08 12:52:40 +02:00
Matija Pintarić
d9464c6ffd
Add InDegree and OutDegree in O(1) (#1217) 2023-09-07 13:16:30 +02:00
Antonio Filipovic
b6b32bec03
Improve performance of delta creation (#1129) 2023-09-06 11:30:21 +02:00
Andi
b5413c6f82
Add edge import mode into the on-disk storage (#1157) 2023-09-05 19:00:53 +02:00
Josipmrden
09fd5939da
Remove double scan with expand from the planner (#1085) 2023-09-05 11:02:52 +02:00
Josipmrden
02eab6ab9c
Set properties C API extension (#1131)
Add SetProperties into the C++ query module API
2023-09-04 16:17:43 +02:00
Gareth Andrew Lloyd
e928eed028
Replication refactor (part 4) (#1211)
More refactoring to isolate generic replication behavior. Making the 
InMemory* types even more decoupled from replication logic.
2023-08-31 16:06:44 +01:00
Matija Pintarić
d516e40841
Add ToString on C++ API mgp types(#1140) 2023-08-29 17:30:23 +02:00
Andi
a6ec81b179
Add deterministic disk vertex_count and edge_count (#1146)
* Add exact vertex_count and edge_count to disk storage
2023-08-29 13:07:23 +02:00
andrejtonev
c526ff2a8f
[master < ] Remove DbAccessor from non-transactional queries (#1201)
* Decouple non-transactional queries from DbAccessor
* Invalidate auth cache after AuthQuery

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2023-08-29 11:13:42 +02:00
Gareth Andrew Lloyd
4bc5d749b2
Refactor replication, part 3 (#1177)
Changes to make replication code agnostic of the storage kind being used.

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
2023-08-25 10:52:07 +01:00
imilinovic
a426ef9cc3
Add Relationship::RemoveProperty to C++ query module API (#1156) 2023-08-24 12:14:00 +02:00
andrejtonev
9355e58e73
Decoupling replication logic from InMemoryStorage (#1169) 2023-08-22 13:29:25 +02:00
gvolfing
476968e2c8
Fix concurrent query module race condition (#1158)
Concurrent access to the same query module had a race condition on the
pointer that was used to handle the custom memory management. With this
commit, a mapping has been added to keep information about what
thread used the pointer to handle the memory resources. This should be
fine since the respected query executions are running on a dedicated
thread. Access to the mapping itself is threadsafe. A simple RAII
wrapper for the mapping container has also been added for simpler
client-side use.
2023-08-21 16:45:36 +02:00
Aidar Samerkhanov
3bf2cf65ab
Optimize splitting keys inside the on-disk storage (#1155) 2023-08-17 18:09:21 +02:00
Gareth Andrew Lloyd
2e51e703c3
Add supernode vertex cache (#1124)
Add supernode vertex cache to account for long delta chains and modifications in the same module being independent of scanning of the nodes in the next iteration of the pulling mechanism.
2023-08-11 10:18:28 +02:00
Antonio Filipovic
509183e985
Improve performance on set properties (#1115) 2023-08-10 09:06:44 +02:00
Andi
762fe6a65d
Improve disk indices (#1139) 2023-08-09 10:16:49 +02:00
Matija Pintarić
514fed51c4
Add implementation of C++ API Node::RemoveProperty (#1128) 2023-08-01 20:11:38 +02:00
imilinovic
2877c343e8
Add implementation of << operator for mgp::Value (#1127) 2023-08-01 19:30:23 +02:00
andrejtonev
e8850549d2
Add multi-tenancy v1 (#952)
* Decouple BoltSession and communication::bolt::Session
* Add CREATE/USE/DROP DATABASE
* Add SHOW DATABASES
* Cover WebSocket session
* Simple session safety implemented via RWLock
* Storage symlinks for backward. compatibility
* Extend the audit log with the DB info
* Add auth part
* Add tenant recovery
2023-08-01 18:49:11 +02:00
gvolfing
210bea83d4
Add GraphQL transpilation compatibility (#1018)
* Add callable mappings feature
* Implement mgps.validate (void procedure)
* Make '_' a valid variable name
2023-07-31 14:48:12 +02:00
Josh Soref
57fe3463f2
Fix a bunch of spelling mistakes (1/n) (#1112) 2023-07-30 14:05:05 +02:00
Antonio Filipovic
285b409927
Improve all shortest paths memory usage (#981)
* Change allocator to PoolResource
2023-07-30 12:58:07 +02:00
Gareth Andrew Lloyd
8ebab84324
Add handling of partial results on timeout (#1046) 2023-07-30 10:48:11 +02:00
imilinovic
3fd9ce4a33
Add mgp::map erase and update (#1103) 2023-07-30 08:36:50 +02:00
Andi
18bd02423a
Fix PropertyStore buffer serialization (#1111) 2023-07-29 19:14:27 +02:00
andrejtonev
58c0c4cebb
Add missing-field-initialization warning flag (#1113) 2023-07-29 17:59:11 +02:00
Matija Pintarić
76ca019f31
Add overload of operator< for mgp::Value (#1090) 2023-07-28 11:35:14 +02:00
imilinovic
609b9a20f1
Add hash on mgp::Value (#1093) 2023-07-28 09:08:36 +02:00
Matija Pintarić
ab4d1efe0b
Overload << operator for mgp::Value::Type (#1080) 2023-07-25 13:30:02 +02:00
Marko Budiselić
992e718a97
Fix v2.9 release build issues (#1087)
* Fix include order inside `tests/unit/query_variable_start_planner.cpp` fails on an Rhel7 based OS
2023-07-20 16:15:27 +02:00
Andi
05eca46267
Fix flaky transaction_queue_multiple unit test (#1077) 2023-07-19 22:58:02 +02:00
Marko Budiselić
9d056e7649
Add experimental/v1 of ON_DISK_TRANSACTIONAL storage (#850)
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-06-29 11:44:55 +02:00
Josipmrden
84721f7e0a
Add vertex degree to index statistics (#1026)
Add graph analysis of vertex degrees when doing ANALYZE GRAPH.
2023-06-27 18:06:20 +02:00
Gareth Andrew Lloyd
3b781bf525
Add HTTP+GZIP support to LOAD CSV (#1027) 2023-06-26 19:10:48 +02:00
Josipmrden
b875649270
Add restoring of replication roles upon database startup (#791)
Fix replica node restoration on startup so it is restored as replica and not as main.
2023-06-21 19:08:58 +02:00
Josipmrden
63f8298033
Fix MATCH + LOAD CSV to load CSV only once (#916)
* update profile query to use poolresource
* Optimize update of indexes
* Add ignore empty strings to load csv
* Add operator changes to support handling of nulls
* Store chunks in memory pools ordered
* Use same max block per chunks number
* Remove redundant return statement
* add hacky cached solution
* change map to set
* remove memory
* Add match load csv invalid behaviour commit
* Accept input on LOAD CSV
* Ommit changes not tied to the PR
* Add tests for match + load csv
* Add gqlalchemy installation for e2e tests
* Modify setup script to update packages
* Revert gqlalchemy to 1.3.3
* Revert gqlalchemy to 1.3.3
* Address PR review comments
* Ommit semicolon
---------

Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
2023-06-21 11:13:40 +02:00
andrejtonev
30ec570bb9
Add Bolt v5 support (#938) 2023-06-12 18:55:15 +02:00
andrejtonev
d842adbed3
Handle user-defined metadata and expose it with SHOW TRANSACTIONS(#945) 2023-05-29 11:40:14 +02:00
Josipmrden
651b6f3a5a Expose system metrics over HTTP Endpoint (#940) 2023-05-18 05:10:57 +00:00
Ante Pušić
0d9bd74a8a
Add support for map projection (#892) 2023-05-16 20:05:35 +02:00
andrejtonev
802f8aceda
Add data directory status and (un)lock query (#933) 2023-05-16 18:36:04 +02:00
gvolfing
c3e4f81026
Include additional info inside storage mode info query (#883) 2023-05-16 14:25:41 +02:00
János Benjamin Antal
6c947947eb Parallelize recovery (#868)
* Parallelize edge recovery

* Load vertex labels and properties parallel

* Add parallel connectivity loading

* Add batches information to snapshot

* Introduce `items_per_batch` and `recovery_thread_count` flags

* Make possible to load snapshots with old version

* Add vertex batches to `RecoveryInfo`

* Extend durability integration tests with v15 test cases

* Add `std::vector` based `InitProperties`

* Use `InitProperties` in snapshot loading
2023-04-25 16:25:25 +02:00
Antonio Filipovic
64e837b355
Introduce analytics mode (#772) 2023-04-04 18:46:26 +02:00
Josipmrden
128771a6ec
Add SHA-256 password encryption (#839) 2023-04-03 16:29:21 +02:00
Josipmrden
f5a49ed29f Add Cypher subqueries (#794) (#851)
Co-authored-by: Bruno Sačarić <bruno.sacaric@gmail.com>
2023-03-31 13:49:10 +00:00
Josipmrden
398503da7a
Add index statistics for better query planning (#812) 2023-03-30 15:34:34 +02:00
Andi
029be10f1d
Add queries to show or terminate active transactions (#790) 2023-03-27 15:46:00 +02:00
Jure Bajic
c4167bafdd
Add support for Amazon Linux 2 and stop generating C++ using Lisp/LCP (#814) 2023-03-14 19:24:55 +01:00
Josipmrden
6abd356d01
[master < E214] WHERE Exists feature (#818)
Add WHERE exists() to filter based on neighbouring pattern expressions
2023-03-07 00:28:41 +01:00
Ante Pušić
173f5430aa
Remove noexcept from functions that may throw (#819) 2023-03-06 17:34:34 +01:00
Antonio Filipovic
d79dd69607
Improve performance with props init on node|edge creation (#788) 2023-02-24 15:40:35 +01:00
Antonio Filipovic
862a1afdf1
Improve Visit performance (#774) 2023-02-17 13:09:25 +01:00
Bruno Sačarić
34dd47ef07
Fix nested FOREACH shadowing bug (#725) 2023-01-25 20:06:05 +01:00
Ante Pušić
aad4bcb7a0
Fix C++ API memory leak on Relationships() (#743) 2023-01-25 17:23:46 +01:00
Josipmrden
8cf51d9f68
Fix bug in query plan to use indexes on optional match and foreach (#736)
* Add fix in query plan to use indexes on optional match and foreach
2023-01-25 12:53:33 +01:00
Antonio Filipovic
1cd1da84fd
Fix bug on (vertex|edge) properties in C++ API (#732) 2023-01-23 12:57:17 +01:00
Bruno Sačarić
1f2a15e7c8
Fix MATCH not allowed on replica (#709) 2022-12-23 14:47:12 +01:00
Antonio Filipovic
e5e37bc14a
Fix LOAD CSV large memory usage (#712) 2022-12-22 19:38:48 +01:00
Andi
0f77c85824
Fix cursor exhaustion by adding EmptyResult operator (#667) 2022-12-09 11:44:07 +01:00
Ante Pušić
45451bae3b
Fix C++ query modules API bugs (#688) 2022-12-06 16:57:50 +01:00
niko4299
3e11f38548
Add aggregation distinct (#654) (#665) 2022-12-03 13:48:44 +02:00
Bruno Sačarić
58e6097664
Fix ALLSHORTEST combined with id function (#636) 2022-11-04 19:36:03 +01:00
Jure Bajic
ff21c0705c
Add multiple license support (#618)
Make license info available through LicenseChecker
Add LicenseInfoSender
Move license library from utils
Rename telemetry_lib to mg-telemetry
2022-11-04 15:23:43 +01:00
Marko Budiselić
b42e47b0be
Reduce the size of TypedValue (#560)
- Reduce the size of TypedValue
- Fix double allocation
- Add `Graph` to `TypedValue` unit tests
- Fix allocator usage in `TypedValue`
- Add graph projection to `long_running.cpp` stress test
2022-09-20 14:21:34 +02:00
Boris Taševski
a0b8871b36
Fix cland tidy errors and other warning (#555) 2022-09-15 15:51:35 +02:00
Marko Budiselic
bb6cf35441 Merge master cpp module API 2022-09-15 11:29:52 +02:00
Ante Pušić
5bc301d21d
Add C++ query modules API (#546)
Co-authored-by: Ante Pusic <ante.pusic@memgraph.com>
Co-authored-by: Josip Mrden <josip.mrden@memgraph.com>
2022-09-15 11:26:26 +02:00
Boris Taševski
0876a8848d
Merge master to epic and fix differences (#552) 2022-09-14 18:36:21 +02:00
Boris Taševski
fb4641a6be
Fix logic in fine grained permissions (#551) 2022-09-14 12:39:23 +02:00
niko4299
201f75e809
Add MG_ENTERPRISE and license checks (#547) 2022-09-14 01:10:28 +02:00
niko4299
dc8dad9794
Add authorization in SetLabels, RemoveLabels, Allshortestpath cursor (#537) 2022-09-13 17:14:23 +02:00
Boris Taševski
aa02745915
[E129-MG < T1030-MG] Tech debts (#540)
* renamed parameters (#539)

* added variable declarations in ifs; minor code improvements; (#541)

* dba parameter removed (#543)

* Accept -> Has rename; HasGlobalPermissionOnVertices/Edges -> HasGlobalPrivilegeOnVertices/Edges (#545)

* replaced passing dba from reference to pointer
2022-09-13 11:37:17 +02:00
Josipmrden
b2d5a8eeca
[E129-MG < T1040-MG] Add exceptions in LBA cursors (#536)
Exceptions added in update and create delete operators instead of logging
2022-09-12 14:04:40 +02:00
Kostas Kyrimis
f1fe77adfb
Graph project feature implementation (#508) (#535) 2022-09-07 16:00:49 +03:00
Josip Mrden
35f8978560 Merge branch 'master' into E129-MG-label-based-authorization 2022-09-07 09:28:32 +02:00