Commit Graph

2249 Commits

Author SHA1 Message Date
imilinovic
404cdf05d3
Add path pop to mgp API (#1249) 2023-09-19 12:37:55 +02:00
gvolfing
fd63944493
Add --query-callable-mappings-path package default (#1203) 2023-09-11 15:12:14 -04:00
Gareth Andrew Lloyd
6694de2dfa
Fix libkrb5 TRUE and FALSE macros leakage (#1243)
Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2023-09-11 12:46:40 -04:00
andrejtonev
5e5f4ffc5d
Add more runtime configurable settings (#1183)
server name, query timeout settings, log.level, log.to_stderr
2023-09-11 11:30:54 -04:00
Ante Pušić
060b9d1c16
[master < T1204] Add detailed operator info to PROFILE and EXPLAIN (#1204) 2023-09-11 14:34:27 +02:00
Ante Pušić
29a505cb38
Property lookup caching (#1168) 2023-09-11 13:03:54 +02:00
Ante Pušić
d4fcd745d2
Sort SHOW INDEX INFO (#1178) 2023-09-11 10:59:41 +02:00
Josipmrden
58546a9fe1
Add detach delete in bulk (#1078) 2023-09-10 18:53:03 +02:00
Gareth Andrew Lloyd
ab56abf4ca
Optimize scanning vertices (#1227) 2023-09-09 10:09:25 -04:00
Gareth Andrew Lloyd
1bd47318cd
Improve PropertyStore (#1142)
Improve AnyVersionHasLabelProperty by doing less work in some instances.
Improve FindSpecificProperty.
2023-09-09 08:00:43 -04:00
Ante Pušić
0403b67073
Fix returning NULL on map projection from a null value (#1119) 2023-09-09 06:43:25 -04:00
DavIvek
9e4babcdbb
Fix segfault based on issue #874 (#1175) 2023-09-09 02:04:46 -04:00
Antonio Filipovic
b094fdbadc
Fix API bug on accessing deleted object (#1209) 2023-09-08 13:52:21 -04:00
Josipmrden
07dea328d8
[master < T1110] Add merge optimization to expand dynamically during runtime (#1110) 2023-09-08 17:12:25 +02:00
imilinovic
9c51dbbb01
Implement changing from and to vertices in relationships (#1221) 2023-09-08 12:52:40 +02:00
Matija Pintarić
d9464c6ffd
Add InDegree and OutDegree in O(1) (#1217) 2023-09-07 13:16:30 +02:00
Ante Javor
312d01bd0c
Remove repeated log lines from TRACE log level. (#1054) 2023-09-06 23:09:51 +02:00
Antonio Filipovic
b6b32bec03
Improve performance of delta creation (#1129) 2023-09-06 11:30:21 +02:00
Antonio Filipovic
93992a275b
Improve NameToId mapper on set properties (#1147) 2023-09-06 00:12:27 +02:00
Andi
b5413c6f82
Add edge import mode into the on-disk storage (#1157) 2023-09-05 19:00:53 +02:00
Josipmrden
09fd5939da
Remove double scan with expand from the planner (#1085) 2023-09-05 11:02:52 +02:00
Josipmrden
02eab6ab9c
Set properties C API extension (#1131)
Add SetProperties into the C++ query module API
2023-09-04 16:17:43 +02:00
Gareth Andrew Lloyd
9661c52179
Introduce a reader writer spin lock (#1187)
It is possible for multiple read only queries to be accessing the same
sequence of vertices/edges. The reader mode of the spin lock will ensure
multiple threads can make progress at the same time.
2023-09-01 14:21:15 +01:00
Gareth Andrew Lloyd
e928eed028
Replication refactor (part 4) (#1211)
More refactoring to isolate generic replication behavior. Making the 
InMemory* types even more decoupled from replication logic.
2023-08-31 16:06:44 +01:00
Andi
a6ec81b179
Add deterministic disk vertex_count and edge_count (#1146)
* Add exact vertex_count and edge_count to disk storage
2023-08-29 13:07:23 +02:00
andrejtonev
c526ff2a8f
[master < ] Remove DbAccessor from non-transactional queries (#1201)
* Decouple non-transactional queries from DbAccessor
* Invalidate auth cache after AuthQuery

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2023-08-29 11:13:42 +02:00
Aidar Samerkhanov
5f509532f2
Add timestamp to DELETE_DESERIALIZED_OBJECT delta at which this object was created. (#1179)
Add timestamp to DELETE_DESERIALIZED_OBJECT delta at which this object was created.
RocksDB currently doesn't provide timestamp() functionality in iterators of TransationDB.
Because of that we are using constant "0" timestamp for DELETE_DESERIALIZED_OBJECT.
2023-08-28 10:56:17 +04:00
Andi
4b3ba908c7
Code improvements on disk storage (#1153)
* Improvements based on a code review

---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-08-26 14:16:12 +02:00
Andi
030b554ffd
Improve concurrency control for on-disk storage (#1154)
* Remove locking vertices and serialization checks
---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-08-25 14:42:52 +02:00
Gareth Andrew Lloyd
4bc5d749b2
Refactor replication, part 3 (#1177)
Changes to make replication code agnostic of the storage kind being used.

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
2023-08-25 10:52:07 +01:00
Ante Pušić
60e167d676
Optimize index and constraint updates (#1159) 2023-08-23 14:52:44 +02:00
andrejtonev
9355e58e73
Decoupling replication logic from InMemoryStorage (#1169) 2023-08-22 13:29:25 +02:00
Gareth Andrew Lloyd
97183fb9da
Fix FLAGS_delta_chain_cache_threshold typo (#1181) 2023-08-21 13:16:02 +02:00
Gareth Andrew Lloyd
adb65b2fff
Refactor memgraph.cpp (#1164) 2023-08-18 18:23:15 +02:00
Aidar Samerkhanov
3bf2cf65ab
Optimize splitting keys inside the on-disk storage (#1155) 2023-08-17 18:09:21 +02:00
ind1xa
8f3f693f20
Fix duration overflow (#1150) 2023-08-16 13:30:26 +02:00
Gareth Andrew Lloyd
2e51e703c3
Add supernode vertex cache (#1124)
Add supernode vertex cache to account for long delta chains and modifications in the same module being independent of scanning of the nodes in the next iteration of the pulling mechanism.
2023-08-11 10:18:28 +02:00
Andi
adf7533751
Optimize import of edges on disk (#1132) 2023-08-10 11:53:07 +02:00
Antonio Filipovic
509183e985
Improve performance on set properties (#1115) 2023-08-10 09:06:44 +02:00
Aidar Samerkhanov
1fe2190747
Filter deleted edges during edge prefetch (#1145) 2023-08-09 13:56:34 +02:00
Andi
762fe6a65d
Improve disk indices (#1139) 2023-08-09 10:16:49 +02:00
Aidar Samerkhanov
271b1a5ddb
Fix bug with on-disk triggers (#1134)
* Fix TriggerContext adaptation for accessors.
* Fix edge deserialization in case of the deleted vertex.
2023-08-08 10:37:14 +02:00
gvolfing
260660f1dd
Fix sequential label-property index recovery (#1135)
The parallel_exec_info should have been passed to this function before,
otherwise, the recovery of label-property indices would never have been
parallelized.
2023-08-05 23:20:15 +02:00
andrejtonev
4a99625287
Fix throwing if user was created without a license (#1067) 2023-08-01 23:36:12 +02:00
andrejtonev
5bbed6ef9a
Implement user caching to speed up PullPlan (#1109) 2023-08-01 23:04:35 +02:00
Andi
f0bac53e7b
Improve restore replication role (#1089) 2023-08-01 21:51:52 +02:00
andrejtonev
e8850549d2
Add multi-tenancy v1 (#952)
* Decouple BoltSession and communication::bolt::Session
* Add CREATE/USE/DROP DATABASE
* Add SHOW DATABASES
* Cover WebSocket session
* Simple session safety implemented via RWLock
* Storage symlinks for backward. compatibility
* Extend the audit log with the DB info
* Add auth part
* Add tenant recovery
2023-08-01 18:49:11 +02:00
Andi
60f4ffc6a1
Improve logging if replica cannot recover using curr WAL file (#1086) 2023-08-01 10:33:46 +02:00
gvolfing
210bea83d4
Add GraphQL transpilation compatibility (#1018)
* Add callable mappings feature
* Implement mgps.validate (void procedure)
* Make '_' a valid variable name
2023-07-31 14:48:12 +02:00
Josh Soref
57fe3463f2
Fix a bunch of spelling mistakes (1/n) (#1112) 2023-07-30 14:05:05 +02:00
Antonio Filipovic
285b409927
Improve all shortest paths memory usage (#981)
* Change allocator to PoolResource
2023-07-30 12:58:07 +02:00
Gareth Andrew Lloyd
8ebab84324
Add handling of partial results on timeout (#1046) 2023-07-30 10:48:11 +02:00
imilinovic
3fd9ce4a33
Add mgp::map erase and update (#1103) 2023-07-30 08:36:50 +02:00
Andi
be4eb95a98
Fix Jepsen replication pause (#1082) 2023-07-30 02:36:11 +02:00
Andi
18bd02423a
Fix PropertyStore buffer serialization (#1111) 2023-07-29 19:14:27 +02:00
andrejtonev
58c0c4cebb
Add missing-field-initialization warning flag (#1113) 2023-07-29 17:59:11 +02:00
andrejtonev
110ca3968c
Fix path generation ignores edge's element_id (#1108) 2023-07-29 15:51:51 +02:00
Vlasta
fae039c215
Improve logging for the on-disk storage (#1079) 2023-07-19 22:54:42 +02:00
Marko Budiselić
3b9133fd5a
Improve e2e and replication testing setup (#1061)
* Add `--replication-restore-state-on-startup` with `false` as default

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
2023-07-19 21:18:43 +02:00
Marko Budiselić
9d056e7649
Add experimental/v1 of ON_DISK_TRANSACTIONAL storage (#850)
Co-authored-by: Andi Skrgat <andi8647@gmail.com>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
2023-06-29 11:44:55 +02:00
Antonio Filipovic
aa4f68a37d
Add error handling on py batched module init (#1052) 2023-06-28 17:23:42 +02:00
Josipmrden
84721f7e0a
Add vertex degree to index statistics (#1026)
Add graph analysis of vertex degrees when doing ANALYZE GRAPH.
2023-06-27 18:06:20 +02:00
Ante Javor
261aa4f49b
Improve replication logging (#1030) 2023-06-27 17:57:51 +02:00
Gareth Andrew Lloyd
3b781bf525
Add HTTP+GZIP support to LOAD CSV (#1027) 2023-06-26 19:10:48 +02:00
Antonio Filipovic
d573eda8bb
Add python & cpp batching option in procedures
* Add API for batching from the procedure 
* Use PoolResource for batched procedures
2023-06-26 15:46:13 +02:00
Gareth Andrew Lloyd
5b1ba10183
Fix IN_MEMORY_ANALYTICAL storage GC (#1025) 2023-06-23 12:50:03 +02:00
Vlasta
b25e9968ee
Update links inside CSV import tool (#834) 2023-06-22 16:00:22 +02:00
Katarina Supe
68e5610566
Fix replica exception message (#930) 2023-06-22 14:41:59 +02:00
Ante Javor
0ea96663ba
Add check for opening snapshots (#966) 2023-06-22 13:29:49 +02:00
Marko Budiselić
e73eac77a9
Improve libstdc++ dependency on RPM systems (#863) 2023-06-22 10:14:59 +02:00
Josipmrden
b875649270
Add restoring of replication roles upon database startup (#791)
Fix replica node restoration on startup so it is restored as replica and not as main.
2023-06-21 19:08:58 +02:00
Josipmrden
05cc35bf93
Add command NULLIF for identifying nulls in LOAD CSV (#914)
Add NULLIF command which turns all row values corresponding to the string to the nullif character sequence.
2023-06-21 14:50:46 +02:00
Josipmrden
63f8298033
Fix MATCH + LOAD CSV to load CSV only once (#916)
* update profile query to use poolresource
* Optimize update of indexes
* Add ignore empty strings to load csv
* Add operator changes to support handling of nulls
* Store chunks in memory pools ordered
* Use same max block per chunks number
* Remove redundant return statement
* add hacky cached solution
* change map to set
* remove memory
* Add match load csv invalid behaviour commit
* Accept input on LOAD CSV
* Ommit changes not tied to the PR
* Add tests for match + load csv
* Add gqlalchemy installation for e2e tests
* Modify setup script to update packages
* Revert gqlalchemy to 1.3.3
* Revert gqlalchemy to 1.3.3
* Address PR review comments
* Ommit semicolon
---------

Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
2023-06-21 11:13:40 +02:00
Josipmrden
df95775222
Fix init file startup in community edition (#974)
* Fix init file startup in community edition

* Add possibility to build binary without MG_ENTERPRISE

* Added trace spdlog for when init file is not present

* Add gqlalchemy and unit tests

* Add init data files which correspond to the right directory by the github actions
2023-06-20 17:54:50 +02:00
andrejtonev
30ec570bb9
Add Bolt v5 support (#938) 2023-06-12 18:55:15 +02:00
Antonio Filipovic
d917c3f0fd
Fix slow IN LIST evaluation (#901) 2023-05-29 17:52:20 +02:00
andrejtonev
d842adbed3
Handle user-defined metadata and expose it with SHOW TRANSACTIONS(#945) 2023-05-29 11:40:14 +02:00
Josipmrden
651b6f3a5a Expose system metrics over HTTP Endpoint (#940) 2023-05-18 05:10:57 +00:00
Ante Pušić
0d9bd74a8a
Add support for map projection (#892) 2023-05-16 20:05:35 +02:00
andrejtonev
802f8aceda
Add data directory status and (un)lock query (#933) 2023-05-16 18:36:04 +02:00
gvolfing
7ddce539fa
Add return build type command (#894) 2023-05-16 16:02:03 +02:00
gvolfing
c3e4f81026
Include additional info inside storage mode info query (#883) 2023-05-16 14:25:41 +02:00
Antonio Filipovic
208705f296
Reduce memory consumption on return from python procedures (#932) 2023-05-16 10:33:09 +02:00
János Benjamin Antal
3a5f140c2b
Order chunks in utils::Pool to speed up deallocation (#898) 2023-05-02 13:08:20 +02:00
Antonio Filipovic
91017b7f36
Update profile query to use PoolResource for LOAD CSV (#885) 2023-04-26 18:04:13 +02:00
gvolfing
00f8d54249
Parallelize index creation (#882) 2023-04-26 16:28:02 +02:00
János Benjamin Antal
4fcdd52f88
Use correct memory resource (#900) 2023-04-26 10:02:55 +02:00
János Benjamin Antal
6c947947eb Parallelize recovery (#868)
* Parallelize edge recovery

* Load vertex labels and properties parallel

* Add parallel connectivity loading

* Add batches information to snapshot

* Introduce `items_per_batch` and `recovery_thread_count` flags

* Make possible to load snapshots with old version

* Add vertex batches to `RecoveryInfo`

* Extend durability integration tests with v15 test cases

* Add `std::vector` based `InitProperties`

* Use `InitProperties` in snapshot loading
2023-04-25 16:25:25 +02:00
János Benjamin Antal
97e250129e
Change AccumulateCursor to use utils::pmr::deque (#888)
* Increase performance by eliminating unnecessary `TypedValue` copies
2023-04-24 16:22:22 +02:00
Antonio Filipovic
2c6a55775d
Fix max block size bug on LOAD CSV(#877) 2023-04-19 16:10:20 +02:00
Andi
26a0866938
Fix index optimization bug (#860) 2023-04-04 23:43:13 +02:00
Antonio Filipovic
64e837b355
Introduce analytics mode (#772) 2023-04-04 18:46:26 +02:00
Antonio Filipovic
a586f2f98d
Change EvalContext and QueryExecution to use PoolResource on LOAD CSV (#825)
* Change PullPlan to use specific PoolResource for LOAD CSV
2023-04-04 16:54:08 +02:00
Josipmrden
9fc51f74a0
Skip label based auth on user with global visibility on graph (#837) 2023-04-04 11:13:25 +02:00
Josipmrden
128771a6ec
Add SHA-256 password encryption (#839) 2023-04-03 16:29:21 +02:00
Josipmrden
f5a49ed29f Add Cypher subqueries (#794) (#851)
Co-authored-by: Bruno Sačarić <bruno.sacaric@gmail.com>
2023-03-31 13:49:10 +00:00
Josipmrden
398503da7a
Add index statistics for better query planning (#812) 2023-03-30 15:34:34 +02:00
Bruno Sačarić
0819b40202
Fix bug on AllShortest with multiple edges between nodes (#832) 2023-03-29 16:39:41 +02:00
Andi
029be10f1d
Add queries to show or terminate active transactions (#790) 2023-03-27 15:46:00 +02:00
Jure Bajic
c4167bafdd
Add support for Amazon Linux 2 and stop generating C++ using Lisp/LCP (#814) 2023-03-14 19:24:55 +01:00