Commit Graph

3741 Commits

Author SHA1 Message Date
gvolfing
6b7e1d8a9f Assert only if metadata flag is enabled 2024-03-22 11:41:02 +01:00
gvolfing
6f18117b0a Merge branch 'master' into Fetch-edge-by-edge-id 2024-03-20 11:39:36 +01:00
Ante Pušić
9629f10166
Text search (, )
Add text search:
* named property search
* all-property search
* regex search
* aggregation over search results

Text search works with:
* non-parallel transactions
* durability (WAL files and snapshots)
* multitenancy
2024-03-20 10:29:24 +01:00
gvolfing
da1189b960 Merge branch 'master' into Fetch-edge-by-edge-id 2024-03-19 16:21:50 +01:00
gvolfing
494b4d2d83 Undo outcommented tests 2024-03-19 16:21:16 +01:00
gvolfing
527d4c864f React to PR comments 2024-03-19 16:18:23 +01:00
Marko Barišić
2ac649f3b5
Upgrade jepsen ()
* Try with jepsen v0.3.5
* Add a few WIP adjustments
* Add replication restore state on startup flag
* Fix some run.sh scripts issues
* Improve cluster commands
* Run Jepsen on debian-12 with toolchain v5
---------
Co-authored-by: Marko Budiselic <mbudiselicbuda@gmail.com>
2024-03-18 16:38:58 +01:00
gvolfing
71fdace31d Generic refactor 2024-03-18 12:52:31 +01:00
Marko Barišić
ec8536e11b
Make diff run on push to master again ()
* Add workflow dispatch and run on push to master
2024-03-18 11:58:34 +01:00
gvolfing
9dff306782 Merge branch 'master' into Fetch-edge-by-edge-id 2024-03-18 11:10:21 +01:00
gvolfing
5d233c2486 Conform cpp-api when changing edges on the fly 2024-03-18 11:02:43 +01:00
Marko Barišić
84fe853169
Fix cargo not found when buidling in mgbuild container ()
*Add source /home/mg/.cargo/env before cmake and make commands in mgbuild.sh
2024-03-18 10:47:59 +01:00
gvolfing
db6b1fc265 Implement edge-visibility checking 2024-03-18 09:48:00 +01:00
gvolfing
9825bdb41b Add snapshot based recovery capabilites 2024-03-15 15:59:01 +01:00
Josipmrden
082f9a7d9b
Add behaviour of no updates if vertex is updated with same value () 2024-03-15 14:45:21 +01:00
gvolfing
4f6d8c10ec Add metadata flag + runtime check during id scan 2024-03-15 13:56:43 +01:00
Aidar Samerkhanov
0ed2d18754
Add RollUpApply operator support to edge type index rewrite. () 2024-03-15 11:39:37 +04:00
Gareth Andrew Lloyd
8bc8e867e4
Pmr allocator unify ()
Query allocator and evaluation allocator were different.
After analysis, was determined they should be the same, this will help 
future development reduce TypeValue copies during queries.

Changes:
- Common allocator, PoolResource backed by MonotonicResource
- Optimized Pool, now O(1) alloc/dealloc as all chunks in Pool form a single 
  free list
- 2nd PoolResource, using bin sizing, not as perfect for memory usage but 
  O(1) bin selection
- Now have jemalloc's background thread to make sure decay and return 
  to OS happens
- Optimized ProperyValue to be faster at destruction/copy/move
- Less temporary memory allocations
  - CSV reader now maintains a common line buffer it reuses on line reads
  - Writing out bolt values, now reuses a values buffer
  - Evaluating an int no longer makes temporary strings for errors it most 
    likely never throws
  - ExpandVariable will reuse existing edge list in frame it one existed
2024-03-14 11:21:59 -07:00
Marko Barišić
b0cdcd3483
Run CI in mgbuilder containers ()
* Update deployment files for mgbuilders because of toolchain upgrade
* Fix args parameter in builder yaml files
* Add fedora 38, 39 and rockylinux 9.3 mgbuilder Dockerfiles
* Change format of ARG TOOLCHAIN_VERSION from toolchain-vX to vX
* Add function to check supported arch, build type, os and toolchain
* Add options to init subcommand
* Add image names to mgbuilders
* Add v2 of the run.sh script
* Add testing to run2.sh
* Add option for threads --thread
* Add options for enterprise license and organization name
* Make stop mgbuild container step run always
* Add --ci flag to init script
* Move init conditionals under build-memgraph flags
* Add --community flag to build-memgraph
* Change target dir inside mgbuild container
* Add node fix to debian 11, ubuntu 20.04 and ubuntu 22.04
* rm memgraph repo after installing deps
* Add mg user in Dockerfile
* Add step to install rust on all OSs
* Chown files copied into mgbuild container
* Add e2e tests
* Add jepsen test
* Bugfix: Using reference in a callback
* Bugfix: Broad target for e2e tests
* Up db info test limit
* Disable e2e streams tests
* Fix default THREADS
* Prioretize docker compose over docker-compose
* Improve selection between docker compose and docker-compose
* Install PyYAML as mg user
* Fix doxygen install for rocky linux 9.3
* Fix rocky-9.3 environment script to properly install sbcl
* Rename all rocky-9 mentions to rocky-9.3
* Add mgdeps-cache and benchgraph-api hostnames to mgbuild images
* Add logic to pull mgbuild image if missing
* Fix build errors on toolchain-v5 ()
* Rename run2 script, remove run script, add small features to mgbuild.sh
* Add --no-copy flag to build-memgraph to resolve TODO
* Add timeouts to diff jobs
* Fix asio flaky clone, try mgdeps-cache first

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Co-authored-by: Ante Pušić <ante.f.pusic@gmail.com>
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-03-14 12:19:59 +01:00
Andi
24f8a14b43
Improve registration queries in HA environment() 2024-03-13 13:04:27 +00:00
Josipmrden
2cab07429e
Add new PR template () 2024-03-13 10:09:22 +01:00
DavIvek
de2e2048ef
Support label creation via property values () 2024-03-12 12:55:40 +00:00
gvolfing
96651170a0 Add query plan test 2024-03-12 08:50:51 +01:00
Gareth Andrew Lloyd
a282542666
Optimise ORDER BY, RANGE, UNWIND ()
* Optimise frame change

* Optimise distinct + orderby memory usage

- dispose collections as earlier as possible
- move values rather than copy

* Better perf, ORDER BY

* Optimise RANGE and UNWIND

* ConstraintVerificationInfo only if at least one constraint

* Optimise TypeValue

* Clang-tidy fix
2024-03-12 00:26:11 +00:00
Josipmrden
462336ff78
Fix early exit for OR expression () 2024-03-11 22:44:15 +01:00
gvolfing
a5ee0fad08 Enhance planner for edge-id indexing 2024-03-11 19:39:32 +01:00
gvolfing
956705293e Add logical operator for id based edge scans 2024-03-11 09:56:00 +01:00
gvolfing
5167c6ee37 Add temporary edge retrieval code 2024-03-10 19:39:30 +01:00
gvolfing
a58f2887b3 Create storage level call-chain 2024-03-10 18:56:15 +01:00
Aidar Samerkhanov
1c71d605ff
Fix PatternVisitor compilation in toolchain-v5 () 2024-03-08 19:20:40 -08:00
Antonio Filipovic
2a5388cea9
Add tests to verify log store works properly () 2024-03-08 15:16:30 +00:00
gvolfing
619b01f3f8
Implement edge type indices ()
Implement edge type indices ( )
2024-03-08 08:44:48 +01:00
Andi
5ca98f9543
Fix snapshot creation in RSM and forbid multiple leaders () 2024-03-07 17:40:32 +00:00
Aidar Samerkhanov
a099417c56
List Pattern Comprehension planner () 2024-03-07 18:41:02 +04:00
Antonio Filipovic
02325f8673
Fix bug prone add server to cluster behavior () 2024-03-07 11:10:33 +00:00
Katarina Supe
6f849a14df
Update cypherl transform script ()
* Update cypherl transform script

* Add new script and fix typo

* Add convert to separate files script

---------

Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2024-03-07 10:04:36 +01:00
Andi
75aad72984
Improve in-memory RAFT state () 2024-03-06 09:16:46 +01:00
Antonio Filipovic
d4d4660af0
Add force sync REPLICA with MAIN () 2024-03-05 16:51:14 +00:00
Andi
1802dc93d1
Improve Raft log serialization () 2024-03-05 07:33:13 +00:00
Andi
822183b62d
Support failure of coordinators () 2024-03-04 07:24:18 +00:00
Antonio Filipovic
33caa27161
Ensure replication works on HA cluster in different scenarios () 2024-03-01 12:32:56 +01:00
Marko Barišić
f316f7db87
Add openssl to MEMGRAPH_BUILD_DEPS for amzn-2 and centos-7 () 2024-02-28 18:21:56 +01:00
Gareth Andrew Lloyd
55f224839e
Do not use UUID_STR_LEN ()
Older libuuid did not have this macro, we need to publish for older
distro with older libs.
2024-02-28 17:46:03 +01:00
Antonio Filipovic
b561c61b64
HA: Add initial logic for choosing new replica () 2024-02-28 09:57:00 +00:00
DavIvek
b7de79d5a0
Fix schema.node_type_properties() and schema.rel_type_properties() () 2024-02-27 21:40:55 +00:00
Gareth Andrew Lloyd
da898be8f9
Compact Delta 80B -> 56B ()
Make special structure for old_disk_key. std::optional<std::string> was
40B, which is the largest member of out action union. Replaced with 8B,
structure.

This makes largest member now vertex_edge at 24B, this means Delta is
now only 56B.

🥳🎉 Now less than a cacheline 🎊
2024-02-27 17:21:52 +00:00
Gareth Andrew Lloyd
a6fcdfd905
Make GC + snapshot, main lock friendly ()
- Only IN_MEMORY_ANALYTICAL requires unique lock during snapshot
- GC in some cases will be provide with unique lock
  - This fact can be used for optimisations
  - In all other cases, optimisations should be done with alternative
    check. Not via getting a unique lock

Also:
- Faster property lookup
- Faster index iteration (better conditional branching)
2024-02-27 15:45:08 +01:00
Marko Barišić
e88c7a0aa5
Add jobs for pushing ARM packages ()
* Add jobs for pushing ARM packages
2024-02-27 12:08:53 +01:00
Marko Barišić
86ff96697d
Minor update to the rc workflow ()
* Increase ARM build timeout to 120 minutes

* Remove PushToS3 job and make each Package job push to S3 individually

* Expand ARM timeout to 150 minutes for added safety; revert this after release
2024-02-26 22:57:21 +01:00
andrejtonev
f4d9a3695d
Introduce multi-tenancy to SHOW REPLICAS ()
---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-02-26 19:05:49 +00:00