Commit Graph

1936 Commits

Author SHA1 Message Date
Andi Skrgat
3a0bd33a91 Test CoordClusterState 2024-03-25 07:22:17 +01:00
Andi
581767b491
Bump googletest to 1.14 () 2024-03-23 14:45:12 +00:00
Antonio Filipovic
13e3a1d0f7
Add distributed locks in HA ()
- Add distributed locks
- Fix the wrong MAIN state on the follower coordinator
- Fix wrong main doing failover
2024-03-22 11:34:33 +00:00
Marko Barišić
89e13109d7
Fix jepsen nodes not starting up healthy ()
* add a loop to check if all nodes started correctly and restart if any failed
2024-03-21 18:39:40 +01:00
DavIvek
56be736d30
Fix and update mgbench () 2024-03-21 12:34:59 +00:00
DavIvek
a3d2474c5b
Fix timestamps saving on-disk () 2024-03-21 10:50:55 +00:00
Andi
0913e95167
Rename HA startup flags () 2024-03-21 09:12:28 +00:00
Andi
f699c0b37f
Support bolt+routing () 2024-03-21 06:41:26 +00:00
Ante Pušić
9629f10166
Text search (, )
Add text search:
* named property search
* all-property search
* regex search
* aggregation over search results

Text search works with:
* non-parallel transactions
* durability (WAL files and snapshots)
* multitenancy
2024-03-20 10:29:24 +01:00
Marko Barišić
2ac649f3b5
Upgrade jepsen ()
* Try with jepsen v0.3.5
* Add a few WIP adjustments
* Add replication restore state on startup flag
* Fix some run.sh scripts issues
* Improve cluster commands
* Run Jepsen on debian-12 with toolchain v5
---------
Co-authored-by: Marko Budiselic <mbudiselicbuda@gmail.com>
2024-03-18 16:38:58 +01:00
Josipmrden
082f9a7d9b
Add behaviour of no updates if vertex is updated with same value () 2024-03-15 14:45:21 +01:00
Gareth Andrew Lloyd
8bc8e867e4
Pmr allocator unify ()
Query allocator and evaluation allocator were different.
After analysis, was determined they should be the same, this will help 
future development reduce TypeValue copies during queries.

Changes:
- Common allocator, PoolResource backed by MonotonicResource
- Optimized Pool, now O(1) alloc/dealloc as all chunks in Pool form a single 
  free list
- 2nd PoolResource, using bin sizing, not as perfect for memory usage but 
  O(1) bin selection
- Now have jemalloc's background thread to make sure decay and return 
  to OS happens
- Optimized ProperyValue to be faster at destruction/copy/move
- Less temporary memory allocations
  - CSV reader now maintains a common line buffer it reuses on line reads
  - Writing out bolt values, now reuses a values buffer
  - Evaluating an int no longer makes temporary strings for errors it most 
    likely never throws
  - ExpandVariable will reuse existing edge list in frame it one existed
2024-03-14 11:21:59 -07:00
Marko Barišić
b0cdcd3483
Run CI in mgbuilder containers ()
* Update deployment files for mgbuilders because of toolchain upgrade
* Fix args parameter in builder yaml files
* Add fedora 38, 39 and rockylinux 9.3 mgbuilder Dockerfiles
* Change format of ARG TOOLCHAIN_VERSION from toolchain-vX to vX
* Add function to check supported arch, build type, os and toolchain
* Add options to init subcommand
* Add image names to mgbuilders
* Add v2 of the run.sh script
* Add testing to run2.sh
* Add option for threads --thread
* Add options for enterprise license and organization name
* Make stop mgbuild container step run always
* Add --ci flag to init script
* Move init conditionals under build-memgraph flags
* Add --community flag to build-memgraph
* Change target dir inside mgbuild container
* Add node fix to debian 11, ubuntu 20.04 and ubuntu 22.04
* rm memgraph repo after installing deps
* Add mg user in Dockerfile
* Add step to install rust on all OSs
* Chown files copied into mgbuild container
* Add e2e tests
* Add jepsen test
* Bugfix: Using reference in a callback
* Bugfix: Broad target for e2e tests
* Up db info test limit
* Disable e2e streams tests
* Fix default THREADS
* Prioretize docker compose over docker-compose
* Improve selection between docker compose and docker-compose
* Install PyYAML as mg user
* Fix doxygen install for rocky linux 9.3
* Fix rocky-9.3 environment script to properly install sbcl
* Rename all rocky-9 mentions to rocky-9.3
* Add mgdeps-cache and benchgraph-api hostnames to mgbuild images
* Add logic to pull mgbuild image if missing
* Fix build errors on toolchain-v5 ()
* Rename run2 script, remove run script, add small features to mgbuild.sh
* Add --no-copy flag to build-memgraph to resolve TODO
* Add timeouts to diff jobs
* Fix asio flaky clone, try mgdeps-cache first

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Co-authored-by: Ante Pušić <ante.f.pusic@gmail.com>
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-03-14 12:19:59 +01:00
Andi
24f8a14b43
Improve registration queries in HA environment() 2024-03-13 13:04:27 +00:00
DavIvek
de2e2048ef
Support label creation via property values () 2024-03-12 12:55:40 +00:00
Gareth Andrew Lloyd
a282542666
Optimise ORDER BY, RANGE, UNWIND ()
* Optimise frame change

* Optimise distinct + orderby memory usage

- dispose collections as earlier as possible
- move values rather than copy

* Better perf, ORDER BY

* Optimise RANGE and UNWIND

* ConstraintVerificationInfo only if at least one constraint

* Optimise TypeValue

* Clang-tidy fix
2024-03-12 00:26:11 +00:00
Antonio Filipovic
2a5388cea9
Add tests to verify log store works properly () 2024-03-08 15:16:30 +00:00
gvolfing
619b01f3f8
Implement edge type indices ()
Implement edge type indices ( )
2024-03-08 08:44:48 +01:00
Andi
5ca98f9543
Fix snapshot creation in RSM and forbid multiple leaders () 2024-03-07 17:40:32 +00:00
Aidar Samerkhanov
a099417c56
List Pattern Comprehension planner () 2024-03-07 18:41:02 +04:00
Antonio Filipovic
02325f8673
Fix bug prone add server to cluster behavior () 2024-03-07 11:10:33 +00:00
Andi
75aad72984
Improve in-memory RAFT state () 2024-03-06 09:16:46 +01:00
Antonio Filipovic
d4d4660af0
Add force sync REPLICA with MAIN () 2024-03-05 16:51:14 +00:00
Andi
1802dc93d1
Improve Raft log serialization () 2024-03-05 07:33:13 +00:00
Andi
822183b62d
Support failure of coordinators () 2024-03-04 07:24:18 +00:00
Antonio Filipovic
33caa27161
Ensure replication works on HA cluster in different scenarios () 2024-03-01 12:32:56 +01:00
Antonio Filipovic
b561c61b64
HA: Add initial logic for choosing new replica () 2024-02-28 09:57:00 +00:00
DavIvek
b7de79d5a0
Fix schema.node_type_properties() and schema.rel_type_properties() () 2024-02-27 21:40:55 +00:00
andrejtonev
f4d9a3695d
Introduce multi-tenancy to SHOW REPLICAS ()
---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-02-26 19:05:49 +00:00
andrejtonev
c2e9df309a
Correctly call driver v1 tests () 2024-02-26 17:28:13 +00:00
andrejtonev
82c47ee80d
GetInfo simplification ()
* Removed force dir in the GetInfo functions
2024-02-26 14:55:45 +00:00
andrejtonev
6a4ef55e90
Better auth user/role handling ()
* Stop auth module from creating users
* Explicit about auth policy (check if no users defined OR auth module used)
* Role supports database access definition
* Authenticate() returns user or role
* AuthChecker generates QueryUserOrRole (can be empty)
* QueryUserOrRole actually authorizes
* Add auth cache invalidation
* Better database access queries (GRANT, DENY, REVOKE DATABASE)
2024-02-22 14:00:39 +00:00
Aidar Samerkhanov
9a20ac494d
In BFS expansion filter by path we should shrink path to restore state prior to expansion only if the path was changed. () 2024-02-22 05:34:08 +00:00
Marko Budiselić
61b9bb0f59
Add toolchain-v5 compatibility Revert to C++20 ()
* Upgrade cppitertools, spdlog, fmt, rapidcheck
* Make compilation work on both v4 and v5 toolchains
2024-02-19 21:09:54 +01:00
Andi
7ec648b4ce
Add --experimental-enabled=high-availability () 2024-02-19 16:28:15 +00:00
Josipmrden
bae3e8a6d3
Add function for property sizes ()
Add function for property sizes
2024-02-19 13:56:01 +01:00
Andi
f3574012c5
Add cpp23 support () 2024-02-19 10:36:51 +00:00
Gareth Andrew Lloyd
33c400fcc1
Fixup memory e2e tests ()
- Remove the e2e that did concurrent mgp_* calls on the same transaction
  (ATM this is unsupported)
- Fix up the concurrent mgp_global_alloc test to be testing it more precisely
- Reduce the memory limit on detach delete test due to recent memory
  optimizations around deltas.
- No longer throw from hook, through jemalloc C, to our C++ on other
  side. This cause mutex unlocks to not happen.
- No longer allocate error messages while inside the hook. This caused
  recursive entry back inside jamalloc which would try to relock a
  non-recursive mutex.
2024-02-16 15:35:08 +00:00
Andi
3e3224f0a2
Forbid having multiple mains in the cluster () 2024-02-16 11:41:15 +00:00
Antonio Filipovic
bfc756c092
HA: Polish flow for replicas from coordinator () 2024-02-16 10:58:01 +01:00
Andi
20b47845f0
Forbid writing to cluster-managed main on restart () 2024-02-15 14:07:04 +01:00
Andi
fb281459b9
Add support for unregistering replication instances () 2024-02-14 14:24:59 +00:00
Gareth Andrew Lloyd
f48151576b
System replication experimental flag ()
- Remove the compile time control
- Introduce the runtime control flag

New flag `--experimental-enabled=system-replication`
2024-02-13 12:57:18 +00:00
Andi
4a7c7f0898
Distributed coordinators () 2024-02-13 08:49:28 +00:00
Ivan Milinović
7688a1b068
Fix unbound variable causing crash inside subquery () 2024-02-13 01:10:03 +01:00
Antonio Filipovic
4f4a569c72
Revert replication tests () 2024-02-12 16:42:57 +01:00
DavIvek
0133673f1d
Add support for query params in load csv () 2024-02-09 18:26:27 +01:00
DavIvek
786cdea260
Fix go driver test () 2024-02-09 17:07:30 +01:00
Antonio Filipovic
54f78f9217
Revert e2e tests and remove flaky ones () 2024-02-09 12:55:31 +01:00
Andi
cf80687d1d
HA: Organize Raft coordinator group () 2024-02-08 09:11:33 +00:00