Commit Graph

3741 Commits

Author SHA1 Message Date
gvolfing
dbda8484ee Merge branch 'master' into Implement-automatic-index-creation 2024-03-25 08:44:02 +01:00
gvolfing
fd0f4a0346 Apply TODOs 2024-03-25 08:43:29 +01:00
Andi
3faa64a53c
Forbid TSAN with Jemalloc (#1842) 2024-03-25 07:20:55 +00:00
Andi
581767b491
Bump googletest to 1.14 (#1845) 2024-03-23 14:45:12 +00:00
Andi
b228b431a8
Fix installation of mgcxx (#1849) 2024-03-22 17:31:01 +01:00
gvolfing
dc2acf15e2 Merge branch 'master' into Implement-automatic-index-creation 2024-03-22 16:00:55 +01:00
gvolfing
d4d6e85398 Extend e2e tests 2024-03-22 16:00:02 +01:00
gvolfing
1558745e13 Update flag-check e2e test 2024-03-22 12:36:44 +01:00
Antonio Filipovic
13e3a1d0f7
Add distributed locks in HA (#1819)
- Add distributed locks
- Fix the wrong MAIN state on the follower coordinator
- Fix wrong main doing failover
2024-03-22 11:34:33 +00:00
Marko Barišić
89e13109d7
Fix jepsen nodes not starting up healthy (#1846)
* add a loop to check if all nodes started correctly and restart if any failed
2024-03-21 18:39:40 +01:00
DavIvek
56be736d30
Fix and update mgbench (#1838) 2024-03-21 12:34:59 +00:00
DavIvek
a3d2474c5b
Fix timestamps saving on-disk (#1811) 2024-03-21 10:50:55 +00:00
Andi
0913e95167
Rename HA startup flags (#1820) 2024-03-21 09:12:28 +00:00
Andi
f699c0b37f
Support bolt+routing (#1796) 2024-03-21 06:41:26 +00:00
gvolfing
1788e60691 Add automatic index creation functionality 2024-03-20 16:13:42 +01:00
Ante Pušić
9629f10166
Text search (#1603, #1739)
Add text search:
* named property search
* all-property search
* regex search
* aggregation over search results

Text search works with:
* non-parallel transactions
* durability (WAL files and snapshots)
* multitenancy
2024-03-20 10:29:24 +01:00
gvolfing
be32e8893d Create flags for enabling auto index creation 2024-03-19 15:00:44 +01:00
Marko Barišić
2ac649f3b5
Upgrade jepsen (#1594)
* Try with jepsen v0.3.5
* Add a few WIP adjustments
* Add replication restore state on startup flag
* Fix some run.sh scripts issues
* Improve cluster commands
* Run Jepsen on debian-12 with toolchain v5
---------
Co-authored-by: Marko Budiselic <mbudiselicbuda@gmail.com>
2024-03-18 16:38:58 +01:00
Marko Barišić
ec8536e11b
Make diff run on push to master again (#1826)
* Add workflow dispatch and run on push to master
2024-03-18 11:58:34 +01:00
Marko Barišić
84fe853169
Fix cargo not found when buidling in mgbuild container (#1825)
*Add source /home/mg/.cargo/env before cmake and make commands in mgbuild.sh
2024-03-18 10:47:59 +01:00
Josipmrden
082f9a7d9b
Add behaviour of no updates if vertex is updated with same value (#1791) 2024-03-15 14:45:21 +01:00
Aidar Samerkhanov
0ed2d18754
Add RollUpApply operator support to edge type index rewrite. (#1816) 2024-03-15 11:39:37 +04:00
Gareth Andrew Lloyd
8bc8e867e4
Pmr allocator unify (#1801)
Query allocator and evaluation allocator were different.
After analysis, was determined they should be the same, this will help 
future development reduce TypeValue copies during queries.

Changes:
- Common allocator, PoolResource backed by MonotonicResource
- Optimized Pool, now O(1) alloc/dealloc as all chunks in Pool form a single 
  free list
- 2nd PoolResource, using bin sizing, not as perfect for memory usage but 
  O(1) bin selection
- Now have jemalloc's background thread to make sure decay and return 
  to OS happens
- Optimized ProperyValue to be faster at destruction/copy/move
- Less temporary memory allocations
  - CSV reader now maintains a common line buffer it reuses on line reads
  - Writing out bolt values, now reuses a values buffer
  - Evaluating an int no longer makes temporary strings for errors it most 
    likely never throws
  - ExpandVariable will reuse existing edge list in frame it one existed
2024-03-14 11:21:59 -07:00
Marko Barišić
b0cdcd3483
Run CI in mgbuilder containers (#1749)
* Update deployment files for mgbuilders because of toolchain upgrade
* Fix args parameter in builder yaml files
* Add fedora 38, 39 and rockylinux 9.3 mgbuilder Dockerfiles
* Change format of ARG TOOLCHAIN_VERSION from toolchain-vX to vX
* Add function to check supported arch, build type, os and toolchain
* Add options to init subcommand
* Add image names to mgbuilders
* Add v2 of the run.sh script
* Add testing to run2.sh
* Add option for threads --thread
* Add options for enterprise license and organization name
* Make stop mgbuild container step run always
* Add --ci flag to init script
* Move init conditionals under build-memgraph flags
* Add --community flag to build-memgraph
* Change target dir inside mgbuild container
* Add node fix to debian 11, ubuntu 20.04 and ubuntu 22.04
* rm memgraph repo after installing deps
* Add mg user in Dockerfile
* Add step to install rust on all OSs
* Chown files copied into mgbuild container
* Add e2e tests
* Add jepsen test
* Bugfix: Using reference in a callback
* Bugfix: Broad target for e2e tests
* Up db info test limit
* Disable e2e streams tests
* Fix default THREADS
* Prioretize docker compose over docker-compose
* Improve selection between docker compose and docker-compose
* Install PyYAML as mg user
* Fix doxygen install for rocky linux 9.3
* Fix rocky-9.3 environment script to properly install sbcl
* Rename all rocky-9 mentions to rocky-9.3
* Add mgdeps-cache and benchgraph-api hostnames to mgbuild images
* Add logic to pull mgbuild image if missing
* Fix build errors on toolchain-v5 (#1806)
* Rename run2 script, remove run script, add small features to mgbuild.sh
* Add --no-copy flag to build-memgraph to resolve TODO
* Add timeouts to diff jobs
* Fix asio flaky clone, try mgdeps-cache first

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Co-authored-by: Ante Pušić <ante.f.pusic@gmail.com>
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-03-14 12:19:59 +01:00
Andi
24f8a14b43
Improve registration queries in HA environment(#1809) 2024-03-13 13:04:27 +00:00
Josipmrden
2cab07429e
Add new PR template (#1798) 2024-03-13 10:09:22 +01:00
DavIvek
de2e2048ef
Support label creation via property values (#1762) 2024-03-12 12:55:40 +00:00
Gareth Andrew Lloyd
a282542666
Optimise ORDER BY, RANGE, UNWIND (#1781)
* Optimise frame change

* Optimise distinct + orderby memory usage

- dispose collections as earlier as possible
- move values rather than copy

* Better perf, ORDER BY

* Optimise RANGE and UNWIND

* ConstraintVerificationInfo only if at least one constraint

* Optimise TypeValue

* Clang-tidy fix
2024-03-12 00:26:11 +00:00
Josipmrden
462336ff78
Fix early exit for OR expression (#1738) 2024-03-11 22:44:15 +01:00
Aidar Samerkhanov
1c71d605ff
Fix PatternVisitor compilation in toolchain-v5 (#1803) 2024-03-08 19:20:40 -08:00
Antonio Filipovic
2a5388cea9
Add tests to verify log store works properly (#1794) 2024-03-08 15:16:30 +00:00
gvolfing
619b01f3f8
Implement edge type indices (#1542)
Implement edge type indices (#1542 )
2024-03-08 08:44:48 +01:00
Andi
5ca98f9543
Fix snapshot creation in RSM and forbid multiple leaders (#1788) 2024-03-07 17:40:32 +00:00
Aidar Samerkhanov
a099417c56
List Pattern Comprehension planner (#1686) 2024-03-07 18:41:02 +04:00
Antonio Filipovic
02325f8673
Fix bug prone add server to cluster behavior (#1792) 2024-03-07 11:10:33 +00:00
Katarina Supe
6f849a14df
Update cypherl transform script (#1701)
* Update cypherl transform script

* Add new script and fix typo

* Add convert to separate files script

---------

Co-authored-by: Marko Budiselić <marko.budiselic@memgraph.com>
2024-03-07 10:04:36 +01:00
Andi
75aad72984
Improve in-memory RAFT state (#1782) 2024-03-06 09:16:46 +01:00
Antonio Filipovic
d4d4660af0
Add force sync REPLICA with MAIN (#1777) 2024-03-05 16:51:14 +00:00
Andi
1802dc93d1
Improve Raft log serialization (#1778) 2024-03-05 07:33:13 +00:00
Andi
822183b62d
Support failure of coordinators (#1728) 2024-03-04 07:24:18 +00:00
Antonio Filipovic
33caa27161
Ensure replication works on HA cluster in different scenarios (#1743) 2024-03-01 12:32:56 +01:00
Marko Barišić
f316f7db87
Add openssl to MEMGRAPH_BUILD_DEPS for amzn-2 and centos-7 (#1771) 2024-02-28 18:21:56 +01:00
Gareth Andrew Lloyd
55f224839e
Do not use UUID_STR_LEN (#1770)
Older libuuid did not have this macro, we need to publish for older
distro with older libs.
2024-02-28 17:46:03 +01:00
Antonio Filipovic
b561c61b64
HA: Add initial logic for choosing new replica (#1729) 2024-02-28 09:57:00 +00:00
DavIvek
b7de79d5a0
Fix schema.node_type_properties() and schema.rel_type_properties() (#1718) 2024-02-27 21:40:55 +00:00
Gareth Andrew Lloyd
da898be8f9
Compact Delta 80B -> 56B (#1747)
Make special structure for old_disk_key. std::optional<std::string> was
40B, which is the largest member of out action union. Replaced with 8B,
structure.

This makes largest member now vertex_edge at 24B, this means Delta is
now only 56B.

🥳🎉 Now less than a cacheline 🎊
2024-02-27 17:21:52 +00:00
Gareth Andrew Lloyd
a6fcdfd905
Make GC + snapshot, main lock friendly (#1759)
- Only IN_MEMORY_ANALYTICAL requires unique lock during snapshot
- GC in some cases will be provide with unique lock
  - This fact can be used for optimisations
  - In all other cases, optimisations should be done with alternative
    check. Not via getting a unique lock

Also:
- Faster property lookup
- Faster index iteration (better conditional branching)
2024-02-27 15:45:08 +01:00
Marko Barišić
e88c7a0aa5
Add jobs for pushing ARM packages (#1765)
* Add jobs for pushing ARM packages
2024-02-27 12:08:53 +01:00
Marko Barišić
86ff96697d
Minor update to the rc workflow (#1760)
* Increase ARM build timeout to 120 minutes

* Remove PushToS3 job and make each Package job push to S3 individually

* Expand ARM timeout to 150 minutes for added safety; revert this after release
2024-02-26 22:57:21 +01:00
andrejtonev
f4d9a3695d
Introduce multi-tenancy to SHOW REPLICAS (#1735)
---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-02-26 19:05:49 +00:00