Commit Graph

2357 Commits

Author SHA1 Message Date
antoniofilipovic
d734cb67c0 add more tests on replication with high availability 2024-02-29 09:47:18 +01:00
antoniofilipovic
c52c0975de add test for instance behind few commits but contains same epoch 2024-02-29 09:47:18 +01:00
Gareth Andrew Lloyd
55f224839e
Do not use UUID_STR_LEN (#1770)
Older libuuid did not have this macro, we need to publish for older
distro with older libs.
2024-02-28 17:46:03 +01:00
Antonio Filipovic
b561c61b64
HA: Add initial logic for choosing new replica (#1729) 2024-02-28 09:57:00 +00:00
Gareth Andrew Lloyd
da898be8f9
Compact Delta 80B -> 56B (#1747)
Make special structure for old_disk_key. std::optional<std::string> was
40B, which is the largest member of out action union. Replaced with 8B,
structure.

This makes largest member now vertex_edge at 24B, this means Delta is
now only 56B.

🥳🎉 Now less than a cacheline 🎊
2024-02-27 17:21:52 +00:00
Gareth Andrew Lloyd
a6fcdfd905
Make GC + snapshot, main lock friendly (#1759)
- Only IN_MEMORY_ANALYTICAL requires unique lock during snapshot
- GC in some cases will be provide with unique lock
  - This fact can be used for optimisations
  - In all other cases, optimisations should be done with alternative
    check. Not via getting a unique lock

Also:
- Faster property lookup
- Faster index iteration (better conditional branching)
2024-02-27 15:45:08 +01:00
andrejtonev
f4d9a3695d
Introduce multi-tenancy to SHOW REPLICAS (#1735)
---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-02-26 19:05:49 +00:00
andrejtonev
82c47ee80d
GetInfo simplification (#1621)
* Removed force dir in the GetInfo functions
2024-02-26 14:55:45 +00:00
andrejtonev
6a4ef55e90
Better auth user/role handling (#1699)
* Stop auth module from creating users
* Explicit about auth policy (check if no users defined OR auth module used)
* Role supports database access definition
* Authenticate() returns user or role
* AuthChecker generates QueryUserOrRole (can be empty)
* QueryUserOrRole actually authorizes
* Add auth cache invalidation
* Better database access queries (GRANT, DENY, REVOKE DATABASE)
2024-02-22 14:00:39 +00:00
Aidar Samerkhanov
9a20ac494d
In BFS expansion filter by path we should shrink path to restore state prior to expansion only if the path was changed. (#1745) 2024-02-22 05:34:08 +00:00
Marko Budiselić
61b9bb0f59
Add toolchain-v5 compatibility Revert to C++20 (#587)
* Upgrade cppitertools, spdlog, fmt, rapidcheck
* Make compilation work on both v4 and v5 toolchains
2024-02-19 21:09:54 +01:00
Andi
7ec648b4ce
Add --experimental-enabled=high-availability (#1720) 2024-02-19 16:28:15 +00:00
Josipmrden
bae3e8a6d3
Add function for property sizes (#1557)
Add function for property sizes
2024-02-19 13:56:01 +01:00
Gareth Andrew Lloyd
33c400fcc1
Fixup memory e2e tests (#1715)
- Remove the e2e that did concurrent mgp_* calls on the same transaction
  (ATM this is unsupported)
- Fix up the concurrent mgp_global_alloc test to be testing it more precisely
- Reduce the memory limit on detach delete test due to recent memory
  optimizations around deltas.
- No longer throw from hook, through jemalloc C, to our C++ on other
  side. This cause mutex unlocks to not happen.
- No longer allocate error messages while inside the hook. This caused
  recursive entry back inside jamalloc which would try to relock a
  non-recursive mutex.
2024-02-16 15:35:08 +00:00
Andi
3e3224f0a2
Forbid having multiple mains in the cluster (#1727) 2024-02-16 11:41:15 +00:00
Antonio Filipovic
bfc756c092
HA: Polish flow for replicas from coordinator (#1711) 2024-02-16 10:58:01 +01:00
Andi
20b47845f0
Forbid writing to cluster-managed main on restart (#1717) 2024-02-15 14:07:04 +01:00
Andi
fb281459b9
Add support for unregistering replication instances (#1712) 2024-02-14 14:24:59 +00:00
Andi
3a7e62f72c
Forbid branching when registering replica in auto-managed cluster (#1709) 2024-02-14 08:02:51 +00:00
Gareth Andrew Lloyd
f48151576b
System replication experimental flag (#1702)
- Remove the compile time control
- Introduce the runtime control flag

New flag `--experimental-enabled=system-replication`
2024-02-13 12:57:18 +00:00
Andi
4a7c7f0898
Distributed coordinators (#1693) 2024-02-13 08:49:28 +00:00
Ivan Milinović
7688a1b068
Fix unbound variable causing crash inside subquery (#1710) 2024-02-13 01:10:03 +01:00
Antonio Filipovic
4f4a569c72
Revert replication tests (#1707) 2024-02-12 16:42:57 +01:00
Ivan Milinović
a511e63c7a
Fix memory tracker counting wrong after OOM (#1651) 2024-02-11 20:29:06 +01:00
DavIvek
0133673f1d
Add support for query params in load csv (#1653) 2024-02-09 18:26:27 +01:00
Andi
cf80687d1d
HA: Organize Raft coordinator group (#1687) 2024-02-08 09:11:33 +00:00
Aidar Samerkhanov
2fa8e00124
Fix accumulated path evaluation in builtin algorithms. (#1642)
Fix accumulated path evaluation in DFS, BFS, WeghtedShortestPath and AllShortestPath algorithm.
2024-02-08 10:48:54 +04:00
Antonio Filipovic
c15b62a88d
HA: Disable replication from old main (#1674) 2024-02-07 11:20:47 +01:00
Gareth Andrew Lloyd
4ef6a1f9c3
Improve memory handling of Deltas (#1688)
- Reduce delta from 104B to 80B
- Hold and pass them around as in a deque
- Detect and deleted deltas within commit if safe to do so
2024-02-06 18:07:38 +01:00
andrejtonev
7ead00f23e
Adding authentication data replication (#1666)
* Add AUTH system tx deltas
* Add auth data RPC and handlers
* Support multiple system deltas in a single transaction
* Added e2e test
* Bugfix: KVStore segfault after move

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-02-05 10:37:00 +00:00
Andi
cb7b88ad92
HA: Support restart of instances (#1672) 2024-02-01 11:55:48 +01:00
Andi
6ab4235cc9
Add NuRaft library (#1678) 2024-01-31 13:13:59 +01:00
Gareth Andrew Lloyd
97b1e67d80
Fix auth durability (#1644)
* Change auth durability to store hash algorithm
* Add Salt to SHA256
2024-01-30 18:17:05 +00:00
Andi
78a88737f8
HA: Add automatic failover (#1646)
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-01-29 15:34:00 +01:00
andrejtonev
ff44d68843
Simplify auth::Auth (#1663)
Moved various auth flags under a single config
Moved all regex logic under auth::Auth
2024-01-29 12:52:32 +00:00
Gareth Andrew Lloyd
9f7118d893
Performance tuning based on stress test (#1572)
Minor changes that speedup the large stress test.
Also now uses a stop token for a more productive shutdown. No need to wait for expensive GC runs.
2024-01-25 17:14:58 +00:00
Andi
38ade99652
HA: Add coordinator to replication cluster (#1608) 2024-01-24 13:07:51 +01:00
DavIvek
6706ebfa2b
Add support for query parameters in return limit (#1654) 2024-01-23 19:17:27 +01:00
Gareth Andrew Lloyd
e7f6a5f4f4
Fix SkipList iterators (#1635)
Fix SkipList iterators and find methods to be as expected by normal C++ iterator usage
2024-01-23 15:31:28 +00:00
andrejtonev
071df2f439
Replication refactor part 7 (#1550)
* Split queries into system and data queries
* System queries are sequentially executed and generate separate transaction deltas
* System transaction try locks for 100ms
* last_commited_system_ts saved to DBMS durability
* Replicating CREATE/DROP DATABASE
* Sending a system snapshot if REPLICA behind
* Passing a copy of the gatekeeper::access as std::any to all functions that could call an async execution
* Removed delete_on_drop flag (we now always delete on drop)
* Using UUID as the directory name for databases
* DBMS durability update (added versioning and salient information)
* Automatic migration from previous version
* Interpreter can run some queries without a target database
* SHOW REPLICA returns the status of the currently active DB
* Returning UUID instead of db name in the RPC responses
* Using UUIDs for database specification in RPC (not name)
* FrequentCheck forces update on reconnect
* TimestampRpc will detect if a replica is behind, and will update client's state
* Safer SLK reads
* Split SHOW DATABASES in two SHOW DATABASES (list of current databases) and SHOW DATABASE a single string naming the current database

---------

Co-authored-by: Gareth Lloyd <gareth.lloyd@memgraph.io>
2024-01-23 12:06:10 +01:00
Gareth Andrew Lloyd
7f10636470
Bugfix don't use _Py_IsFinalizing (#1657)
This is an unstable function and would bump our dependency to python 3.7
2024-01-22 16:14:41 +00:00
andrejtonev
9c89fce249
Bugfix: Shutdown blocks due to wrong execution order (#1649)
* Bugfix: Destorying settings before stopping license checker
* Bugfix: Python GC running while shutting down
2024-01-19 17:05:47 +00:00
Ivan Milinović
23dff58d22
Improve memory tracking (#1631) 2024-01-14 11:14:46 +01:00
Aidar Samerkhanov
c772cab766
ToString function now returns double values with precision 15 (#1576)
The DoubleToString function has been updated to handle higher precision doubles correctly. The unnecessary string length restriction has been removed, allowing the function to convert the full double value without prematurely truncating it. This change ensures that the string representation of doubles is more accurate, especially for very large or very small numbers. Unit tests have been added to verify the correct behavior for a range of double values.
2024-01-12 12:32:34 +04:00
Aidar Samerkhanov
2e4d27c59a
Add List Pattern Comprehension grammar. (#1588) 2024-01-11 18:20:21 +04:00
DavIvek
31f15b3651
Fix index hints (#1606) 2024-01-11 10:10:06 +01:00
DavIvek
b3d0c2ccc2
Add query parameters support for labels (#1602) 2024-01-10 15:08:21 +01:00
DavIvek
d4bcdb77ad
Fix using path identifier after CREATE (#1629) 2024-01-10 11:46:20 +01:00
DavIvek
4bb9238679
Add support for triggers in database dump (#1610) 2024-01-09 13:05:54 +01:00
DavIvek
bd11266f82
Extend ToBoolean function (#1620) 2024-01-08 13:17:55 +01:00