Commit Graph

97 Commits

Author SHA1 Message Date
Gareth Andrew Lloyd
8bc8e867e4
Pmr allocator unify (#1801)
Query allocator and evaluation allocator were different.
After analysis, was determined they should be the same, this will help 
future development reduce TypeValue copies during queries.

Changes:
- Common allocator, PoolResource backed by MonotonicResource
- Optimized Pool, now O(1) alloc/dealloc as all chunks in Pool form a single 
  free list
- 2nd PoolResource, using bin sizing, not as perfect for memory usage but 
  O(1) bin selection
- Now have jemalloc's background thread to make sure decay and return 
  to OS happens
- Optimized ProperyValue to be faster at destruction/copy/move
- Less temporary memory allocations
  - CSV reader now maintains a common line buffer it reuses on line reads
  - Writing out bolt values, now reuses a values buffer
  - Evaluating an int no longer makes temporary strings for errors it most 
    likely never throws
  - ExpandVariable will reuse existing edge list in frame it one existed
2024-03-14 11:21:59 -07:00
Marko Barišić
b0cdcd3483
Run CI in mgbuilder containers (#1749)
* Update deployment files for mgbuilders because of toolchain upgrade
* Fix args parameter in builder yaml files
* Add fedora 38, 39 and rockylinux 9.3 mgbuilder Dockerfiles
* Change format of ARG TOOLCHAIN_VERSION from toolchain-vX to vX
* Add function to check supported arch, build type, os and toolchain
* Add options to init subcommand
* Add image names to mgbuilders
* Add v2 of the run.sh script
* Add testing to run2.sh
* Add option for threads --thread
* Add options for enterprise license and organization name
* Make stop mgbuild container step run always
* Add --ci flag to init script
* Move init conditionals under build-memgraph flags
* Add --community flag to build-memgraph
* Change target dir inside mgbuild container
* Add node fix to debian 11, ubuntu 20.04 and ubuntu 22.04
* rm memgraph repo after installing deps
* Add mg user in Dockerfile
* Add step to install rust on all OSs
* Chown files copied into mgbuild container
* Add e2e tests
* Add jepsen test
* Bugfix: Using reference in a callback
* Bugfix: Broad target for e2e tests
* Up db info test limit
* Disable e2e streams tests
* Fix default THREADS
* Prioretize docker compose over docker-compose
* Improve selection between docker compose and docker-compose
* Install PyYAML as mg user
* Fix doxygen install for rocky linux 9.3
* Fix rocky-9.3 environment script to properly install sbcl
* Rename all rocky-9 mentions to rocky-9.3
* Add mgdeps-cache and benchgraph-api hostnames to mgbuild images
* Add logic to pull mgbuild image if missing
* Fix build errors on toolchain-v5 (#1806)
* Rename run2 script, remove run script, add small features to mgbuild.sh
* Add --no-copy flag to build-memgraph to resolve TODO
* Add timeouts to diff jobs
* Fix asio flaky clone, try mgdeps-cache first

---------

Co-authored-by: Andreja Tonev <andreja.tonev@memgraph.io>
Co-authored-by: Ante Pušić <ante.f.pusic@gmail.com>
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
2024-03-14 12:19:59 +01:00
Marko Budiselić
61b9bb0f59
Add toolchain-v5 compatibility Revert to C++20 (#587)
* Upgrade cppitertools, spdlog, fmt, rapidcheck
* Make compilation work on both v4 and v5 toolchains
2024-02-19 21:09:54 +01:00
Marko Budiselić
f098a9d5e3
Patch NuRaft for clang-17 compilation (#1733) 2024-02-19 14:50:37 +01:00
Marko Barišić
dcdbd0a19a
Fix primary urls (#1700) 2024-02-08 14:19:30 +01:00
Gareth Andrew Lloyd
4ef6a1f9c3
Improve memory handling of Deltas (#1688)
- Reduce delta from 104B to 80B
- Hold and pass them around as in a deque
- Detect and deleted deltas within commit if safe to do so
2024-02-06 18:07:38 +01:00
Andi
6ab4235cc9
Add NuRaft library (#1678) 2024-01-31 13:13:59 +01:00
Ante Pušić
71e76cc980
Fix unresolved host errors by switching to using jemalloc’s primary URL (#1579) 2023-12-18 23:31:00 +01:00
Josipmrden
e617ff9b59
Provide textual information for inefficient plans with notifications (#1343) 2023-10-24 22:20:05 +02:00
Antonio Filipovic
7f7f3adfcb
Implement jemalloc extent hooks memory tracker (#1250)
Should improve/fix memory usage exceeds --memory-limit issues
2023-10-23 12:48:26 +02:00
Aidar Samerkhanov
3bf2cf65ab
Optimize splitting keys inside the on-disk storage (#1155) 2023-08-17 18:09:21 +02:00
Gareth Andrew Lloyd
2e51e703c3
Add supernode vertex cache (#1124)
Add supernode vertex cache to account for long delta chains and modifications in the same module being independent of scanning of the nodes in the next iteration of the pulling mechanism.
2023-08-11 10:18:28 +02:00
Marko Budiselić
e5350a011c
Upgrade to mgconsole v1.4.0 (#1144) 2023-08-05 15:52:31 +02:00
Marko Budiselić
f0aca2d23b
Improve init script logging (#1096) 2023-07-22 13:08:39 +02:00
Gareth Andrew Lloyd
3b781bf525
Add HTTP+GZIP support to LOAD CSV (#1027) 2023-06-26 19:10:48 +02:00
Marko Budiselić
7fb3f62703
Upgrade to RocksDB 8.1.1 (#1013) 2023-06-15 11:54:24 +02:00
Ante Javor
940bf6722c
Add mgbench tutorial (#836)
* Add Docker runner
* Add Docker client
* Add benchgraph.sh script
* Add package script
2023-04-19 08:21:55 +02:00
Jure Bajic
6e4047a847
Bump mgconsole version (#660) 2022-12-01 13:10:08 +01:00
Marko Budiselić
80e0e439b7
Add parser stress test (#463) 2022-07-26 20:54:56 +02:00
Marko Budiselić
74d3663821
Upgrade Antrl to 4.10.1 and remove antlr_lock (#441) 2022-07-26 08:31:38 +02:00
János Benjamin Antal
693dab78d2
Remove unused URLs from libs/setup.sh (#352) 2022-02-11 13:56:43 +01:00
Antonio Andelic
69eca9b043
Procedures for handling modules (#330) 2022-02-11 11:29:41 +01:00
Antonio Andelic
265b203b00
Build for ARM64 (#340) 2022-02-03 13:03:35 +01:00
Antonio Andelic
ced84e17b6
Update nlohmann and remove guards (#336) 2022-01-26 08:12:09 +01:00
Antonio Andelic
bb1308acc7
Use libs from toolchain (#326) 2022-01-21 10:22:36 +01:00
János Benjamin Antal
e1f31d3d02
Toolchain v4 with fbthrift (#318) 2022-01-18 12:51:27 +01:00
Antonio Andelic
f3a3a4ed87
Unsafe git protocol fix (#324) 2022-01-12 11:36:06 +01:00
Antonio Andelic
8716b8e992
Update to newest spdlog and fmt (#309) 2021-12-07 11:05:33 +01:00
Antonio Andelic
e8976e0f1c Update mgp API for Pulsar (#298) 2021-11-16 16:00:25 +01:00
Antonio Andelic
b66cc66503 Add support for Pulsar queries (#292) 2021-11-16 15:52:58 +01:00
Antonio Andelic
0e4719018a Introduce pulsar dependency (#286) 2021-11-16 15:47:40 +01:00
Antonio Andelic
f868b51483
Update mgconsole version (#260) 2021-10-08 11:19:21 +02:00
Kostas Kyrimis
18b801a722 Add e2e tests for temporal types (#241) 2021-10-02 15:31:00 +02:00
antonio2368
e51954fc94
Update toolchain to v3 (#189)
* Make memgraph buildable with new toolchain

* Use toolchain v3 in workflows
2021-07-08 14:20:48 +02:00
János Benjamin Antal
7776160836
Add third party licenses (#195)
* Add third party licences

* Package the third-party licences
2021-07-08 08:55:09 +02:00
János Benjamin Antal
36afc6c5f3 Add Kafka consumer (#167)
* Add CMake config for Kafka integration

* Add Consumer

* Add simple unit test for consumer

* Add explicit offset handling and test for it
2021-07-07 15:38:51 +02:00
Josip Seljan
cd03e13443
Upgrade Antlr to v4.9.2 (#161) 2021-06-09 12:50:27 +02:00
Josip Seljan
2a0b0d969f
Replace mg_client with mgconsole (#153) 2021-05-26 11:59:36 +02:00
antonio2368
999b3ef79f
Fix memory tracking issues (#150) 2021-05-18 18:42:19 +02:00
Marko Budiselić
1def0c9104
Fix libs/setup.sh nocached download (#148) 2021-05-13 12:05:36 +02:00
Marko Budiselić
59105f68bd
Add primary and secondary dependency links (#137)
* Add explicit exit in case of git checkout or cherry-pick
2021-04-20 10:54:54 +02:00
antonio2368
e8810a4152 Add and use jemalloc lib (#90)
* Add and use jemalloc lib

* Add autoconf

* Silence macro redefinition warning
2021-03-24 14:53:15 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72)
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
Marko Budiselić
afbf672915 Add end2end replication tests based on mgclient (#69)
* Remove old HA benchmark and integration tests
2021-01-21 15:56:21 +01:00
antonio2368
4e7c569071 Migrate code to C++20 (#44) 2020-11-17 11:03:08 +01:00
Marko Budiselić
958bc870b3
Migrate to toolchain-v2 (#33)
* GCC_VERSION=10.2.0
* BINUTILS_VERSION=2.35.1
* GDB_VERSION=10.1 (except centos-7 8.3)
* CMAKE_VERSION=3.18.4
* CPPCHECK_VERSION=2.2
* LLVM_VERSION=11.0.0
* SWIG_VERSION=4.0.2
2020-11-12 20:18:11 +01:00
Marko Budiselić
6fab357430
Upgrade cppitertools to v2.0 (#17) 2020-10-27 12:14:18 +01:00
Matej Ferencevic
c12e4a49b1
Migrate deps to S3 (#3) 2020-09-22 19:52:26 +02:00
Matej Ferencevic
e9ea8693fa Fix CentOS compilation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2755
2020-04-06 16:01:35 +02:00
Matej Ferencevic
87511460f1 Remove unused CMake options
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2656
2020-02-05 14:52:22 +01:00