Commit Graph

49 Commits

Author SHA1 Message Date
Marko Budiselic
888c6a4bca Add support for the id function
Reviewers: dgleich, teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: dgleich, mferencevic, teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1462
2018-07-04 22:14:13 +02:00
Marko Budiselic
a7d50992db Fix documentation and config issues for v0.11.0
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1449
2018-06-22 23:48:05 +02:00
Marko Budiselic
d320af1af2 Update community config with properties-on-disk placeholder
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1446
2018-06-21 21:03:45 +02:00
Matej Ferencevic
1d448d40ca Implement SSL support for servers and clients
Summary:
This diff implements OpenSSL support in the network stack.
Currently SSL support is only enabled for Bolt connections,
support for RPC connections will be added in another diff.

Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1328
2018-06-20 17:56:47 +02:00
Matej Ferencevic
44821a918c Initial implementation of telemetry
Summary:
Add telemetry to main memgraph binary

Add resource usage collector

Add telemetry flag

Change telemetry collector logic

Fix utils compilation

Add timestamp

Add first version of interactive test

Started working on test runner

Implement all tests

Flake8 on runner.py

Integrate test with Apollo

Add TODO

Reviewers: buda, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1419
2018-06-20 14:49:07 +02:00
Marko Budiselic
44474b55e3 Add durability stress test
Reviewers: dgleich, mferencevic

Reviewed By: dgleich, mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1371
2018-04-22 08:32:44 +02:00
Dominik Gleich
0eb30db8db Remove executor not run option
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1062
2017-12-18 14:36:15 +01:00
florijan
8bbf1af525 Cleanup durability config, docs, CHANGELOG
Reviewers: teon.banek, buda, mislav.bradac, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D992
2017-11-21 10:17:13 +01:00
florijan
1e0ac8ab8f Write-ahead log
Summary:
My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed...

What's done:
- Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr.
- WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk.
- Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info.
- Durability testing completely reworked.

What's not done (and should be when we decide how):
- Old WAL file purging.
- Config refactor (naming and organization). Will do when we discuss what we want.
- Changelog and new feature documentation (both depending on the point above).
- Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment).
- Moving the implementation of WAL stuff to .cpp where possible.
- Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into.
- True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature.

Reviewers: buda, mislav.bradac, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D958
2017-11-13 09:51:39 +01:00
Matej Ferencevic
cbf8dacc11 Add parallel macro benchmark suite
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D922
2017-10-24 13:53:09 +02:00
Teon Banek
8bf0a9cf92 Fill the alpha.conf with all settings and document them
Summary:
This should be the basis for default configuration that is to be shipped
in release builds/packages.

Reviewers: mislav.bradac, florijan, buda, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D851
2017-10-06 15:15:19 +02:00
Matej Ferencevic
e6c3d42e02 Changed long running stress parameters.
Summary:
Increased query execution timeout.

Enabled global queries by default.

Implemented faster RandomElement for vertices and edges.

Changed long running verify message format.

Changed vertex and edge count to be per worker.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D864
2017-10-05 22:10:39 +02:00
Teon Banek
eb821aa615 Remove ast-cache flag
Summary:
AST caching should be well tested by now.
We should consider removing `Context.is_query_cached_` member as well as the
implementation and tests for `CypherMainVisitor`.

Reviewers: mislav.bradac, mferencevic, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D833
2017-09-27 09:04:31 +02:00
Teon Banek
9d4ae833cc Setup basic installation for CPackDeb
Summary:
Remove traces of plan_compiler in cmake.
Add memgraph executable to install target.
Fix indentation in tests/CMakeLists.txt.
Install alpha.conf to /etc/memgraph/config.
Rename executable to 'memgraph' and install /var dirs.

Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D829
2017-09-26 14:54:13 +02:00
florijan
ff37c04b38 Snapshot flags name consistency
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D821
2017-09-22 14:08:51 +02:00
Matej Ferencevic
686dc452ee LDBC: Integrate with Apollo
Summary:
Run neo4j and memgraph from run_benchmark script.
This makes mg and neo scripts obsolete.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D806
2017-09-21 22:29:42 +02:00
Mislav Bradac
7e99e93e47 Start work on parallel benchmark
Summary: First version of our benchmark

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D740
2017-09-12 16:58:21 +02:00
Mislav Bradac
d40319c111 Use 1 worker in benchmarks
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D722
2017-08-29 12:35:34 +02:00
Marko Budiselic
a8e0792609 LDBC
Summary: Add LDBC helper scripts.

Reviewers: teon.banek, mferencevic, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D563
2017-08-28 09:34:22 +02:00
Matej Ferencevic
0914c5a941 Migrate harness to use our bolt client
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D711
2017-08-25 11:10:15 +02:00
Matej Ferencevic
2ba3df942b Added continuous_integration script to stress tests.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D620
2017-08-02 12:06:59 +02:00
Mislav Bradac
06d3629fd0 Migrate to new jail faker
Reviewers: mferencevic

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D599
2017-07-29 18:06:44 +02:00
Mislav Bradac
68e78c417b NeoRunner + Harness bugfix
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D588
2017-07-29 17:46:24 +02:00
Mislav Bradac
3119ae5343 Supstitute named expressions in ast cache
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D510
2017-06-26 16:33:17 +02:00
florijan
e28fc6ed41 FLAG 'snapshot_on_destruction' renamed to 'snapshot_on_exit'
Reviewers: teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D471
2017-06-16 16:25:26 +02:00
Marko Budiselic
13e7988c5f Testing config
Summary: Testing config update. AST cache false.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: buda

Differential Revision: https://phabricator.memgraph.io/D487
2017-06-16 15:40:32 +02:00
Marko Budiselic
9ed4102897 Release preparation.
Summary: Alpha config fix. Docker volume support.

Reviewers: teon.banek, dgleich

Reviewed By: teon.banek, dgleich

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D484
2017-06-16 15:16:11 +02:00
Marko Budiselic
c081281944 Docker mount volume documentation.
Summary: Add snapshots mount documentation. By default, build_interpreter takes alpha.conf as a config file.

Reviewers: teon.banek, dtomicevic

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D464
2017-06-14 10:15:05 +02:00
Dominik Gleich
9f4e6243d8 Fix.
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D448
2017-06-09 11:40:06 +02:00
Marko Budiselic
f84f9af244 Alpha package scripts.
Summary: Alpha package scripts. Alpha version is going to be shipped within docker.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D436
2017-06-08 14:15:55 +02:00
Dominik Gleich
65507da9eb Load configs from flagfiles.
Summary: Migrate configto gflags.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D433
2017-06-07 15:38:17 +02:00
matej.gradicek
15b57e2d52 Implemented recovery and tests.
Summary: Implemented durability recovery and tesats.

Reviewers: mislav.bradac, dgleich, buda

Reviewed By: mislav.bradac, dgleich, buda

Subscribers: dtomicevic, mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D374
2017-06-06 15:12:34 +00:00
matej.gradicek
e791e9b660 Summary:Added files for snapshot durability
Summary:
File buffer added

Implemented little more of snapshoter.

Resolved conflicts

More stuff implemented of snapshot durability.

More things in snapshoter.

Refactored, added comments

Merge branch 'dev' into durability_snapshot

Merge branch 'dev' into durability_snapshot

Resolved bug in scheduler, snapshoter is running in grpah_db.

Reviewers: mferencevic, buda, dgleich, mislav.bradac

Reviewed By: mferencevic, buda, dgleich, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D232
2017-05-17 08:42:00 +00:00
Marko Budiselic
38c40096c5 Interpreter build script + dockerfile.
Summary: Interpreter build script + dockerfile.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D275
2017-04-13 14:58:14 +02:00
Mislav Bradac
a58e269176 Fix bugs needed to run qa tests
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D216
2017-04-04 17:28:08 +02:00
Dominik Gleich
bd0b81526e Initial version of mvcc_gc.
Summary:
Tested MVCC garbage collector.
Also refactors graph_db forward declarations of vertex and edge which were causing issues.

Reviewers: mislav.bradac, dtomicevic, florijan, teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D177
2017-03-29 12:38:24 +02:00
Marko Budiselic
3b42cd2579 Everything is merged (communication + query engine) but we have to fix the PULL_ALL issue. 2017-03-22 17:57:06 +01:00
Marko Budiselic
3642fb1312 examples folder has been removed; src/test folder has been removed; the existing tests were moved to tests folder; StacktraceException has been introduced; query_plan_templated has been moved to query folder; hazard pointers have been deleted because they are not used any more 2017-02-18 18:03:48 +01:00
Marko Budiselic
0fcda94162 Hardcoded query infrastructure - first concrete version - USEFUL FOR: POCs & pilots
Summary: Hardcoded query infrastructure - first concrete version - USEFUL FOR: POCs & pilots

Test Plan: manual + jenkins

Reviewers: sale, florijan

Reviewed By: florijan

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D45
2017-02-14 09:40:31 +01:00
Marko Budiselic
0a0e5e93da src/demo was removed, src/examples and src/benchmarks were moved to the top level 2016-12-04 10:20:14 +01:00
Marko Budiselic
21788d003a Dressipi CRUD queries are dummy implemented; Fixes T99 and T131
Summary: Dressipi CRUD queries are dummy implemented; Fixes T99 and T131

Test Plan: manual

Reviewers: sale

Subscribers: buda, sale

Maniphest Tasks: T131, T99

Differential Revision: https://memgraph.phacility.com/D9
2016-11-29 04:08:49 +01:00
Marko Budiselic
f5cf9c72d2 skiplist benchmark 2016-10-04 15:09:21 +02:00
Kruno Tomola Fabro
fd202b1ed7 Finished stack allocator. 2016-09-15 20:19:31 +01:00
Kruno Tomola Fabro
efbae39f41 Added logic for cleaning older snapshots. 2016-09-11 22:22:52 +01:00
Kruno Tomola Fabro
4443cb8b7b Finished creating part of snapshot. 2016-09-08 13:25:52 +01:00
Marko Budiselic
8e5dbd4703 config is better now (yaml-cpp) + web::Client + web::Logger 2016-09-05 01:39:30 +01:00
Marko Budiselic
52625e39a3 The folder structure was changed, all source code is now within the src dir. 2016-05-08 11:33:07 +02:00
Marko Budiselic
6eac9a5ac5 first proposal of compile process 2016-02-10 23:34:49 +01:00
Marko Budiselic
7d8c5a7f7e query engine can now generate, compile and run dummy code, everything is ready to generate code for dummy the query: CREATE (n {...}) RETURN n 2016-02-07 22:56:52 +01:00