Commit Graph

90 Commits

Author SHA1 Message Date
Teon Banek
c0cd9bd22b Use correct snapshot directory in mg_import_csv
Reviewers: florijan, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1025
2017-12-05 13:18:17 +01:00
Dominik Gleich
3ddbcad0d9 Refactor global ids and prepare for distributed
Summary:
Change ids to global ids

Fix tests

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1019
2017-12-05 13:05:55 +01:00
Matej Ferencevic
3c6306985d Link macro benchmark runs together
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1024
2017-12-05 10:06:05 +01:00
Teon Banek
a17261038c Include tools CMakeLists in the top level CMakeLists
Summary:
Split main CMakeLists into src/CMakeLists

The main CMakeLists duty is to make all the required libraries and
variables visible to all of the other sub-CMakeLists. After doing that,
it should include those sub-CMakeLists according to configuration
options.

This should make global configurations easier to reuse without polluting
the global space with locally related configurations. It is a necessary
step for including other projects like 'tools' in the release
installation.

Building tools is automatically disabled, but can be enabled by setting
the TOOLS option to ON when running cmake. This should allow on demand
building as well as combined installation of Memgraph and its tools.

Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1018
2017-12-04 15:18:18 +01:00
florijan
e26456d5ad Fix durability test flakyness
Summary:
It occurred that part of the durability flakyness test might be that the
same durability directory is used always. If the test is run
simultaneously on a single system, there will be interference.

This might not actually fix all the flakyness :(

I also made the `utils::RandomString` function since that's now used in
multiple places, tested it etc.

Reviewers: buda, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1020
2017-12-04 13:46:56 +01:00
florijan
ce3638b25e Prepare transactional engine for distributed
Summary:
The current idea is that the same MG binary can be used for single-node,
distributed master and distributed worker. The transactional engine in
the single-node and distributed master is the same: it determines the
transactional time and exposes all the "global" functionalities. In the
distributed worker the "global" functions must contact the master.

Reviewers: dgleich, mislav.bradac, buda

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1013
2017-12-01 09:17:44 +01:00
Dominik Gleich
73c1206e81 Remove extension from Python executables
Reviewers: mferencevic, buda

Reviewed By: buda

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D988
2017-11-27 14:39:51 +01:00
Teon Banek
5a41478789 Add maintainer scripts for DEB package
Summary:
Add postinst script for DEB package

The script creates a 'memgraph' group and sets permission on installed
'/var/*/memgraph' directories. Only the group is created, while
'memgraph' user is not. It seems more sane only to require group
membership for using memgraph.

Add conffiles for DEB package

This allows for `dpkg` to detect changes in configuration files and
present them to the user. Therefore, we don't need to care whether the
configuration merges are handled correctly nor if we accidentally
overwrite them.

Add postrm script for DEB packaging

The script is only used so that `dpkg --purge` removes '/var/*/memgraph'
directories, even if they contain something.

Add email, longer description and license file to DEB packaging, as well
as a systemd service.

Provide a logrotate configuration and support it in memgraph.

Use DEB package for Docker installation

This way, the whole installation process and testing should go through
DEB.

Generate release archives in Apollo with standard names

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D989
2017-11-22 16:41:25 +01:00
florijan
8bbf1af525 Cleanup durability config, docs, CHANGELOG
Reviewers: teon.banek, buda, mislav.bradac, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D992
2017-11-21 10:17:13 +01:00
Dominik Gleich
f16246604f Add recovery speed measurement tool
Reviewers: mferencevic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D985
2017-11-16 13:29:06 +01:00
florijan
b7188c296c Fix CSV multiple edge file import
Summary:
The CSV importer used to generate non-unique edge IDs when
generating edges from multiple CSV files. This is incompatible with the
unique ID requirements introduced by the WAL. Tested and fixed in this
diff.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D980
2017-11-16 09:18:38 +01:00
florijan
885d19401f Move WAL implementation to .cpp
Summary: No implementation changes, just code migration, rebase, import cleanup.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D983
2017-11-16 09:12:34 +01:00
Dominik Gleich
85ef12def5 Add memory usage tool
Summary:
Tool measures memory usage per one vertex, per one edge,
and per one vertex by adding a new label to it.

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D973
2017-11-13 13:35:57 +01:00
florijan
1e0ac8ab8f Write-ahead log
Summary:
My dear fellow Memgraphians. It's friday afternoon, and I am as ready to pop as WAL is to get reviewed...

What's done:
- Vertices and Edges have global IDs, stored in `VersionList`. Main storage is now a concurrent map ID->vlist_ptr.
- WriteAheadLog class added. It's based around buffering WAL::Op objects (elementraly DB changes) and periodically serializing and flusing them to disk.
- Snapshot recovery refactored, WAL recovery added. Snapshot format changed again to include necessary info.
- Durability testing completely reworked.

What's not done (and should be when we decide how):
- Old WAL file purging.
- Config refactor (naming and organization). Will do when we discuss what we want.
- Changelog and new feature documentation (both depending on the point above).
- Better error handling and recovery feedback. Currently it's all returning bools, which is not fine-grained enough (neither for errors nor partial successes, also EOF is reported as a failure at the moment).
- Moving the implementation of WAL stuff to .cpp where possible.
- Not sure if there are transactions being created outside of `GraphDbAccessor` and it's `BuildIndex`. Need to look into.
- True write-ahead logic (flag controlled): not committing a DB transaction if the WAL has not flushed it's data. We can discuss the gain/effort ratio for this feature.

Reviewers: buda, mislav.bradac, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D958
2017-11-13 09:51:39 +01:00
Dominik Gleich
1d5245cb13 Add edge creation and bfs parallel tests
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D953
2017-11-10 16:03:06 +01:00
Dominik Gleich
f50d87f736 Take care of warnings/errors created by cppcheck.
Summary:
Warnings I ignored:

Creates a new stacktrace object and then dumps it
	102.570384 (102.495075) E[1]: [src/utils/exceptions.hpp:116]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.
	102.570390 (102.495081) E[1]: [src/utils/exceptions.hpp:127]: (performance) Variable 'stacktrace_' is assigned in constructor body. Consider performing initialization in initialization list.

Used all over the codebase without explicit cast
102.570412 (102.495103) E[1]: [src/utils/stacktrace.hpp:14]: (style) Class 'Line' has a constructor with 1 argument that is not explicit.

Not really used anywhere before initialized:
102.570526 (102.495217) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::preds' is not initialized in the constructor.
102.570530 (102.495221) E[1]: [src/data_structures/concurrent/skiplist.hpp:467]: (warning) Member variable 'Accessor::succs' is not initialized in the constructor.

Implicit conversions between types are used all over the codebase:
102.570548 (102.495239) E[1]: [src/storage/property_value.hpp:41]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570552 (102.495243) E[1]: [src/storage/property_value.hpp:42]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570557 (102.495248) E[1]: [src/storage/property_value.hpp:43]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570561 (102.495252) E[1]: [src/storage/property_value.hpp:44]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570566 (102.495257) E[1]: [src/storage/property_value.hpp:47]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570570 (102.495261) E[1]: [src/storage/property_value.hpp:50]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570578 (102.495269) E[1]: [src/storage/property_value.hpp:53]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570582 (102.495273) E[1]: [src/storage/property_value.hpp:57]: (style) Class 'PropertyValue' has a constructor with 1 argument that is not explicit.
102.570591 (102.495282) E[1]: [src/query/typed_value.hpp:80]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570596 (102.495287) E[1]: [src/query/typed_value.hpp:81]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570601 (102.495292) E[1]: [src/query/typed_value.hpp:82]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570605 (102.495296) E[1]: [src/query/typed_value.hpp:83]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570609 (102.495300) E[1]: [src/query/typed_value.hpp:89]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570614 (102.495305) E[1]: [src/query/typed_value.hpp:92]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570618 (102.495309) E[1]: [src/query/typed_value.hpp:95]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570623 (102.495314) E[1]: [src/query/typed_value.hpp:98]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570627 (102.495318) E[1]: [src/query/typed_value.hpp:102]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570632 (102.495323) E[1]: [src/query/typed_value.hpp:105]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570636 (102.495327) E[1]: [src/query/typed_value.hpp:108]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570641 (102.495332) E[1]: [src/query/typed_value.hpp:109]: (style) Class 'TypedValue' has a constructor with 1 argument that is not explicit.
102.570645 (102.495336) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:88]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570650 (102.495341) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:89]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570654 (102.495345) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:90]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570659 (102.495350) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:91]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570663 (102.495354) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:94]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570668 (102.495359) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:97]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570672 (102.495363) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:100]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570677 (102.495368) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:104]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570681 (102.495372) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:107]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570690 (102.495381) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:110]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.
102.570694 (102.495385) E[1]: [src/communication/bolt/v1/decoder/decoded_value.hpp:113]: (style) Class 'DecodedValue' has a constructor with 1 argument that is not explicit.

CypherParser:

102.570767 (102.495458) E[1]: [src/query/frontend/opencypher/generated/CypherParser.h:69]: (style) Class 'CypherParser' has a constructor with 1 argument that is not explicit.
102.570772 (102.495463) E[1]: [src/query/frontend/opencypher/generated/CypherLexer.h:40]: (style) Class 'CypherLexer' has a constructor with 1 argument that is not explicit.
102.570776 (102.495467) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:86]: (style) The scope of the variable '_la' can be reduced.
102.570781 (102.495472) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:311]: (style) The scope of the variable '_la' can be reduced.
102.570785 (102.495476) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:402]: (style) The scope of the variable '_la' can be reduced.
102.570789 (102.495480) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:497]: (style) The scope of the variable '_la' can be reduced.
102.570797 (102.495488) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:778]: (style) The scope of the variable '_la' can be reduced.
102.570802 (102.495493) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:895]: (style) The scope of the variable '_la' can be reduced.
102.570806 (102.495497) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:991]: (style) The scope of the variable '_la' can be reduced.
102.570811 (102.495502) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1190]: (style) The scope of the variable '_la' can be reduced.
102.570815 (102.495506) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1274]: (style) The scope of the variable '_la' can be reduced.
102.570820 (102.495511) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1393]: (style) The scope of the variable '_la' can be reduced.
102.570824 (102.495515) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1570]: (style) The scope of the variable '_la' can be reduced.
102.570829 (102.495520) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1695]: (style) The scope of the variable '_la' can be reduced.
102.570834 (102.495525) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1800]: (style) The scope of the variable '_la' can be reduced.
102.570839 (102.495530) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:1903]: (style) The scope of the variable '_la' can be reduced.
102.570843 (102.495534) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2019]: (style) The scope of the variable '_la' can be reduced.
102.570848 (102.495539) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2228]: (style) The scope of the variable '_la' can be reduced.
102.570852 (102.495543) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2542]: (style) The scope of the variable '_la' can be reduced.
102.570857 (102.495548) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2797]: (style) The scope of the variable '_la' can be reduced.
102.570861 (102.495552) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:2966]: (style) The scope of the variable '_la' can be reduced.
102.570866 (102.495557) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3067]: (style) The scope of the variable '_la' can be reduced.
102.570870 (102.495561) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3289]: (style) The scope of the variable '_la' can be reduced.
102.570875 (102.495566) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3295]: (style) The scope of the variable 'alt' can be reduced.
102.570879 (102.495570) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3419]: (style) The scope of the variable '_la' can be reduced.
102.570884 (102.495575) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3596]: (style) The scope of the variable '_la' can be reduced.
102.570888 (102.495579) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3688]: (style) The scope of the variable '_la' can be reduced.
102.570893 (102.495584) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:3963]: (style) The scope of the variable '_la' can be reduced.
102.570897 (102.495588) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4452]: (style) The scope of the variable '_la' can be reduced.
102.570902 (102.495593) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4586]: (style) The scope of the variable '_la' can be reduced.
102.570906 (102.495597) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4813]: (style) The scope of the variable '_la' can be reduced.
102.570911 (102.495602) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:4943]: (style) The scope of the variable '_la' can be reduced.
102.570918 (102.495609) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5026]: (style) The scope of the variable '_la' can be reduced.
102.570923 (102.495614) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5569]: (style) The scope of the variable '_la' can be reduced.
102.570928 (102.495619) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5664]: (style) The scope of the variable '_la' can be reduced.
102.570932 (102.495623) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5755]: (style) The scope of the variable '_la' can be reduced.
102.570937 (102.495628) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:5888]: (style) The scope of the variable '_la' can be reduced.
102.570941 (102.495632) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6045]: (style) The scope of the variable '_la' can be reduced.
102.570946 (102.495637) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6142]: (style) The scope of the variable '_la' can be reduced.
102.570950 (102.495641) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6347]: (style) The scope of the variable '_la' can be reduced.
102.570955 (102.495646) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6523]: (style) The scope of the variable '_la' can be reduced.
102.570959 (102.495650) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6614]: (style) The scope of the variable '_la' can be reduced.
102.570964 (102.495655) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6899]: (style) The scope of the variable '_la' can be reduced.
102.570968 (102.495659) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:6992]: (style) The scope of the variable '_la' can be reduced.
102.570973 (102.495664) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7147]: (style) The scope of the variable '_la' can be reduced.
102.570977 (102.495668) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7680]: (style) The scope of the variable '_la' can be reduced.
102.570982 (102.495673) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7759]: (style) The scope of the variable '_la' can be reduced.
102.570986 (102.495677) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:7938]: (style) The scope of the variable '_la' can be reduced.
102.570991 (102.495682) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8126]: (style) The scope of the variable '_la' can be reduced.
102.570995 (102.495686) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8220]: (style) The scope of the variable '_la' can be reduced.
102.571000 (102.495691) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8313]: (style) The scope of the variable '_la' can be reduced.
102.571004 (102.495695) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8491]: (style) The scope of the variable '_la' can be reduced.
102.571009 (102.495700) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8703]: (style) The scope of the variable '_la' can be reduced.
102.571013 (102.495704) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8783]: (style) The scope of the variable '_la' can be reduced.
102.571018 (102.495709) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:8914]: (style) The scope of the variable '_la' can be reduced.
102.571022 (102.495713) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9119]: (style) The scope of the variable '_la' can be reduced.
102.571027 (102.495718) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9220]: (style) The scope of the variable '_la' can be reduced.
102.571034 (102.495725) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9414]: (style) The scope of the variable '_la' can be reduced.
102.571039 (102.495730) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:9660]: (style) The scope of the variable '_la' can be reduced.
102.571043 (102.495734) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10008]: (style) The scope of the variable '_la' can be reduced.
102.571048 (102.495739) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10158]: (style) The scope of the variable '_la' can be reduced.
102.571052 (102.495743) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10250]: (style) The scope of the variable '_la' can be reduced.
102.571057 (102.495748) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10370]: (style) The scope of the variable '_la' can be reduced.
102.571061 (102.495752) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10637]: (style) The scope of the variable '_la' can be reduced.
102.571065 (102.495756) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10749]: (style) The scope of the variable '_la' can be reduced.
102.571070 (102.495761) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10815]: (style) The scope of the variable '_la' can be reduced.
102.571075 (102.495766) E[1]: [src/query/frontend/opencypher/generated/CypherParser.cpp:10881]: (style) The scope of the variable '_la' can be reduced.

We know that we represented it correctly in memory:
102.571079 (102.495770) E[1]: [src/communication/bolt/v1/decoder/decoder.hpp:252]: (portability) Casting between integer* and double* which have an incompatible binary data representation.

Cont assigned but not used after:
102.571101 (102.495792) E[1]: [src/query/frontend/ast/ast.hpp:1008]: (style) Variable 'cont' is assigned a value that is never used.

Reviewers: teon.banek, buda, mferencevic

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D967
2017-11-10 10:28:02 +01:00
Matej Ferencevic
6720ad6aaf Fixed macro benchmark summary indentation.
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D965
2017-11-07 10:54:15 +01:00
Matej Ferencevic
f2ebd935b9 Added more cppcheck suppressions.
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D959
2017-11-06 16:04:07 +01:00
Matej Ferencevic
8ed95a2b33 Don't run benchmark tests on diffs.
Reviewers: buda, dgleich

Reviewed By: buda, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D945
2017-10-31 11:05:58 +01:00
Teon Banek
55456b4214 Remove Dbms
Summary:
Remove name from GraphDb.
Take GraphDb in query test macros instead of accessor.
Add is_accepting_transactions flag to GraphDb.

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D940
2017-10-30 12:33:29 +01:00
florijan
be9c875fa9 Durability - snapshot summary refactor
Summary:
- Removed durability::Summary because it was wired into reader and stopped me from recovering WAL files.
- Refactored and renamed BufferedFile(Reader/Writer) to HashedFile(Reader/Writer).
- Vertex and edge counts in the snapshot are now hashed.

Breaking snapshot compatibility again (hashing), but since the previous version was not released, and we are not caching snapshots, the previous version does not need to be supported.

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek, mislav.bradac

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D932
2017-10-27 13:51:02 +02:00
Teon Banek
df72861b90 Make csv_to_snapshot more user friendly
Summary:
Time csv_to_snapshot conversion and log it.
Check if writing csv_to_snapshot failed.
Extract LoadConfig from memgraph_bolt to config.hpp.
Read memgraph config in csv_to_snapshot for snapshot_directory.
Rename csv_to_snapshot to mg_import_csv.
Add tests for tools.
Run tools tests in apollo.

Reviewers: mislav.bradac, florijan, mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D931
2017-10-26 09:37:56 +02:00
Matej Ferencevic
cfb8db9e20 Lowered treshold for macro benchmark memory measurements.
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D927
2017-10-25 20:24:34 +02:00
Matej Ferencevic
6b80d472cb Fixed macro benchmark parallel summary.
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D928
2017-10-24 16:26:31 +02:00
Matej Ferencevic
cbf8dacc11 Add parallel macro benchmark suite
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D922
2017-10-24 13:53:09 +02:00
florijan
4460dd79f9 Durability - snapshot magic num, version and tx snapshot added
Summary:
New snapshot structure:
- magic number
- snapshot version (old-version recovery not yet implemented)
- transaction snapshot (will be used in the WAL)
- the rest is as before (indices, vertices, edges)

Not backward compatible with the old snapshotting.

Does not improve error handling (user feedback). A task for that has been added.

Reviewers: buda, mislav.bradac, mferencevic, teon.banek

Reviewed By: teon.banek

Subscribers: teon.banek, dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D912
2017-10-23 15:51:35 +02:00
Dominik Gleich
0df240f9d5 Remove macro benchmark results if nothing really changed.
Reviewers: mislav.bradac, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D913
2017-10-19 16:30:40 +02:00
Mislav Bradac
03c50bada9 Pass assigned cpus to harness by flags
Summary:
Since we have different kind of workers in Apollo we should pass
assigned cpus to harness from apollo generate script and not define them
in harness or in benchmarks.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D916
2017-10-19 13:29:45 +02:00
Teon Banek
06b0111ddc Remove unused stuff from CMakeLists
Summary: In the process, make experimental/distributed compilable.

Reviewers: mislav.bradac, buda, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D906
2017-10-16 13:45:18 +02:00
Matej Ferencevic
6a144db165 Integrated release files generation into Apollo.
Summary:
Don't store unnecessary files in documentation archive.

Apollo generate script cleanup.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D887
2017-10-09 21:13:29 +02:00
Marko Budiselic
7a14e09dfc Add Dockerfile for csv_to_snapshot packaging.
Summary:
C++ implementation of CSV to Memgraph's snapshot file is much more efficient then
the Python implementation. It includes a huge part of the Memgraph's source
code and the code has to be compiled (which is not easy). Because of that it
has to be packaged.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D854
2017-10-09 12:15:46 +01:00
Marko Budiselic
2d2ebaca2f LDBC plot script update
Summary: Plot's width, height, and PPI are now parameterized.

Reviewers: teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D847
2017-10-08 14:23:53 +01:00
Matej Ferencevic
37b2caae63 Implemented hidden flags.
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D861
2017-10-07 15:08:48 +02:00
Teon Banek
4c25123d83 Make docker installation equivalent to regular install
Summary:
This puts the whole installation and packaging under a single point of
entry. (Docker, DEB, RPM, etc.)

Rename alpha.dockerfile to beta.dockerfile
Use Debian Stretch for docker
Remove building old hardcoded compiler
Rename build_interpreter to build_memgraph
Remove unused config-file

Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D857
2017-10-06 13:23:34 +02:00
Teon Banek
f6a8d0e910 Fix flag parsing and update usage message
Summary:
csv_to_snapshot: Don't skip flags when parsing
csv_to_snapshot: Update usage message

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D865
2017-10-06 12:45:34 +02:00
Teon Banek
1cd83d2c19 Add option whether to link with libreadline or not
Reviewers: buda, mislav.bradac, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D850
2017-10-04 15:23:11 +02:00
Teon Banek
0c186fe76f Add convenience functions and a macro to libs/CMakeLists.txt
Summary:
Add json and cppitertools to libs/CMakeLists.txt.
Import external projects as libraries.
This removes the need to use `add_dependencies` in order to link with
external project.
Extract common ExternalProject_Add function.
Add macro for easier addition of external libraries.

Reviewers: mislav.bradac, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D845
2017-10-03 14:28:16 +02:00
Teon Banek
e70f4de208 Make almost all libs external
Summary:
Make 'benchmark' external project
Make 'fmt' external project
Make 'gtest' external project
Make 'gflags' external project
Skip fmt tests

Reviewers: mferencevic, mislav.bradac, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D837
2017-09-27 13:46:33 +02:00
Mislav Bradac
f91aa7b8fe Add medium sized pokec scenario
Summary: Add window size to plot throughput

Reviewers: florijan

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D828
2017-09-25 13:20:08 +02:00
Matej Ferencevic
99494d77e3 Integrated C++ CSV to snapshot into LDBC.
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D826
2017-09-23 23:12:23 +02:00
Teon Banek
acb102de65 Replace Python csv_to_snapshot with C++
Summary:
Setup scaffolding for building Memgraph tools.

Change `utils::Split` without delimiter to split on whitespace.
This should make `Split` behave just like Python's `str.split`, which is
more practical for splitting on word boundaries.

Add `utils::StartsWith` function.

Rewrite csv_to_snapshot to C++.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D822
2017-09-22 13:59:42 +02:00
Matej Ferencevic
686dc452ee LDBC: Integrate with Apollo
Summary:
Run neo4j and memgraph from run_benchmark script.
This makes mg and neo scripts obsolete.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D806
2017-09-21 22:29:42 +02:00
Mislav Bradac
cf7190ecc6 Refactor long running benchmark
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D801
2017-09-20 10:36:51 +02:00
Matej Ferencevic
2603209849 Replaced Python long running test with C++ version
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D802
2017-09-18 15:06:22 +02:00
Mislav Bradac
960c5e6092 Change parent macrobenchmark generate
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D800
2017-09-15 14:51:39 +02:00
Mislav Bradac
7e5dddecae Move harness stuff to parent directory
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D799
2017-09-15 14:05:26 +02:00
Matej Ferencevic
84b0d03a5f Added max memory usage to harness
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D790
2017-09-13 22:13:22 +02:00
Matej Ferencevic
9d63dfa9ca Added slave groups to Apollo generate.
Summary: Added large stress to daily release.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D786
2017-09-13 17:02:41 +02:00
Teon Banek
bfc56440a1 Add max-label-width argument to plot_ldbc_latency
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D779
2017-09-13 09:39:03 +02:00
Mislav Bradac
7e99e93e47 Start work on parallel benchmark
Summary: First version of our benchmark

Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D740
2017-09-12 16:58:21 +02:00