Summary:
Add additional structs and functions for handling C++ meta information.
Add capnp-file and capnp-id arguments to lcp:process-file.
Generate cpp along with hpp and capnp in lcp.
Wrap LogicalOperator base class in lcp:define-class.
Modify logical operators for capnp serialization.
Add query/common.capnp.
Reviewers: mculinovic, buda, mtomic, msantl, ipaljak, dgleich, mferencevic
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1391
Summary:
In order to enhance C++ metaprogramming capabilities, a custom
preprocessing step is added before compilation. C++ code may be mixed
with Lisp code in order to generate a complete C++ source code. The
mechanism is hooked into cmake. To notify cmake of .lcp files, `add_lcp`
function in src/CMakeLists.txt needs to be invoked.
The main executable entry point is in tools/lcp, while the source code
is in src/lisp/lcp.lisp
The main goal of LCP is to auto generate class serialization code and
member variable getter functions. This should now be significantly less
error prone, since you cannot forget to serialize a member variable
through this mechanism. Future uses should be generating other repeating
code, such as `Clone` methods or perhaps some debug information.
.lcp files may contain mixed C++ code (enclosed in #>cpp ... cpp<#
blocks) with Common Lisp code.
NOTE: With great power comes great responsibility. Lisp metaprogramming
capabilities are incredibly powerful. To keep the sanity of the team
intact, use Lisp preprocessing only when *really* necessary.
Reviewers: buda, mferencevic, msantl, dgleich, ipaljak, mculinovic, mtomic
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1361
Summary:
Snapshots should have the transaction from which they were created because we need this info for recovery later on.
Otherwise we wouldn't be able to tell the workers from which snapshots to recover. The whole cluster should be recovered
from the same transaction snapshot.
Reviewers: msantl
Reviewed By: msantl
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1338
Summary:
Flag cluster-prefix is added because we want to differentiate
between stats collected on different memgraph clusters.
Reviewers: mtomic, buda
Reviewed By: mtomic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1245
Summary:
Memgraph has flag --rpc-num-workers, so support for this
was added to script.
Reviewers: buda, mtomic
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1243
Summary:
Previously, the RPC stack used the network stack only to receive messages. The
messages were then added to a separate queue that was processed by different
thread pools. This design was inefficient because there was a lock when
inserting and getting messages from the common queue.
This diff removes the need for separate thread pools by utilising the new
network stack design. This is possible because the new network stack allows
full processing of the network request without blocking the whole queue.
Reviewers: buda, florijan, teon.banek, dgleich, mislav.bradac
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1229
Summary:
Stats server wasn't connecting to the right service on statsd.
Also, benchmark client stats now have prefix `client` instead
of machine name to be consistent with memgraphs stats naming
which starts with `master` or `worker`.
Reviewers: mtomic, buda
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1209
Summary:
Get rid of client class
Fix cppcheck errors
Add documentation to metrics.hpp
Add documentation to stats.hpp
Remove stats from global namespace
Fix build failures
Refactor a bit
Refactor stopwatch into a function
Add rpc execution time stats
Fix segmentation fault
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1200
Summary:
We have been using `Edges::VertexAddress` and `Edges::EdgeAddress` a lot
in other parts of the codebase because it's cleaner to write then
`Address<mvcc::VersionList<Edge>>`, especially in code what should not
really be MVCC-aware. However, a lot of that code should not really be
`Edges` aware either, as that's a storage datastructure that should not
be exposed.
This became annoying, so I extracted these addresses into a type-file. I
don't really like this approach, it might be better to have
`Vertex::Address` and `Edge::Address`, but that means we'd have to
import those headers and we'd get circular dependencies.
“The horror! The horror!”
- Joseph Conrad, Heart of Darkness
Reviewers: teon.banek, buda
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1204
Summary:
* add run_pokec script because more than one step is required
* refactor of plot_throughput script
* move all plot scripts under tools/plot
Reviewers: mferencevic, teon.banek, mislav.bradac
Reviewed By: mferencevic
Subscribers: florijan, pullbot, buda
Differential Revision: https://phabricator.memgraph.io/D1106
Summary:
Adds worker id to snapshot and wal filename.
Adds a new worker_id flag to be used for recovering a worker with a distributed snapshot.
Adds worker_id field to snapshot to check for consistency.
Reviewers: florijan
Reviewed By: florijan
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1096
Summary:
The directory was never actually copied on apollo, so tests weren't even
doing anything...
Also remove fswatcher unit test, it should be rewritten correctly.
Reviewers: mislav.bradac, mferencevic, buda
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1108
Summary:
GraphDb is refactored to become an API exposing different parts
necessary for the database to function. These different parts can have
different implementations in SingleNode or distributed Master/Server
GraphDb implementations.
Interally GraphDb is implemented using two class heirarchies. One
contains all the members and correct wiring for each situation. The
other takes care of initialization and shutdown. This architecture is
practical because it can guarantee that the initialization of the
object structure is complete, before initializing state.
Reviewers: buda, mislav.bradac, dgleich, teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1093
Summary:
Improve Apollo config files
Add name to apollo_build
Remove old generate script from build
Add build_release symlink to release build
Rename 'args' to 'arguments'
Add run definition for cppcheck
Host doxygen documentation
Reviewers: teon.banek
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1095
Summary:
Memgraph's executable target is unnecessarily complicated by having a
name which depends on the commit hash and build type. It is now renamed
to simply 'memgraph', while the executable output uses the encoding of
commit hash and build type as the name. This simplifies the dependency
handling in cmake, where now the full path to the executable can be
easily obtained by using the generator expression:
`$<TARGET_FILE:memgraph>`.
In addition to the above simplification, memgraph_link_target is now
replaced with a post build step on memgraph target itself. This
guarantees that the symlink always exists after building the executable,
as well as simplify the make invocation. The invocation now boils down
to `make memgraph` instead of `make memgraph_link_target`.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1043
Summary:
Split main CMakeLists into src/CMakeLists
The main CMakeLists duty is to make all the required libraries and
variables visible to all of the other sub-CMakeLists. After doing that,
it should include those sub-CMakeLists according to configuration
options.
This should make global configurations easier to reuse without polluting
the global space with locally related configurations. It is a necessary
step for including other projects like 'tools' in the release
installation.
Building tools is automatically disabled, but can be enabled by setting
the TOOLS option to ON when running cmake. This should allow on demand
building as well as combined installation of Memgraph and its tools.
Reviewers: mferencevic, buda
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1018
Summary:
It occurred that part of the durability flakyness test might be that the
same durability directory is used always. If the test is run
simultaneously on a single system, there will be interference.
This might not actually fix all the flakyness :(
I also made the `utils::RandomString` function since that's now used in
multiple places, tested it etc.
Reviewers: buda, dgleich
Reviewed By: dgleich
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1020
Summary:
The current idea is that the same MG binary can be used for single-node,
distributed master and distributed worker. The transactional engine in
the single-node and distributed master is the same: it determines the
transactional time and exposes all the "global" functionalities. In the
distributed worker the "global" functions must contact the master.
Reviewers: dgleich, mislav.bradac, buda
Reviewed By: dgleich
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1013
Summary:
Add postinst script for DEB package
The script creates a 'memgraph' group and sets permission on installed
'/var/*/memgraph' directories. Only the group is created, while
'memgraph' user is not. It seems more sane only to require group
membership for using memgraph.
Add conffiles for DEB package
This allows for `dpkg` to detect changes in configuration files and
present them to the user. Therefore, we don't need to care whether the
configuration merges are handled correctly nor if we accidentally
overwrite them.
Add postrm script for DEB packaging
The script is only used so that `dpkg --purge` removes '/var/*/memgraph'
directories, even if they contain something.
Add email, longer description and license file to DEB packaging, as well
as a systemd service.
Provide a logrotate configuration and support it in memgraph.
Use DEB package for Docker installation
This way, the whole installation process and testing should go through
DEB.
Generate release archives in Apollo with standard names
Reviewers: buda, mferencevic
Reviewed By: mferencevic
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D989
