Summary:
This diff changes the RPC layer to directly return `TResponse` to the user when
issuing a `Call<...>` RPC call. The call throws an exception on failure
(instead of the previous return `nullopt`).
All servers (network, RPC and distributed) are set to have explicit `Shutdown`
methods so that a controlled shutdown can always be performed. The object
destructors now have `CHECK`s to enforce that the `AwaitShutdown` methods were
called.
The distributed memgraph is changed that none of the binaries (master/workers)
crash when there is a communication failure. Instead, the whole cluster starts
a graceful shutdown when a persistent communication error is detected.
Transient errors are allowed during execution. The transaction that errored out
will be aborted on the whole cluster. The cluster state is managed using a new
Heartbeat RPC call.
Reviewers: buda, teon.banek, msantl
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1604
Summary:
This diff implements OpenSSL support in the network stack.
Currently SSL support is only enabled for Bolt connections,
support for RPC connections will be added in another diff.
Reviewers: buda, teon.banek
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1328
Summary:
The network layer now has a `Session` that handles all things that should be
done before the `Execute` method is called on sessions. Also, all sessions
now communicate using streams instead of holding the input buffer and writing
to the `Socket`. This design will allow implementation of a SSL middleware.
Reviewers: buda, dgleich
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1314
Summary:
To see the thread names in htop press <F2> to enter setup and under
'Display options' choose 'Show custom thread names'.
Reviewers: buda, teon.banek, florijan
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1234
Summary:
Previously, the network stack `communication::Server` accepted connections and
assigned them statically in a round-robin fashion to `communication::Worker`.
That meant that if two compute intensive connections were assigned to the same
worker they would block each other while the other workers would do nothing.
This implementation replaces `communication::Worker` with
`communication::Listener` which holds all accepted connections in one pool and
ensures that all workers execute all connections.
Reviewers: buda, florijan, teon.banek
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1220
Summary:
Start removal of old logic
Remove more obsolete classes
Move Message class to RPC
Remove client logic from system
Remove messaging namespace
Move protocol from messaging to rpc
Move System from messaging to rpc
Remove unnecessary namespace
Remove System from RPC Client
Split Client and Server into separate files
Start implementing new client logic
First semi-working state
Changed network protocol layout
Rewrite client
Fix client receive bug
Cleanup code of debug lines
Migrate to accessors
Migrate back to binary boost archives
Remove debug logging from server
Disable timeout test
Reduce message_id from uint64_t to uint32_t
Add multiple workers to server
Fix compiler warnings
Apply clang-format
Reviewers: teon.banek, florijan, dgleich, buda, mtomic
Reviewed By: teon.banek
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D1129
Summary:
There was only two files in dbms directory so I moved them to database
directory.
Reviewers: buda
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D540
Summary:
There were a couple of issues with handling the above 2 signals.
1) Calling `std::exit` from a signal handler is undefined behaviour.
The only defined way for a signal handler to stop the program is calling
one of: `std::_Exit`, `std::abort` or `std::quick_exit`. Neither of them
will completely clean the resources, so a clean exit is not possible.
Since SIGSEGV and SIGABRT happen in extraordinary circumstances that we
wish to debug 99% of the time, it makes sense to generate a core dump
which can be inspected by a debugger. Of the 3 termination functions,
only `std::abort` will generate a core dump, so it makes sense to use
that to stop the program.
Also, since we are now aborting as is the default behaviour on SIGSEGV
and SIGABRT, it becomes questionable why have a custom handler at all.
2) Raising an exception inside a signal handler is undefined behaviour
Although the handler by itself does not raise an exception, it is
possible for the logging facility to raise one. This is a real case when
logging a stack trace in particular. Stack trace is generated by
creating a string "<function name> <line location>". It is possible that
a function name will contain '{}' somewhere inside. This is usually the
case with anonymous functions. The generated string is then passed to
logging, which uses the `fmt` library to fill '{}' with remaining
arguments. Since only a single argument (the stack trace string) is
passed for formatting, naturally the `fmt::format` throws an exception,
that it is missing a format argument.
We could provide an overload which takes a single string, but that
defeats the purpose of `fmt::format` raising an exception in regular
code if we forget to pass an argument. Another solution is to escape the
whole stack trace string, so it is valid for formatting, but this only
complicates the handler even further. The simplest solution is to send
the stack trace to `stderr` and avoid logging altogether.
Simplify Shutdown, so it can be used in a signal handler
Reviewers: florijan, mferencevic, buda, mislav.bradac
Reviewed By: mferencevic, buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D474
Summary:
Remove utils/stacktrace/log.hpp
The single function defined in log.hpp is used only in
memgraph_bolt.cpp. Therefore, the function has been moved and the file
removed.
Move utils/stacktrace/stacktrace.hpp one level up
Move some logging from memgraph_bolt to Server
Reviewers: buda, dtomicevic
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D465
Summary: Bolt PullAll works. ChunkedBuffer is implemented by the specification (the tests can be found in the tests/unit/bolt_chunked_buffer.cpp). Bolt session states method are refactored (all run methods have only one argument - active Session). Modifications related to the style guide.
Reviewers: mferencevic
Reviewed By: mferencevic
Subscribers: pullbot, buda
Differential Revision: https://phabricator.memgraph.io/D180
Summary:
Removed old encoder.
Changed namespace from bolt to communication::bolt.
Removed old include from new encoder.
Added an empty message success to encoder.
Changed order in communication::Server.
Changed bolt session to use new encoder.
Merge remote-tracking branch 'origin/dev' into mg_hardcoded_queries
Fixed PrintRecordStream.
Reviewers: buda, dgleich
Reviewed By: buda
Subscribers: pullbot
Differential Revision: https://phabricator.memgraph.io/D158
Summary:
Moved server and worker from bolt to communication. Started
templatization.
Started removal of Bolt class.
Removed unnecessary files from network.
Converted states to template functions.
Bolt::Session is now a template.
Merge remote-tracking branch 'origin/dev' into mg_refactor_network
Merged bolt_serializer.cpp into hpp.
Removed obsolete include.
Initial version of bolt session unit test.
Uncommented leftover log commands.
Reimplemented io::Socket.
Added client-stress.sh script.
Reviewers: dgleich, buda
Reviewed By: dgleich, buda
Subscribers: pullbot, mferencevic, buda
Differential Revision: https://phabricator.memgraph.io/D64
