Commit Graph

55 Commits

Author SHA1 Message Date
Teon Banek
55456b4214 Remove Dbms
Summary:
Remove name from GraphDb.
Take GraphDb in query test macros instead of accessor.
Add is_accepting_transactions flag to GraphDb.

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D940
2017-10-30 12:33:29 +01:00
Teon Banek
df72861b90 Make csv_to_snapshot more user friendly
Summary:
Time csv_to_snapshot conversion and log it.
Check if writing csv_to_snapshot failed.
Extract LoadConfig from memgraph_bolt to config.hpp.
Read memgraph config in csv_to_snapshot for snapshot_directory.
Rename csv_to_snapshot to mg_import_csv.
Add tests for tools.
Run tools tests in apollo.

Reviewers: mislav.bradac, florijan, mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D931
2017-10-26 09:37:56 +02:00
Mislav Bradac
19a44a7d94 Close session on timeouts
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D897
2017-10-17 14:34:31 +02:00
Mislav Bradac
f10380a861 Begin refactor of network layer
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D855
2017-10-10 14:55:13 +02:00
Teon Banek
8bf0a9cf92 Fill the alpha.conf with all settings and document them
Summary:
This should be the basis for default configuration that is to be shipped
in release builds/packages.

Reviewers: mislav.bradac, florijan, buda, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D851
2017-10-06 15:15:19 +02:00
Matej Ferencevic
d249ff11a5 Misc glog and gflags fixes.
Summary:
Log files aren't created by default anymore.
All logs are reported to stderr by default.
Normalized flag names.
Removed unnecessary flags from gflags.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D856
2017-10-04 17:29:53 +02:00
Matej Ferencevic
a40248ab65 Removed filename strings from release binary.
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D839
2017-10-03 11:06:42 +02:00
Matej Ferencevic
30573d4fd5 Removed chdir from main binary.
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D842
2017-10-02 14:00:08 +02:00
Mislav Bradac
56017e598e Abort all transactions on SIGTERM
Reviewers: buda, teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D818
2017-09-27 10:56:44 +02:00
Teon Banek
8c9ff2d79a Add version.hpp.in and display it on '--version' flag
Reviewers: mferencevic, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D831
2017-09-27 09:04:44 +02:00
florijan
d9503d6b65 Flags cleanup and QueryEngine removal
Summary:
I started with cleaning flags up (removing unused ones, documenting undocumented ones). There were some flags to remove in `QueryEngine`. Seeing how we never use hardcoded queries (AFAIK last Mislav's testing also indicated they aren't faster then interpretation), when removing those unused flags the `QueryEngine` becomes obsolete. That means that a bunch of other stuff becomes obsolete, along with the hardcoded queries. So I removed it all (this has been discussed and approved on the daily).

Some flags that were previously undocumented in `docs/user_technical/installation` are now documented. The following flags are NOT documented and in my opinion should not be displayed when starting `./memgraph --help` (@mferencevic):
```
query_vertex_count_to_expand_existsing (from rule_based_planner.cpp)
query_max_plans (rule_based_planner.cpp)
```

If you think that another organization is needed w.r.t. flag visibility, comment.

@teon.banek: I had to remove some stuff from CMakeLists to make it buildable. Please review what I removed and clean up if necessary if/when this lands. If the needed changes are minor, you can also comment.

Reviewers: buda, mislav.bradac, teon.banek, mferencevic

Reviewed By: buda, mislav.bradac

Subscribers: pullbot, mferencevic, teon.banek

Differential Revision: https://phabricator.memgraph.io/D825
2017-09-22 17:03:45 +02:00
florijan
b2f3dc1916 Available memory logging, utils/sysinfo/memory cleanup, tests/concurrent cleanup.
Summary:
Added a warning to the log. Every 3 seconds (let's not make that configurable). Can be turned off.

I still don't like this. We are raising another thread and reading a file to do monitoring. We're developing a DB, not a sys monitor. Serious admins do that themselves. But, here it is.

UPDATE:
Cleaned up `utils/sysinfo/memory`. Removed all unused functions. Removed the faulty memory check in `tests/concurrent`.

Reviewers: buda, mferencevic, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D819
2017-09-22 16:06:30 +02:00
Matej Ferencevic
9a080ba024 Initialize Dbms before starting the network server
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D805
2017-09-18 15:29:20 +02:00
Mislav Bradac
4d02a4f607 Fail if MEMGRAPH_CONFIG path exists
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D776
2017-09-11 15:38:05 +02:00
Matej Ferencevic
7939b04117 Removed unused template parameter from network stack.
Reviewers: mislav.bradac, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D632
2017-08-04 11:12:08 +02:00
Matej Ferencevic
71ded22b65 Removed Dbms and QueryEngine from the Network stack.
Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D628
2017-08-03 16:33:10 +02:00
Mislav Bradac
aac5205802 sptr to uptr in worker/server
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D543
2017-07-12 14:10:22 +02:00
Mislav Bradac
e08c6bf3ff Add cppitertools to ycm, small changes in loogging
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D541
2017-07-12 13:07:37 +02:00
Mislav Bradac
6068a95a0e Move dbms to database
Summary:
There was only two files in dbms directory so I moved them to database
directory.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D540
2017-07-12 12:44:11 +02:00
Mislav Bradac
cbe6648eb8 Migrate command line args to gflgs in tests
Summary: Some other minor cleanups

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D523
2017-07-06 13:54:12 +02:00
Dominik Gleich
e2f3aba332 Use GLogger instead of broken memgraph Logger.
Summary:
http://rpg.ifi.uzh.ch/docs/glog.html

Second phase before tests complete.

Delete logging test.

Finish relase loging.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D500
2017-06-21 15:33:24 +02:00
Teon Banek
d0016ab98c Add simple logging to a file
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D476
2017-06-16 10:19:37 +02:00
Teon Banek
527d69d382 Fix errors with handling SIGSEGV and SIGABRT
Summary:
There were a couple of issues with handling the above 2 signals.

1) Calling `std::exit` from a signal handler is undefined behaviour.

The only defined way for a signal handler to stop the program is calling
one of: `std::_Exit`, `std::abort` or `std::quick_exit`. Neither of them
will completely clean the resources, so a clean exit is not possible.
Since SIGSEGV and SIGABRT happen in extraordinary circumstances that we
wish to debug 99% of the time, it makes sense to generate a core dump
which can be inspected by a debugger. Of the 3 termination functions,
only `std::abort` will generate a core dump, so it makes sense to use
that to stop the program.

Also, since we are now aborting as is the default behaviour on SIGSEGV
and SIGABRT, it becomes questionable why have a custom handler at all.

2) Raising an exception inside a signal handler is undefined behaviour

Although the handler by itself does not raise an exception, it is
possible for the logging facility to raise one. This is a real case when
logging a stack trace in particular. Stack trace is generated by
creating a string "<function name> <line location>". It is possible that
a function name will contain '{}' somewhere inside. This is usually the
case with anonymous functions. The generated string is then passed to
logging, which uses the `fmt` library to fill '{}' with remaining
arguments. Since only a single argument (the stack trace string) is
passed for formatting, naturally the `fmt::format` throws an exception,
that it is missing a format argument.

We could provide an overload which takes a single string, but that
defeats the purpose of `fmt::format` raising an exception in regular
code if we forget to pass an argument. Another solution is to escape the
whole stack trace string, so it is valid for formatting, but this only
complicates the handler even further. The simplest solution is to send
the stack trace to `stderr` and avoid logging altogether.

Simplify Shutdown, so it can be used in a signal handler

Reviewers: florijan, mferencevic, buda, mislav.bradac

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D474
2017-06-16 08:53:07 +02:00
Teon Banek
7de8fd4b1b Reorganize logging in memgraph_bolt.cpp
Summary:
Remove utils/stacktrace/log.hpp
The single function defined in log.hpp is used only in
memgraph_bolt.cpp. Therefore, the function has been moved and the file
removed.

Move utils/stacktrace/stacktrace.hpp one level up

Move some logging from memgraph_bolt to Server

Reviewers: buda, dtomicevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D465
2017-06-14 10:11:31 +02:00
Teon Banek
025e557204 Validate the --num_workers flag
Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D449
2017-06-09 16:16:38 +02:00
Dominik Gleich
972c62e102 Fix signals. Add recovery status.
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D451
2017-06-09 13:25:24 +02:00
Dominik Gleich
1057d253ca Rename config flags.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D446
2017-06-09 10:52:26 +02:00
Marko Budiselic
f84f9af244 Alpha package scripts.
Summary: Alpha package scripts. Alpha version is going to be shipped within docker.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D436
2017-06-08 14:15:55 +02:00
Matej Ferencevic
e9ca6d8243 Fixed SIGTERM handler.
Summary: Merge branch 'dev' into fix_signals

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D432
2017-06-07 15:56:34 +02:00
Dominik Gleich
65507da9eb Load configs from flagfiles.
Summary: Migrate configto gflags.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D433
2017-06-07 15:38:17 +02:00
Mislav Bradac
9acdc632e9 Add google flag for number of workers
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D429
2017-06-06 18:52:03 +02:00
Mislav Bradac
a5e2ccb94f Change working dir to dir of memgraph binary
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D408
2017-05-30 16:49:56 +02:00
Dominik Gleich
a2d3577f0f Add google flags in preparation for deleting old logger.
Summary: Implement gflags and add optional port.

Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D391
2017-05-25 09:45:48 +02:00
Matej Ferencevic
e5c814e022 First version of new bolt decoder and tests.
Summary:
Extracted constants to codes.hpp.

Extracted bolt constants.

Extracted StreamBuffer and fixed data type.

Extracted bolt testdata.

Added bolt buffer and tests.

Added bolt decoder buffer and tests.

Renamed bolt testdata.

Reviewers: dgleich, buda, matej.gradicek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D220
2017-04-06 14:30:19 +02:00
Marko Budiselic
d00c08bf15 Bolt PullAll works.
Summary: Bolt PullAll works. ChunkedBuffer is implemented by the specification (the tests can be found in the tests/unit/bolt_chunked_buffer.cpp). Bolt session states method are refactored (all run methods have only one argument - active Session). Modifications related to the style guide.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D180
2017-03-28 12:43:04 +02:00
Matej Ferencevic
5a5ffface3 Modified hardcoded queries to use new encoder.
Summary:
Removed old encoder.

Changed namespace from bolt to communication::bolt.

Removed old include from new encoder.

Added an empty message success to encoder.

Changed order in communication::Server.

Changed bolt session to use new encoder.

Merge remote-tracking branch 'origin/dev' into mg_hardcoded_queries

Fixed PrintRecordStream.

Reviewers: buda, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D158
2017-03-22 17:09:22 +01:00
Matej Ferencevic
813a3b9eed Started network refactorization.
Summary:
Moved server and worker from bolt to communication. Started
templatization.

Started removal of Bolt class.

Removed unnecessary files from network.

Converted states to template functions.

Bolt::Session is now a template.

Merge remote-tracking branch 'origin/dev' into mg_refactor_network

Merged bolt_serializer.cpp into hpp.

Removed obsolete include.

Initial version of bolt session unit test.

Uncommented leftover log commands.

Reimplemented io::Socket.

Added client-stress.sh script.

Reviewers: dgleich, buda

Reviewed By: dgleich, buda

Subscribers: pullbot, mferencevic, buda

Differential Revision: https://phabricator.memgraph.io/D64
2017-03-06 13:52:34 +01:00
Marko Budiselic
28ad69b536 use of Google Style (clang-format)
Summary: use of Google Style (clang-format)

Test Plan: manual

Reviewers: sale

Reviewed By: sale

Subscribers: buda

Differential Revision: https://phabricator.memgraph.io/D39
2017-01-23 12:04:54 +01:00
Marko Budiselic
d7de5cfe08 merge + conflict resolution 2017-01-13 11:31:56 +01:00
sale
d475b79a58 Merge 2017-01-11 09:18:10 +00:00
sale
1c1cf1ad88 Review changes fixed 2016-12-28 11:56:53 +00:00
sale
02f332c34f Update on Configuration and CppCheck refactor 2016-12-21 17:48:38 +00:00
sale
08c8bd4aee Initial configuration memgraph which needs review
Summary: MG configuration commit for review

Test Plan: manual

Reviewers: buda

Subscribers: buda

Differential Revision: https://memgraph.phacility.com/D24
2016-12-20 22:26:03 +00:00
Marko Budiselic
362bc3ba48 Bug fixes: RELEASE MODE - asserts 2016-12-19 18:32:44 +01:00
sale
18612a3063 Added dump methods to stacktrace 2016-12-19 10:02:24 +00:00
sale
b4c65d9680 Stacktrace dump method added
Summary: Stacktrace dump method added

Test Plan: manual

Reviewers: buda

Subscribers: buda

Differential Revision: https://memgraph.phacility.com/D19
2016-12-16 14:50:19 +00:00
sale
9154f9b719 Added signal handler and refactored exception handler
Summary: Added signal handler and refactored exception handler

Test Plan: manual

Reviewers: buda

Subscribers: buda

Differential Revision: https://memgraph.phacility.com/D17
2016-12-16 12:58:49 +00:00
Marko Budiselic
8362b64e9f DRESSIPI FIRST PILOT RELEASE (release script update) 2016-11-04 12:05:35 +01:00
Marko Budiselic
d45121a1a2 scenario 000001 pass, dressipi scenario 1 - 3 more queries have to be implemented 2016-11-02 16:05:02 +01:00
Marko Budiselic
e22cb3aee4 review fixes + T70 2016-08-19 01:28:37 +01:00