Commit Graph

523 Commits

Author SHA1 Message Date
Teon Banek
f5db9d65b0 Add CreateIndex operator
Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D516
2017-07-03 11:03:15 +02:00
Mislav Bradac
592ba4df98 Minor cleanup
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D513
2017-06-28 15:56:05 +02:00
Mislav Bradac
3119ae5343 Supstitute named expressions in ast cache
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D510
2017-06-26 16:33:17 +02:00
Teon Banek
f89ef14823 Fix a minor typo when logging
Reviewers: mferencevic, mislav.bradac

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D507
2017-06-23 09:30:40 +02:00
Mislav Bradac
3a922de963 Implementation of new lexer
Summary: WORK IN PROGRESS

Reviewers: buda, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D496
2017-06-21 16:16:17 +02:00
Dominik Gleich
e2f3aba332 Use GLogger instead of broken memgraph Logger.
Summary:
http://rpg.ifi.uzh.ch/docs/glog.html

Second phase before tests complete.

Delete logging test.

Finish relase loging.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D500
2017-06-21 15:33:24 +02:00
florijan
19c0dfe084 SkipList - added position_and_count function
Summary: This is the first implementation that seems to work. I am not happy with it's complexity. Might attempt a simpler implementation, at the cost of some performance.

Reviewers: dgleich, buda

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D502
2017-06-21 14:12:29 +02:00
florijan
a726ac0023 Tests - long-running test added
Summary:
Tests - long-running test added

 - common functionality refactor
 - long running tests WIP

tests/stress/common: minor refactor, long_running WIP

tests/stress/long_running - stable single-threaded, unstable concurrent

tests/stress/long_running - added error summary output

test/stress/long_running - add/remove vertex/edge stable multithreaded

test/stress/long_running - added vertex label testing

tests/stress/long_running - time limit added

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D489
2017-06-21 12:10:36 +02:00
florijan
ad148692c3 Concurrent::PushQueue added
Summary:
PushQueue with concurrent lock-free pushing, and single-threaded deletion. Iteration without modification can also be concurrent. Deletion should NOT be concurrent with iteration and other deletions, but can be concurrent with pushing.

There is no const iteraton at the moment, we can add it when necessary. Also I've not handled std::iterator_traits, might be fun getting into that :D

Reviewers: buda, dgleich, mislav.bradac

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D420
2017-06-21 09:03:33 +02:00
Mislav Bradac
b6ecc5631f Disallow keywords as symbolic names
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D497
2017-06-20 11:58:14 +02:00
florijan
492b4919f1 Concurrent data structures -> multi_X stuff removed. Skiplist cleanup.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D495
2017-06-20 08:42:20 +02:00
florijan
e28fc6ed41 FLAG 'snapshot_on_destruction' renamed to 'snapshot_on_exit'
Reviewers: teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D471
2017-06-16 16:25:26 +02:00
Mislav Bradac
6403599a5d Generate plan_compiler_flags only if needed
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D485
2017-06-16 15:58:32 +02:00
Dominik Gleich
f034c29724 Previous iterator should only be set when the node is alive.
Summary: There was a big performance hit induced when removing consecutive nodes from the concurrent list. The reason why it was happening is that the iterator has a pointer to previous node, and uses that pointer to re-link the whole list after the deletion. That node wasn't alive because it was deleted earlier and was always being updated to the next deleted entry in the list while incrementing the iterator. This behaviour caused find_and_disconnect method to be invoked, which has an O(n) complexity. That made our removal of O(n) entries from the list run in O(n^2) time, which is obviously slow.

Reviewers: buda, mislav.bradac, florijan

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D483
2017-06-16 15:28:05 +02:00
Mislav Bradac
1862b04ac2 Fix bug in StrippedQuery
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D482
2017-06-16 15:04:48 +02:00
Mislav Bradac
5403b99e18 Remove logger from StrippedQuery
Summary:
benchmark/query/stripped.cpp outputs garbage because of some race
condition in logger. Sometime even crashes, just be sure this
should be removed.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D477
2017-06-16 12:13:23 +02:00
Teon Banek
7d00c4eac0 Add query_cost_planner gflag
Reviewers: mislav.bradac, buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D479
2017-06-16 10:20:02 +02:00
Teon Banek
d0016ab98c Add simple logging to a file
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D476
2017-06-16 10:19:37 +02:00
Teon Banek
527d69d382 Fix errors with handling SIGSEGV and SIGABRT
Summary:
There were a couple of issues with handling the above 2 signals.

1) Calling `std::exit` from a signal handler is undefined behaviour.

The only defined way for a signal handler to stop the program is calling
one of: `std::_Exit`, `std::abort` or `std::quick_exit`. Neither of them
will completely clean the resources, so a clean exit is not possible.
Since SIGSEGV and SIGABRT happen in extraordinary circumstances that we
wish to debug 99% of the time, it makes sense to generate a core dump
which can be inspected by a debugger. Of the 3 termination functions,
only `std::abort` will generate a core dump, so it makes sense to use
that to stop the program.

Also, since we are now aborting as is the default behaviour on SIGSEGV
and SIGABRT, it becomes questionable why have a custom handler at all.

2) Raising an exception inside a signal handler is undefined behaviour

Although the handler by itself does not raise an exception, it is
possible for the logging facility to raise one. This is a real case when
logging a stack trace in particular. Stack trace is generated by
creating a string "<function name> <line location>". It is possible that
a function name will contain '{}' somewhere inside. This is usually the
case with anonymous functions. The generated string is then passed to
logging, which uses the `fmt` library to fill '{}' with remaining
arguments. Since only a single argument (the stack trace string) is
passed for formatting, naturally the `fmt::format` throws an exception,
that it is missing a format argument.

We could provide an overload which takes a single string, but that
defeats the purpose of `fmt::format` raising an exception in regular
code if we forget to pass an argument. Another solution is to escape the
whole stack trace string, so it is valid for formatting, but this only
complicates the handler even further. The simplest solution is to send
the stack trace to `stderr` and avoid logging altogether.

Simplify Shutdown, so it can be used in a signal handler

Reviewers: florijan, mferencevic, buda, mislav.bradac

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D474
2017-06-16 08:53:07 +02:00
Teon Banek
13ae13bf43 Make asserts abort the program instead of exiting cleanly
Summary:
Since assertions should signify an abnormal program condition, it makes
sense for them to call `std::abort`. This way, we'll get a core dump
which can be inspected in a debugger.

Update utils/assert.hpp documentation

Reviewers: florijan, buda, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D475
2017-06-16 08:52:36 +02:00
Mislav Bradac
67b859cf13 Add AST cache
Reviewers: buda, teon.banek, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D468
2017-06-14 18:59:31 +02:00
florijan
b6da65b9e7 Skiplist - GC - race condition fix
Summary: An unpleasant race-condition detected. Solution proposed. It's not very pretty. Perhaps consider using the ConcurrentPushQueue. Not 100% sure, but it should make the GC code easier to work with.

Reviewers: buda, mislav.bradac, teon.banek, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D469
2017-06-14 16:16:53 +02:00
Teon Banek
7de8fd4b1b Reorganize logging in memgraph_bolt.cpp
Summary:
Remove utils/stacktrace/log.hpp
The single function defined in log.hpp is used only in
memgraph_bolt.cpp. Therefore, the function has been moved and the file
removed.

Move utils/stacktrace/stacktrace.hpp one level up

Move some logging from memgraph_bolt to Server

Reviewers: buda, dtomicevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D465
2017-06-14 10:11:31 +02:00
Mislav Bradac
c56eb4310e Remember token positions for literals
Summary: Remove yaml

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D466
2017-06-13 17:11:27 +02:00
Mislav Bradac
c8e25987b8 Add asserts to placeholder
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D463
2017-06-13 14:54:17 +02:00
Teon Banek
69cfd197d8 Don't raise TypedValueException during query execution
Summary:
Handle TypedValueExceptions in query/plan/operator.cpp

Raise QueryRuntimeException during expression evaluation

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D457
2017-06-13 14:46:56 +02:00
florijan
9e6260d0c3 tx::Engine - commit log encapsulation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D460
2017-06-13 12:32:48 +02:00
Mislav Bradac
f7d540829a Add deep clone to Ast
Summary: Prepare your aspergillums for blessing this magnificent diff.

Reviewers: buda, teon.banek, florijan

Reviewed By: teon.banek

Subscribers: lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D458
2017-06-13 12:22:25 +02:00
florijan
c12990ae33 DynamicBitset - const correctness, tests, docs
Reviewers: dtomicevic, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D453
2017-06-13 09:18:20 +02:00
Teon Banek
3b88d7429f Visit edge properties during symbol generation
Summary:
This is an obvious bug, caused by an oversight. A test has been added
for this case.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D455
2017-06-12 15:27:15 +02:00
Mislav Bradac
d46945f259 Make GraphDbTypes const
Reviewers: florijan

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D456
2017-06-12 15:24:30 +02:00
florijan
afbb940a05 GC bugfixes, MVCC and transaction refactoring
Summary:
- GC changed to evaluate old records w.r.t. the oldest transaction's id AND snapshot, as opposed to only id
- MVCC hints exp+aborted race condition prevented
- minor MVCC refactors and cleanups
- minor Transaction refactors and cleanups

Reviewers: buda, dgleich

Reviewed By: buda, dgleich

Subscribers: dtomicevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D434
2017-06-12 10:46:12 +02:00
Teon Banek
025e557204 Validate the --num_workers flag
Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D449
2017-06-09 16:16:38 +02:00
Dominik Gleich
972c62e102 Fix signals. Add recovery status.
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D451
2017-06-09 13:25:24 +02:00
Dominik Gleich
0fb29a7ce0 Add release logging without core dump.
Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D447
2017-06-09 11:26:19 +02:00
Dominik Gleich
1057d253ca Rename config flags.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D446
2017-06-09 10:52:26 +02:00
Matej Ferencevic
6308ec6a34 Added javascript and java driver tests.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D438
2017-06-09 10:12:25 +02:00
Teon Banek
04a5bdeb82 Remove unused config source code files
Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D442
2017-06-08 16:10:55 +02:00
Marko Budiselic
f84f9af244 Alpha package scripts.
Summary: Alpha package scripts. Alpha version is going to be shipped within docker.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D436
2017-06-08 14:15:55 +02:00
Mislav Bradac
fea9031605 Refactor stripper
Summary: Fix tests

Reviewers: buda, florijan, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D435
2017-06-07 18:47:09 +02:00
Matej Ferencevic
e9ca6d8243 Fixed SIGTERM handler.
Summary: Merge branch 'dev' into fix_signals

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D432
2017-06-07 15:56:34 +02:00
Mislav Bradac
a45e5a9a38 Remove old snapshot
Reviewers: buda, matej.gradicek

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D437
2017-06-07 15:47:24 +02:00
Dominik Gleich
65507da9eb Load configs from flagfiles.
Summary: Migrate configto gflags.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D433
2017-06-07 15:38:17 +02:00
Dominik Gleich
ebdee4e509 Skiplist garbage collector rework.
Summary:
Drawing:
https://drive.google.com/open?id=0B-W7PQZqMD9hcG04b0lKaGZGOWM

Reviewers: mislav.bradac, buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D395
2017-06-07 10:47:02 +02:00
Dominik Gleich
141d9b3bb8 Rework of index api. Remove implicit default value of current_state in graph_db_accessor.
Reviewers: buda, florijan

Reviewed By: buda, florijan

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D392
2017-06-07 10:14:55 +02:00
Mislav Bradac
9acdc632e9 Add google flag for number of workers
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D429
2017-06-06 18:52:03 +02:00
matej.gradicek
15b57e2d52 Implemented recovery and tests.
Summary: Implemented durability recovery and tesats.

Reviewers: mislav.bradac, dgleich, buda

Reviewed By: mislav.bradac, dgleich, buda

Subscribers: dtomicevic, mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D374
2017-06-06 15:12:34 +00:00
Teon Banek
7278bdff94 Check for aggregation in all Expression types
Summary:
This fixes an issue when aggregations and/or group by expressions
weren't picked up from certain operators. In addition to that, we would
segfault in cases when the `has_aggregation_` is empty. For example,
function calls without arguments: `RETURN PI()`.

Test aggregations inside some operators

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D425
2017-06-06 15:05:43 +02:00
Teon Banek
b2c523b93a Generate multiple plans and use the least costly one
Summary:
This is a simple integration of multiple plan generation and cost
estimation. Each plan produced by VariableStartPlanner is cost estimated
and the best is used for execution.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D424
2017-06-06 14:03:41 +02:00
florijan
509d5db67a Query - plan - cost estimator basic. REPL mods
Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: teon.banek, buda

Subscribers: pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D399
2017-06-02 12:44:17 +02:00