Commit Graph

102 Commits

Author SHA1 Message Date
florijan
b2f3dc1916 Available memory logging, utils/sysinfo/memory cleanup, tests/concurrent cleanup.
Summary:
Added a warning to the log. Every 3 seconds (let's not make that configurable). Can be turned off.

I still don't like this. We are raising another thread and reading a file to do monitoring. We're developing a DB, not a sys monitor. Serious admins do that themselves. But, here it is.

UPDATE:
Cleaned up `utils/sysinfo/memory`. Removed all unused functions. Removed the faulty memory check in `tests/concurrent`.

Reviewers: buda, mferencevic, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D819
2017-09-22 16:06:30 +02:00
Teon Banek
acb102de65 Replace Python csv_to_snapshot with C++
Summary:
Setup scaffolding for building Memgraph tools.

Change `utils::Split` without delimiter to split on whitespace.
This should make `Split` behave just like Python's `str.split`, which is
more practical for splitting on word boundaries.

Add `utils::StartsWith` function.

Rewrite csv_to_snapshot to C++.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D822
2017-09-22 13:59:42 +02:00
Teon Banek
08eaaba77f Add caching logical plans
Summary: Add const to MakeCursor method

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D791
2017-09-20 13:31:26 +02:00
florijan
347611edfd Named path support
Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D794
2017-09-19 09:45:50 +02:00
Teon Banek
35f726dfd2 Use EdgeType in Expand and ExpandVariable
Summary:
Add function First to utils.
Insert EdgeType into Expand during planning.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D769
2017-09-11 11:16:04 +02:00
Mislav Bradac
38c83d80cb Rename Executioner to Executor
Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D764
2017-09-07 15:13:17 +02:00
florijan
4975743f35 Assert function added
Summary: `assert` function added. Useful to us for asserting DB state in harness tests. Potentially useful to the client for breaking out of a query as soon as a predicate fails, as opposed to collecting result and checking them client-side.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D750
2017-09-05 13:57:56 +02:00
Mislav Bradac
bba5d134c0 Refactor harness
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D732
2017-08-31 11:23:11 +02:00
Teon Banek
7fc821ad25 Setup basic interactive planner testing
Summary:
Add basic InteractiveDbAccessor.
Add printing logical operator tree.
Cache some InteractiveDbAccessor vertex counts.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D707
2017-08-28 10:25:17 +02:00
florijan
a656ba3343 Scheduler - removed templatization
Summary: Because it was unnecessary and also implemented wrong (if someone tried using a Schduler with something other then std::mutex, it would not compile). We can trivially add this if it ever becomes necessary.

Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D666
2017-08-17 16:10:41 +02:00
Goran Zuzic
58453794a6 [Rename diff] Change the terminology to match the rest.
Reviewers: sasa.stanko

Reviewed By: sasa.stanko

Subscribers: buda, lion

Differential Revision: https://phabricator.memgraph.io/D672
2017-08-17 15:37:12 +02:00
florijan
5c921a21c4 utils::auto_scope refactor
Summary: Changed on-scope-exit-mechanism from macro (with two auto-generated variables and an all-capturing lambda) to an explicitly created variable that takes an std::function argument.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D659
2017-08-11 09:43:10 +02:00
florijan
1d112e1141 GraphDbAccessor - style change
Summary: Not strictly neccessary, but it's been itching me. It took an hour.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D648
2017-08-09 16:09:08 +02:00
florijan
47c1cd6e3d Degree(Vertex) function added
Summary:
- added only one function for getting the total (in + out) vertex degree, it's required for the Ravelin use-case
- specific `degree_in` and `degree_out` functions can be added as necessary
- also fixed random_graph_generator bug (needed it for testing)

Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D652
2017-08-09 15:06:10 +02:00
Mislav Bradac
599e5651af Make debug_assert noop on NDEBUG
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D634
2017-08-04 15:05:26 +02:00
florijan
57dea09b5b Console and random-graph generation improvements
Summary: Random graph generation is now parallel. Slow, though.

Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D586
2017-08-02 12:23:51 +02:00
Teon Banek
16d94c8aaf Make the message mandatory for NotYetImplemented exception
Reviewers: florijan, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D565
2017-07-18 16:40:49 +02:00
Mislav Bradac
50cd53e5c9 Migrate timer to use walltime, instead of cputime
Summary:
 - Move some stuff to poc
 - Use steady_clock instead of system_clock

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D555
2017-07-17 13:42:44 +02:00
Mislav Bradac
0588de76bb Move unused datastructures to poc
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D526
2017-07-10 12:03:11 +02:00
Teon Banek
d9a724f0cd Plan scanning by label & property index for node atoms
Summary:
Add Filters class for storing additional info

Add FindOr to utils/algorithm.hpp

Use all collected labels when scanning by them

Collect label filters inside WHERE

Document the Filters class

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot, lion

Differential Revision: https://phabricator.memgraph.io/D515
2017-07-07 11:30:54 +02:00
Mislav Bradac
cbe6648eb8 Migrate command line args to gflgs in tests
Summary: Some other minor cleanups

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D523
2017-07-06 13:54:12 +02:00
florijan
c22ac38ea2 GraphDbAccessor - label+property index cardinalities for exact value and value ranges
Summary:
- added functionality to `GraphDbAccessor` for cardinality estimates
- changed all `GraphDbAccessor::Count...` functions to return `int64_t`
- added the need functionality into `LabelPropertyIndex`
- modified `SkipList::position_and_count` to accept a custom `equals` function. Equality could not be implemented using only the custom `less` because it compares a templated `TItem` with skiplist element type `T`, and is therefore not symetrical.

Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D521
2017-07-06 10:22:19 +02:00
Teon Banek
1f7a146b95 Replace utils::Border with utils::Bound
Summary:
The new Bound class does not have comparison operators defined. The
reason being, we want to support having values which we may not want to
compare. For example, having an Expression which should first be
evaluated and then compared.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D520
2017-07-04 10:00:38 +02:00
Dominik Gleich
e2f3aba332 Use GLogger instead of broken memgraph Logger.
Summary:
http://rpg.ifi.uzh.ch/docs/glog.html

Second phase before tests complete.

Delete logging test.

Finish relase loging.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D500
2017-06-21 15:33:24 +02:00
Teon Banek
13ae13bf43 Make asserts abort the program instead of exiting cleanly
Summary:
Since assertions should signify an abnormal program condition, it makes
sense for them to call `std::abort`. This way, we'll get a core dump
which can be inspected in a debugger.

Update utils/assert.hpp documentation

Reviewers: florijan, buda, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D475
2017-06-16 08:52:36 +02:00
Teon Banek
7de8fd4b1b Reorganize logging in memgraph_bolt.cpp
Summary:
Remove utils/stacktrace/log.hpp
The single function defined in log.hpp is used only in
memgraph_bolt.cpp. Therefore, the function has been moved and the file
removed.

Move utils/stacktrace/stacktrace.hpp one level up

Move some logging from memgraph_bolt to Server

Reviewers: buda, dtomicevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D465
2017-06-14 10:11:31 +02:00
Mislav Bradac
c8e25987b8 Add asserts to placeholder
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D463
2017-06-13 14:54:17 +02:00
Teon Banek
025e557204 Validate the --num_workers flag
Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D449
2017-06-09 16:16:38 +02:00
Teon Banek
04a5bdeb82 Remove unused config source code files
Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D442
2017-06-08 16:10:55 +02:00
Mislav Bradac
fea9031605 Refactor stripper
Summary: Fix tests

Reviewers: buda, florijan, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D435
2017-06-07 18:47:09 +02:00
Dominik Gleich
ebdee4e509 Skiplist garbage collector rework.
Summary:
Drawing:
https://drive.google.com/open?id=0B-W7PQZqMD9hcG04b0lKaGZGOWM

Reviewers: mislav.bradac, buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D395
2017-06-07 10:47:02 +02:00
Dominik Gleich
db5f4469fe Change time to duration.
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D403
2017-05-30 10:39:58 +02:00
Teon Banek
8d8ef19694 Raise in ParseDouble with empty string or float overflow
Summary:
The check is made more robust by using the `fail` accessor along with
`eof`.

Reviewers: florijan, buda, mislav.bradac, dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D376
2017-05-17 14:52:42 +02:00
matej.gradicek
e791e9b660 Summary:Added files for snapshot durability
Summary:
File buffer added

Implemented little more of snapshoter.

Resolved conflicts

More stuff implemented of snapshot durability.

More things in snapshoter.

Refactored, added comments

Merge branch 'dev' into durability_snapshot

Merge branch 'dev' into durability_snapshot

Resolved bug in scheduler, snapshoter is running in grpah_db.

Reviewers: mferencevic, buda, dgleich, mislav.bradac

Reviewed By: mferencevic, buda, dgleich, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D232
2017-05-17 08:42:00 +00:00
Teon Banek
15d7226515 Add classic and hierarchical visiting
Summary:
Merge utils/visitor directory into single file.
Rename Visitor to HierarchicalVisitor.
Add regular Visitor.
Split HierarchicalVisitor into LeafVisitor and CompositeVisitor.
Add more documentation on visitor pattern.
Make PostVisit and Visit return bool.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D364
2017-05-16 12:35:09 +02:00
florijan
2e5eccf197 Query::AST::Literal refactor. Repl and TypedValue mods.
Summary:
- Query::AST::Literal refactor (LiteralBase introduced, ListLiteral added)
- Repl now prints out list TypedValues properly
- TypedValue to string conversion refactors

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D327
2017-04-28 14:58:16 +02:00
Mislav Bradac
0fa4555cad Implement opencypher functions
Summary: Implement opencypher functions

Reviewers: florijan, buda, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D299
2017-04-26 13:27:42 +02:00
Teon Banek
893df584f6 Merge utils/exceptions into single file
Summary: Update documentation of `utils/exceptions.hpp`

Reviewers: mislav.bradac, florijan, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D300
2017-04-20 16:53:21 +02:00
Mislav Bradac
5434e79ea6 Merge string utils to one file
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D281
2017-04-18 17:39:58 +02:00
florijan
231b84e834 Query::Plan - Reconstruct fails after delete
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D287
2017-04-18 12:25:41 +02:00
Matej Ferencevic
f05fcd91c3 Refactored bolt session to use new decoder.
Summary:
Bolt buffer is now a template.

Communication worker now has a new interface.

Fixed network tests to use new interface.

Fixed bolt tests to use new interface.

Added more functions to bolt decoder.

Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D256
2017-04-15 16:35:09 +02:00
Dominik Gleich
461ea65ad4 Deferred deleter added.
Summary:
Add tests.

Merge branch 'dev' into remove_locks

Everything works. Refactor complete.

Documentation:
https://phabricator.memgraph.io/w/memgraph_implementation/indexing/

Reviewers: buda, mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D258
2017-04-14 17:46:16 +02:00
florijan
0a09a6ac64 Storage, Query - deletion bugfix and improvements
Summary:
Multiple attempts to delete a record dont crash anymore.
Deleting a vertex and its blocking edge in the same delete op now supported.
Utils::Assert - permanent_fail bug fix

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D282
2017-04-14 16:34:38 +02:00
Mislav Bradac
bba20cf89c Remove unneccessary files from utils
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D276
2017-04-13 17:55:33 +02:00
Teon Banek
b69c162c9f Add base QueryException class
Summary:
Also, inherit `TypedValueException` from `BasicException` instead of
`StacktraceException`.

This is meant as a basic API change, so that catching query exceptions becomes
simpler. The error message can be obtained from `what` method, inherited from
`BasicException`. In the future, we may extend the `QueryException`. It could
take column/row position in query source, as well as other information. Then
provide (or override `what`) a method to format the error message nicely for
the user.

Reviewers: florijan, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D267
2017-04-12 15:35:51 +02:00
florijan
593e4e72b9 Query::Plan::Aggregate
Summary:
- Aggregation LogicalOperation added, with tests.

- Added capabilities to TypedValue (hash, bool-equality)
to support std::unordered_map<TypedValue>.

- Removed some bad code from utils/hashing/fnv and added
a hashing function for collections.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D252
2017-04-11 15:12:06 +02:00
Teon Banek
804d0b09b9 Move typed_value to query top level and namespace it
Summary:
Upgraded old uses of TypedValue.
Also, implemented operator<< for EdgeAccessor and VertexAccessor.

Reviewers: buda, florijan, mislav.bradac, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D251
2017-04-11 08:50:46 +02:00
Dominik Gleich
fd2780155a Refactor GC to use scheduler.
Reviewers: buda, matej.gradicek

Reviewed By: matej.gradicek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D247
2017-04-10 15:57:45 +02:00
Teon Banek
7d0958b1a4 Add PreVisit method to utils::Visitor
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D242
2017-04-10 15:03:02 +02:00
Dominik Gleich
2032466e2a New version of scheduler.
Summary: New scheduler declaration.

Reviewers: matej.gradicek, buda

Reviewed By: matej.gradicek

Differential Revision: https://phabricator.memgraph.io/D246
2017-04-07 11:55:10 +02:00