Commit Graph

1278 Commits

Author SHA1 Message Date
Mislav Bradac
e2a53b82c4 Use our version of cppitertools
Summary:
Our version of cppitertools should improve performance of aggregation
benchmarks by ~50%.

Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D668
2017-08-17 14:38:11 +02:00
Matej Ferencevic
26c4651897 Removed global state from long_running stress test.
Reviewers: florijan, buda

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D637
2017-08-17 10:06:16 +02:00
florijan
71d8062af1 GraphDb - index garbage collection fix
Summary:
A single line (graph_db.cpp:109 in the new code) was missing. This should have been done in D355 (made by DGleich, approved by Flor AND Buda AND Mislav :D).

Converted a lambda to a method for convenience.

Reviewers: buda, dgleich, mislav.bradac

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D665
2017-08-17 09:14:00 +02:00
Sasa Stanko
23652fee15 Example of starting simple distributed system
Reviewers: zuza, buda, lion

Reviewed By: zuza

Subscribers: lion, pullbot

Differential Revision: https://phabricator.memgraph.io/D663
2017-08-16 15:20:21 +02:00
Teon Banek
9e85ccc892 Iterate matching variants and limit expanding
Summary:
Matching variants are generated iteratively and limited. Additionally,
When generating expansions, if there is nothing to continue on the
expanded nodes, simply append all the remaining expansions to the
currently generated one. This should speed up queries with large number
of `MATCH` clauses.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D612
2017-08-16 11:34:04 +02:00
Teon Banek
777b6a8101 Store paths to glog include and library in CMakeLists
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D615
2017-08-16 11:32:52 +02:00
Teon Banek
2ec1080ac9 Add pretty printer to gdb for TypedValue
Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D619
2017-08-16 11:32:25 +02:00
Sasa Stanko
43015f164c Implement basic master and worker
Summary:
Add FindChannel to System which works over the network

Give example of master and worker

Add script to start master and worker

Rewrite worker and master

Reviewers: zuza, buda

Reviewed By: zuza

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D662
2017-08-14 17:21:31 +02:00
Goran Zuzic
98a1b652d1 Change network_chat to work with events
Reviewers: sasa.stanko, buda

Reviewed By: sasa.stanko

Subscribers: mferencevic

Differential Revision: https://phabricator.memgraph.io/D661
2017-08-14 11:50:28 +02:00
Goran Zuzic
34121a676c Connected RemoteChannel to Callbacks by removing type_index from where it isn't needed
Reviewers: sasa.stanko, buda

Reviewed By: sasa.stanko

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D660
2017-08-11 17:01:33 +02:00
florijan
0703724295 GraphDbAccessor::BuildIndex - deadlock bugfix
Reviewers: buda, mislav.bradac, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D658
2017-08-11 10:34:18 +02:00
florijan
5c921a21c4 utils::auto_scope refactor
Summary: Changed on-scope-exit-mechanism from macro (with two auto-generated variables and an all-capturing lambda) to an explicitly created variable that takes an std::function argument.

Reviewers: buda, mislav.bradac, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D659
2017-08-11 09:43:10 +02:00
Sasa Stanko
b5a61681d2 Add few connector tests
Reviewers: zuza, buda

Reviewed By: zuza

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D656
2017-08-10 10:36:51 +02:00
Goran Zuzic
3fecc4f357 Callback Chainer Convenience Class
Reviewers: buda, sasa.stanko, lion

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D653
2017-08-09 16:54:58 +02:00
Goran Zuzic
2446571770 Added a README.md file in the experimental/distributed subdirectory.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D654
2017-08-09 16:43:57 +02:00
Mislav Bradac
98241f99a6 Fix remaining falling tests in release mode
Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D655
2017-08-09 16:39:21 +02:00
florijan
1d112e1141 GraphDbAccessor - style change
Summary: Not strictly neccessary, but it's been itching me. It took an hour.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D648
2017-08-09 16:09:08 +02:00
florijan
47c1cd6e3d Degree(Vertex) function added
Summary:
- added only one function for getting the total (in + out) vertex degree, it's required for the Ravelin use-case
- specific `degree_in` and `degree_out` functions can be added as necessary
- also fixed random_graph_generator bug (needed it for testing)

Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D652
2017-08-09 15:06:10 +02:00
Matej Ferencevic
6a2d123f30 Temporary IDs now sent to Bolt client. Documentation updated.
Reviewers: buda, florijan

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D643
2017-08-09 14:45:09 +02:00
Goran Zuzic
5c4c4919a2 Better OnEvent usability
Reviewers: lion, buda, sasa.stanko

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D650
2017-08-09 11:59:17 +02:00
florijan
caff5e4066 map_operations.feature added to QA
Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D651
2017-08-09 11:53:35 +02:00
Sasa Stanko
d88380c602 Replace SendHelper with Send call
Reviewers: zuza

Reviewed By: zuza

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D649
2017-08-08 18:47:06 +02:00
Goran Zuzic
045e14e139 Callback interface consistency changes
Summary:
1. add a test for unsubscription service
2. change the callback to accept const subscription&
3. renamed Send -> SendHelper
4. created a more usable Send
5. changed the Send usages

Reviewers: buda, lion, sasa.stanko

Reviewed By: sasa.stanko

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D647
2017-08-08 17:22:26 +02:00
Sasa Stanko
54b7851f84 Add typed version of pop/await event functions
Reviewers: zuza

Reviewed By: zuza

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D646
2017-08-08 16:47:44 +02:00
Goran Zuzic
6ce504349f Adding callback by type (Work in Progress)
Summary:
Merge branch 'dev' of https://phabricator.memgraph.io/diffusion/MG/memgraph into dev

Finished adding callback by type.

Reviewers: buda, lion, sasa.stanko

Reviewed By: sasa.stanko

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D644
2017-08-08 15:56:49 +02:00
Goran Zuzic
16170d22da Set .ycm_extra_config to work with other editors
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D645
2017-08-08 15:51:56 +02:00
florijan
b8957c999d Map type now supported
Summary:
- MapLiteral added
- PropertyLookup on maps added

This is the basic implementation, missing are:
- unit tests
- feature and TCK tests
- documentation
- changelog

That stuff is coming. Please review the implementation (Mislav).

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D640
2017-08-08 14:34:26 +02:00
Sasa Stanko
bfe9ec0ab5 Fix bug in StopServer function
Summary: Add test for system start and shutdown

Reviewers: mferencevic, zuza

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D642
2017-08-08 13:19:22 +02:00
Sasa Stanko
82f1cb333d Test for exception throws when channel name exist
Reviewers: zuza

Reviewed By: zuza

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D641
2017-08-07 18:00:50 +02:00
Mislav Bradac
7fe799f232 Fix tests that expect debug_assert to fail
Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D639
2017-08-07 16:38:15 +02:00
Matej Ferencevic
2ba0f0cdea Initial version of distributed networking stack.
Reviewers: buda, lion

Reviewed By: buda

Subscribers: pullbot, zuza

Differential Revision: https://phabricator.memgraph.io/D636
2017-08-07 16:26:43 +02:00
Goran Zuzic
59b395626c Implemented EventStream::Close
Summary:
1. Consistency Changes in Reactors
2. Renamed Channel -> Connector where appropriate

Merge branch 'dev' of https://phabricator.memgraph.io/diffusion/MG/memgraph into dev

Reviewers: lion, buda

Reviewed By: buda

Subscribers: pullbot, sasa.stanko

Differential Revision: https://phabricator.memgraph.io/D638
2017-08-07 14:12:55 +02:00
Teon Banek
0b8d71ee8f Plan BreadthFirstExpand
Summary:
Test planning BreadthFirstExpand

Add bfs tests to memgraph qa

Allow pointers in `print-operator-tree` for gdb

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D618
2017-08-07 10:44:10 +02:00
Goran Zuzic
b68265823c Removed Double Locking, Simplified Ownership or LocalEventStream, ....
Summary:
1. added TAGS to .gitignore
2. Commented: LockedGetPendingMessages is not fair.
3. Removed lock from EventQueue dtor.
4. Changed recursive_mutex -> mutex in Reactor and EventQueue.
5. [main change] made LocalEventStream a composite part of EventQueue, the new and only way to close EQ is to call close() in Reactor
6. [main change] make LocalChannel non-unique, it can be freely created and shared around.
7. renamed Reactor::Close into Reactor::CloseConnector

Reviewers: lion, buda

Reviewed By: lion

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D630
2017-08-04 17:58:22 +02:00
Mislav Bradac
599e5651af Make debug_assert noop on NDEBUG
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D634
2017-08-04 15:05:26 +02:00
Mislav Bradac
cc8e9f2996 Fix of stupid, stupid bug
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: florijan, pullbot

Differential Revision: https://phabricator.memgraph.io/D635
2017-08-04 15:03:59 +02:00
florijan
bce4407ebe CostEstimator - adding new ScanAll and Expand ops
Summary:
Now all ScanAll and Expand ops are covered by the cost estimator. For ScanAll with indices cost estimation is pretty good, for new Expand ops it is tragically bad (Expand to the power of expansion depth, plus arbitrary filtering). Static cost estimation is wrong wrong wrong.

Currently cost estimation of even trivial plans that use indices is wrong because the planner leaves filtering expressions that are implicitly handled by the index in the operator tree, IIRC. Tasking Teon to revise this, even though I'm not sure how bad an influence this has on cost estimation and it's use in plan choosing.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D633
2017-08-04 14:32:22 +02:00
florijan
6c22caa80e Query::Plan::Expand with preceeding optional bug fix
Summary:
This diff contains a bug fix for the expansion operators that are currently on dev.
More importantly, it proposes end-to-end testing for edge-cases for which it's a
pain to write single-phase tests. In my opinion this is OK, you're all reviewers so
you can comment.

The test relies on left-to-right query execution. We need this guarantee in tests
like this. I propose renaming "RuleBasedPlanner" to "LeftToRightPlanner" to make
this explicit. As Teon is not here at the moment, will make this a task/discussion.

Reviewers: buda, mislav.bradac, teon.banek, lion

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D626
2017-08-04 12:16:26 +02:00
florijan
0f73c2451b Query::Plan::ExpandVariable - graph state bugfix
Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D631
2017-08-04 12:12:44 +02:00
Matej Ferencevic
7939b04117 Removed unused template parameter from network stack.
Reviewers: mislav.bradac, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D632
2017-08-04 11:12:08 +02:00
Mislav Bradac
d750970381 Lock when accessing parser to prevent antlr bugs
Reviewers: buda, mferencevic

Reviewed By: buda, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D625
2017-08-03 16:38:08 +02:00
Matej Ferencevic
71ded22b65 Removed Dbms and QueryEngine from the Network stack.
Reviewers: buda, mislav.bradac

Reviewed By: buda, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D628
2017-08-03 16:33:10 +02:00
florijan
e45ae4c4b6 Query::Plan::ExpandBreadthFirst added
Summary:
Not complete (but review can start):
- implementation should be done
- still need to finish tests
- documentation missing

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D616
2017-08-03 16:22:38 +02:00
Mislav Bradac
da0e4a5b12 Implement explicitly started transactions
Summary: Fix tests

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D623
2017-08-03 16:17:19 +02:00
florijan
bb5d06e276 Skiplist::position_and_count fix
Summary: Fixed bug for SkipList::position_and_count for an item lesser then all skiplist elements.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D629
2017-08-03 14:57:11 +02:00
florijan
31ee8a148f Long running test speedup
Summary: Index added to long running test to speed up edge creation and vertex deletion.

Reviewers: buda, mferencevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D621
2017-08-03 14:48:55 +02:00
Matej Ferencevic
797fa10f7c Added antlr sigsegv test.
Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D622
2017-08-03 13:06:18 +02:00
Matej Ferencevic
a11c9885ad Merged experimental repo.
Summary:
Fixed distributed init.
Add CMakeLists to build experimentall/distribuedClosing unused Channels, work in progress.
Make System the owner of Reactor.
This entails changing shared_ptr -> unique_ptr and some pointers to references.
Merged experimental repository into memgraph.
Moved experimental repo to experimental directory.
Removed obsolete experimental files.
Added comments.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Subscription service unsubscribe.
Add Close method on EventStream.
Add placeholder for the configuration class.
Remove comments.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Clean-up parameters for EventQueue.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add Channel::serialize method implementation.
Merge.
Add docs on event stream.
Clang-format merge conflicts.
First implementations of serialize methods.
Add hostname, port, and names as methods in Channel base class.
Add reactor name and name methods to LocalChannel.
Add reactor name to LocalChannel.
Add name to LocalChannel.
Add serialization service.
Serialize_test removed.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Move Message to the end of communications files.
Full example of serialization with cereal.
Fix constructor calls.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Avoid using `FindChannel` in the transaction code.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Init script creates libs folder.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add System pointer to Network.
serialized_test binary is removed from the repo.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Cereal basic example.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Callbacks finished.
Always open the main channel by default.
Fixed callbacks, wrong number of emplace arguments.
Callbacks WIP.
Raise connector mutex to reactor level.
Add argument to LockedPush.
Fix data race in connector closing.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add functional header.
Fixed to make the changes work.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Refactored connectors into Reactors
Use shared pointer for the mutex.
Rename to Open and Close in implementation file.
Rename Create to Open and Destroy to Close.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Adding callback to Reactors; work in progress
Add stubs for asynchronous channel resolution.
Add stubs for the networking service.
Replace reactor pointers with shared ptrs, disable System assignment.
Forbid assignment.
Replace raw channel pointers with shared pointers.
Replace raw event stream pointer with shared pointer.
Rename default stream name.
Use recursive mutex in System.
Main uses Spawn method. All files are formatted.
Move thread local to a cpp file.
Work in progress on Spawn method.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Kill out graph.hpp to make it compile
Add Spawn method prototype.
Fix return type.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add method used to create nameless channels.
Add format script.
Introduce the Reactor base class.
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add compile script.
added comments about class terminology
Spinner rewrite (graph data structures and algo)
Organize Spinner code
Create working version
Improves Spinner implementation and testing
Spinner fix
.arcconfig
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add graph
Spinner work
Spinner added
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Add communication
.clang-format + ycm config.
Init. Distributed hackaton.
Implementation of lock-free list from Petar Sirkovic.
pro compiler
Merge branch 'master' of https://phabricator.memgraph.io/source/experimental
Implement Match
Add test data.
Insert quotes before and after props and labels
Multiple node declarations, along with edges.
After merge.
Node property creations work now.
Bug fix in visitor
After merge.
Implement node creation with labels.
Implement boolean operators
Tidy up ImplementedVistor.
Implement expression6 (addition)
Implement basic type visitor functions
Cypher Visitor Implementation class created.
Fix style.
Fix template synrax in main.cpp
Merge remote-tracking branch 'origin/master'
Add pretty_print
Update main and BaseVisitor to present return value.
Headers included. Temporary fix.
Antlr4 module reintroduced.
Updateao git config.
Fix trailing space.
CMake 2.8 fix rerolled, 3.1 minimum version req.
Fix for Cmake version 2.8 compatibility.
Build works.
Tidy src folder. Include generated files for antlr.
Included antlr generated files.
Changed directory structure.
Cmake: include subdirectory.
GenerateRuntime, partial.
Add GenerateParser target to cmake.
Remove main.cpp
Merge remote-tracking branch 'origin/master'
Add requirements
Main file added. Run the lexer and parser with this.
Add antlr_generated to baby_compiler
Experimental memory_tracker and opencypher tck tests

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D627
2017-08-03 12:48:19 +02:00
Mislav Bradac
6bc9deba5f Replace map with bitset in stripped query
Summary:
Implement trie and use it in stripper

Make it nicer

Reviewers: buda, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D614
2017-08-02 22:17:36 +02:00
florijan
57dea09b5b Console and random-graph generation improvements
Summary: Random graph generation is now parallel. Slow, though.

Reviewers: teon.banek, buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D586
2017-08-02 12:23:51 +02:00