Commit Graph

1977 Commits

Author SHA1 Message Date
Matej Ferencevic
d338e753c8 Extract Kafka integration to its own target
Reviewers: buda, teon.banek, msantl

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1583
2018-09-03 13:04:19 +02:00
Marko Budiselic
d9153921b8 Add distributed Id Cypher function
Reviewers: msantl, vkasljevic, teon.banek, ipaljak

Reviewed By: msantl, teon.banek

Subscribers: pullbot, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1477
2018-09-03 13:04:03 +02:00
Marko Budiselic
e80c49f856 Add dgp related docs
Summary:
The following documents related to dynamic graph partitioning
are added:
  * Dependency Diagram
  * Feature Specification
  * Feature Reference

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1584
2018-09-03 12:06:44 +02:00
Teon Banek
88de3422d0 Use stack allocation for cypher function arguments
Summary:
This is a simple change which modifies interface of
awesome_memgraph_functions to accept C-style pointer to array with
count. Doing things this way, allows us to easily try out different
allocation schemes for function arguments. In this diff, we are now
using stack allocation of arguments in a plain fixed size array. This is
done when the number of arguments is small. According to heaptrack, this
small change should yield noticeable improvements to heap usage.

Obviously, this doesn't solve the problem of heap allocations inside
TypedValue arguments themselves. These allocations appear when
std::string and std::vector is used inside TypedValue.

Micro benchmarks show that there is some performance improvement,
mostly around the limits of using array vs std::vector. The improvement is
more noticeable with multiple threads, due to primary gain being in avoiding
calls to memory allocation.

Reviewers: mtomic, msantl, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1581
2018-09-03 11:23:29 +02:00
Teon Banek
ee889b98fc Add default LCP save of primitive types in optional
Reviewers: msantl, mtomic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1579
2018-08-31 14:25:39 +02:00
Teon Banek
5c69ae8a0c Split database/counters into single node and distributed
Reviewers: msantl, vkasljevic, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1580
2018-08-31 13:52:36 +02:00
Teon Banek
ceffaf3d85 Move add_lcp and add_capnp to cmake/functions.cmake
Summary:
These functions were defined in multiple places. They are moved to
cmake/functions.cmake to keep only one source of truth.

Reviewers: mferencevic, msantl, mculinovic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1578
2018-08-30 16:34:39 +02:00
Teon Banek
c4958d9960 Extract distributed Expand
Summary: Depends on D1575

Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1576
2018-08-30 14:43:01 +02:00
Teon Banek
80f649bcd1 Extract distributed create operators
Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1575
2018-08-30 14:41:23 +02:00
Teon Banek
2c732f3ea1 Remove type method from GraphDb
Reviewers: msantl, vkasljevic, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1577
2018-08-30 09:47:12 +02:00
Ivan Paljak
146c35ec4a Add magic number to WAL and fix version consistency check
Summary: Recovery file is version inconsistent iff it starts with a correct magic number, has at least one integer written after that and that integer differs from kVersion.

Reviewers: mferencevic, ipaljak, vkasljevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1565
2018-08-29 16:53:10 +02:00
Matija Santl
57b84f2da3 Add kafka benchmark
Summary:
In order to add kafka benchmark, `memgraph_bolt.cpp` has been split.
Now we have `memgraph_init.cpp/hpp` files with common memgraph startup code.
Kafka benchmark implements a new `main` function that doesn't start a bolt
server, it just creates and starts a stream. Then it waits for the stream to
start consuming and measures the time it took to import the given number of
entries.

This benchmark is in a new folder, `feature_benchmark`, and so should any new
bechmark that measures performance of memgraphs features.

Reviewers: mferencevic, teon.banek, ipaljak, vkasljevic

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1552
2018-08-29 16:35:31 +02:00
Ivan Paljak
d106aff88f Implement full durability mode
Summary:
This diff introduces a new flags
* `--synchronous-commit`

The `--synchronous-commit` tells the WAL when should the deltas be flushed to
the disk drive. By default this is off and the WAL flushes deltas every `N`
milliseconds. If it's turned on, on every transaction end, commit or abort, the
WAL will first flush the deltas and only after that will return from ending a
transaction.

Reviewers: buda, vkasljevic, mferencevic, teon.banek, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1542
2018-08-29 16:05:07 +02:00
Marin Tomic
bff56bcf89 Revise user visible error messages
Summary: A quick clean-up of user visible error messages. Tried to make them gramatically correct by capitalizing the first word in the sentence and putting a dot at the end.

Reviewers: teon.banek, buda, ipaljak

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1571
2018-08-29 12:58:15 +02:00
Marin Tomic
28ba872668 Implement single node two-sided BFS
Reviewers: mculinovic, teon.banek, ipaljak, buda, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1567
2018-08-29 12:56:39 +02:00
Teon Banek
24e2b31367 Extract distributed BFS as a new operator
Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1570
2018-08-29 11:31:41 +02:00
Marin Tomic
f6a56db19e Small cleanup of auth queries
Summary:
Changed GRANT ROLE to SET ROLE. Now it is `SET ROLE FOR user TO ROLE` instead of `GRANT ROLE role TO user`. It makes more sense because our users can only have 1 role.

Changed REVOKE ROLE to CLEAR ROLE. Now it is `CLEAR ROLE FOR user` instead of `REVOKE ROLE role FOR user`. REVOKE ROLE would throw exception if user was not a member of role. CLEAR ROLE clears the role whatever it is. I find that the latter makes more sense combined with SET ROLE.

Changed `SHOW ROLE FOR USER user` to `SHOW ROLE FOR user`.

Changed `SHOW USERS FOR ROLE role` to `SHOW USERS FOR role`.

Reviewers: mferencevic, teon.banek, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1572
2018-08-29 10:07:55 +02:00
Marin Tomic
7b4196cd7c Allow keywords as stream names
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1573
2018-08-29 10:07:03 +02:00
Matej Ferencevic
e5028d91ea Improve Kafka test
Reviewers: msantl, mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1574
2018-08-29 09:48:22 +02:00
Teon Banek
4db62b18f0 Decouple visiting of distributed operators from regular
Summary:
Visitor pattern's main issue is cyclical dependency between classes that
are visited and the visitor instance itself. We need to decouple this
dependency if we want to open source part of the code, namely
non-distributed part. This decoupling is achieved through the use of
`dynamic_cast` in distributed operators. Hopefully the solution is good
enough and doesn't cause performance issues. An alternative solution is
to build our own custom double dispatch solution, but that will
basically boil down to our implementation of runtime type information
and casts.

Note, this only decouples the distributed operators. If and when we
decide that other operators shouldn't be open sourced, the same
`dynamic_cast` pattern should be applied in them also.

Depends on D1563

Reviewers: mtomic, msantl, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1566
2018-08-28 14:53:02 +02:00
Teon Banek
bb679a4b1d Move distributed operators to its own file
Summary:
This is the first step in separating the implementation of distributed
features in operators. Following steps are:

  * decoupling distributed visitors
  * injecting distributed details in operator state
  * minor cleanup or anything else that was overlooked

Reviewers: mtomic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1563
2018-08-28 14:47:14 +02:00
Teon Banek
6427902920 Extract distributed interpretation out of Interpreter
Reviewers: mtomic, mferencevic, msantl, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1560
2018-08-27 09:31:39 +02:00
Matej Ferencevic
e7cde4b4ef Make analyze_rpc_calls compatible with Cap'n Proto
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1561
2018-08-24 15:40:00 +02:00
Ivan Paljak
c0de946f91 Polish user_technical
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1562
2018-08-24 13:35:57 +02:00
Ivan Paljak
8c2ba7d564 Fix issue with wal_file initialization
Reviewers: msantl

Reviewed By: msantl

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1559
2018-08-24 11:40:58 +02:00
Ivan Paljak
74cfdd5c94 Remove couscous from user_technical
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1557
2018-08-24 11:03:37 +02:00
Matija Santl
d98ecca2da Verify ScanAllByLabelProperty
Summary:
We want to make sure that index doesn't miss any results.

This test inserts vertices in the db and checks that the number of inserted
vertices is the same as the number of results from a scan all and a scan all by
label property.

https://app.asana.com/0/478665099752750/762723827276182/f

Reviewers: ipaljak, vkasljevic, teon.banek

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1558
2018-08-24 10:47:35 +02:00
Marin Tomic
ce31ff4625 Fix possible memory leak in AstStorage ctor
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1553
2018-08-23 14:04:55 +02:00
Teon Banek
50c75c56a4 Add EXPLAIN to openCypher
Summary:
  * Move PlanPrinter from test to memgraph
  * Add explainQuery to MemgraphCypher.g4
  * Add Explain operator
  * Update changelog

Reviewers: mtomic, buda, ipaljak

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1555
2018-08-23 14:05:32 +02:00
Ivan Paljak
6615a9de53 Restructure user-technical
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D1547
2018-08-23 11:13:03 +02:00
Marko Culinovic
dab95af366 Extract stats to static lib
Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1546
2018-08-23 10:33:50 +02:00
Vinko Kasljevic
51c9f4d0d3 Refactor Cache and DataManager
Summary:
Instead of DataManager returning Cache which can fetch data when needed,
I refactored the code so that the Cache is simple wrapper around
unordered_map and the DataManager is one that is fetching data. Also Cache
is not visible from outside of the DataManager so we can add LRU policy
without changing anything else.

Reviewers: msantl, ipaljak, teon.banek, buda

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1545
2018-08-23 09:03:40 +02:00
Matej Ferencevic
f62d764649 Integrate driver tests with Apollo
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1551
2018-08-22 18:31:37 +02:00
Teon Banek
de16b7ee82 Add LaTeX template for presentations
Reviewers: buda, mtomic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1550
2018-08-22 14:51:27 +02:00
Matej Ferencevic
1b643958b6 Integrate auth checks into query execution
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1544
2018-08-22 11:44:09 +02:00
Ivan Paljak
0249a280f8 Handle durability versions on start-up
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1539
2018-08-22 11:30:42 +02:00
Teon Banek
da9dc10373 Add a LCP syntax highlighter for vim
Reviewers: buda, msantl, mtomic, mferencevic, ipaljak, mculinovic, vkasljevic, mpetricevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1548
2018-08-22 11:06:25 +02:00
Teon Banek
da3630f8a9 Add lcp-mode for Emacs
Reviewers: mtomic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1549
2018-08-21 17:22:24 +02:00
Ivan Paljak
8717eb0734 Add graph algorithnm concepts to user_technical
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1532
2018-08-20 15:32:44 +02:00
Teon Banek
8a43ac461c Add documentation of LCP features
Reviewers: mtomic, msantl, buda, ipaljak, vkasljevic, mferencevic, mculinovic, mpetricevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1527
2018-08-20 14:07:46 +02:00
Teon Banek
256fd038f9 Correctly parse pointers in lcp::parse-cpp-type-declaration
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1531
2018-08-20 13:59:44 +02:00
Matej Ferencevic
805b86f5e0 Fix memory warning errors
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1543
2018-08-17 16:29:46 +02:00
Marin Tomic
327c3c5d9b Add required privileges for query to Results
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1537
2018-08-16 15:59:10 +02:00
Matej Ferencevic
b448db245b Fix network buffer resize bug
Reviewers: mculinovic

Reviewed By: mculinovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1541
2018-08-14 16:33:31 +02:00
Matej Ferencevic
94ad18326c Implement leftover Auth queries
Reviewers: mtomic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1535
2018-08-14 13:00:39 +02:00
Matej Ferencevic
1febc15d68 Implement kafka integration test
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1538
2018-08-14 11:47:54 +02:00
Marko Culinovic
158f97206d Add bfs benchmark on pokec dataset
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1533
2018-08-13 09:50:39 +02:00
Matija Santl
cd3210fb9b Durability utility
Summary:
Added WAL and Snapshot explorer utility executables that read a wal or a
snapshot file and print the content of it, but the main purpose is that they
check it.

This is useful when debugging durability and want to know where it gets stuck.

Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: ipaljak, vkasljevic, teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1422
2018-08-10 09:21:09 +02:00
Matija Santl
4ee3db80b0 Add kafka documentation
Summary:
Updated the feature specs, the changelog and added a new section in
user technical.

Reviewers: mferencevic, mculinovic, buda, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1534
2018-08-09 16:52:52 +02:00
Matej Ferencevic
705c43a816 Add memgraph coverage script
Reviewers: mculinovic, msantl

Reviewed By: mculinovic

Differential Revision: https://phabricator.memgraph.io/D1536
2018-08-08 15:35:09 +02:00