Commit Graph

1945 Commits

Author SHA1 Message Date
Matej Ferencevic
f62d764649 Integrate driver tests with Apollo
Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1551
2018-08-22 18:31:37 +02:00
Teon Banek
de16b7ee82 Add LaTeX template for presentations
Reviewers: buda, mtomic, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1550
2018-08-22 14:51:27 +02:00
Matej Ferencevic
1b643958b6 Integrate auth checks into query execution
Reviewers: mtomic, teon.banek

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1544
2018-08-22 11:44:09 +02:00
Ivan Paljak
0249a280f8 Handle durability versions on start-up
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1539
2018-08-22 11:30:42 +02:00
Teon Banek
da9dc10373 Add a LCP syntax highlighter for vim
Reviewers: buda, msantl, mtomic, mferencevic, ipaljak, mculinovic, vkasljevic, mpetricevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1548
2018-08-22 11:06:25 +02:00
Teon Banek
da3630f8a9 Add lcp-mode for Emacs
Reviewers: mtomic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1549
2018-08-21 17:22:24 +02:00
Ivan Paljak
8717eb0734 Add graph algorithnm concepts to user_technical
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1532
2018-08-20 15:32:44 +02:00
Teon Banek
8a43ac461c Add documentation of LCP features
Reviewers: mtomic, msantl, buda, ipaljak, vkasljevic, mferencevic, mculinovic, mpetricevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1527
2018-08-20 14:07:46 +02:00
Teon Banek
256fd038f9 Correctly parse pointers in lcp::parse-cpp-type-declaration
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1531
2018-08-20 13:59:44 +02:00
Matej Ferencevic
805b86f5e0 Fix memory warning errors
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1543
2018-08-17 16:29:46 +02:00
Marin Tomic
327c3c5d9b Add required privileges for query to Results
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1537
2018-08-16 15:59:10 +02:00
Matej Ferencevic
b448db245b Fix network buffer resize bug
Reviewers: mculinovic

Reviewed By: mculinovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1541
2018-08-14 16:33:31 +02:00
Matej Ferencevic
94ad18326c Implement leftover Auth queries
Reviewers: mtomic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1535
2018-08-14 13:00:39 +02:00
Matej Ferencevic
1febc15d68 Implement kafka integration test
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1538
2018-08-14 11:47:54 +02:00
Marko Culinovic
158f97206d Add bfs benchmark on pokec dataset
Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1533
2018-08-13 09:50:39 +02:00
Matija Santl
cd3210fb9b Durability utility
Summary:
Added WAL and Snapshot explorer utility executables that read a wal or a
snapshot file and print the content of it, but the main purpose is that they
check it.

This is useful when debugging durability and want to know where it gets stuck.

Reviewers: dgleich, buda

Reviewed By: buda

Subscribers: ipaljak, vkasljevic, teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D1422
2018-08-10 09:21:09 +02:00
Matija Santl
4ee3db80b0 Add kafka documentation
Summary:
Updated the feature specs, the changelog and added a new section in
user technical.

Reviewers: mferencevic, mculinovic, buda, ipaljak

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1534
2018-08-09 16:52:52 +02:00
Matej Ferencevic
705c43a816 Add memgraph coverage script
Reviewers: mculinovic, msantl

Reviewed By: mculinovic

Differential Revision: https://phabricator.memgraph.io/D1536
2018-08-08 15:35:09 +02:00
Marin Tomic
2a5fce8464 Add rest of user auth queries
Reviewers: mferencevic, teon.banek

Reviewed By: mferencevic, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1522
2018-08-06 16:16:48 +02:00
Matej Ferencevic
ce306a4c21 Implement Kafka Python transform
Summary:
The Kafka Python transform functionality uses a Python script to transform
incoming Kafka data into queries and parameters that are executed against the
database. When starting the Python transform script it is started in a
sandboxed environment so that it can't do harm to the host system or the
database.

Reviewers: msantl, teon.banek

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1509
2018-08-06 13:53:26 +02:00
Ivan Paljak
3d1e5f2ebd Add version in WAL
Reviewers: msantl, vkasljevic, buda

Reviewed By: msantl, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1528
2018-08-06 10:50:40 +02:00
Ivan Paljak
ff5eba73e0 Add indexing concept and reference in user_techincal
Reviewers: buda, dtomicevic

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1485
2018-08-06 10:36:55 +02:00
Teon Banek
d51be890d2 Add default saving of known enums in LCP
Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1529
2018-08-03 14:17:32 +02:00
Teon Banek
0bb23df27b Update our code conventions and add required reading
Reviewers: mtomic, buda, ipaljak, mpetricevic, mferencevic, vkasljevic, mculinovic, msantl

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1525
2018-08-02 16:07:39 +02:00
Ivan Paljak
da89dcb3ee Add tutorial articles to docs/user_technical
Summary:
Quick start summary:
  - Quick Start
  -- Installation
  --- Debian Package Installation
  --- RPM Package Installation
  --- Docker Installation
  ---- Note about Named Volumes
  ---- Note for OS X/macOS users
  -- Querying
  --- Supported Languages
  -- Telemetry
  -- Where to Next

Example (TED/football/europe) summary:
  - Article Name
  -- Introduction
  -- Data Model
  -- Importing the Snapshot
  -- Example Queries

Programmatic Querying summary:
  - Programmatic Querying
  -- Supported Languages
  -- Secure Sockets Layer (SSL)
  -- Examples
  --- Python Example
  --- Java Example
  --- JavaScript Example
  --- C# Example
  -- Limitations
  --- Multiple Users & Authorization

Reviewers: buda, dtomicevic, teon.banek, mtomic

Reviewed By: buda, teon.banek

Differential Revision: https://phabricator.memgraph.io/D1516
2018-08-02 13:57:10 +02:00
Ivan Paljak
4d3c0a2fa5 Fix terminology consistencies (vertex to node and relationship to edge)
Reviewers: buda, mtomic

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1526
2018-08-02 13:43:23 +02:00
Marin Tomic
23d4391c29 Fix segfault with all/single predicate
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1523
2018-08-02 10:43:22 +02:00
Marin Tomic
265c22f9bc Move ted_data.cyp out of memgraph/docs/user_technical
Summary: It seems like it is not used anywhere. Queries necessary for generating a snapshot are in release/examples/queries/TEDTalk.

Reviewers: mculinovic, buda

Reviewed By: mculinovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1524
2018-08-02 10:42:16 +02:00
Teon Banek
eeb03b132f Remove dependency on distributed from durability
Reviewers: msantl, vkasljevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1521
2018-08-02 09:40:34 +02:00
Teon Banek
09bc9cb164 Update save/load hooks in LCP
Reviewers: mtomic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1517
2018-08-02 09:33:33 +02:00
Marko Budiselic
4fd5b1ebc4 Add support for distributed tck tests
Reviewers: mferencevic, ipaljak, mculinovic

Reviewed By: mculinovic

Subscribers: teon.banek, msantl, pullbot

Differential Revision: https://phabricator.memgraph.io/D1395
2018-08-01 10:51:49 +02:00
Ivan Paljak
e89fe953b3 Fix inconsistencies in existing docs/user_technical
Summary:
  - Uppercase keywords
  - Add newlines before listing items

Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D1520
2018-07-31 17:05:09 +02:00
Marko Culinovic
b59edf1640 Add support for node label, rel type and quote
Summary:
New flags are introduced.
1) `node-label` flag can be specified multiple times to add
extra labels to all nodes
2) `relationship-type` overwrites relationship types from file
with given flag value
3) `quote` is used to specify quotation character

Added tests for new flags.

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1514
2018-07-31 13:28:58 +02:00
Teon Banek
02e7cbf16c Add pure interface class to RecordAccessor
Summary:
Since RecordAccessor is often instantiated and copied, using a
factory-like function to allocate concrete types on the heap would be
too costly. The approach in this diff uses a Strategy pattern (see
"Design Patterns" by Gamma et al.), where the Strategy interface is
given as RecordAccessor::Impl. Concrete implementations are then created
for each GraphDb. This allows us to instantiate the concrete
RecordAccessors::Impl *once* and *share* it among all RecordAccessors.

Reviewers: msantl, vkasljevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1510
2018-07-31 08:48:28 +02:00
Matej Ferencevic
5d73c64fd8 Fix alter user query
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1519
2018-07-30 17:02:27 +02:00
Matej Ferencevic
6546d1de07 Fix handling of Bolt init auth metadata
Summary:
Our implementation of the Bolt protocol now correctly handles INIT message
metadata used for authentification. The related description from the Bolt
standard is: 'The token must contain either just the entry {"scheme": "none"}
or the keys scheme, principal, and credentials. Example {"scheme": "basic",
"principal": "user", "credentials": "secret"}". If no scheme is provided, it
defaults to "none".'

Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1518
2018-07-30 16:51:41 +02:00
Teon Banek
f7f7ccde60 Remove boost from the codebase
Summary:
Since we switched to Cap'n Proto serialization there's no need for
keeping boost around anymore.

Reviewers: mtomic, mferencevic, buda

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1515
2018-07-30 15:34:12 +02:00
Teon Banek
8d934ed801 Remove raft from the codebase
Summary:
The raft implementation has been stale for a while now. It doesn't
compile, nor uses Cap'n Proto for serialization. In the future we would
probably rewrite it, so it doesn't need to be part of the repo at this
moment.

Reviewers: mferencevic, mtomic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1513
2018-07-30 14:14:10 +02:00
Teon Banek
7262f7c5c1 Remove couple of linking redundancies in CMakeLists
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1512
2018-07-30 13:55:30 +02:00
Marin Tomic
9a6801cb98 Separate auth/stream queries from rest of cypher
Summary: first step for a cleaner query front end that might be easier to open source

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1511
2018-07-30 12:29:36 +02:00
Matej Ferencevic
2ecb660790 Initial implementation of authentication
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: mtomic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1488
2018-07-27 13:08:17 +02:00
Marin Tomic
7e92a7f41c Split lexer and parser grammars
Summary: first step of grammar cleanup

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1508
2018-07-26 10:33:25 +02:00
Teon Banek
9f460914ed Separate distributed implementation of GraphDbAccessor
Summary:
GraphDbAccessor is now constructed only through GraphDb. This allows the
concrete GraphDb to instantiate a concrete GraphDbAccessor. This allows
us to use virtual calls, so that the implementation may be kept
separate. The major downside of doing things this way is heap allocation
of GraphDbAccessor. In case it turns out to be a real performance
issues, another solution with pointer to static implementation may be
used.

InsertVertexIntoRemote is now a non-member function, which reduces
coupling. It made no sense for it to be member function because it used
only the public parts of GraphDbAccessor.

Reviewers: msantl, mtomic, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1504
2018-07-26 09:16:39 +02:00
Matej Ferencevic
e28fd2025d Rename bolt::DecodedValue to bolt::Value
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1506
2018-07-24 15:59:20 +02:00
Matej Ferencevic
5c13c2d22c Change Kafka transform script download logic
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1505
2018-07-24 12:33:48 +02:00
Dominik Gleich
0681040395 Durability distributed wal
Summary:
Recover only snapshot

Return recovery info on worker recovery

Update tests

Start wal recovery

Single node wal split

Keep track of wal possible to recover

Fix comment

Wal tx intersection

Merge branch 'master' into sync_wal_tx

Reviewers: buda, ipaljak, dgleich, vkasljevic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1489
2018-07-24 08:55:13 +02:00
Marko Budiselic
5db39ac501 Add initial version of dynamic partitioning feature spec
Reviewers: dgleich, dtomicevic, msantl

Reviewed By: msantl

Differential Revision: https://phabricator.memgraph.io/D1388
2018-07-23 17:39:22 +02:00
Marin Tomic
98350274ad Fix repl/single_query
Reviewers: mferencevic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1503
2018-07-20 14:31:35 +02:00
Teon Banek
2c50ea41d5 Split GraphDb to distributed and single node files
Summary:
This change, hopefully, simplifies the implementation of different kinds
of GraphDb. The pimpl idiom is now simplified by removing all of the
crazy inheritance. Implementations classes are just plain data stores,
without any methods. The interface classes now have a more flat
hierarchy:

```
    GraphDb (pure interface)
         |
    +----+---------- DistributedGraphDb (pure interface)
    |                         |
Single Node             +-----+------+
                        |            |
                      Master       Worker
```

DistributedGraphDb is used as an intermediate interface for all the
things that should work only in distributed. Therefore, virtual calls
for distributed stuff have been removed from GraphDb. Some are exposed
via DistributedGraphDb, other's are only in concrete Master and Worker
classes. The code which relied on those virtual calls has been
refactored to either use DistributedGraphDb, take a pointer to what is
actually needed or use dynamic_cast. Obviously, dynamic_cast is a
temporary solution and should be replaced with another mechanism (e.g.
virtual call, or some other function pointer style).

The cost of the above change is some code duplication in constructors
and destructors of classes. This duplication has a lot of little tweaks
that make it hard to generalize, not to mention that virtual calls do
not work in constructor and destructor. If we really care about
generalizing this, we should think about abandoning RAII in favor of
constructor + Init method.

The next steps for splitting the dependencies that seem logical are:

  1) Split GraphDbAccessor implementation, either via inheritance or
     passing in an implementation pointer. GraphDbAccessor should then
     only be created by a virtual call on GraphDb.
  2) Split Interpreter implementation. Besides allowing single node
     interpreter to exist without depending on distributed, this will
     enable the planner and operators to be correctly separated.

Reviewers: msantl, mferencevic, ipaljak

Reviewed By: msantl

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D1493
2018-07-20 10:48:38 +02:00
Marin Tomic
4a3808b982 Remove keyword lowercasing from stripper
Summary: This is a simple thing that enables us to use keywords as symbolic names. A bad thing is that planning will be done for queries which differ only in keyword case (e.g. "MATCH (x) RETURN x" and "match (x) return x"), but that should not be a significant performance impact as queries are done programmaticaly when high throughput is expected and we don't expect letter case change in that case.

Reviewers: buda, mferencevic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1496
2018-07-19 17:08:45 +02:00