Commit Graph

667 Commits

Author SHA1 Message Date
Teon Banek
796e4f50e5 Add named path symbols to new symbols of a match
Summary:
Previously, named path symbols remained untracked as `new_symbols` during planning. This meant that
operator `Optional` would be left unaware of those symbols, and therefore not reset them to `Null`
if optional matching failed.

Test Optional operator will be aware of path symbols

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D974
2017-11-10 15:09:47 +01:00
Teon Banek
7fabdda938 Fix an error where null AND false returned null
Summary:
Tests have been updated to catch this error and other behaviour. Other
than this change, `AND` should behave as before.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D970
2017-11-09 16:46:25 +01:00
Teon Banek
4b8076f41a Fix issues with lambda symbols and returning *
Summary:
Correctly propagate user declared symbols for lambda.
Unbind lambda symbols after inlining filters.
Also update unit tests.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D971
2017-11-09 16:46:06 +01:00
Marin Tomic
dddfe52a45 Add Watchdog utility
Summary: see documentation

Reviewers: mislav.bradac, buda, teon.banek, dgleich

Reviewed By: dgleich

Subscribers: teon.banek, dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D951
2017-11-09 11:23:11 +01:00
Teon Banek
b36386cfe9 Prevent double termination signals causing crashes
Summary:
Use sigaction to register signal handlers.
This is preferred over `signal` function, according to `man 3p signal`.
Add global sig_atomic_t flag when shutting down.
Block other signal handlers when shutting down.

Reviewers: mislav.bradac, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D943
2017-11-09 10:41:16 +01:00
Matej Ferencevic
0ae88bc06b Added apollo data to harness.
Reviewers: dgleich, teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D963
2017-11-07 22:10:27 +01:00
Matej Ferencevic
3fc1ef2770 Reduced database wait times in harness.
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D964
2017-11-07 10:54:30 +01:00
Matej Ferencevic
56fcca0412 Fixed memgraph snapshot path in LDBC test.
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D960
2017-11-06 16:03:47 +01:00
Dominik Gleich
b3aa6bea17 Disable measurments output.
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D957
2017-11-06 09:58:33 +01:00
florijan
f2aad117cf RingBuffer - buffer on heap instead of stack
Summary: RingBuffer - buffer in vector

Reviewers: mislav.bradac, buda, dgleich

Reviewed By: mislav.bradac

Differential Revision: https://phabricator.memgraph.io/D952
2017-11-02 12:07:23 +01:00
Mislav Bradac
b39231f7df Add tests for string utils
Reviewers: florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D941
2017-10-30 14:15:51 +01:00
Mislav Bradac
00f76d1391 Add nodes and relationships functions
Reviewers: teon.banek, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D942
2017-10-30 12:38:12 +01:00
Teon Banek
55456b4214 Remove Dbms
Summary:
Remove name from GraphDb.
Take GraphDb in query test macros instead of accessor.
Add is_accepting_transactions flag to GraphDb.

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D940
2017-10-30 12:33:29 +01:00
Dominik Gleich
7ab6da5f6a Update query suite.
Summary: Add a perf flag.

Reviewers: mferencevic, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot, mferencevic

Differential Revision: https://phabricator.memgraph.io/D937
2017-10-30 09:55:23 +01:00
florijan
be9c875fa9 Durability - snapshot summary refactor
Summary:
- Removed durability::Summary because it was wired into reader and stopped me from recovering WAL files.
- Refactored and renamed BufferedFile(Reader/Writer) to HashedFile(Reader/Writer).
- Vertex and edge counts in the snapshot are now hashed.

Breaking snapshot compatibility again (hashing), but since the previous version was not released, and we are not caching snapshots, the previous version does not need to be supported.

Reviewers: teon.banek, mislav.bradac, buda

Reviewed By: teon.banek, mislav.bradac

Subscribers: dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D932
2017-10-27 13:51:02 +02:00
Mislav Bradac
e6d3edf9a9 Fix distributed reactors
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D935
2017-10-27 11:26:04 +02:00
Teon Banek
df72861b90 Make csv_to_snapshot more user friendly
Summary:
Time csv_to_snapshot conversion and log it.
Check if writing csv_to_snapshot failed.
Extract LoadConfig from memgraph_bolt to config.hpp.
Read memgraph config in csv_to_snapshot for snapshot_directory.
Rename csv_to_snapshot to mg_import_csv.
Add tests for tools.
Run tools tests in apollo.

Reviewers: mislav.bradac, florijan, mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D931
2017-10-26 09:37:56 +02:00
Mislav Bradac
fe3d752904 Revise reactors code
Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D923
2017-10-25 15:36:18 +02:00
florijan
9f7ef8e0e9 utils::string - split functions extended
Summary:
- RSplit added
- split limits added
- tests

Reviewers: teon.banek, dgleich

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D930
2017-10-25 15:26:31 +02:00
Matej Ferencevic
df4933ea0f Disable hyperthreading on Apollo for macro benchmark tests.
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D929
2017-10-25 11:32:10 +02:00
Teon Banek
db34cb2a40 Refactor collecting filters during planning
Summary:
Move QueryParts and Filters to a new file.
Reorganize FilterInfo struct.
Remove label filter if we do indexed scan by label.
Remove property filter used in indexed scan.

Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D915
2017-10-24 14:13:14 +02:00
Matej Ferencevic
cbf8dacc11 Add parallel macro benchmark suite
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D922
2017-10-24 13:53:09 +02:00
florijan
add801a80a Ring buffer added
Summary: Locked version. There are some benchmarks, it seems the lock won't be the bottleneck in the WAL (DB ops causing WAL delta insertions into it will be slower, flushing the WAL be slower).

Reviewers: buda, mislav.bradac, dgleich

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D919
2017-10-24 08:53:35 +02:00
florijan
4460dd79f9 Durability - snapshot magic num, version and tx snapshot added
Summary:
New snapshot structure:
- magic number
- snapshot version (old-version recovery not yet implemented)
- transaction snapshot (will be used in the WAL)
- the rest is as before (indices, vertices, edges)

Not backward compatible with the old snapshotting.

Does not improve error handling (user feedback). A task for that has been added.

Reviewers: buda, mislav.bradac, mferencevic, teon.banek

Reviewed By: teon.banek

Subscribers: teon.banek, dgleich, pullbot

Differential Revision: https://phabricator.memgraph.io/D912
2017-10-23 15:51:35 +02:00
Mislav Bradac
03c50bada9 Pass assigned cpus to harness by flags
Summary:
Since we have different kind of workers in Apollo we should pass
assigned cpus to harness from apollo generate script and not define them
in harness or in benchmarks.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D916
2017-10-19 13:29:45 +02:00
Mislav Bradac
088a177939 Fix mvcc tests
Summary: Daily release build was failing because of these tests.

Reviewers: florijan, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D917
2017-10-19 12:58:23 +02:00
Mislav Bradac
19a44a7d94 Close session on timeouts
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D897
2017-10-17 14:34:31 +02:00
florijan
5525af558b Long running test - RemoveEdges never called
Summary: RemoveEdges is an extremely slow operation because it iterates over all the vertices to find the appropriate edge. It kind of messes up the DB usage. This diff stops it ever getting called, but does not delete the function. We might want it to happen **very rarely**, but it's probably best never to call it.

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D910
2017-10-17 11:59:39 +02:00
florijan
14fabe2125 Removed one function from version_list
Summary: It wasn't used in MG, only in tests.

Reviewers: buda, dgleich, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D909
2017-10-17 11:21:56 +02:00
Mislav Bradac
f6b1c9b6e2 Fix mvcc tests
Summary: This diff fixs failing release daily builds (hopefully)..

Reviewers: buda, dgleich

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D908
2017-10-16 17:19:32 +02:00
Teon Banek
06b0111ddc Remove unused stuff from CMakeLists
Summary: In the process, make experimental/distributed compilable.

Reviewers: mislav.bradac, buda, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D906
2017-10-16 13:45:18 +02:00
Teon Banek
8da3839a1d Convert variable start planning to iterative process
Summary: This change increases the planning time, but should reduce memory consumption.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D901
2017-10-16 10:12:09 +02:00
Teon Banek
96d9846b25 Remove traces of custom assert
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D905
2017-10-13 09:34:13 +02:00
Dominik Gleich
ebc7b2b1b9 Const map/set iteration.
Summary: In the current state, it was not possible to iterate, or even access a const map, or const set structure because of an incorrect implementation of "ConstAccessors".

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D902
2017-10-12 16:35:39 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Dominik Gleich
cce6db3442 Const iterator works.
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D886
2017-10-11 11:01:24 +02:00
Mislav Bradac
f10380a861 Begin refactor of network layer
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D855
2017-10-10 14:55:13 +02:00
florijan
ef5a381654 Snapshot generator improvements
Summary: Added support for optional properties, random integers (uniform distr) and random strings.

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D885
2017-10-10 10:20:15 +02:00
Mislav Bradac
31798eb957 Handle index creation correctly
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D891
2017-10-09 19:22:39 +02:00
Mislav Bradac
bac99cc267 Return TransientError when possible
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D889
2017-10-09 17:58:54 +02:00
Mislav Bradac
2fd43019d3 Change scheduler to first wait then execute
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D884
2017-10-09 15:40:00 +02:00
Matej Ferencevic
def612622e Decreased long running test size.
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D880
2017-10-08 14:19:53 +02:00
Mislav Bradac
c6a13ae5f6 Fix cmake for benchmarks
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D875
2017-10-06 18:04:10 +02:00
florijan
880a563cf3 Generate snapshot added
Summary:
A cool new graph generator. It's really cool!

It is currently being used for generating the Ravelin graph with the following config:
```
{
  "indexes": ["Node.id"],
  "nodes": [
    {
      "count": 40000000,
      "labels": ["Node"],
      "properties": {
              "id" : {"type": "counter", "param" : "Node.id"},
              "fradulent" : {"type" : "bernoulli", "param" : 0.0005}
      }
    }
  ],
  "edges": [
    {
      "count": 80000000,
      "from": "Node",
      "to": "Node",
      "type": "Edge"
    }
  ]
}
```

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D872
2017-10-06 15:53:02 +02:00
Mislav Bradac
c6f1920f8b Remove EdgeType index - not used in interpreter
Reviewers: teon.banek, florijan

Reviewed By: teon.banek, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D870
2017-10-06 15:02:34 +02:00
Mislav Bradac
773d1f1876 Add match nodes with cross referencing properties test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D869
2017-10-06 15:01:35 +02:00
Teon Banek
4c25123d83 Make docker installation equivalent to regular install
Summary:
This puts the whole installation and packaging under a single point of
entry. (Docker, DEB, RPM, etc.)

Rename alpha.dockerfile to beta.dockerfile
Use Debian Stretch for docker
Remove building old hardcoded compiler
Rename build_interpreter to build_memgraph
Remove unused config-file

Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D857
2017-10-06 13:23:34 +02:00
Mislav Bradac
11b2b83c96 Make gc more aggressive and optimise is_not_visible function
Reviewers: florijan, mferencevic

Reviewed By: florijan

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D849
2017-10-06 10:07:55 +02:00
florijan
d0fb4d7242 BFS - lower bound support
Summary: This is not a very important functionality, but it turned out simple to do, so let's add it to have a consistent query support.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D862
2017-10-06 09:33:20 +02:00
Matej Ferencevic
e6c3d42e02 Changed long running stress parameters.
Summary:
Increased query execution timeout.

Enabled global queries by default.

Implemented faster RandomElement for vertices and edges.

Changed long running verify message format.

Changed vertex and edge count to be per worker.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D864
2017-10-05 22:10:39 +02:00