Commit Graph

1351 Commits

Author SHA1 Message Date
Teon Banek
06b0111ddc Remove unused stuff from CMakeLists
Summary: In the process, make experimental/distributed compilable.

Reviewers: mislav.bradac, buda, mferencevic

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D906
2017-10-16 13:45:18 +02:00
Teon Banek
8da3839a1d Convert variable start planning to iterative process
Summary: This change increases the planning time, but should reduce memory consumption.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D901
2017-10-16 10:12:09 +02:00
Teon Banek
96d9846b25 Remove traces of custom assert
Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D905
2017-10-13 09:34:13 +02:00
Dominik Gleich
ebc7b2b1b9 Const map/set iteration.
Summary: In the current state, it was not possible to iterate, or even access a const map, or const set structure because of an incorrect implementation of "ConstAccessors".

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D902
2017-10-12 16:35:39 +02:00
Dominik Gleich
fcecb14545 Replace debug_assert, permanent_assert with DCHECK/CHECK
Summary:
Phase 2.

Phase 3.

Phase 4.

Phase 5.

Complete refactor.

Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D895
2017-10-11 14:43:32 +02:00
Matej Ferencevic
4f7f59f9fc Fixed openCypher documentation query indentation.
Reviewers: teon.banek, florijan, buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D899
2017-10-11 13:55:46 +02:00
Teon Banek
f85de1e8e1 Add tagging docker image as latest
Summary: Update technical documentation, installation

Reviewers: florijan, buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D893
2017-10-11 13:05:18 +02:00
Mislav Bradac
e078c9f7cd Refactor Interpreter
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D896
2017-10-11 11:06:25 +02:00
Dominik Gleich
cce6db3442 Const iterator works.
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D886
2017-10-11 11:01:24 +02:00
Mislav Bradac
f10380a861 Begin refactor of network layer
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D855
2017-10-10 14:55:13 +02:00
florijan
0f5c2bb6c3 Differences from Cypher documented
Summary:
I made some corrections to the existing docs and added a paragraph on Memgraph vs Cypher. It inevitably mentions Neo4j.

My main source of Neo4j functionality was the refcard, so it's possible I missed something, but I think the list is pretty exhaustive. Please validate.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D883
2017-10-10 13:25:12 +02:00
florijan
ef5a381654 Snapshot generator improvements
Summary: Added support for optional properties, random integers (uniform distr) and random strings.

Reviewers: mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D885
2017-10-10 10:20:15 +02:00
Matej Ferencevic
6a144db165 Integrated release files generation into Apollo.
Summary:
Don't store unnecessary files in documentation archive.

Apollo generate script cleanup.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D887
2017-10-09 21:13:29 +02:00
Mislav Bradac
31798eb957 Handle index creation correctly
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D891
2017-10-09 19:22:39 +02:00
Mislav Bradac
bac99cc267 Return TransientError when possible
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D889
2017-10-09 17:58:54 +02:00
Teon Banek
57144518d0 Remove rechecking external projects
Summary:
This is probably a bug in CMake. When the external projects are forced
to be rechecked, CPack takes their outputs and puts them in binary
release.

Since rechecking is removed, this means that any changes to external
projects (like gflags or glog) will not be recompiled with plain
`make all`. You need to call `make clean` first. The same applies to
catching configuration like removal of filenames in glog when creating a
release build.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D890
2017-10-09 17:12:38 +02:00
Mislav Bradac
93b9592c63 Add malloc_trim
Summary: Lets run daily tonight with malloc_trim and see what is memory usage

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D888
2017-10-09 15:48:19 +02:00
Mislav Bradac
2fd43019d3 Change scheduler to first wait then execute
Reviewers: dgleich

Reviewed By: dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D884
2017-10-09 15:40:00 +02:00
florijan
cffa5f6c96 Clients: ravelin
Summary: A proposition on where and how we could keep some basic client info.

Reviewers: buda, mislav.bradac, teon.banek, mferencevic

Reviewed By: buda

Subscribers: dtomicevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D873
2017-10-09 14:34:46 +02:00
Matej Ferencevic
c4c22417b9 Added user documentation community bundle script.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D878
2017-10-09 13:44:54 +02:00
Marko Budiselic
7a14e09dfc Add Dockerfile for csv_to_snapshot packaging.
Summary:
C++ implementation of CSV to Memgraph's snapshot file is much more efficient then
the Python implementation. It includes a huge part of the Memgraph's source
code and the code has to be compiled (which is not easy). Because of that it
has to be packaged.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D854
2017-10-09 12:15:46 +01:00
Teon Banek
9e744bc7dc Bump Memgraph version to 0.8.0
Reviewers: buda, mferencevic, mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D881
2017-10-09 10:30:31 +02:00
Dominik Gleich
888a63b784 Add functional include - to compile on clang5.0
Reviewers: teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D882
2017-10-09 10:29:37 +02:00
Marko Budiselic
2d2ebaca2f LDBC plot script update
Summary: Plot's width, height, and PPI are now parameterized.

Reviewers: teon.banek, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D847
2017-10-08 14:23:53 +01:00
Matej Ferencevic
def612622e Decreased long running test size.
Reviewers: buda

Reviewed By: buda

Differential Revision: https://phabricator.memgraph.io/D880
2017-10-08 14:19:53 +02:00
Matej Ferencevic
45a02b56f8 Fixed bolt client endless loop when server dies.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D879
2017-10-08 13:31:05 +02:00
Marko Budiselic
1f8d532fe9 Remove docs update version script.
Summary: Remove docs update version script.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D877
2017-10-07 21:15:06 +01:00
Matej Ferencevic
37b2caae63 Implemented hidden flags.
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D861
2017-10-07 15:08:48 +02:00
Mislav Bradac
e11216f058 Remove malloc_trim
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D876
2017-10-06 19:13:01 +02:00
Mislav Bradac
c6a13ae5f6 Fix cmake for benchmarks
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D875
2017-10-06 18:04:10 +02:00
Teon Banek
04fe004423 Force external project recheck earlier
Summary:
This fixes an issue when the configuration changed (e.g. gflags CMake
options).

Reviewers: mferencevic

Reviewed By: mferencevic

Differential Revision: https://phabricator.memgraph.io/D874
2017-10-06 16:00:47 +02:00
florijan
880a563cf3 Generate snapshot added
Summary:
A cool new graph generator. It's really cool!

It is currently being used for generating the Ravelin graph with the following config:
```
{
  "indexes": ["Node.id"],
  "nodes": [
    {
      "count": 40000000,
      "labels": ["Node"],
      "properties": {
              "id" : {"type": "counter", "param" : "Node.id"},
              "fradulent" : {"type" : "bernoulli", "param" : 0.0005}
      }
    }
  ],
  "edges": [
    {
      "count": 80000000,
      "from": "Node",
      "to": "Node",
      "type": "Edge"
    }
  ]
}
```

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D872
2017-10-06 15:53:02 +02:00
Teon Banek
8bf0a9cf92 Fill the alpha.conf with all settings and document them
Summary:
This should be the basis for default configuration that is to be shipped
in release builds/packages.

Reviewers: mislav.bradac, florijan, buda, mferencevic

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D851
2017-10-06 15:15:19 +02:00
Mislav Bradac
da3e46c2d6 Fix race condition in BuildIndex
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D871
2017-10-06 15:13:26 +02:00
Mislav Bradac
c6f1920f8b Remove EdgeType index - not used in interpreter
Reviewers: teon.banek, florijan

Reviewed By: teon.banek, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D870
2017-10-06 15:02:34 +02:00
Mislav Bradac
773d1f1876 Add match nodes with cross referencing properties test
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D869
2017-10-06 15:01:35 +02:00
Mislav Bradac
c3a211fd4e Add message to HintedAbortError
Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D868
2017-10-06 14:39:50 +02:00
Teon Banek
4c25123d83 Make docker installation equivalent to regular install
Summary:
This puts the whole installation and packaging under a single point of
entry. (Docker, DEB, RPM, etc.)

Rename alpha.dockerfile to beta.dockerfile
Use Debian Stretch for docker
Remove building old hardcoded compiler
Rename build_interpreter to build_memgraph
Remove unused config-file

Reviewers: mferencevic, buda

Reviewed By: mferencevic, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D857
2017-10-06 13:23:34 +02:00
Teon Banek
f6a8d0e910 Fix flag parsing and update usage message
Summary:
csv_to_snapshot: Don't skip flags when parsing
csv_to_snapshot: Update usage message

Reviewers: buda, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D865
2017-10-06 12:45:34 +02:00
Teon Banek
69f178b347 Force CMake to recheck external project source
Reviewers: mferencevic, buda

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D867
2017-10-06 12:45:00 +02:00
Mislav Bradac
11b2b83c96 Make gc more aggressive and optimise is_not_visible function
Reviewers: florijan, mferencevic

Reviewed By: florijan

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D849
2017-10-06 10:07:55 +02:00
florijan
d0fb4d7242 BFS - lower bound support
Summary: This is not a very important functionality, but it turned out simple to do, so let's add it to have a consistent query support.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D862
2017-10-06 09:33:20 +02:00
Matej Ferencevic
e6c3d42e02 Changed long running stress parameters.
Summary:
Increased query execution timeout.

Enabled global queries by default.

Implemented faster RandomElement for vertices and edges.

Changed long running verify message format.

Changed vertex and edge count to be per worker.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D864
2017-10-05 22:10:39 +02:00
florijan
8e1626d4fb LongRunning - batched edge creation improved. Verification before loop
Summary:
Batch creation now does not iterate over V**2 vertex pairs, but over V vertices and generates E/V random edges for each one. Assuming there is an index over :V(id), this is much more efficient and supports larger vertex counts (Ferenc: modify the test config as desired).

An additional requirement is that E/V (for each worker) is a whole number.

Reviewers: mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D860
2017-10-05 13:51:54 +02:00
Mislav Bradac
1f01ef6cd1 Rerun antlr on original query to generate appropriate error message
Reviewers: buda, florijan, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D858
2017-10-05 13:35:39 +02:00
florijan
76fe8bfadf Variable expansion consolidaton
Summary:
- Removed BreadthFirstAtom, using EdgeAtom only with a Type enum.
- Both variable expansions (breadth and depth first) now have mandatory inner node and edge Identifiers.
- Both variable expansions use inline property filtering and support inline lambdas.
- BFS and variable expansion now have the same planning process.
- Planner modified in the following ways:
	- Variable expansions support inline property filtering (two filters added to all_filters, one for inline, one for post-expand).
	- Asserting against existing_edge since we don't support that anymore.
	- Edge and node symbols bound after variable expansion to disallow post-expand filters to get inlined.
	- Some things simplified due to different handling.
- BreadthFirstExpand logical operator merged into ExpandVariable. Two Cursor classes remain and are dynamically chosen from.

As part of planned planner refactor we should ensure that a filter is applied only once. The current implementation is very suboptimal for property filtering in variable expansions.

@buda: we will start refactoring this these days. This current planner logic is too dense and complex. It is becoming technical debt. Most of the time I spent working on this has been spent figuring the planning out, and I still needed Teon's help at times. Implementing the correct and optimal version of query execution (avoiding multiple potentially expensive filterings) was out of reach also due to tech debt.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D852
2017-10-05 13:12:39 +02:00
Matej Ferencevic
d249ff11a5 Misc glog and gflags fixes.
Summary:
Log files aren't created by default anymore.
All logs are reported to stderr by default.
Normalized flag names.
Removed unnecessary flags from gflags.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D856
2017-10-04 17:29:53 +02:00
Teon Banek
1cd83d2c19 Add option whether to link with libreadline or not
Reviewers: buda, mislav.bradac, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D850
2017-10-04 15:23:11 +02:00
Matej Ferencevic
73052dcff4 Changed long running large stress parameters.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D853
2017-10-04 09:57:42 +02:00
florijan
0370e83b4d mvcc::Hints reduction
Summary: It's a sketch. I can tidy it up if you like this approach.

Reviewers: mislav.bradac, mferencevic

Reviewed By: mislav.bradac

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D841
2017-10-03 16:56:06 +02:00