Commit Graph

56 Commits

Author SHA1 Message Date
Antonio Andelic
bd21bc82b7
Add license to cpp/hpp/py test files (#283) 2021-10-26 08:53:56 +02:00
Jure Bajic
f560293657
Add Property Map support in CREATE clause (#220) 2021-09-09 12:39:13 +02:00
antonio2368
3f3c55a4aa
Format all the memgraph and test source files (#97) 2021-02-18 15:32:43 +01:00
antonio2368
28413fd626 Change log library to spdlog, expose log levels to user (#72)
* Change from glog to spdlog

* Remove HA tests

* Remove logrotate log configuration

* Define custom main for unit gtests
2021-01-21 16:30:55 +01:00
Matej Ferencevic
98dc7e2849 Remove old storage types
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2633
2020-01-23 14:32:33 +01:00
Matej Ferencevic
f61a8c3358 Migrate tests to storage v2 part 2
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2559
2019-11-22 10:21:30 +01:00
Matej Ferencevic
9cc10e131d Migrate tests to storage v2 part 1
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2554
2019-11-21 19:00:32 +01:00
Teon Banek
7bd45f8714 Make query execution work with storage_v2
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2203
2019-09-12 10:22:00 +02:00
Teon Banek
f02617f2bd Replace query::GraphView with storage::View
Reviewers: mferencevic, ipaljak

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2356
2019-09-06 14:58:27 +02:00
Teon Banek
ed802073f6 Make PropertyValue constructors explicit
Summary: Depends on D2332

Reviewers: ipaljak, mferencevic

Reviewed By: ipaljak

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2333
2019-09-02 12:31:53 +02:00
Teon Banek
97b20a9384 Use non-template Value invocation
Reviewers: mtomic, mferencevic

Reviewed By: mtomic, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2317
2019-08-22 16:20:31 +02:00
Teon Banek
5bc0be50c5 Remove GraphDbAccessor parameter from MakeCursor
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2189
2019-07-09 14:22:55 +02:00
Teon Banek
57d967786c Make TypedValue constructor explicit for non-primitive types
Reviewers: mtomic, mferencevic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2128
2019-06-12 14:55:58 +02:00
Teon Banek
6d3e3ac4aa Remove TypedValue::Null
Summary:
The global variable may hide the fact that it uses the default
utils::NewDeleteResource() for allocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2121
2019-06-12 09:55:18 +02:00
Teon Banek
2909ef63d2 Use utils::MonotonicBufferResource in query execution
Reviewers: mferencevic, mtomic, msantl

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2016
2019-05-13 11:36:19 +02:00
Teon Banek
e3fbe56588 Accept utils::MemoryResource in MakeCursor
Summary:
This API change is needed in order to propagate the memory allocation
scheme for the execution of LogicalOperator::Cursor

Depends on D1990

Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1980
2019-04-29 10:34:54 +02:00
Matej Ferencevic
026c796e06 Move GraphDbAccessor to stack in SN and HA
Reviewers: msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1944
2019-04-17 11:29:43 +02:00
Marin Tomic
c03ca5f8f0 Remove UID tracking from AstStorage
Summary:
All AST nodes had a member `uid_` that was used as a key in
`SymbolTable`. It is renamed to `symbol_pos_` and it appears only in
`Identifier`, `NamedExpression` and `Aggregation`, since only those types were
used in `SymbolTable`. SymbolGenerator is now responsible for creating symbols
in `SymbolTable` and assigning positions to AST nodes.

Cloning and serialization code is now simpler since there is no need to track
UIDs.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1836
2019-02-05 14:19:48 +01:00
Teon Banek
d7422a16d2 Cleanup query/context.hpp
Summary:
Rename Context to ExecutionContext and make it struct
Move ParsingContext to cypher_main_visitor.hpp

Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1810
2019-01-17 10:21:34 +01:00
Teon Banek
b90375c3ae Remove GraphDbAccessor and storage types from Ast
Summary:
This diff removes the need for a database when parsing a query and
creating an Ast. Instead of storing storage::{Label,Property,EdgeType}
in Ast nodes, we store the name and an index into all of the names. This
allows for easy creation of a map from {Label,Property,EdgeType} index
into the concrete storage type. Obviously, this comes with a performance
penalty during execution, but it should be minor. The upside is that the
query/frontend minimally depends on storage (PropertyValue), which makes
writing tests easier as well as running them a lot faster (there is no
database setup). This is most noticeable in the ast_serialization test
which took a long time due to start up of a distributed database.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D1774
2019-01-16 09:47:42 +01:00
Teon Banek
72ee3fda85 Replace NodeAtom and EdgeAtom with CreationInfo
Summary:
This (almost) removes the dependency of operators on NodeAtom and
EdgeAtom. Only EdgeAtom::Direction is needed. The change was done as the
initial step of removing dependency on storage from Ast. Additionally,
it makes sense for LogicalOperator to only depend on Expression classes.

Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1779
2018-12-20 13:06:33 +01:00
Marin Tomic
eee8b57daf Separate query types in AST and interpreter
Summary:
`Query` is now an abstract class which has `CypherQuery`,
`ExplainQuery`, `IndexQuery`, `AuthQuery` and `StreamQuery` as derived
classes. Only `CypherQuery` is forwarded to planner and the rest of the
queries are handled directly in the interpreter. This enabled us to
remove auth, explain and stream operators, clean up `Context` class and
remove coupling between `Results` class and plan cache. This should make
it easier to add similar functionality because no logical operator
boilerplate is needed. It should also be easier to separate community
and enterprise features for open source.

Remove Explain logical operator
Separate IndexQuery in AST
Handle index creation in interpreter
Remove CreateIndex operator and ast nodes
Remove plan cache reference from Results
Move auth queries out of operator tree
Remove auth from context
Fix tests, separate stream queries
Remove in_explicit_transaction and streams from context

Reviewers: teon.banek, mferencevic, msantl

Reviewed By: teon.banek, mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1664
2018-10-22 16:43:42 +02:00
Matija Santl
fb6284cb99 Add unique constraint to label property index
Summary:
LabelPropertyIndex now has the ability to enforce unique constraint.

This doesn't lock the tx engine.

Reviewers: teon.banek, mferencevic

Reviewed By: teon.banek

Subscribers: pullbot, vkasljevic, buda

Differential Revision: https://phabricator.memgraph.io/D1660
2018-10-16 14:20:47 +02:00
Teon Banek
1b6a6c15e5 Add syntax for creating unique indexes
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1653
2018-10-12 12:50:58 +02:00
Matej Ferencevic
4e5fe37dd6 Remove virtual and pimpl from single node
Summary:
This diff removes: `SingleNodeRecoveryTransactions`, `TypemapPack`
It also removes virtual and/or pimpl from: `SingleNodeCounters`,
`StorageGcSingleNode`, `SingleNodeConcurrentIdMapper`,
accessors (revert D1510), transaction engine, `GraphDbAccessor`, `GraphDb`

Reviewers: msantl, teon.banek

Reviewed By: msantl, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1639
2018-10-09 11:48:30 +02:00
Teon Banek
80f649bcd1 Extract distributed create operators
Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1575
2018-08-30 14:41:23 +02:00
Teon Banek
9f460914ed Separate distributed implementation of GraphDbAccessor
Summary:
GraphDbAccessor is now constructed only through GraphDb. This allows the
concrete GraphDb to instantiate a concrete GraphDbAccessor. This allows
us to use virtual calls, so that the implementation may be kept
separate. The major downside of doing things this way is heap allocation
of GraphDbAccessor. In case it turns out to be a real performance
issues, another solution with pointer to static implementation may be
used.

InsertVertexIntoRemote is now a non-member function, which reduces
coupling. It made no sense for it to be member function because it used
only the public parts of GraphDbAccessor.

Reviewers: msantl, mtomic, mferencevic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1504
2018-07-26 09:16:39 +02:00
Marin Tomic
3948cea83c Rename AstTreeStorage to AstStorage
Summary: happiness

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1403
2018-06-14 13:39:03 +02:00
Marin Tomic
0a6b8cdf4f Remove AS_IS from GraphView
Summary:
Removing `AS_IS` from GraphView because it doesn't seem like it is necessary for query execution and it also has weird semantics (you might get a mix of old and new records). `Unwind`, `OrderBy` and `PullRemoteOrderBy` now use `OLD` graph view.

Remove AS_IS from GraphView

Fix query_cost_estimator tests

Fix query_expression_evaluator tests

Fix query_plan_match_filter_return tests

Fix query_plan_create_set_remove_delete tests

Fix query_plan_accumulate_aggregate tests

Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1390
2018-05-15 13:10:19 +02:00
florijan
ae95e9480c Use RemoteCreateVertex in CreateNode operator
Summary:
- Add `database::GraphDb::GetWorkerIds()`
- Change `CreateNode` constructor API
- Make `CreateNode` distribute nodes uniformly over workers

Did not yet modify `CreateExpand`, coming in the next diff.

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1216
2018-02-21 12:17:43 +01:00
florijan
45da645a4b Fix remote pull
Summary:
- Extracted TypedValue reconstruction into utils (will need that soon)
- Added more error handling in RemotePull

Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1183
2018-02-08 15:00:40 +01:00
Teon Banek
93de41e717 Throw RecordDeletedError if updating a deleted record
Summary:
Previously, we would have a `DCHECK` which crashes the application. This
was evident when testing a queries, such as:

    MATCH (n) DELETE n SET n.prop = 42

Since the argument to update clauses is evaluated during execution, it
makes it very difficult to prevent such errors during semantic analysis.
For example:

    MATCH (n)--(m) WITH collect(n) as ns, m
    DETACH DELETE ns[m.prop] SET head(ns).prop = 42

Test query updates on deleted graph elements

Reviewers: florijan, dgleich

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1114
2018-01-17 14:30:57 +01:00
florijan
813d37e939 Migrate db::types to storage::
Reviewers: teon.banek, dgleich

Reviewed By: teon.banek, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1110
2018-01-17 10:35:12 +01:00
florijan
6fc6a27288 Refactor GraphDb
Summary:
GraphDb is refactored to become an API exposing different parts
necessary for the database to function. These different parts can have
different implementations in SingleNode or distributed Master/Server
GraphDb implementations.

Interally GraphDb is implemented using two class heirarchies. One
contains all the members and correct wiring for each situation. The
other takes care of initialization and shutdown. This architecture is
practical because it can guarantee that the initialization of the
object structure is complete, before initializing state.

Reviewers: buda, mislav.bradac, dgleich, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1093
2018-01-12 16:47:24 +01:00
Dominik Gleich
67538aceeb Migrate labels/properties/edgetypes to ids
Summary:
In preparation for distributed storage we need to have labels/properties/edgetypes uniquely identifiable by their ids, which will be global in near future.
The old design has to be abandoned because it's not possible to keep track of global labels/properties/edgetypes while they are local pointers.

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D993
2017-11-23 17:12:37 +01:00
Teon Banek
55456b4214 Remove Dbms
Summary:
Remove name from GraphDb.
Take GraphDb in query test macros instead of accessor.
Add is_accepting_transactions flag to GraphDb.

Reviewers: mislav.bradac, florijan, mferencevic

Reviewed By: mislav.bradac

Subscribers: mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D940
2017-10-30 12:33:29 +01:00
florijan
76fe8bfadf Variable expansion consolidaton
Summary:
- Removed BreadthFirstAtom, using EdgeAtom only with a Type enum.
- Both variable expansions (breadth and depth first) now have mandatory inner node and edge Identifiers.
- Both variable expansions use inline property filtering and support inline lambdas.
- BFS and variable expansion now have the same planning process.
- Planner modified in the following ways:
	- Variable expansions support inline property filtering (two filters added to all_filters, one for inline, one for post-expand).
	- Asserting against existing_edge since we don't support that anymore.
	- Edge and node symbols bound after variable expansion to disallow post-expand filters to get inlined.
	- Some things simplified due to different handling.
- BreadthFirstExpand logical operator merged into ExpandVariable. Two Cursor classes remain and are dynamically chosen from.

As part of planned planner refactor we should ensure that a filter is applied only once. The current implementation is very suboptimal for property filtering in variable expansions.

@buda: we will start refactoring this these days. This current planner logic is too dense and complex. It is becoming technical debt. Most of the time I spent working on this has been spent figuring the planning out, and I still needed Teon's help at times. Implementing the correct and optimal version of query execution (avoiding multiple potentially expensive filterings) was out of reach also due to tech debt.

Reviewers: buda, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D852
2017-10-05 13:12:39 +02:00
florijan
a9381df09e Edges data structure now supports multiple edge filtering (implicit OR)
Summary: - modified all utils/algorithm functions to be inline and in the utils namespace

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D830
2017-09-26 13:46:18 +02:00
Teon Banek
a83bea0b74 Add ParameterLookup to AST
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D782
2017-09-13 14:28:48 +02:00
Teon Banek
35f726dfd2 Use EdgeType in Expand and ExpandVariable
Summary:
Add function First to utils.
Insert EdgeType into Expand during planning.

Reviewers: florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D769
2017-09-11 11:16:04 +02:00
florijan
1d112e1141 GraphDbAccessor - style change
Summary: Not strictly neccessary, but it's been itching me. It took an hour.

Reviewers: buda, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D648
2017-08-09 16:09:08 +02:00
florijan
b8957c999d Map type now supported
Summary:
- MapLiteral added
- PropertyLookup on maps added

This is the basic implementation, missing are:
- unit tests
- feature and TCK tests
- documentation
- changelog

That stuff is coming. Please review the implementation (Mislav).

Reviewers: mislav.bradac, buda, teon.banek

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D640
2017-08-08 14:34:26 +02:00
Mislav Bradac
6068a95a0e Move dbms to database
Summary:
There was only two files in dbms directory so I moved them to database
directory.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D540
2017-07-12 12:44:11 +02:00
Teon Banek
f5db9d65b0 Add CreateIndex operator
Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D516
2017-07-03 11:03:15 +02:00
Dominik Gleich
141d9b3bb8 Rework of index api. Remove implicit default value of current_state in graph_db_accessor.
Reviewers: buda, florijan

Reviewed By: buda, florijan

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D392
2017-06-07 10:14:55 +02:00
Teon Banek
ffcc8b8d12 Change CollectProduce to immediately return results
Reviewers: florijan, mislav.bradac

Reviewed By: florijan, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D406
2017-05-30 16:55:37 +02:00
Teon Banek
e631eb4eb2 Rename directions from LEFT/RIGHT to IN/OUT
Reviewers: florijan, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D405
2017-05-30 13:14:34 +02:00
Teon Banek
30a4f40093 Skip Null in SET and REMOVE clauses
Summary:
openCypher expects removing/setting properties and labels on Null
vertices/edges does not produce an error. Instead, Nulls are simply
skipped.

Reviewers: florijan, mislav.bradac, buda

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D375
2017-05-17 15:07:03 +02:00
Teon Banek
87e5dc0dfb Plan '*' in RETURN and WITH
Summary:
Make Symbol members read only.
Check WITH/RETURN * in SymbolGenerator.
Test semantic checks for WITH/RETURN *.
Sort expanded user identifiers by name.
Test planning WITH/RETURN *.

Reviewers: buda, florijan, mislav.bradac

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D357
2017-05-12 13:05:40 +01:00
Teon Banek
be81751db3 Allow filter symbols in Match to be out of binding order
Summary:
Check symbols in property maps after visiting Match.
Plan Filters as soon as possible.
Take AstTreeStorage in MakeLogicalPlan instead of Query.
Plan generic Filter instead of specialized operators.
Remove traces of EdgeFilter and NodeFilter.

Reviewers: buda, mislav.bradac, florijan

Reviewed By: mislav.bradac, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D344
2017-05-08 12:03:38 +02:00