Commit Graph

165 Commits

Author SHA1 Message Date
Teon Banek
f5db9d65b0 Add CreateIndex operator
Reviewers: florijan, mislav.bradac, buda

Reviewed By: florijan, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D516
2017-07-03 11:03:15 +02:00
Mislav Bradac
592ba4df98 Minor cleanup
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D513
2017-06-28 15:56:05 +02:00
Dominik Gleich
e2f3aba332 Use GLogger instead of broken memgraph Logger.
Summary:
http://rpg.ifi.uzh.ch/docs/glog.html

Second phase before tests complete.

Delete logging test.

Finish relase loging.

Reviewers: mislav.bradac, teon.banek, buda

Reviewed By: teon.banek

Subscribers: buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D500
2017-06-21 15:33:24 +02:00
florijan
e28fc6ed41 FLAG 'snapshot_on_destruction' renamed to 'snapshot_on_exit'
Reviewers: teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D471
2017-06-16 16:25:26 +02:00
florijan
9e6260d0c3 tx::Engine - commit log encapsulation
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D460
2017-06-13 12:32:48 +02:00
Mislav Bradac
d46945f259 Make GraphDbTypes const
Reviewers: florijan

Reviewed By: florijan

Differential Revision: https://phabricator.memgraph.io/D456
2017-06-12 15:24:30 +02:00
florijan
afbb940a05 GC bugfixes, MVCC and transaction refactoring
Summary:
- GC changed to evaluate old records w.r.t. the oldest transaction's id AND snapshot, as opposed to only id
- MVCC hints exp+aborted race condition prevented
- minor MVCC refactors and cleanups
- minor Transaction refactors and cleanups

Reviewers: buda, dgleich

Reviewed By: buda, dgleich

Subscribers: dtomicevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D434
2017-06-12 10:46:12 +02:00
Dominik Gleich
972c62e102 Fix signals. Add recovery status.
Reviewers: teon.banek, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D451
2017-06-09 13:25:24 +02:00
Dominik Gleich
1057d253ca Rename config flags.
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D446
2017-06-09 10:52:26 +02:00
Dominik Gleich
65507da9eb Load configs from flagfiles.
Summary: Migrate configto gflags.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D433
2017-06-07 15:38:17 +02:00
Dominik Gleich
141d9b3bb8 Rework of index api. Remove implicit default value of current_state in graph_db_accessor.
Reviewers: buda, florijan

Reviewed By: buda, florijan

Subscribers: mislav.bradac, pullbot

Differential Revision: https://phabricator.memgraph.io/D392
2017-06-07 10:14:55 +02:00
matej.gradicek
15b57e2d52 Implemented recovery and tests.
Summary: Implemented durability recovery and tesats.

Reviewers: mislav.bradac, dgleich, buda

Reviewed By: mislav.bradac, dgleich, buda

Subscribers: dtomicevic, mferencevic, pullbot

Differential Revision: https://phabricator.memgraph.io/D374
2017-06-06 15:12:34 +00:00
Dominik Gleich
db5f4469fe Change time to duration.
Reviewers: florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D403
2017-05-30 10:39:58 +02:00
matej.gradicek
e791e9b660 Summary:Added files for snapshot durability
Summary:
File buffer added

Implemented little more of snapshoter.

Resolved conflicts

More stuff implemented of snapshot durability.

More things in snapshoter.

Refactored, added comments

Merge branch 'dev' into durability_snapshot

Merge branch 'dev' into durability_snapshot

Resolved bug in scheduler, snapshoter is running in grpah_db.

Reviewers: mferencevic, buda, dgleich, mislav.bradac

Reviewed By: mferencevic, buda, dgleich, mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D232
2017-05-17 08:42:00 +00:00
Dominik Gleich
842901ecd2 LabelProperty index.
Summary:
Add return values.

After merge.

Inital working version. Still missing comments.

Update documentation.

Add checking for previous vlist and value equality.

After merge.

Remove functor, add boolean ffunction.

Build index.

More functionality. Start implementing tests.

Add tests.

Reviewers: matej.gradicek, mislav.bradac, mferencevic, buda, florijan

Reviewed By: mislav.bradac, buda, florijan

Subscribers: lion, florijan, teon.banek, buda, pullbot

Differential Revision: https://phabricator.memgraph.io/D355
2017-05-16 16:33:20 +02:00
florijan
0a33a555a6 GraphDdAccessor and KeyIndex API upgrades. Test refactoring
Summary:
The GraphDbAccessor and KeyIndex APIs can now also return records for the current transaction+command graph state. This is necessary to correctly implement MERGE. The new logic is has increased the MVCC+Accessor related chaos and should be revised when refactoring MVCC (as planned).

Previous index testing was separated into VertexIndex and EdgeIndex testing. This is inappropriate since most of the logic is exaclty the same. Also it was not clearly defined what gets tested via the GraphDbAccessor API, and what directly through the KeyIndex API. This has also been refactored, but it needs additional work (Gleich).

Reviewers: buda, dgleich

Reviewed By: buda, dgleich

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D306
2017-05-03 15:42:04 +02:00
Teon Banek
893df584f6 Merge utils/exceptions into single file
Summary: Update documentation of `utils/exceptions.hpp`

Reviewers: mislav.bradac, florijan, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D300
2017-04-20 16:53:21 +02:00
florijan
231b84e834 Query::Plan - Reconstruct fails after delete
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D287
2017-04-18 12:25:41 +02:00
Dominik Gleich
461ea65ad4 Deferred deleter added.
Summary:
Add tests.

Merge branch 'dev' into remove_locks

Everything works. Refactor complete.

Documentation:
https://phabricator.memgraph.io/w/memgraph_implementation/indexing/

Reviewers: buda, mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D258
2017-04-14 17:46:16 +02:00
florijan
0a09a6ac64 Storage, Query - deletion bugfix and improvements
Summary:
Multiple attempts to delete a record dont crash anymore.
Deleting a vertex and its blocking edge in the same delete op now supported.
Utils::Assert - permanent_fail bug fix

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D282
2017-04-14 16:34:38 +02:00
Mislav Bradac
bba20cf89c Remove unneccessary files from utils
Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D276
2017-04-13 17:55:33 +02:00
Dominik Gleich
fd2780155a Refactor GC to use scheduler.
Reviewers: buda, matej.gradicek

Reviewed By: matej.gradicek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D247
2017-04-10 15:57:45 +02:00
Dominik Gleich
8c7ed8c186 Add edge_types indexes.
Summary: Refactor label_index. Also add edge_type indexes.

Reviewers: mislav.bradac, florijan

Reviewed By: florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D217
2017-04-05 17:23:12 +02:00
florijan
9ce2081103 Storage::RecordAccessor - added multiple pointers to RecordAcessors and switching capabilities. Rewired GraphDbAccessor and VersionList accordingly. Added tests for new RecordAccessor functions.
Reviewers: teon.banek, dtomicevic, mislav.bradac, dgleich, buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D207
2017-04-03 12:29:56 +02:00
Dominik Gleich
863f55dc2f Labels implementation.
Summary: Labels implementation

Reviewers: buda, teon.banek

Reviewed By: buda, teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D201
2017-04-03 11:50:22 +02:00
Dominik Gleich
4a6d17658c Fix unnecessary multiple inserts.
Summary: Unnecessary inserts into skiplist. Skiplist when calls insert alredy blocks untill it inserts or finds that record.

Reviewers: mislav.bradac, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D210
2017-03-31 14:02:54 +02:00
Dominik Gleich
bd0b81526e Initial version of mvcc_gc.
Summary:
Tested MVCC garbage collector.
Also refactors graph_db forward declarations of vertex and edge which were causing issues.

Reviewers: mislav.bradac, dtomicevic, florijan, teon.banek, buda

Reviewed By: teon.banek, buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D177
2017-03-29 12:38:24 +02:00
florijan
63aae05ec0 GraphDbAccessor - DetachRemove - commands re-ordered for better (assumed) efficiency
Summary: See above

Reviewers: buda, dtomicevic

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D174
2017-03-24 14:17:04 +01:00
florijan
1a6aaaed58 DBMS now returns a unique pointer to a GraphDbAccessor
Summary: See above.

Reviewers: mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D120
2017-03-14 13:37:23 +01:00
florijan
09999e877f RecordAccessor uses pointers for storage so it's move/copy assignable
Summary: See above.

Reviewers: buda, teon.banek, mislav.bradac

Reviewed By: mislav.bradac

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D118
2017-03-14 13:04:07 +01:00
Dominik Gleich
57f1b40d8d Refactor transaction inside db_accessor. Also introduce advance_command.
Summary:
Modification to MVCC - CHECK!

Refactor transaction_read

Add automatic commit/abort functionality to graph_db_accessor.

Fix failing test in graph_db_accessor.

Reviewers: buda, dtomicevic, florijan

Reviewed By: buda, florijan

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D87
2017-03-07 09:42:04 +01:00
florijan
2aeef25881 Traversal namespace specialization added for real DB primitives
Summary: database/GraphDbAccessor and storage/...Accessor now returns iterators over lazily created Accessors (filtered on visibility). Dependecies and forward declarations reorganized to support new code

Reviewers: buda, teon.banek

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D83
2017-03-03 14:01:01 +01:00
Dominik Gleich
82414b9111 Fix naming of asserts.runtime_assert & assert -> debug_assert
Summary:
Merge branch 'dev' into rename_assert

Fix new files.

Remove cassert.

Reviewers: buda

Reviewed By: buda

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D66
2017-02-24 11:41:55 +01:00
Marko Budiselic
3642fb1312 examples folder has been removed; src/test folder has been removed; the existing tests were moved to tests folder; StacktraceException has been introduced; query_plan_templated has been moved to query folder; hazard pointers have been deleted because they are not used any more 2017-02-18 18:03:48 +01:00
Marko Budiselic
d4e3c4cd10 clang format has been run on all hpp and cpp files under src and tests 2017-02-18 11:54:37 +01:00
Marko Budiselic
e7f5bd4c21 antlr integration, *.hpp and *.cpp inside src dir, cleanup
Summary: antlr integration, *.hpp and *.cpp inside src dir, cleanup

Test Plan: manual

Reviewers: mislav.bradac, dgleich, florijan

Reviewed By: florijan

Subscribers: pullbot, buda

Differential Revision: https://phabricator.memgraph.io/D49
2017-02-17 16:20:31 +01:00
florijan
55f1912910 Properties refactoring: STABLE STATE. Memgraph compiles. Properties are out. Tests are in progress. 2017-02-16 15:47:55 +01:00
florijan
70a8b93b0b Major properties system and database accessor refactor: first stable state (compiles). 2017-02-15 14:10:16 +01:00
Florijan Stamenkovic
9adf5699d9 Major properties and storage refactoring in progress. UNSTABLE STATE 2017-02-13 09:21:35 +01:00
Florijan Stamenkovic
b38704391c Typed-value migration in progress. UNSTABLE STATE 2017-02-06 12:40:55 +01:00
Florijan Stamenkovic
b374ae1dbb storage and database major refactor in progress, also switching from properties to typed-value system. a lot of overdesigned functionalities cut down, new implementations just started. VERY UNSTABLE STATE 2017-02-04 09:01:56 +01:00
Florijan Stamenkovic
e508482ab9 Deleted storage/model/properties, started refactoring 2017-02-02 09:03:48 +01:00
Marko Budiselic
55a62f9640 Bugs from concurrent tests aren't solved. TODO: we have to introduce memory tracker for tests 2016-12-21 21:33:58 +01:00
Marko Budiselic
cfd36be2e0 All possible request changes from D9#146 are done 2016-11-30 11:05:48 +01:00
Marko Budiselic
21788d003a Dressipi CRUD queries are dummy implemented; Fixes T99 and T131
Summary: Dressipi CRUD queries are dummy implemented; Fixes T99 and T131

Test Plan: manual

Reviewers: sale

Subscribers: buda, sale

Maniphest Tasks: T131, T99

Differential Revision: https://memgraph.phacility.com/D9
2016-11-29 04:08:49 +01:00
Kruno Tomola Fabro
d806d635f9 Added documentation.
Fixed test for index.
2016-09-18 23:22:36 +01:00
Kruno Tomola Fabro
266d8ed055 Transformed dressipi astar into a query. 2016-09-13 14:17:50 +01:00
Kruno Tomola Fabro
6e4096d619 Added flag for optional import of database. 2016-09-13 12:21:07 +01:00
Kruno Tomola Fabro
2a59ed8906 Minor refactorings:
Importes now use logger.

Refactored order of constrution of objects in Db.

Moved index creation/removing from Db to Indexes.

Completed Garbage class.
Cleaner now calls garbage.clean() for databases.

Renamed List to ConcurrentList which better names it.
2016-09-12 20:13:04 +01:00
Kruno Tomola Fabro
cf497dd522 Extracted from snapshoter => snapshot_engine. 2016-09-09 18:48:15 +01:00
Kruno Tomola Fabro
911eeff1d9 Snapshot store and load works. 2016-09-09 16:14:20 +01:00
Kruno Tomola Fabro
4443cb8b7b Finished creating part of snapshot. 2016-09-08 13:25:52 +01:00
Kruno Tomola Fabro
5d235b51f3 tmp commit
tmp commit

tmp commit v2

Finished reimplementation of propertys.
They now can be placed in a holder with different source of type information.

Tmp commit
2016-09-05 10:02:48 +01:00
Kruno Tomola Fabro
b2ce3d58a4 Added cleaner.
Added multithreading to dbms.
Skiplist now supports emplace insert.
2016-08-30 00:45:07 +01:00
Kruno Tomola Fabro
e908d6588b Tidied up barrier files.
MEMGRAPH has linked/definistion errors during compile.
2016-08-28 20:46:30 +01:00
Kruno Tomola Fabro
77a3298d1e tmp 2016-08-28 15:47:13 +01:00
Kruno Tomola Fabro
8a89f6601d EdgeType indexes added.
Implemented untested UniqueOrderedIndex.

Introduced TypeGroupEdge/Vertex into database.

Added Index capabilityes to PropertyFamily.
Added method for adding index.
Added method for removing index.
2016-08-25 15:29:45 +01:00
Kruno Tomola Fabro
fbd9ca8420 First version of CSVImport tool.
Query and import works.

Properties now use unordered_map.
Reduced memory footprint of properties by more than half.
2016-08-22 19:03:45 +01:00
Kruno Tomola Fabro
eba7fd8be4 Fixed Query compiler to use new interface. 2016-08-19 18:40:04 +01:00
Marko Budiselic
427a7bbe4b merged interface branch 2016-08-19 11:28:10 +01:00
Kruno Tomola Fabro
5a42e15c4a Alpha version of label indexes.
Squashed messages from 9 commits:

9.
Properties now uses PropertyFamily and contained classes.
Fetching,seting,clearing properties can be done with PropertyFamilyKey or PropertyTypeKey.
Hierarchy of newly added clases is:
Vertices -n-> PropertyFamily {name: String} <-1-n-> PropertyType {type: Property::Flags}
Edges -n-> PropertyFamily {name: String} <-1-n-> PropertyType {type: Property::Flags}

PropertyFamilyKey -> PropertyType
PropertyTypeKey -> PropertyType

PropertyType t0,t1;
let t0!=t1 be true
let t0.family==t1.family be true

then next is true
PropertyTypeKey{&t0}!=PropertyTypeKey{&t1}
PropertyFamilyKey{&t0}==PropertyFamilyKey{&t1}
PropertyFamilyKey{&t0}==PropertyTypeKey{&t1}
PropertyTypeKey{&t0}==PropertyFamilyKey{&t1}

8.
Intermedate commit.
Noticed that integration queries throw SEGFAULT.

7.
Defined interface for indexes.
Fixed three memory leaks.
Fixed integration_queries test which now passes.

6.
Commit which return Xorshift128plus to valid shape.

5.
Tmp commit.

4.
Label Index is compiling.

3.
tmp

2.
Vertex::Accessor now updates Label index.

1.
Applied changes for code review.
2016-08-18 15:34:36 +01:00
Kruno Tomola Fabro
df0bf6fa5f Database interface refactor.
DbAccessor:
 -Guarantees that access to Vertex and Edge is possible only through
 Vertex::Accessor and Edge::Accessor.
 -Guarantees that changing Vertex and Edge is possible only using
 Vertex::Accessor returned by vertex_insert() method and
 Edge::Accessor returned by edge_insert() method.
 -Offers CRUD for Vertex and Edge except iterating over all edges.

Squashed commit messages:

First step in database accessor refactoring done.
It's compiling.
All tests with exception of integration_querys pass

Tests now initialize logging facilities.

Refactored accessors.
RecordAccessor now has 3 states.
From,To,Out,In in there respecive Accessors return unfilled RecordAccessor.
Added iterator classes into utils/itearator/.
2016-08-15 00:09:58 +01:00
Kruno Tomola Fabro
1849514159 First step in database accessor refactoring done.
It's compiling.
All tests with exception of integration_querys pass
2016-08-12 23:01:39 +01:00
Marko Budiselic
2b1fc70695 some header files are splitted (WORK IN PROGRESS) 2016-07-05 13:20:33 +01:00
Marko Budiselic
52625e39a3 The folder structure was changed, all source code is now within the src dir. 2016-05-08 11:33:07 +02:00