This PR adds support for generating randomized workloads that will be executed
against a simulated cluster, as well as against a correctness model. Initially
this just generates ScanAll and CreateVertex requests, and anything that it
creates, it also inserts into a `std::set`, and when we do a ScanAll, it asserts
that we get the same number of requests back. This will become much more
sophisticated over time, but it's already hitting pay-dirt.
ScanVerticesrequest was not able to utilize filtering capabilities
before. With these modification it is now able to filter the scanned
vertices based on the filter_expressions field in the
ScanVerticesRequest message type.
Add testing capabilities similar to what existed before on the older
architecture. With this test specifically, test if we can create
vertices and return them with ScanAll. There is a hard-coded 3s wait in
the test which is needed because the MachineManager need this time to
initialize itself.
The communication between the ShardRequestManager and the RsmClient
used to be direct. In this PR this changes into a future-based
communication type. The RsmClient stores state about the currently
processed future (either read or write request) and exposes blocking
and non-blocking functionality to obtain the filled future. The
ShardRequestManager -for now- will send of the set of requests present
in the ExecutionState and block on each of them until the requests are
completed or the set of paginated responses(caused by, for example the
batch-limit in ScanAll) are ready for the next round.
* Use query-v2 in the main executable
* Set up machine manager in memgraph
* Add `ShardRequestManager` to `Interpreter`
* Make vertex creation work
* Make scan all work
* Add edge type map in shard request manager
* Send schema over request
* Empty out DbAccessor
* Store shard mapping at creation
* Remove failing CI steps
Cooltura is the best place in Zagreb!
Co-authored-by: János Benjamin Antal <benjamin.antal@memgraph.io>
- Enable `mg-expr` in mg-storage-v3
- Adapt id_mapper
- Change conversion function from PropertyValue to TypedValue
- Remove memgraph functions
- Enable expression tests on for storage
Create shard-side handlers for basic messages
Implement the handlers for CreateVertices, CreateEdges and ScanAll. Use
or modify the defined messages to interact with individual Shards and
test their behavior. Shard is currently being owned by ShardRsm
instances. The two top level dispatching functions Read() and Apply()
are responsible for read- and write operations respectively. Currently
there are a handful of messages that are defined but not utilized, these
will be used in the near future, as well as a couple of handler
functions with empty implementations.
- Integrate schema and keystore on vertex creation
- Add GC test for storage v3
- Add tests for accessors
- Fix all tests related to this except for query v2
- Fix labels not returning primary label
* Create LocalTransport Io provider for sending messages to components on the same machine
* Move src/io/simulation/message_conversion.hpp to src/io/message_conversion.hpp for use in other Io providers
* Move schema to storage v3
* Remove schema from v2
* Move schema to query v2
* Remove schema from query v1
* Make glue v2
* Move schema related tests to newer versions of query and storage
* Fix typo in CMake
* Fix interpreter test
* Fix clang tidy errors
* Change temp dir name
* Fix aggregation functions on `null` and group-by inputs (#448)
* Upgrade Antrl to 4.10.1 and remove antlr_lock (#441)
* Update clang-tidy job (#476)
* Add parser stress test (#463)
NOTE: Doing this to have buildable comments on the project-pineapples branch
Co-authored-by: gvolfing <107616712+gvolfing@users.noreply.github.com>
Co-authored-by: Jure Bajic <jure.bajic@memgraph.com>
* Make `IfOperator` return the `else_expression_` in case of `NULL`
* Add gql_behave tests
* Add gql_behave test to specifically check for the case when the test expression itself is null
* Modify `toString` to be able to handle `Date`, `LocalTime`, `LocalDateTime` and `Duration`
* Add unit tests
* Make `operator<<` use the `ToString()` implementations
* Add tests to verify the correctness of negative durations
* Add more tests to look for cases when the individual duration entities overflow.
* Add initial schema implementation
* Add index to schema
* List schemas and enable multiple properties
* Implement SchemaTypes
* Apply suggestions from code review
Co-authored-by: Jeremy B <97525434+42jeremy@users.noreply.github.com>
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
* Address review comments
* Remove Map and List
* Add schema operations in storage
* Add create and show schema queries
* Add privileges for schema
* Add missing keywords into lexer
* Add drop schema query
* Add schema visitors
* Update metadata
* Add PrepareSchemaQuery function
* Implement show schemas
* Add show schema query
* Fix schema visitor
* Add common schema type
* Fix grammar
* Temporary create ddl logic
* Fix naming for schemaproperty type to schema type
* Rename schemaproperty to schemapropertytype
* Enable Create schema ddl
* Override visitPropertyType
* Add initial schema implementation
* Add initial schema implementation
* Add index to schema
* List schemas and enable multiple properties
* Implement SchemaTypes
* Apply suggestions from code review
Co-authored-by: Jeremy B <97525434+42jeremy@users.noreply.github.com>
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
* Address review comments
* Remove Map and List
* Apply suggestions from code review
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
Co-authored-by: Jeremy B <97525434+42jeremy@users.noreply.github.com>
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
* Add verification on creation and deletion
* Rename DeleteSchema to DropSchema
* Remove list and map from lexer
* Fix grammar with schemaTypeMap
* Add privilege and cypher visitor tests
* Catch repeating type name in schema definition
* Fix conflicting keywords
* Add notifications
* Drop float support
* Finish interpreter tests
* Fix tests
* Fix clang tidy errors
* Fix GetSchema
* Replace for with transfrom
* Add cloning og schema_property_map
* Address review comments
* Rename SchemaPropertyType to SchemaType
* Remove inline
* Assert of schema properties
Co-authored-by: Jeremy B <97525434+42jeremy@users.noreply.github.com>
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
* Storage takes care of the saving of setting when a new replica is added
* Restore replicas at startup
* Modify interactive_mg_runner + memgraph to support that data-directory can be configured in CONTEXT
* Extend e2e test
* Correct typo
* Add flag to config to specify when replication should be stored (true by default when starting Memgraph)
* Remove un-necessary "--" in yaml file
* Make sure Memgraph stops if a replica can't be restored.
* Add UT covering the parsing of ReplicaStatus to/from json
* Add assert in e2e script to check that a port is free before using it
* Add test covering crash on Jepsen
* Make sure applciaiton crashes if it starts on corrupted replications' info
Starting with a non-reponsive replica is allowed.
* Add temporary startup flag: this is needed so jepsen do not automatically restore replica on startup of main. This will be removed in T0835
* Copy storage v2 to v3
* Integrate v3 to cmake
* Fix clang-tidy warnings
* Add dummy unit test for storage-v3 to trigger build for code analysis builds
