573b6cb045
* Add initial behave test improvements * Make HTML human-readable * Remove tutorial tests * Migrate to neo4j-driver==4.1.1
61 lines
1.9 KiB
Markdown
61 lines
1.9 KiB
Markdown
# Memgraph GQL Behave Tests
|
|
|
|
Python script used to run graph query language behavior tests against Memgraph.
|
|
|
|
To run the script please execute:
|
|
```
|
|
source ve3/bin/activate
|
|
./run.py --help
|
|
./run.py memgraph_V1
|
|
```
|
|
|
|
The script requires one positional parameter that specifies which test suite
|
|
should be executed. All available test suites can be found in the `tests/`
|
|
directory.
|
|
|
|
## Graph Sizes
|
|
|
|
IMPORTANT: Please prepare small graphs, up to 1000 nodes+edges because this
|
|
engine is not optimized to run on a huge graphs. Furthermore, semantics should
|
|
always be tested on a small scale.
|
|
|
|
## openCypher TCK Tests
|
|
|
|
The script uses Behave to run Cucumber tests.
|
|
|
|
Some gotchas exist when adding openCypher TCK tests to the engine:
|
|
|
|
- In some tests example injection did not work. Behave stores the first row in
|
|
Cucumber tables as headings and the example injection failed to work. To
|
|
correct this behavior, one row was added to tables where injection was used.
|
|
|
|
- Some tests don't have fully defined result ordering. Because the tests rely
|
|
on result order, some tests fail. If you find a flaky test to ignore output
|
|
ordering you should change the tag "the result should be" to "the result
|
|
should be (ignoring element order for lists)".
|
|
|
|
- Behave can't escape character '|' and it throws a parse error. The query was
|
|
then changed and the result was returned with a different name.
|
|
|
|
`Comparability.feature` tests are failing because integers are compared to
|
|
strings what is not allowed in openCypher.
|
|
|
|
## The Engine Issues
|
|
|
|
Comparing tables with ordering doesn't always work, example:
|
|
```
|
|
ORDER BY x DESC
|
|
| x | y | | x | y |
|
|
| 3 | 2 | | 3 | 1 |
|
|
| 3 | 1 | | 3 | 2 |
|
|
| 1 | 4 | | 1 | 4 |
|
|
```
|
|
|
|
Side effect aren't tracked or verified, example:
|
|
```
|
|
| +properties | 1 |
|
|
| -properties | 1 |
|
|
```
|
|
This is because Memgraph currently doesn't give out the list of side effects
|
|
that happend during query execution.
|