tursom/memgraph
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Refactored kpi service

Browse Source 
Reviewers: buda

Reviewed By: buda

Subscribers: matej.gradicek

Differential Revision: https://phabricator.memgraph.io/D142
This commit is contained in:
Matej Gradiček 2017-03-20 12:32:09 +00:00
parent 9048d6002f
commit 1013c3eaee
13 changed files with 364 additions and 174 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
README.md
View File

@ -34,6 +34,21 @@ The following tck tests have been changed:
Comparability.feature tests are failing because integers are compared to strings
what is not allowed in openCypher.
TCK Engine problems:
1. Comparing tables with ordering.
ORDER BY x DESC
| x | y | | x | y |
| 3 | 2 | | 3 | 1 |
| 3 | 1 | | 3 | 2 |
| 1 | 4 | | 1 | 4 |
2. Properties side effects
| +properties | 1 |
| -properties | 1 |
Database is returning properties_set, not properties_created and properties_deleted.
## KPI Service
Flask application used to get results from executing tests with TCK Engine.

160
kpi_service/kpi_service.py
View File

@ -1,22 +1,27 @@
from flask import Flask, jsonify, request
import os, json
import os
import json
from argparse import ArgumentParser
app = Flask(__name__)
"""
Script runs web application used for getting test results.
File with results is "../tck_engine/results".
Script runs web application used for getting test results.
Default file with results is "../tck_engine/results".
Application lists files with results or returnes result
file as json.
Default host is 127.0.0.1, and default port is 5000.
Host and port can be passed as arguments of a script.
Default host is 0.0.0.0, and default port is 5000.
Host, port and result file can be passed as arguments of a script.
"""
def parse_args():
argp = ArgumentParser(description=__doc__)
argp.add_argument("--host", default="0.0.0.0", help="Application host ip, default is 127.0.0.1.")
argp.add_argument("--port", default="5000", help="Application host ip, default is 5000.")
argp.add_argument("--results", default="../tck_engine/results", help="Path where the results are stored.")
argp.add_argument("--host", default="0.0.0.0",
help="Application host ip, default is 0.0.0.0.")
argp.add_argument("--port", default="5000",
help="Application host ip, default is 5000.")
argp.add_argument("--results", default="tck_engine/results",
help="Path where the results are stored.")
return argp.parse_args()
@ -24,83 +29,138 @@ def parse_args():
def results():
"""
Function accessed with route /result. Function lists
names of last tail result files added to results and
separeted by whitespace. Tail is a parameter given in
route. If tail is not given, function lists all files
from the last added file.
last tail result files added. Tail is a parameter given
in the route. If the tail is not given, function lists
last ten added files. If parameter last is true, only last
test result is returned.
@return:
string of file names separated by whitespace
json list of test results
"""
tail = request.args.get("tail")
return list_to_str(os.listdir(app.config["RESULTS_PATH"]), tail)
l = [f for f in os.listdir(app.config["RESULTS_PATH"])
if f != ".gitignore"]
return get_ret_list(l)
@app.route("/results/<dbname>")
def results_for_db(dbname):
"""
Function accessed with route /result/<dbname>. Function
lists names of last tail result files of database <dbname>
added to results and separeted by whitespace. Tail is a
parameter given in route. If tail is not given, function
lists all files of database <dbname> from the last added
file.
Function accessed with route /result/<dbname>. Function
lists last tail result files added of database <dbname.
Tail is a parameter given in the route. If tail is not
given, function lists last ten added files of database
<dbname>. If param last is true, only last test result
is returned.
@param dbname:
string, database name
@return:
string of file names separated by whitespace
json list of test results
"""
tail = request.args.get("tail")
return list_to_str(([f for f in os.listdir(app.config["RESULTS_PATH"]) if f.startswith(dbname)]), tail)
print(os.listdir(app.config["RESULTS_PATH"]))
l = [f for f in os.listdir(app.config["RESULTS_PATH"])
if f != ".gitignore" and f.split('-')[1] == dbname]
return get_ret_list(l)
@app.route("/results/<dbname>/<timestamp>")
def result(dbname, timestamp):
@app.route("/results/<dbname>/<test_suite>")
def result(dbname, test_suite):
"""
Function accessed with route /results/<dbname>/<timestamp>
Returns json of result file with name <dbname>_<timestamp>.json.
<timestamp> is in format yyyy_mm_dd__HH_MM.
Function accessed with route /results/<dbname>/<test_suite>
Function lists last tail result files added of database <dbname>
tested on <test_suite>. Tail is a parameter given in the
route. If tail is not given, function lists last ten results.
If param last is true, only last test result is returned.
@param dbname:
string, database name
@param timestamp:
string, timestamp from description
@param test_suite:
string, test suite of result file
@return:
json of a file.
json list of test results
"""
fname = dbname + "_" + timestamp + ".json"
with open(app.config["RESULTS_PATH"] + "/" + fname) as f:
json_data = json.load(f)
return jsonify(json_data)
fname = dbname + "-" + test_suite + ".json"
l = [f for f in os.listdir(app.config["RESULTS_PATH"])
if f.endswith(fname)]
return get_ret_list(l)
def list_to_str(l, tail):
def get_ret_list(l):
"""
Function returns first tail results of list l in decreasing
order as string separated by whitespace. If tail is None,
function returns string of whole list.
Function returns json list of test results of files given in
list l.
@param l:
list to return as string
@param tail:
number of results
list of file names
@return:
list as string
json list of test results
"""
l.sort()
ret_list = []
for f in l:
ret_list.append(get_content(f))
return list_to_json(
ret_list,
request.args.get("last"),
request.args.get("tail")
)
def get_content(fname):
"""
Function returns data of the json file fname located in
results directory in json format.
@param fname:
string, name of the file
@return:
json of a file
"""
with open(app.config["RESULTS_PATH"] + "/" + fname) as f:
json_data = json.load(f)
return json_data
def list_to_json(l, last, tail):
"""
Function converts list to json format. If last is true,
only the first item in list is returned list, else last
tail results are returned in json list. If tail is not
given, last ten results are returned in json list.
@param l:
list to convert to json format
@param last:
string from description
@param tail:
string from description
"""
l.reverse()
if len(l) == 0:
return jsonify(results=[])
if last == "true":
return jsonify(results=[l[0]])
if tail is None:
tail = len(l)
tail = max(tail, len(l))
return ' '.join(l[0:int(tail)])
tail = 10
else:
tail = int(tail)
if tail > len(l):
tail = len(l)
return jsonify(results=l[0:tail])
def main():
args = parse_args()
app.config.update(dict(
RESULTS_PATH=args.results
RESULTS_PATH=os.path.abspath(args.results)
))
app.run(
host = args.host,
port = int(args.port)
host=args.host,
port=int(args.port)
)
if __name__ == "__main__":
main()

33
tck_engine/environment.py
View File

@ -1,4 +1,7 @@
import logging, datetime, time, json
import logging
import datetime
import time
import json
from steps.test_parameters import TestParameters
from neo4j.v1 import GraphDatabase, basic_auth
from steps.graph_properties import GraphProperties
@ -6,34 +9,54 @@ from test_results import TestResults
test_results = TestResults()
def before_scenario(context, step):
context.test_parameters = TestParameters()
context.graph_properties = GraphProperties()
context.exception = None
def after_scenario(context, scenario):
test_results.add_test(scenario.status)
def before_all(context):
set_logging(context)
context.driver = create_db_driver(context)
def after_all(context):
ts = time.time()
timestamp = datetime.datetime.fromtimestamp(ts).strftime("%Y_%m_%d__%H_%M")
file_name = context.config.output_folder + context.config.database + "_" + timestamp + ".json"
js = {"total": test_results.num_total(), "passed": test_results.num_passed(), "test_suite": context.config.root}
root = context.config.root
if root.endswith("/"):
root = root[0:len(root) - 1]
if root.endswith("features"):
root = root[0: len(root) - len("features") - 1]
test_suite = root.split('/')[-1]
file_name = context.config.output_folder + timestamp + \
"-" + context.config.database + "-" + test_suite + ".json"
js = {
"total": test_results.num_total(), "passed": test_results.num_passed(),
"test_suite": test_suite, "timestamp": timestamp, "db": context.config.database}
with open(file_name, 'w') as f:
json.dump(js, f)
def set_logging(context):
logging.basicConfig(level="DEBUG")
log = logging.getLogger(__name__)
context.log = log
def create_db_driver(context):
uri = context.config.database_uri
auth_token = basic_auth(context.config.database_username, context.config.database_password)
auth_token = basic_auth(
context.config.database_username, context.config.database_password)
if context.config.database == "neo4j" or context.config.database == "memgraph":
driver = GraphDatabase.driver(uri, auth=auth_token, encrypted=0)
else:

4
tck_engine/steps/binary_tree.py
View File

@ -1,9 +1,11 @@
from behave import *
import graph
@given(u'the binary-tree-1 graph')
def step_impl(context):
graph.create_graph('binary-tree-1', context)
graph.create_graph('binary-tree-1', context)
@given(u'the binary-tree-2 graph')
def step_impl(context):

23
tck_engine/steps/database.py
View File

@ -1,10 +1,10 @@
def query(q, context, params={}):
"""
Function used to execute query on database. Query results are
Function used to execute query on database. Query results are
set in context.result_list. If exception occurs, it is set on
context.exception.
@param q:
@param q:
String, database query.
@param context:
behave.runner.Context, context of all tests.
@ -17,7 +17,7 @@ def query(q, context, params={}):
if context.config.database == "neo4j":
session = driver.session()
try:
#executing query
# executing query
with session.begin_transaction() as tx:
results = tx.run(q, params)
summary = results.summary()
@ -27,11 +27,11 @@ def query(q, context, params={}):
tx.success = True
session.close()
except Exception as e:
#exception
# exception
context.exception = e
context.log.info('%s', str(e))
session.close()
#not working if removed
# not working if removed
query("match (n) detach delete(n)", context)
return results_list
@ -44,23 +44,22 @@ def add_side_effects(context, counters):
behave.runner.Context, context of all tests.
"""
graph_properties = context.graph_properties
#check nodes
# check nodes
if counters.nodes_deleted > 0:
graph_properties.change_nodes(-counters.nodes_deleted)
if counters.nodes_created > 0:
graph_properties.change_nodes(counters.nodes_created)
#check relationships
# check relationships
if counters.relationships_deleted > 0:
graph_properties.change_relationships(-counters.relationships_deleted)
if counters.relationships_created > 0:
graph_properties.change_relationships(counters.relationships_created)
#check labels
graph_properties.change_relationships(counters.relationships_created)
# check labels
if counters.labels_removed > 0:
graph_properties.change_labels(-counters.labels_removed)
if counters.labels_added > 0:
graph_properties.change_labels(counters.labels_added)
#check properties
# check properties
if counters.properties_set > 0:
graph_properties.change_properties(counters.properties_set)

48
tck_engine/steps/errors.py
View File

@ -1,6 +1,8 @@
from behave import *
#TODO check for exact error?
# TODO check for exact error?
def handle_error(context):
"""
Function checks if exception exists in context.
@ -11,167 +13,207 @@ def handle_error(context):
"""
assert(context.exception is not None)
@then('a SyntaxError should be raised at compile time: NestedAggregation')
def syntax_error(context):
handle_error(context)
@then('TypeError should be raised at compile time: IncomparableValues')
def type_error(context):
handle_error(context)
@then(u'a TypeError should be raised at compile time: IncomparableValues')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: RequiresDirectedRelationship')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidRelationshipPattern')
def syntax_error(context):
handle_error(context)
@then(u'a TypeError should be raised at runtime: MapElementAccessByNonString')
def type_error(context):
handle_error(context)
@then(u'a ConstraintVerificationFailed should be raised at runtime: DeleteConnectedNode')
def step(context):
handle_error(context)
@then(u'a TypeError should be raised at runtime: ListElementAccessByNonInteger')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidArgumentType')
def step(context):
handle_error(context)
@then(u'a TypeError should be raised at runtime: InvalidElementAccess')
def step(context):
handle_error(context)
@then(u'a ArgumentError should be raised at runtime: NumberOutOfRange')
def step(context):
handle_error(context)
@then(u'a TypeError should be raised at runtime: InvalidArgumentValue')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: VariableAlreadyBound')
def step(context):
handle_error(context)
@then(u'a TypeError should be raised at runtime: IncomparableValues')
def step(context):
handle_error(context)
@then(u'a TypeError should be raised at runtime: PropertyAccessOnNonMap')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidUnicodeLiteral')
def step(context):
handle_error(context)
@then(u'a SemanticError should be raised at compile time: MergeReadOwnWrites')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidAggregation')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: NoExpressionAlias')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: UndefinedVariable')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: VariableTypeConflict')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: DifferentColumnsInUnion')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidClauseComposition')
def step(context):
handle_error(context)
@then(u'a TypeError should be raised at compile time: InvalidPropertyType')
def step(context):
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: ColumnNameConflict')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: NoVariablesInScope')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidDelete')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: NegativeIntegerArgument')
def step(context):
handle_error(context)
@then(u'a EntityNotFound should be raised at runtime: DeletedEntityAccess')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: RelationshipUniquenessViolation')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: CreatingVarLength')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidParameterUse')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: FloatingPointOverflow')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time InvalidArgumentExpression')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time InvalidUnicodeCharacter')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: NonConstantExpression')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: NoSingleRelationshipType')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: UnknownFunction')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidNumberLiteral')
def step_impl(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidArgumentExpression')
def step(context):
handle_error(context)
@then(u'a SyntaxError should be raised at compile time: InvalidUnicodeCharacter')
def step(context):
handle_error(context)

18
tck_engine/steps/graph.py
View File

@ -1,5 +1,5 @@
import database, os
from graph_properties import GraphProperties
import database
import os
from behave import *
@ -8,23 +8,26 @@ def empty_graph_step(context):
database.query("MATCH (n) DETACH DELETE n", context)
context.graph_properties.set_beginning_parameters()
@given('any graph')
def any_graph_step(context):
database.query("MATCH (n) DETACH DELETE n", context)
context.graph_properties.set_beginning_parameters()
@given('graph "{name}"')
def graph_name(context, name):
create_graph(name, context)
def create_graph(name, context):
"""
Function deletes everything from database and creates a new
graph. Graph file name is an argument of function. Function
graph. Graph file name is an argument of function. Function
executes queries written in a .cypher file separated by ';'
and sets graph properties to beginning values.
"""
database.query("MATCH (n) DETACH DELETE n", context)
database.query("MATCH (n) DETACH DELETE n", context)
path = find_graph_path(name, context.config.graphs_root)
q_marks = ["'", '"', '`']
@ -36,8 +39,8 @@ def create_graph(name, context):
i = 0
while i < len(content):
ch = content[i]
if ch == '\\' and i != len(content)-1 and content[i+1] in q_marks:
q += ch + content[i+1]
if ch == '\\' and i != len(content) - 1 and content[i + 1] in q_marks:
q += ch + content[i + 1]
i += 2
else:
q += ch
@ -45,7 +48,6 @@ def create_graph(name, context):
in_string = False
elif ch in q_marks:
in_string = True
q_mark = ch
if ch == ';' and not in_string:
database.query(q, context)
q = ''
@ -58,7 +60,7 @@ def create_graph(name, context):
def find_graph_path(name, path):
"""
Function returns path to .cypher file with given name in
given folder or subfolders. Argument path is path to a given
given folder or subfolders. Argument path is path to a given
folder.
"""
for root, dirs, files in os.walk(path):

19
tck_engine/steps/graph_properties.py
View File

@ -1,15 +1,15 @@
import steps.database
class GraphProperties:
class GraphProperties:
"""
Class used to store changes(side effects of queries)
to graph parameters(nodes, relationships, labels and
Class used to store changes(side effects of queries)
to graph parameters(nodes, relationships, labels and
properties) when executing queries.
"""
def set_beginning_parameters(self):
"""
Method sets parameters to empty lists.
@param self:
Instance of a class.
"""
@ -21,7 +21,7 @@ class GraphProperties:
def __init__(self):
"""
Method sets parameters to empty lists.
@param self:
Instance of a class.
"""
@ -49,7 +49,7 @@ class GraphProperties:
@param self:
Instance of a class.
@param dif:
Int, difference between number of relationships
Int, difference between number of relationships
before and after executing query.
"""
self.relationships.append(dif)
@ -77,7 +77,8 @@ class GraphProperties:
"""
self.properties.append(dif)
def compare(self, nodes_dif, relationships_dif, labels_dif, properties_dif):
def compare(self, nodes_dif, relationships_dif, labels_dif,
properties_dif):
"""
Method used to compare side effects from executing
queries and an expected result from a cucumber test.
@ -88,7 +89,7 @@ class GraphProperties:
List of all expected node side effects in order
when executing query.
@param relationships_dif:
List of all expected relationship side effects
List of all expected relationship side effects
in order when executing query.
@param labels_dif:
List of all expected label side effects in order

34
tck_engine/steps/parser.py
View File

@ -28,6 +28,7 @@ def parse(el, ignore_order):
return parse_rel(el, ignore_order)
return el
def is_list(el):
"""
Function returns true if string el is a list, else false.
@ -40,6 +41,7 @@ def is_list(el):
return False
return True
def parse_path(path, ignore_order):
"""
Function used to parse path.
@ -49,8 +51,8 @@ def parse_path(path, ignore_order):
parsed path
"""
parsed_path = '<'
dif_open_closed_brackets = 0;
for i in range(1, len(path)-1):
dif_open_closed_brackets = 0
for i in range(1, len(path) - 1):
if path[i] == '(' or path[i] == '{' or path[i] == '[':
dif_open_closed_brackets += 1
if dif_open_closed_brackets == 1:
@ -58,12 +60,13 @@ def parse_path(path, ignore_order):
if path[i] == ')' or path[i] == '}' or path[i] == ']':
dif_open_closed_brackets -= 1
if dif_open_closed_brackets == 0:
parsed_path += parse(path[start:(i+1)], ignore_order)
parsed_path += parse(path[start:(i + 1)], ignore_order)
elif dif_open_closed_brackets == 0:
parsed_path += path[i]
parsed_path += '>'
return parsed_path
def parse_node(node_str, ignore_order):
"""
Function used to parse node.
@ -75,7 +78,7 @@ def parse_node(node_str, ignore_order):
label = ''
labels = []
props_start = None
for i in range(1, len(node_str)):
for i in range(1, len(node_str)):
if node_str[i] == ':' or node_str[i] == ')' or node_str[i] == '{':
if label.startswith(':'):
labels.append(label)
@ -88,13 +91,15 @@ def parse_node(node_str, ignore_order):
labels.sort()
parsed_node = '('
for label in labels:
for label in labels:
parsed_node += label
if props_start is not None:
parsed_node += parse_map(node_str[props_start:len(node_str)-1], ignore_order)
parsed_node += parse_map(
node_str[props_start:len(node_str) - 1], ignore_order)
parsed_node += ')'
return parsed_node
def parse_map(props, ignore_order):
"""
Function used to parse map.
@ -106,7 +111,7 @@ def parse_map(props, ignore_order):
dif_open_closed_brackets = 0
prop = ''
list_props = []
for i in range(1, len(props)-1):
for i in range(1, len(props) - 1):
if props[i] == ',' and dif_open_closed_brackets == 0:
list_props.append(prop_to_str(prop, ignore_order))
prop = ''
@ -119,10 +124,10 @@ def parse_map(props, ignore_order):
if prop != '':
list_props.append(prop_to_str(prop, ignore_order))
list_props.sort()
return '{' + ','.join(list_props) + '}'
def prop_to_str(prop, ignore_order):
"""
Function used to parse one pair of key, value in format 'key:value'.
@ -137,6 +142,7 @@ def prop_to_str(prop, ignore_order):
val = prop.split(':', 1)[1]
return key + ":" + parse(val, ignore_order)
def parse_list(l, ignore_order):
"""
Function used to parse list.
@ -148,7 +154,7 @@ def parse_list(l, ignore_order):
dif_open_closed_brackets = 0
el = ''
list_el = []
for i in range(1, len(l)-1):
for i in range(1, len(l) - 1):
if l[i] == ',' and dif_open_closed_brackets == 0:
list_el.append(parse(el, ignore_order))
el = ''
@ -163,9 +169,10 @@ def parse_list(l, ignore_order):
if ignore_order:
list_el.sort()
return '[' + ','.join(list_el) + ']'
def parse_rel(rel, ignore_order):
"""
Function used to parse relationship.
@ -177,7 +184,7 @@ def parse_rel(rel, ignore_order):
label = ''
labels = []
props_start = None
for i in range(1, len(rel)):
for i in range(1, len(rel)):
if rel[i] == ':' or rel[i] == ']' or rel[i] == '{':
if label.startswith(':'):
labels.append(label)
@ -190,10 +197,9 @@ def parse_rel(rel, ignore_order):
labels.sort()
parsed_rel = '['
for label in labels:
for label in labels:
parsed_rel += label
if props_start is not None:
parsed_rel += parse_map(rel[props_start:len(rel)-1], ignore_order)
parsed_rel += parse_map(rel[props_start:len(rel) - 1], ignore_order)
parsed_rel += ']'
return parsed_rel

93
tck_engine/steps/query.py
View File

@ -1,29 +1,38 @@
import json
import database, parser
import database
import parser
from behave import *
from neo4j.v1.types import Node, Path, Relationship
@given('parameters are')
def parameters_step(context):
context.test_parameters.set_parameters_from_table(context.table)
@then('parameters are')
@then('parameters are')
def parameters_step(context):
context.test_parameters.set_parameters_from_table(context.table)
@step('having executed')
def having_executed_step(context):
context.results = database.query(context.text, context, context.test_parameters.get_parameters())
context.results = database.query(
context.text, context, context.test_parameters.get_parameters())
context.graph_properties.set_beginning_parameters()
@when('executing query')
def executing_query_step(context):
context.results = database.query(context.text, context, context.test_parameters.get_parameters())
context.results = database.query(
context.text, context, context.test_parameters.get_parameters())
@when('executing control query')
def executing_query_step(context):
context.results = database.query(context.text, context, context.test_parameters.get_parameters())
context.results = database.query(
context.text, context, context.test_parameters.get_parameters())
def parse_props(prop_json):
"""
@ -67,25 +76,25 @@ def to_string(element):
String of parsed element.
"""
if element is None:
#parsing None
# parsing None
return "null"
if isinstance(element, Node):
#parsing Node
# parsing Node
sol = "("
if element.labels:
sol += ':' + ': '.join(element.labels)
if element.properties:
if element.labels:
sol += ' '
sol += parse_props(element.properties)
sol += parse_props(element.properties)
sol += ")"
return sol
elif isinstance(element, Relationship):
#parsing Relationship
# parsing Relationship
sol = "[:"
if element.type:
sol += element.type
@ -96,7 +105,7 @@ def to_string(element):
return sol
elif isinstance(element, Path):
#parsing Path
# parsing Path
# TODO add longer paths
edges = []
nodes = []
@ -113,18 +122,18 @@ def to_string(element):
sol += nodes[i][1] + "-" + edges[i][1] + "->"
else:
sol += nodes[i][1] + "<-" + edges[i][1] + "-"
sol += nodes[len(edges)][1]
sol += ">"
return sol
elif isinstance(element, str):
#parsing string
# parsing string
return "'" + element + "'"
elif isinstance(element, list):
#parsing list
# parsing list
sol = '['
el_str = []
for el in element:
@ -135,13 +144,13 @@ def to_string(element):
return sol
elif isinstance(element, bool):
#parsing bool
# parsing bool
if element:
return "true"
return "false"
elif isinstance(element, dict):
#parsing map
# parsing map
if len(element) == 0:
return '{}'
sol = '{'
@ -149,19 +158,19 @@ def to_string(element):
sol += key + ':' + to_string(val) + ','
sol = sol[:-1] + '}'
return sol
elif isinstance(element, float):
#parsing float, scientific
# parsing float, scientific
if 'e' in str(element):
if str(element)[-3] == '-':
zeroes = int(str(element)[-2:])-1
zeroes = int(str(element)[-2:]) - 1
num_str = ''
if str(element)[0] == '-':
num_str += '-'
num_str += '.' + zeroes * '0' + str(element)[:-4].replace("-", "").replace(".", "")
num_str += '.' + zeroes * '0' + \
str(element)[:-4].replace("-", "").replace(".", "")
return num_str
return str(element)
@ -174,7 +183,7 @@ def get_result_rows(context, ignore_order):
behave.runner.Context, behave context.
@param ignore_order:
bool, ignore order in result and expected list.
@return
@return
Result rows.
"""
result_rows = []
@ -182,7 +191,8 @@ def get_result_rows(context, ignore_order):
keys = result.keys()
values = result.values()
for i in range(0, len(keys)):
result_rows.append(keys[i] + ":" + parser.parse(to_string(values[i]).replace("\n", "\\n").replace(" ", ""), ignore_order))
result_rows.append(keys[i] + ":" + parser.parse(
to_string(values[i]).replace("\n", "\\n").replace(" ", ""), ignore_order))
return result_rows
@ -200,9 +210,11 @@ def get_expected_rows(context, ignore_order):
expected_rows = []
for row in context.table:
for col in context.table.headings:
expected_rows.append(col + ":" + parser.parse(row[col].replace(" ", ""), ignore_order))
expected_rows.append(
col + ":" + parser.parse(row[col].replace(" ", ""), ignore_order))
return expected_rows
def validate(context, ignore_order):
"""
Function used to check if results from database are same
@ -218,7 +230,7 @@ def validate(context, ignore_order):
context.log.info("Expected: %s", str(expected_rows))
context.log.info("Results: %s", str(result_rows))
assert(len(expected_rows) == len(result_rows))
assert(len(expected_rows) == len(result_rows))
for i in range(0, len(expected_rows)):
if expected_rows[i] in result_rows:
@ -243,37 +255,43 @@ def validate_in_order(context, ignore_order):
context.log.info("Expected: %s", str(expected_rows))
context.log.info("Results: %s", str(result_rows))
assert(len(expected_rows) == len(result_rows))
assert(len(expected_rows) == len(result_rows))
for i in range(0, len(expected_rows)):
if expected_rows[i] != result_rows[i]:
assert(False)
@then('the result should be')
def expected_result_step(context):
validate(context, False)
check_exception(context)
@then('the result should be, in order')
def expected_result_step(context):
validate_in_order(context, False)
check_exception(context)
@then('the result should be (ignoring element order for lists)')
def expected_result_step(context):
validate(context, True)
check_exception(context)
def check_exception(context):
if context.exception is not None:
context.log.info("Exception when eqecuting query!")
assert(False)
@then('the result should be empty')
def empty_result_step(context):
assert(len(context.results) == 0)
check_exception(context)
def side_effects_number(prop, table):
"""
Function returns an expected list of side effects for property prop
@ -284,7 +302,7 @@ def side_effects_number(prop, table):
labels or properties.
@param table:
behave.model.Table, context table with side effects.
@return
@return
Description.
"""
ret = []
@ -293,32 +311,35 @@ def side_effects_number(prop, table):
if row[0][0] == '+':
sign = 1
if row[0][1:] == prop:
ret.append(int(row[1])*sign)
ret.append(int(row[1]) * sign)
sign = -1
row = table.headings
if row[0][0] == '+':
sign = 1
if row[0][1:] == prop:
ret.append(int(row[1])*sign)
ret.append(int(row[1]) * sign)
ret.sort()
return ret
@then('the side effects should be')
def side_effects_step(context):
if context.config.no_side_effects:
return
table = context.table
#get side effects from db queries
# get side effects from db queries
nodes_dif = side_effects_number("nodes", table)
relationships_dif = side_effects_number("relationships", table)
labels_dif = side_effects_number("labels", table)
properties_dif = side_effects_number("properties", table)
#compare side effects
assert(context.graph_properties.compare(nodes_dif, relationships_dif, labels_dif, properties_dif) == True)
# compare side effects
assert(context.graph_properties.compare(nodes_dif,
relationships_dif, labels_dif, properties_dif) == True)
@then('no side effects')
def side_effects_step(context):
if context.config.no_side_effects:
return
#check if side effects are non existing
# check if side effects are non existing
assert(context.graph_properties.compare([], [], [], []) == True)

25
tck_engine/steps/test_parameters.py
View File

@ -1,9 +1,12 @@
import yaml
class TestParameters:
"""
Class used to store parameters from a cucumber test.
"""
def __init__(self):
"""
Constructor initializes parameters to empty dict.
@ -24,18 +27,22 @@ class TestParameters:
par = dict()
for row in table:
par[row[0]] = self.parse_parameters(row[1])
if isinstance(par[row[0]], str) and par[row[0]].startswith("'") and par[row[0]].endswith("'"):
par[row[0]] = par[row[0]][1:len(par[row[0]])-1]
if isinstance(par[row[0]], str) and par[row[0]].startswith("'") \
and par[row[0]].endswith("'"):
par[row[0]] = par[row[0]][1:len(par[row[0]]) - 1]
par[table.headings[0]] = self.parse_parameters(table.headings[1])
if isinstance(par[table.headings[0]], str) and par[table.headings[0]].startswith("'") and par[table.headings[0]].endswith("'"):
par[table.headings[0]] = par[table.headings[0]][1:len(par[table.headings[0]])-1]
if isinstance(par[table.headings[0]], str) and \
par[table.headings[0]].startswith("'") and \
par[table.headings[0]].endswith("'"):
par[table.headings[0]] = \
par[table.headings[0]][1:len(par[table.headings[0]]) - 1]
self.parameters = par
def get_parameters(self):
"""
Method returns parameters.
@param self:
Instance of a class.
return:
@ -45,10 +52,10 @@ class TestParameters:
def parse_parameters(self, val):
"""
Method used for parsing parameters given in a cucumber test table
Method used for parsing parameters given in a cucumber test table
to a readable format for a database.
Integers are parsed to int values, floats to float values, bools
to bool values, null to None and structures are recursively
to bool values, null to None and structures are recursively
parsed and returned.
@param val:
@ -58,5 +65,3 @@ class TestParameters:
"""
return yaml.load(val)

60
tck_engine/test_executor.py
View File

@ -3,44 +3,56 @@ from behave import configuration
from argparse import ArgumentParser
import os
def parse_args():
argp = ArgumentParser(description=__doc__)
argp.add_argument("--root", default="tck_engine/tests/openCypher_M05/tck/features",
help="Path to folder where tests are located, default is openCypher_M05/tck/features.")
argp.add_argument("--graphs-root", default="tck_engine/tests/openCypher_M05/tck/graphs",
argp.add_argument("--root", default="tck_engine/tests/openCypher_M05",
help="Path to folder where tests are located, default is openCypher_M05/tck/features.")
argp.add_argument(
"--graphs-root", default="tck_engine/tests/openCypher_M05/tck/graphs",
help="Path to folder where files with graphs queries are located, default is openCypher_M05/tck/graphs.")
argp.add_argument("--stop", action="store_true", help="Stop testing after first fail.")
argp.add_argument("--no-side-effects", action="store_true", help="Check for side effects in tests.")
argp.add_argument("--db", default="neo4j", choices=["neo4j", "memgraph"], help="Default is neo4j.")
argp.add_argument(
"--stop", action="store_true", help="Stop testing after first fail.")
argp.add_argument("--no-side-effects", action="store_true",
help="Check for side effects in tests.")
argp.add_argument("--db", default="neo4j", choices=[
"neo4j", "memgraph"], help="Default is neo4j.")
argp.add_argument("--db-user", default="neo4j", help="Default is neo4j.")
argp.add_argument("--db-pass", default="memgraph", help="Default is memgraph.")
argp.add_argument("--db-uri", default="bolt://localhost:7687", help="Default is bolt://localhost:7687.")
argp.add_argument("--output-folder", default="tck_engine/results/", help="Test result output folder, default is results/.")
argp.add_argument("--logging", default="DEBUG", choices=["INFO", "DEBUG"], help="Logging level, default is DEBUG.")
argp.add_argument(
"--db-pass", default="memgraph", help="Default is memgraph.")
argp.add_argument("--db-uri", default="bolt://localhost:7687",
help="Default is bolt://localhost:7687.")
argp.add_argument("--output-folder", default="tck_engine/results/",
help="Test result output folder, default is results/.")
argp.add_argument("--logging", default="DEBUG", choices=[
"INFO", "DEBUG"], help="Logging level, default is DEBUG.")
return argp.parse_args()
def main():
"""
Script used to run behave tests with given options. List of
Script used to run behave tests with given options. List of
options is available when running python test_executor.py -help.
"""
args = parse_args()
tests_root = os.path.abspath(args.root)
#adds options to cucumber configuration
add_config("--no-side-effects", dict(action="store_true", help = "Exclude side effects."))
add_config("--database", dict(help = "Choose database(memgraph/neo4j)."))
add_config("--database-password", dict(help = "Database password."))
add_config("--database-username", dict(help = "Database username."))
add_config("--database-uri", dict(help = "Database uri."))
add_config("--graphs-root", dict(help = "Path to folder where graphs are given."))
add_config("--output-folder", dict(help = "Folder where results of tests are written."))
add_config("--root", dict(help = "Folder with test features."))
# adds options to cucumber configuration
add_config("--no-side-effects",
dict(action="store_true", help="Exclude side effects."))
add_config("--database", dict(help="Choose database(memgraph/neo4j)."))
add_config("--database-password", dict(help="Database password."))
add_config("--database-username", dict(help="Database username."))
add_config("--database-uri", dict(help="Database uri."))
add_config("--graphs-root",
dict(help="Path to folder where graphs are given."))
add_config("--output-folder", dict(
help="Folder where results of tests are written."))
add_config("--root", dict(help="Folder with test features."))
#list with all options
#options will be passed to the cucumber engine
# list with all options
# options will be passed to the cucumber engine
behave_options = [tests_root]
if args.stop:
behave_options.append("--stop")
@ -61,7 +73,7 @@ def main():
behave_options.append("--root")
behave_options.append(args.root)
#runs tests with options
# runs tests with options
behave_main(behave_options)

6
tck_engine/test_results.py
View File

@ -1,6 +1,7 @@
class TestResults:
"""
Clas used to store test results. It has parameters total
Clas used to store test results. It has parameters total
and passed.
@attribute total:
@ -8,6 +9,7 @@ class TestResults:
@attribute passed:
int, number of passed scenarios.
"""
def __init__(self):
self.total = 0
self.passed = 0
@ -26,7 +28,7 @@ class TestResults:
def add_test(self, status):
"""
Method adds one scenario to current results. If
Method adds one scenario to current results. If
scenario passed, number of passed scenarios increases.
@param status: