Commit Graph

64 Commits

Author SHA1 Message Date
jbajic
60ad05acff Address review comments 2022-03-14 15:47:41 +01:00
jbajic
4f593c7fca Adapt precommit to run through all files 2022-03-14 15:47:41 +01:00
jbajic
12b4ec1589 Add memgraph namespace 2022-03-14 15:47:41 +01:00
Antonio Andelic
df689dccd5
Make Memgraph work with fetch content () 2021-10-14 18:58:42 +02:00
Matej Ferencevic
de48548164 Move RPC to root source directory
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2641
2020-01-27 11:50:24 +01:00
Teon Banek
6a2dc1eb9d cmake: Allow absolute paths in define_add_lcp
Reviewers: mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2240
2019-07-25 16:45:24 +02:00
Lovro Lugovic
0b2240e6e4 LCP: Split up PROCESS-FILE
Summary:
Split up `process-file` so that looking at the generated code for an LCP form is
easier from the REPL.

`process-lcp`, `generate-hpp` and `generate-cpp` now perform the generation of
C++ code, but take a list of "C++ elements" (the results of LCP forms) as input
and write their output to streams. They do no reading/evaluating of LCP forms of
their own.

`read-lcp` and `read-lcp-file` are used to read and evaluate a stream of LCP
forms. The latter is a specialized version for file streams which also reports
the position of the form within the file when an error happens.

`process-lcp-string` and `process-lcp-file` are convenient wrappers around the
main functionality that take a string (file) and output to strings (files).
Using `read-lcp` and `read-lcp-file` they process LCP forms and pass them off to
`process-lcp` for code generation.

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2097
2019-06-12 09:38:41 +02:00
Lovro Lugovic
21d05e9360 LCP: Add resolution for SLK-OPTS and CLONE-OPTS
Summary: Depends on D2094

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2095
2019-06-12 09:38:05 +02:00
Lovro Lugovic
dbd226d05d LCP: Minor improvements
Summary:
- Add a unified `generate_lcp` target
- Simplify SLK-ERROR and CLONE-ERROR
- Rework WITH-VARS
- Expect CPP-CLASS, not CPP-TYPE
- Fix CPP-TYPE-REFERENCE-P
- Add CPP-GENSYM
- Add util.lisp to LCP's source files in CMake
- Make the CMake variable `lcp_src_files` a cache (persistent) variable
- Add `lcp_src_files` as a dependency to `test_lcp`
- Rename `lcp_compile` to `compile-lcp`
- Fetch docstrings for CPP-NAME-* functions at run-time
- Remove existing C++ entities on redefinition

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2094
2019-06-12 09:38:02 +02:00
Lovro Lugovic
0732264fd5 LCP: Make :PUBLIC, :PROTECTED and :PRIVATE options aggregating
Summary: Depends on D2092

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2093
2019-06-12 09:37:59 +02:00
Lovro Lugovic
2736f31dcf LCP: Rework DEFINE-CLASS and DEFINE-ENUM
Summary: Depends on D2091

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2092
2019-06-12 09:37:55 +02:00
Lovro Lugovic
b8514ccf04 LCP: Update documentation
Summary: Depends on D2090

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2091
2019-06-12 09:37:52 +02:00
Lovro Lugovic
59af45f94e LCP: Fix up LCP warnings
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2090
2019-06-12 09:37:47 +02:00
Lovro Lugovic
017dec8c0e LCP: Refactor the type representation
Summary:
# Summary

## Concepts and terminology

A **namestring for <object>** is a Lisp string that names the C++ language element
`<object>` (such as a namespace, a variable, a class, etc.). Therefore we have a
"namestring for a namespace", a "namestring for a variable", etc.

A **designator for a namestring for <object>** is a Lisp object that denotes a
"namestring for <object>". These are symbols and strings themselves.

A **typestring** is a Lisp string that represents a C++ type, i.e. its
declaration.

A **typestring designator** is a Lisp object that denotes a typestring.

A typestring (and the corresponding C++ type) is said to be **supported** if it
can be parsed using `parse-cpp-type-declaration`. This concept is important and
should form the base of our design because we can't really hope to ever support
all of C++'s type declarations.

A typestring (and the corresponding C++ type) that is not supported is
**unsupported**.

A **processed typestring** is a typestring that is either fully qualified or
unqualified.

A C++ type is said to be **known** if LCP knows extra information about the type,
rather than just what kind of type it is, in which namespace it lives, etc. For
now, the only known types are those that are defined within LCP itself using
`define-class` & co.

A C++ type is **unknown** if it is not known.

**Typestring resolution** is the process of resolving a (processed) typestring
into an instance of `cpp-type` or `unsupported-cpp-type`.

**Resolving accessors** are accessors which perform typestring resolution.

## Changes

Explicitly introduce the concept of supported and known types. `cpp-type` models
supported types while `unsupported-cpp-type` models unsupported types.
Subclasses of `cpp-type` model known types. `general-cpp-type` is either a
`cpp-type` or an `unsupported-cpp-type`.

Add various type queries.

Fix `define-rpc`'s `:initarg` (remove it from the `cpp-member` struct).

Introduce namestrings and their designators in `names.lisp`.

Introduce typestrings and their designators.

Introduce **processed typestrings**. Our DSL's macros (`define-class` & co.)
convert all of the given typestrings into processed typestrings because we don't
attempt to support partially qualified names and relative name lookup yet. A
warning is signalled when a partially qualified name is treated as a fully
qualified name.

The slots of `cpp-type`, `cpp-class`, `cpp-member` and `cpp-capnp-opts` now
store processed typestrings which are lazily resolved into their corresponding
C++ types.

The only thing that instances of `unsupported-cpp-type` are good for is getting
the typestring that was used to construct them. Most of LCP's functions only
work with known C++ types, i.e. `cpp-type` instances. The only function so far
that works for both of them is `cpp-type-decl`.

Since "unsupportedness" is now explicitly part of LCP, client code is expected
to manually check whether a returned type is unsupported or not (or face
receiving an error otherwise), unless a function is documented to return only
`cpp-type` instances.

A similar thing goes for "knowness". Client code is expected to manually check
whether a returned type is known or not, unless a function is documented to
return only (instances of `cpp-type` subclasses) known types.

## TODO

Resolution still has to be done for the following slots of the
following structures:

-   `slk-opts`
    -   `save-args`
    -   `load-args`

-   `clone-opts`
    -   `args`
    -   `return-type`

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1962
2019-05-10 16:18:31 +02:00
Lovro Lugovic
e8c82e36e2 LCP: Remove Cap'n Proto
Summary: Depends on D1947

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2034
2019-05-10 16:10:22 +02:00
Lovro Lugovic
390b6c1557 LCP: Small test fixes
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1947
2019-05-10 16:10:17 +02:00
Lovro Lugovic
9d6e025304 LCP: SLIME debugging support when used as a tool
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1929
2019-05-10 16:09:28 +02:00
Matej Ferencevic
5c244c1ad4 Remove Cap'n Proto
Summary:
There will be a lot of leftover files, execute the following commands inside
`src/` to remove them:
```
git clean -xf
rm -r rpc/ storage/single_node_ha/rpc/
```

Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D2011
2019-05-08 10:51:10 +02:00
Matej Ferencevic
129c6c0242 Finish SLK implementation
Reviewers: teon.banek, msantl

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1999
2019-05-02 15:47:38 +02:00
Teon Banek
95f4d1c3fa Generate static Save and Load methods for RPCs
Reviewers: mtomic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1995
2019-04-29 13:42:17 +02:00
Lovro Lugovic
5dc362d9f5 LCP: Use named-readtables
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1928
2019-04-29 12:20:06 +02:00
Matej Ferencevic
9291a5fc4d Migrate to C++17
Reviewers: teon.banek, buda

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1974
2019-04-23 14:46:44 +02:00
Lovro Lugovic
11f3cb8838 Undefine CPP types at the end of LCP's test
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1915
2019-03-11 14:45:32 +01:00
Lovro Lugovic
0679997e8f Use the FIVEAM reporter in LCP's test
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D1914
2019-03-11 13:21:38 +01:00
Lovro Lugovic
c15c16c577 Prevent result accumulation within LCP's tests
Summary:
PROVE:PLAN clears the previously stored results for the current suite (which is
the suite associated with the current package) and prevents "result
accumulation" (and the accompanying huge and partly outdated reports).

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1909
2019-03-11 13:12:39 +01:00
Matej Ferencevic
bb052be002 Remove serialization from utils
Reviewers: teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1852
2019-02-13 15:41:40 +01:00
Teon Banek
f91428f23f Generate virtual GetTypeInfo with LCP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1827
2019-01-25 14:12:50 +01:00
Marin Tomic
914f40411a Implement plan cloning with LCP
Summary:
Implement proper plan cloning using LCP instead of hacking it with
serialization.

depends on D1815

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1816
2019-01-22 14:52:06 +01:00
Marin Tomic
f8e76efa3e Implement cloning in LCP
Summary:
Add automatic generating of cloning (deep-copy) functions for LCP
defined classes. This enables us to remove a bunch of manually written `Clone`
functions from AST and also to implement logical plan cloning properly (before
it was using serialization).

Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1808
2019-01-18 10:34:35 +01:00
Lovro Lugovic
0436bf77ce LCP: Minor fixes
Summary:
- Rename `type` to `cpp-type` for consistency
- Fix docs
- Guard against simultaneous `type-params` and `type-args`
- Explicitly initialize to the empty list
- Fix test

Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1806
2019-01-15 16:55:12 +01:00
Lovro Lugovic
bd9f900722 LCP: Properly print instances of CPP-TYPE
Reviewers: mtomic, teon.banek

Reviewed By: teon.banek

Differential Revision: https://phabricator.memgraph.io/D1805
2019-01-15 16:04:43 +01:00
Teon Banek
68b2dcc490 Serialize RPC messages using SLK
Reviewers: msantl, mtomic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1771
2018-12-14 14:24:39 +01:00
Teon Banek
7638b09867 Generate SLK serialization from LCP
Summary:
Classes marked with `:serialize (:slk)` will now generate SLK
serialization code. This diff also changes how the `:serialize` option
is parsed, so that multiple different serialization backends are
supported.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1755
2018-12-05 14:58:39 +01:00
Teon Banek
7a8e6b52e0 Implement LCP class serialization for SLK
Summary:
This should cover the minimum required feature set for generating the
serialization code for SLK. There are some TODO comments, mostly
concerning quality of life improvements. The documentation on LCP has
been updated.

Additionally, any previous CHECK which would trigger if loading went
wrong is now replaced by raising SlkDecodeException. Other assertions of
code misuse are left as CHECK invocations.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1754
2018-12-04 12:54:58 +01:00
Teon Banek
7d01ba5178 Use generic :dont-save mechanism in LCP
Summary:
It doesn't make sense to tie skipping a member for serialization to a
particular serialization implementation.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1743
2018-11-22 10:01:38 +01:00
Teon Banek
a75008f94b Support SLK serialization of enums in LCP
Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1737
2018-11-21 13:28:41 +01:00
Teon Banek
8ed1fbbbe1 Generate TypeInfo object for all LCP defined classes
Summary:
TypeInfo will be needed for upcoming serialization via SLK. This diff
changes the already defined TypeInfo by removing the reliance on
capnp::typeId calls. The struct itself is now in utils.

Hopefully, this shouldn't break our RPC stack due to new ID generation.

Reviewers: mferencevic, mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1735
2018-11-16 13:27:58 +01:00
Teon Banek
a272fa2e6b Add basic support for saving classes with SLK
Summary:
This is the initial work on transferring our serialization code from
Cap'n Proto to SaveLoadKit. The commit contains tests for generated
code, but still requires work on supporting some features. Most notably,
generating and storing type IDs for derived classes so that they can be
loaded from a base pointer. Naturally, loading implementation hasn't
even been started yet.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1733
2018-11-15 13:26:38 +01:00
Teon Banek
08c14bfafc Split C++ meta information types to a new file
Reviewers: mtomic, llugovic

Reviewed By: llugovic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1716
2018-11-05 09:37:09 +01:00
Teon Banek
218384786d Replace TreeVisitor with ExpressionVisitor
Summary:
This is the initial step in making the visitors more granular in what
they visit.

Also add ignoring multiple inheritance in LCP and update LCP docs.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1704
2018-10-30 09:24:45 +01:00
Teon Banek
66dcd536f0 Support serializing LCP enum vector
Summary:
Generate conversion functions from C++ enum to Cap'n Proto and vice
verse. This should reduce the generated code size and hopefully improve
readability.

Use conversion functions for enums in default serialization generation.
`capnp-save-enum` and `capnp-load-enum` should now be used exclusively
to serialize non LCP defined enumerations. They now always require
`enum-values` argument.

Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1696
2018-10-26 16:05:47 +02:00
Teon Banek
30e18026a5 Regenerate LCP files if the LCP itself changed
Summary:
Unfortunately, CMake macros cannot access variables defined in the file
where macro is defined. This means that we have to repeat the values by
hand in order to track the correct dependency of LCP.

Reviewers: mtomic, llugovic, mferencevic

Reviewed By: mferencevic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1701
2018-10-26 10:06:04 +02:00
Teon Banek
6ee7d02657 Don't generate unused builder/reader
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1689
2018-10-22 17:02:54 +02:00
Teon Banek
84305f7423 Support looking for nested enums in LCP
Reviewers: mtomic, llugovic

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1686
2018-10-22 13:21:50 +02:00
Lovro Lugovic
a6a621e08b Add LCP tests
Summary: Create an ASDF system for LCP. Add LCP tests.

Reviewers: teon.banek, mtomic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1645
2018-10-15 11:52:20 +02:00
Marin Tomic
ce6085fa06 Fix pointer tracking in AST serialization
Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1640
2018-10-10 11:24:39 +02:00
Marin Tomic
db52b35ab6 Convert AST to LCP
Reviewers: teon.banek, llugovic

Reviewed By: teon.banek

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1628
2018-10-04 12:38:29 +02:00
Teon Banek
1fd9a72e10 Generate Load functions from LCP as top level
Summary: Depends on D1596

Reviewers: mtomic, msantl

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1601
2018-09-28 10:34:20 +02:00
Teon Banek
a5926b4e0f Generate Save functions from LCP as top level
Summary:
This should allow us to more easily decouple the code which should be
open sourced. Unfortunately, the downside of this approach is that we
cannot rely on virtual calls to dispatch the serialization to correct
type. Another downside is that members need to be publicly accessible
for serialization.

Reviewers: mtomic, msantl

Reviewed By: mtomic

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1596
2018-09-28 10:26:56 +02:00
Teon Banek
ee889b98fc Add default LCP save of primitive types in optional
Reviewers: msantl, mtomic

Reviewed By: msantl

Subscribers: pullbot

Differential Revision: https://phabricator.memgraph.io/D1579
2018-08-31 14:25:39 +02:00