diff --git a/docs/dev/lcp.md b/docs/dev/lcp.md index 35829dfc2..55a4e51c0 100644 --- a/docs/dev/lcp.md +++ b/docs/dev/lcp.md @@ -22,6 +22,7 @@ Contents - [C++ Classes & Structs](#cpp-classes) - [Defining an RPC](#defining-an-rpc) - [Cap'n Proto Serialization](#capnp-serial) + - [SaveLoadKit Serialization](#slk-serial) ## Running LCP @@ -773,3 +774,243 @@ Example: "capnp::SomeEnum" "SomeEnum" '(first-value second-value))))) ``` + +### SaveLoadKit Serialization {#slk-serial} + +LCP supports generating serialization code for use with our own simple +serialization framework, SaveLoadKit (SLK). + +To specify a class or structure for serialization, pass a `:serialize :slk` +class option. For example: + +```lisp +(lcp:define-struct my-struct () + ((member :int64_t)) + (:serialize :slk)) +``` + +The above will generate C++ functions for saving and loading all members of +the defined type. The generated code is put inside the `slk` namespace. For +the above example, we would get the following declarations: + +```cpp +namespace slk { +void Save(const MyStruct &self, slk::Builder *builder); +void Load(MyStruct *self, slk::Reader *reader); +} +``` + +Since we use top level (i.e. non-member) functions, the class members need to +have public access. The primary reason why we use non-member functions is the +ability to have them decoupled from types. This in turn allows us to easily +compile the code with and without serialization. The obvious downside is the +requirement of public access which could potentially allow for erroneous use +of classes and its members. Therefore, the recommended way to use +serialization is with plain old data types. The programmer needs be aware of +that and use POD as an immutable type as much as possible. This +recommendation of using POD types will also help minimize the complexity of +serialization code as well as minimize required features in LCP. + +Another requirement on serialized types is that they need to be default +constructible. This keeps the serialization implementation simple and uniform. +Each type is first default constructed, potentially on stack memory. Then the +`slk::Load` function is invoked with the pointer to that instance. We could +add support for having a pointer to an uninitialized memory and perform the +construct in `slk::Load` to allow types which aren't default constructible. +At the moment, implementing this support would needlessly complicate our code +where most of the types can be and are default constructible. + +#### Single Inheritance + +The first and most common step out of the POD zone is having classes with +inheritance. LCP supports serializing classes with single inheritance. + +A minor complication appears when loading a pointer to a base class. When we +have a pointer to a base class, serializing it may save the data of some +concrete, derived type. Loading the pointer back will need to determine which +type was actually serialized. When we know the concrete type, we need to +construct it and load it. Finally, we can return a base pointer to that. For +this reason, we generate 2 loading functions: regular `Load` and +`ConstructAndLoad`. The latter function is used to do the whole process of +determining the type, constructing it and invoking regular `Load`. Since we +cannot know the type of the serialized pointer upfront, we cannot allocate the +exact required memory on the stack. For that reason, `ConstructAndLoad` will +perform a heap allocation for you. Obviously, this could be a performance +issue. In cases when we know the exact concrete type, then we can use the +regular `Load` which expects the pointer to that type. If you are using `Load` +instead of `ConstructAndLoad`, read the next paragraph carefully! + +Determining which type was serialized works by storing the `id` of +`utils::TypeInfo` when saving a class which is anywhere in the inheritance +hierarchy. This is the *first* thing the invocation to `Save` does. Later, +when we call `ConstructAndLoad` it will read that type `id` and dispatch on it +to construct the instance of that type and call the appropriate `Load` +function. Beware when invoking `Load` of polymorphic types yourself! You +*need* to read the type `id` yourself *first* and then invoke the `Load` +function. Things will not work correctly if you forget to do that, because +`Load` expects to read the serialized data members and not the type +information. + +For example: + +```lisp +(lcp:define-class base () + ... + (:serialize :slk)) + +(lcp:define-class derived (base) + ... + (:serialize :slk)) +``` + +We get the following declarations generated: + +```cpp +namespace slk { +// Save will correctly forward to derived class using `dynamic_cast`! +void Save(const Base &self, slk::Builder *builder); +// Load only the Base instance, does *not* forward! +void Load(Base *self, slk::Reader *reader); +// Construct the concrete type (could be Base or any derived) and call the +// correct Load. Raises `slk::SlkDecodeException` if an unknown type is +// serialized. +void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader); + +void Save(const Derived &self, slk::Builder *builder); +void Load(Derived *self, slk::Reader *reader); +// This will raise slk::SlkDecodeException, if something other than `Derived` +// was serialized. `Derived` does not have any subclassses. +void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader); +``` + +#### Multiple Inheritance + +Serializing classes with multiple inheritance is *not* supported! + +Usually, multiple inheritance is used to satisfy some interface which doesn't +carry data for serialization. In such cases, you can ignore the multiple +inheritance by specifying `:ignore-other-base-classes` option. For example: + +```lisp +(lcp:define-class derived (primary-base some-interface ...) + ... + (:serialize :slk (:ignore-other-base-classes t))) +``` + +The above will produce serialization code as if `derived` is inheriting *only* +from `primary-base`. + +#### Templated Types + +Serializing templated types is *not* supported! + +You may still write your own serialization code in C++, but LCP will not +generate it for you. + +#### Custom Save and Load Hooks + +In cases when default serialization is not adequate, you may wish to provide +your own serialization code. LCP provides `:slk-save` and `:slk-load` options +for each member. + +These hooks for custom serialization expect a function with a single argument, +`member`, representing the member currently being serialized. This allows to +have a more generic function which works with any member of some type. The +return value of the function needs to be C++ code. The generated code may +expect to have `self` and `builder` variables in scope, just like they are +found in the generated `Save` and `Load` declarations. + +For example, one of the most common use cases is saving and loading +a `std::shared_ptr`. You need to provide an argument which is used to track +which pointers were already (de)serialized. Let's take a look how this could +be done in LCP. + +```lisp +(lcp:define-struct my-struct () + ((some-ptr "std::shared_ptr" + :slk-save (lambda (member) + #>cpp + std::vector already_saved; + slk::Save(self.${member}, builder, &already_saved); + cpp<#) + :slk-load (lambda (member) + #>cpp + std::vector> already_loaded; + slk::Load(&self->${member}, reader, &already_loaded); + cpp<#))) + (:serialize :slk)) +``` + +The above use is very artificial, because we usually have multiple shared +pointers across different members. In such cases we would like to share the +tracking data. One way to do that is explained in the next section. + +#### Additional Arguments to Generated Save and Load + +As you may have noticed, primary arguments for `Save` and `Load` are the type +instance and a `slk::Builder` or a `slk::Reader`. In some cases we would like +to accept additional arguments to help us with the serialization process. +Let's see how this is done in LCP using the `:save-args` and `:load-args` +options for `:slk` serialization. + +Both `:save-args` and `:load-args` options expect a list of pairs. Each pair +designates one argument. The first element of the pair is the argument name +and the second is the C++ type of that argument. + +As mentioned in the previous section, one of the most common cases where +default serialization doesn't cut it is when we have a `std::shared_ptr`. +Here, we would like to track already serialized pointers. Instead of having +some kind of a global variable, we could pass the tracking data as an +additional argument. Let's take the example from the previous section, and +have it take tracking data as an argument to `Save` and `Load` of `my-struct` +type. + +```lisp +(lcp:define-struct my-struct () + ((some-ptr "std::shared_ptr" + :slk-save (lambda (member) + #>cpp + slk::Save(self.${member}, builder, already_saved); + cpp<#) + :slk-load (lambda (member) + #>cpp + slk::Load(&self->${member}, reader, already_loaded); + cpp<#))) + (:serialize :slk (:save-args '((already-saved "std::vector *")) + :load-args '((already-loaded "std::vector> *"))))) +``` + +The generated declarations now look like the following: + +```cpp +void Save(const MyStruct &self, slk::Builder *builder, + std::vector *already_saved); +void Load(MyStruct *self, slk::Builder *builder, + std::vector> *already_loaded); +``` + +This can now be handy when serializing multiple instances of `my-struct`. For +example: + +```lisp +(lcp:define-struct my-array-of-struct () + ((structs "std::vector" + :slk-save (lambda (member) + #>cpp + slk::Save(self.${member}.size(), builder); + std::vector already_saved; + for (const auto &my_struct : structs) + slk::Save(my_struct, builder, &already_saved); + cpp<#) + :slk-load (lambda (member) + #>cpp + size_t size = 0; + slk::Load(&size, reader); + self->${member}.resize(size); + std::vector> already_loaded; + for (size_t i = 0; i < size; ++i) + slk::Load(&self->${member}[i], reader, &already_loaded); + cpp<#))) + (:serialize :slk)) +``` + diff --git a/src/lisp/code-gen.lisp b/src/lisp/code-gen.lisp index 07931c909..d142b6aca 100644 --- a/src/lisp/code-gen.lisp +++ b/src/lisp/code-gen.lisp @@ -18,6 +18,46 @@ see `CALL-WITH-CPP-BLOCK-OUTPUT' documentation." (declare (ignorable semicolonp name)) `(call-with-cpp-block-output ,out (lambda () ,@body) ,@rest)) +(defun call-with-namespaced-output (out fun) + "Invoke FUN with a function for opening C++ namespaces. The function takes +care to write namespaces to OUT without redundantly opening already open +namespaces." + (declare (type stream out)) + (declare (type (function (function)) fun)) + (let (open-namespaces) + (funcall fun (lambda (namespaces) + ;; No namespaces is global namespace + (unless namespaces + (dolist (to-close open-namespaces) + (declare (ignore to-close)) + (format out "~%}"))) + ;; Check if we need to open or close namespaces + (loop for namespace in namespaces + with unmatched = open-namespaces do + (if (string= namespace (car unmatched)) + (setf unmatched (cdr unmatched)) + (progn + (dolist (to-close unmatched) + (declare (ignore to-close)) + (format out "~%}")) + (format out "namespace ~A {~2%" namespace)))) + (setf open-namespaces namespaces))) + ;; Close remaining namespaces + (dolist (to-close open-namespaces) + (declare (ignore to-close)) + (format out "~%}")))) + +(defmacro with-namespaced-output ((out open-namespace-fun) &body body) + "Use `CALL-WITH-NAMESPACED-OUTPUT' more conveniently by executing BODY in a +context which binds OPEN-NAMESPACE-FUN function for opening namespaces." + (let ((open-namespace (gensym))) + `(call-with-namespaced-output + ,out + (lambda (,open-namespace) + (flet ((,open-namespace-fun (namespaces) + (funcall ,open-namespace namespaces))) + ,@body))))) + (defun cpp-documentation (documentation) "Convert DOCUMENTATION to Doxygen style string." (declare (type string documentation)) diff --git a/src/lisp/lcp-test.lisp b/src/lisp/lcp-test.lisp index 8f591ce88..8dfc42b0f 100644 --- a/src/lisp/lcp-test.lisp +++ b/src/lisp/lcp-test.lisp @@ -161,15 +161,42 @@ (deftest "slk" (subtest "function declarations" (undefine-cpp-types) - (is-generated (lcp.slk:save-function-declaration-for-class - (lcp:define-struct test-struct () - ())) - "void Save(const TestStruct &self, slk::Builder *builder)") + (let ((test-struct (lcp:define-struct test-struct () + ()))) + (is-generated (lcp.slk:save-function-declaration-for-class test-struct) + "void Save(const TestStruct &self, slk::Builder *builder)") + (is-generated (lcp.slk:load-function-declaration-for-class test-struct) + "void Load(TestStruct *self, slk::Reader *reader)")) (undefine-cpp-types) - (is-generated (lcp.slk:save-function-declaration-for-class - (lcp:define-class derived (base) - ())) - "void Save(const Derived &self, slk::Builder *builder)") + (let ((derived (lcp:define-class derived (base) + ()))) + (is-generated (lcp.slk:save-function-declaration-for-class derived) + "void Save(const Derived &self, slk::Builder *builder)") + (is-generated (lcp.slk:load-function-declaration-for-class derived) + "void Load(Derived *self, slk::Reader *reader)")) + (undefine-cpp-types) + (let ((test-struct (lcp:define-struct test-struct () + () + (:serialize :slk (:save-args '((extra-arg "SaveArgType")) + :load-args '((extra-arg "LoadArgType"))))))) + (is-generated (lcp.slk:save-function-declaration-for-class test-struct) + "void Save(const TestStruct &self, slk::Builder *builder, SaveArgType extra_arg)") + (is-generated (lcp.slk:load-function-declaration-for-class test-struct) + "void Load(TestStruct *self, slk::Reader *reader, LoadArgType extra_arg)")) + (undefine-cpp-types) + (let ((base-class (lcp:define-struct base () + () + (:serialize :slk (:save-args '((extra-arg "SaveArgType")) + :load-args '((extra-arg "LoadArgType")))))) + (derived-class (lcp:define-struct derived (base) + ()))) + (declare (ignore base-class)) + (is-generated (lcp.slk:save-function-declaration-for-class derived-class) + "void Save(const Derived &self, slk::Builder *builder, SaveArgType extra_arg)") + (is-generated (lcp.slk:load-function-declaration-for-class derived-class) + "void Load(Derived *self, slk::Reader *reader, LoadArgType extra_arg)") + (is-generated (lcp.slk:construct-and-load-function-declaration-for-class derived-class) + "void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader, LoadArgType extra_arg)")) (undefine-cpp-types) (let ((my-enum (lcp:define-enum my-enum (first-value second-value)))) @@ -178,11 +205,20 @@ (is-generated (lcp.slk:load-function-declaration-for-enum my-enum) "void Load(MyEnum *self, slk::Reader *reader)")) (undefine-cpp-types) + ;; Unsupported multiple inheritance (is-error (lcp.slk:save-function-declaration-for-class (lcp:define-class derived (fst-base snd-base) ())) 'lcp.slk:slk-error) (undefine-cpp-types) + ;; Ignoring multiple inheritance + (is-generated (lcp.slk:save-function-declaration-for-class + (lcp:define-class derived (fst-base snd-base) + () + (:serialize :slk (:ignore-other-base-classes t)))) + "void Save(const Derived &self, slk::Builder *builder)") + (undefine-cpp-types) + ;; Unsupported template classes (is-error (lcp.slk:save-function-declaration-for-class (lcp:define-class (derived t-param) (base) ())) @@ -194,121 +230,326 @@ 'lcp.slk:slk-error) (undefine-cpp-types)) - (subtest "save definitions" - (undefine-cpp-types) - (is-generated (lcp.slk:save-function-definition-for-class - (lcp:define-struct test-struct () - ((int-member :int64_t) - (vec-member "std::vector")))) - "void Save(const TestStruct &self, slk::Builder *builder) { - slk::Save(self.int_member, builder); - slk::Save(self.vec_member, builder); - }") - (undefine-cpp-types) - (is-generated (lcp.slk:save-function-definition-for-class - (lcp:define-struct test-struct () - ((skip-member :int64_t :dont-save t)))) - "void Save(const TestStruct &self, slk::Builder *builder) {}") - (undefine-cpp-types) - (is-generated (lcp.slk:save-function-definition-for-class - (lcp:define-struct test-struct () - ((custom-member "SomeType" - :slk-save (lambda (member-name) - (check-type member-name string) - (format nil "self.~A.CustomSave(builder);" member-name)))))) - "void Save(const TestStruct &self, slk::Builder *builder) { - self.custom_member.CustomSave(builder); - }") - (undefine-cpp-types) - (is-generated (lcp.slk:save-function-definition-for-enum - (lcp:define-enum test-enum - (first-value second-value))) - "void Save(const TestEnum &self, slk::Builder *builder) { - uint8_t enum_value; - switch (self) { - case TestEnum::FIRST_VALUE: enum_value = 0; break; - case TestEnum::SECOND_VALUE: enum_value = 1; break; - } - slk::Save(enum_value, builder); - }") + (subtest "enum serialization" (undefine-cpp-types) + (let ((my-enum (lcp:define-enum my-enum + (first-value second-value)))) + (is-generated (lcp.slk:save-function-definition-for-enum my-enum) + "void Save(const MyEnum &self, slk::Builder *builder) { + uint8_t enum_value; + switch (self) { + case MyEnum::FIRST_VALUE: enum_value = 0; break; + case MyEnum::SECOND_VALUE: enum_value = 1; break; + } + slk::Save(enum_value, builder); + }") + (is-generated (lcp.slk:load-function-definition-for-enum my-enum) + "void Load(MyEnum *self, slk::Reader *reader) { + uint8_t enum_value; + slk::Load(&enum_value, reader); + switch (enum_value) { + case static_cast(0): *self = MyEnum::FIRST_VALUE; break; + case static_cast(1): *self = MyEnum::SECOND_VALUE; break; + default: throw slk::SlkDecodeException(\"Trying to load unknown enum value!\"); + } + }"))) - (subtest "inheritance" - (undefine-cpp-types) - (is-error (lcp.slk:save-function-declaration-for-class - (lcp:define-struct derived (fst-base snd-base) - ())) - 'lcp.slk:slk-error) - (undefine-cpp-types) + (subtest "plain class serialization" + (undefine-cpp-types) + (let ((test-struct (lcp:define-struct test-struct () + ((int-member :int64_t) + (vec-member "std::vector"))))) + (is-generated (lcp.slk:save-function-definition-for-class test-struct) + "void Save(const TestStruct &self, slk::Builder *builder) { + slk::Save(self.int_member, builder); + slk::Save(self.vec_member, builder); + }") + (is-generated (lcp.slk:load-function-definition-for-class test-struct) + "void Load (TestStruct *self, slk::Reader *reader) { + slk::Load(&self->int_member, reader); + slk::Load(&self->vec_member, reader); + }")) + (undefine-cpp-types) + (let ((test-struct (lcp:define-struct test-struct () + ((skip-member :int64_t :dont-save t))))) + (is-generated (lcp.slk:save-function-definition-for-class test-struct) + "void Save(const TestStruct &self, slk::Builder *builder) {}") + (is-generated (lcp.slk:load-function-definition-for-class test-struct) + "void Load(TestStruct *self, slk::Reader *reader) {}")) + (undefine-cpp-types) + (let ((test-struct + (lcp:define-struct test-struct () + ((custom-member "SomeType" + :slk-save (lambda (member-name) + (check-type member-name string) + (format nil "self.~A.CustomSave(builder);" member-name)) + :slk-load (lambda (member-name) + (check-type member-name string) + (format nil "self->~A.CustomLoad(reader);" member-name))))))) + (is-generated (lcp.slk:save-function-definition-for-class test-struct) + "void Save(const TestStruct &self, slk::Builder *builder) { + { self.custom_member.CustomSave(builder); } + }") + (is-generated (lcp.slk:load-function-definition-for-class test-struct) + "void Load(TestStruct *self, slk::Reader *reader) { + { self->custom_member.CustomLoad(reader); } + }")) + (undefine-cpp-types) + (let ((raw-ptr-class (lcp:define-struct has-raw-ptr () + ((raw-ptr "SomeType *")))) + (shared-ptr-class (lcp:define-struct has-shared-ptr () + ((shared-ptr "std::shared_ptr")))) + (unique-ptr-class (lcp:define-struct has-unique-ptr() + ((unique-ptr "std::unique_ptr"))))) + (dolist (ptr-class (list raw-ptr-class shared-ptr-class unique-ptr-class)) + (is-error (lcp.slk:save-function-definition-for-class ptr-class) + 'lcp.slk:slk-error) + (is-error (lcp.slk:load-function-definition-for-class ptr-class) + 'lcp.slk:slk-error))) + (undefine-cpp-types) + (flet ((custom-save (m) + (declare (ignore m)) + "CustomSave();") + (custom-load (m) + (declare (ignore m)) + "CustomLoad();")) + (let ((raw-ptr-class (lcp:define-struct has-raw-ptr () + ((member "SomeType *" + :slk-save #'custom-save + :slk-load #'custom-load)))) + (shared-ptr-class (lcp:define-struct has-shared-ptr () + ((member "std::shared_ptr" + :slk-save #'custom-save + :slk-load #'custom-load)))) + (unique-ptr-class (lcp:define-struct has-unique-ptr() + ((member "std::unique_ptr" + :slk-save #'custom-save + :slk-load #'custom-load))))) + (dolist (ptr-class (list raw-ptr-class shared-ptr-class unique-ptr-class)) + (is-generated (lcp.slk:save-function-definition-for-class ptr-class) + (format nil "void Save(const ~A &self, slk::Builder *builder) { { CustomSave(); } }" + (lcp::cpp-type-decl ptr-class))) + (is-generated (lcp.slk:load-function-definition-for-class ptr-class) + (format nil "void Load(~A *self, slk::Reader *reader) { { CustomLoad(); } }" + (lcp::cpp-type-decl ptr-class))))))) + + (subtest "class inheritance serialization" + (undefine-cpp-types) + ;; Unsupported multiple inheritance + (is-error (lcp.slk:save-function-declaration-for-class + (lcp:define-struct derived (fst-base snd-base) + ())) + 'lcp.slk:slk-error) + + (undefine-cpp-types) + ;; We will test single inheritance and ignored multiple inheritance, both + ;; should generate the same code that follows. + (let ((base-save-code + "void Save(const Base &self, slk::Builder *builder) { + if (const auto *derived_derived = dynamic_cast(&self)) { + return slk::Save(*derived_derived, builder); + } + slk::Save(Base::kType.id, builder); + slk::Save(self.base_member, builder); + }") + (base-construct-code + "void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader) { + uint64_t type_id; + slk::Load(&type_id, reader); + if (Base::kType.id == type_id) { + auto base_instance = std::make_unique(); + slk::Load(base_instance.get(), reader); + *self = std::move(base_instance); + return; + } + if (Derived::kType.id == type_id) { + auto derived_instance = std::make_unique(); + slk::Load(derived_instance.get(), reader); + *self = std::move(derived_instance); + return; + } + throw slk::SlkDecodeException(\"Trying to load unknown derived type!\"); + }") + (base-load-code + "void Load(Base *self, slk::Reader *reader) { + // CHECK(self->GetTypeInfo() == Base::kType); + slk::Load(&self->base_member, reader); + }") + (derived-save-code + "void Save(const Derived &self, slk::Builder *builder) { + slk::Save(Derived::kType.id, builder); + // Save parent Base + { slk::Save(self.base_member, builder); } + slk::Save(self.derived_member, builder); + }") + (derived-construct-code + "void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader) { + uint64_t type_id; + slk::Load(&type_id, reader); + if (Derived::kType.id == type_id) { + auto derived_instance = std::make_unique(); + slk::Load(derived_instance.get(), reader); + *self = std::move(derived_instance); + return; + } + throw slk::SlkDecodeException(\"Trying to load unknown derived type!\"); + }") + (derived-load-code + "void Load(Derived *self, slk::Reader *reader) { + // Load parent Base + { slk::Load(&self->base_member, reader); } + slk::Load(&self->derived_member, reader); + }")) + ;; Single inheritance (let ((base-class (lcp:define-struct base () ((base-member :bool)))) (derived-class (lcp:define-struct derived (base) ((derived-member :int64_t))))) (is-generated (lcp.slk:save-function-definition-for-class base-class) - "void Save(const Base &self, slk::Builder *builder) { - if (const auto &derived_derived = dynamic_cast(self)) { - return slk::Save(derived_derived, builder); - } - slk::Save(self.base_member, builder); - }") + base-save-code) + (is-generated (lcp.slk:construct-and-load-function-definition-for-class base-class) + base-construct-code) + (is-generated (lcp.slk:load-function-definition-for-class base-class) + base-load-code) (is-generated (lcp.slk:save-function-definition-for-class derived-class) - "void Save(const Derived &self, slk::Builder *builder) { - // Save parent Base - { slk::Save(self.base_member, builder); } - slk::Save(self.derived_member, builder); - }")) - (undefine-cpp-types) - (let ((abstract-base-class (lcp:define-struct abstract-base () - ((base-member :bool)) - (:abstractp t))) - (derived-class (lcp:define-struct derived (abstract-base) - ((derived-member :int64_t))))) - (is-generated (lcp.slk:save-function-definition-for-class abstract-base-class) - "void Save(const AbstractBase &self, slk::Builder *builder) { - if (const auto &derived_derived = dynamic_cast(self)) { - return slk::Save(derived_derived, builder); - } - LOG(FATAL) << \"`AbstractBase` is marked as an abstract class!\"; - }") - (is-generated (lcp.slk:save-function-definition-for-class derived-class) - "void Save(const Derived &self, slk::Builder *builder) { - // Save parent AbstractBase - { slk::Save(self.base_member, builder); } - slk::Save(self.derived_member, builder); - }")) - (undefine-cpp-types) - (let ((base-templated-class (lcp:define-struct (base t-param) () - ((base-member :bool)))) - (derived-class (lcp:define-struct derived (base) - ((derived-member :int64_t))))) - (is-error (lcp.slk:save-function-definition-for-class base-templated-class) - 'lcp.slk:slk-error) - (is-error (lcp.slk:save-function-definition-for-class derived-class) - 'lcp.slk:slk-error)) + derived-save-code) + (is-generated (lcp.slk:construct-and-load-function-definition-for-class derived-class) + derived-construct-code) + (is-generated (lcp.slk:load-function-definition-for-class derived-class) + derived-load-code)) (undefine-cpp-types) + ;; Ignored multiple inheritance should be the same as single (let ((base-class (lcp:define-struct base () ((base-member :bool)))) - (derived-templated-class (lcp:define-struct (derived t-param) (base) - ((derived-member :int64_t))))) - (is-error (lcp.slk:save-function-definition-for-class base-class) - 'lcp.slk:slk-error) - (is-error (lcp.slk:save-function-definition-for-class derived-templated-class) - 'lcp.slk:slk-error)))) + (derived-class (lcp:define-struct derived (base ignored-base) + ((derived-member :int64_t)) + (:serialize :slk (:ignore-other-base-classes t))))) + (is-generated (lcp.slk:save-function-definition-for-class base-class) + base-save-code) + (is-generated (lcp.slk:construct-and-load-function-definition-for-class base-class) + base-construct-code) + (is-generated (lcp.slk:load-function-definition-for-class base-class) + base-load-code) + (is-generated (lcp.slk:save-function-definition-for-class derived-class) + derived-save-code) + (is-generated (lcp.slk:construct-and-load-function-definition-for-class derived-class) + derived-construct-code) + (is-generated (lcp.slk:load-function-definition-for-class derived-class) + derived-load-code))) - (subtest "load definitions" (undefine-cpp-types) - (is-generated (lcp.slk:load-function-definition-for-enum - (lcp:define-enum my-enum - (first-value second-value))) - "void Load(MyEnum *self, slk::Reader *reader) { - uint8_t enum_value; - slk::Load(&enum_value, reader); - switch (enum_value) { - case static_cast(0): *self = MyEnum::FIRST_VALUE; break; - case static_cast(1): *self = MyEnum::SECOND_VALUE; break; - default: LOG(FATAL) << \"Trying to load unknown enum value!\"; - } - }")) + (let ((abstract-base-class (lcp:define-struct abstract-base () + ((base-member :bool)) + (:abstractp t))) + (derived-class (lcp:define-struct derived (abstract-base) + ((derived-member :int64_t))))) + (is-generated (lcp.slk:save-function-definition-for-class abstract-base-class) + "void Save(const AbstractBase &self, slk::Builder *builder) { + if (const auto *derived_derived = dynamic_cast(&self)) { + return slk::Save(*derived_derived, builder); + } + LOG(FATAL) << \"`AbstractBase` is marked as an abstract class!\"; + }") + (is-generated (lcp.slk:construct-and-load-function-definition-for-class abstract-base-class) + "void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader) { + uint64_t type_id; + slk::Load(&type_id, reader); + if (Derived::kType.id == type_id) { + auto derived_instance = std::make_unique(); + slk::Load(derived_instance.get(), reader); + *self = std::move(derived_instance); + return; + } + throw slk::SlkDecodeException(\"Trying to load unknown derived type!\"); + }") + (is-generated (lcp.slk:save-function-definition-for-class derived-class) + "void Save(const Derived &self, slk::Builder *builder) { + slk::Save(Derived::kType.id, builder); + // Save parent AbstractBase + { slk::Save(self.base_member, builder); } + slk::Save(self.derived_member, builder); + }") + (is-generated (lcp.slk:construct-and-load-function-definition-for-class derived-class) + "void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader) { + uint64_t type_id; + slk::Load(&type_id, reader); + if (Derived::kType.id == type_id) { + auto derived_instance = std::make_unique(); + slk::Load(derived_instance.get(), reader); + *self = std::move(derived_instance); + return; + } + throw slk::SlkDecodeException(\"Trying to load unknown derived type!\"); + }") + (is-generated (lcp.slk:load-function-definition-for-class derived-class) + "void Load(Derived *self, slk::Reader *reader) { + // Load parent AbstractBase + { slk::Load(&self->base_member, reader); } + slk::Load(&self->derived_member, reader); + }")) + (undefine-cpp-types) + (let ((base-templated-class (lcp:define-struct (base t-param) () + ((base-member :bool)))) + (derived-class (lcp:define-struct derived (base) + ((derived-member :int64_t))))) + (is-error (lcp.slk:save-function-definition-for-class base-templated-class) + 'lcp.slk:slk-error) + (is-error (lcp.slk:save-function-definition-for-class derived-class) + 'lcp.slk:slk-error)) + (undefine-cpp-types) + (let ((base-class (lcp:define-struct base () + ((base-member :bool)))) + (derived-templated-class (lcp:define-struct (derived t-param) (base) + ((derived-member :int64_t))))) + (is-error (lcp.slk:save-function-definition-for-class base-class) + 'lcp.slk:slk-error) + (is-error (lcp.slk:save-function-definition-for-class derived-templated-class) + 'lcp.slk:slk-error)) + + (undefine-cpp-types) + (let ((class (lcp:define-struct derived ("UnknownBase") + ((member :bool))))) + (is-error (lcp.slk:save-function-definition-for-class class) + 'lcp.slk:slk-error) + (is-error (lcp.slk:load-function-definition-for-class class) + 'lcp.slk:slk-error)) + (undefine-cpp-types) + (let ((class (lcp:define-struct derived ("UnknownBase") + ((member :bool)) + (:serialize :slk (:base t))))) + (is-generated (lcp.slk:save-function-definition-for-class class) + "void Save(const Derived &self, slk::Builder *builder) { slk::Save(self.member, builder); }") + (is-generated (lcp.slk:load-function-definition-for-class class) + "void Load(Derived *self, slk::Reader *reader) { slk::Load(&self->member, reader); }")) + + (undefine-cpp-types) + (let ((base-class (lcp:define-struct base () + () + (:abstractp t) + (:serialize :slk (:save-args '((extra-arg "SaveArg")) + :load-args '((extra-arg "LoadArg")))))) + (derived-class (lcp:define-struct derived (base) + ()))) + (declare (ignore derived-class)) + (is-generated (lcp.slk:save-function-definition-for-class base-class) + "void Save(const Base &self, slk::Builder *builder, SaveArg extra_arg) { + if (const auto *derived_derived = dynamic_cast(&self)) { + return slk::Save(*derived_derived, builder, extra_arg); + } + LOG(FATAL) << \"`Base` is marked as an abstract class!\"; + }") + (is-generated (lcp.slk:construct-and-load-function-definition-for-class base-class) + "void ConstructAndLoad(std::unique_ptr *self, slk::Reader *reader, LoadArg extra_arg) { + uint64_t type_id; + slk::Load(&type_id, reader); + if (Derived::kType.id == type_id) { + auto derived_instance = std::make_unique(); + slk::Load(derived_instance.get(), reader, extra_arg); + *self = std::move(derived_instance); + return; + } + throw slk::SlkDecodeException(\"Trying to load unknown derived type!\"); + }"))) (subtest "non-public members" (undefine-cpp-types) diff --git a/src/lisp/lcp.lisp b/src/lisp/lcp.lisp index 48cd90f30..59c5ab32c 100644 --- a/src/lisp/lcp.lisp +++ b/src/lisp/lcp.lisp @@ -1370,46 +1370,6 @@ formatted and output." (count-newlines in-stream :stop-position (1+ stream-pos)))))))) -(defun call-with-namespaced-output (out fun) - "Invoke FUN with a function for opening C++ namespaces. The function takes -care to write namespaces to OUT without redundantly opening already open -namespaces." - (declare (type stream out)) - (declare (type (function (function)) fun)) - (let (open-namespaces) - (funcall fun (lambda (namespaces) - ;; No namespaces is global namespace - (unless namespaces - (dolist (to-close open-namespaces) - (declare (ignore to-close)) - (format out "~%}"))) - ;; Check if we need to open or close namespaces - (loop for namespace in namespaces - with unmatched = open-namespaces do - (if (string= namespace (car unmatched)) - (setf unmatched (cdr unmatched)) - (progn - (dolist (to-close unmatched) - (declare (ignore to-close)) - (format out "~%}")) - (format out "namespace ~A {~2%" namespace)))) - (setf open-namespaces namespaces))) - ;; Close remaining namespaces - (dolist (to-close open-namespaces) - (declare (ignore to-close)) - (format out "~%}")))) - -(defmacro with-namespaced-output ((out open-namespace-fun) &body body) - "Use `CALL-WITH-NAMESPACED-OUTPUT' more conveniently by executing BODY in a -context which binds OPEN-NAMESPACE-FUN function for opening namespaces." - (let ((open-namespace (gensym))) - `(call-with-namespaced-output - ,out - (lambda (,open-namespace) - (flet ((,open-namespace-fun (namespaces) - (funcall ,open-namespace namespaces))) - ,@body))))) - (defun generate-capnp (cpp-types &key capnp-file capnp-id cpp-out lcp-file) "Generate Cap'n Proto serialization code for given CPP-TYPES. The schema is written to CAPNP-FILE using the CAPNP-ID. The C++ serialization code is diff --git a/src/lisp/package.lisp b/src/lisp/package.lisp index 935fce0d8..63e7e5c75 100644 --- a/src/lisp/package.lisp +++ b/src/lisp/package.lisp @@ -24,6 +24,10 @@ (:export #:slk-error #:save-function-declaration-for-class #:save-function-definition-for-class + #:construct-and-load-function-declaration-for-class + #:construct-and-load-function-definition-for-class + #:load-function-declaration-for-class + #:load-function-definition-for-class #:save-function-declaration-for-enum #:save-function-definition-for-enum #:load-function-declaration-for-enum diff --git a/src/lisp/slk.lisp b/src/lisp/slk.lisp index 6969b8bac..2e6afb8e3 100644 --- a/src/lisp/slk.lisp +++ b/src/lisp/slk.lisp @@ -5,30 +5,118 @@ (in-package #:lcp.slk) (define-condition slk-error (error) - ((message :type string :initarg :message :reader slk-error-message)) + ((message :type string :initarg :message :reader slk-error-message) + (format-args :type list :initform nil :initarg :format-args :reader slk-error-format-args)) (:report (lambda (condition stream) - (write-string (slk-error-message condition) stream)))) + (apply #'format stream + (slk-error-message condition) + (slk-error-format-args condition))))) + +(defun slk-error (message &rest format-args) + (error 'slk-error :message message :format-args format-args)) + +;;; CPP-CLASS serialization generation + +(defun cpp-class-super-classes-for-slk (cpp-class) + (let ((supers (lcp::cpp-class-super-classes cpp-class)) + (opts (lcp::cpp-class-slk-opts cpp-class))) + (unless (and opts (lcp::slk-opts-base opts)) + (if (and supers opts (lcp::slk-opts-ignore-other-base-classes opts)) + (list (car supers)) + supers)))) + +(defun save-extra-args (cpp-class) + "Get additional arguments to Save function for CPP-CLASS. Note, returned +extra arguments are of the first class encountered when traversing the +hierarchy from CPP-CLASS to parents." + (let ((opts (lcp::cpp-class-slk-opts cpp-class))) + (if (and opts (lcp::slk-opts-save-args opts)) + (lcp::slk-opts-save-args opts) + (let ((parents (cpp-class-super-classes-for-slk cpp-class))) + (dolist (parent parents) + (let ((parent-class (lcp::find-cpp-class parent))) + (when parent-class + (return (save-extra-args parent-class))))))))) + +(defun load-extra-args (cpp-class) + "Get additional arguments to Load function for CPP-CLASS. Note, returned +extra arguments are of the first class encountered when traversing the +hierarchy from CPP-CLASS to parents." + (let ((opts (lcp::cpp-class-slk-opts cpp-class))) + (if (and opts (lcp::slk-opts-load-args opts)) + (lcp::slk-opts-load-args opts) + (let ((parents (cpp-class-super-classes-for-slk cpp-class))) + (dolist (parent parents) + (let ((parent-class (lcp::find-cpp-class parent))) + (when parent-class + (return (load-extra-args parent-class))))))))) (defun save-function-declaration-for-class (cpp-class) "Generate SLK save function declaration for CPP-CLASS. Note that the code generation expects the declarations and definitions to be in `slk` namespace." (check-type cpp-class lcp::cpp-class) (when (lcp::cpp-type-type-params cpp-class) - (error 'slk-error :message - (format nil "Don't know how to save templated class '~A'" - (lcp::cpp-type-base-name cpp-class)))) - (when (< 1 (list-length (lcp::cpp-class-super-classes cpp-class))) - (error 'slk-error :message - (format nil "Don't know how to save multiple parents of '~A'" - (lcp::cpp-type-base-name cpp-class)))) + (slk-error "Don't know how to save templated class '~A'" + (lcp::cpp-type-base-name cpp-class))) + (when (< 1 (list-length (cpp-class-super-classes-for-slk cpp-class))) + (slk-error "Don't know how to save multiple parents of '~A'" + (lcp::cpp-type-base-name cpp-class))) (let ((self-arg (list 'self (format nil "const ~A &" - (lcp::cpp-type-decl cpp-class :namespace nil)))) + (lcp::cpp-type-decl cpp-class)))) (builder-arg (list 'builder "slk::Builder *"))) (lcp::cpp-function-declaration - "Save" :args (list self-arg builder-arg) + "Save" :args (cons self-arg (cons builder-arg (save-extra-args cpp-class))) :type-params (lcp::cpp-type-type-params cpp-class)))) +(defun construct-and-load-function-declaration-for-class (cpp-class) + "Generate SLK construct and load function declaration for CPP-CLASS. This +function needs to be used to load pointers to polymorphic types. Note that +the code generation expects the declarations and definitions to be in `slk` +namespace." + (check-type cpp-class lcp::cpp-class) + (when (lcp::cpp-type-type-params cpp-class) + (slk-error "Don't know how to load templated class '~A'" + (lcp::cpp-type-base-name cpp-class))) + (when (< 1 (list-length (cpp-class-super-classes-for-slk cpp-class))) + (slk-error "Don't know how to load multiple parents of '~A'" + (lcp::cpp-type-base-name cpp-class))) + (let ((self-arg + (list 'self (format nil "std::unique_ptr<~A> *" (lcp::cpp-type-decl cpp-class)))) + (reader-arg (list 'reader "slk::Reader *"))) + (lcp::cpp-function-declaration + "ConstructAndLoad" :args (cons self-arg (cons reader-arg (load-extra-args cpp-class))) + :type-params (lcp::cpp-type-type-params cpp-class)))) + +(defun load-function-declaration-for-class (cpp-class) + "Generate SLK load function declaration for CPP-CLASS. Note that the code +generation expects the declarations and definitions to be in `slk` namespace." + (check-type cpp-class lcp::cpp-class) + (when (lcp::cpp-type-type-params cpp-class) + (slk-error "Don't know how to load templated class '~A'" + (lcp::cpp-type-base-name cpp-class))) + (when (< 1 (list-length (cpp-class-super-classes-for-slk cpp-class))) + (slk-error "Don't know how to load multiple parents of '~A'" + (lcp::cpp-type-base-name cpp-class))) + (let ((self-arg + (list 'self (format nil "~A *" (lcp::cpp-type-decl cpp-class)))) + (reader-arg (list 'reader "slk::Reader *"))) + (lcp::cpp-function-declaration + "Load" :args (cons self-arg (cons reader-arg (load-extra-args cpp-class))) + :type-params (lcp::cpp-type-type-params cpp-class)))) + +(defun cpp-type-pointer-p (cpp-type) + (check-type cpp-type (or lcp::cpp-type string lcp::cpp-primitive-type-keywords)) + (typecase cpp-type + (string (cpp-type-pointer-p (lcp::parse-cpp-type-declaration cpp-type))) + (lcp::cpp-type + (or + (string= "*" (lcp::cpp-type-name cpp-type)) + (string= "shared_ptr" (lcp::cpp-type-name cpp-type)) + ;; Note, we could forward to default slk::Load for unique_ptr and hope + ;; everything is alright w.r.t to inheritance. + (string= "unique_ptr" (lcp::cpp-type-name cpp-type)))))) + (defun save-members (cpp-class) "Generate code for saving members of CPP-CLASS. Raise `SLK-ERROR' if the serializable member has no public access." @@ -36,41 +124,101 @@ serializable member has no public access." (dolist (member (lcp::cpp-class-members-for-save cpp-class)) (let ((member-name (lcp::cpp-member-name member :struct (lcp::cpp-class-structp cpp-class)))) (when (not (eq :public (lcp::cpp-member-scope member))) - (error 'slk-error :message - (format nil "Cannot save non-public member '~A' of '~A'" - (lcp::cpp-member-symbol member) (lcp::cpp-type-base-name cpp-class)))) + (slk-error "Cannot save non-public member '~A' of '~A'" + (lcp::cpp-member-symbol member) (lcp::cpp-type-base-name cpp-class))) (cond ((lcp::cpp-member-slk-save member) ;; Custom save function - (write-line (lcp::cpp-code (funcall (lcp::cpp-member-slk-save member) - member-name)) - s)) + (lcp::with-cpp-block-output (s) + (write-line (lcp::cpp-code (funcall (lcp::cpp-member-slk-save member) + member-name)) + s))) + ;; TODO: Maybe support saving (but not loading) unique_ptr. + ((cpp-type-pointer-p (lcp::cpp-member-type member)) + (slk-error "Don't know how to save pointer '~A' in '~A'" + (lcp::cpp-member-type member) + (lcp::cpp-type-base-name cpp-class))) ;; TODO: Extra args for cpp-class members (t (format s "slk::Save(self.~A, builder);~%" member-name))))))) +(defun members-for-load (cpp-class) + (remove-if (lambda (m) + (and (lcp::cpp-member-dont-save m) + (not (lcp::cpp-member-slk-load m)))) + (lcp::cpp-class-members cpp-class))) + +(defun load-members (cpp-class) + "Generate code for loading members of CPP-CLASS. Raise `SLK-ERROR' if the +serializable member has no public access." + (with-output-to-string (s) + (dolist (member (members-for-load cpp-class)) + (let ((member-name (lcp::cpp-member-name member :struct (lcp::cpp-class-structp cpp-class)))) + (when (not (eq :public (lcp::cpp-member-scope member))) + (slk-error "Cannot save non-public member '~A' of '~A'" + (lcp::cpp-member-symbol member) (lcp::cpp-type-base-name cpp-class))) + (cond + ((lcp::cpp-member-slk-load member) + ;; Custom load function + (lcp::with-cpp-block-output (s) + (write-line (lcp::cpp-code (funcall (lcp::cpp-member-slk-load member) + member-name)) + s))) + ((cpp-type-pointer-p (lcp::cpp-member-type member)) + (slk-error "Don't know how to load pointer '~A' in '~A'" + (lcp::cpp-member-type member) + (lcp::cpp-type-base-name cpp-class))) + ;; TODO: Extra args for cpp-class members + (t + (format s "slk::Load(&self->~A, reader);~%" member-name))))))) + (defun save-parents-recursively (cpp-class) "Generate code for saving members of all parents, recursively. Raise `SLK-ERROR' if trying to save templated parent class or if using multiple inheritance." - (when (< 1 (list-length (lcp::cpp-class-super-classes cpp-class))) - (error 'slk-error :message - (format nil "Don't know how to save multiple parents of '~A'" - (lcp::cpp-type-base-name cpp-class)))) + (when (< 1 (list-length (cpp-class-super-classes-for-slk cpp-class))) + (slk-error "Don't know how to save multiple parents of '~A'" + (lcp::cpp-type-base-name cpp-class))) (with-output-to-string (s) - ;; TODO: Stop recursing to parents if CPP-CLASS is marked as base for - ;; serialization. - (dolist (parent (lcp::cpp-class-super-classes cpp-class)) + (dolist (parent (cpp-class-super-classes-for-slk cpp-class)) (let ((parent-class (lcp::find-cpp-class parent))) - (assert parent-class) - (when (lcp::cpp-type-type-params parent-class) - (error 'slk-error :message - (format nil "Don't know how to save templated parent class '~A'" - (lcp::cpp-type-base-name parent-class)))) - (format s "// Save parent ~A~%" (lcp::cpp-type-name parent)) - (lcp::with-cpp-block-output (s) - (write-string (save-parents-recursively parent-class) s) - (write-string (save-members parent-class) s)))))) + (cond + ((not parent-class) + (slk-error + "Class '~A' has an unknown parent '~A', serialization is incomplete. Did you forget to mark '~A' as base?" + (lcp::cpp-type-base-name cpp-class) parent (lcp::cpp-type-base-name cpp-class))) + ((lcp::cpp-type-type-params parent-class) + (slk-error "Don't know how to save templated parent class '~A'" + (lcp::cpp-type-base-name parent-class))) + (t + (format s "// Save parent ~A~%" (lcp::cpp-type-name parent)) + (lcp::with-cpp-block-output (s) + (write-string (save-parents-recursively parent-class) s) + (write-string (save-members parent-class) s)))))))) + +(defun load-parents-recursively (cpp-class) + "Generate code for loading members of all parents, recursively. Raise +`SLK-ERROR' if trying to load templated parent class or if using multiple +inheritance." + (when (< 1 (list-length (cpp-class-super-classes-for-slk cpp-class))) + (slk-error "Don't know how to load multiple parents of '~A'" + (lcp::cpp-type-base-name cpp-class))) + (with-output-to-string (s) + (dolist (parent (cpp-class-super-classes-for-slk cpp-class)) + (let ((parent-class (lcp::find-cpp-class parent))) + (cond + ((not parent-class) + (slk-error + "Class '~A' has an unknown parent '~A', serialization is incomplete. Did you forget to mark '~A' as base?" + (lcp::cpp-type-base-name cpp-class) parent (lcp::cpp-type-base-name cpp-class))) + ((lcp::cpp-type-type-params parent-class) + (slk-error "Don't know how to load templated parent class '~A'" + (lcp::cpp-type-base-name parent-class))) + (t + (format s "// Load parent ~A~%" (lcp::cpp-type-name parent)) + (lcp::with-cpp-block-output (s) + (write-string (load-parents-recursively parent-class) s) + (write-string (load-members parent-class) s)))))))) (defun forward-save-to-subclasses (cpp-class) "Generate code which forwards the serialization to derived classes of @@ -79,39 +227,97 @@ CPP-CLASS. Raise `SLK-ERROR' if a derived class has template parameters." (let ((subclasses (lcp::direct-subclasses-of cpp-class))) (dolist (subclass subclasses) (when (lcp::cpp-type-type-params subclass) - (error 'slk-error :message - (format nil "Don't know how to save derived templated class '~A'" - (lcp::cpp-type-base-name subclass)))) - (let ((derived-class (lcp::cpp-type-name subclass)) + (slk-error "Don't know how to save derived templated class '~A'" + (lcp::cpp-type-base-name subclass))) + (let ((derived-class (lcp::cpp-type-decl subclass)) (derived-var (lcp::cpp-variable-name (lcp::cpp-type-base-name subclass))) - ;; TODO: Extra save arguments - (extra-args nil)) - (format s "if (const auto &~A_derived = dynamic_cast(self)) { - return slk::Save(~A_derived, builder~{, ~A~}); }~%" + (extra-args (mapcar (lambda (name-and-type) + (lcp::cpp-variable-name (first name-and-type))) + (save-extra-args cpp-class)))) + (format s "if (const auto *~A_derived = dynamic_cast(&self)) { + return slk::Save(*~A_derived, builder~{, ~A~}); }~%" derived-var derived-class derived-var extra-args)))))) (defun save-function-code-for-class (cpp-class) "Generate code for serializing CPP-CLASS. Raise `SLK-ERROR' on unsupported C++ constructs, mostly related to templates." (when (lcp::cpp-type-type-params cpp-class) - (error 'slk-error :message - (format nil "Don't know how to save templated class '~A'" - (lcp::cpp-type-base-name cpp-class)))) + (slk-error "Don't know how to save templated class '~A'" + (lcp::cpp-type-base-name cpp-class))) (with-output-to-string (s) (cond ((lcp::direct-subclasses-of cpp-class) + ;; We have more derived classes, so forward the call to them. (write-string (forward-save-to-subclasses cpp-class) s) (if (lcp::cpp-class-abstractp cpp-class) (format s "LOG(FATAL) << \"`~A` is marked as an abstract class!\";" (lcp::cpp-type-name cpp-class)) (progn + ;; We aren't abstract, so save our data. + (format s "slk::Save(~A::kType.id, builder);~%" + (lcp::cpp-type-decl cpp-class)) (write-string (save-parents-recursively cpp-class) s) (write-string (save-members cpp-class) s)))) (t - ;; TODO: Write some sort of type ID for derived classes + (when (cpp-class-super-classes-for-slk cpp-class) + ;; Write type ID for the (final) derived classes. + (format s "slk::Save(~A::kType.id, builder);~%" + (lcp::cpp-type-decl cpp-class))) (write-string (save-parents-recursively cpp-class) s) (write-string (save-members cpp-class) s))))) +(defun construct-and-load-function-code-for-class (cpp-class) + "Generate code for serializing CPP-CLASS. Raise `SLK-ERROR' on unsupported +C++ constructs, mostly related to templates." + (assert (or (cpp-class-super-classes-for-slk cpp-class) + (lcp::direct-subclasses-of cpp-class))) + (when (lcp::cpp-type-type-params cpp-class) + (slk-error "Don't know how to load templated class '~A'" + (lcp::cpp-type-base-name cpp-class))) + (labels ((concrete-subclasses-rec (class) + (let ((concrete-classes nil)) + (dolist (subclass (lcp::direct-subclasses-of class) concrete-classes) + (unless (lcp::cpp-class-abstractp subclass) + (push subclass concrete-classes)) + (setf concrete-classes + (append concrete-classes (concrete-subclasses-rec subclass))))))) + (with-output-to-string (s) + (write-line "uint64_t type_id;" s) + (write-line "slk::Load(&type_id, reader);" s) + (let ((concrete-classes (concrete-subclasses-rec cpp-class))) + (unless (lcp::cpp-class-abstractp cpp-class) + (push cpp-class concrete-classes)) + (dolist (concrete-class concrete-classes) + (let ((type-decl (lcp::cpp-type-decl concrete-class)) + (var-name (lcp::cpp-variable-name (lcp::cpp-type-base-name concrete-class))) + (extra-args (mapcar (lambda (name-and-type) + (lcp::cpp-variable-name (first name-and-type))) + (load-extra-args cpp-class)))) + (lcp::with-cpp-block-output + (s :name (format nil "if (~A::kType.id == type_id)" type-decl)) + (format s "auto ~A_instance = std::make_unique<~A>();~%" var-name type-decl) + (format s "slk::Load(~A_instance.get(), reader~{, ~A~});~%" var-name extra-args) + (format s "*self = std::move(~A_instance); return;~%" var-name)))) + (write-line "throw slk::SlkDecodeException(\"Trying to load unknown derived type!\");" s))))) + +(defun load-function-code-for-class (cpp-class) + "Generate code for serializing CPP-CLASS. Raise `SLK-ERROR' on unsupported +C++ constructs, mostly related to templates." + (when (lcp::cpp-type-type-params cpp-class) + (slk-error "Don't know how to load templated class '~A'" + (lcp::cpp-type-base-name cpp-class))) + (assert (not (lcp::cpp-class-abstractp cpp-class))) + (with-output-to-string (s) + ;; We are assuming that the generated code is called only in cases when we + ;; really have this particular class instantiated and not any of the + ;; derived ones. + ;; TODO: Add the following check when we have derived classes and + ;; virtual GetTypeInfo method. + (when (lcp::direct-subclasses-of cpp-class) + (format s "// CHECK(self->GetTypeInfo() == ~A::kType);~%" (lcp::cpp-type-decl cpp-class))) + (write-string (load-parents-recursively cpp-class) s) + (write-string (load-members cpp-class) s))) + (defun save-function-definition-for-class (cpp-class) "Generate SLK save function. Raise `SLK-ERROR' if an unsupported or invalid class definition is encountered during code generation. Note that the code @@ -122,6 +328,30 @@ generation expects the declarations and definitions to be in `slk` namespace." (cpp-out :name (save-function-declaration-for-class cpp-class)) (write-line (save-function-code-for-class cpp-class) cpp-out)))) +(defun load-function-definition-for-class (cpp-class) + "Generate SLK load function. Raise `SLK-ERROR' if an unsupported or invalid +class definition is encountered during code generation. Note that the code +generation expects the declarations and definitions to be in `slk` namespace." + (check-type cpp-class lcp::cpp-class) + (with-output-to-string (cpp-out) + (lcp::with-cpp-block-output + (cpp-out :name (load-function-declaration-for-class cpp-class)) + (write-line (load-function-code-for-class cpp-class) cpp-out)))) + +(defun construct-and-load-function-definition-for-class (cpp-class) + "Generate SLK construct and load function. This function needs to be used +to load pointers to polymorphic types. Raise `SLK-ERROR' if an unsupported or +invalid class definition is encountered during code generation. Note that the +code generation expects the declarations and definitions to be in `slk` +namespace." + (check-type cpp-class lcp::cpp-class) + (with-output-to-string (cpp-out) + (lcp::with-cpp-block-output + (cpp-out :name (construct-and-load-function-declaration-for-class cpp-class)) + (write-line (construct-and-load-function-code-for-class cpp-class) cpp-out)))) + +;;; CPP-ENUM serialization generation + (defun save-function-declaration-for-enum (cpp-enum) "Generate SLK save function declaration for CPP-ENUM. Note that the code generation expects the declarations and definitions to be in `slk` namespace." @@ -172,7 +402,7 @@ generation expects the declarations and definitions to be in `slk` namespace." enum-ix (lcp::cpp-type-decl cpp-enum) (lcp::cpp-enumerator-name enum-value))) - (write-line "default: LOG(FATAL) << \"Trying to load unknown enum value!\";" s)))) + (write-line "default: throw slk::SlkDecodeException(\"Trying to load unknown enum value!\");" s)))) (defun load-function-definition-for-enum (cpp-enum) "Generate SLK save function. Note that the code generation expects the diff --git a/src/lisp/types.lisp b/src/lisp/types.lisp index 90752e754..b86c9c5d3 100644 --- a/src/lisp/types.lisp +++ b/src/lisp/types.lisp @@ -160,6 +160,17 @@ ;; In case of multiple inheritance, pretend we only inherit the 1st base class. (ignore-other-base-classes nil :type boolean :read-only t)) +(defstruct slk-opts + "SLK serialization options for C++ class." + ;; BASE is T if the class should be treated as a base class for SLK, even + ;; though it may have parents. + (base nil :type boolean :read-only t) + ;; Extra arguments to the generated save function. List of (name cpp-type). + (save-args nil :read-only t) + (load-args nil :read-only t) + ;; In case of multiple inheritance, pretend we only inherit the 1st base class. + (ignore-other-base-classes nil :type boolean :read-only t)) + (defclass cpp-class (cpp-type) ((structp :type boolean :initarg :structp :initform nil :reader cpp-class-structp) @@ -173,6 +184,8 @@ (private :initarg :private :initform nil :accessor cpp-class-private) (capnp-opts :type (or null capnp-opts) :initarg :capnp-opts :initform nil :reader cpp-class-capnp-opts) + (slk-opts :type (or null slk-opts) :initarg :slk-opts :initform nil + :reader cpp-class-slk-opts) (inner-types :initarg :inner-types :initform nil :reader cpp-class-inner-types) (abstractp :initarg :abstractp :initform nil :reader cpp-class-abstractp)) (:documentation "Meta information on a C++ class (or struct).")) @@ -567,6 +580,8 @@ Generates C++: :capnp-opts ,(when (member :capnp serialize) `(and *capnp-serialize-p* (make-capnp-opts ,@(cdr (member :capnp serialize))))) + :slk-opts ,(when (member :slk serialize) + `(make-slk-opts ,@(cadr (member :slk serialize)))) :abstractp ,abstractp :namespace (reverse *cpp-namespaces*) ;; Set inner types at the end. This works