POD serialization, rocksdb integration and Gleich's optimization

Reviewers: buda, dgleich, mferencevic, msantl, teon.banek Reviewed By: buda, dgleich, teon.banek Subscribers: teon.banek, pullbot Differential Revision: https://phabricator.memgraph.io/D1399
2018-06-12 11:29:22 +02:00 · 2018-06-12 11:29:22 +02:00 · 035540c598
commit 035540c598
parent a22ca94d16
32 changed files with 1268 additions and 412 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -8,6 +8,7 @@

 * Dynamic graph partitioner added.
 * Static vertices/edges id generators exposed through the Id Cypher function.
+* Properties on disk added.

 ### Bug Fixes and Other Changes

--- a/docs/dev/memgraph/storage/property-storage.md
+++ b/docs/dev/memgraph/storage/property-storage.md
@ -0,0 +1,131 @@
+# Property storage
+
+Although the reader is probably familiar with properties in *Memgraph*, let's
+briefly recap. 
+
+Both vertices and edges can store an arbitrary number of properties. Properties
+are, in essence, ordered pairs of property names and property values. Each
+property name within a single graph element (edge/node) can store a single
+property value. Property names are represented as strings, while property values
+must be one of the following types:
+
+ Type      | Description
+-----------|------------
+ `Null`    | Denotes that the property has no value. This is the same as if the property does not exist.
+ `String`  | A character string, i.e. text.
+ `Boolean` | A boolean value, either `true` or `false`.
+ `Integer` | An integer number.
+ `Float`   | A floating-point number, i.e. a real number.
+ `List`    | A list containing any number of property values of any supported type. It can be used to store multiple values under a single property name.
+ `Map`     | A mapping of string keys to values of any supported type.
+
+Property values are modeled in a class conveniently called `PropertyValue`.
+
+## Mapping between property names and property keys.
+
+Although users think of property names in terms of descriptive strings 
+(e.g. "location" or "department"), *Memgraph* internally converts those names
+into property keys which are, essentially, unsigned 16-bit integers.
+
+Property keys are modelled by a not-so-conveniently named class called
+`Property` which can be found in `storage/types.hpp`. The actual conversion
+between property names and property keys is done within the `ConcurrentIdMapper`
+but the internals of that implementation are out of scope for understanding
+property storage.
+
+## PropertyValueStore
+
+Both `Edge` and `Vertex` objects contain an instance of `PropertyValueStore`
+object which is responsible for storing properties of a corresponding graph
+element.
+
+An interface of `PropertyValueStore` is as follows:
+
+ Method    | Description
+-----------|------------
+ `at`      | Returns the `PropertyValue` for a given `Property` (key).
+ `set`     | Stores a given `PropertyValue` under a given `Property` (key).
+ `erase`   | Deletes a given `Property` (key) alongside its corresponding `PropertyValue`.
+ `clear`   | Clears the storage.
+ `iterator`| Provides an extension of `std::input_iterator` that iterates over storage.
+
+## Storage location
+
+By default, *Memgraph* is an in-memory database and all properties are therefore
+stored in working memory unless specified otherwise by the user. User has an
+option to specify via the command line which properties they wish to be stored
+on disk.
+
+Storage location of each property is encapsulated within a `Property` object
+which is ensured by the `ConcurrentIdMapper`. More precisely, the unsigned 16-bit
+property key has the following format:
+
+```
+|---location--|------id------|
+|-Memory|Disk-|-----2^15-----|
+``` 
+
+In other words, the most significant bit determines the location where the
+property will be stored. 
+
+### In-memory storage
+
+The underlying implementation of in-memory storage for the time being is
+`std::vector<std::pair<Property, PropertyValue>>`. Implementations of`at`, `set`
+and `erase` are linear in time. This implementation is arguably more efficient
+than `std::map` or `std::unordered_map` when the average number of properties of
+a record is relatively small (up to 10) which seems to be the case.
+
+### On-disk storage
+
+#### KVStore
+
+Disk storage is modeled by an abstraction of key-value storage as implemented in
+`storage/kvstore.hpp'. An interface of this abstraction is as follows:
+
+ Method         | Description
+----------------|------------
+ `Put`          | Stores the given value under the given key.
+ `Get`          | Obtains the given value stored under the given key.
+ `Delete`       | Deletes a given (key, value) pair from storage..
+ `DeletePrefix` | Deletes all (key, value) pairs where key begins with a given prefix.
+ `Size`         | Returns the size of the storage or, optionally, the number of stored pairs that begin with a given prefix.
+ `iterator`     | Provides an extension of `std::input_iterator` that iterates over storage.
+
+Keys and values in this context are of type `std::string`.
+
+The actual underlying implementation of this abstraction uses
+[RocksDB]{https://rocksdb.org} &mdash; a persistent key-value store for fast
+storage.
+
+It is worthy to note that the custom iterator implementation allows the user
+to iterate over a given prefix. Otherwise, the implementation follows familiar
+c++ constructs and can be used as follows:
+
+```
+KVStore storage = ...;
+for (auto it = storage.begin(); it != storage.end(); ++it) {}
+for (auto kv : storage) {}
+for (auto it = storage.begin("prefix"); it != storage.end("prefix"); ++it) {}
+```
+
+Note that it is not possible to scan over multiple prefixes. For instance, one
+might assume that you can scan over all keys that fall in a certain
+lexicographical range. Unfortunately, that is not the case and running the
+following code will result in an infinite loop with a touch of undefined
+behavior.
+
+```
+KVStore storage = ...;
+for (auto it = storage.begin("alpha"); it != storage.end("omega"); ++it) {}
+```
+
+#### Data organization on disk
+
+Each `PropertyValueStore` instance can access a static `KVStore` object that can
+store `(key, value)` pairs on disk. The key of each property on disk consists of
+two parts &mdash; a unique identifier (unsigned 64-bit integer) of the current
+record version (see mvcc docummentation for further clarification) and a 
+property key as described above. The actual value of the property is serialized
+into a bytestring using bolt `BaseEncoder`. Similarly, deserialization is
+performed by bolt `Decoder`.
--- a/docs/user_technical/storage.md
+++ b/docs/user_technical/storage.md
@ -75,3 +75,27 @@ However, these queries are not:

    MATCH (n:Node) SET n.property[0] = 0
    MATCH (n:Node) SET n.property.key = "other value"
+
+### Cold data on disk
+
+Although *Memgraph* is an in-memory database by default, it offers an option
+to store a certain amount of data on disk. More precisely, the user can pass
+a list of properties they wish to keep stored on disk via the command line.
+In certain cases, this might result in a significant performance boost due to
+reduced memory usage. It is recommended to use this feature on large,
+cold properties, i.e. properties that are rarely accessed.
+
+For example, a user of a library database might identify author biographies
+and book summaries as cold properties. In that case, the user should run
+*Memgraph* as follows:
+
+```
+/usr/lib/memgraph/memgraph --properties-on-disk biography,summary
+```
+
+Note that the usage of *Memgraph* has not changed, i.e. durability and
+data recovery mechanisms are still in place and the query language remains
+the same. It is also important to note that the user cannot change the storage
+location of a property while *Memgraph* is running. Naturally, the user can
+reload their database from snapshot, provide a different list of properties on
+disk and rest assured that only those properties will be stored on disk.
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@ -213,6 +213,10 @@ add_dependencies(memgraph_lib generate_capnp)
 add_library(kvstore_lib STATIC storage/kvstore.cpp)
 target_link_libraries(kvstore_lib stdc++fs mg-utils rocksdb bzip2 zlib)

+# STATIC library for dummy key-value storage
+add_library(kvstore_dummy_lib STATIC storage/kvstore_dummy.cpp)
+target_link_libraries(kvstore_dummy_lib mg-utils)
+
 # Generate a version.hpp file
 set(VERSION_STRING ${memgraph_VERSION})
 configure_file(version.hpp.in version.hpp @ONLY)
@ -220,7 +224,7 @@ include_directories(${CMAKE_CURRENT_BINARY_DIR})

 # memgraph main executable
 add_executable(memgraph memgraph_bolt.cpp)
-target_link_libraries(memgraph memgraph_lib)
+target_link_libraries(memgraph memgraph_lib kvstore_lib)
 set_target_properties(memgraph PROPERTIES
                      # Set the executable output name to include version information.
                      OUTPUT_NAME "memgraph-${memgraph_VERSION}-${COMMIT_HASH}_${CMAKE_BUILD_TYPE}"
--- a/src/database/storage.hpp
+++ b/src/database/storage.hpp
@ -10,7 +10,7 @@
 #include "mvcc/version_list.hpp"
 #include "storage/address.hpp"
 #include "storage/edge.hpp"
-#include "storage/kvstore_mock.hpp"
+#include "storage/kvstore.hpp"
 #include "storage/types.hpp"
 #include "storage/vertex.hpp"
 #include "transactions/type.hpp"
--- a/src/storage/kvstore.cpp
+++ b/src/storage/kvstore.cpp
@ -1,40 +1,137 @@
-#include "storage/kvstore.hpp"
+#include <rocksdb/db.h>
+#include <rocksdb/options.h>

+#include "storage/kvstore.hpp"
 #include "utils/file.hpp"

 namespace storage {

-KVStore::KVStore(fs::path storage) : storage_(storage) {
-  if (!utils::EnsureDir(storage))
-    throw KVStoreError("Folder for the key-value store " + storage.string() +
-                       " couldn't be initialized!");
-  options_.create_if_missing = true;
+struct KVStore::impl {
+  std::experimental::filesystem::path storage;
+  std::unique_ptr<rocksdb::DB> db;
+  rocksdb::Options options;
+};
+
+KVStore::KVStore(fs::path storage) : pimpl_(std::make_unique<impl>()) {
+  pimpl_->storage = storage;
+  if (!utils::EnsureDir(pimpl_->storage))
+    throw KVStoreError("Folder for the key-value store " +
+                       pimpl_->storage.string() + " couldn't be initialized!");
+  pimpl_->options.create_if_missing = true;
  rocksdb::DB *db = nullptr;
-  auto s = rocksdb::DB::Open(options_, storage.c_str(), &db);
+  auto s = rocksdb::DB::Open(pimpl_->options, storage.c_str(), &db);
  if (!s.ok())
    throw KVStoreError("RocksDB couldn't be initialized inside " +
-                       storage.string() + "!");
-  db_.reset(db);
+                       storage.string() + " -- " + std::string(s.ToString()));
+  pimpl_->db.reset(db);
+}
+
+KVStore::~KVStore() {}
+
+KVStore::KVStore(KVStore &&other) { pimpl_ = std::move(other.pimpl_); }
+
+KVStore &KVStore::operator=(KVStore &&other) {
+  pimpl_ = std::move(other.pimpl_);
+  return *this;
 }

 bool KVStore::Put(const std::string &key, const std::string &value) {
-  auto s = db_->Put(rocksdb::WriteOptions(), key.c_str(), value.c_str());
-  if (!s.ok()) return false;
-  return true;
+  auto s = pimpl_->db->Put(rocksdb::WriteOptions(), key, value);
+  return s.ok();
 }

 std::experimental::optional<std::string> KVStore::Get(
    const std::string &key) const noexcept {
  std::string value;
-  auto s = db_->Get(rocksdb::ReadOptions(), key.c_str(), &value);
+  auto s = pimpl_->db->Get(rocksdb::ReadOptions(), key, &value);
  if (!s.ok()) return std::experimental::nullopt;
  return value;
 }

 bool KVStore::Delete(const std::string &key) {
-  auto s = db_->SingleDelete(rocksdb::WriteOptions(), key.c_str());
-  if (!s.ok()) return false;
+  auto s = pimpl_->db->Delete(rocksdb::WriteOptions(), key);
+  return s.ok();
+}
+
+bool KVStore::DeletePrefix(const std::string &prefix) {
+  std::unique_ptr<rocksdb::Iterator> iter = std::unique_ptr<rocksdb::Iterator>(
+      pimpl_->db->NewIterator(rocksdb::ReadOptions()));
+  for (iter->Seek(prefix); iter->Valid() && iter->key().starts_with(prefix);
+       iter->Next()) {
+    if (!pimpl_->db->Delete(rocksdb::WriteOptions(), iter->key()).ok())
+      return false;
+  }
  return true;
 }

+// iterator
+
+struct KVStore::iterator::impl {
+  const KVStore *kvstore;
+  std::string prefix;
+  std::unique_ptr<rocksdb::Iterator> it;
+  std::pair<std::string, std::string> disk_prop;
+};
+
+KVStore::iterator::iterator(const KVStore *kvstore, const std::string &prefix,
+                            bool at_end)
+    : pimpl_(std::make_unique<impl>()) {
+  pimpl_->kvstore = kvstore;
+  pimpl_->prefix = prefix;
+  pimpl_->it = std::unique_ptr<rocksdb::Iterator>(
+      pimpl_->kvstore->pimpl_->db->NewIterator(rocksdb::ReadOptions()));
+  pimpl_->it->Seek(pimpl_->prefix);
+  if (!pimpl_->it->Valid() || !pimpl_->it->key().starts_with(pimpl_->prefix) ||
+      at_end)
+    pimpl_->it = nullptr;
+}
+
+KVStore::iterator::iterator(KVStore::iterator &&other) {
+  pimpl_ = std::move(other.pimpl_);
+}
+
+KVStore::iterator::~iterator() {}
+
+KVStore::iterator &KVStore::iterator::operator=(KVStore::iterator &&other) {
+  pimpl_ = std::move(other.pimpl_);
+  return *this;
+}
+
+KVStore::iterator &KVStore::iterator::operator++() {
+  pimpl_->it->Next();
+  if (!pimpl_->it->Valid() || !pimpl_->it->key().starts_with(pimpl_->prefix))
+    pimpl_->it = nullptr;
+  return *this;
+}
+
+bool KVStore::iterator::operator==(const iterator &other) const {
+  return pimpl_->kvstore == other.pimpl_->kvstore &&
+         pimpl_->prefix == other.pimpl_->prefix &&
+         pimpl_->it == other.pimpl_->it;
+}
+
+bool KVStore::iterator::operator!=(const iterator &other) const {
+  return !(*this == other);
+}
+
+KVStore::iterator::reference KVStore::iterator::operator*() {
+  pimpl_->disk_prop = {pimpl_->it->key().ToString(),
+                       pimpl_->it->value().ToString()};
+  return pimpl_->disk_prop;
+}
+
+KVStore::iterator::pointer KVStore::iterator::operator->() { return &**this; }
+
+void KVStore::iterator::SetInvalid() { pimpl_->it = nullptr; }
+
+bool KVStore::iterator::IsValid() { return pimpl_->it != nullptr; }
+
+// TODO(ipaljak) The complexity of the size function should be at most
+//               logarithmic.
+size_t KVStore::Size(const std::string &prefix) {
+  size_t size = 0;
+  for (auto it = this->begin(prefix); it != this->end(prefix); ++it) ++size;
+  return size;
+}
+
 }  // namespace storage
--- a/src/storage/kvstore.hpp
+++ b/src/storage/kvstore.hpp
@ -5,9 +5,6 @@
 #include <memory>
 #include <string>

-#include <rocksdb/db.h>
-#include <rocksdb/options.h>
-
 #include "utils/exceptions.hpp"

 namespace storage {
@ -23,6 +20,8 @@ class KVStoreError : public utils::BasicException {
 */
 class KVStore final {
 public:
+  KVStore() = delete;
+
  /**
   * @param storage Path to a directory where the data is persisted.
   *
@ -31,6 +30,14 @@ class KVStore final {
   */
  explicit KVStore(std::experimental::filesystem::path storage);

+  KVStore(const KVStore &other) = delete;
+  KVStore(KVStore &&other);
+
+  KVStore &operator=(const KVStore &other) = delete;
+  KVStore &operator=(KVStore &&other);
+
+  ~KVStore();
+
  /**
   * Store value under the given key.
   *
@ -54,7 +61,7 @@ class KVStore final {
      noexcept;

  /**
-   * Delete value under the given key.
+   * Deletes the key and corresponding value from storage.
   *
   * @param key
   *
@ -64,10 +71,89 @@ class KVStore final {
   */
  bool Delete(const std::string &key);

+  /**
+   * Delete all (key, value) pairs where key begins with a given prefix.
+   *
+   * @param prefix - prefix of the keys in (key, value) pairs to be deleted.
+   *                 This parameter is optional and is empty by default.
+   *
+   * @return True on success, false on error. The return value is
+   *         true if the key doesn't exist and underlying storage
+   *         didn't encounter any error.
+   */
+  bool DeletePrefix(const std::string &prefix = "");
+
+  /**
+   * Returns total number of stored (key, value) pairs. The function takes an
+   * optional prefix parameter used for filtering keys that start with that
+   * prefix.
+   *
+   * @param prefix - prefix on which the keys should be filtered. This parameter
+   *                 is optional and is empty by default.
+   *
+   * @return - number of stored pairs.
+   */
+  size_t Size(const std::string &prefix = "");
+
+  /**
+   * Custom prefix-based iterator over kvstore.
+   *
+   * It filters all (key, value) pairs where the key has a certain prefix
+   * and behaves as if all of those pairs are stored in a single iterable
+   * collection of std::pair<std::string, std::string>.
+   */
+  class iterator final
+      : public std::iterator<
+            std::input_iterator_tag,                      // iterator_category
+            std::pair<std::string, std::string>,          // value_type
+            long,                                         // difference_type
+            const std::pair<std::string, std::string> *,  // pointer
+            const std::pair<std::string, std::string> &   // reference
+            > {
+   public:
+    explicit iterator(const KVStore *kvstore, const std::string &prefix = "",
+                      bool at_end = false);
+
+    iterator(const iterator &other) = delete;
+
+    iterator(iterator &&other);
+
+    ~iterator();
+
+    iterator &operator=(iterator &&other);
+
+    iterator &operator=(const iterator &other) = delete;
+
+    iterator &operator++();
+
+    bool operator==(const iterator &other) const;
+
+    bool operator!=(const iterator &other) const;
+
+    reference operator*();
+
+    pointer operator->();
+
+    void SetInvalid();
+
+    bool IsValid();
+
+   private:
+    struct impl;
+    std::unique_ptr<impl> pimpl_;
+  };
+
+  iterator begin(const std::string &prefix = "") {
+    return iterator(this, prefix);
+  }
+
+  iterator end(const std::string &prefix = "") {
+    return iterator(this, prefix, true);
+  }
+
 private:
-  std::experimental::filesystem::path storage_;
-  std::unique_ptr<rocksdb::DB> db_;
-  rocksdb::Options options_;
+  struct impl;
+  std::unique_ptr<impl> pimpl_;
 };

 }  // namespace storage
--- a/src/storage/kvstore_dummy.cpp
+++ b/src/storage/kvstore_dummy.cpp
@ -0,0 +1,81 @@
+#include "storage/kvstore.hpp"
+
+#include "glog/logging.h"
+#include "utils/file.hpp"
+
+namespace storage {
+
+struct KVStore::impl {};
+
+KVStore::KVStore(fs::path storage) {}
+
+KVStore::~KVStore() {}
+
+bool KVStore::Put(const std::string &key, const std::string &value) {
+  CHECK(false)
+      << "Unsupported operation (KVStore::Put) -- this is a dummy kvstore";
+}
+
+std::experimental::optional<std::string> KVStore::Get(
+    const std::string &key) const noexcept {
+  CHECK(false)
+      << "Unsupported operation (KVStore::Get) -- this is a dummy kvstore";
+}
+
+bool KVStore::Delete(const std::string &key) {
+  CHECK(false)
+      << "Unsupported operation (KVStore::Delete) -- this is a dummy kvstore";
+}
+
+bool KVStore::DeletePrefix(const std::string &prefix) {
+  CHECK(false) << "Unsupported operation (KVStore::DeletePrefix) -- this is a "
+                  "dummy kvstore";
+}
+
+// iterator
+
+struct KVStore::iterator::impl {};
+
+KVStore::iterator::iterator(const KVStore *kvstore, const std::string &prefix,
+                            bool at_end)
+    : pimpl_(new impl()) {}
+
+KVStore::iterator::iterator(KVStore::iterator &&other) {
+  pimpl_ = std::move(other.pimpl_);
+}
+
+KVStore::iterator::~iterator() {}
+
+KVStore::iterator &KVStore::iterator::operator=(KVStore::iterator &&other) {
+  pimpl_ = std::move(other.pimpl_);
+  return *this;
+}
+
+KVStore::iterator &KVStore::iterator::operator++() {
+  CHECK(false) << "Unsupported operation (&KVStore::iterator::operator++) -- "
+                  "this is a dummy kvstore";
+}
+
+bool KVStore::iterator::operator==(const iterator &other) const { return true; }
+
+bool KVStore::iterator::operator!=(const iterator &other) const {
+  return false;
+}
+
+KVStore::iterator::reference KVStore::iterator::operator*() {
+  CHECK(false) << "Unsupported operation (KVStore::iterator::operator*)-- this "
+                  "is a dummy kvstore";
+}
+
+KVStore::iterator::pointer KVStore::iterator::operator->() {
+  CHECK(false) << "Unsupported operation (KVStore::iterator::operator->) -- "
+                  "this is a dummy kvstore";
+}
+
+void KVStore::iterator::SetInvalid() {}
+
+bool KVStore::iterator::IsValid() { return false; }
+
+size_t KVStore::Size(const std::string &prefix) { return 0; }
+
+}  // namespace storage
--- a/src/storage/kvstore_mock.hpp
+++ b/src/storage/kvstore_mock.hpp
@ -1,40 +0,0 @@
-#pragma once
-
-#include <experimental/optional>
-#include <map>
-#include <string>
-
-#include <glog/logging.h>
-
-// TODO(ipaljak): replace with the real implementation
-
-namespace storage {
-
-class KVStore {
- public:
-  explicit KVStore(const std::string &) {}
-
-  bool Put(const std::string &key, const std::string &value) {
-    VLOG(31) << "PUT: " << key << " : " << value;
-    storage_[key] = value;
-    return true;
-  }
-
-  std::experimental::optional<std::string> Get(const std::string &key) const {
-    VLOG(31) << "GET: " << key;
-    auto it = storage_.find(key);
-    if (it == storage_.end()) return std::experimental::nullopt;
-    return it->second;
-  }
-
-  bool Delete(const std::string &key) {
-    VLOG(31) << "DELETE: " << key;
-    storage_.erase(key);
-    return true;
-  }
-
- private:
-  std::map<std::string, std::string> storage_;
-};
-
-} // namespace storage
--- a/src/storage/pod_buffer.hpp
+++ b/src/storage/pod_buffer.hpp
@ -0,0 +1,46 @@
+#pragma once
+
+#include "communication/bolt/v1/encoder/base_encoder.hpp"
+
+namespace storage {
+
+/**
+ * Buffer used for serialization of disk properties. The buffer
+ * implements a template parameter Buffer interface from BaseEncoder
+ * and Decoder classes for bolt serialization.
+ */
+class PODBuffer {
+ public:
+  PODBuffer() = default;
+  explicit PODBuffer(const std::string &s) {
+    buffer = std::vector<uint8_t>{s.begin(), s.end()};
+  }
+
+  /**
+   * Writes data to buffer
+   *
+   * @param data - Pointer to data to be written.
+   * @param len - Data length.
+   */
+  void Write(const uint8_t *data, size_t len) {
+    for (size_t i = 0; i < len; ++i) buffer.push_back(data[i]);
+  }
+
+  /**
+   * Reads raw data from buffer.
+   *
+   * @param data - pointer to where data should be stored.
+   * @param len - data length
+   * @return - True if successful, False otherwise.
+   */
+  bool Read(uint8_t *data, size_t len) {
+    if (len > buffer.size()) return false;
+    memcpy(data, buffer.data(), len);
+    buffer.erase(buffer.begin(), buffer.begin() + len);
+    return true;
+  }
+
+  std::vector<uint8_t> buffer;
+};
+
+}  // namespace storage
--- a/src/storage/property_value_store.cpp
+++ b/src/storage/property_value_store.cpp
@ -1,10 +1,52 @@
+#include <experimental/filesystem>
+
+#include "gflags/gflags.h"
+#include "glog/logging.h"
+
+#include "storage/pod_buffer.hpp"
 #include "storage/property_value_store.hpp"

-using Property = storage::Property;
-using Location = storage::Location;
+namespace fs = std::experimental::filesystem;

-std::atomic<uint64_t> PropertyValueStore::disk_key_cnt_ = {0};
-storage::KVStore PropertyValueStore::disk_storage_("properties");
+using namespace communication::bolt;
+
+std::atomic<uint64_t> PropertyValueStore::global_key_cnt_ = {0};
+
+// properties on disk are stored in a directory named properties within the
+// durability directory
+DECLARE_string(durability_directory);
+DECLARE_string(properties_on_disk);
+
+std::string DiskKeyPrefix(const std::string &version_key) {
+  return version_key + disk_key_separator;
+}
+
+std::string DiskKey(const std::string &version_key,
+                    const std::string &property_id) {
+  return DiskKeyPrefix(version_key) + property_id;
+}
+
+PropertyValueStore::PropertyValueStore(const PropertyValueStore &old)
+    : props_(old.props_) {
+  // We need to update disk key and disk key counter when calling a copy
+  // constructor due to mvcc.
+  if (!FLAGS_properties_on_disk.empty()) {
+    version_key_ = global_key_cnt_++;
+    storage::KVStore::iterator old_disk_it(
+        &DiskStorage(), DiskKeyPrefix(std::to_string(old.version_key_)));
+    iterator it(&old, old.props_.end(), std::move(old_disk_it));
+
+    while (it != old.end()) {
+      this->set(it->first, it->second);
+      ++it;
+    }
+  }
+}
+
+PropertyValueStore::~PropertyValueStore() {
+  if (!FLAGS_properties_on_disk.empty())
+    DiskStorage().DeletePrefix(DiskKeyPrefix(std::to_string(version_key_)));
+}

 PropertyValue PropertyValueStore::at(const Property &key) const {
  auto GetValue = [&key](const auto &props) {
@ -15,7 +57,11 @@ PropertyValue PropertyValueStore::at(const Property &key) const {

  if (key.Location() == Location::Memory) return GetValue(props_);

-  return GetValue(PropsOnDisk(std::to_string(disk_key_)));
+  std::string disk_key =
+      DiskKey(std::to_string(version_key_), std::to_string(key.Id()));
+  auto serialized_prop = DiskStorage().Get(disk_key);
+  if (serialized_prop) return DeserializeProp(serialized_prop.value());
+  return PropertyValue::Null;
 }

 void PropertyValueStore::set(const Property &key, const char *value) {
@ -40,80 +86,63 @@ void PropertyValueStore::set(const Property &key, const PropertyValue &value) {
  if (key.Location() == Location::Memory) {
    SetValue(props_);
  } else {
-    auto props_on_disk = PropsOnDisk(std::to_string(disk_key_));
-    SetValue(props_on_disk);
-    disk_storage_.Put(std::to_string(disk_key_), SerializeProps(props_on_disk));
+    std::string disk_key =
+        DiskKey(std::to_string(version_key_), std::to_string(key.Id()));
+    DiskStorage().Put(disk_key, SerializeProp(value));
  }
 }

-size_t PropertyValueStore::erase(const Property &key) {
+bool PropertyValueStore::erase(const Property &key) {
  auto EraseKey = [&key](auto &props) {
    auto found = std::find_if(props.begin(), props.end(),
                              [&key](std::pair<Property, PropertyValue> &kv) {
                                return kv.first == key;
                              });
-    if (found != props.end()) {
-      props.erase(found);
-      return true;
-    }
-    return false;
+    if (found != props.end()) props.erase(found);
+    return true;
  };

  if (key.Location() == Location::Memory) return EraseKey(props_);

-  auto props_on_disk = PropsOnDisk(std::to_string(disk_key_));
-  if (EraseKey(props_on_disk)) {
-    if (props_on_disk.empty())
-      return disk_storage_.Delete(std::to_string(disk_key_));
-    return disk_storage_.Put(std::to_string(disk_key_),
-                             SerializeProps(props_on_disk));
-  }
-
-  return false;
+  std::string disk_key =
+      DiskKey(std::to_string(version_key_), std::to_string(key.Id()));
+  return DiskStorage().Delete(disk_key);
 }

 void PropertyValueStore::clear() {
  props_.clear();
-  disk_storage_.Delete(std::to_string(disk_key_));
+  if (!FLAGS_properties_on_disk.empty())
+    DiskStorage().DeletePrefix(DiskKeyPrefix(std::to_string(version_key_)));
 }

-/* TODO(ipaljak): replace serialize/deserialize with the real implementation.
- * Currently supporting a only one property on disk per record and that property
- * must be a string.
- * */
-std::string PropertyValueStore::SerializeProps(
-    const std::vector<std::pair<Property, PropertyValue>> &props) const {
-  if (props.size() > 1) throw std::runtime_error("Unsupported operation");
-  std::stringstream strstream;
-  strstream << props[0].first.Id() << "," << props[0].second;
-  return strstream.str();
+storage::KVStore &PropertyValueStore::DiskStorage() const {
+  static auto disk_storage = ConstructDiskStorage();
+  return disk_storage;
 }

-std::vector<std::pair<Property, PropertyValue>> PropertyValueStore::Deserialize(
-    const std::string &serialized_props) const {
-  std::istringstream strstream(serialized_props);
-
-  std::string s;
-  std::getline(strstream, s, ',');
-
-  uint16_t id;
-  std::istringstream ss(s);
-  ss >> id;
-
-  Property key(id, Location::Disk);
-
-  std::getline(strstream, s, ',');
-  PropertyValue value(s);
-
-  std::vector<std::pair<Property, PropertyValue>> ret;
-  ret.emplace_back(key, value);
-
-  return ret;
+std::string PropertyValueStore::SerializeProp(const PropertyValue &prop) const {
+  storage::PODBuffer pod_buffer;
+  BaseEncoder<storage::PODBuffer> encoder{pod_buffer};
+  encoder.WriteTypedValue(prop);
+  return std::string(reinterpret_cast<char *>(pod_buffer.buffer.data()),
+                     pod_buffer.buffer.size());
 }

-std::vector<std::pair<Property, PropertyValue>> PropertyValueStore::PropsOnDisk(
-    const std::string &disk_key) const {
-  auto serialized = disk_storage_.Get(disk_key);
-  if (serialized) return Deserialize(disk_storage_.Get(disk_key).value());
-  return {};
+PropertyValue PropertyValueStore::DeserializeProp(
+    const std::string &serialized_prop) const {
+  storage::PODBuffer pod_buffer{serialized_prop};
+  Decoder<storage::PODBuffer> decoder{pod_buffer};
+
+  DecodedValue dv;
+  if (!decoder.ReadValue(&dv)) {
+    DLOG(WARNING) << "Unable to read property value";
+    return PropertyValue::Null;
+  }
+  return dv.operator PropertyValue();
+}
+
+storage::KVStore PropertyValueStore::ConstructDiskStorage() const {
+  auto storage_path = fs::path() / FLAGS_durability_directory / "properties";
+  if (fs::exists(storage_path)) fs::remove_all(storage_path);
+  return storage::KVStore(storage_path);
 }
--- a/src/storage/property_value_store.hpp
+++ b/src/storage/property_value_store.hpp
@ -1,32 +1,33 @@
 #pragma once
-#include <algorithm>
+
 #include <atomic>
 #include <experimental/optional>
-#include <map>
-#include <memory>
+#include <string>
 #include <vector>

-#include "glog/logging.h"
-
-#include "storage/kvstore_mock.hpp"
+#include "storage/kvstore.hpp"
 #include "storage/property_value.hpp"
 #include "storage/types.hpp"
+
+const std::string disk_key_separator = "_";
+
+std::string DiskKey(const std::string &version_key,
+                    const std::string &property_id);
+
+std::string DiskKeyPrefix(const std::string &version_key);
+
 /**
 * A collection of properties accessed in a map-like way using a key of type
- * Properties::Property.
+ * Storage::Property.
 *
 * PropertyValueStore handles storage on disk or in memory. Property key defines
 * where the corresponding property should be stored. Each instance of
- * PropertyValueStore contains a disk_key_ member which specifies where on
+ * PropertyValueStore contains a version_key_ member which specifies where on
 * disk should the properties be stored. That key is inferred from a static
- * global counter disk_key_cnt_.
+ * global counter global_key_cnt_.
 *
 * The underlying implementation of in-memory storage is not necessarily
 * std::map.
- *
- * TODO(ipaljak) modify on-disk storage so that each property has its own
- * key for storage. Storage key should, in essence, be an ordered pair
- * (global key, property key).
 */
 class PropertyValueStore {
  using Property = storage::Property;
@ -38,18 +39,9 @@ class PropertyValueStore {
  static constexpr char IdPropertyName[] = "__id__";

  PropertyValueStore() = default;
+  PropertyValueStore(const PropertyValueStore &old);

-  PropertyValueStore(const PropertyValueStore &old) {
-    // We need to update disk key and disk key counter when calling a copy
-    // constructor due to mvcc.
-    props_ = old.props_;
-    disk_key_ = disk_key_cnt_++;
-    auto old_value = disk_storage_.Get(std::to_string(old.disk_key_));
-    if (old_value)
-      disk_storage_.Put(std::to_string(disk_key_), old_value.value());
-  }
-
-  ~PropertyValueStore() { disk_storage_.Delete(std::to_string(disk_key_)); }
+  ~PropertyValueStore();

  /**
   * Returns a PropertyValue (by reference) at the given key.
@ -63,39 +55,6 @@ class PropertyValueStore {
   */
  PropertyValue at(const Property &key) const;

-  /**
-   * Sets the value for the given key. A new PropertyValue instance
-   * is created for the given value (which is of raw type). If the property
-   * is to be stored on disk then that instance does not represent an additional
-   * memory overhead as it goes out of scope at the end of this method.
-   *
-   * @tparam TValue Type of value. It must be one of the
-   * predefined possible PropertyValue values (bool, string, int...)
-   * @param key  The key for which the property is set. The previous
-   * value at the same key (if there was one) is replaced.
-   * @param value  The value to set.
-   */
-  template <typename TValue>
-  void set(const Property &key, const TValue &value) {
-    auto SetValue = [&key, &value](auto &props) {
-      for (auto &kv : props)
-        if (kv.first == key) {
-          kv.second = value;
-          return;
-        }
-      props.emplace_back(key, value);
-    };
-
-    if (key.Location() == Location::Memory) {
-      SetValue(props_);
-    } else {
-      auto props_on_disk = PropsOnDisk(std::to_string(disk_key_));
-      SetValue(props_on_disk);
-      disk_storage_.Put(std::to_string(disk_key_),
-                        SerializeProps(props_on_disk));
-    }
-  }
-
  /**
   * Set overriding for character constants. Forces conversion
   * to std::string, otherwise templating might cast the pointer
@ -112,52 +71,74 @@ class PropertyValueStore {
  /**
   * Removes the PropertyValue for the given key.
   *
-   * @param key The key for which to remove the property.
-   * @return  The number of removed properties (0 or 1).
+   * @param key - The key for which to remove the property.
+   *
+   * @return true if the operation was successful and there is nothing stored
+   *         under given key after this operation.
   */
-  size_t erase(const Property &key);
+  bool erase(const Property &key);

  /** Removes all the properties (both in-mem and on-disk) from this store. */
  void clear();

+  /**
+   * Returns a static storage::kvstore instance used for storing properties on
+   * disk. This hack is needed due to statics that are internal to rocksdb and
+   * availability of durability_directory flag.
+   */
+  storage::KVStore &DiskStorage() const;
+
  /**
   * Custom PVS iterator behaves as if all properties are stored in a single
   * iterable collection of std::pair<Property, PropertyValue>.
-   * */
-  class iterator : public std::iterator<
-                       std::input_iterator_tag,             // iterator_category
-                       std::pair<Property, PropertyValue>,  // value_type
-                       long,                                // difference_type
-                       const std::pair<Property, PropertyValue> *,  // pointer
-                       const std::pair<Property, PropertyValue> &   // reference
-                       > {
+   */
+  class iterator final
+      : public std::iterator<
+            std::input_iterator_tag,                     // iterator_category
+            std::pair<Property, PropertyValue>,          // value_type
+            long,                                        // difference_type
+            const std::pair<Property, PropertyValue> *,  // pointer
+            const std::pair<Property, PropertyValue> &   // reference
+            > {
   public:
-    explicit iterator(const PropertyValueStore *init, int it)
-        : PVS_(init), it_(it) {}
+    iterator() = delete;
+
+    iterator(const PropertyValueStore *PVS,
+             std::vector<std::pair<Property, PropertyValue>>::const_iterator
+                 memory_it,
+             storage::KVStore::iterator disk_it)
+        : PVS_(PVS), memory_it_(memory_it), disk_it_(std::move(disk_it)) {}
+
+    iterator(const iterator &other) = delete;
+
+    iterator(iterator &&other) = default;
+
+    iterator &operator=(iterator &&other) = default;
+
+    iterator &operator=(const iterator &other) = delete;

    iterator &operator++() {
-      ++it_;
+      if (memory_it_ != PVS_->props_.end())
+        ++memory_it_;
+      else
+        ++disk_it_;
      return *this;
    }

-    iterator operator++(int) {
-      iterator ret = *this;
-      ++(*this);
-      return ret;
-    }
-
    bool operator==(const iterator &other) const {
-      return PVS_ == other.PVS_ && it_ == other.it_;
+      return PVS_ == other.PVS_ && memory_it_ == other.memory_it_ &&
+             disk_it_ == other.disk_it_;
    }

-    bool operator!=(const iterator &other) const {
-      return PVS_ != other.PVS_ || it_ != other.it_;
-    }
+    bool operator!=(const iterator &other) const { return !(*this == other); }

    reference operator*() {
-      if (it_ < static_cast<int>(PVS_->props_.size())) return PVS_->props_[it_];
-      auto disk_props = PVS_->PropsOnDisk(std::to_string(PVS_->disk_key_));
-      disk_prop_ = disk_props[it_ - PVS_->props_.size()];
+      if (memory_it_ != PVS_->props_.end()) return *memory_it_;
+      std::pair<std::string, std::string> kv = *disk_it_;
+      std::string prop_id =
+          kv.first.substr(kv.first.find(disk_key_separator) + 1);
+      disk_prop_ = {Property(std::stoi(prop_id), Location::Disk),
+                    PVS_->DeserializeProp(kv.second)};
      return disk_prop_.value();
    }

@ -165,33 +146,52 @@ class PropertyValueStore {

   private:
    const PropertyValueStore *PVS_;
-    int32_t it_;
+    std::vector<std::pair<Property, PropertyValue>>::const_iterator memory_it_;
+    storage::KVStore::iterator disk_it_;
    std::experimental::optional<std::pair<Property, PropertyValue>> disk_prop_;
  };

  size_t size() const {
-    size_t ram_size = props_.size();
-    size_t disk_size = PropsOnDisk(std::to_string(disk_key_)).size();
-    return ram_size + disk_size;
+    return props_.size() +
+           DiskStorage().Size(DiskKeyPrefix(std::to_string(version_key_)));
  }

-  auto begin() const { return iterator(this, 0); }
-  auto end() const { return iterator(this, size()); }
+  iterator begin() const {
+    return iterator(
+        this, props_.begin(),
+        DiskStorage().begin(DiskKeyPrefix(std::to_string(version_key_))));
+  }
+
+  iterator end() const {
+    return iterator(
+        this, props_.end(),
+        DiskStorage().end(DiskKeyPrefix(std::to_string(version_key_))));
+  }

 private:
-  static std::atomic<uint64_t> disk_key_cnt_;
-  uint64_t disk_key_ = disk_key_cnt_++;
-
-  static storage::KVStore disk_storage_;
+  static std::atomic<uint64_t> global_key_cnt_;
+  uint64_t version_key_ = global_key_cnt_++;

  std::vector<std::pair<Property, PropertyValue>> props_;

-  std::string SerializeProps(
-      const std::vector<std::pair<Property, PropertyValue>> &props) const;
+  /**
+   * Serializes a single PropertyValue into std::string.
+   *
+   * @param prop - Property to be serialized.
+   *
+   * @return Serialized property.
+   */
+  std::string SerializeProp(const PropertyValue &prop) const;

-  std::vector<std::pair<Property, PropertyValue>> Deserialize(
-      const std::string &serialized_props) const;
+  /**
+   * Deserializes a single PropertyValue from std::string.
+   *
+   * @param serialized_prop - Serialized property.
+   *
+   * @return Deserialized property.
+   */
+  PropertyValue DeserializeProp(const std::string &serialized_prop) const;

-  std::vector<std::pair<Property, PropertyValue>> PropsOnDisk(
-      const std::string &disk_key) const;
+  storage::KVStore ConstructDiskStorage() const;
 };
+
--- a/tests/benchmark/CMakeLists.txt
+++ b/tests/benchmark/CMakeLists.txt
@ -24,7 +24,7 @@ foreach(test_cpp ${test_type_cpps})
    set_target_properties(${target_name} PROPERTIES OUTPUT_NAME ${exec_name})

    # link libraries
-    target_link_libraries(${target_name} memgraph_lib)
+    target_link_libraries(${target_name} memgraph_lib kvstore_dummy_lib)
    # google-benchmark
    target_link_libraries(${target_name} benchmark Threads::Threads)

--- a/tests/distributed/raft/CMakeLists.txt
+++ b/tests/distributed/raft/CMakeLists.txt
@ -22,7 +22,7 @@ foreach(test_cpp ${test_type_cpps})
    set_target_properties(${target_name} PROPERTIES OUTPUT_NAME ${exec_name})

    # link libraries
-    target_link_libraries(${target_name} memgraph_lib)
+    target_link_libraries(${target_name} memgraph_lib kvstore_dummy_lib)

    set(output_path ${CMAKE_BINARY_DIR}/test_results/unit/${target_name}.xml)

--- a/tests/macro_benchmark/CMakeLists.txt
+++ b/tests/macro_benchmark/CMakeLists.txt
@ -26,7 +26,7 @@ foreach(test_cpp ${test_type_cpps})
    set_target_properties(${target_name} PROPERTIES OUTPUT_NAME ${exec_name})

    # link libraries
-    target_link_libraries(${target_name} memgraph_lib)
+    target_link_libraries(${target_name} memgraph_lib kvstore_dummy_lib)

    # add target to dependencies
    add_dependencies(${all_targets_target} ${target_name})
--- a/tests/manual/CMakeLists.txt
+++ b/tests/manual/CMakeLists.txt
@ -28,48 +28,45 @@ add_manual_test(binomial.cpp)
 target_link_libraries(${test_prefix}binomial mg-utils)

 add_manual_test(bolt_client.cpp)
-target_link_libraries(${test_prefix}bolt_client memgraph_lib)
+target_link_libraries(${test_prefix}bolt_client memgraph_lib kvstore_dummy_lib)

 add_manual_test(card_fraud_generate_snapshot.cpp)
-target_link_libraries(${test_prefix}card_fraud_generate_snapshot memgraph_lib)
+target_link_libraries(${test_prefix}card_fraud_generate_snapshot memgraph_lib kvstore_dummy_lib)

 add_manual_test(card_fraud_local.cpp)
-target_link_libraries(${test_prefix}card_fraud_local memgraph_lib)
+target_link_libraries(${test_prefix}card_fraud_local memgraph_lib kvstore_dummy_lib)

 add_manual_test(distributed_repl.cpp)
-target_link_libraries(${test_prefix}distributed_repl memgraph_lib)
+target_link_libraries(${test_prefix}distributed_repl memgraph_lib kvstore_dummy_lib)

 add_manual_test(endinan.cpp)

 add_manual_test(generate_snapshot.cpp)
-target_link_libraries(${test_prefix}generate_snapshot memgraph_lib)
+target_link_libraries(${test_prefix}generate_snapshot memgraph_lib kvstore_dummy_lib)

 add_manual_test(graph_500_generate_snapshot.cpp)
-target_link_libraries(${test_prefix}graph_500_generate_snapshot memgraph_lib)
-
-add_manual_test(kvstore.cpp)
-target_link_libraries(${test_prefix}kvstore gtest gtest_main memgraph_lib kvstore_lib)
+target_link_libraries(${test_prefix}graph_500_generate_snapshot memgraph_lib kvstore_dummy_lib)

 add_manual_test(query_hash.cpp)
-target_link_libraries(${test_prefix}query_hash memgraph_lib)
+target_link_libraries(${test_prefix}query_hash memgraph_lib kvstore_dummy_lib)

 add_manual_test(query_planner.cpp)
-target_link_libraries(${test_prefix}query_planner memgraph_lib)
+target_link_libraries(${test_prefix}query_planner memgraph_lib kvstore_dummy_lib)

 add_manual_test(raft_rpc.cpp)
-target_link_libraries(${test_prefix}raft_rpc memgraph_lib)
+target_link_libraries(${test_prefix}raft_rpc memgraph_lib kvstore_dummy_lib)

 add_manual_test(repl.cpp)
-target_link_libraries(${test_prefix}repl memgraph_lib)
+target_link_libraries(${test_prefix}repl memgraph_lib kvstore_dummy_lib)

 add_manual_test(single_query.cpp)
-target_link_libraries(${test_prefix}single_query memgraph_lib)
+target_link_libraries(${test_prefix}single_query memgraph_lib kvstore_dummy_lib)

 add_manual_test(sl_position_and_count.cpp)
-target_link_libraries(${test_prefix}sl_position_and_count memgraph_lib)
+target_link_libraries(${test_prefix}sl_position_and_count memgraph_lib kvstore_dummy_lib)

 add_manual_test(stripped_timing.cpp)
-target_link_libraries(${test_prefix}stripped_timing memgraph_lib)
+target_link_libraries(${test_prefix}stripped_timing memgraph_lib kvstore_dummy_lib)

 add_manual_test(xorshift.cpp)
 target_link_libraries(${test_prefix}xorshift mg-utils)
--- a/tests/manual/kvstore.cpp
+++ b/tests/manual/kvstore.cpp
@ -1,48 +0,0 @@
-#include <glog/logging.h>
-#include <gtest/gtest.h>
-
-#include "durability/paths.hpp"
-#include "storage/kvstore.hpp"
-#include "utils/file.hpp"
-
-namespace fs = std::experimental::filesystem;
-
-class KVStore : public ::testing::Test {
- protected:
-  virtual void SetUp() { utils::EnsureDir(test_folder_); }
-
-  virtual void TearDown() { fs::remove_all(test_folder_); }
-
-  fs::path test_folder_{fs::path("kvstore_test")};
-};
-
-TEST_F(KVStore, PutGet) {
-  storage::KVStore kvstore(test_folder_ / "PutGet");
-  ASSERT_TRUE(kvstore.Put("key", "value"));
-  ASSERT_EQ(kvstore.Get("key").value(), "value");
-}
-
-TEST_F(KVStore, PutGetDeleteGet) {
-  storage::KVStore kvstore(test_folder_ / "PutGetDeleteGet");
-  ASSERT_TRUE(kvstore.Put("key", "value"));
-  ASSERT_EQ(kvstore.Get("key").value(), "value");
-  ASSERT_TRUE(kvstore.Delete("key"));
-  ASSERT_FALSE(static_cast<bool>(kvstore.Get("key")));
-}
-
-TEST_F(KVStore, Durability) {
-  {
-    storage::KVStore kvstore(test_folder_ / "Durability");
-    ASSERT_TRUE(kvstore.Put("key", "value"));
-  }
-  {
-    storage::KVStore kvstore(test_folder_ / "Durability");
-    ASSERT_EQ(kvstore.Get("key").value(), "value");
-  }
-}
-
-int main(int argc, char **argv) {
-  google::InitGoogleLogging(argv[0]);
-  ::testing::InitGoogleTest(&argc, argv);
-  return RUN_ALL_TESTS();
-}
--- a/tests/property_based/CMakeLists.txt
+++ b/tests/property_based/CMakeLists.txt
@ -22,7 +22,7 @@ foreach(test_cpp ${test_type_cpps})
    set_target_properties(${target_name} PROPERTIES OUTPUT_NAME ${exec_name})

    # link libraries
-    target_link_libraries(${target_name} memgraph_lib)
+    target_link_libraries(${target_name} memgraph_lib kvstore_dummy_lib)
    # gtest
    target_link_libraries(${target_name} gtest gtest_main)
    target_link_libraries(${target_name} rapidcheck rapidcheck_gtest)
--- a/tests/qa/continuous_integration
+++ b/tests/qa/continuous_integration
@ -54,7 +54,7 @@ suites = [
    Test(
        name="memgraph_V1_POD",
        test_suite="memgraph_V1",
-        memgraph_params="--properties-on-disk=surname,location",
+        memgraph_params="--properties-on-disk=x,y,z,w,k,v,a,b,c,d,e,f,r,t,o,prop,age,name,surname,location",
        mandatory=True
    ),
    Test(
@ -62,7 +62,7 @@ suites = [
        test_suite="openCypher_M09",
        memgraph_params="",
        mandatory=False
-    )
+    ),
 ]

 results_folder = os.path.join("tck_engine", "results")
--- a/tests/stress/CMakeLists.txt
+++ b/tests/stress/CMakeLists.txt
@ -26,7 +26,7 @@ foreach(test_cpp ${test_type_cpps})
    set_target_properties(${target_name} PROPERTIES OUTPUT_NAME ${exec_name})

    # link libraries
-    target_link_libraries(${target_name} memgraph_lib)
+    target_link_libraries(${target_name} memgraph_lib kvstore_dummy_lib)

    # add target to dependencies
    add_dependencies(${all_targets_target} ${target_name})
--- a/tests/stress/apollo_runs.yaml
+++ b/tests/stress/apollo_runs.yaml
@ -6,6 +6,10 @@
    - ../../config # directory with config files
    - ../../build_release/tests/stress # stress client binaries

+- name: stress_properties_on_disk
+  commands: TIMEOUT=600 ./continuous_integration --properties-on-disk
+  infiles: *STRESS_INFILES
+
 - name: stress_large
  project: release
  commands: TIMEOUT=43200 ./continuous_integration --large-dataset
--- a/tests/stress/continuous_integration
+++ b/tests/stress/continuous_integration
@ -127,6 +127,7 @@ parser.add_argument("--config", default = os.path.join(CONFIG_DIR,
        "stress.conf"))
 parser.add_argument("--log-file", default = "")
 parser.add_argument("--durability-directory", default = "")
+parser.add_argument("--properties-on-disk", action = "store_true")
 parser.add_argument("--python", default = os.path.join(SCRIPT_DIR,
        "ve3", "bin", "python3"), type = str)
 parser.add_argument("--large-dataset", action = "store_const",
@ -148,6 +149,8 @@ if args.log_file:
    cmd += ["--log-file", args.log_file]
 if args.durability_directory:
    cmd += ["--durability-directory", args.durability_directory]
+if args.properties_on_disk:
+    cmd += ["--properties-on-disk", "id,x"]
 proc_mg = subprocess.Popen(cmd, cwd = cwd,
        env = {"MEMGRAPH_CONFIG": args.config})
 time.sleep(1.0)
--- a/tests/unit/CMakeLists.txt
+++ b/tests/unit/CMakeLists.txt
@ -20,223 +20,229 @@ function(add_unit_test test_cpp)
 endfunction(add_unit_test)

 add_unit_test(bolt_chunked_decoder_buffer.cpp)
-target_link_libraries(${test_prefix}bolt_chunked_decoder_buffer memgraph_lib)
+target_link_libraries(${test_prefix}bolt_chunked_decoder_buffer memgraph_lib kvstore_dummy_lib)

 add_unit_test(bolt_chunked_encoder_buffer.cpp)
-target_link_libraries(${test_prefix}bolt_chunked_encoder_buffer memgraph_lib)
+target_link_libraries(${test_prefix}bolt_chunked_encoder_buffer memgraph_lib kvstore_dummy_lib)

 add_unit_test(bolt_decoder.cpp)
-target_link_libraries(${test_prefix}bolt_decoder memgraph_lib)
+target_link_libraries(${test_prefix}bolt_decoder memgraph_lib kvstore_dummy_lib)

 add_unit_test(bolt_encoder.cpp)
-target_link_libraries(${test_prefix}bolt_encoder memgraph_lib)
+target_link_libraries(${test_prefix}bolt_encoder memgraph_lib kvstore_dummy_lib)

 add_unit_test(bolt_result_stream.cpp)
-target_link_libraries(${test_prefix}bolt_result_stream memgraph_lib)
+target_link_libraries(${test_prefix}bolt_result_stream memgraph_lib kvstore_dummy_lib)

 add_unit_test(bolt_session.cpp)
-target_link_libraries(${test_prefix}bolt_session memgraph_lib)
+target_link_libraries(${test_prefix}bolt_session memgraph_lib kvstore_dummy_lib)

 add_unit_test(communication_buffer.cpp)
-target_link_libraries(${test_prefix}communication_buffer memgraph_lib)
+target_link_libraries(${test_prefix}communication_buffer memgraph_lib kvstore_dummy_lib)

 add_unit_test(concurrent_id_mapper_distributed.cpp)
-target_link_libraries(${test_prefix}concurrent_id_mapper_distributed memgraph_lib)
+target_link_libraries(${test_prefix}concurrent_id_mapper_distributed memgraph_lib kvstore_dummy_lib)

 add_unit_test(concurrent_id_mapper_single_node.cpp)
-target_link_libraries(${test_prefix}concurrent_id_mapper_single_node memgraph_lib)
+target_link_libraries(${test_prefix}concurrent_id_mapper_single_node memgraph_lib kvstore_dummy_lib)

 add_unit_test(concurrent_map_access.cpp)
-target_link_libraries(${test_prefix}concurrent_map_access memgraph_lib)
+target_link_libraries(${test_prefix}concurrent_map_access memgraph_lib kvstore_dummy_lib)

 add_unit_test(concurrent_map.cpp)
-target_link_libraries(${test_prefix}concurrent_map memgraph_lib)
+target_link_libraries(${test_prefix}concurrent_map memgraph_lib kvstore_dummy_lib)

 add_unit_test(counters.cpp)
-target_link_libraries(${test_prefix}counters memgraph_lib)
+target_link_libraries(${test_prefix}counters memgraph_lib kvstore_dummy_lib)

 add_unit_test(cypher_main_visitor.cpp)
-target_link_libraries(${test_prefix}cypher_main_visitor memgraph_lib)
+target_link_libraries(${test_prefix}cypher_main_visitor memgraph_lib kvstore_dummy_lib)

 add_unit_test(database_key_index.cpp)
-target_link_libraries(${test_prefix}database_key_index memgraph_lib)
+target_link_libraries(${test_prefix}database_key_index memgraph_lib kvstore_dummy_lib)

 add_unit_test(database_label_property_index.cpp)
-target_link_libraries(${test_prefix}database_label_property_index memgraph_lib)
+target_link_libraries(${test_prefix}database_label_property_index memgraph_lib kvstore_dummy_lib)

 add_unit_test(database_master.cpp)
-target_link_libraries(${test_prefix}database_master memgraph_lib)
+target_link_libraries(${test_prefix}database_master memgraph_lib kvstore_dummy_lib)

 add_unit_test(database_transaction_timeout.cpp)
-target_link_libraries(${test_prefix}database_transaction_timeout memgraph_lib)
+target_link_libraries(${test_prefix}database_transaction_timeout memgraph_lib kvstore_dummy_lib)

 add_unit_test(datastructure_union_find.cpp)
-target_link_libraries(${test_prefix}datastructure_union_find memgraph_lib)
+target_link_libraries(${test_prefix}datastructure_union_find memgraph_lib kvstore_dummy_lib)

 add_unit_test(deferred_deleter.cpp)
-target_link_libraries(${test_prefix}deferred_deleter memgraph_lib)
+target_link_libraries(${test_prefix}deferred_deleter memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_bfs.cpp)
-target_link_libraries(${test_prefix}distributed_bfs memgraph_lib)
+target_link_libraries(${test_prefix}distributed_bfs memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_coordination.cpp)
-target_link_libraries(${test_prefix}distributed_coordination memgraph_lib)
+target_link_libraries(${test_prefix}distributed_coordination memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_data_exchange.cpp)
-target_link_libraries(${test_prefix}distributed_data_exchange memgraph_lib)
+target_link_libraries(${test_prefix}distributed_data_exchange memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_durability.cpp)
-target_link_libraries(${test_prefix}distributed_durability memgraph_lib)
+target_link_libraries(${test_prefix}distributed_durability memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_gc.cpp)
-target_link_libraries(${test_prefix}distributed_gc memgraph_lib)
+target_link_libraries(${test_prefix}distributed_gc memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_graph_db.cpp)
-target_link_libraries(${test_prefix}distributed_graph_db memgraph_lib)
+target_link_libraries(${test_prefix}distributed_graph_db memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_interpretation.cpp)
-target_link_libraries(${test_prefix}distributed_interpretation memgraph_lib)
+target_link_libraries(${test_prefix}distributed_interpretation memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_query_plan.cpp)
-target_link_libraries(${test_prefix}distributed_query_plan memgraph_lib)
+target_link_libraries(${test_prefix}distributed_query_plan memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_serialization.cpp)
-target_link_libraries(${test_prefix}distributed_serialization memgraph_lib)
+target_link_libraries(${test_prefix}distributed_serialization memgraph_lib kvstore_dummy_lib)

 add_unit_test(distributed_updates.cpp)
-target_link_libraries(${test_prefix}distributed_updates memgraph_lib)
+target_link_libraries(${test_prefix}distributed_updates memgraph_lib kvstore_dummy_lib)

 add_unit_test(durability.cpp)
-target_link_libraries(${test_prefix}durability memgraph_lib)
+target_link_libraries(${test_prefix}durability memgraph_lib kvstore_dummy_lib)

 add_unit_test(dynamic_bitset.cpp)
-target_link_libraries(${test_prefix}dynamic_bitset memgraph_lib)
+target_link_libraries(${test_prefix}dynamic_bitset memgraph_lib kvstore_dummy_lib)

 add_unit_test(gid.cpp)
-target_link_libraries(${test_prefix}gid memgraph_lib)
+target_link_libraries(${test_prefix}gid memgraph_lib kvstore_dummy_lib)

 add_unit_test(graph_db_accessor.cpp)
-target_link_libraries(${test_prefix}graph_db_accessor memgraph_lib)
+target_link_libraries(${test_prefix}graph_db_accessor memgraph_lib kvstore_dummy_lib)

 add_unit_test(graph_db_accessor_index_api.cpp)
-target_link_libraries(${test_prefix}graph_db_accessor_index_api memgraph_lib)
+target_link_libraries(${test_prefix}graph_db_accessor_index_api memgraph_lib kvstore_dummy_lib)

 add_unit_test(graph_db.cpp)
-target_link_libraries(${test_prefix}graph_db memgraph_lib)
+target_link_libraries(${test_prefix}graph_db memgraph_lib kvstore_dummy_lib)

 add_unit_test(interpreter.cpp)
-target_link_libraries(${test_prefix}interpreter memgraph_lib)
+target_link_libraries(${test_prefix}interpreter memgraph_lib kvstore_dummy_lib)
+
+add_unit_test(kvstore.cpp)
+target_link_libraries(${test_prefix}kvstore gtest gtest_main memgraph_lib kvstore_lib)

 add_unit_test(metrics.cpp)
-target_link_libraries(${test_prefix}metrics memgraph_lib)
+target_link_libraries(${test_prefix}metrics memgraph_lib kvstore_dummy_lib)

 add_unit_test(mvcc.cpp)
-target_link_libraries(${test_prefix}mvcc memgraph_lib)
+target_link_libraries(${test_prefix}mvcc memgraph_lib kvstore_dummy_lib)

 add_unit_test(mvcc_find.cpp)
-target_link_libraries(${test_prefix}mvcc_find memgraph_lib)
+target_link_libraries(${test_prefix}mvcc_find memgraph_lib kvstore_dummy_lib)

 add_unit_test(mvcc_gc.cpp)
-target_link_libraries(${test_prefix}mvcc_gc memgraph_lib)
+target_link_libraries(${test_prefix}mvcc_gc memgraph_lib kvstore_dummy_lib)

 add_unit_test(mvcc_one_transaction.cpp)
-target_link_libraries(${test_prefix}mvcc_one_transaction memgraph_lib)
+target_link_libraries(${test_prefix}mvcc_one_transaction memgraph_lib kvstore_dummy_lib)

 add_unit_test(mvcc_parallel_update.cpp)
-target_link_libraries(${test_prefix}mvcc_parallel_update memgraph_lib)
+target_link_libraries(${test_prefix}mvcc_parallel_update memgraph_lib kvstore_dummy_lib)

 add_unit_test(network_timeouts.cpp)
-target_link_libraries(${test_prefix}network_timeouts memgraph_lib)
+target_link_libraries(${test_prefix}network_timeouts memgraph_lib kvstore_dummy_lib)
+
+add_unit_test(pod_buffer.cpp)
+target_link_libraries(${test_prefix}pod_buffer memgraph_lib kvstore_dummy_lib)

 add_unit_test(property_value_store.cpp)
-target_link_libraries(${test_prefix}property_value_store memgraph_lib)
+target_link_libraries(${test_prefix}property_value_store memgraph_lib kvstore_lib)

 add_unit_test(query_cost_estimator.cpp)
-target_link_libraries(${test_prefix}query_cost_estimator memgraph_lib)
+target_link_libraries(${test_prefix}query_cost_estimator memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_expression_evaluator.cpp)
-target_link_libraries(${test_prefix}query_expression_evaluator memgraph_lib)
+target_link_libraries(${test_prefix}query_expression_evaluator memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_plan_accumulate_aggregate.cpp)
-target_link_libraries(${test_prefix}query_plan_accumulate_aggregate memgraph_lib)
+target_link_libraries(${test_prefix}query_plan_accumulate_aggregate memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_plan_bag_semantics.cpp)
-target_link_libraries(${test_prefix}query_plan_bag_semantics memgraph_lib)
+target_link_libraries(${test_prefix}query_plan_bag_semantics memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_plan_create_set_remove_delete.cpp)
-target_link_libraries(${test_prefix}query_plan_create_set_remove_delete memgraph_lib)
+target_link_libraries(${test_prefix}query_plan_create_set_remove_delete memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_plan_edge_cases.cpp)
-target_link_libraries(${test_prefix}query_plan_edge_cases memgraph_lib)
+target_link_libraries(${test_prefix}query_plan_edge_cases memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_plan_match_filter_return.cpp)
-target_link_libraries(${test_prefix}query_plan_match_filter_return memgraph_lib)
+target_link_libraries(${test_prefix}query_plan_match_filter_return memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_planner.cpp)
-target_link_libraries(${test_prefix}query_planner memgraph_lib)
+target_link_libraries(${test_prefix}query_planner memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_semantic.cpp)
-target_link_libraries(${test_prefix}query_semantic memgraph_lib)
+target_link_libraries(${test_prefix}query_semantic memgraph_lib kvstore_dummy_lib)

 add_unit_test(query_variable_start_planner.cpp)
-target_link_libraries(${test_prefix}query_variable_start_planner memgraph_lib)
+target_link_libraries(${test_prefix}query_variable_start_planner memgraph_lib kvstore_dummy_lib)

 add_unit_test(queue.cpp)
-target_link_libraries(${test_prefix}queue memgraph_lib)
+target_link_libraries(${test_prefix}queue memgraph_lib kvstore_dummy_lib)

 add_unit_test(raft.cpp)
-target_link_libraries(${test_prefix}raft memgraph_lib)
+target_link_libraries(${test_prefix}raft memgraph_lib kvstore_dummy_lib)

 add_unit_test(raft_storage.cpp)
-target_link_libraries(${test_prefix}raft_storage memgraph_lib)
+target_link_libraries(${test_prefix}raft_storage memgraph_lib kvstore_dummy_lib)

 add_unit_test(record_edge_vertex_accessor.cpp)
-target_link_libraries(${test_prefix}record_edge_vertex_accessor memgraph_lib)
+target_link_libraries(${test_prefix}record_edge_vertex_accessor memgraph_lib kvstore_dummy_lib)

 add_unit_test(rpc.cpp)
-target_link_libraries(${test_prefix}rpc memgraph_lib)
+target_link_libraries(${test_prefix}rpc memgraph_lib kvstore_dummy_lib)

 add_unit_test(rpc_worker_clients.cpp)
-target_link_libraries(${test_prefix}rpc_worker_clients memgraph_lib)
+target_link_libraries(${test_prefix}rpc_worker_clients memgraph_lib kvstore_dummy_lib)

 add_unit_test(serialization.cpp)
-target_link_libraries(${test_prefix}serialization memgraph_lib)
+target_link_libraries(${test_prefix}serialization memgraph_lib kvstore_dummy_lib)

 add_unit_test(skiplist_access.cpp)
-target_link_libraries(${test_prefix}skiplist_access memgraph_lib)
+target_link_libraries(${test_prefix}skiplist_access memgraph_lib kvstore_dummy_lib)

 add_unit_test(skiplist_gc.cpp)
-target_link_libraries(${test_prefix}skiplist_gc memgraph_lib)
+target_link_libraries(${test_prefix}skiplist_gc memgraph_lib kvstore_dummy_lib)

 add_unit_test(skiplist_position_and_count.cpp)
-target_link_libraries(${test_prefix}skiplist_position_and_count memgraph_lib)
+target_link_libraries(${test_prefix}skiplist_position_and_count memgraph_lib kvstore_dummy_lib)

 add_unit_test(skiplist_reverse_iteration.cpp)
-target_link_libraries(${test_prefix}skiplist_reverse_iteration memgraph_lib)
+target_link_libraries(${test_prefix}skiplist_reverse_iteration memgraph_lib kvstore_dummy_lib)

 add_unit_test(skiplist_suffix.cpp)
-target_link_libraries(${test_prefix}skiplist_suffix memgraph_lib)
+target_link_libraries(${test_prefix}skiplist_suffix memgraph_lib kvstore_dummy_lib)

 add_unit_test(state_delta.cpp)
-target_link_libraries(${test_prefix}state_delta memgraph_lib)
+target_link_libraries(${test_prefix}state_delta memgraph_lib kvstore_dummy_lib)

 add_unit_test(static_bitset.cpp)
-target_link_libraries(${test_prefix}static_bitset memgraph_lib)
+target_link_libraries(${test_prefix}static_bitset memgraph_lib kvstore_dummy_lib)

 add_unit_test(storage_address.cpp)
-target_link_libraries(${test_prefix}storage_address memgraph_lib)
+target_link_libraries(${test_prefix}storage_address memgraph_lib kvstore_dummy_lib)

 add_unit_test(stripped.cpp)
-target_link_libraries(${test_prefix}stripped memgraph_lib)
+target_link_libraries(${test_prefix}stripped memgraph_lib kvstore_dummy_lib)

 add_unit_test(transaction_engine_distributed.cpp)
-target_link_libraries(${test_prefix}transaction_engine_distributed memgraph_lib)
+target_link_libraries(${test_prefix}transaction_engine_distributed memgraph_lib kvstore_dummy_lib)

 add_unit_test(transaction_engine_single_node.cpp)
-target_link_libraries(${test_prefix}transaction_engine_single_node memgraph_lib)
+target_link_libraries(${test_prefix}transaction_engine_single_node memgraph_lib kvstore_dummy_lib)

 add_unit_test(typed_value.cpp)
-target_link_libraries(${test_prefix}typed_value memgraph_lib)
+target_link_libraries(${test_prefix}typed_value memgraph_lib kvstore_dummy_lib)

 # Test data structures

--- a/tests/unit/database_label_property_index.cpp
+++ b/tests/unit/database_label_property_index.cpp
@ -30,7 +30,7 @@ class LabelPropertyIndexComplexTest : public ::testing::Test {
    vertex = vlist->find(*t);
    ASSERT_NE(vertex, nullptr);
    vertex->labels_.push_back(label);
-    vertex->properties_.set(property, 0);
+    vertex->properties_.set(property, PropertyValue(0));

    EXPECT_EQ(index.Count(*key), 0);
  }
--- a/tests/unit/distributed_common.hpp
+++ b/tests/unit/distributed_common.hpp
@ -2,6 +2,7 @@
 #include <memory>
 #include <thread>

+#include <gflags/gflags.h>
 #include <gtest/gtest.h>

 #include "database/graph_db.hpp"
@ -11,6 +12,8 @@
 #include "storage/address_types.hpp"
 #include "transactions/engine_master.hpp"

+DECLARE_string(durability_directory);
+
 namespace fs = std::experimental::filesystem;

 class WorkerInThread {
@ -62,6 +65,9 @@ class DistributedGraphDbTest : public ::testing::Test {
      return config;
    };

+    // Flag needs to be updated due to props on disk storage.
+    FLAGS_durability_directory = tmp_dir_;
+
    for (int i = 0; i < kWorkerCount; ++i) {
      workers_.emplace_back(std::make_unique<WorkerInThread>(
          modify_config(worker_config(i + 1))));
--- a/tests/unit/durability.cpp
+++ b/tests/unit/durability.cpp
@ -25,6 +25,8 @@
 DECLARE_int32(wal_flush_interval_millis);
 DECLARE_int32(wal_rotate_deltas_count);

+DECLARE_string(durability_directory);
+
 namespace fs = std::experimental::filesystem;

 // Helper class for performing random CRUD ops on a database.
@ -309,6 +311,7 @@ class Durability : public ::testing::Test {
    snapshot_dir_ = durability_dir_ / durability::kSnapshotDir;
    wal_dir_ = durability_dir_ / durability::kWalDir;
    FLAGS_wal_rotate_deltas_count = 1000;
+    FLAGS_durability_directory = "MG_test_unit_durability";
    CleanDurability();
  }

@ -818,9 +821,8 @@ TEST_F(Durability, SequentialRecovery) {
    return threads;
  };

-  auto make_updates = [&run_updates, this](database::GraphDb &db,
-                                           bool snapshot_during,
-                                           bool snapshot_after) {
+  auto make_updates = [&run_updates, this](
+      database::GraphDb &db, bool snapshot_during, bool snapshot_after) {
    std::atomic<bool> keep_running{true};
    auto update_theads = run_updates(db, keep_running);
    std::this_thread::sleep_for(25ms);
--- a/tests/unit/kvstore.cpp
+++ b/tests/unit/kvstore.cpp
@ -0,0 +1,198 @@
+#include <glog/logging.h>
+#include <gtest/gtest.h>
+
+#include "durability/paths.hpp"
+#include "storage/kvstore.hpp"
+#include "utils/file.hpp"
+
+namespace fs = std::experimental::filesystem;
+
+class KVStore : public ::testing::Test {
+ protected:
+  virtual void SetUp() { utils::EnsureDir(test_folder_); }
+
+  virtual void TearDown() { fs::remove_all(test_folder_); }
+
+  fs::path test_folder_{fs::path("kvstore_test")};
+};
+
+TEST_F(KVStore, PutGet) {
+  storage::KVStore kvstore(test_folder_ / "PutGet");
+  ASSERT_TRUE(kvstore.Put("key", "value"));
+  ASSERT_EQ(kvstore.Get("key").value(), "value");
+}
+
+TEST_F(KVStore, PutGetDeleteGet) {
+  storage::KVStore kvstore(test_folder_ / "PutGetDeleteGet");
+  ASSERT_TRUE(kvstore.Put("key", "value"));
+  ASSERT_EQ(kvstore.Get("key").value(), "value");
+  ASSERT_TRUE(kvstore.Delete("key"));
+  ASSERT_FALSE(static_cast<bool>(kvstore.Get("key")));
+}
+
+TEST_F(KVStore, Durability) {
+  {
+    storage::KVStore kvstore(test_folder_ / "Durability");
+    ASSERT_TRUE(kvstore.Put("key", "value"));
+  }
+  {
+    storage::KVStore kvstore(test_folder_ / "Durability");
+    ASSERT_EQ(kvstore.Get("key").value(), "value");
+  }
+}
+
+TEST_F(KVStore, Size) {
+  storage::KVStore kvstore(test_folder_ / "Size");
+
+  ASSERT_TRUE(kvstore.Put("prefix_1", "jedan"));
+  ASSERT_TRUE(kvstore.Put("prefix_2", "dva"));
+  ASSERT_TRUE(kvstore.Put("prefix_3", "tri"));
+  ASSERT_TRUE(kvstore.Put("prefix_4", "cetiri"));
+  ASSERT_TRUE(kvstore.Put("prefix_5", "pet"));
+
+  EXPECT_EQ(kvstore.Size("a"), 0);
+  EXPECT_EQ(kvstore.Size(), 5);
+  EXPECT_EQ(kvstore.Size("prefix_"), 5);
+  EXPECT_EQ(kvstore.Size("prefix_1"), 1);
+
+  ASSERT_TRUE(kvstore.Put("predmetak_1", "jedan"));
+  ASSERT_TRUE(kvstore.Put("predmetak_2", "dva"));
+  ASSERT_TRUE(kvstore.Put("predmetak_3", "tri"));
+  ASSERT_TRUE(kvstore.Put("predmetak_4", "cetiri"));
+
+  EXPECT_EQ(kvstore.Size("a"), 0);
+  EXPECT_EQ(kvstore.Size(), 9);
+  EXPECT_EQ(kvstore.Size("prefix_"), 5);
+  EXPECT_EQ(kvstore.Size("predmetak_"), 4);
+  EXPECT_EQ(kvstore.Size("p"), 9);
+  EXPECT_EQ(kvstore.Size("pre"), 9);
+  EXPECT_EQ(kvstore.Size("pred"), 4);
+  EXPECT_EQ(kvstore.Size("pref"), 5);
+  EXPECT_EQ(kvstore.Size("prex"), 0);
+}
+
+TEST_F(KVStore, DeletePrefix) {
+  storage::KVStore kvstore(test_folder_ / "DeletePrefix");
+
+  ASSERT_TRUE(kvstore.Put("prefix_1", "jedan"));
+  ASSERT_TRUE(kvstore.Put("prefix_2", "dva"));
+  ASSERT_TRUE(kvstore.Put("prefix_3", "tri"));
+  ASSERT_TRUE(kvstore.Put("prefix_4", "cetiri"));
+  ASSERT_TRUE(kvstore.Put("prefix_5", "pet"));
+
+  EXPECT_EQ(kvstore.Size(), 5);
+
+  ASSERT_TRUE(kvstore.DeletePrefix("prefix_5"));
+
+  EXPECT_EQ(kvstore.Size(), 4);
+  EXPECT_EQ(kvstore.Size("prefix_"), 4);
+  EXPECT_EQ(kvstore.Size("prefix_1"), 1);
+
+  ASSERT_TRUE(kvstore.Put("predmetak_1", "jedan"));
+  ASSERT_TRUE(kvstore.Put("predmetak_2", "dva"));
+  ASSERT_TRUE(kvstore.Put("predmetak_3", "tri"));
+  ASSERT_TRUE(kvstore.Put("predmetak_4", "cetiri"));
+
+  EXPECT_EQ(kvstore.Size(), 8);
+
+  ASSERT_TRUE(kvstore.DeletePrefix("predmetak_1"));
+  EXPECT_EQ(kvstore.Size(), 7);
+
+  ASSERT_TRUE(kvstore.DeletePrefix("prefix_"));
+  EXPECT_EQ(kvstore.Size(), 3);
+
+  ASSERT_TRUE(kvstore.DeletePrefix("predmetak_"));
+  EXPECT_EQ(kvstore.Size(), 0);
+
+  ASSERT_TRUE(kvstore.DeletePrefix("whatever"));
+  EXPECT_EQ(kvstore.Size(), 0);
+  EXPECT_EQ(kvstore.Size("any_prefix"), 0);
+}
+
+TEST_F(KVStore, Iterator) {
+  storage::KVStore kvstore(test_folder_ / "Iterator");
+
+  for (int i = 1; i <= 4; ++i)
+    ASSERT_TRUE(
+        kvstore.Put("key" + std::to_string(i), "value" + std::to_string(i)));
+
+  auto it = kvstore.begin();
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "key1");
+  EXPECT_EQ((*it).second, "value1");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ((*it).first, "key2");
+  EXPECT_EQ(it->second, "value2");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "key3");
+  EXPECT_EQ((*it).second, "value3");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ((*it).first, "key4");
+  EXPECT_EQ(it->second, "value4");
+
+  ++it;
+  ASSERT_FALSE(it.IsValid());
+}
+
+TEST_F(KVStore, IteratorPrefix) {
+  storage::KVStore kvstore(test_folder_ / "Iterator");
+
+  ASSERT_TRUE(kvstore.Put("a_1", "value1"));
+  ASSERT_TRUE(kvstore.Put("a_2", "value2"));
+
+  ASSERT_TRUE(kvstore.Put("aa_1", "value1"));
+  ASSERT_TRUE(kvstore.Put("aa_2", "value2"));
+
+  ASSERT_TRUE(kvstore.Put("b_1", "value1"));
+  ASSERT_TRUE(kvstore.Put("b_2", "value2"));
+
+  auto it = kvstore.begin("a");
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "a_1");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "a_2");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "aa_1");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "aa_2");
+
+  ++it;
+  ASSERT_FALSE(it.IsValid());
+
+  it = kvstore.begin("aa_");
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "aa_1");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "aa_2");
+
+  ++it;
+  ASSERT_FALSE(it.IsValid());
+
+  it = kvstore.begin("b_");
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "b_1");
+
+  ++it;
+  ASSERT_TRUE(it.IsValid());
+  EXPECT_EQ(it->first, "b_2");
+
+  ++it;
+  ASSERT_FALSE(it.IsValid());
+
+  it = kvstore.begin("unexisting_prefix");
+  ASSERT_FALSE(it.IsValid());
+}
--- a/tests/unit/pod_buffer.cpp
+++ b/tests/unit/pod_buffer.cpp
@ -0,0 +1,60 @@
+#include <glog/logging.h>
+#include <gtest/gtest.h>
+
+#include "storage/pod_buffer.hpp"
+
+class PODBufferTest : public ::testing::Test {
+ protected:
+  storage::PODBuffer buffer_;
+
+  void SetUp() override { buffer_ = storage::PODBuffer(""); }
+
+  void Write(const uint8_t *data, size_t len) { buffer_.Write(data, len); }
+
+  bool Read(uint8_t *data, size_t len) { return buffer_.Read(data, len); }
+};
+
+TEST_F(PODBufferTest, ReadEmpty) {
+  uint8_t data[10];
+  ASSERT_TRUE(Read(data, 0));
+  for (int i = 1; i <= 5; ++i) ASSERT_FALSE(Read(data, i));
+}
+
+TEST_F(PODBufferTest, ReadNonEmpty) {
+  uint8_t input_data[10];
+  uint8_t output_data[10];
+
+  for (int i = 0; i < 10; ++i) input_data[i] = i;
+
+  Write(input_data, 10);
+  ASSERT_TRUE(Read(output_data, 10));
+
+  for (int i = 0; i < 10; ++i) ASSERT_EQ(output_data[i], i);
+
+  ASSERT_FALSE(Read(output_data, 1));
+}
+
+TEST_F(PODBufferTest, WriteRead) {
+  uint8_t input_data[10];
+  uint8_t output_data[10];
+
+  for (int i = 0; i < 10; ++i) input_data[i] = i;
+
+  Write(input_data, 10);
+  ASSERT_TRUE(Read(output_data, 5));
+
+  for (int i = 0; i < 5; ++i) ASSERT_EQ(output_data[i], i);
+
+  ASSERT_TRUE(Read(output_data, 5));
+
+  for (int i = 0; i < 5; ++i) ASSERT_EQ(output_data[i], i + 5);
+
+  ASSERT_FALSE(Read(output_data, 1));
+
+  Write(input_data + 5, 5);
+  ASSERT_TRUE(Read(output_data, 5));
+
+  for (int i = 0; i < 5; ++i) ASSERT_EQ(output_data[i], i + 5);
+
+  ASSERT_FALSE(Read(output_data, 1));
+}
--- a/tests/unit/property_value_store.cpp
+++ b/tests/unit/property_value_store.cpp
@ -1,5 +1,6 @@
 #include <vector>

+#include "gflags/gflags.h"
 #include "gtest/gtest.h"

 #include "storage/property_value.hpp"
@ -8,10 +9,17 @@
 using std::string;
 using Location = storage::Location;

+DECLARE_string(properties_on_disk);
+
 class PropertyValueStoreTest : public ::testing::Test {
 protected:
  PropertyValueStore props_;

+  void SetUp() override {
+    // we need this to test the copy constructor
+    FLAGS_properties_on_disk = "not empty";
+  }
+
  void Set(int key, Location location, PropertyValue value) {
    props_.set(storage::Property(key, location), value);
  }
@ -24,12 +32,15 @@ class PropertyValueStoreTest : public ::testing::Test {
    return props_.erase(storage::Property(key, location));
  }

+  auto Begin() { return props_.begin(); }
+
+  auto End() { return props_.end(); }
+
  void TearDown() override { props_.clear(); }
 };

-TEST_F(PropertyValueStoreTest, At) {
+TEST_F(PropertyValueStoreTest, AtMemory) {
  std::string some_string = "something";
-  std::string other_string = "something completely different";

  EXPECT_EQ(PropertyValue(At(0, Location::Memory)).type(),
            PropertyValue::Type::Null);
@ -38,10 +49,17 @@ TEST_F(PropertyValueStoreTest, At) {
            some_string);
  Set(120, Location::Memory, 42);
  EXPECT_EQ(PropertyValue(At(120, Location::Memory)).Value<int64_t>(), 42);
+}

-  Set(100, Location::Disk, other_string);
-  EXPECT_EQ(PropertyValue(At(100, Location::Disk)).Value<string>(),
-            other_string);
+TEST_F(PropertyValueStoreTest, AtDisk) {
+  std::string some_string = "something";
+
+  EXPECT_EQ(PropertyValue(At(0, Location::Disk)).type(),
+            PropertyValue::Type::Null);
+  Set(0, Location::Disk, some_string);
+  EXPECT_EQ(PropertyValue(At(0, Location::Disk)).Value<string>(), some_string);
+  Set(120, Location::Disk, 42);
+  EXPECT_EQ(PropertyValue(At(120, Location::Disk)).Value<int64_t>(), 42);
 }

 TEST_F(PropertyValueStoreTest, AtNull) {
@ -69,7 +87,7 @@ TEST_F(PropertyValueStoreTest, SetNull) {
  EXPECT_EQ(0, props_.size());
 }

-TEST_F(PropertyValueStoreTest, Remove) {
+TEST_F(PropertyValueStoreTest, RemoveMemory) {
  // set some props
  Set(11, Location::Memory, "a");
  Set(30, Location::Memory, "b");
@ -85,48 +103,59 @@ TEST_F(PropertyValueStoreTest, Remove) {
  EXPECT_EQ(props_.size(), 0);
  EXPECT_EQ(At(30, Location::Memory).type(), PropertyValue::Type::Null);

-  EXPECT_EQ(Erase(1000, Location::Memory), 0);
-
-  props_.clear();
-
-  Set(110, Location::Disk, "a");
-  EXPECT_NE(At(110, Location::Disk).type(), PropertyValue::Type::Null);
-  EXPECT_EQ(props_.size(), 1);
-
-  Erase(110, Location::Disk);
-  EXPECT_EQ(props_.size(), 0);
-  EXPECT_EQ(At(110, Location::Disk).type(), PropertyValue::Type::Null);
-  EXPECT_EQ(Erase(1000, Location::Disk), 0);
+  EXPECT_EQ(Erase(1000, Location::Memory), 1);
 }

-TEST_F(PropertyValueStoreTest, Clear) {
+TEST_F(PropertyValueStoreTest, RemoveDisk) {
+  // set some props
+  Set(11, Location::Disk, "a");
+  Set(30, Location::Disk, "b");
+  EXPECT_NE(At(11, Location::Disk).type(), PropertyValue::Type::Null);
+  EXPECT_NE(At(30, Location::Disk).type(), PropertyValue::Type::Null);
+  EXPECT_EQ(props_.size(), 2);
+
+  Erase(11, Location::Disk);
+  EXPECT_EQ(props_.size(), 1);
+  EXPECT_EQ(At(11, Location::Disk).type(), PropertyValue::Type::Null);
+
+  EXPECT_EQ(Erase(30, Location::Disk), 1);
+  EXPECT_EQ(props_.size(), 0);
+  EXPECT_EQ(At(30, Location::Disk).type(), PropertyValue::Type::Null);
+
+  EXPECT_EQ(Erase(1000, Location::Disk), 1);
+}
+
+TEST_F(PropertyValueStoreTest, ClearMemory) {
  EXPECT_EQ(props_.size(), 0);
  Set(11, Location::Memory, "a");
  Set(30, Location::Memory, "b");
  EXPECT_EQ(props_.size(), 2);
-  props_.clear();
-  EXPECT_EQ(props_.size(), 0);
-
-  Set(11, Location::Disk, "a");
-  EXPECT_EQ(props_.size(), 1);
-  props_.clear();
-  EXPECT_EQ(props_.size(), 0);
 }

-TEST_F(PropertyValueStoreTest, Replace) {
+TEST_F(PropertyValueStoreTest, ClearDisk) {
+  EXPECT_EQ(props_.size(), 0);
+  Set(11, Location::Disk, "a");
+  Set(30, Location::Disk, "b");
+  EXPECT_EQ(props_.size(), 2);
+}
+
+TEST_F(PropertyValueStoreTest, ReplaceMemory) {
  Set(10, Location::Memory, 42);
  EXPECT_EQ(At(10, Location::Memory).Value<int64_t>(), 42);
  Set(10, Location::Memory, 0.25f);
  EXPECT_EQ(At(10, Location::Memory).type(), PropertyValue::Type::Double);
  EXPECT_FLOAT_EQ(At(10, Location::Memory).Value<double>(), 0.25);
-
-  Set(100, Location::Disk, "some text");
-  EXPECT_EQ(At(100, Location::Disk).Value<string>(), "some text");
-  Set(100, Location::Disk, "some other text");
-  EXPECT_EQ(At(100, Location::Disk).Value<string>(), "some other text");
 }

-TEST_F(PropertyValueStoreTest, Size) {
+TEST_F(PropertyValueStoreTest, ReplaceDisk) {
+  Set(10, Location::Disk, 42);
+  EXPECT_EQ(At(10, Location::Disk).Value<int64_t>(), 42);
+  Set(10, Location::Disk, 0.25f);
+  EXPECT_EQ(At(10, Location::Disk).type(), PropertyValue::Type::Double);
+  EXPECT_FLOAT_EQ(At(10, Location::Disk).Value<double>(), 0.25);
+}
+
+TEST_F(PropertyValueStoreTest, SizeMemory) {
  EXPECT_EQ(props_.size(), 0);

  Set(0, Location::Memory, "something");
@ -145,14 +174,49 @@ TEST_F(PropertyValueStoreTest, Size) {
  EXPECT_EQ(props_.size(), 101);
  Erase(1, Location::Memory);
  EXPECT_EQ(props_.size(), 100);
+}

-  Set(101, Location::Disk, "dalmatians");
+TEST_F(PropertyValueStoreTest, SizeDisk) {
+  EXPECT_EQ(props_.size(), 0);
+
+  Set(0, Location::Disk, "something");
+  EXPECT_EQ(props_.size(), 1);
+  Set(0, Location::Disk, true);
+  EXPECT_EQ(props_.size(), 1);
+  Set(1, Location::Disk, true);
+  EXPECT_EQ(props_.size(), 2);
+
+  for (int i = 0; i < 100; ++i) Set(i + 20, Location::Disk, true);
+  EXPECT_EQ(props_.size(), 102);
+
+  Erase(0, Location::Disk);
  EXPECT_EQ(props_.size(), 101);
-  Erase(101, Location::Disk);
+  Erase(0, Location::Disk);
+  EXPECT_EQ(props_.size(), 101);
+  Erase(1, Location::Disk);
  EXPECT_EQ(props_.size(), 100);
 }

-TEST_F(PropertyValueStoreTest, InsertRetrieveList) {
+TEST_F(PropertyValueStoreTest, Size) {
+  EXPECT_EQ(props_.size(), 0);
+
+  for (int i = 0; i < 100; ++i) Set(i, Location::Disk, true);
+  EXPECT_EQ(props_.size(), 100);
+
+  for (int i = 0; i < 200; ++i) Set(i + 100, Location::Memory, true);
+  EXPECT_EQ(props_.size(), 300);
+
+  Erase(0, Location::Disk);
+  EXPECT_EQ(props_.size(), 299);
+  Erase(99, Location::Disk);
+  EXPECT_EQ(props_.size(), 298);
+  Erase(100, Location::Memory);
+  EXPECT_EQ(props_.size(), 297);
+  Erase(299, Location::Memory);
+  EXPECT_EQ(props_.size(), 296);
+}
+
+TEST_F(PropertyValueStoreTest, InsertRetrieveListMemory) {
  Set(0, Location::Memory, std::vector<PropertyValue>{1, true, 2.5, "something",
                                                      PropertyValue::Null});
  auto p = At(0, Location::Memory);
@ -171,6 +235,25 @@ TEST_F(PropertyValueStoreTest, InsertRetrieveList) {
  EXPECT_EQ(l[4].type(), PropertyValue::Type::Null);
 }

+TEST_F(PropertyValueStoreTest, InsertRetrieveListDisk) {
+  Set(0, Location::Disk, std::vector<PropertyValue>{1, true, 2.5, "something",
+                                                    PropertyValue::Null});
+  auto p = At(0, Location::Disk);
+
+  EXPECT_EQ(p.type(), PropertyValue::Type::List);
+  auto l = p.Value<std::vector<PropertyValue>>();
+  EXPECT_EQ(l.size(), 5);
+  EXPECT_EQ(l[0].type(), PropertyValue::Type::Int);
+  EXPECT_EQ(l[0].Value<int64_t>(), 1);
+  EXPECT_EQ(l[1].type(), PropertyValue::Type::Bool);
+  EXPECT_EQ(l[1].Value<bool>(), true);
+  EXPECT_EQ(l[2].type(), PropertyValue::Type::Double);
+  EXPECT_EQ(l[2].Value<double>(), 2.5);
+  EXPECT_EQ(l[3].type(), PropertyValue::Type::String);
+  EXPECT_EQ(l[3].Value<std::string>(), "something");
+  EXPECT_EQ(l[4].type(), PropertyValue::Type::Null);
+}
+
 TEST_F(PropertyValueStoreTest, InsertRetrieveMap) {
  Set(0, Location::Memory, std::map<std::string, PropertyValue>{
                               {"a", 1}, {"b", true}, {"c", "something"}});
@ -189,3 +272,89 @@ TEST_F(PropertyValueStoreTest, InsertRetrieveMap) {
  EXPECT_EQ(get("c").type(), PropertyValue::Type::String);
  EXPECT_EQ(get("c").Value<std::string>(), "something");
 }
+
+TEST_F(PropertyValueStoreTest, InsertRetrieveMapDisk) {
+  Set(0, Location::Disk, std::map<std::string, PropertyValue>{
+                             {"a", 1}, {"b", true}, {"c", "something"}});
+
+  auto p = At(0, Location::Disk);
+  EXPECT_EQ(p.type(), PropertyValue::Type::Map);
+  auto m = p.Value<std::map<std::string, PropertyValue>>();
+  EXPECT_EQ(m.size(), 3);
+  auto get = [&m](const std::string &prop_name) {
+    return m.find(prop_name)->second;
+  };
+  EXPECT_EQ(get("a").type(), PropertyValue::Type::Int);
+  EXPECT_EQ(get("a").Value<int64_t>(), 1);
+  EXPECT_EQ(get("b").type(), PropertyValue::Type::Bool);
+  EXPECT_EQ(get("b").Value<bool>(), true);
+  EXPECT_EQ(get("c").type(), PropertyValue::Type::String);
+  EXPECT_EQ(get("c").Value<std::string>(), "something");
+}
+
+TEST_F(PropertyValueStoreTest, Iterator) {
+  Set(0, Location::Memory, "a");
+  Set(1, Location::Memory, 1);
+  Set(2, Location::Disk, "b");
+  Set(3, Location::Disk, 2);
+
+  auto it = Begin();
+  ASSERT_TRUE(it != End());
+  EXPECT_EQ(it->first.Id(), 0);
+  EXPECT_EQ((*it).second.Value<std::string>(), "a");
+
+  ++it;
+  ASSERT_TRUE(it != End());
+  EXPECT_EQ((*it).first.Id(), 1);
+  EXPECT_EQ(it->second.Value<int64_t>(), 1);
+
+  ++it;
+  ASSERT_TRUE(it != End());
+  EXPECT_EQ(it->first.Id(), 2);
+  EXPECT_EQ((*it).second.Value<std::string>(), "b");
+
+  ++it;
+  ASSERT_TRUE(it != End());
+  EXPECT_EQ((*it).first.Id(), 3);
+  EXPECT_EQ(it->second.Value<int64_t>(), 2);
+
+  ++it;
+  ASSERT_TRUE(it == End());
+}
+
+TEST_F(PropertyValueStoreTest, CopyConstructor) {
+  PropertyValueStore props;
+  for (int i = 1; i <= 3; ++i)
+    props.set(storage::Property(i, Location::Memory),
+              "mem_" + std::to_string(i));
+  for (int i = 4; i <= 5; ++i)
+    props.set(storage::Property(i, Location::Disk),
+              "disk_" + std::to_string(i));
+
+  PropertyValueStore new_props = props;
+  for (int i = 1; i <= 3; ++i)
+    EXPECT_EQ(
+        new_props.at(storage::Property(i, Location::Memory)).Value<string>(),
+        "mem_" + std::to_string(i));
+  for (int i = 4; i <= 5; ++i)
+    EXPECT_EQ(
+        new_props.at(storage::Property(i, Location::Disk)).Value<string>(),
+        "disk_" + std::to_string(i));
+
+  props.set(storage::Property(1, Location::Memory), "mem_1_update");
+  EXPECT_EQ(
+      new_props.at(storage::Property(1, Location::Memory)).Value<string>(),
+      "mem_1");
+
+  new_props.set(storage::Property(2, Location::Memory), "mem_2_update");
+  EXPECT_EQ(props.at(storage::Property(2, Location::Memory)).Value<string>(),
+            "mem_2");
+
+  props.set(storage::Property(4, Location::Disk), "disk_4_update");
+  EXPECT_EQ(new_props.at(storage::Property(4, Location::Disk)).Value<string>(),
+            "disk_4");
+
+  new_props.set(storage::Property(5, Location::Disk), "disk_5_update");
+  EXPECT_EQ(props.at(storage::Property(5, Location::Disk)).Value<string>(),
+            "disk_5");
+}
--- a/tools/src/CMakeLists.txt
+++ b/tools/src/CMakeLists.txt
@ -1,10 +1,10 @@
 # CSV Import Tool Target
 add_executable(mg_import_csv mg_import_csv/main.cpp)
-target_link_libraries(mg_import_csv memgraph_lib)
+target_link_libraries(mg_import_csv memgraph_lib kvstore_dummy_lib)

 # StatsD Target
 add_executable(mg_statsd mg_statsd/main.cpp)
-target_link_libraries(mg_statsd memgraph_lib)
+target_link_libraries(mg_statsd memgraph_lib kvstore_dummy_lib)

 # Strip the executable in release build.
 string(TOLOWER ${CMAKE_BUILD_TYPE} lower_build_type)
--- a/tools/src/mg_import_csv/main.cpp
+++ b/tools/src/mg_import_csv/main.cpp
@ -446,7 +446,7 @@ static const char *usage =
    "Create a Memgraph recovery snapshot file from CSV.\n";

 // Used only to get the value from memgraph's configuration files.
-DEFINE_HIDDEN_string(durability_directory, "", "Durability directory");
+DECLARE_string(durability_directory);

 std::string GetOutputPath() {
  // If we have the 'out' flag, use that.
--- a/tools/tests/CMakeLists.txt
+++ b/tools/tests/CMakeLists.txt
@ -1,10 +1,10 @@
 include_directories(SYSTEM ${GTEST_INCLUDE_DIR})

 add_executable(mg_recovery_check mg_recovery_check.cpp)
-target_link_libraries(mg_recovery_check memgraph_lib gtest gtest_main)
+target_link_libraries(mg_recovery_check memgraph_lib gtest gtest_main kvstore_dummy_lib)

 add_executable(mg_statsd_client statsd/mg_statsd_client.cpp)
-target_link_libraries(mg_statsd_client memgraph_lib)
+target_link_libraries(mg_statsd_client memgraph_lib kvstore_dummy_lib)

 # Copy CSV data to CMake build dir
 configure_file(csv/comment_nodes.csv csv/comment_nodes.csv COPYONLY)