Commit Graph

476 Commits

Author SHA1 Message Date
Raúl Kripalani
fec786e9ef pstore ds: make gc lookahead window optional. 2019-02-06 09:35:10 +00:00
Raúl Kripalani
9da98d7fcf factor out cyclic batch into file; and gc into type. 2019-02-05 17:53:50 +00:00
Raúl Kripalani
20a0dc649d adopt new ds.Write interface. 2019-02-05 16:53:23 +00:00
Raúl Kripalani
d88042c422 improve code readability. 2019-02-05 16:52:27 +00:00
Raúl Kripalani
0baddf577e Merge branch 'master' into feat/pstore-ds-gc 2019-02-05 16:15:11 +00:00
Raúl Kripalani
1e2fd43772
Merge pull request #52 from raulk/master
Add unit test to verify AddAddr doesn't shorten TTL
2019-02-05 15:09:02 +00:00
Raúl Kripalani
f0eb25b3c5 pstore ds: enforce order in gc queries. 2019-02-05 15:07:30 +00:00
Raúl Kripalani
6959c64874 add unit test to verify AddAddr doesn't shorten TTL. 2019-01-23 13:13:11 +00:00
Hector Sanjuan
99108065a0
Merge pull request #50 from libp2p/update/manet
gx publish 2.0.13
2018-12-18 18:34:47 +01:00
Hector Sanjuan
23d213ec8c gx publish 2.0.13 2018-12-18 18:22:44 +01:00
Steven Allen
0ef521ea9d gx publish 2.0.12 2018-12-07 14:34:08 -08:00
Steven Allen
3bee9fde6a
Merge pull request #49 from libp2p/fix/disable-inline-pid-test
disable inline-peer id test
2018-12-07 13:51:25 -08:00
Steven Allen
f32bc1c26d disable inline-peer id test 2018-12-07 13:47:26 -08:00
Raúl Kripalani
fd9c9287e2 return error when committing a closed cyclic batch. 2018-12-04 11:47:44 +00:00
Raúl Kripalani
53d45e9406 make gc operations run on cyclic batches. 2018-12-03 17:40:01 +00:00
Raúl Kripalani
a306a6eadb add a benchmark for lookahead cycles. 2018-12-03 15:54:41 +00:00
Raúl Kripalani
e43a2fc519 drop the need for a transactional datastore; introduce buffer pool. 2018-12-03 15:43:37 +00:00
Raúl Kripalani
8fa350be35 fix raciness in init logic; dedicated goroutines for GC and flushing. 2018-11-30 17:32:51 +00:00
Raúl Kripalani
19eea773fa rename addrsRecord.Refresh() => addrsRecord.Clean(). 2018-11-30 16:56:37 +00:00
Raúl Kripalani
943429ff4b Revert "use a Pool for address record objects." 2018-11-30 16:19:57 +00:00
Raúl Kripalani
dd2a07ea4a Revert "gx: update golang-lru dependency."
This reverts commit c7107ffb52.
2018-11-30 16:19:57 +00:00
Raúl Kripalani
fd57c55d87 document custom gogo types; reorg code. 2018-11-30 16:19:57 +00:00
Raúl Kripalani
c3c24b227a rename GCPruneInterval to GCPurgeInterval for consistency. 2018-11-28 01:29:09 +00:00
Raúl Kripalani
bbff6b7522 improve test coverage of new GC routine. 2018-11-28 01:08:25 +00:00
Raúl Kripalani
ffe9586203 codecov: ignore protobuf go sources. 2018-11-28 00:57:08 +00:00
Raúl Kripalani
03a40a855a minor cleanup. 2018-11-28 00:56:51 +00:00
Raúl Kripalani
3a85498ae8 add tests for new GC routines. 2018-11-28 00:13:13 +00:00
Raúl Kripalani
35444fbc6a implement two-tiered lookahead-based GC procedure.
The GC procedure consists of two tiers: lookahead and purge. The
lookahead tier runs less frequently (default 12 hours) than the purge
tier (default 5 minutes).

A lookahead round traverses the entire store and picks entries that
need to be visited in the current lookahead window, pinning those
entries in a separate region in the KV store, indexed by the next visit
timestamp.

A purge round iterates over the lookahead window only, and refreshes
the entries that require a visit. It removes them from the lookahead
region unless the entry requires another visit within the current
window.
2018-11-28 00:13:02 +00:00
Raúl Kripalani
87b51603b1
Merge pull request #48 from anshukla/master
README: Update contributing guideline linkrot.
2018-11-26 12:11:22 +00:00
Ansh Shukla
eb26a53a64 README: Update contributing guideline linkrot. 2018-11-25 12:45:28 -08:00
Raúl Kripalani
c7107ffb52 gx: update golang-lru dependency. 2018-11-16 11:11:00 +00:00
Raúl Kripalani
ef8e8ff359 use a Pool for address record objects. 2018-11-15 18:48:40 +00:00
Raúl Kripalani
390e8c3f44 add Keys() function to private cache interface for completeness. 2018-11-15 18:39:38 +00:00
Raúl Kripalani
bd80609ef6 adjust GC interval in tests. 2018-11-15 13:55:53 +00:00
Raúl Kripalani
e57f21a66c Merge branch 'master' into feat/pstore-ds-gc 2018-11-15 13:44:13 +00:00
Raúl Kripalani
22f8aa142c fix makefile for protobuf. 2018-11-15 13:40:48 +00:00
Raúl Kripalani
4c43736fe9 pstoreds: tighten up gc-based peerstore.
Introduces custom types in protobuf to serde directly into multiaddrs
and peer IDs. Simplify purge by ordering addrs by expiry. In general,
getting this readier for merge.
2018-11-15 13:32:08 +00:00
Raúl Kripalani
0c6cf4a799
Merge pull request #43 from lanzafame/feat/more-benchs
Deterministic benchmark order; Keybook interface benchmarks
2018-11-14 19:09:15 +00:00
Raúl Kripalani
c423e9e997 pstoreds: migrate from strict TTL to GC-based expiry.
Each entry in the address book KV store now represents a full peer
with all its addresses. Expiries and TTLs are kept inline. Records are
serialised with Protobuf.

Housekeeping is performed on retrieve, and via a periodic GC routine.
The architecture mimics a write-through cache, although not strictly.
2018-11-13 16:57:51 +00:00
Hannah Howard
3a5fba8af2
Merge pull request #46 from libp2p/gx/update-kque0y
gx publish 2.0.11
2018-11-09 15:47:08 -08:00
hannahhoward
129aa29181 gx publish 2.0.11 2018-11-09 15:31:38 -08:00
Steven Allen
00679cccdb
Merge pull request #45 from zhizouxiao/fix/peerinfo-unmarshal
PeerInfo UnMarshal Error #393
2018-11-08 20:54:45 -08:00
wzp
de5683cb85 add marshal unittest #393 2018-11-09 11:13:06 +08:00
Eli Wang
ab8c6cbfd1
bug fix: UnmarshalJSON should take a pointer 2018-11-08 10:40:39 +08:00
wzp
e9ca112fea PeerInfo UnMarshal Error #393 2018-11-08 10:12:34 +08:00
Adrian Lanzafame
c04ff2d27b
add keybook benchmarks
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-07 15:48:31 +10:00
Adrian Lanzafame
308d998d76
sort suite so benchmarks always run in the same order
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-07 15:48:31 +10:00
Adrian Lanzafame
ee43bc1951
prevent timeout by un-nesting tb.Runs
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-07 15:48:31 +10:00
Steven Allen
545f62f160 gx publish 2.0.10 2018-11-02 21:10:55 -07:00
Steven Allen
d351b9ad3b gx publish 2.0.9 2018-11-02 18:06:06 -07:00