Commit Graph

82 Commits

Author SHA1 Message Date
Steven Allen
2fb6d7a48f fix multiple TTL bugs
The first fix independently extends the address expiration time and the address
TTL:

By example:

* We have an address with a TTL of 4s that will expire in 1s.
* We update it with a TTL of 3s.

Before this change:

* We end up with an address with a TTL of 3s that will expire in 3s.

After this change:

* We end up with an address with a TTL of 4s that will expire in 3s.

---

The second fix prevents the in-memory addressbook from announcing existing
addresses every time their TTLs get updated.

---

The third fix correctly updates TTLs for existing addresses in the on-disk
addressbook.

This fixes https://github.com/libp2p/go-libp2p-identify/issues/2
2019-07-24 17:04:37 -07:00
Raúl Kripalani
ad0faef7c7
make RemoveProtocols take the write lock. (#90) 2019-07-10 15:45:49 +01:00
Raúl Kripalani
3d3a10dd54
protobook: support removing protocols. (#84) 2019-06-19 23:53:07 +01:00
Steven Allen
0b5b389640 dep: switch to core (#80)
* dep: switch to core

Specifically, remove go-libp2p-{crypto,peer} deps.

* catch a few more usages of deprecated packages.
2019-05-31 14:51:16 +01:00
vyzo
7fac6b6d98 use segments in ds protobook instead of a lock array 2019-05-18 11:42:47 +03:00
vyzo
07ee3fb062 make lock methods private 2019-05-17 23:41:05 +03:00
vyzo
afca355218 index the peer ID strings directly to avoid copying the byte slice 2019-05-17 23:36:46 +03:00
vyzo
3140aaa2c3 implement protobook for pstoreds 2019-05-17 13:13:18 +03:00
Raúl Kripalani
662be3b0a4 fix error handling: return on err. 2019-04-16 22:11:14 +01:00
Raúl Kripalani
2c5bc31271 migrate to multiformats/go-base32. 2019-02-28 18:34:51 +00:00
Raúl Kripalani
4806c742b5 pstoreds: remove dirty flag when deleting from db. 2019-02-21 19:31:19 +00:00
Raúl Kripalani
135471f291 add io.Closer interface and implementation to Peerstore. 2019-02-19 11:45:32 +00:00
Raúl Kripalani
4e7d772db3 pstoreds: remove async flushing on read: now sync. 2019-02-18 16:35:48 +00:00
Raúl Kripalani
25f53120b8 gc: drain async flush queue with grace period before closing. 2019-02-15 18:44:05 +00:00
Raúl Kripalani
d7e9f78273 increment WaitGroups before spawning goroutines. 2019-02-15 18:43:34 +00:00
Raúl Kripalani
86f1626c51 fix tests; run lookahead after initial delay. 2019-02-06 16:05:02 +00:00
Raúl Kripalani
25e4df45aa change default purge interval to 2 hours. 2019-02-06 14:43:10 +00:00
Raúl Kripalani
e16cac9f5e make addrsRecord.Clean() private. 2019-02-06 14:40:51 +00:00
Raúl Kripalani
3966498df4 improve docs; cyclic batch arg; validations. 2019-02-06 14:36:51 +00:00
Raúl Kripalani
fec786e9ef pstore ds: make gc lookahead window optional. 2019-02-06 09:35:10 +00:00
Raúl Kripalani
9da98d7fcf factor out cyclic batch into file; and gc into type. 2019-02-05 17:53:50 +00:00
Raúl Kripalani
20a0dc649d adopt new ds.Write interface. 2019-02-05 16:53:23 +00:00
Raúl Kripalani
d88042c422 improve code readability. 2019-02-05 16:52:27 +00:00
Raúl Kripalani
f0eb25b3c5 pstore ds: enforce order in gc queries. 2019-02-05 15:07:30 +00:00
Raúl Kripalani
fd9c9287e2 return error when committing a closed cyclic batch. 2018-12-04 11:47:44 +00:00
Raúl Kripalani
53d45e9406 make gc operations run on cyclic batches. 2018-12-03 17:40:01 +00:00
Raúl Kripalani
a306a6eadb add a benchmark for lookahead cycles. 2018-12-03 15:54:41 +00:00
Raúl Kripalani
e43a2fc519 drop the need for a transactional datastore; introduce buffer pool. 2018-12-03 15:43:37 +00:00
Raúl Kripalani
8fa350be35 fix raciness in init logic; dedicated goroutines for GC and flushing. 2018-11-30 17:32:51 +00:00
Raúl Kripalani
19eea773fa rename addrsRecord.Refresh() => addrsRecord.Clean(). 2018-11-30 16:56:37 +00:00
Raúl Kripalani
943429ff4b Revert "use a Pool for address record objects." 2018-11-30 16:19:57 +00:00
Raúl Kripalani
c3c24b227a rename GCPruneInterval to GCPurgeInterval for consistency. 2018-11-28 01:29:09 +00:00
Raúl Kripalani
bbff6b7522 improve test coverage of new GC routine. 2018-11-28 01:08:25 +00:00
Raúl Kripalani
03a40a855a minor cleanup. 2018-11-28 00:56:51 +00:00
Raúl Kripalani
3a85498ae8 add tests for new GC routines. 2018-11-28 00:13:13 +00:00
Raúl Kripalani
35444fbc6a implement two-tiered lookahead-based GC procedure.
The GC procedure consists of two tiers: lookahead and purge. The
lookahead tier runs less frequently (default 12 hours) than the purge
tier (default 5 minutes).

A lookahead round traverses the entire store and picks entries that
need to be visited in the current lookahead window, pinning those
entries in a separate region in the KV store, indexed by the next visit
timestamp.

A purge round iterates over the lookahead window only, and refreshes
the entries that require a visit. It removes them from the lookahead
region unless the entry requires another visit within the current
window.
2018-11-28 00:13:02 +00:00
Raúl Kripalani
ef8e8ff359 use a Pool for address record objects. 2018-11-15 18:48:40 +00:00
Raúl Kripalani
390e8c3f44 add Keys() function to private cache interface for completeness. 2018-11-15 18:39:38 +00:00
Raúl Kripalani
bd80609ef6 adjust GC interval in tests. 2018-11-15 13:55:53 +00:00
Raúl Kripalani
e57f21a66c Merge branch 'master' into feat/pstore-ds-gc 2018-11-15 13:44:13 +00:00
Raúl Kripalani
4c43736fe9 pstoreds: tighten up gc-based peerstore.
Introduces custom types in protobuf to serde directly into multiaddrs
and peer IDs. Simplify purge by ordering addrs by expiry. In general,
getting this readier for merge.
2018-11-15 13:32:08 +00:00
Raúl Kripalani
c423e9e997 pstoreds: migrate from strict TTL to GC-based expiry.
Each entry in the address book KV store now represents a full peer
with all its addresses. Expiries and TTLs are kept inline. Records are
serialised with Protobuf.

Housekeeping is performed on retrieve, and via a periodic GC routine.
The architecture mimics a write-through cache, although not strictly.
2018-11-13 16:57:51 +00:00
Adrian Lanzafame
c04ff2d27b
add keybook benchmarks
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-07 15:48:31 +10:00
Adrian Lanzafame
ee43bc1951
prevent timeout by un-nesting tb.Runs
License: MIT
Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
2018-11-07 15:48:31 +10:00
Steven Allen
588ab36ce8 update to match new ExtractPublicKey API
This is a "more correct" API and we might as well fix this before it becomes
used all over the place. The new API returns ErrNoPublicKey when there is no
inlined key.

This *also* fixes a bug where the datastore-backed keystore would panic if
`ExtractPublicKey` returned `nil, nil`.
2018-10-16 11:31:32 +01:00
Raúl Kripalani
4e2e2de65f ds key encoding: b58 => b32 (no padding). 2018-10-04 10:51:57 +01:00
Raúl Kripalani
61b5355499 run tests against multiple datastores.
leveldb disabled for now, until TTL shim is implemented.
2018-10-04 10:50:45 +01:00
Raúl Kripalani
3e80eb1873 review comments. 2018-09-28 18:47:41 +01:00
Raúl Kripalani
dd3cb20421 add explicit import aliases. 2018-09-28 14:18:05 +01:00
Raúl Kripalani
1b72b920ed datastore-backed impls of KeyBook and PeerMetadata. 2018-09-28 14:04:52 +01:00