go-libp2p-core/peerstore/peerstore.go

256 lines
9.8 KiB
Go
Raw Normal View History

// Package peerstore provides types and interfaces for local storage of address information,
// metadata, and public key material about libp2p peers.
package peerstore
import (
"context"
"errors"
"io"
"math"
"time"
ic "github.com/libp2p/go-libp2p-core/crypto"
Signed envelopes & routing records (#73) * add SignedEnvelope type * use struct for SignedEnvelope instead of exposing protobuf directly * doc comments for envelopes * tests for SignedEnvelopes * add helpers to make routing records for Host * fix doc comment * go fmt * add method to peerstore to retrieve signed routing records * update to match spec changes * just use nanoseconds * use proto3 & rename fields to match spec changes * use proto3 for routing records * make envelope fields private & validate on unmarshal * use buffer pool for envelope signatures * tests for RoutingState * go fmt * rename Equals -> Equal, add some comments * use test helpers * get rid of unsigned RoutingState struct, only expose SignedRoutingState * rm batching SignedRoutingStates accessor in peerstore the datastore peerstore implementation doesn't support batched reads, so it's no more efficient to get a bunch of states at once than it is to call SignedRoutingState multiple times. * whitespace * expose struct fields & remove accessors * use camelCase in protos for consistency * use multiformats uvarint for length-prefixes * remove payloadType check when unmarhaling * rm stray ref to golang/protobuf * define CertifiedAddrBook to avoid breaking API change * add events for updated addresses and routing state * remove SignedRoutingStateFromHost helper moving this to go-libp2p * add routing state records, extend peerstore API * fix: rebuild protos with new gogofaster generator * filter private addrs from signed routing records * envelope: use byte slices from pool; adjust interface. * move envelope to record package. * move protobuf files; adjust imports everywhere. * rename RoutingStateRecord -> PeerRecord also removes embedded reference to Envelope from the record, as that was confusing. as a result, the CertifiedAddrBook now accepts/returns record.SignedEnvelope instead of a specialized type. * hoist Seq from PeerRecord to SignedEnvelope * test that PeerRecords can't be signed by wrong key * commit go.sum * add Seq field to envelope signature * fix proto_path in Makefile * fix import ordering * comments for PeerRecord proto message also removes the seq field from PeerMessage proto, since it was moved to the SignedEnvelope * use Record type for envelope payloads * rename SignedEnvelope -> Envelope, unmarshal payload in ConsumeEnvelope * return buffer to pool before early return * doc comments * rename CertifiedAddrBook methods, update comments * cache unmarshalled Record payload inside Envelope * doc comments * store reflect.Type when registering Record * Revert "return buffer to pool before early return" 8d8da386f26482e06dc21989a6b5ade69f0a46d9 misread this - unsigned will be nil if there's an error, so it was right the way it was * use a DefaultRecord for unregistered PayloadTypes instead of returning an error if we don't have a registered Record for a given PayloadType, we can have a catch-all DefaultRecord type that just preserves the original payload as a []byte * cleanup DefaultRecord code a bit - removes unused error return from blankRecordForPayloadType - just references instead of copying in DefaultRecord.UnmarshalRecord I figure this is likely safe, since we'll be unmarshalling from the payload of an Envelope, which shouldn't get altered after it's created. * use explicit payloadType in MakeEnvelopeWithRecord * Revert DefaultRecord commits ae3bc7bdfb657c232229229706854a56effca80b a26c845a766b45ceabd87c17c0801d191650f0d4 * doc comments * move Seq field back to PeerRecord * make diffs optional in EvtLocalAddressesUpdated * more envelope tests * replace MakeEnvelope with record.Seal also: - add Domain and Codec fields to Record interface * fix import * add interface check * rename ProcessPeerRecord -> ConsumePeerRecord also, adds bool `accepted` return value * rename event field, add doc comment * peer record protobuf: fix field casing. * record protobuf: add docs and fix casing. * cleanup: group imports. * nit: split test/utils.go => test/{addrs,errors}.go. Co-authored-by: Raúl Kripalani <raul.kripalani@gmail.com>
2020-02-11 03:53:24 +08:00
"github.com/libp2p/go-libp2p-core/peer"
"github.com/libp2p/go-libp2p-core/record"
ma "github.com/multiformats/go-multiaddr"
)
var ErrNotFound = errors.New("item not found")
var (
// AddressTTL is the expiration time of addresses.
AddressTTL = time.Hour
2020-03-04 01:16:38 +08:00
// TempAddrTTL is the ttl used for a short lived address.
TempAddrTTL = time.Minute * 2
// ProviderAddrTTL is the TTL of an address we've received from a provider.
// This is also a temporary address, but lasts longer. After this expires,
// the records we return will require an extra lookup.
ProviderAddrTTL = time.Minute * 30
// RecentlyConnectedAddrTTL is used when we recently connected to a peer.
// It means that we are reasonably certain of the peer's address.
RecentlyConnectedAddrTTL = time.Minute * 30
// OwnObservedAddrTTL is used for our own external addresses observed by peers.
OwnObservedAddrTTL = time.Minute * 30
)
// Permanent TTLs (distinct so we can distinguish between them, constant as they
// are, in fact, permanent)
const (
// PermanentAddrTTL is the ttl for a "permanent address" (e.g. bootstrap nodes).
PermanentAddrTTL = math.MaxInt64 - iota
// ConnectedAddrTTL is the ttl used for the addresses of a peer to whom
// we're connected directly. This is basically permanent, as we will
// clear them + re-add under a TempAddrTTL after disconnecting.
ConnectedAddrTTL
)
// Peerstore provides a threadsafe store of Peer related
// information.
type Peerstore interface {
io.Closer
AddrBook
KeyBook
PeerMetadata
Metrics
ProtoBook
// PeerInfo returns a peer.PeerInfo struct for given peer.ID.
// This is a small slice of the information Peerstore has on
// that peer, useful to other services.
PeerInfo(peer.ID) peer.AddrInfo
// Peers returns all of the peer IDs stored across all inner stores.
Peers() peer.IDSlice
}
// PeerMetadata can handle values of any type. Serializing values is
// up to the implementation. Dynamic type introspection may not be
// supported, in which case explicitly enlisting types in the
// serializer may be required.
//
// Refer to the docs of the underlying implementation for more
// information.
type PeerMetadata interface {
// Get / Put is a simple registry for other peer-related key/value pairs.
// If we find something we use often, it should become its own set of
// methods. This is a last resort.
Get(p peer.ID, key string) (interface{}, error)
Put(p peer.ID, key string, val interface{}) error
// RemovePeer removes all values stored for a peer.
RemovePeer(peer.ID)
}
// AddrBook holds the multiaddrs of peers.
type AddrBook interface {
// AddAddr calls AddAddrs(p, []ma.Multiaddr{addr}, ttl)
AddAddr(p peer.ID, addr ma.Multiaddr, ttl time.Duration)
// AddAddrs gives this AddrBook addresses to use, with a given ttl
// (time-to-live), after which the address is no longer valid.
// If the manager has a longer TTL, the operation is a no-op for that address
AddAddrs(p peer.ID, addrs []ma.Multiaddr, ttl time.Duration)
// SetAddr calls mgr.SetAddrs(p, addr, ttl)
SetAddr(p peer.ID, addr ma.Multiaddr, ttl time.Duration)
// SetAddrs sets the ttl on addresses. This clears any TTL there previously.
// This is used when we receive the best estimate of the validity of an address.
SetAddrs(p peer.ID, addrs []ma.Multiaddr, ttl time.Duration)
// UpdateAddrs updates the addresses associated with the given peer that have
// the given oldTTL to have the given newTTL.
UpdateAddrs(p peer.ID, oldTTL time.Duration, newTTL time.Duration)
Signed envelopes & routing records (#73) * add SignedEnvelope type * use struct for SignedEnvelope instead of exposing protobuf directly * doc comments for envelopes * tests for SignedEnvelopes * add helpers to make routing records for Host * fix doc comment * go fmt * add method to peerstore to retrieve signed routing records * update to match spec changes * just use nanoseconds * use proto3 & rename fields to match spec changes * use proto3 for routing records * make envelope fields private & validate on unmarshal * use buffer pool for envelope signatures * tests for RoutingState * go fmt * rename Equals -> Equal, add some comments * use test helpers * get rid of unsigned RoutingState struct, only expose SignedRoutingState * rm batching SignedRoutingStates accessor in peerstore the datastore peerstore implementation doesn't support batched reads, so it's no more efficient to get a bunch of states at once than it is to call SignedRoutingState multiple times. * whitespace * expose struct fields & remove accessors * use camelCase in protos for consistency * use multiformats uvarint for length-prefixes * remove payloadType check when unmarhaling * rm stray ref to golang/protobuf * define CertifiedAddrBook to avoid breaking API change * add events for updated addresses and routing state * remove SignedRoutingStateFromHost helper moving this to go-libp2p * add routing state records, extend peerstore API * fix: rebuild protos with new gogofaster generator * filter private addrs from signed routing records * envelope: use byte slices from pool; adjust interface. * move envelope to record package. * move protobuf files; adjust imports everywhere. * rename RoutingStateRecord -> PeerRecord also removes embedded reference to Envelope from the record, as that was confusing. as a result, the CertifiedAddrBook now accepts/returns record.SignedEnvelope instead of a specialized type. * hoist Seq from PeerRecord to SignedEnvelope * test that PeerRecords can't be signed by wrong key * commit go.sum * add Seq field to envelope signature * fix proto_path in Makefile * fix import ordering * comments for PeerRecord proto message also removes the seq field from PeerMessage proto, since it was moved to the SignedEnvelope * use Record type for envelope payloads * rename SignedEnvelope -> Envelope, unmarshal payload in ConsumeEnvelope * return buffer to pool before early return * doc comments * rename CertifiedAddrBook methods, update comments * cache unmarshalled Record payload inside Envelope * doc comments * store reflect.Type when registering Record * Revert "return buffer to pool before early return" 8d8da386f26482e06dc21989a6b5ade69f0a46d9 misread this - unsigned will be nil if there's an error, so it was right the way it was * use a DefaultRecord for unregistered PayloadTypes instead of returning an error if we don't have a registered Record for a given PayloadType, we can have a catch-all DefaultRecord type that just preserves the original payload as a []byte * cleanup DefaultRecord code a bit - removes unused error return from blankRecordForPayloadType - just references instead of copying in DefaultRecord.UnmarshalRecord I figure this is likely safe, since we'll be unmarshalling from the payload of an Envelope, which shouldn't get altered after it's created. * use explicit payloadType in MakeEnvelopeWithRecord * Revert DefaultRecord commits ae3bc7bdfb657c232229229706854a56effca80b a26c845a766b45ceabd87c17c0801d191650f0d4 * doc comments * move Seq field back to PeerRecord * make diffs optional in EvtLocalAddressesUpdated * more envelope tests * replace MakeEnvelope with record.Seal also: - add Domain and Codec fields to Record interface * fix import * add interface check * rename ProcessPeerRecord -> ConsumePeerRecord also, adds bool `accepted` return value * rename event field, add doc comment * peer record protobuf: fix field casing. * record protobuf: add docs and fix casing. * cleanup: group imports. * nit: split test/utils.go => test/{addrs,errors}.go. Co-authored-by: Raúl Kripalani <raul.kripalani@gmail.com>
2020-02-11 03:53:24 +08:00
// Addrs returns all known (and valid) addresses for a given peer.
Addrs(p peer.ID) []ma.Multiaddr
// AddrStream returns a channel that gets all addresses for a given
// peer sent on it. If new addresses are added after the call is made
// they will be sent along through the channel as well.
AddrStream(context.Context, peer.ID) <-chan ma.Multiaddr
2020-03-04 01:16:38 +08:00
// ClearAddresses removes all previously stored addresses.
ClearAddrs(p peer.ID)
2020-03-04 01:16:38 +08:00
// PeersWithAddrs returns all of the peer IDs stored in the AddrBook.
PeersWithAddrs() peer.IDSlice
}
Signed envelopes & routing records (#73) * add SignedEnvelope type * use struct for SignedEnvelope instead of exposing protobuf directly * doc comments for envelopes * tests for SignedEnvelopes * add helpers to make routing records for Host * fix doc comment * go fmt * add method to peerstore to retrieve signed routing records * update to match spec changes * just use nanoseconds * use proto3 & rename fields to match spec changes * use proto3 for routing records * make envelope fields private & validate on unmarshal * use buffer pool for envelope signatures * tests for RoutingState * go fmt * rename Equals -> Equal, add some comments * use test helpers * get rid of unsigned RoutingState struct, only expose SignedRoutingState * rm batching SignedRoutingStates accessor in peerstore the datastore peerstore implementation doesn't support batched reads, so it's no more efficient to get a bunch of states at once than it is to call SignedRoutingState multiple times. * whitespace * expose struct fields & remove accessors * use camelCase in protos for consistency * use multiformats uvarint for length-prefixes * remove payloadType check when unmarhaling * rm stray ref to golang/protobuf * define CertifiedAddrBook to avoid breaking API change * add events for updated addresses and routing state * remove SignedRoutingStateFromHost helper moving this to go-libp2p * add routing state records, extend peerstore API * fix: rebuild protos with new gogofaster generator * filter private addrs from signed routing records * envelope: use byte slices from pool; adjust interface. * move envelope to record package. * move protobuf files; adjust imports everywhere. * rename RoutingStateRecord -> PeerRecord also removes embedded reference to Envelope from the record, as that was confusing. as a result, the CertifiedAddrBook now accepts/returns record.SignedEnvelope instead of a specialized type. * hoist Seq from PeerRecord to SignedEnvelope * test that PeerRecords can't be signed by wrong key * commit go.sum * add Seq field to envelope signature * fix proto_path in Makefile * fix import ordering * comments for PeerRecord proto message also removes the seq field from PeerMessage proto, since it was moved to the SignedEnvelope * use Record type for envelope payloads * rename SignedEnvelope -> Envelope, unmarshal payload in ConsumeEnvelope * return buffer to pool before early return * doc comments * rename CertifiedAddrBook methods, update comments * cache unmarshalled Record payload inside Envelope * doc comments * store reflect.Type when registering Record * Revert "return buffer to pool before early return" 8d8da386f26482e06dc21989a6b5ade69f0a46d9 misread this - unsigned will be nil if there's an error, so it was right the way it was * use a DefaultRecord for unregistered PayloadTypes instead of returning an error if we don't have a registered Record for a given PayloadType, we can have a catch-all DefaultRecord type that just preserves the original payload as a []byte * cleanup DefaultRecord code a bit - removes unused error return from blankRecordForPayloadType - just references instead of copying in DefaultRecord.UnmarshalRecord I figure this is likely safe, since we'll be unmarshalling from the payload of an Envelope, which shouldn't get altered after it's created. * use explicit payloadType in MakeEnvelopeWithRecord * Revert DefaultRecord commits ae3bc7bdfb657c232229229706854a56effca80b a26c845a766b45ceabd87c17c0801d191650f0d4 * doc comments * move Seq field back to PeerRecord * make diffs optional in EvtLocalAddressesUpdated * more envelope tests * replace MakeEnvelope with record.Seal also: - add Domain and Codec fields to Record interface * fix import * add interface check * rename ProcessPeerRecord -> ConsumePeerRecord also, adds bool `accepted` return value * rename event field, add doc comment * peer record protobuf: fix field casing. * record protobuf: add docs and fix casing. * cleanup: group imports. * nit: split test/utils.go => test/{addrs,errors}.go. Co-authored-by: Raúl Kripalani <raul.kripalani@gmail.com>
2020-02-11 03:53:24 +08:00
// CertifiedAddrBook manages "self-certified" addresses for remote peers.
// Self-certified addresses are contained in peer.PeerRecords
// which are wrapped in a record.Envelope and signed by the peer
// to whom they belong.
//
// Certified addresses (CA) are generally more secure than uncertified
// addresses (UA). Consequently, CAs beat and displace UAs. When the
// peerstore learns CAs for a peer, it will reject UAs for the same peer
// (as long as the former haven't expired).
// Furthermore, peer records act like sequenced snapshots of CAs. Therefore,
// processing a peer record that's newer than the last one seen overwrites
// all addresses with the incoming ones.
//
// This interface is most useful when combined with AddrBook.
// To test whether a given AddrBook / Peerstore implementation supports
// certified addresses, callers should use the GetCertifiedAddrBook helper or
// type-assert on the CertifiedAddrBook interface:
//
// if cab, ok := aPeerstore.(CertifiedAddrBook); ok {
// cab.ConsumePeerRecord(signedPeerRecord, aTTL)
// }
//
type CertifiedAddrBook interface {
// ConsumePeerRecord adds addresses from a signed peer.PeerRecord (contained in
// a record.Envelope), which will expire after the given TTL.
//
// The 'accepted' return value indicates that the record was successfully processed
// and integrated into the CertifiedAddrBook state. If 'accepted' is false but no
// error is returned, it means that the record was ignored, most likely because
// a newer record exists for the same peer.
//
// Signed records added via this method will be stored without
// alteration as long as the address TTLs remain valid. The Envelopes
// containing the PeerRecords can be retrieved by calling GetPeerRecord(peerID).
//
// If the signed PeerRecord belongs to a peer that already has certified
// addresses in the CertifiedAddrBook, the new addresses will replace the
2020-03-04 01:16:38 +08:00
// older ones, if the new record has a higher sequence number than the
Signed envelopes & routing records (#73) * add SignedEnvelope type * use struct for SignedEnvelope instead of exposing protobuf directly * doc comments for envelopes * tests for SignedEnvelopes * add helpers to make routing records for Host * fix doc comment * go fmt * add method to peerstore to retrieve signed routing records * update to match spec changes * just use nanoseconds * use proto3 & rename fields to match spec changes * use proto3 for routing records * make envelope fields private & validate on unmarshal * use buffer pool for envelope signatures * tests for RoutingState * go fmt * rename Equals -> Equal, add some comments * use test helpers * get rid of unsigned RoutingState struct, only expose SignedRoutingState * rm batching SignedRoutingStates accessor in peerstore the datastore peerstore implementation doesn't support batched reads, so it's no more efficient to get a bunch of states at once than it is to call SignedRoutingState multiple times. * whitespace * expose struct fields & remove accessors * use camelCase in protos for consistency * use multiformats uvarint for length-prefixes * remove payloadType check when unmarhaling * rm stray ref to golang/protobuf * define CertifiedAddrBook to avoid breaking API change * add events for updated addresses and routing state * remove SignedRoutingStateFromHost helper moving this to go-libp2p * add routing state records, extend peerstore API * fix: rebuild protos with new gogofaster generator * filter private addrs from signed routing records * envelope: use byte slices from pool; adjust interface. * move envelope to record package. * move protobuf files; adjust imports everywhere. * rename RoutingStateRecord -> PeerRecord also removes embedded reference to Envelope from the record, as that was confusing. as a result, the CertifiedAddrBook now accepts/returns record.SignedEnvelope instead of a specialized type. * hoist Seq from PeerRecord to SignedEnvelope * test that PeerRecords can't be signed by wrong key * commit go.sum * add Seq field to envelope signature * fix proto_path in Makefile * fix import ordering * comments for PeerRecord proto message also removes the seq field from PeerMessage proto, since it was moved to the SignedEnvelope * use Record type for envelope payloads * rename SignedEnvelope -> Envelope, unmarshal payload in ConsumeEnvelope * return buffer to pool before early return * doc comments * rename CertifiedAddrBook methods, update comments * cache unmarshalled Record payload inside Envelope * doc comments * store reflect.Type when registering Record * Revert "return buffer to pool before early return" 8d8da386f26482e06dc21989a6b5ade69f0a46d9 misread this - unsigned will be nil if there's an error, so it was right the way it was * use a DefaultRecord for unregistered PayloadTypes instead of returning an error if we don't have a registered Record for a given PayloadType, we can have a catch-all DefaultRecord type that just preserves the original payload as a []byte * cleanup DefaultRecord code a bit - removes unused error return from blankRecordForPayloadType - just references instead of copying in DefaultRecord.UnmarshalRecord I figure this is likely safe, since we'll be unmarshalling from the payload of an Envelope, which shouldn't get altered after it's created. * use explicit payloadType in MakeEnvelopeWithRecord * Revert DefaultRecord commits ae3bc7bdfb657c232229229706854a56effca80b a26c845a766b45ceabd87c17c0801d191650f0d4 * doc comments * move Seq field back to PeerRecord * make diffs optional in EvtLocalAddressesUpdated * more envelope tests * replace MakeEnvelope with record.Seal also: - add Domain and Codec fields to Record interface * fix import * add interface check * rename ProcessPeerRecord -> ConsumePeerRecord also, adds bool `accepted` return value * rename event field, add doc comment * peer record protobuf: fix field casing. * record protobuf: add docs and fix casing. * cleanup: group imports. * nit: split test/utils.go => test/{addrs,errors}.go. Co-authored-by: Raúl Kripalani <raul.kripalani@gmail.com>
2020-02-11 03:53:24 +08:00
// existing record. Attempting to add a peer record with a
// sequence number that's <= an existing record for the same peer will not
// result in an error, but the record will be ignored, and the 'accepted'
// bool return value will be false.
//
// If the CertifiedAddrBook is also an AddrBook (which is most likely the case),
// adding certified addresses for a peer will *replace* any
// existing non-certified addresses for that peer, and only the certified
// addresses will be returned from AddrBook.Addrs thereafter.
//
// Likewise, once certified addresses have been added for a given peer,
// any non-certified addresses added via AddrBook.AddAddrs or
// AddrBook.SetAddrs will be ignored. AddrBook.SetAddrs may still be used
// to update the TTL of certified addresses that have previously been
// added via ConsumePeerRecord.
ConsumePeerRecord(s *record.Envelope, ttl time.Duration) (accepted bool, err error)
// GetPeerRecord returns a Envelope containing a PeerRecord for the
// given peer id, if one exists.
// Returns nil if no signed PeerRecord exists for the peer.
GetPeerRecord(p peer.ID) *record.Envelope
}
// GetCertifiedAddrBook is a helper to "upcast" an AddrBook to a
// CertifiedAddrBook by using type assertion. If the given AddrBook
// is also a CertifiedAddrBook, it will be returned, and the ok return
// value will be true. Returns (nil, false) if the AddrBook is not a
// CertifiedAddrBook.
//
// Note that since Peerstore embeds the AddrBook interface, you can also
// call GetCertifiedAddrBook(myPeerstore).
func GetCertifiedAddrBook(ab AddrBook) (cab CertifiedAddrBook, ok bool) {
cab, ok = ab.(CertifiedAddrBook)
return cab, ok
}
// KeyBook tracks the keys of Peers.
type KeyBook interface {
// PubKey stores the public key of a peer.
PubKey(peer.ID) ic.PubKey
// AddPubKey stores the public key of a peer.
AddPubKey(peer.ID, ic.PubKey) error
// PrivKey returns the private key of a peer, if known. Generally this might only be our own
// private key, see
// https://discuss.libp2p.io/t/what-is-the-purpose-of-having-map-peer-id-privatekey-in-peerstore/74.
PrivKey(peer.ID) ic.PrivKey
// AddPrivKey stores the private key of a peer.
AddPrivKey(peer.ID, ic.PrivKey) error
// PeersWithKeys returns all the peer IDs stored in the KeyBook.
PeersWithKeys() peer.IDSlice
// RemovePeer removes all keys associated with a peer.
RemovePeer(peer.ID)
}
// Metrics tracks metrics across a set of peers.
type Metrics interface {
// RecordLatency records a new latency measurement
RecordLatency(peer.ID, time.Duration)
// LatencyEWMA returns an exponentially-weighted moving avg.
// of all measurements of a peer's latency.
LatencyEWMA(peer.ID) time.Duration
// RemovePeer removes all metrics stored for a peer.
RemovePeer(peer.ID)
}
2020-03-04 01:16:38 +08:00
// ProtoBook tracks the protocols supported by peers.
type ProtoBook interface {
GetProtocols(peer.ID) ([]string, error)
AddProtocols(peer.ID, ...string) error
SetProtocols(peer.ID, ...string) error
RemoveProtocols(peer.ID, ...string) error
// SupportsProtocols returns the set of protocols the peer supports from among the given protocols.
// If the returned error is not nil, the result is indeterminate.
SupportsProtocols(peer.ID, ...string) ([]string, error)
2020-05-11 15:30:05 +08:00
// FirstSupportedProtocol returns the first protocol that the peer supports among the given protocols.
// If the peer does not support any of the given protocols, this function will return an empty string and a nil error.
// If the returned error is not nil, the result is indeterminate.
2020-05-11 15:30:05 +08:00
FirstSupportedProtocol(peer.ID, ...string) (string, error)
// RemovePeer removes all protocols associated with a peer.
RemovePeer(peer.ID)
}