mirror of
https://github.com/libp2p/go-libp2p-resource-manager.git
synced 2025-02-05 01:00:19 +08:00
docs: describe how the limiter is configured, and how limits are scaled (#59)
* docs: describe how the limiter is configured, and how limits are scaled * Apply suggestions from code review Co-authored-by: Marco Munizaga <marco@marcopolo.io> Co-authored-by: Marco Munizaga <marco@marcopolo.io>
This commit is contained in:
parent
9320db8806
commit
df257cee0c
108
README.md
108
README.md
@ -281,64 +281,78 @@ aid understanding of applicable limits. Note that the (wrapped) error
|
|||||||
implements `net.Error` and is marked as temporary, so that the
|
implements `net.Error` and is marked as temporary, so that the
|
||||||
programmer can handle by backoff retry.
|
programmer can handle by backoff retry.
|
||||||
|
|
||||||
### Default Limits
|
## Usage
|
||||||
|
|
||||||
The provided default static limiters apply the following limits, where
|
This package provides a limiter implementation that applies fixed limits:
|
||||||
memoryCap is provided by the programmer, either as a fixed number (in
|
```go
|
||||||
bytes) or a fraction of total system memory:
|
limiter := NewFixedLimiter(limits)
|
||||||
```
|
```
|
||||||
DefaultSystemBaseLimit:
|
The `limits` allows fine-grained control of resource usage on all scopes.
|
||||||
StreamsInbound: 4096,
|
|
||||||
StreamsOutbound: 16384,
|
|
||||||
ConnsInbound: 256,
|
|
||||||
ConnsOutbound: 512,
|
|
||||||
FD: 512,
|
|
||||||
|
|
||||||
DefaultTransientBaseLimit:
|
### Scaling Limits
|
||||||
StreamsInbound: 128,
|
|
||||||
StreamsOutbound: 512,
|
|
||||||
ConnsInbound: 32,
|
|
||||||
ConnsOutbound: 128,
|
|
||||||
FD: 128,
|
|
||||||
|
|
||||||
DefaultProtocolBaseLimit:
|
When building software that is supposed to run on many different kind of machines,
|
||||||
StreamsInbound: 1024,
|
with various memory and CPU configurations, it is desireable to have limits that
|
||||||
StreamsOutbound: 4096,
|
scale with the size of the machine.
|
||||||
|
|
||||||
DefaultServiceBaseLimit:
|
This is done using the `ScalingLimitConfig`. For every scope, this configuration
|
||||||
StreamsInbound: 2048,
|
struct defines the absolutely bare minimum limits, and an (optional) increase of
|
||||||
StreamsOutbound: 8192,
|
these limits, which will be applied on nodes that have sufficient memory.
|
||||||
|
|
||||||
system:
|
A `ScalingLimitConfig` can be converted into a `LimitConfig` (which can then be
|
||||||
Memory: memoryCap
|
used to initialize a fixed limiter as shown above) by calling the `Scale` method.
|
||||||
BaseLimit: DefaultSystemBaseLimit
|
The `Scale` method takes two parameters: the amount of memory and the number of file
|
||||||
|
descriptors that an application is willing to dedicate to libp2p.
|
||||||
|
|
||||||
transient:
|
These amounts will differ between use cases: A blockchain node running on a dedicated
|
||||||
Memory: memoryCap / 16
|
server might have a lot of memory, and dedicate 1/4 of that memory to libp2p. On the
|
||||||
BaseLimit: DefaultTransientBaseLimit
|
other end of the spectrum, a desktop companion application running as a background
|
||||||
|
task on a consumer laptop will probably dedicate significantly less than 1/4 of its system
|
||||||
|
memory to libp2p.
|
||||||
|
|
||||||
svc =
|
For convenience, the `ScalingLimitConfig` also provides an `AutoScale` method,
|
||||||
Memory: memoryCap / 2
|
which determines the amount of memory and file descriptors available on the
|
||||||
BaseLimit: DefaultServiceBaseLimit
|
system, and dedicates up to 1/8 of the memory and 1/2 of the file descriptors to libp2p.
|
||||||
|
|
||||||
proto:
|
For example, one might set:
|
||||||
Memory: memoryCap / 4
|
```go
|
||||||
BaseLimit: DefaultProtocolBaseLimit
|
var scalingLimits = ScalingLimitConfig{
|
||||||
|
SystemBaseLimit: BaseLimit{
|
||||||
peer:
|
ConnsInbound: 64,
|
||||||
Memory: memoryCap / 16
|
ConnsOutbound: 128,
|
||||||
BaseLimit: DefaultPeerBaseLimit
|
Conns: 128,
|
||||||
|
StreamsInbound: 512,
|
||||||
conn:
|
StreamsOutbound: 1024,
|
||||||
Memory: 16 << 20,
|
Streams: 1024,
|
||||||
|
Memory: 128 << 20,
|
||||||
stream:
|
FD: 256,
|
||||||
Memory: 16 << 20,
|
},
|
||||||
|
SystemLimitIncrease: BaseLimitIncrease{
|
||||||
|
ConnsInbound: 32,
|
||||||
|
ConnsOutbound: 64,
|
||||||
|
Conns: 64,
|
||||||
|
StreamsInbound: 256,
|
||||||
|
StreamsOutbound: 512,
|
||||||
|
Streams: 512,
|
||||||
|
Memory: 256 << 20,
|
||||||
|
FDFraction: 1,
|
||||||
|
},
|
||||||
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
We also provide a dynamic limiter which uses the same base limits, but
|
The base limit (`SystemBaseLimit`) here is the minimum configuration that any
|
||||||
the memory limit is dynamically computed at each memory reservation check
|
node will have, no matter how little memory it possesses. For every GB of memory
|
||||||
based on free memory.
|
passed into the `Scale` method, an increase of (`SystemLimitIncrease`) is added.
|
||||||
|
|
||||||
|
For Example, calling `Scale` with 4 GB of memory will result in a limit of 384 for
|
||||||
|
`Conns` (128 + 4*64).
|
||||||
|
|
||||||
|
The `FDFraction` defines how many of the file descriptors are allocated to this
|
||||||
|
scope. In the example above, when called with a file descriptor value of 1000,
|
||||||
|
this would result in a limit of 1256 file descriptors for the system scope.
|
||||||
|
|
||||||
|
Note that we only showed the configuration for the system scope here, equivalent
|
||||||
|
configuration options apply to all other scopes as well.
|
||||||
|
|
||||||
## Implementation Notes
|
## Implementation Notes
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user