WIP write up how this thing works

trace-analyzer/README.md

@@ -0,0 +1,45 @@
+# Resource Manager Trace Analyzer
+
+This directory contains a script to analyze the trace generated by the resource manager, by:
+1. for every resource (de)allocation, emitting a log line with the current state of the applicable scope
+2. assigning a "class" to any scope. Classes respond to the different categories in the [`DefaultLimitConfig`](https://github.com/libp2p/go-libp2p-resource-manager/blob/master/limit_defaults.go).
+
+This script is most useful to determine appropriate limits for a node running in its typical position in the network. In order to establish a baseline, the node can be run with a resource manager configuration in which all allocations are allowed (by using the `InfiniteLimits` configuration) for a few hours / days.
+
+Keep in mind that the number of file descriptors (each TCP connection consumes one file descriptor, for example) is limited by the operating system. The resource manager has no way of modifying this limit. See [this article](https://www.linuxtechi.com/set-ulimit-file-descriptors-limit-linux-servers/) for details. Depending on the system configuration, it might make sense to increase this limit.
+
+## Usage
+```bash
+go run main.go <path to rcmgr.json.gz> <path to output.json>
+```
+
+## Examples
+
+### System Scope
+
+```json
+{"Time":"2022-06-04T14:45:11.917046+02:00","Class":"system","Stat":{"StreamsIn":2,"StreamsOut":13,"ConnsIn":0,"ConnsOut":32,"FD":16,"Memory":20480}}
+{"Time":"2022-06-04T14:45:11.927064+02:00","Class":"system","Stat":{"StreamsIn":3,"StreamsOut":13,"ConnsIn":0,"ConnsOut":32,"FD":16,"Memory":20480}}
+```
+
+At the beginning, the node is handling 32 outgoing connections (using 16 file descriptors), with 13 outgoing and 2 incoming streams. The second log line is emitted because an additional stream was opened by the remote peer.
+
+### Protocol-Peer Scope
+
+```json
+{"Time":"2022-06-04T13:08:51.873356976Z","Class":"protocol-peer","Protocol":"/ipfs/kad/1.0.0","Peer":"12D3KooWFXc3Tht1FgrFVNg7SigUJvWrk15ELYtXvs3hLrDFdtEA","Stat":{"StreamsIn":0,"StreamsOut":1,"ConnsIn":0,"ConnsOut":0,"FD":0,"Memory":0}}
+{"Time":"2022-06-04T13:08:51.873523513Z","Class":"protocol-peer","Protocol":"/ipfs/kad/1.0.0","Peer":"12D3KooWFXc3Tht1FgrFVNg7SigUJvWrk15ELYtXvs3hLrDFdtEA","Stat":{"StreamsIn":0,"StreamsOut":2,"ConnsIn":0,"ConnsOut":0,"FD":0,"Memory":0}}
+```
+
+The Protocol-Peer scope is a bit more interesting. It shows the resource consumption of a protocol (here Kademlia, `/ipfs/kad/1.0.0`) for a certain peer (`12D3KooWFXc3Tht1FgrFVNg7SigUJvWrk15ELYtXvs3hLrDFdtEA`). Limiting based on individual peer IDs is not interesting, but determining how many streams a certain protocol opens _per peer_ is. Protocols _should_ limit their stream usage, thus observing a protocol that opens a very large number of streams concurrently might show that the protocol is overly greedily consuming resources and might need to be fixed.
+
+## Resource Manager Limit Configuration
+
+The script also calculates a `DefaultLimitConfig` that can be used as a starting point to determine appropriate limit for a node. The calculated configuration has limits set such that each and every allocation contained in the trace would have been allowed, but no more than that.
+
+While it is possible to simply copy-and-paste this configuration, it is a good idea to do a few sanity checks:
+* It is easier for an attacker to open connections / streams to us and exhaust our resources, than to trick our node into opening a lot of connections / streams. Resource limits for inbound resources therefore should be tighter in the incoming than the outgoing direction, i.e. `StreamsOutbound > StreamsInbound` and `ConnsOutbound > ConnsInbound` (in every scope).
+* The calculated config sets `Streams = StreamsInbound + StreamOutbound` and `Conns = ConnsInbound + ConnsOutbound`. When using increased outbound limits, it might make sense to set `Streams = StreamsOutbound` and `Conns = ConnsOutbound` to enforce a limit on the total number of streams.
+* TODO: comment on transient limits
+* TODO: comment on memory limits
+* TODO: comment on ConnMemory and StreamMemory