Merge pull request #6356 from explosic4/master

translated talk/20170320 Education of a Programmer.md

sources/talk/20170320 Education of a Programmer.md

@@ -1,157 +0,0 @@
-"amesy translating."
-
-Education of a Programmer
-============================================================
-
-_When I left Microsoft in October 2016 after almost 21 years there and almost 35 years in the industry, I took some time to reflect on what I had learned over all those years. This is a lightly edited version of that post. Pardon the length!_ 
-
-There are an amazing number of things you need to know to be a proficient programmer — details of languages, APIs, algorithms, data structures, systems and tools. These things change all the time — new languages and programming environments spring up and there always seems to be some hot new tool or language that “everyone” is using. It is important to stay current and proficient. A carpenter needs to know how to pick the right hammer and nail for the job and needs to be competent at driving the nail straight and true.
-
-At the same time, I’ve found that there are some concepts and strategies that are applicable over a wide range of scenarios and across decades. We have seen multiple orders of magnitude change in the performance and capability of our underlying devices and yet certain ways of thinking about the design of systems still say relevant. These are more fundamental than any specific implementation. Understanding these recurring themes is hugely helpful in both the analysis and design of the complex systems we build.
-
-Humility and Ego
-
-This is not limited to programming, but in an area like computing which exhibits so much constant change, one needs a healthy balance of humility and ego. There is always more to learn and there is always someone who can help you learn it — if you are willing and open to that learning. One needs both the humility to recognize and acknowledge what you don’t know and the ego that gives you confidence to master a new area and apply what you already know. The biggest challenges I have seen are when someone works in a single deep area for a long time and “forgets” how good they are at learning new things. The best learning comes from actually getting hands dirty and building something, even if it is just a prototype or hack. The best programmers I know have had both a broad understanding of technology while at the same time have taken the time to go deep into some technology and become the expert. The deepest learning happens when you struggle with truly hard problems.
-
-End to End Argument
-
-Back in 1981, Jerry Saltzer, Dave Reed and Dave Clark were doing early work on the Internet and distributed systems and wrote up their [classic description][4] of the end to end argument. There is much misinformation out there on the Internet so it can be useful to go back and read the original paper. They were humble in not claiming invention — from their perspective this was a common engineering strategy that applies in many areas, not just in communications. They were simply writing it down and gathering examples. A minor paraphrasing is:
-
-When implementing some function in a system, it can be implemented correctly and completely only with the knowledge and participation of the endpoints of the system. In some cases, a partial implementation in some internal component of the system may be important for performance reasons.
-
-The SRC paper calls this an “argument”, although it has been elevated to a “principle” on Wikipedia and in other places. In fact, it is better to think of it as an argument — as they detail, one of the hardest problem for a system designer is to determine how to divide responsibilities between components of a system. This ends up being a discussion that involves weighing the pros and cons as you divide up functionality, isolate complexity and try to design a reliable, performant system that will be flexible to evolving requirements. There is no simple set of rules to follow.
-
-Much of the discussion on the Internet focuses on communications systems, but the end-to-end argument applies in a much wider set of circumstances. One example in distributed systems is the idea of “eventual consistency”. An eventually consistent system can optimize and simplify by letting elements of the system get into a temporarily inconsistent state, knowing that there is a larger end-to-end process that can resolve these inconsistencies. I like the example of a scaled-out ordering system (e.g. as used by Amazon) that doesn’t require every request go through a central inventory control choke point. This lack of a central control point might allow two endpoints to sell the same last book copy, but the overall system needs some type of resolution system in any case, e.g. by notifying the customer that the book has been backordered. That last book might end up getting run over by a forklift in the warehouse before the order is fulfilled anyway. Once you realize an end-to-end resolution system is required and is in place, the internal design of the system can be optimized to take advantage of it.
-
-In fact, it is this design flexibility in the service of either ongoing performance optimization or delivering other system features that makes this end-to-end approach so powerful. End-to-end thinking often allows internal performance flexibility which makes the overall system more robust and adaptable to changes in the characteristics of each of the components. This makes an end-to-end approach “anti-fragile” and resilient to change over time.
-
-An implication of the end-to-end approach is that you want to be extremely careful about adding layers and functionality that eliminates overall performance flexibility. (Or other flexibility, but performance, especially latency, tends to be special.) If you expose the raw performance of the layers you are built on, end-to-end approaches can take advantage of that performance to optimize for their specific requirements. If you chew up that performance, even in the service of providing significant value-add functionality, you eliminate design flexibility.
-
-The end-to-end argument intersects with organizational design when you have a system that is large and complex enough to assign whole teams to internal components. The natural tendency of those teams is to extend the functionality of those components, often in ways that start to eliminate design flexibility for applications trying to deliver end-to-end functionality built on top of them.
-
-One of the challenges in applying the end-to-end approach is determining where the end is. “Little fleas have lesser fleas… and so on ad infinitum.”
-
-Concentrating Complexity
-
-Coding is an incredibly precise art, with each line of execution required for correct operation of the program. But this is misleading. Programs are not uniform in the overall complexity of their components or the complexity of how those components interact. The most robust programs isolate complexity in a way that lets significant parts of the system appear simple and straightforward and interact in simple ways with other components in the system. Complexity hiding can be isomorphic with other design approaches like information hiding and data abstraction but I find there is a different design sensibility if you really focus on identifying where the complexity lies and how you are isolating it.
-
-The example I’ve returned to over and over again in my [writing][5] is the screen repaint algorithm that was used by early character video terminal editors like VI and EMACS. The early video terminals implemented control sequences for the core action of painting characters as well as additional display functions to optimize redisplay like scrolling the current lines up or down or inserting new lines or moving characters within a line. Each of those commands had different costs and those costs varied across different manufacturer’s devices. (See [TERMCAP][6] for links to code and a fuller history.) A full-screen application like a text editor wanted to update the screen as quickly as possible and therefore needed to optimize its use of these control sequences to transition the screen from one state to another.
-
-These applications were designed so this underlying complexity was hidden. The parts of the system that modify the text buffer (where most innovation in functionality happens) completely ignore how these changes are converted into screen update commands. This is possible because the performance cost of computing the optimal set of updates for  _any_  change in the content is swamped by the performance cost of actually executing the update commands on the terminal itself. It is a common pattern in systems design that performance analysis plays a key part in determining how and where to hide complexity. The screen update process can be asynchronous to the changes in the underlying text buffer and can be independent of the actual historical sequence of changes to the buffer. It is not important  _how_  the buffer changed, but only  _what_  changed. This combination of asynchronous coupling, elimination of the combinatorics of historical path dependence in the interaction between components and having a natural way for interactions to efficiently batch together are common characteristics used to hide coupling complexity.
-
-Success in hiding complexity is determined not by the component doing the hiding but by the consumers of that component. This is one reason why it is often so critical for a component provider to actually be responsible for at least some piece of the end-to-end use of that component. They need to have clear optics into how the rest of the system interacts with their component and how (and whether) complexity leaks out. This often shows up as feedback like “this component is hard to use” — which typically means that it is not effectively hiding the internal complexity or did not pick a functional boundary that was amenable to hiding that complexity.
-
-Layering and Componentization
-
-It is the fundamental role of a system designer to determine how to break down a system into components and layers; to make decisions about what to build and what to pick up from elsewhere. Open Source may keep money from changing hands in this “build vs. buy” decision but the dynamics are the same. An important element in large scale engineering is understanding how these decisions will play out over time. Change fundamentally underlies everything we do as programmers, so these design choices are not only evaluated in the moment, but are evaluated in the years to come as the product continues to evolve.
-
-Here are a few things about system decomposition that end up having a large element of time in them and therefore tend to take longer to learn and appreciate.
-
-*   Layers are leaky. Layers (or abstractions) are [fundamentally leaky][1]. These leaks have consequences immediately but also have consequences over time, in two ways. One consequence is that the characteristics of the layer leak through and permeate more of the system than you realize. These might be assumptions about specific performance characteristics or behavior ordering that is not an explicit part of the layer contract. This means that you generally are more  _vulnerable_  to changes in the internal behavior of the component that you understood. A second consequence is it also means you are more  _dependent_  on that internal behavior than is obvious, so if you consider changing that layer the consequences and challenges are probably larger than you thought.
-*   Layers are too functional. It is almost a truism that a component you adopt will have more functionality than you actually require. In some cases, the decision to use it is based on leveraging that functionality for future uses. You adopt specifically because you want to “get on the train” and leverage the ongoing work that will go into that component. There are a few consequences of building on this highly functional layer. 1) The component will often make trade-offs that are biased by functionality that you do not actually require. 2) The component will embed complexity and constraints because of functionality you do not require and those constraints will impede future evolution of that component. 3) There will be more surface area to leak into your application. Some of that leakage will be due to true “leaky abstractions” and some will be explicit (but generally poorly controlled) increased dependence on the full capabilities of the component. Office is big enough that we found that for any layer we built on, we eventually fully explored its functionality in some part of the system. While that might appear to be positive (we are more completely leveraging the component), all uses are not equally valuable. So we end up having a massive cost to move from one layer to another based on this long-tail of often lower value and poorly recognized use cases. 4) The additional functionality creates complexity and opportunities for misuse. An XML validation API we used would optionally dynamically download the schema definition if it was specified as part of the XML tree. This was mistakenly turned on in our basic file parsing code which resulted in both a massive performance degradation as well as an (unintentional) distributed denial of service attack on a w3c.org web server. (These are colloquially known as “land mine” APIs.)
-*   Layers get replaced. Requirements evolve, systems evolve, components are abandoned. You eventually need to replace that layer or component. This is true for external component dependencies as well as internal ones. This means that the issues above will end up becoming important.
-*   Your build vs. buy decision will change. This is partly a corollary of above. This does not mean the decision to build or buy was wrong at the time. Often there was no appropriate component when you started and it only becomes available later. Or alternatively, you use a component but eventually find that it does not match your evolving requirements and your requirements are narrow enough, well-understood or so core to your value proposition that it makes sense to own it yourself. It does mean that you need to be just as concerned about leaky layers permeating more of the system for layers you build as well as for layers you adopt.
-*   Layers get thick. As soon as you have defined a layer, it starts to accrete functionality. The layer is the natural throttle point to optimize for your usage patterns. The difficulty with a thick layer is that it tends to reduce your ability to leverage ongoing innovation in underlying layers. In some sense this is why OS companies hate thick layers built on top of their core evolving functionality — the pace at which innovation can be adopted is inherently slowed. One disciplined approach to avoid this is to disallow any additional state storage in an adaptor layer. Microsoft Foundation Classes took this general approach in building on top of Win32\. It is inevitably cheaper in the short term to just accrete functionality on to an existing layer (leading to all the eventual problems above) rather than refactoring and recomponentizing. A system designer who understands this looks for opportunities to break apart and simplify components rather than accrete more and more functionality within them.
-
-Einsteinian Universe
-
-I had been designing asynchronous distributed systems for decades but was struck by this quote from Pat Helland, a SQL architect, at an internal Microsoft talk. “We live in an Einsteinian universe — there is no such thing as simultaneity. “ When building distributed systems — and virtually everything we build is a distributed system — you cannot hide the distributed nature of the system. It’s just physics. This is one of the reasons I’ve always felt Remote Procedure Call, and especially “transparent” RPC that explicitly tries to hide the distributed nature of the interaction, is fundamentally wrong-headed. You need to embrace the distributed nature of the system since the implications almost always need to be plumbed completely through the system design and into the user experience.
-
-Embracing the distributed nature of the system leads to a number of things:
-
-*   You think through the implications to the user experience from the start rather than trying to patch on error handling, cancellation and status reporting as an afterthought.
-*   You use asynchronous techniques to couple components. Synchronous coupling is  _impossible._  If something appears synchronous, it’s because some internal layer has tried to hide the asynchrony and in doing so has obscured (but definitely not hidden) a fundamental characteristic of the runtime behavior of the system.
-*   You recognize and explicitly design for interacting state machines and that these states represent robust long-lived internal system states (rather than ad-hoc, ephemeral and undiscoverable state encoded by the value of variables in a deep call stack).
-*   You recognize that failure is expected. The only guaranteed way to detect failure in a distributed system is to simply decide you have waited “too long”. This naturally means that [cancellation is first-class][2]. Some layer of the system (perhaps plumbed through to the user) will need to decide it has waited too long and cancel the interaction. Cancelling is only about reestablishing local state and reclaiming local resources — there is no way to reliably propagate that cancellation through the system. It can sometimes be useful to have a low-cost, unreliable way to attempt to propagate cancellation as a performance optimization.
-*   You recognize that cancellation is not rollback since it is just reclaiming local resources and state. If rollback is necessary, it needs to be an end-to-end feature.
-*   You accept that you can never really know the state of a distributed component. As soon as you discover the state, it may have changed. When you send an operation, it may be lost in transit, it might be processed but the response is lost, or it may take some significant amount of time to process so the remote state ultimately transitions at some arbitrary time in the future. This leads to approaches like idempotent operations and the ability to robustly and efficiently rediscover remote state rather than expecting that distributed components can reliably track state in parallel. The concept of “[eventual consistency][3]” succinctly captures many of these ideas.
-
-I like to say you should “revel in the asynchrony”. Rather than trying to hide it, you accept it and design for it. When you see a technique like idempotency or immutability, you recognize them as ways of embracing the fundamental nature of the universe, not just one more design tool in your toolbox.
-
-Performance
-
-I am sure Don Knuth is horrified by how misunderstood his partial quote “Premature optimization is the root of all evil” has been. In fact, performance, and the incredible exponential improvements in performance that have continued for over 6 decades (or more than 10 decades depending on how willing you are to project these trends through discrete transistors, vacuum tubes and electromechanical relays), underlie all of the amazing innovation we have seen in our industry and all the change rippling through the economy as “software eats the world”.
-
-A key thing to recognize about this exponential change is that while all components of the system are experiencing exponential change, these exponentials are divergent. So the rate of increase in capacity of a hard disk changes at a different rate from the capacity of memory or the speed of the CPU or the latency between memory and CPU. Even when trends are driven by the same underlying technology, exponentials diverge. [Latency improvements fundamentally trail bandwidth improvements][7]. Exponential change tends to look linear when you are close to it or over short periods but the effects over time can be overwhelming. This overwhelming change in the relationship between the performance of components of the system forces reevaluation of design decisions on a regular basis.
-
-A consequence of this is that design decisions that made sense at one point no longer make sense after a few years. Or in some cases an approach that made sense two decades ago starts to look like a good trade-off again. Modern memory mapping has characteristics that look more like process swapping of the early time-sharing days than it does like demand paging. (This does sometimes result in old codgers like myself claiming that “that’s just the same approach we used back in ‘75” — ignoring the fact that it didn’t make sense for 40 years and now does again because some balance between two components — maybe flash and NAND rather than disk and core memory — has come to resemble a previous relationship).
-
-Important transitions happen when these exponentials cross human constraints. So you move from a limit of two to the sixteenth characters (which a single user can type in a few hours) to two to the thirty-second (which is beyond what a single person can type). So you can capture a digital image with higher resolution than the human eye can perceive. Or you can store an entire music collection on a hard disk small enough to fit in your pocket. Or you can store a digitized video recording on a hard disk. And then later the ability to stream that recording in real time makes it possible to “record” it by storing it once centrally rather than repeatedly on thousands of local hard disks.
-
-The things that stay as a fundamental constraint are three dimensions and the speed of light. We’re back to that Einsteinian universe. We will always have memory hierarchies — they are fundamental to the laws of physics. You will always have stable storage and IO, memory, computation and communications. The relative capacity, latency and bandwidth of these elements will change, but the system is always about how these elements fit together and the balance and tradeoffs between them. Jim Gray was the master of this analysis.
-
-Another consequence of the fundamentals of 3D and the speed of light is that much of performance analysis is about three things: locality, locality, locality. Whether it is packing data on disk, managing processor cache hierarchies, or coalescing data into a communications packet, how data is packed together, the patterns for how you touch that data with locality over time and the patterns of how you transfer that data between components is fundamental to performance. Focusing on less code operating on less data with more locality over space and time is a good way to cut through the noise.
-
-Jon Devaan used to say “design the data, not the code”. This also generally means when looking at the structure of a system, I’m less interested in seeing how the code interacts — I want to see how the data interacts and flows. If someone tries to explain a system by describing the code structure and does not understand the rate and volume of data flow, they do not understand the system.
-
-A memory hierarchy also implies we will always have caches — even if some system layer is trying to hide it. Caches are fundamental but also dangerous. Caches are trying to leverage the runtime behavior of the code to change the pattern of interaction between different components in the system. They inherently need to model that behavior, even if that model is implicit in how they fill and invalidate the cache and test for a cache hit. If the model is poor  _or becomes_  poor as the behavior changes, the cache will not operate as expected. A simple guideline is that caches  _must_  be instrumented — their behavior will degrade over time because of changing behavior of the application and the changing nature and balance of the performance characteristics of the components you are modeling. Every long-time programmer has cache horror stories.
-
-I was lucky that my early career was spent at BBN, one of the birthplaces of the Internet. It was very natural to think about communications between asynchronous components as the natural way systems connect. Flow control and queueing theory are fundamental to communications systems and more generally the way that any asynchronous system operates. Flow control is inherently resource management (managing the capacity of a channel) but resource management is the more fundamental concern. Flow control also is inherently an end-to-end responsibility, so thinking about asynchronous systems in an end-to-end way comes very naturally. The story of [buffer bloat][8]is well worth understanding in this context because it demonstrates how lack of understanding the dynamics of end-to-end behavior coupled with technology “improvements” (larger buffers in routers) resulted in very long-running problems in the overall network infrastructure.
-
-The concept of “light speed” is one that I’ve found useful in analyzing any system. A light speed analysis doesn’t start with the current performance, it asks “what is the best theoretical performance I could achieve with this design?” What is the real information content being transferred and at what rate of change? What is the underlying latency and bandwidth between components? A light speed analysis forces a designer to have a deeper appreciation for whether their approach could ever achieve the performance goals or whether they need to rethink their basic approach. It also forces a deeper understanding of where performance is being consumed and whether this is inherent or potentially due to some misbehavior. From a constructive point of view, it forces a system designer to understand what are the true performance characteristics of their building blocks rather than focusing on the other functional characteristics.
-
-I spent much of my career building graphical applications. A user sitting at one end of the system defines a key constant and constraint in any such system. The human visual and nervous system is not experiencing exponential change. The system is inherently constrained, which means a system designer can leverage ( _must_  leverage) those constraints, e.g. by virtualization (limiting how much of the underlying data model needs to be mapped into view data structures) or by limiting the rate of screen update to the perception limits of the human visual system.
-
-The Nature of Complexity
-
-I have struggled with complexity my entire career. Why do systems and apps get complex? Why doesn’t development within an application domain get easier over time as the infrastructure gets more powerful rather than getting harder and more constrained? In fact, one of our key approaches for managing complexity is to “walk away” and start fresh. Often new tools or languages force us to start from scratch which means that developers end up conflating the benefits of the tool with the benefits of the clean start. The clean start is what is fundamental. This is not to say that some new tool, platform or language might not be a great thing, but I can guarantee it will not solve the problem of complexity growth. The simplest way of controlling complexity growth is to build a smaller system with fewer developers.
-
-Of course, in many cases “walking away” is not an alternative — the Office business is built on hugely valuable and complex assets. With OneNote, Office “walked away” from the complexity of Word in order to innovate along a different dimension. Sway is another example where Office decided that we needed to free ourselves from constraints in order to really leverage key environmental changes and the opportunity to take fundamentally different design approaches. With the Word, Excel and PowerPoint web apps, we decided that the linkage with our immensely valuable data formats was too fundamental to walk away from and that has served as a significant and ongoing constraint on development.
-
-I was influenced by Fred Brook’s “[No Silver Bullet][9]” essay about accident and essence in software development. There is much irreducible complexity embedded in the essence of what the software is trying to model. I just recently re-read that essay and found it surprising on re-reading that two of the trends he imbued with the most power to impact future developer productivity were increasing emphasis on “buy” in the “build vs. buy” decision — foreshadowing the change that open-source and cloud infrastructure has had. The other trend was the move to more “organic” or “biological” incremental approaches over more purely constructivist approaches. A modern reader sees that as the shift to agile and continuous development processes. This in 1986!
-
-I have been much taken with the work of Stuart Kauffman on the fundamental nature of complexity. Kauffman builds up from a simple model of Boolean networks (“[NK models][10]”) and then explores the application of this fundamentally mathematical construct to things like systems of interacting molecules, genetic networks, ecosystems, economic systems and (in a limited way) computer systems to understand the mathematical underpinning to emergent ordered behavior and its relationship to chaotic behavior. In a highly connected system, you inherently have a system of conflicting constraints that makes it (mathematically) hard to evolve that system forward (viewed as an optimization problem over a rugged landscape). A fundamental way of controlling this complexity is to batch the system into independent elements and limit the interconnections between elements (essentially reducing both “N” and “K” in the NK model). Of course this feels natural to a system designer applying techniques of complexity hiding, information hiding and data abstraction and using loose asynchronous coupling to limit interactions between components.
-
-A challenge we always face is that many of the ways we want to evolve our systems cut across all dimensions. Real-time co-authoring has been a very concrete (and complex) recent example for the Office apps.
-
-Complexity in our data models often equates with “power”. An inherent challenge in designing user experiences is that we need to map a limited set of gestures into a transition in the underlying data model state space. Increasing the dimensions of the state space inevitably creates ambiguity in the user gesture. This is “[just math][11]” which means that often times the most fundamental way to ensure that a system stays “easy to use” is to constrain the underlying data model.
-
-Management
-
-I started taking leadership roles in high school (student council president!) and always found it natural to take on larger responsibilities. At the same time, I was always proud that I continued to be a full-time programmer through every management stage. VP of development for Office finally pushed me over the edge and away from day-to-day programming. I’ve enjoyed returning to programming as I stepped away from that job over the last year — it is an incredibly creative and fulfilling activity (and maybe a little frustrating at times as you chase down that “last” bug).
-
-Despite having been a “manager” for over a decade by the time I arrived at Microsoft, I really learned about management after my arrival in 1996\. Microsoft reinforced that “engineering leadership is technical leadership”. This aligned with my perspective and helped me both accept and grow into larger management responsibilities.
-
-The thing that most resonated with me on my arrival was the fundamental culture of transparency in Office. The manager’s job was to design and use transparent processes to drive the project. Transparency is not simple, automatic, or a matter of good intentions — it needs to be designed into the system. The best transparency comes by being able to track progress as the granular output of individual engineers in their day-to-day activity (work items completed, bugs opened and fixed, scenarios complete). Beware subjective red/green/yellow, thumbs-up/thumbs-down dashboards!
-
-I used to say my job was to design feedback loops. Transparent processes provide a way for every participant in the process — from individual engineer to manager to exec to use the data being tracked to drive the process and result and understand the role they are playing in the overall project goals. Ultimately transparency ends up being a great tool for empowerment — the manager can invest more and more local control in those closest to the problem because of confidence they have visibility to the progress being made. Coordination emerges naturally.
-
-Key to this is that the goal has actually been properly framed (including key resource constraints like ship schedule). Decision-making that needs to constantly flow up and down the management chain usually reflects poor framing of goals and constraints by management.
-
-I was at Beyond Software when I really internalized the importance of having a singular leader over a project. The engineering manager departed (later to hire me away for FrontPage) and all four of the leads were hesitant to step into the role — not least because we did not know how long we were going to stick around. We were all very technically sharp and got along well so we decided to work as peers to lead the project. It was a mess. The one obvious problem is that we had no strategy for allocating resources between the pre-existing groups — one of the top responsibilities of management! The deep accountability one feels when you know you are personally in charge was missing. We had no leader really accountable for unifying goals and defining constraints.
-
-I have a visceral memory of the first time I fully appreciated the importance of  _listening_  for a leader. I had just taken on the role of Group Development Manager for Word, OneNote, Publisher and Text Services. There was a significant controversy about how we were organizing the text services team and I went around to each of the key participants, heard what they had to say and then integrated and wrote up all I had heard. When I showed the write-up to one of the key participants, his reaction was “wow, you really heard what I had to say”! All of the largest issues I drove as a manager (e.g. cross-platform and the shift to continuous engineering) involved carefully listening to all the players. Listening is an active process that involves trying to understand the perspectives and then writing up what I learned and testing it to validate my understanding. When a key hard decision needed to happen, by the time the call was made everyone knew they had been heard and understood (whether they agreed with the decision or not).
-
-It was the previous job, as FrontPage development manager, where I internalized the “operational dilemma” inherent in decision making with partial information. The longer you wait, the more information you will have to make a decision. But the longer you wait, the less flexibility you will have to actually implement it. At some point you just need to make a call.
-
-Designing an organization involves a similar tension. You want to increase the resource domain so that a consistent prioritization framework can be applied across a larger set of resources. But the larger the resource domain, the harder it is to actually have all the information you need to make good decisions. An organizational design is about balancing these two factors. Software complicates this because characteristics of the software can cut across the design in an arbitrary dimensionality. Office has used [shared teams][12] to address both these issues (prioritization and resources) by having cross-cutting teams that can share work (add resources) with the teams they are building for.
-
-One dirty little secret you learn as you move up the management ladder is that you and your new peers aren’t suddenly smarter because you now have more responsibility. This reinforces that the organization as a whole better be smarter than the leader at the top. Empowering every level to own their decisions within a consistent framing is the key approach to making this true. Listening and making yourself accountable to the organization for articulating and explaining the reasoning behind your decisions is another key strategy. Surprisingly, fear of making a dumb decision can be a useful motivator for ensuring you articulate your reasoning clearly and make sure you listen to all inputs.
-
-Conclusion
-
-At the end of my interview round for my first job out of college, the recruiter asked if I was more interested in working on “systems” or “apps”. I didn’t really understand the question. Hard, interesting problems arise at every level of the software stack and I’ve had fun plumbing all of them. Keep learning.
-
---------------------------------------------------------------------------------
-
-via: https://hackernoon.com/education-of-a-programmer-aaecf2d35312
-
-作者：[ Terry Crowley][a]
-译者：[译者ID](https://github.com/译者ID)
-校对：[校对者ID](https://github.com/校对者ID)
-
-本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
-
-[a]:https://hackernoon.com/@terrycrowley
-[1]:https://medium.com/@terrycrowley/leaky-by-design-7b423142ece0#.x67udeg0a
-[2]:https://medium.com/@terrycrowley/how-to-think-about-cancellation-3516fc342ae#.3pfjc5b54
-[3]:http://queue.acm.org/detail.cfm?id=2462076
-[4]:http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf
-[5]:https://medium.com/@terrycrowley/model-view-controller-and-loose-coupling-6370f76e9cde#.o4gnupqzq
-[6]:https://en.wikipedia.org/wiki/Termcap
-[7]:http://www.ll.mit.edu/HPEC/agendas/proc04/invited/patterson_keynote.pdf
-[8]:https://en.wikipedia.org/wiki/Bufferbloat
-[9]:http://worrydream.com/refs/Brooks-NoSilverBullet.pdf
-[10]:https://en.wikipedia.org/wiki/NK_model
-[11]:https://medium.com/@terrycrowley/the-math-of-easy-to-use-14645f819201#.untmk9eq7
-[12]:https://medium.com/@terrycrowley/breaking-conways-law-a0fdf8500413#.gqaqf1c5k

translated/talk/20170320 Education of a Programmer.md

@@ -0,0 +1,167 @@
+程序员的学习之路
+============================================================
+
+*2016 年 10 月，当我从微软离职时，我已经在微软工作了近 21 年，在工业界也快 35 年了。我花了一些时间反思我这些年来学到的东西，这些文字是那篇帖子稍加修改后得到。请见谅，文章有一点长。*
+  
+要成为一名专业的程序员，你需要知道的事情多得令人吃惊：语言的细节，API，算法，数据结构，系统和工具。这些东西一直在随着时间变化——新的语言和编程环境不断出现，似乎总有热门的新工具或新语言是“每个人”都在使用的。紧跟潮流，保持专业，这很重要。木匠需要知道如何为工作选择合适的锤子和钉子，并且要有能力笔直精准地钉入钉子。
+
+与此同时，我也发现有一些理论和方法有着广泛的应用场景，它们能使用几十年。底层设备的性能和容量在这几十年来增长了几个数量级，但系统设计的思考方式还是互相有关联的，这些思考方式比具体的实现更根本。理解这些重复出现的主题对分析与设计我们所负责的系统大有帮助。
+
+谦卑和自我
+
+这不仅仅局限于编程，但在编程这个持续发展的领域，一个人需要在谦卑和自我中保持平衡。总有新的东西需要学习，并且总有人能帮助你学习——如果你愿意学习的话。一个人即需要保持谦卑，认识到自己不懂并承认它，也要保持自我，相信自己能掌握一个新的领域，并且能运用你已经掌握的知识。我见过的最大的挑战就是一些人在某个领域深入专研了很长时间，“忘记”了自己擅长学习新的东西。最好的学习来自放手去做，建造一些东西，即便只是一个原型或者　hack。我知道的最好的程序员对技术有广泛的认识，但同时他们对某个技术深入研究，成为了专家。而深入的学习来自努力解决真正困难的问题。
+
+端到端观点
+
+1981 年，Jerry Saltzer, Dave Reed 和 Dave Clark 在做因特网和分布式系统的早期工作，他们提出了端到端观点，并作出了[经典的阐述][4]。网络上的文章有许多误传，所以更应该阅读论文本身。论文的作者很谦虚，没有声称这是他们自己的创造——从他们的角度看，这只是一个常见的工程策略，不只在通讯领域中，在其他领域中也有运用。他们只是将其写下来并收集了一些例子。下面是文章的一个小片段：
+
+> 当我们设计系统的一个功能时，仅依靠端点的知识和端点的参与，就能正确地完整地实现这个功能。在一些情况下，系统的内部模块局部实现这个功能，可能会对性能有重要的提升。
+
+论文称这是一个“观点”，虽然在维基百科和其他地方它已经被上升成“原则”。实际上，还是把它看作一个观点比较好，正如作者们所说，系统设计者面临的最难的问题之一就是如何在系统组件之间划分责任，这会引发不断的讨论：怎样在划分功能时权衡利弊，怎样隔离复杂性，怎样设计一个灵活的高性能系统来满足不断变化的需求。没有简单的原则可以直接遵循。
+
+互联网上的大部分讨论集中在通信系统上，但端到端观点的适用范围其实更广泛。分布式系统中的“最终一致性”就是一个例子。一个满足“最终一致性”的系统，可以让系统中的元素暂时进入不一致的状态，从而简化系统，优化性能，因为有一个更大的端到端过程来解决不一致的状态。我喜欢横向拓展的订购系统的例子（例如亚马逊），它不要求每个请求都通过中央库存的控制点。缺少中央控制点可能允许两个终端出售相同的最后一本书，所以系统需要用某种方法来解决这个问题，如通知客户该书会延期交货。不论怎样设计，想购买的最后一本书在订单完成前都有可能被仓库中的叉车运出厍（译者注：比如被其他人下单购买）。一旦你意识到你需要一个端到端的解决方案，并实现了这个方案，那系统内部的设计就可以被优化，并利用这个解决方案。
+
+事实上，这种设计上的灵活性可以优化系统的性能，或者提供其他的系统功能，从而使得端到端的方法变得如此强大。端到端的思考往往允许内部进行灵活的操作，使整个系统更加健壮，并且能适应每个组件特性的变化。这些都让端到端的方法变得健壮，并能适应变化。
+
+端到端方法意味着，添加会牺牲整体性能灵活性的抽象层和功能时要非常小心（也可能是其他的灵活性，但性能，特别是延迟，往往是特殊的）。如果你展示出底层的原始性能（performance, 也可能指操作），端到端的方法可以根据这个性能（操作）来优化，实现特定的需求。如果你破坏了底层性能（操作），即使你实现了重要的有附加价值的功能，你也牺牲了设计灵活性。
+
+如果系统足够庞大而且足够复杂，需要把整个开发团队分配给系统内部的组件，那么端到端观点可以和团队组织相结合。这些团队自然要扩展这些组件的功能，他们通常从牺牲设计上的灵活性开始，尝试在组件上实现端到端的功能。
+
+应用端到端方法面临的挑战之一是确定端点在哪里。 俗话说，“大跳蚤上有小跳蚤，小跳蚤上有更少的跳蚤……等等”。
+
+关注复杂性
+
+编程是一门精确的艺术，每一行代码都要确保程序的正确执行。但这是带有误导的。编程的复杂性不在于各个部分的整合，也不在于各个部分之间如何相互交互。最健壮的程序将复杂性隔离开，让最重要的部分变的简单直接，通过简单的方式与其他部分交互。虽然隐藏复杂性和信息隐藏、数据抽象等其他设计方法一样，但我仍然觉得，如果你真的要定位出系统的复杂所在，并将其隔离开，那你需要对设计特别敏锐。
+
+在我的[文章][5]中反复提到的例子是早期的终端编辑器 VI 和 Emacs 中使用的屏幕重绘算法。早期的视频终端实现了控制序列，来控制绘制字符核心操作，也实现了附加的显示功能，来优化重新绘制屏幕，如向上向下滚动当前行，或者插入新行，或在当前行中移动字符。这些命令都具有不同的开销，并且这些开销在不同制造商的设备中也是不同的。（参见[TERMCAP][6]以获取代码链接和更完整的历史记录。）像文本编辑器这样的全屏应用程序希望尽快更新屏幕，因此需要优化使用这些控制序列来转换屏幕从一个状态到另一个状态。
+
+这些程序在设计上隐藏了底层的复杂性。系统中修改文本缓冲区的部分（功能上大多数创新都在这里）完全忽略了这些改变如何被转换成屏幕更新命令。这是可以接受的，因为针对*任何*内容的改变计算最佳命令所消耗的性能代价，远不及被终端本身实际执行这些更新命令的性能代价。在确定如何隐藏复杂性，以及隐藏哪些复杂性时，性能分析扮演着重要的角色，这一点在系统设计中非常常见。屏幕的更新与底层文本缓冲区的更改是异步的，并且可以独立于缓冲区的实际历史变化顺序。缓冲区*怎样*改变的并不重要，重要的是改变了*什么*。异步耦合，在组件交互时消除组件对历史路径依赖的组合，以及用自然的交互方式以有效地将组件组合在一起是隐藏耦合复杂度的常见特征。
+
+隐藏复杂性的成功不是由隐藏复杂性的组件决定的，而是由使用该模块的使用者决定的。这就是为什么组件的提供者至少要为组件的某些端到端过程负责。他们需要清晰的知道系统的其他部分如何与组件相互作用，复杂性是如何泄漏出来的（以及是否泄漏出来）。这常常表现为“这个组件很难使用”这样的反馈——这通常意味着它不能有效地隐藏内部复杂性，或者没有选择一个隐藏复杂性的功能边界。
+
+分层与组件化
+
+系统设计人员的一个基本工作是确定如何将系统分解成组件和层；决定自己要开发什么，以及从别的地方获取什么。开源项目在决定自己开发组件还是购买服务时，大多会选择自己开发，但组件之间交互的过程是一样的。在大规模工程中，理解这些决策将如何随着时间的推移而发挥作用是非常重要的。从根本上说，变化是程序员所做的一切的基础，所以这些设计决定不仅在当下被评估，还要随着产品的不断发展而在未来几年得到评估。
+
+以下是关于系统分解的一些事情，它们最终会占用大量的时间，因此往往需要更长的时间来学习和欣赏。
+
+* 层泄漏。层（或抽象）[基本上是泄漏的][1]。这些泄漏会立即产生后果，也会随着时间的推移而产生两方面的后果。其中一方面就是该抽象层的特性渗透到了系统的其他部分，渗透的程度比你意识到得更深入。这些渗透可能是关于具体的性能特征的假设，以及抽象层的文档中没有明确的指出的行为发生的顺序。这意味着假如内部组件的行为发生变化，你的系统会比想象中更加脆弱。第二方面是你比表面上看起来更依赖组件内部的行为，所以如果你考虑改变这个抽象层，后果和挑战可能超出你的想象。
+
+* 层具有太多功能了。您所采用的组件具有比实际需要更多的功能，这几乎是一个真理。在某些情况下，你决定采用这个组件是因为你想在将来使用那些尚未用到的功能。有时，你采用组件是想“上快车”，利用组件完成正在进行的工作。在功能强大的抽象层上开发会带来一些后果。1) 组件往往会根据你并不需要的功能作出取舍。 2) 为了实现那些你并不没有用到的功能，组件引入了复杂性和约束，这些约束将阻碍该组件的未来的演变。3) 层泄漏的范围更大。一些泄漏是由于真正的“抽象泄漏”，另一些是由于明显的，逐渐增加的对组件全部功能的依赖（但这些依赖通常都没有处理好）。Office 太大了，我们发现，对于我们建立的任何抽象层，我们最终都在系统的某个部分完全运用了它的功能。虽然这看起来是积极的（我们完全地利用了这个组件），但并不是所用的使用都有同样的价值。所以，我们最终要付出巨大的代价才能从一个抽象层往另一个抽象层迁移，这种“长尾巴”没什么价值，并且对使用场景认识不足。4) 附加的功能会增加复杂性，并增加功能滥用的可能。如果将验证 XML 的 API 指定为 XML 树的一部分，那这个 API 可以选择动态下载 XML 的模式定义。这在我们的基本文件解析代码中被错误地执行，导致 w3c.org 服务器上的大量性能下降以及（无意）分布式拒绝服务攻击。（这些被通俗地称为“地雷”API）。
+
+* 抽象层被更换。需求发展，系统发展，组件被放弃。您最终需要更换该抽象层或组件。不管是对外部组件的依赖还是对内部组件的依赖都是如此。这意味着上述问题将变得重要起来。
+
+* 自己构建还是购买的决定将会改变。这是上面几方面的必然结果。这并不意味着自己构建还是购买的决定在当时是错误的。一开始时往往没有合适的组件，一段时间之后才有合适的组件出现。或者，也可能你使用了一个组件，但最终发现它不符合您不断变化的要求，而且你的要求非常窄，能被理解，或着对你的价值体系来说是非常重要的，以至于拥有自己的模块是有意义的。这意味着你像关心自己构造的模块一样，关心购买的模块，关心它们是怎样泄漏并深入你的系统中的。
+
+* 抽象层会变臃肿。一旦你定义了一个抽象层，它就开始增加功能。层是对使用模式优化的自然分界点。臃肿的层的困难在于，它往往会降低您利用底层的不断创新的能力。从某种意义上说，这就是操作系统公司憎恨构建在其核心功能之上的臃肿的层的原因——采用创新的速度放缓了。避免这种情况的一种比较规矩的方法是禁止在适配器层中进行任何额外的状态存储。微软基础类在 Win32 上采用这个一般方法。在短期内，将功能集成到现有层（最终会导致上述所有问题）而不是重构和重新推导是不可避免的。理解这一点的系统设计人员寻找分解和简化组件的方法，而不是在其中增加越来越多的功能。
+
+爱因斯坦宇宙
+
+几十年来，我一直在设计异步分布式系统，但是在微软内部的一次演讲中，SQL 架构师 Pat Helland 的一句话震惊了我。 “我们生活在爱因斯坦的宇宙中，没有同时性。”在构建分布式系统时（基本上我们构建的都是分布式系统），你无法隐藏系统的分布式特性。这是物理的。我一直感到远程过程调用在根本上错误的，这是一个原因，尤其是那些“透明的”远程过程调用，它们就是想隐藏分布式的交互本质。你需要拥抱系统的分布式特性，因为这些意义几乎总是需要通过系统设计和用户体验来完成。
+
+拥抱分布式系统的本质则要遵循以下几个方面：
+
+* 一开始就要思考设计对用户体验的影响，而不是试图在处理错误，取消请求和报告状态上打补丁。
+
+* 使用异步技术来耦合组件。同步耦合是*不可能*的。如果某些行为看起来是同步的，是因为某些内部层尝试隐藏异步，这样做会遮蔽（但绝对不隐藏）系统运行时的基本行为特征。
+
+* 认识到并且明确设计了交互状态机，这些状态表示长期的可靠的内部系统状态（而不是由深度调用堆栈中的变量值编码的临时，短暂和不可发现的状态）。
+
+* 认识到失败是在所难免的。要保证能检测出分布式系统中的失败，唯一的办法就是直接看你的等待时间是否“太长”。这自然意味着[取消的等级最高][2]。系统的某一层（可能直接通向用户）需要决定等待时间是否过长，并取消操作。取消只是为了重建局部状态，回收局部的资源——没有办法在系统内广泛使用取消机制。有时用一种低成本，不可靠的方法广泛使用取消机制对优化性能可能有用。
+
+* 认识到取消不是回滚，因为它只是回收本地资源和状态。如果回滚是必要的，它必须实现成一个端到端的功能。
+
+* 承认永远不会真正知道分布式组件的状态。只要你发现一个状态，它可能就已经改变了。当你发送一个操作时，请求可能在传输过程中丢失，也可能被处理了但是返回的响应丢失了，或者请求需要一定的时间来处理，这样远程状态最终会在未来的某个任意的时间转换。这需要像幂等操作这样的方法，并且要能够稳健有效地重新发现远程状态，而不是期望可靠地跟踪分布式组件的状态。“[最终一致性][3]”的概念简洁地捕捉了这其中大多数想法。
+
+我喜欢说你应该“陶醉在异步”。与其试图隐藏异步，不如接受异步，为异步而设计。当你看到像幂等性或不变性这样的技术时，你就认识到它们是拥抱宇宙本质的方法，而不仅仅是工具箱中的一个设计工具。
+
+性能
+
+我确信 Don Knuth 会对人们怎样误解他的名言“过早的优化是一切罪恶的根源”而感到震惊。事实上，性能，及性能持续超过60年的指数增长（或超过10年，取决于您是否愿意将晶体管，真空管和机电继电器的发展算入其中），为所有行业内的惊人创新和影响经济的“软件吃遍世界”的变化打下了基础。
+
+要认识到这种指数变化的一个关键是，虽然系统的所有组件正在经历指数变化，但这些指数是不同的。硬盘容量的增长速度与内存容量的增长速度不同，与 CPU 的增长速度不同，与内存 CPU 之间的延迟的性能改善速度也不用。即使性能发展的趋势是由相同的基础技术驱动的，增长的指数也会有分歧。[延迟的改进从根本上改善了带宽][7]。指数变化在近距离或者短期内看起来是线性的，但随着时间的推移可能是压倒性的。系统不同组件的性能的增长不同，会出现压倒性的变化，并迫使对设计决策定期进行重新评估。
+
+这样做的结果是，几年后，一度有意义的设计决定就不再有意义了。或者在某些情况下，二十年前有意义的方法又开始变成一个好的决定。现代内存映射的特点看起来更像是早期分时的进程切换，而不像分页那样。 （这样做有时会让我这样的老人说“这就是我们在 1975 年时用的方法”——忽略了这种方法在 40 年都没有意义，但现在又重新成为好的方法，因为两个组件之间的关系——可能是闪存和 NAND 而不是磁盘和核心内存——已经变得像以前一样了）。
+
+当这些指数超越人自身的限制时，重要的转变就发生了。你能从 2 的 16 次方个字符（一个人可以在几个小时打这么多字）过渡到 2 的 3 次方个字符（远超出了一个人打字的范围）。你可以捕捉比人眼能感知的分辨率更高的数字图像。或者你可以将整个音乐专辑存在小巧的磁盘上，放在口袋里。或者你可以将数字化视频录制存储在硬盘上。再通过实时流式传输的能力，可以在一个地方集中存储一次，不需要在数千个本地硬盘上重复记录。
+
+但有的东西仍然是根本的限制条件，那就是空间的三维和光速。我们又回到了爱因斯坦的宇宙。内存的分级结构将始终存在——它是物理定律的基础。稳定的存储和 IO，内存，计算和通信也都将一直存在。这些模块的相对容量，延迟和带宽将会改变，但是系统始终要考虑这些元素如何组合在一起，以及它们之间的平衡和折衷。Jim Gary 是这方面的大师。
+
+空间和光速的根本限制造成的另一个后果是，性能分析主要是关于三件事：局部化 (locality)，局部化，局部化。无论是将数据打包在磁盘上，管理处理器缓存的层次结构，还是将数据合并到通信数据包中，数据如何打包在一起，如何在一段时间内从局部获取数据，数据如何在组件之间传输数据是性能的基础。把重点放在减少管理数据的代码上，增加空间和时间上的局部性，是消除噪声的好办法。
+
+Jon Devaan 曾经说过：“设计数据，而不是设计代码”。这也通常意味着当查看系统结构时，我不太关心代码如何交互——我想看看数据如何交互和流动。如果有人试图通过描述代码结构来解释一个系统，而不理解数据流的速率和数量，他们就不了解这个系统。
+
+内存的层级结构也意味着我缓存将会一直存在——即使某些系统层正在试图隐藏它。缓存是根本的，但也是危险的。缓存试图利用代码的运行时行为，来改变系统中不同组件之间的交互模式。它们需要对运行时行为进行建模，即使模型填充缓存并使缓存失效，并测试缓存命中。如果模型由于行为改变而变差或变得不佳，缓存将无法按预期运行。一个简单的指导方针是，缓存必须被检测——由于应用程序行为的改变，事物不断变化的性质和组件之间性能的平衡，缓存的行为将随着时间的推移而退化。每一个老程序员都有缓存变糟的经历。
+
+我很幸运，我的早期职业生涯是在互联网的发源地之一 BBN 度过的。 我们很自然地将将异步组件之间的通信视为系统连接的自然方式。流量控制和队列理论是通信系统的基础，更是任何异步系统运行的方式。流量控制本质上是资源管理（管理通道的容量），但资源管理是更根本的关注点。流量控制本质上也应该由端到端的应用负责，所以用端到端的方式思考异步系统是自然的。[缓冲区膨胀][8]的故事在这种情况下值得研究，因为它展示了当对端到端行为的动态性以及技术“改进”（路由器中更大的缓冲区）缺乏理解时，在整个网络基础设施中导致的长久的问题。
+
+ 我发现“光速”的概念在分析任何系统时都非常有用。光速分析并不是从当前的性能开始分析，而是问“这个设计理论上能达到的最佳性能是多少？”真正传递的信息是什么，以什么样的速度变化？组件之间的底层延迟和带宽是多少？光速分析迫使设计师深入思考他们的方法能否达到性能目标，或者否需要重新考虑设计的基本方法。它也迫使人们更深入地了解性能在哪里损耗，以及损耗是由固有的，还是由于一些不当行为产生的。从构建的角度来看，它迫使系统设计人员了解其构建的模块的真实性能特征，而不是关注其他功能特性。
+
+我的职业生涯大多花费在构建图形应用程序上。用户坐在系统的一端，定义关键的常量和约束。人类的视觉和神经系统没有经历过指数性的变化。它们固有地受到限制，这意味着系统设计者可以利用（必须利用）这些限制，例如，通过虚拟化（限制底层数据模型需要映射到视图数据结构中的数量），或者通过将屏幕更新的速率限制到人类视觉系统的感知限制。
+
+复杂性的本质
+
+我的整个职业生涯都在与复杂性做斗争。为什么系统和应用变得复杂呢？为什么在一个应用领域内进行开发并没有随着时间变得简单，而基础设施却没有变得更复杂，反而变得更强大了？事实上，管理复杂性的一个关键方法就是“走开”然后重新开始。通常新的工具或语言迫使我们从头开始，这意味着开发人员将工具的优点与从新开始的优点结合起来。从新开始是重要的。这并不是说新工具，新平台，或新语言可能不好，但我保证它们不能解决复杂性增长的问题。控制复杂性的最简单的方法就是用更少的程序员，建立一个更小的系统。
+
+当然，很多情况下“走开”并不是一个选择——Office 建立在有巨大的价值的复杂的资源上。通过 OneNote， Office 从 Word 的复杂性上“走开”，从而在另一个维度上进行创新。Sway 是另一个例子， Office 决定从限制中跳出来，利用关键的环境变化，抓住机会从底层上采取全新的设计方案。我们有 Word，Excel，PowerPoint 这些应用，它们的数据结构非常有价值，我们并不能完全放弃这些数据结构，它们成为了开发中持续的显著的限制条件。
+
+我受到 Fred Brook 讨论软件开发中的意外和本质的文章[《没有银子弹》][9]的影响，他希望用两个趋势来尽可能地推动程序员的生产力：一是在选择自己开发还是购买时，更多地关注购买——这预示了开源社区和云架构的改变；二是从单纯的构建方法转型到更“有机”或者“生态”的增量开发方法。现代的读者可以认为是向敏捷开发和持续开发的转型。但那篇文章可是写于 1986 年！
+
+我很欣赏 Stuart Kauffman 的在复杂性的基本性上的研究工作。Kauffman 从一个简单的布尔网络模型（“[NK 模型][10]”）开始建立起来，然后探索这个基本的数学结构在相互作用的分子，基因网络，生态系统，经济系统，计算机系统（以有限的方式）等系统中的应用，来理解紧急有序行为的数学基础及其与混沌行为的关系。在一个高度连接的系统中，你固有地有一个相互冲突的约束系统，使得它（在数学上）很难向前发展（这被看作是在崎岖景观上的优化问题）。控制这种复杂性的基本方法是将系统分成独立元素并限制元素之间的相互连接（实质上减少 NK 模型中的“N”和“K”）。当然对那些使用复杂隐藏，信息隐藏和数据抽象，并且使用松散异步耦合来限制组件之间的交互的技术的系统设计者来说，这是很自然的。
+
+
+我们一直面临的一个挑战是，我们想到的许多拓展系统的方法，都跨越了所有的方面。实时共同编辑是 Office 应用程序最近的一个非常具体的（也是最复杂的）例子。
+
+我们的数据模型的复杂性往往等同于“能力”。设计用户体验的固有挑战是我们需要将有限的一组手势，映射到底层数据模型状态空间的转换。增加状态空间的维度不可避免地在用户手势中产生模糊性。这是“[纯数学][11]”，这意味着确保系统保持“易于使用”的最基本的方式常常是约束底层的数据模型。
+
+管理
+
+我从高中开始着手一些领导角色（学生会主席！），对承担更多的责任感到理所当然。同时，我一直为自己在每个管理阶段都坚持担任全职程序员而感到自豪。但 Office 的开发副总裁最终还是让我从事管理，离开了日常的编程工作。当我在去年离开那份工作时，我很享受重返编程——这是一个出奇地充满创造力的充实的活动（当修完“最后”的 bug 时，也许也会有一点令人沮丧）。
+
+尽管在我加入微软前已经做了十多年的“主管”，但是到了 1996 年我加入微软才真正了解到管理。微软强调“工程领导是技术领导”。这与我的观点一致，帮助我接受并承担更大的管理责任。
+
+主管的工作是设计项目并透明地推进项目。透明并不简单，它不是自动的，也不仅仅是有好的意愿就行。透明需要被设计进系统中去。透明工作的最好方式是能够记录每个工程师每天活动的产出，以此来追踪项目进度（完成任务，发现 bug 并修复，完成一个情景）。留意主观上的红/绿/黄，点赞或踩的仪表板。
+
+我过去说我的工作是设计反馈回路。独立工程师，经理，行政人员，每一个项目的参与者都能通过分析记录的项目数据，推进项目，产出结果，了解自己在整个项目中扮演的角色。最终，透明化最终成为增强能力的一个很好的工具——管理者可以将更多的局部控制权给予那些最接近问题的人，因为他们对所取得的进展有信心。这样的话，合作自然就会出现。
+
+Key to this is that the goal has actually been properly framed (including key resource constraints like ship schedule). Decision-making that needs to constantly flow up and down the management chain usually reflects poor framing of goals and constraints by management.
+
+关键需要确定目标框架（包括关键资源的约束，如发布的时间表）。如果决策需要在管理链上下不断流动，那说明管理层对目标和约束的框架不好。
+
+当我在 Beyond Software 工作时，我真正理解了一个项目拥有一个唯一领导的重要性。原来的项目经理离职了（后来从 FrontPage 雇佣了我）。我们四个主管在是否接任这个岗位上都有所犹豫，这不仅仅由于我们都不知道要在这家公司坚持多久。我们都技术高超，并且相处融洽，所以我们决定以同级的身份一起来领导这个项目。然而这槽糕透了。有一个显而易见的问题，我们没有相应的战略用来在原有的组织之间分配资源——这应当是管理者的首要职责之一！当你知道你是唯一的负责人时，你会有很深的责任感，但在这个例子中，这种责任感缺失了。我们没有真正的领导来负责统一目标和界定约束。
+
+我有清晰地记得，我第一次充分认识到*倾听*对一个领导者的重要性。那时我刚刚担任了 Word，OneNote，Publisher 和 Text Services 团队的开发经理。关于我们如何组织文本服务团队，我们有一个很大的争议，我走到了每个关键参与者身边，听他们想说的话，然后整合起来，写下了我所听到的一切。当我向其中一位主要参与者展示我写下的东西时，他的反应是“哇，你真的听了我想说的话”！作为一名管理人员，我所经历的所有最大的问题（例如，跨平台和转型持续工程）涉及到仔细倾听所有的参与者。倾听是一个积极的过程，它包括：尝试以别人的角度去理解，然后写出我学到的东西，并对其进行测试，以验证我的理解。当一个关键的艰难决定需要发生的时候，在最终决定前，每个人都知道他们的想法都已经被听到并理解（不论他们是否同意最后的决定）。
+
+在 FrontPage 担任开发经理的工作，让我理解了在只有部分信息的情况下做决定的“操作困境”。你等待的时间越长，你就会有更多的信息做出决定。但是等待的时间越长，实际执行的灵活性就越低。在某个时候，你仅需要做出决定。
+
+设计一个组织涉及类似的两难情形。您希望增加资源领域，以便可以在更大的一组资源上应用一致的优先级划分框架。但资源领域越大，越难获得作出决定所需要的所有信息。组织设计就是要平衡这两个因素。软件复杂化，因为软件的特点可以在任意维度切入设计。Office 已经使用[共享团队][12]来解决这两个问题（优先次序和资源），让跨领域的团队能与需要产品的团队分享工作（增加资源）。
+
+随着管理阶梯的提升，你会懂一个小秘密：你和你的新同事不会因为你现在承担更多的责任，就突然变得更聪明。这强调了整个组织比顶层领导者更聪明。赋予每个级别在一致框架下拥有自己的决定是实现这一目标的关键方法。听取并使自己对组织负责，阐明和解释决策背后的原因是另一个关键策略。令人惊讶的是，害怕做出一个愚蠢的决定可能是一个有用的激励因素，以确保你清楚地阐明你的推理，并确保你听取所有的信息。
+
+结语
+
+我离开大学寻找第一份工作时，面试官在最后一轮面试时问我对做“系统”和做“应用”哪一个更感兴趣。我当时并没有真正理解这个问题。在软件技术栈的每一个层面都会有趣的难题，我很高兴深入研究这些问题。保持学习。
+
+--------------------------------------------------------------------------------
+
+via: https://hackernoon.com/education-of-a-programmer-aaecf2d35312
+
+作者：[ Terry Crowley][a]
+译者：[explosic4](https://github.com/explosic4)
+校对：[校对者ID](https://github.com/校对者ID)
+
+本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译，[Linux中国](https://linux.cn/) 荣誉推出
+
+[a]:https://hackernoon.com/@terrycrowley
+[1]:https://medium.com/@terrycrowley/leaky-by-design-7b423142ece0#.x67udeg0a
+[2]:https://medium.com/@terrycrowley/how-to-think-about-cancellation-3516fc342ae#.3pfjc5b54
+[3]:http://queue.acm.org/detail.cfm?id=2462076
+[4]:http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf
+[5]:https://medium.com/@terrycrowley/model-view-controller-and-loose-coupling-6370f76e9cde#.o4gnupqzq
+[6]:https://en.wikipedia.org/wiki/Termcap
+[7]:http://www.ll.mit.edu/HPEC/agendas/proc04/invited/patterson_keynote.pdf
+[8]:https://en.wikipedia.org/wiki/Bufferbloat
+[9]:http://worrydream.com/refs/Brooks-NoSilverBullet.pdf
+[10]:https://en.wikipedia.org/wiki/NK_model
+[11]:https://medium.com/@terrycrowley/the-math-of-easy-to-use-14645f819201#.untmk9eq7
+[12]:https://medium.com/@terrycrowley/breaking-conways-law-a0fdf8500413#.gqaqf1c5k