Self-Hosted LLM Inference

Sriram Panyam has spent over 20 years building large-scale distributed systems, cloud and AI platforms and the data analytics that run on them, at Google, LinkedIn, Amazon, several startups and now General Motors. At Google Cloud he led engineering teams working on petabyte-scale workloads and control planes on Cloud Dataflow. At LinkedIn he built and managed identity and analytics platforms serving more than a billion members. He has mentored senior technical leaders, and writes about systems engineering and AI infrastructure.
His recent work is on the Model Context Protocol. He contributes conformance suites to it, the tests that check an implementation against the specification, including the scenarios for the tasks extension and for the accepted skills-server proposal. He also maintains mcpkit, a Go implementation of MCP listed among the project's community SDKs.
He holds engineering degrees from the University of Melbourne.

- PART I: FOUNDATIONS.- Chapter 1: Introduction to Self-Hosted Inference.- Chapter 2: How LLMs Work.- Chapter 3: Hardware Fundamentals.- Chapter 4: Model Formats and Quantization.- Chapter 5: Inference Engines.- Chapter 6: Building the Control Plane (v0.1).- PART II: SERVING REAL WORKLOADS.- Chapter 7: Routing, Queueing, and the Control Plane in the Data Path.- Chapter 8: Stateful Routing and Prefix-Cache Reuse.- Chapter 9: Scaling to 30B+ Models.- PART III: DISTRIBUTED INFERENCE.- Chapter 10: Multi-GPU and Distributed Inference.- Chapter 11: Capacity Across Providers.- Chapter 12: 70B Deployment.- PART IV: LET’S GO BIG.- Chapter 13: Serving a Frontier Model, and What the Market Will Sell You.- Chapter 14: What It Costs, and Why Batching Stops Helping.- Chapter 15: The Ceiling, and What Comes Next.