Catalogue
/
Artificial Intelligence and Applied AI
/
vLLM for Production-Grade LLM Serving and Inference Optimization

vLLM for Production-Grade LLM Serving and Inference Optimization

A practical two day technical course for participants who are new to vLLM and want to deploy large language models efficiently in production-oriented environments. The agenda progresses from model serving and runtime architecture through inference optimization, continuous batching, configuration, distributed serving, and scaling. The course focuses on the practical decisions required to achieve high throughput, predictable latency, and reliable operation while maintaining compatibility with modern AI application stacks.

What will you learn?

You will learn how to serve open-weight language models with vLLM, configure the inference engine, and expose models through OpenAI-compatible APIs. You will understand how PagedAttention, continuous batching, prefix caching, quantization, and parallel execution improve inference efficiency. You will also learn how to configure, benchmark, scale, monitor, and operate vLLM services for demanding production workloads.

  • Deploy language models through vLLM and its OpenAI-compatible serving interface
  • Configure batching, memory management, caching, quantization, and inference parameters
  • Scale model serving across multiple GPUs and serving instances
  • Monitor and optimize throughput, latency, reliability, and resource utilization

Requirements:

  • Technical background in software development, infrastructure, DevOps, machine learning, or a related field
  • Basic familiarity with Python, command-line tools, and HTTP APIs
  • General understanding of large language models and GPU-based inference
  • Previous vLLM experience is not required

Course Outline*:

*We customize the course outline and content to your specific needs and relevant use cases.

Module 1: vLLM architecture and model serving fundamentals

  • Understanding vLLM as a high-throughput inference and serving engine
  • Installing vLLM and selecting compatible models from common model repositories
  • Starting models for offline inference and online serving
  • Understanding model loading, tokenization, generation, and request processing

Module 2: OpenAI-compatible model serving

  • Starting and configuring the vLLM HTTP server
  • Working with chat completions, completions, embeddings, and supported API interfaces
  • Connecting existing applications and OpenAI-compatible clients to vLLM
  • Configuring models, chat templates, generation parameters, and API access

Module 3: PagedAttention, batching, and memory efficiency

  • Understanding KV cache memory and why it becomes a serving bottleneck
  • Understanding how PagedAttention improves KV cache utilization
  • Using continuous batching to process concurrent requests efficiently
  • Understanding prefix caching and chunked prefill for repeated and long-context workloads

Module 4: Inference configuration and optimization

  • Configuring sequence length, GPU memory utilization, batch behavior, and concurrency
  • Understanding quantization options and their impact on memory, quality, and throughput
  • Matching model size and configuration to available GPU resources
  • Balancing throughput, time to first token, inter-token latency, and memory consumption

Module 5: Multi-GPU serving and parallelism

  • Understanding when a model requires or benefits from multiple GPUs
  • Configuring tensor parallelism to distribute model computation across GPUs
  • Introducing pipeline and other parallel execution strategies for larger deployments
  • Choosing parallelism based on model size, hardware topology, and workload characteristics

Module 6: Scaling concurrent inference workloads

  • Using data parallel deployment to increase serving capacity across model replicas
  • Understanding request distribution and batching across serving instances
  • Separating model scaling from application and API scaling
  • Designing scalable serving architectures for variable and high-concurrency workloads

Module 7: Benchmarking, monitoring, and performance tuning

  • Measuring throughput, latency, token generation rates, and concurrency behavior
  • Comparing configurations under representative request patterns
  • Identifying GPU memory pressure, queueing, long prompts, and inefficient batch behavior
  • Using metrics and profiling information to guide configuration changes

Module 8: Production deployment and operational readiness

  • Deploying vLLM behind application gateways, reverse proxies, or container platforms
  • Understanding authentication boundaries, network exposure, and service security
  • Planning health checks, failure recovery, model updates, and capacity management
  • Building a practical production readiness checklist covering performance, scaling, security, observability, and maintenance

Hands-on learning with expert instructors at your location for organizations.

4,122€*
Graph Icon - Education X Webflow Template
Level:
advanced
Clock Icon - Education X Webflow Template
Duration:
14
Hours (days:
2
)
Camera Icon - Education X Webflow Template
Training customized to your needs
Star Icon - Education X Webflow Template
Immersive hands-on experience in a dedicated setting
*Price can range depending on number of participants, change of outline, location etc.

Master new skills guided by experienced instructors from anywhere.

3,097€*
Graph Icon - Education X Webflow Template
Level:
advanced
Clock Icon - Education X Webflow Template
Duration:
14
Hours (days:
2
)
Camera Icon - Education X Webflow Template
Training customized to your needs
Star Icon - Education X Webflow Template
Reduced training costs
*Price can range depending on number of participants, change of outline, location etc.

Upcoming Sessions

27-28 Jan 2027
Brussels
9-10 Mar 2027
Frankfurt
23-24 Mar 2027
London
31 Mar - 1 Apr 2027
Madrid
11-12 May 2027
Milan
25-26 May 2027
Frankfurt
16-17 Jun 2027
Vienna
6-7 Jul 2027
Amsterdam

Can't find a suitable date? Get in touch and we'll arrange one that works for you.

We use cookies to improve site navigation, analyse how the site is used, and support our marketing. You can accept all cookies, reject non-essential ones, or choose individual categories. Read our Cookie Notice