Skip to content
#

serving-infrastructure

Star

Here are 3 public repositories matching this topic...

Language: All
Filter by language
All 3 Python 2 Jupyter Notebook 1

ksm26 / Efficiently-Serving-LLMs

Star 20

Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.

text-generation batch-processing server-optimization model-serving model-acceleration inference-optimization optimization-techniques machine-learning-operations deep-learning-techniques model-inference-service performance-enhancement scalability-strategies serving-infrastructure large-scale-deployment
  • Updated Apr 12, 2024
  • Jupyter Notebook

Improve this page

Add a description, image, and links to the serving-infrastructure topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the serving-infrastructure topic, visit your repo's landing page and select "manage topics."

Learn more