A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Info
Stars★ 85,411
Forks18,964
Watchers85,411
Open Issues5,548
LicenseApache License 2.0
Last Pushed5h ago
Related AI Projects
View all →The agent that grows with you
★ 209.5kPython
Details →AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
★ 185.4kPython
Details →🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
★ 162.3kPython
Details →