​
🐦 What is Glide?

Glide is a lightweight blazing-fast cloud-native open-source LLM gateway, delivering high-performance LLMOps in one single binary. Glide provides a simplified way to build application resilience, reduce latency, and manage API keys among many other features.

Glide is currently under development and the docs are updated frequently.

​
Why Glide?

The True Gateway

Focus on your business logic and application while letting Glide handle common LLMOps problems (like fallbacks, cost control, routing, caching, etc.)

Blazing-Fast & Cloud-Native

High performance and scalability are our priority, so we don’t slow you down and scale out as you grow.

Resilient

Glide is designed around resiliency and failures, so your application can achieve high availability working with external models and survive the toughest real-world situations.

Configurable

A Glide cluster can serve multiple applications with totally different needs.

No Vendor Lock-Ins

Switch your model providers seamlessly for your application without a need to modify them.

Open Source

We welcome all kinds of feedback and contributions in Glide. Let’s make it work in the best possible way for you!

​
Features

  • Unified API: Change your model providers without a need to modify your applications
  • High Performance: Written in Golang with the best practices of software development in a cloud-native environments in mind
  • Flexible routing across several models and various strategies like the least latency, weighted round-robin, round-robin, priority-based routing. All strategies include low-latency fallback mechanisms to ensure resiliency
  • Support Popular LLM providers
  • Centralized configuration and API key management
  • Production-ready Observability: Always know what’s going on with the heart of your system via logs, metrics, and traces. OpenTelemetry is natively supported (coming soon).
  • and much more is coming

​
Installation

Docker

Get started with Glide in a few minutes with Docker (recommended)

Command Line Interface (CLI)

Install Glide locally into your developer machine or bare metal setup via CLI (MacOS, Linux)

Kubernetes

Install Glide into a Kubernetes cluster

​
License

Glide is provided under Apache 2.0 open source license.

Developed with ❤️ by Roma, Max, and the rest of the EinStack Community.

Was this page helpful?

Docker