Dropāin proxy with caching across providers
OpenAIācompatible endpoint with intelligent caching, multiāprovider routing, and automatic failover. Zero code changes required.
Up to 70% Faster
Potential response time improvement with intelligent caching and provider optimization
99.9% Uptime Goal
Multi-provider failover designed with health checks and circuit breakers
5+ Providers
Unified interface supporting OpenAI, Anthropic, Google, Mistral, and more
100% OpenAI Compatible
Drop-in replacement for OpenAI's API with zero code changes. Simply update your base URL and add your Bastio API key header.
Supported Endpoints
Chat completions with streaming support
Legacy text completions
Text embeddings generation
Available models listing
Quick Integration
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
baseURL: "https://api.openai.com/v1"
});const openai = new OpenAI({
apiKey: "unused", // Provider keys managed by Bastio
baseURL: "https://api.bastio.com/v1",
defaultHeaders: {
"X-API-Key": process.env.BASTIO_API_KEY
}
});Intelligent Multi-Provider Routing
Automatic failover, load balancing, and cost optimization across multiple AI providers with configurable routing policies.
OpenAI
- ā¢ GPT-4, GPT-4 Turbo, GPT-3.5
- ā¢ Primary for chat completions
- ā¢ Best performance for code generation
Anthropic
- ā¢ Claude 3, Claude 2.1, Claude Instant
- ā¢ Excellent for long-form content
- ā¢ Strong reasoning capabilities
- ā¢ Gemini Pro, Gemini Ultra
- ā¢ Multimodal capabilities
- ā¢ Cost-effective for high volume
Routing Strategies
Cost-Optimized
Route to the lowest-cost provider that meets quality requirements
Performance-First
Prioritize fastest response times and lowest latency
Quality-Based
Route based on model capabilities for specific task types
Load Balancing
Distribute requests evenly to prevent rate limiting
Intelligent Response Caching
Advanced caching algorithms reduce costs and improve performance with semantic similarity matching and configurable TTL policies.
Semantic Caching
Match similar queries using embedding-based similarity, not just exact string matching.
- ā¢ Vector similarity scoring
- ā¢ Configurable similarity thresholds
- ā¢ Context-aware matching
Smart TTL Policies
Adaptive cache expiration based on content type, user patterns, and update frequency.
- ā¢ Dynamic TTL adjustment
- ā¢ Content-type specific policies
- ā¢ User behavior analysis
Cache Performance Metrics
Cache Configuration
{
"cache_policy": "semantic",
"similarity_threshold": 0.85,
"ttl_minutes": 60,
"max_cache_size": "10GB",
"exclude_patterns": ["password", "token"]
}Advanced Streaming Support
Full support for Server-Sent Events (SSE) streaming with security analysis, caching, and provider failover for real-time applications.
Streaming Features
- Real-time security analysis on streaming content
- Partial response caching for common prefixes
- Seamless failover during streaming
- Backpressure handling and flow control
Example Implementation
const stream = await openai.chat.completions.create({
model: "gpt-4",
messages: [{ role: "user", content: "Hello" }],
stream: true
});
for await (const chunk of stream) {
// Security analysis happens in real-time
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}Reliability & Circuit Breakers
Health Monitoring
Continuous health checks across all providers with intelligent circuit breaker patterns that prevent cascading failures.
Example Target Scenarios
SaaS Customer Support Use Case
Designed for customer success platforms to reduce AI costs while improving response times through intelligent caching and provider optimization.
- ā¢ Target: 80%+ cache hit rate on common queries
- ā¢ Goal: Sub-3s average response times
- ā¢ Multi-provider failover for high availability
- ā¢ Potential: $10K+ monthly savings
EdTech Content Generation
Built for educational platforms targeting 99.9% uptime and 40%+ cost reduction through multi-provider routing and semantic caching.
- ā¢ Seamless failover during provider outages
- ā¢ Target: 70%+ cache hit rate on content
- ā¢ Simplified provider integration management
- ā¢ Target: 100ms+ latency improvement
Ready to Optimize Your AI Gateway?
Start with a drop-in replacement that reduces costs, improves performance, and eliminates provider downtime risk.