1
Dual-Queue Priority & Decoupled Concurrency
Clients push tasks via Redis or HTTP. Dr.AI prioritizes drai:ai_fast_queue (line-skipping for POST /api/image receipt OCR and urgent prompts) before polling standard drai:ai_queue.
The worker runs decoupled concurrency: 1 task per Gemini API key concurrently plus independent parallel slots for local Ollama and AGY CLI.
2
Flash-First Models & Thinking Level Tiers
Tasks specify complexity as low, medium, or high (or legacy fast/simple/complex).
Dr.AI prioritizes flash-latest models across all tiers, paired with automated thinking_level controls:
- Low Tier (
low/fast): Flash models with minimal thinking level for instant summaries & lookups.
- Medium Tier (
medium/simple): Flash models with medium thinking level for balanced extraction & formatting.
- High Tier (
high/complex): Flash models with high thinking level for deep multi-step logic.
3
Instant Response Caching (<10ms)
Dr.AI calculates deterministic SHA-256 hashes of incoming tasks in Redis (drai:cache:response:<hash>).
Identical prompt requests return immediately in sub-10ms without consuming cloud rate limits, GPU slots, or API quotas.
4
Instant 429 Failover & Local OCI Fallback
When an API key encounters an HTTP 429 Rate Limit, the key/model is marked inactive in Redis and instantly rotates to the next available API key with zero backoff delay.
If all cloud keys are occupied or exhausted, tasks seamlessly fall back to the private OCI AI server running Ollama (llama3.2:1b for low/medium, qwen2.5:3b with thinking for high) or AGY CLI.
5
Real-time WebSocket Streaming & Structured Storage
Task lifecycle events (processing, completed, error) and queue depth updates are broadcast to connected dashboard clients in real time via Socket.io.
Results are logged to Redis transaction history (drai:transactions).