What is Feenion?
Feenion is an open-source, 100% self-hosted observability and debugging platform built specifically for LLM applications, RAG pipelines, and autonomous multi-agent tool loops.
Traditional APMs only capture raw HTTP status codes or generic server errors. When an agent loops infinitely, retrieves irrelevant vector chunks, or hallucinates due to prompt bloat, traditional tools remain blind. Feenion captures the full causal execution graph: exact prompts, completions, tokens, calculated costs, tool arguments, and retrieval scores.
Docker Quickstart
Spin up the complete Feenion server and React dashboard with embedded SQLite WAL storage in one command:
git clone https://github.com/DarshanAguru/feenion.git
cd feenion
docker compose up -d
Open http://localhost:8000 in your browser to view the live dashboard.
Python SDK Setup
Install the official lightweight Python SDK in your application environment:
pip install feenion
Your First 5-Minute Trace
Configure the client and annotate your functions with @trace and span():
from feenion import trace, span, configure
# Point SDK to your local or self-hosted server
configure(server_url="http://localhost:8000")
@trace(name="customer_support_agent", span_type="agent")
def handle_query(user_query: str):
with span("vector_kb_search", span_type="retrieval", input={"q": user_query}):
docs = ["Reset password at /settings", "API keys are under Security"]
with span("llm_synthesis", span_type="llm"):
return f"Answer for {user_query}: {docs[0]}"
handle_query("How do I reset my password?")
Traces, Spans & Events Data Model
Feenion models execution as a hierarchical Directed Acyclic Graph (DAG):
- Trace: The top-level root operation representing the end-to-end request (e.g. an API endpoint or agent task).
- Span: A timed unit of work within a trace (e.g., LLM generation, vector search, tool execution, function call).
- Event: An instantaneous timestamped milestone attached to a span (e.g. streaming first token received).
Context Propagation
Feenion uses Python's native contextvars.ContextVar to maintain the current active trace and parent span IDs across nested sync and async calls without requiring manual ID passing.
Pluggable Telemetry Exporters
Feenion provides 5 built-in, decoupled exporters for any production or testing setup:
- AsyncExporter(inner): Non-blocking daemon worker with an in-memory queue flushing batches every 500ms without slowing down request loops.
- HTTPExporter(endpoint, api_key, project_id): Direct HTTP ingestion transport to the Feenion server with tenant API key authentication.
- JSONLExporter(filepath): Structured append-only local file logging for air-gapped environments, CI pipelines, and offline trace replay.
- ConsoleExporter(verbose): Real-time formatted terminal output for local developer interactive debugging.
- CompositeExporter([exporters]): Broadcasts telemetry to multiple targets simultaneously (e.g. streaming to remote HTTP while saving local JSONL audit files).
from feenion import configure
from feenion.exporters import AsyncExporter, HTTPExporter, CompositeExporter, JSONLExporter
# Composite: Async HTTP upload + local JSONL audit trail
configure(
exporter=CompositeExporter([
AsyncExporter(HTTPExporter("http://localhost:8000", api_key="feenion_key", project_id="finance-prod")),
JSONLExporter("traces_audit.jsonl"),
])
)
Multi-Tenant Workspace Routing & Authentication
Telemetry in Feenion is routed strictly using each workspace's unique Workspace ID (obtained from the Settings page):
API key is optional for local development on the same PC. When deploying Feenion in production with authentication enabled, provide your workspace's API key.
import feenion
from feenion import trace
# Option 1: Global Workspace Configuration by Workspace ID
feenion.configure(
server_url="http://localhost:8000",
workspace_id="60d03b94-82a1-4328-874e-7b5fbfbc4402",
api_key="fn_live_...", # Optional for local instance, required for auth
)
# Option 2: Dynamic Per-Agent / Per-Trace Workspace Routing (with Auth)
@trace(
name="compliance_scanner",
workspace_id="60d03b94-82a1-4328-874e-7b5fbfbc4402",
api_key="fn_live_...",
)
def run_compliance():
# Or attach dynamically inside any function:
feenion.set_workspace_id("60d03b94-82a1-4328-874e-7b5fbfbc4402")
feenion.set_api_key("fn_live_...")
return "completed"
@trace & @async_trace Decorators
Decorate any synchronous or asynchronous Python function to automatically track execution:
from feenion import async_trace
@async_trace(name="async_pipeline", span_type="agent")
async def process_stream(payload: dict):
await asyncio.sleep(0.05)
return {"status": "success"}
span() Context Manager
Use the span() context manager for sub-operations:
with span("db_lookup", span_type="tool", input={"user_id": 42}) as s:
result = db.query(...)
s.set_output(result)
s.set_metric("rows_returned", len(result))
Token & Cost Tracking • Multi-Currency Support
Feenion contains built-in pricing tables for standard models (e.g. GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash, DeepSeek). You can also set custom token usage and model overrides in code or via the Web Dashboard:
s.set_tokens(prompt_tokens=512, completion_tokens=128)
s.set_attribute("model", "gpt-4o")
# Override pricing directly in code (per 1M tokens):
from feenion import set_model_pricing
set_model_pricing("my-fine-tuned-model", prompt_per_1m=1.20, completion_per_1m=3.80)
In the Web Dashboard under Settings → Currency & FX Display Preferences, you can switch between USD ($), INR (โน), EUR (โฌ), GBP (ยฃ), CNY (ยฅ), and JPY (ยฅ) with customizable exchange rates, and edit token rates for any model with a live simulation calculator.
Client-Side Sensitive PII & Secret Redaction
Mask sensitive fields in payload dicts before they leave application memory:
from feenion.redaction import Redactor
redactor = Redactor(sensitive_keys={"api_key", "password", "auth_token", "ssn"})
safe_payload = redactor.redact(raw_payload)
Azure OpenAI & Azure AI Foundry
Wrap Azure OpenAI (openai.AzureOpenAI or LangChain langchain_openai.AzureChatOpenAI) in one call:
from langchain_openai import AzureChatOpenAI
from feenion.integrations import wrap_azure_openai
# Auto-instruments .invoke(), .ainvoke(), tokens, and pricing
llm = wrap_azure_openai(AzureChatOpenAI(
azure_endpoint="https://my-resource.openai.azure.com/",
api_key="AZURE_API_KEY",
azure_deployment="gpt-4o",
api_version="2024-02-01",
))
response = llm.invoke([("user", "Analyze compliance report #1024")])
OpenAI Auto-Instrumentation
Wrap the official OpenAI client or LangChain ChatOpenAI:
from openai import OpenAI
from feenion.integrations.openai import wrap_openai
client = wrap_openai(OpenAI())
Google Gemini Auto-Instrumentation
Wrap Google Gemini clients (google-genai or google-generativeai) in one line:
from google import genai
from feenion.integrations.gemini import instrument_gemini
client = genai.Client()
instrument_gemini(client)
response = client.models.generate_content(
model="gemini-2.0-flash",
contents="Explain vector embeddings in machine learning."
)
Anthropic Claude Auto-Instrumentation
from anthropic import Anthropic
from feenion.integrations.anthropic import instrument_anthropic
client = Anthropic()
instrument_anthropic(client)
LangChain & LangGraph Callbacks
from feenion.integrations.langchain import FeenionCallbackHandler
handler = FeenionCallbackHandler(trace_name="rag_chain")
chain.invoke({"topic": "AI Debugging"}, config={"callbacks": [handler]})
RAG & Vector Retrieval Observability
Instrument retrieval spans to capture similarity scores, chunk IDs, and query text:
with span("chroma_kb_query", span_type="retrieval", input={"query": q, "top_k": 5}) as s:
chunks = vector_store.similarity_search_with_score(q, k=5)
s.set_output([{"text": doc.page_content, "score": score} for doc, score in chunks])
Autonomous Agents & Tool Calling
Track multi-turn reasoning loops, tool call arguments, execution outputs, and error recovery:
with span("sql_executor", span_type="tool", input={"sql": query_str}) as s:
s.set_attribute("tool_name", "database_reader")
res = db.execute(query_str)
s.set_output(res)
Waterfall Timeline & Critical Path
The Waterfall Timeline view provides an interactive flamegraph of all child spans. Feenion automatically highlights the critical path that contributed most to the overall request duration, allowing you to instantly spot latency bottlenecks.
D3 Execution DAG Mind Map
The Mind Map view renders the full parent-child causality tree of your AI execution. Link distances scale proportionally to latency, and color-coded nodes represent LLMs (purple), retrievals (amber), tools (cyan), and exceptions (rose).
Semantic Error Intelligence
Feenion clusters similar exceptions using semantic error fingerprinting. Click any fingerprint to see the full stack trace, occurrence frequency, affected models, and deep links to sample traces.
Comparative Trace Regression Diff
Open two traces side-by-side to immediately compare durations, token inflations, cost variances, and prompt differences to determine why a regression occurred.
Production Docker Deployment
Deploy Feenion as a single all-in-one container using SQLite Write-Ahead Logging:
version: '3.8'
services:
feenion:
image: feenion/feenion:latest
ports:
- "8000:8000"
environment:
- FEENION_DATABASE_URL=sqlite:////app/data/feenion.db
- FEENION_RETENTION_DAYS=30
volumes:
- feenion_data:/app/data
restart: unless-stopped
Environment Variables Reference
| Variable | Default | Description |
|---|---|---|
| FEENION_DATABASE_URL | sqlite:////app/data/feenion.db | Database connection string (SQLite WAL or PostgreSQL) |
| FEENION_REDIS_URL | "" | Optional Redis queue URL for distributed workers |
| FEENION_LOG_LEVEL | INFO | Logging level (DEBUG, INFO, WARNING, ERROR) |
| FEENION_RETENTION_DAYS | 30 | Automatic retention period for traces and spans |
Workspaces & Isolation Architecture
Feenion provides built-in multi-tenant workspace isolation, allowing teams to separate telemetry data across environments (e.g. Production, Staging, Dev) or distinct AI microservices under a single self-hosted server.
The Workspace Lifecycle Flow
Create a workspace (e.g. prod-agent-service) via the Web Dashboard Settings or POST /api/v1/projects.
The server generates a unique cryptographic ingestion API key (fn_...) stored as a SHA-256 hash.
Pass the API key to configure(api_key="fn_...") in your Python backend or HTTP exporter.
Switch workspaces instantly in the dashboard top navigation bar to view isolated traces, analytics, and costs.
Configuring Python SDK with Workspace Key
from feenion import configure, trace, span
# Bind all telemetry from this process to your dedicated workspace:
configure(
server_url="http://localhost:8000",
api_key="fn_abc123yourSecretKeyHere...",
)
@trace(name="customer_support_pipeline", span_type="agent")
def run_task(query: str):
with span("kb_lookup", span_type="retrieval"):
pass
๐ Zero Data Bleed
Traces, span error stacks, token costs, and LLM prompts are partitioned by workspace ID at the database layer. One workspace cannot access telemetry from another.
๐๏ธ Protected Cascade Deletion
Deleting a workspace requires entering the exact confirmation text delete workspace and cascades through all associated spans, events, and API keys. The system prevents deleting the last remaining workspace.
REST & WebSocket API Reference
The Feenion server exposes high-performance asynchronous endpoints for ingestion, query filtering, deep analytics, project isolation, and live WebSocket telemetry push.
๐ฅ 1. Ingestion & Live Streaming
Ingests an array of execution traces, child spans, token metrics, and execution events with asynchronous persistence. Supports Gzip compression via Content-Encoding: gzip.
Content-Type: application/json • Content-Encoding: gzip (optional) • X-Feenion-Api-Key: <key> (optional)Response (200):
{"accepted": 10, "schema_version": "1.0"}
Bi-directional WebSocket connection for live telemetry streaming. The server broadcasts events (trace_ingested, trace_deleted, data_cleared) directly to connected dashboard sessions.
๐ 2. Traces & Spans Exploration
List and search traces with multi-dimensional filtering, free-text prompt search, and sorting.
time_window:15m,1h,6h,24h,7d,30d,allstatus:all,ok,errorenvironment:all,production,staging,developmentspan_type:all,llm,retrieval,tool,agentmodel: Filter by LLM model name (e.g.gemini-2.0-flash,gpt-4o)search: Full-text substring search across trace names, prompt inputs, model names, and IDsmin_duration_ms/max_duration_ms: Filter by execution latencysort_by:newest,oldest,slowest,fastest,most_tokens,most_cost,most_spans,errorlimit(1-500, default 100) •offset(default 0)
Retrieves the complete causality tree for a single trace, including child spans, prompt inputs, completions, error stack traces, token breakdowns, and calculated dollar costs.
Returns the ordered flat list of child spans with start timestamps, latencies, and parent IDs for timeline and flamegraph visualization.
Returns fast count of total ingested traces: {"count": 1420}.
๐ 3. Observability & Deep Analytics
Returns system health score (0-100), latency percentiles (p50, p75, p90, p95, p99), KPI deltas vs previous window, time breakdown (LLM vs Retrieval vs Tools vs Other), and automated "What Changed?" regression analysis.
Aggregates performance across LLM providers and models: requests, prompt & completion tokens, total spend, p50/p95 latency, error rates, and average cost per query.
Monitors tool invocations, execution failures, p50/p95 latency per tool, and latest invocation timestamps.
Analyzes vector search and retrieval pipelines: average document chunks retrieved, average cosine/relevance scores, slow retrievals (>1s), and empty result alerts.
Evaluates autonomous agent execution runs: average step count, loop candidate detection (repeated tool calling >3 times), total tokens, and goal completion rates.
Groups exceptions into unique semantic fingerprints with stack traces, first seen / latest occurrence, occurrence counts, and list of affected model names and trace IDs.
๐ข 4. Multi-Tenant Workspaces & API Keys
Lists all active workspaces and their creation dates.
Creates a new workspace and generates a raw API key (feenion_live_...) for SDK telemetry ingestion.
Payload: {"name": "production-llm-service"}
Deletes a workspace and cascade-deletes all its API keys, traces, spans, and telemetry logs. (Safety rule: cannot delete the only remaining workspace).
๐ก๏ธ 5. Admin & Data Maintenance
Permanently purges all database and in-memory telemetry data. Requires explicit confirmation payload {"confirmation": "delete everything"}.
Deletes multiple selected traces by IDs with {"trace_ids": ["..."], "confirmation": "delete selected"}.
Deletes a specific trace and its child spans and events, broadcasting a trace_deleted event over WebSocket.
โ๏ธ 6. System Probes & Health
Liveness probe for load balancers and container orchestrators: {"status": "ok"}.
Readiness check verifying database connection and background worker thread status.
Troubleshooting & FAQ
Does Feenion slow down my application?
No. The Python SDK pushes spans to an in-memory queue, and a background thread flushes them asynchronously. Spans execute with zero network delay on user requests.
Can I use Feenion with Ollama or local LLMs?
Yes. Simply use the span() context manager or OpenAI client wrapper pointed to your local Ollama/vLLM base URL.