feat(usage): add feature attribution to microdollar usage#307
feat(usage): add feature attribution to microdollar usage#307pedroheyerdahl merged 8 commits intomainfrom
Conversation
51ebabc to
2fc5096
Compare
ea84e39 to
057da53
Compare
057da53 to
c180567
Compare
1caac0a to
3459b8e
Compare
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Clean implementation of feature attribution for microdollar usage tracking. The PR adds a Files Reviewed (18 files)
|
| @@ -1 +1,15 @@ | |||
| export { POST } from '@/app/api/openrouter/[...path]/route'; | |||
| import { NextRequest } from 'next/server'; | |||
There was a problem hiding this comment.
@jrf0110 I changed the gateway route from a re-export to a thin wrapper that clones the request with X-KILOCODE-FEATURE: direct-gateway before forwarding to the openrouter handler. This lets us positively identify external gateway consumers in microdollar_usage without depending on them to send any headers. Tested locally and confirmed the feature value is stored correctly. Any concerns with this approach or the request cloning pattern?
There was a problem hiding this comment.
I'm not so sure about relying on that the route. There are currently direct-gateway users that use the openrouter endpoint
There was a problem hiding this comment.
@jrf0110 Thanks for flagging. Is it actually intended behavior or is that just where we ended up?
If external consumers can hit both /api/gateway/ and /api/openrouter/, we lose a clean way to identify direct gateway traffic. Ideally, each feature should have its own positive signal so we don’t end up with misattribution or silent failure modes.
Can we enforce separation moving forward? If not, what’s the right way to make direct gateway traffic reliably identifiable without relying on a catch all fallback from excluding all other known internal features signals?
There was a problem hiding this comment.
Is it actually intended behavior or is that just where we ended up?
I suppose it's both. I think we never should have had our gateway route at /api/openrouter, but that's the way it was. Clients (Kilo or otherwise) hit the gateway all the same. IMO kilo feature clients should hit the gateway with some identifying header (e.g. x-kilocode-feature). Of course this is spoofable, so it should only be used for reporting purposes
There was a problem hiding this comment.
I agree we should not introduce behavioral differences between the endpoints (and we should get rid of the poorly named openrouter endpoint eventually).
There was a problem hiding this comment.
kilo feature clients should hit the gateway with some identifying header
That's exactly what this PR along with others in kilo and kilocode repo is doing. The open question is specifically about direct-gateway. It's a feature that needs a positive signal just like the others. Since we can't rely on external consumers to send the header, how should we identify their traffic? I'd appreciate your help defining that so we don't end up using "no header" as a catch-all, which would mask tracking issues.
There was a problem hiding this comment.
reverted the wrapper. will create an issue for direct-gateway usage attribution to unblock the current work
Add `feature` column to `microdollar_usage` to track which product generates each token usage record. Every caller sends an `X-KILOCODE-FEATURE` header validated against an allow-list. - Create `src/lib/feature-detection.ts` with FEATURE_VALUES allow-list, FEATURE_HEADER constant, and validateFeatureHeader() function - Add nullable `feature` text column to microdollar_usage with partial index on non-null values - Wire feature extraction into openrouter and FIM gateway routes - Replace gateway re-export with wrapper that injects `direct-gateway` - Add `feature` field to ProxiedChatCompletionRequest and set it in security-agent and slack-bot callers - Pass KILOCODE_FEATURE env var through cloud-agent server-manager - Update processUsage pipeline, schema, view, migration, and all test helpers - Add implementation plan in plans/microdollar-feature-tracking.md
Migrate the `feature` text column from `microdollar_usage` core table to a dedicated `feature` lookup table with a `feature_id` foreign key on `microdollar_usage_metadata`, consistent with other normalized columns (editor_name, finish_reason, etc.). - Create `feature` lookup table with unique index - Add `feature_id` to `microdollar_usage_metadata` - Remove `feature` column and its partial index from `microdollar_usage` - Update view to join through the new lookup table - Update insert logic to use upsert CTE pattern for feature
5c02e0a to
08c7e7b
Compare
…t gateway route - Replace manual Set-based feature validation with zod schema parsing - Revert gateway route to a re-export - Update tracking plan: remove DEFAULT_FEATURE, add CLI entry point default logic (cli vs unknown for kilo serve), clarify feature coverage
Adds a nullable
featurecolumn tomicrodollar_usageso we can track which product feature generated each token usage record. This is the backend portion of a cross-repo change.What this does:
featurecolumn tomicrodollar_usagetable andmicrodollar_usage_viewsrc/lib/feature-detection.tswith an allow-list of 14 feature values and avalidateFeatureHeader()function/api/openrouterand/api/fim/completions) read the newX-KILOCODE-FEATUREheader and store the validated valuesendProxiedChatCompletionaccepts an optionalfeaturefield and forwards it as a headercloud-agent-nextserver manager accepts an optionalfeatureparameter to setKILOCODE_FEATUREenv var when launchingkilo serveHow it works:
Callers send
X-KILOCODE-FEATURE: <value>. The gateway validates against the allow-list and stores it. No header = NULL. No fallback heuristics in the write path.Companion changes needed (separate PRs):
Kilo-Org/kilo: AddHEADER_FEATUREconstant to kilo-gateway, readKILOCODE_FEATUREenv varKilo-Org/kilocode: AddX-KiloCode-Featureheader tocustomRequestOptions()andstreamFim()See
plans/microdollar-feature-tracking.mdfor the full implementation plan.