Measure both phases
Distinguish prompt processing from output generation. Record time to first token, output-token timing, concurrency and prompt/output lengths. Cache behaviour and batching should remain visible in any comparison.
Count accepted outcomes
Define what constitutes an acceptable answer or completed task. Rejected outputs and retries still consume resources. A cost-per-token figure is meaningful only with a stated workload, quality threshold and cost boundary.
Plan the full path
Include model loading, data access, CPU work, tool dependencies and operational recovery. Evaluate a representative traffic distribution before sizing production. Our inference tool is arithmetic using supplied measurements; it is not a serving simulator.
Prepare a comparable test
Keep the workload, software version, data path, service objective and cost boundary attached to the result. Changing these assumptions changes what a comparison means.