Quick Start

Install, configure, run evals, submit results.

1. Install

pip install pramana-ai

Requires Python 3.11+.

2. Set API key

Set the environment variable for your provider:

# OpenAI (recommended for reproducibility)
export OPENAI_API_KEY=sk-...

# Anthropic
export ANTHROPIC_API_KEY=sk-ant-...

# Google Gemini
export GEMINI_API_KEY=AI...

For scientific drift detection, use OpenAI API with explicit keys. It is the only provider with true deterministic output. See Reproducibility.

3. Check available models

$ pramana models
ANTHROPIC
  • claude-opus-4-6
  • claude-sonnet-4-6
  • claude-haiku-4-5

OPENAI
  • gpt-5.2
  • gpt-4o
  • o3-mini
  ...

GOOGLE
  • gemini-2.5-pro
  • gemini-3-flash-preview
  ...

4. Run evals

# Smoke test (10 tests, ~$0.05)
$ pramana run --tier cheap --model gpt-5.2

# Regular monitoring (25 tests, ~$0.25)
$ pramana run --tier moderate --model gpt-5.2

# Full evaluation (75 tests, ~$1.50)
$ pramana run --tier comprehensive --model gpt-5.2

Defaults: temperature=0.0, seed=42, output to results.json.

Short aliases work:

$ pramana run --tier cheap --model opus
# Resolves to claude-opus-4-6

5. Submit results

$ pramana submit results.json
✓ Submitted 10 results

Results go to the public dashboard. No account required.

6. (Optional) Log in

$ pramana login       # GitHub/Google OAuth
$ pramana whoami      # Check status
$ pramana logout      # Clear token
$ pramana delete      # GDPR: delete all your data

Authentication enables personalized tracking but is not required for running or submitting.

Environment variables

Variable	Provider	Required
`OPENAI_API_KEY`	OpenAI	For OpenAI models
`ANTHROPIC_API_KEY`	Anthropic	For Claude models
`GEMINI_API_KEY`	Google	For Gemini models

Alternatively, pass --api-key directly:

$ pramana run --tier cheap --model gpt-5.2 --api-key sk-...