๐Ÿช™

Know what every dbt model actually costs.

Penny reads BigQuery's own record of every query it ran, traces each one back to the dbt model that caused it, and turns bytes into dollars. Per model, per day, with trends and anomaly flags.

After every run

A cost summary where you are already looking

Penny prints what the run cost to your console, split into the models themselves and the hooks around them.

dbt build --select penny
$ dbt build --select penny

โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
  ๐Ÿช™  Penny โ€” Run Summary
โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
  Models run:        5
  Seeds loaded:      1
  Tests run:         35
  Queries executed:  40 (5 build, 35 hook, 0 overhead)
  Total TB billed:   0.0004 TB
  Estimated cost:    $0.002283 USD
    builds:          $0.000376
    hooks:           $0.001907
    overhead:        $0.0
  Largest model:     0.02 GB (stg_bigquery__job_history)
โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
Why bother

Cost you can act on

A warehouse bill tells you what you spent. It does not tell you which model to look at on Monday morning.

Per model, not per run

Every job is traced to the model that caused it, including the hooks that ran around it and the temporary tables an incremental build creates.

Trends and anomalies

Daily cost against a rolling 30-day average, a weekly trend per model, and a red, amber or green flag when a model costs more than it should.

Useful with no setup

Penny works with zero labels and gets sharper as you give it more identity. Two lines of configuration is the whole installation.

Mixed billing handled

On-demand and reserved jobs are priced separately, job by job, so a project that mixes both is still costed correctly.

History that outlives BigQuery's

BigQuery keeps job history for 180 days. Penny's tables do not expire, so running it daily is how you keep cost history for longer.

Nothing to run

No agent, no scheduler, no external service. It is a dbt package that reads metadata already sitting in your warehouse.

Install

Three steps

Configuration only. No change to the SQL you run or the data it produces.

Add the package

In packages.yml:

packages:
  - git: "https://github.com/alexander-caldwell/penny.git"
    revision: v0.1.10

Label your jobs

In dbt_project.yml. The query comment stamps each BigQuery job with the model that ran it; the hook prints the summary.

query-comment:
  comment: "{{ penny.penny_query_comment(node) }}"
  job-label: true

on-run-end:
  - "{{ penny.log_run_costs() }}"

Build

Penny writes its own models into a penny schema.

dbt deps
dbt build --select penny

Install v0.1.9 or later. Earlier tags from v0.1.6 read a column that INFORMATION_SCHEMA.JOBS does not have in every region.

What you get

Five tables, raw jobs up to a per-model view

TableOne row perTells you
stg_bigquery__job_history BigQuery job The raw facts, straight from job metadata
int_penny_model_runs BigQuery job Cost per job, tagged build, hook or overhead
rpt_penny_cost_summary Model per day Daily cost, 30-day rolling average, anomaly flag
rpt_penny_cost_by_layer Layer per day Where spend concentrates across your layers
rpt_penny_model_latest Model Latest state, weekly trend, red/amber/green status
Pricing

How a job becomes a number

Two formulas, chosen per job by whether it ran inside a reservation.

On-demand

The job ran outside a reservation, so you paid for bytes.

bytes_billed รท 1 TiB ร— price per TiB

Slot-based

The job ran inside one, so you paid for time.

slot_ms รท 1 hour ร— price per slot-hour

Prices come from your configuration, not from the warehouse, so set them to your contracted rate. Penny reports gross cost: BigQuery's free tier is account-level and cannot be split across models.

Read next

Go deeper

How Penny works

What it reads, how a job is traced back to a model, why one model produces many jobs, and how cost is worked out.

README

Every configuration variable, the full column reference for each table, and troubleshooting.