Docs / Core concepts

The PII registry: how resources get declared and protected

Before Chameleon can encrypt or delete anything, it needs to know what counts as PII and where it lives. The registry is that map — and it's built additively from three different sources, so nothing you already declared gets silently overwritten as coverage grows.


One registry, three ways in

Manual, dbt-managed, and discovered — composed, not competing

A resource can enter the registry three different ways, and all three compose on top of each other rather than fighting for the same slot. A manual declaration always wins if one exists; dbt-managed and discovered entries fill in the gaps around it, and none of them overwrite a manual declaration once it's made.

This matters in practice: you can start with a handful of manually declared tables, turn on the dbt package to auto-cover your transformation layer, and later enable the discovery crawler to catch anything both of those missed — without redoing earlier work or risking a declaration getting clobbered by a later automated pass.


The three sources

How a resource actually gets declared

Manual

You declare a resource directly — through the console's declare panel or the API — specifying its fields, classification, and how it should be handled. This is the fastest path for a specific table you already know matters.

dbt-managed

The chameleon_pii dbt package inspects your dbt project's own graph metadata and auto-registers the resources it finds, layered on top of (never replacing) whatever you've already declared manually.

Discovery crawler

A scheduled crawl diffs your actual warehouse schema against the registry, surfacing tables that are undeclared or have drifted — new columns, removed columns, changed types — as a queue you review and declare with one click.


What a declaration contains
A resource declaration
{
  "resourceId": "bigquery:acme-corp.analytics.users",
  "system": "bigquery",
  "resourceLayer": "STAGING",
  "visibility": "CUSTOMER_FACING",
  "tenantIdColumn": "tenant_id",
  "userIdColumn": "user_id",
  "piiFields": [
    { "name": "email", "classification": "DIRECT_IDENTIFIER", "handling": "ENCRYPT" },
    { "name": "signup_ip", "classification": "INDIRECT_IDENTIFIER", "handling": "TOKENIZE" }
  ],
  "deletionStrategy": "CRYPTO_SHRED"
}

Field vocabulary

What the fields actually mean

FieldValuesWhat it controls
resourceLayerRAW, STAGING, INTERMEDIATE, MART, SAASWhere in your pipeline this resource sits — used to set sensible defaults, like excluding raw ingestion tables from ghost-data scans by default.
visibilityCUSTOMER_FACING, INTERNALWhether this is policy metadata meant to be reviewed by a compliance team, or an implementation-only location Chameleon manages internally.
handlingENCRYPT, TOKENIZE, REDACT, HASH_SURROGATE, ALLOW_AGGREGATE_ONLY, MANUAL_REVIEWHow a declared field should actually be protected.
deletionStrategyCRYPTO_SHRED, DELETE_ROWS, REDACT_FIELDS, EXTERNAL_WIPE, MANUAL_REVIEWWhat happens to this resource's data when a user is deleted.

Ghost data

Ghost data is PII the discovery crawler finds somewhere outside the path you'd expect — a table nobody declared, holding real personal data. Findings feed directly into deletion planning and the coverage score below, so undeclared PII doesn't just sit there invisibly until an audit finds it first.


Coverage score

A single number for "how protected are we, really"

Every declared and discovered resource rolls up into one coverage score, bucketed as protected, partial, or exposed. It's a pure read over the registry, policy evaluation, and discovery queue — nothing extra to configure.

The score is meant to move in both directions honestly: it drops the moment the crawler finds a new undeclared table, and rises again once that table is actually declared and protected — a live signal instead of a point-in-time audit finding.


FAQ

Common questions

What happens if a manual declaration and a dbt-managed one disagree?

The manual declaration always wins. dbt-managed and discovered entries only fill gaps — they never overwrite something you declared by hand, so automating coverage later never risks quietly changing a decision you already made.

Do I have to declare every table myself?

No — that's what the dbt package and discovery crawler are for. A common pattern is declaring a handful of known-important resources manually first, then turning on the other two sources to cover the rest of your warehouse over time.

What's the difference between ghost data and a normal undeclared resource?

They're closely related: an undeclared resource is anything the crawler finds that isn't in the registry yet. "Ghost data" specifically means that resource was found to actually contain real PII — it's the subset of undeclared findings that represent genuine, unprotected exposure, not just administrative gaps.


Keep reading

Control plane vs. data plane, the two encryption schemes, and the single-key-per-user model.