Personal information & call-site provenance

ScaiLog treats personal information as a field-level, typed thing you declare at the moment you log it, and it makes every log line traceable to the exact source line that emitted it. This page covers both halves: how you mark a field as PI with pi(), and how the developer-assigned call_site_id turns GDPR remediation into "fix one call site."

Marking a field as PI#

You wrap a value with pi(value, pi_type) inside the fields of a log call. The marker attaches to that one named field — never to the message, never to the whole entry.

1
2
3
4
5
6
7

from scailog import log, pi

log.info(
    "SCAI-DEMO-USER-LOGIN-001", "user logged in",
    subject="usr_8842",
    fields={"method": "oidc", "email": pi("jan@example.nl", "direct_identifier")},
)

Here method is an ordinary plaintext field and email is PI of type direct_identifier. At ingest the policy engine resolves an action for the email field; with the default policy that means the value is encrypted under the subject's key and stored as ciphertext, while method and the message stay in the clear.

PI values must be a scalar (string, number, boolean, null) or a flat list of scalars — you cannot mark a nested object or a list of objects. This keeps every PI field individually encryptable and individually erasable.

The message must stay PI-free#

The free-text message is indexed in plaintext by SQLite FTS5 so you can search your logs. Because the full-text index never covers the encrypted pi_fields, anything you want searchable is, by construction, not encrypted. That is exactly why the message must never contain personal information: put variable and personal data in typed fields, and keep the message a constant template.

scailog-ci enforces this at build time — a message that interpolates a variable (f-string, .format(), %) is rejected, because interpolation is how PII leaks into the searchable message text.

The PI taxonomy#

Every PI marker carries a pi_type from a fixed taxonomy. The type drives the default action the policy engine applies when no more specific rule matches.

An unknown pi_type is rejected at ingest (PI_TYPE_UNKNOWN). credentials and special_category default to drop — they are never stored, even encrypted — while pseudonymous is the one type kept in plaintext by default. These are defaults; a policy can override the action per service and environment.

Subject required for PI#

Any entry that carries a PI marker must also carry a subject= — the data subject the PI belongs to. It is the anchor for encryption (a per-subject key) and for erasure (destroy that key). Emit PI without a subject and the entry is rejected with SUBJECT_REQUIRED, both in the SDK and at the server. For PI-like fields that belong to no natural person, the reserved subject _service is used.

Call-site provenance#

Every entry carries a developer-assigned call_site_id: a stable string constant naming the exact place in your code that logged it (SCAI-DEMO-USER-LOGIN-001 above). It is not derived from runtime data — it is a literal you write once and keep. This is what makes remediation tractable: when an audit finds PI being logged incorrectly, the fix is one call site, and the call_site_id points straight at it.

scailog-ci validates call sites at build time and fails the build on any violation:

• The call_site_id must be a string constant, not computed at runtime.
• It must match ^[A-Z0-9]+(-[A-Z0-9]+){2,}$ — uppercase alphanumeric segments, at least three of them, joined by hyphens.
• It must be unique within the scanned service.
• It must not use the reserved prefixes SCAILOG- or AUTO-. SCAILOG-* is reserved for the system's own entries; AUTO-* marks sites the system auto-named because a valid one wasn't supplied.
• A pi() marker at the site requires subject=, and it must wrap only a scalar or flat list.

The server tracks each observed site in a call_sites table, and scailog-ci can push a manifest of declared sites so the operator can spot drift — for example a site emitting PI it never declared — via scailog sites --diff.

This first-class provenance is design invariant P4: GDPR remediation is "fix one call site," not "grep the whole codebase."