Field catalog
Status: Working registry for firstdraft.foundation-plan.sketch/0.23.
The current format permits twenty Field types, and all eight hand-authored examples pass its JSON Schema. The live Postgres graph persists Counter definitions, Position partition terms, and aggregates as normalized links, and one immutable, target-independent input graph captures every Field and those links. Pure target-independent Analyzers check Counter Association ownership plus Position source ownership and self-partitioning without SQL. Position partition-Field queryability, aggregate ownership, type compatibility and cycles, and target lowering remain absent. The support inventory (repository-only) records which Field kinds the bounded Rails Compiler emits, and the Rails state-transition page states the bounded AASM behavior. An unsupported Field kind is skipped, so its column is not emitted, and the reviewed GapSet lists the omission before Compile.
This page is the living explanation of the Field vocabulary. The JSON Schema defines the shape currently permitted by the format. The data-model chapter explains how Fields relate to the other Plan subjects. The Rails target profile records target-specific lowering directions.
What a Field means
A Field is one named application value owned by an Entity. It is not merely a database column.
For example, Oscar Party describes a production budget like this:
{
"subject_uuid": "019f9425-5412-73f5-890a-233db81aa267",
"key": "production_budget",
"name": "Production budget",
"type": "money",
"required": false,
"settings": {
"currency": "USD"
}
}
That one Field may lower to a subunit column, a Money-valued model attribute, controls, formatting, query translation, validation, and generated tests. An uploaded image may instead lower to framework-owned attachment records, validation, Cloudinary-optimized delivery, and generated tests.
The Field name should describe product meaning. The target profile decides how to store and implement that meaning when only one supported lowering exists. The semantic Field lowering exercise (repository-only) works through one-value and multi-column consequences without reducing Field to Column.
Several boundaries keep the vocabulary focused:
- A relationship is a Reference. Its foreign-key columns are Compiler output.
- A target storage choice such as PostgreSQL
jsonborcitextis not a portable Field type. - A value that needs independent identity, several instances, metadata, order, or lifecycle may be an Entity.
- Behavior shared by several value types may be a Field property rather than another type.
- Application-specific behavior without a bounded structured shape remains outside the Plan.
Admission test
A candidate should enter the format when it passes all five checks:
- Recurring product meaning. Independent applications need the concept. One real application plus a durable Rails or gem category may provide comparable evidence.
- Useful added semantics. Knowing the type improves forms, validation, queries, integrity, lifecycle, or generated tests beyond what an existing type can express.
- Bounded shape. The type and its parameters fit validated data without hiding an arbitrary program in JSON.
- Plausible lowering. At least one target can emit and verify the meaning without making an undisclosed design choice.
- Fixture pressure. A representative Foundation Plan becomes materially clearer, or a focused stress test exposes the concept's boundary.
Before adding a type, check whether the need is instead an Entity, Reference, cross-type property, target-profile choice, or work outside the Plan.
Current scalar and bounded-value types
These types primarily describe a value's semantic shape. That shape can still influence controls, validation, query behavior, and presentation.
| Type | Meaning | Current example |
|---|---|---|
boolean |
A true or false value. | language.right_to_left |
date |
A calendar date without a time of day. | movie.released_on |
datetime |
A timestamp. | message.sent_at |
decimal |
A fixed-precision numeric value. | post_media_item.duration_seconds |
integer |
A whole-number value. | rating.score |
language_code |
A language identifier. | language.code |
long_text |
A longer plain-text value. | message.original_content |
short_text |
A short textual value. | movie.title |
time_zone |
A time-zone identifier. | user.time_zone |
url |
A web address with URL-oriented controls and validation. | profile.website |
decimal remains the current general exact numeric type. The format does not yet express precision or scale.
Current semantic and behavioral types
These types carry more meaning than a conventional scalar column.
attachment
An uploaded file. Dunbar150 uses post_media_item.file.
Active Storage is the first Rails-profile lowering for explicit uploaded Fields as well as the substrate required by Action Text. Cloudinary is the intended first production storage service. Neither choice is repeated as a per-Field realization.
The earlier direction used Shrine for explicit Fields and confined Active Storage to Action Text and Action
Mailbox. Dunbar150 later exercised Active Storage with an ordinary PostMediaItem Entity. Identity, order,
processing state, metadata, authorization, and lifecycle remained relational application data while Active Storage
owned only the bytes. The official Cloudinary gem now supports Active Storage directly, so the extra upload stack
no longer buys enough to outweigh the conventional Rails path. The
dated attachment decision (repository-only) preserves the
comparison and revisit triggers.
Shrine remains deferred until a demonstrated requirement, such as resumable uploads, cannot be met coherently by the selected stack. Attachment content types, byte limits, multiplicity, direct uploads, protected delivery, and cleanup still need bounded Plan meaning or Compiler proof.
counter
A stored count maintained from one Association. settings.counts names that Association.
{
"subject_uuid": "019f9425-5412-7b0f-bc03-ce0b08673f5b",
"key": "likes_count",
"name": "Likes count",
"type": "counter",
"required": true,
"settings": {
"counts": "post.likes"
}
}
A counter is always required in the current format. It does not accept default or immutable.
The audited Rails candidate is counter_culture, including a repair path rather than only write-time callbacks; it
is not yet a profile pin or generated-output claim. count is therefore not repeated as a general aggregate
operation. The dedicated type carries the common zero-initialized, nonnegative, repairable count semantics.
The pure target-independent Counter Analyzer requires the counted Association to belong to the Counter Field's
Entity and consumes complete, same-generation Association analysis. An upstream-blocked Association silently
blocks only dependent Counters. Its immutable, query-free result exactly matches all six bindings from frozen
Compiler revision f8334e3 across the six established design-and-parity Plans, including authored source-pointer
order. The frozen
owner-mismatch diagnostic also matches. Independent valid bindings survive, intentionally replacing the legacy
Field-link pass's all-or-nothing receipt.
Association cardinality is not a Counter validity restriction. Counting a singular Association produces zero or one; counting a collection produces the number of related records. Rails maintenance and lowering remain separate target work.
enum
A closed set of values without a transition graph. settings.values contains the complete stable domain as
ordered { "subject_uuid", "key", "name" } objects. settings.ordinal: true means the order carries semantic
rank instead of presentation order alone.
Oscar Party uses an ordinal enum for bookmark.priority and a non-ordinal enum for credit.role.
An enum is more informative than text plus an inclusion rule. The closed domain can drive controls, validation, queries, and database integrity. The target profile chooses the physical Rails backing.
The current Rails Compiler admits a required enum Field with its canonical nonempty ordered domain only when Rails'
pluralized mapping method name is target-safe, unique among sibling enums, and distinct from authored Predicate and
Ordering scope names. A rejected mapping name omits the Field under its exact
foundation_plan.gap.field_kind.not_generated reason. The reviewed
Plans contribute seven: Oscar Party's ordinal bookmark.priority and non-ordinal credit.role; Case Chat's
case.status, participant.role, participant.notification_level, and notification.kind; and Photogram's
feed_occurrence.reason. Each lowers to a PostgreSQL string column with NOT NULL plus an explicit stable-key Rails
enum mapping. The declaration uses validate: true and collision-aware native helper names. It emits no
native PostgreSQL enum or database CHECK. Membership is therefore model-only: direct SQL and
validation-bypassing operations such as update_all are outside this proof. Oscar's exact Credit uniqueness tuple
may include the emitted role string in a unique index; that enforces tuple uniqueness without widening database
proof of enum membership. An immutable admitted enum reuses ordinary attr_readonly.
Value scopes and instance helpers use their ordinary names unless they conflict. The Rails target then selects native prefix or suffix options, preserving enum behavior. This routine target naming choice does not change the Field's meaning or require a Plan option. The Rails profile owns the allocation rules.
One narrow path realizes a valid in-domain literal-key default on any admitted required enum. Oscar's
bookmark.priority stores normal; Case Chat's case.status and participant.notification_level store open and
all_activity. The generated enum declaration owns that default, so Active Record materializes it on a new model
and an explicit compatible key still wins. The database column retains the same literal default for direct inserts
and storage parity. A nonliteral default or a literal outside the Field's closed domain remains a precise
foundation_plan.gap.field_modifier.default record. An enum Field the Domain does not emit keeps its encompassing
Field gap. Admitted Scaffold enum inputs and read-only projections share generated Rails I18n labels; the initial
catalog preserves authored names exactly, while selects submit stable keys in authored order. Runtime locale edits
change both displays without recompiling the Plan. Screens (repository-only) owns that display
contract. Storage and enum admission remain unchanged.
Outside the descending ordinal Ordering pattern, optional enums, enum Primary Descriptors,
other enum Orderings, Field-owned enum Validations, and ordinal rank comparisons remain unsupported. Credit's exact
Entity-owned two-Reference composite uniqueness may include one required
non-ordinal enum tuple member. This bounded slice does not add rank-aware query semantics. It does feed the existing
generic local
ExpressionSql rule. An independently admitted predicated direct Association may consume such a scope under the
relationship rule's existing gates; the reviewed Plans add no Association consumer.
image
An uploaded image. It differs from a general attachment because the target can derive image-specific validation,
analysis, controls, and delivery behavior. Dunbar150 uses profile.avatar.
The Rails profile stores the image through Active Storage and delivers production images through Cloudinary.
Generated image delivery applies Cloudinary's automatic format and quality optimization rather than calling the
Active Storage variant method, which Cloudinary does not support.
The Plan does not currently describe named image renditions. No Scaffold or other structured subject selects one, so dimensions, crops, responsive sources, and art direction would not affect any other compiled output. They remain ordinary work after compilation. Revisit a bounded rendition vocabulary when generated views can consume it.
json
One intentionally opaque JSON-compatible document. Its internal keys are not separate Plan subjects.
Shinar stores a translation provider's response in translation.raw_response. It keeps content, status, source
version, and Language relationships explicit because application behavior depends on those facts.
The Rails profile lowers json to PostgreSQL jsonb. jsonb is not a second portable type. A product that needs
the exact original bytes should store them separately from the parsed json value.
money
A monetary value with one fixed currency. settings.currency is required and uses an uppercase three-letter
code.
Oscar Party's movie.production_budget uses USD. The Rails-profile direction is money-rails.
Per-record currency is deferred. Currency sourcing, conversion, ordering, allocation, and rounding require more meaning than adding a second column.
position
Mutable contiguous order within a named partition. settings.within lists the Fields or References that
partition records into separate lists. An empty list means one global list and is the intentional exception to
the format's ordinary rule against empty collections.
Oscar Party's credit.position is partitioned by credit.movie. Dunbar150 also conditions media position changes
on the owning Post remaining a draft.
A position is always required in the current format. It does not accept default or immutable. The
Rails-profile direction is positioning.
The live graph stores within as ordered FieldPartitionTerm components owned by the Position Field. Each term
selects exactly one same-Project Field or Reference; the Position Field itself records the global-list case when it
owns no terms. Deferred position and source constraints allow one transaction to reorder or replace a partition.
Deleting the owner cascades, while independently deleting a non-owner selected source is rejected.
The pure target-independent Position Analyzer requires every partition source to belong to the Position Field's Entity and rejects a Position Field that partitions itself. Its immutable, query-free result exactly matches the frozen corpus's four Position roots and four Reference terms in authored source-pointer order. Focused tests preserve the global zero-term case and prove Field terms, exact owner and self-partition diagnostics, and independent-root retention. Partition-Field queryability, generated maintenance, and lowering remain Rails-target work rather than Active Record validation.
rich_text
Formatted text with embedded attachments. Oscar Party uses it for rating.review.
The Rails-profile direction is Action Text. Its storage therefore spans framework-owned rich-text and attachment records rather than one ordinary Entity column.
secure_token
An application-generated token with secure generation and lookup behavior. Shinar's invite.token is immutable,
required, and deliberately publicly shareable.
This remains a semantic type because generation, uniqueness, indexing, form omission, lookup, and sensitive-value handling form one recurring application value. A device token or OAuth credential received from another system remains ordinary text with the appropriate security properties.
The generated token is the Field's exclusive creation source, so secure_token does not accept default. It may
be immutable when the application does not support token rotation.
Rails has_secure_token is strong ecosystem evidence. The exact first-profile lowering still needs to settle
plain versus digest storage, lookup helpers, format parameters, and regeneration behavior.
state_machine
A stored state plus its complete transition graph under settings. The Field owns:
initial_state;- the closed
statesset; and transitions, whose events name allowedfromandtostates.
A transition may include ordered tagged effects when each assignment belongs to the transition itself. The only
current effect is set_field, whose Value uses the same tagged algebra as defaults and create bindings. Oscar
Party's bookmark.status sets bookmark.watched_at from current_time during mark_watched.
Most state-machine Fields are required. An optional one needs when to describe where the graph applies. A
required one omits when. The type does not accept default or immutable.
The Rails-profile direction is AASM over an explicit stored state attribute. The graph remains Entity-local. Concern extraction is a Compiler implementation detail rather than authored reuse in the Plan.
Derived Fields
A numeric Field may carry derivation when its current value is determined by other declared data rather than a
user assignment or creation fallback. Oscar Party derives a Movie's current average rating from its Ratings:
{
"subject_uuid": "019f9425-5412-71be-88a2-f3ebab3915a9",
"key": "average_rating",
"name": "Average rating",
"type": "decimal",
"required": false,
"derivation": {
"aggregate": {
"association": "movie.ratings",
"operation": "average",
"field": "rating.score"
}
}
}
The first bounded derivation variant is an aggregate over one Association. association starts from the Entity
that owns the derived Field. field belongs to the associated records. operation is one of:
| Operation | Meaning |
|---|---|
sum |
Add the source values. |
average |
Calculate their arithmetic mean. |
minimum |
Select the least source value. |
maximum |
Select the greatest source value. |
average always produces a decimal Field, including when its source Field is an integer.
The live Project graph stores one FieldDerivation component for each authored aggregate. It retains the derived
Field, source Association, operation, and source Field through same-Project foreign keys. Deleting the derived
Field removes its component, while a selected source Association or source Field cannot disappear silently. A
deferred owner constraint permits insert-first replacement inside the future Project-locked mutation boundary.
This storage shape does not decide whether those links form a valid aggregate.
An Association's Predicate determines which records participate. The derivation does not embed another filter language. The frozen draft Compiler's working-graph pass checked that the Association starts at the owning Entity, its complete analyzed result set owns the source Field, the source and result types are compatible, and the dependency graph contains no cycle. It rejected the whole pass without writing partial derivation rows when those checks failed. Consumer analysis and target lowering remain separate work.
derivation means an ongoing relationship. If a Rating score or the membership of movie.ratings changes,
movie.average_rating must reflect the change. It is not an initial snapshot. A value captured only during record
creation would instead be a computed default.
The derivation is the Field's exclusive value author. A derived Field therefore accepts neither default nor
immutable, and generated create and edit forms do not submit it.
An approved Plan also requires at least one structured consumer for every derived Field. Initial consumers are:
- a Scaffold that displays the Field;
- a Predicate that reads the Field;
- an Ordering that sorts by the Field; or
- an Association that applies such a Predicate.
This requirement gives the Compiler a concrete use to satisfy. It avoids preserving a general calculation language for values that the user's agent could add after compilation. A draft authoring surface may warn about an unused derivation while the document is incomplete. The Compiler's semantic validator rejects it before compilation. JSON Schema alone cannot enforce this cross-document rule.
The consumers determine the required Rails lowering. A value shown only on a Scaffold show surface can remain a
fresh record-level aggregate. Collection display, a Predicate, an Ordering, or an Association query requires one
set-based, queryable value. The Plan does not serialize that derived consequence as another realization choice.
derivation therefore does not itself request a cache.
For Oscar Party, movie.average_rating appears on the Movie index and in
movie.highest_rated. The isolated Compiler projection records:
- a decimal
ratings_score_sumsupport column maintained bycounter_culture; - an integer
ratings_countsupport column maintained bycounter_culture; and - a stored PostgreSQL-generated
average_ratingover those two columns.
counter_culture can maintain numeric totals with atomic deltas and
repair drift. It does not directly maintain averages. PostgreSQL generated columns cannot
read other rows, but they can derive one column from values on the same row. Splitting the
average into an additive sum and count made a stored generated average_rating calculated as
ratings_score_sum / NULLIF(ratings_count, 0) a plausible earlier extension. The public semantic Field still
appears once in the Plan even when its Rails realization produces support columns.
That stored result column predates Decision 0024. The current projection records
production_wiring_complete: false; neither the result nor its Counter Culture support is generated output. Before
generating it, the target must compare a conventional Rails or common-gem queryable result and either keep the
behavior in Rails or record a narrow owner decision. The semantic requirement is a maintained, queryable average—not
this mechanism. Counter Culture 3.14.0 remains an audited candidate, not a profile pin.
Oscar Party requires every rating.score, so the Rating count is also the non-null score count used by SQL
AVG. Nullable source values need separate non-null-count semantics before this cached lowering can support them.
If selected, Counter Culture would keep its updates in the Rating transaction. Generation would also need repair
guidance and tests because bulk SQL can bypass model callbacks.
Counter Culture 3.14.0 registers a child before_destroy callback for its decrement. Any generated deletion path
that claims to maintain the Counter must therefore call the child's Active Record destroy lifecycle. PostgreSQL
ON DELETE CASCADE, delete_all, and raw SQL do not satisfy that claim. Use conventional dependent: behavior
only when the referenced-side Association is admitted and namespace-checked. The
emitted-code audit (repository-only) found no conventional
namespace-safe Rails path without that inverse. The successor target must retain the Counter and its dependents as
gaps in that shape; a later packet may revisit it only with new ecosystem evidence. Do not synthesize an inverse
reader or generate a callback scheduler.
Decision 0024 (repository-only)
owns that mechanism boundary.
sum can plausibly use the same Counter Culture total without the generated quotient. minimum and maximum
still need a selected Rails lowering. Their deletion and update behavior is not additive.
counter remains a dedicated semantic type rather than
{ "derivation": { "aggregate": { "operation": "count" } } }. Counts are common enough to carry a stronger
zero-initialized, nonnegative meaning and a selected Rails-profile repair direction. Later evidence may add typed
row arithmetic or another derivation variant without turning the Plan into arbitrary code.
Delegated Fields are deferred
A future derivation variant could make a value reached through a Reference into a named, read-only Field on the
owning Entity. For example, like.title could read movie.title through like.movie. Other Plan subjects could
then consume like.title without copying that path.
Rails can lower a simple record read with delegate :title, to: :movie. That macro is not the
complete semantic model. A Scaffold collection may need eager loading, while a Predicate or Ordering needs a
supported SQL join rather than a Ruby method. Optional References, multi-target References, type compatibility,
writes, validations, and longer paths also need bounded rules.
The initial format omits this variant. Generated Scaffold views can use explicit reader chains such as
like.movie.title, and ordinary post-compilation code can add a delegate when it improves the model API. Revisit
the Field shape when a fixture needs the same related value as a named input to several structured consumers.
delegate_missing_to and arbitrary unmodeled method names remain ordinary Rails code.
Current Field properties
Every Field carries these keys:
| Key | Meaning |
|---|---|
subject_uuid |
Project-scoped identity retained across accepted submissions. |
key |
Owner-local lower-snake-case key; typed links use the derived entity.key path. |
name |
Human-facing name. It remains explicit even when derivable from the key. |
type |
One of the twenty current semantic types. |
required |
Whether a stored record may omit the value. |
The current schema also permits these optional properties:
| Key | Current meaning |
|---|---|
notes |
Explanatory prose for people reviewing the Field. It does not request generated behavior. |
validations |
Nonempty rules whose errors belong to this Field. Omit when there are none. |
default |
A literal, environment value, environment path, or reference-data record. |
derivation |
A typed rule that continuously and exclusively determines a numeric Field's current value. |
settings |
Configuration owned by the selected Field type, required only for types that need it. |
immutable |
Whether creation may assign the value while later writes may not change it. Defaults to false. |
normalizations |
An ordered pipeline that changes incoming text before validation and storage. |
comparison |
Comparison behavior that does not rewrite the stored value; currently only case_insensitive. |
encrypted_at_rest |
Whether target storage must protect the stored value. Defaults to false. |
redact_from_logs |
Whether generated diagnostics and logging must hide the value. Defaults to false. |
Optional boolean settings accept both values and default to false. Omission and explicit false mean the same
thing when the property applies to that Field type. Inapplicable properties remain invalid even when set to their
default. Examples omit default-valued settings to stay readable. Ordinary empty collections, including
validations, are omitted.
notes can clarify meaning that does not fit in the display name. Oscar Party uses it to say that movie.status
tracks only Best Picture. A Compiler may preserve these notes in generated documentation, but it does not infer
Fields, validations, callbacks, or other behavior from the prose.
The properties answer different questions:
requireddescribes presence.defaultsupplies a fallback when a new record has no explicit value.derivationcontinuously determines a system-managed value from declared inputs.immutabledescribes changes after creation.normalizationschanges an incoming representation before storage.comparisonchanges how preserved values are compared.- a Scaffold create or update definition decides whether a user can submit the Field;
- an Entity Policy controls access to the record and any attachment bytes it owns;
encrypted_at_restdescribes storage protection;redact_from_logsdescribes disclosure through diagnostics; and- a Predicate, uniqueness rule, or other structured query consumer records actual query use.
A default is creation fallback, not an overwrite rule. A caller-supplied compatible value wins. For a Rails model,
the current_time environment value means Time.current when Active Record materializes the new record's default;
it is not an exact database-commit timestamp and does not make the Field immutable. The
Rails target profile owns the exact model declaration and storage boundary.
There is no used_for_lookup boolean. The use site preserves whether a lookup is global or scoped and simple or
compound. The target Field supplies its comparison behavior.
The Rails profile currently lowers immutable on an emitted scalar or admitted required enum Field to model-level
attr_readonly; its target page owns the bounded behavior and verification.
encrypted_at_rest can lower to Active Record Encryption. On an emitted ordinary scalar or admitted required enum
Field, redact_from_logs appends the attribute to self.filter_attributes and registers the model-qualified request
parameter. Case Chat therefore filters notification.deduplication_key, while the same key nested under another
model remains visible. This is neither encryption nor a claim about arbitrary application, provider, or external
logs. The modifier remains a gap on every other or unemitted Field. Querying encrypted data still needs an explicit
supported strategy at its use site.
Conditional mutation
The current format has no generic conditional-mutation property. Dunbar150 wants a Post's media membership,
attachment, and order to stop changing after the Post leaves draft. That is one application rule spanning
several mechanisms rather than evidence for one cross-type Field primitive.
An ordinary stored attribute could reject a dirty change in an Active Record validation. An attachment changes
through Active Storage. Reordering a position Field can update several sibling rows. Forms and permitted
parameters provide useful affordances but cannot enforce the rule against direct model writes or concurrent
operations. A single condition would therefore promise a uniform lowering that does not yet exist.
The Dunbar150 design-pressure brief keeps the desired outcome for the user's agent. Revisit narrower type- or operation-specific shapes after another application needs them and the Rails Compiler has verified lowerings for model enforcement, forms, direct writes, parent-state changes, and concurrency.
Normalization and comparison
Normalization and comparison are separate Field semantics.
{
"subject_uuid": "019f9425-5412-7dc2-b184-4e356afc8e02",
"key": "username",
"name": "Username",
"type": "short_text",
"required": true,
"normalizations": [
"trim",
"blank_to_null"
],
"comparison": "case_insensitive"
}
The normalizations change " Raghu " to "Raghu" before validation and storage. The comparison retains that
capitalization while treating "Raghu" and "raghu" as equal in generated lookups, Predicates, Orderings, and
uniqueness enforcement.
The current normalization operations are:
| Operation | Meaning |
|---|---|
trim |
Remove Unicode whitespace and the six invisible edge characters listed below from both ends. |
blank_to_null |
Replace an empty or Unicode-whitespace-only string with null. |
collapse_whitespace |
Replace each Unicode whitespace run with U+0020, then remove edge spaces and U+0000. |
downcase |
Store text using the selected target profile's documented Unicode lowercase mapping. |
Operations run in listed order. Null is absorbing: a null input remains null, and once an operation produces null,
later operations leave it null rather than invoking string behavior. A repeated operation, an empty array, or an
unknown operation is invalid. Omitting normalizations means that the Plan requests no general-purpose
normalization for that Field. Semantic types can still carry intrinsic parsing or canonicalization; for example,
a future email type may define email-specific behavior rather than requiring every Plan to reconstruct it from
text operations.
trim removes characters in Unicode White_Space plus U+180E, U+200B, U+200C, U+200D, U+2060, and U+FEFF
at the two edges. It preserves interior whitespace, paragraphs, joiners, and NUL. collapse_whitespace collapses
Unicode whitespace, including tabs and line separators, and removes edge NUL. It preserves interior NUL and those
six extra invisible characters, including joined emoji. An invisible-only value can therefore remain nonblank after
collapse. Interior NUL still encounters ordinary PostgreSQL storage rejection. The normalization
qualification (repository-only) records the source checks behind these
character policies.
Do not combine trim with collapse_whitespace; they have distinct edge policies. If blank_to_null occurs with
either cleanup operation, place it after that operation. For example, ["trim", "downcase", "blank_to_null"] and
["collapse_whitespace", "blank_to_null", "downcase"] are valid, but ["blank_to_null", "trim"] and
["blank_to_null", "downcase", "collapse_whitespace"] are invalid. Downcase can appear anywhere else in the
pipeline. The Plan validator reports the offending pipeline and the required order. It does not reorder the
operations or repeat them until stable. This keeps admitted pipelines idempotent: a lone U+200B becomes null after
trim then blank-to-null, and a lone NUL becomes null after collapse then blank-to-null, on the first pass.
Choose normalization for the content, independently of column type. short_text uses a single-line input and
long_text uses a textarea; neither selects a pipeline automatically.
- Names and titles can request
["collapse_whitespace", "blank_to_null"]when internal whitespace has no meaning. - Ordinary multiline prose can request
["trim", "blank_to_null"]while retaining interior paragraphs and spaces. - Code, Markdown, and other format-sensitive content can omit normalization or request only
blank_to_null. Whole-value trimming removes first-line indentation and trailing newlines. - Identifier-like text can use
trimwhile a format Validation rejects internal whitespace.
url currently admits only trim and blank_to_null. Whole-value downcasing can corrupt case-sensitive paths
and query values, while collapsing internal whitespace is not URL canonicalization. Component-aware URL
normalization remains deferred until representative applications require it.
comparison is Field-owned because every consumer should agree about equality. Omission means ordinary
case-sensitive comparison. A uniqueness Validation inherits comparison behavior from each target Field instead
of repeating one setting for the whole tuple. This matters when a compound tuple mixes a case-insensitive text
Field with exact Fields or References.
The current Rails Compiler uses normalizes. Within one model, Fields with identical pipelines
share one inline callable and one declaration:
class Profile < ApplicationRecord
normalizes :username, :website, with: ->(value) { StripAttributes.strip(value) }
normalizes :display_name, with: ->(value) { value.squish.presence }
end
The first declaration realizes trim, blank_to_null with the public formatter from
strip_attributes 2.0.1. Trim alone passes allow_empty: true; blank-to-null alone uses
value.presence. Rails' squish realizes collapse, with .presence appended only when requested. Ruby
String#downcase supplies the profile-qualified Unicode lowercase mapping. Rails skips normalization for nil inputs;
inline pipelines keep nil absorbing when an intermediate operation returns it.
The grouped inline callable keeps generated behavior beside the affected attributes without claiming an additional top-level constant or Concern. Rails owns assignment and supported-query normalization. The gem's model callback is not installed. The broader trim policy and edge-NUL collapse behavior intentionally supersede the older Unicode-whitespace-only cleanup semantics; they are not equivalent-code refactors.
The formatter's public string API corrects the earlier reason for rejecting StripAttributes: its optional
before_validation model callback does not limit use inside Rails normalizes. Native squish
supplies the separately chosen collapse policy. The cleanup-before-blank rule belongs to Plan admission; it adds
no application validation or silent operation reordering.
Rails normalizes ordinary scalar hash equality operands. QueryAttribute#nil? also notices
when normalization produces nil, so generated equality and route lookups should prefer Active Record hashes:
Profile.find_by!(username: input)
Typed Arel attributes cast non-null values through the decorated type and apply the normalizer:
profiles = Profile.arel_table
Profile.where(profiles[:username].eq(input))
This differs for normalized blanks. Casted#nil? chooses the SQL null operator from the raw
operand, so attribute.eq(blank) emits = NULL. ArrayHandler detects null members before
normalization, so a normalized-null array member does not add IS NULL. Arel, array, untyped, quoted, and raw-SQL
consumers must call Model.normalize_value_for first and branch on nil. Existing rows are not rewritten merely
because a normalization is added; a migration or repair must handle pre-existing values.
The first case-insensitive Rails direction lowers a case_insensitive Field to PostgreSQL
citext. It preserves entered case while ordinary equality, Arel, ordering, and B-tree indexes
use case-insensitive operators:
enable_extension "citext"
create_table :profiles do |t|
t.citext :username, null: false
end
add_index :profiles, :username, unique: true
The Compiler must install the extension in a schema on the application's PostgreSQL search_path in every
environment. citext is broader than equality: PostgreSQL also gives its LIKE, regular-expression, and several
string-function operations case-insensitive behavior. A rare case-sensitive operation must cast its operands to
text. Its case folding follows the database's LC_CTYPE, which is selected when the database is created. A
released target profile must pin or verify a supported database locale, or disclose that external prerequisite
when First Draft does not provision the database.
Rails 8.1 recognizes citext as inherently case-insensitive. Generated uniqueness validators
should use ordinary equality rather than passing case_sensitive: false: the column already supplies the
semantics, and omitting the option lets Rails recognize when a matching unique index makes another persisted-record
check unnecessary. An ordinary compound unique index also retains each member's physical semantics: a citext
Field compares case-insensitively while text, numbers, and foreign keys compare normally.
This is a Rails-profile storage decision, not another Foundation Plan Field type. PostgreSQL recommends considering ICU nondeterministic collations for more complete Unicode case behavior. They also introduce broader collation, indexing, and pattern-matching consequences. Revisit that lowering when multilingual identifiers or Unicode case-folding failures appear in representative applications.
PostgreSQL can resolve a direct comparison between citext and exact text as case-sensitive text equality.
Entity and nested Field comparison Validations admit Field operands, so the current semantic type pass requires
those Fields to have compatible comparison behavior. Expressions apply the same rule when another Field appears as
the right operand. An explicitly designed mixed comparison could instead lower with a deliberate cast.
StripAttributes' optional collapse_spaces and replace_newlines flags do not implement this Plan operation.
Version 2.0.1 can split interior joiners under those flags and does not collapse Unicode line separators in the
same way as Rails. The profile uses only its default edge formatter. A future upstream change calls for a fresh
semantic comparison rather than an automatic switch from Rails collapse.
Issue #705 (repository-only) tracks the joiner contribution candidate separately.
Query profiles
A Field type supplies a fixed query profile. The profile describes comparison meaning; it is not another author-selected Field property. The Expression design owns the recursive algebra, operands, null behavior, and relationship quantifiers.
The current working matrix is:
| Field types | Expression operators | Refinement |
|---|---|---|
boolean |
Equality-shaped and in |
false remains a value rather than absence. |
integer, decimal |
Equality-shaped, in, ordered comparisons |
Integer operands reject fractional values. |
counter, position |
Equality-shaped, in, ordered comparisons |
Both types are always non-null. |
money |
Equality-shaped, in, ordered comparisons |
Both operands use the same fixed Field currency. |
date |
Equality-shaped, in, ordered comparisons |
current_date is compatible. |
datetime |
Equality-shaped, in, ordered comparisons |
current_time is compatible. |
enum |
Equality-shaped and in |
Ordered comparisons require ordinal: true. |
state_machine |
Equality-shaped and in |
Transition order is not value order. |
short_text, long_text |
Equality-shaped, in, literal text matching |
Field comparison behavior applies. |
language_code, time_zone, url |
Equality-shaped and in |
Semantic components are not substring targets. |
secure_token |
Equality-shaped and in |
Caller-supplied token lookup remains a future consumer. |
attachment, image, json, rich_text |
None | Presence, JSON traversal, and search need dedicated meaning. |
“Equality-shaped” means equals and not_equals; other negative forms use the Expression-level not.
“Ordered comparisons” means
less_than, less_than_or_equal_to, greater_than, and greater_than_or_equal_to.
“Literal text matching” means contains, starts_with, and ends_with.
is_null is cross-cutting rather than a comparison profile. It applies when an optional Field has one queryable
scalar null representation. A missing singular Association hop is unknown rather than a null terminal Field.
Association presence uses exists. Attachment and image presence, Action Text blankness, and PostgreSQL jsonb
null need type-specific semantics before they can share a query operator.
A derived Field inherits its declared result type's profile only when the selected target can provide a queryable
realization for the consumer. A Field-to-Field comparison also requires compatible comparison behavior. An exact
text Field and a case_insensitive text Field are not compatible merely because both store strings.
This matrix is a v1 query boundary, not proof that all twenty Field types are otherwise Compiler-supported. Adding a Field type should normally assign one of these profiles or explicitly remain non-queryable. It should not require another Expression object shape.
At frozen revision f8334e3, all six Rails Compiler fixtures complete Predicate Expression type checking in the
full pipeline. Focused tests over the earlier three-Plan corpus exercise this complete matrix. The checker retains
text comparison mode, fixed money currency, enum or State Machine Field domain, ordinal enum meaning, and possible
record Entity sets. It rejects unsupported operators, cross-Field comparison-mode or currency mismatches,
undeclared closed values, nonqueryable Fields, and invalid contextual literal representations before advancing its
all-or-nothing lifecycle marker. Complete SQL and record visitors remain unimplemented. The later bounded Rails SQL
visitor now realizes 14 local Predicate roots, including Oscar Party's credit.directors and Case Chat's
participant.wants_all_activity over admitted enum storage; that does not establish complete-matrix query support.
Type-specific keys and restrictions
| Type | Required type-specific keys | Important restrictions |
|---|---|---|
counter |
settings.counts |
Required; no default, immutability, or conditional mutation. |
enum |
settings.values |
Named values are required; settings.ordinal is optional. |
money |
settings.currency |
Uppercase three-letter code. |
position |
settings.within |
Required; no default or immutability. |
state_machine |
Initial state, states, and transitions under settings |
Optionality uses settings.when. |
short_text |
None | May carry normalizations and case_insensitive comparison. |
long_text |
None | May carry any current normalizations; comparison remains ordinary. |
url |
None | May carry only trim and blank_to_null; comparison remains ordinary. |
JSON Schema validation alone does not prove that settings.counts names an Association or that transition states
exist. The frozen draft Compiler resolved those links and rejected broken Counter, position, aggregate, and State
Machine topology atomically. Later semantic passes still need to validate the JSON-backed Expressions, Values,
Validations, and consumer-specific target requirements that these local link passes deliberately preserve.
Property work still to design
The following recurring needs may extend existing types rather than introduce new types:
- numeric precision and scale;
- attachment content types, byte limits, multiplicity, and protected-delivery requirements;
- nonliteral or out-of-domain enum defaults, and ordinal rank and Ordering semantics;
- generated or system-managed values and their create and edit eligibility.
The current Validation registry exposes bounded Rails-familiar families, restricts them by compatible Field type in JSON Schema, and leaves custom Ruby validation for the user's agent after compilation. Its schema is structurally complete for the current sketch, and pure semantic passes now resolve and type Validation targets, conditions, comparisons, tuples, and error targets. Complete family semantics remain unimplemented; the executable first admission slice lists the Rails declarations admitted so far.
Rails exposing an option is evidence for a candidate. It is not sufficient by itself to add Plan vocabulary.
Deferred extensions to current types
These extensions have plausible demand, but the current format does not yet model them.
State machines
Deferred work includes shared graphs, richer callbacks, timers, and cross-record effects. State machines often read sibling Fields, and independent models rarely share the same complete graph and effects. Revisit shared definitions after repeated generated applications show that Entity-local graphs cause meaningful duplication.
Counters
Deferred work includes arbitrary deltas, multi-level paths, time-dependent conditions, soft-deletion interaction, and an exact repair guarantee. Revisit each extension when an application cannot describe a recurring maintained count by naming one Association.
Positions
Deferred work includes nested lists, cross-partition moves, sparse or ranked alternatives, and concurrent reorder semantics. Revisit when a fixture needs behavior that a contiguous list within declared partition keys cannot express.
Money
Deferred work includes per-record or ancestor-sourced currency, allowed currency sets, exchange-rate sources, cross-currency ordering, allocation, and rounding policy. Revisit when a real application needs more than one fixed currency and can pressure the complete semantics.
Rich text
Deferred work includes custom embeds, attachment policy, plain-text or search projections, alternative editors, and non-Rails target behavior. Revisit when one of those outcomes becomes a repeated generated-app requirement.
Secure tokens
Deferred work includes token format and length, plain or digest storage, expiration, regeneration, and whether lookup helpers are intrinsic or supplied by the feature that uses the token. Revisit while exercising Shinar's invite flow and another independent token use.
Format candidates
These candidates have application or ecosystem evidence. None is part of the current format.
Next focused candidate: slug
A slug is a human-facing locator derived from, or initially suggested by, another Field. It can affect uniqueness, routing, mutation history, and collision handling.
Before inclusion, settle the source Field, scope, update policy, history or redirects, collision strategy, and
generated versus user-chosen values. friendly_id is a plausible Rails lowering, not a selected or exercised one.
email
Email meaning can drive a dedicated control, normalization, comparison, intrinsic format handling, and case-insensitive uniqueness. This is useful outside an Account identifier managed by authentication.
Before inclusion, settle intrinsic normalization and validation, display-case preservation, and uniqueness semantics. Revisit when a non-Account email Field pressures those choices.
address
An Address is a postal, delivery, billing, or contact value with coordinated country-sensitive components. Its baseline generated control is a manual component form that supports browser autofill.
Before inclusion, settle the component set, international formatting, structural validation, optional deliverability checks, and whether components can be referenced independently. Use an Entity instead when reuse, identity, or lifecycle becomes important.
place
A Place is a selected venue, business, landmark, or pickup point. Its baseline generated control is provider autocomplete, while stored data should remain useful without a live lookup.
Before inclusion, settle the provider-neutral snapshot, optional contained Address, manual fallback, coordinates, provider-ID refresh, and Entity boundary. Google Places is the initial candidate lowering. A second supported provider could become a material realization choice.
array
An Array is a bounded collection of same-kind scalar values whose members have no independent identity, metadata, authorization, or lifecycle. Fizzy's subscribed webhook actions provide one example.
Before inclusion, decide whether this is a type or a collection wrapper. Also settle item types, order, duplicates, blank elements, validation, querying, indexing, and generated controls. PostgreSQL arrays and JSON are possible target lowerings rather than portable meaning.
date_range
A Date Range could keep repeated start and end values together and supply end-after-start and overlap meaning.
Before inclusion, settle inclusive versus exclusive bounds, open ends, overlap semantics, time zones, and whether two ordinary named Fields remain clearer.
percentage
A Percentage could prevent missing numeric bounds and improve controls and display.
Before inclusion, settle 0..100 versus 0..1, precision, and whether values above one hundred are meaningful.
Revisit when two applications need the bounded concept rather than an ordinary decimal Validation.
sequence
A Sequence is a race-safe, human-facing monotonic value such as a ticket or invoice number.
Before inclusion, settle scope, formatting, gap policy, reset policy, and assignment timing. Revisit when a real application pressures failure and concurrency behavior.
public_id
A Public ID is a stable, non-enumerable external identifier distinct from an Entity's internal primary key.
Before inclusion, settle generation, format, lookup, rotation, and whether routes expose it by default. Revisit when an application needs the value independently of the target profile's primary-key strategy.
Names that are not separate Field types now
These names remain recorded so later work does not reopen the same question without new evidence.
| Name | Current treatment |
|---|---|
string, text |
Storage names replaced by semantic short_text and long_text. |
reference, foreign_key, belongs_to |
Stored edges are References; relationship readers are Associations. |
citext |
PostgreSQL strategy for case-insensitive behavior. |
uuid |
Primary-key strategy; use the public_id candidate for a separate product value. |
password, password_digest |
Authentication-managed Account behavior and storage. |
opaque_secret |
Use the meaningful type plus security properties. |
encrypted_text |
Encryption is a cross-type storage requirement. |
normalized_text |
Normalization is cross-type input behavior. |
counter_cache |
Rails mechanism for realizing a semantic counter. |
generates_token_for |
Stateless purpose-scoped behavior without a stored Field. |
polymorphic |
A Reference realization strategy. |
multiple_attachment |
Attachment configuration or a collection of child Entities. |
color |
Usually an enum or short text with current evidence. |
float |
decimal is the safer current numeric meaning. |
Revisit these judgments only when the boundary changes:
- Revisit
colorafter repeated needs for color controls, parsing, normalization, or contrast checks. - Revisit
floatwhen an application intentionally needs approximate floating-point behavior. - Revisit attachment multiplicity if repeated unstructured collections cannot fit configuration cleanly.
- Revisit target-specific storage names as profile choices only when supported alternatives create a material product tradeoff. They still would not become semantic Field types.
secure_token remains distinct from opaque_secret. Secure generation and lifecycle are part of its recurring
meaning. Secrecy by itself does not define an application value.
Maintaining the catalog
For a proposed type, record:
- the product-language value it represents;
- two independent examples, or one example plus durable framework or gem evidence;
- intrinsic validation, comparison, query, and control behavior;
- parameters and references to other Schema subjects;
- the Entity and Reference boundary check;
- one plausible target lowering and meaningful alternatives; and
- tests that distinguish format inclusion, Compiler exercise, and generated-app evidence.
When the current format changes, update the JSON Schema, this page, schema tests, and at least one focused example together. Record Compiler and generated-app evidence separately from format inclusion. Open focused Issues when a candidate becomes active research or implementation work, and synthesize any conclusion here before closing knowledge-heavy discussion.