First Draft · Foundation Plan Guide

Supporting reference · Working design and evidence, not a promise of complete support.

On this page

Query intents: Predicates, Expressions, and Orderings

Status: Working synthesis for firstdraft.foundation-plan.sketch/0.23, with implemented target-independent Predicate analysis, authored Ordering retention and bounded import, and bounded Rails Predicate- and Ordering-scope lowerings. Bounded referenced-side direct, predicated-through and collection-source indirect, and one-level nested-through Association consumers are implemented. The same is true for two root unfiltered collection-existence shapes, one root unfiltered singular-reference existence shape, and two exact current-Account relationship shapes. General Predicate consumers and broader Ordering semantics remain open.

For a guided introduction, read Understanding Expressions and their consumers (repository-only).

Current answer

A query intent is structured application meaning that helps select or sequence records. It is an explanatory umbrella, not another serialized subject or collection key. The current Foundation Plan represents reusable selection intent with Predicates and reusable sequence intent with Orderings. Future user-supplied filtering or search may add other bounded subjects without turning Predicate into a general query language.

A Predicate is a named Boolean classification of one Entity's records. Its Expression is the recursive Boolean tree that defines whether one record matches. An Ordering remains separate because qualification and sequence are different application facts.

Every base Expression is one exact tagged variant. The current variants are:

The same JSON key kind labels both Expression nodes and Value operands. The containing schema position determines which closed vocabulary applies.

A Policy uses the same condition semantics through a separate recursive Policy Expression. That grammar adds Policy-only matches_policy delegation without making Policy invocation available to Predicates or Validation conditions. The Policies page (repository-only) owns that authorization extension.

A condition is the role a base or Policy Expression plays when a Validation, State Machine, or Policy asks whether behavior applies. It is not another reusable Plan subject.

Search, pagination, arbitrary SQL, eager loading, projection, and imperative query methods are not Predicates.

Standards lineage and boundary

The scalar and logical core is informed by OGC Common Query Language 2. Basic CQL2 supplies the same three-valued truth model, typed property comparisons, explicit null test, Boolean composition, and rule that only true selects a record. CQL2 calls the third predicate result NULL when a value is unknown; Foundation calls the truth result unknown and uses null for the operand value that can produce it. Hasura NDC supplies a direct JSON precedent for related-collection exists and nested record scope. OpenDD shows semantic operator names being checked and then mapped to target-specific implementations. The query and Expression research report (repository-only) records the source inspection, including CQL2, Hasura NDC and OpenDD, Prisma, OData, SCIM, CEL, Substrait, Calcite, Django, SQLAlchemy, Ransack, Arel, and counterexamples.

Foundation adopts those semantic lessons without claiming CQL2 conformance. Its JSON is a closed application artifact, not a general filter interchange format. kind plus named members such as left, right, and where supports exact variants, readable typed paths, nested Association scope, and useful authoring diagnostics. CQL2's op plus positional args encoding cannot represent the Foundation-specific meanings without extensions, so copying its spelling would not create interoperability.

A complete small example

Oscar Party defines upcoming Movies:

{
  "subject_uuid": "019f9425-5412-7fa2-96a3-ce8510876d21",
  "key": "upcoming",
  "name": "Upcoming",
  "expression": {
    "kind": "comparison",
    "left": {
      "target": {
        "field": "movie.released_on"
      }
    },
    "operator": "greater_than",
    "right": {
      "kind": "environment",
      "name": "current_date"
    }
  }
}

left identifies the typed value being tested. operator selects behavior admitted by that value's query profile. right supplies a compatible value.

The Rails profile can lower this Predicate to a query API:

scope :upcoming, -> { where(arel_table[:released_on].gt(Date.current)) }

When a record-level consumer needs the same meaning, the Compiler may also need a Boolean evaluator:

def upcoming?
  released_on > Time.zone.today
end

Those snippets illustrate two target projections. They are not executable Compiler evidence.

Why Predicate, not Scope

Rails uses scope for any named method that returns an ActiveRecord::Relation. The Active Record implementation permits a scope to filter, join, select, preload, or otherwise transform a relation. The Foundation Plan concept is narrower and can be used outside a query. It answers one Boolean question about one record.

Naming the subject Predicate preserves the ordinary Rails meaning of scope while allowing one definition to produce:

Django Q objects are composable query Expressions. CEL separates parsed expressions from checked expressions. SQLAlchemy hybrid methods show why one meaning may still need distinct instance and SQL implementations.

Query Intent is too broad for this Boolean subject because it also includes Ordering and may later include search. Filter overemphasizes collection use. Rule usually means a condition plus an outcome. Condition works at the point of use but poorly as a reusable definition. Predicate is the least overloaded name for the shared application meaning.

Closed base Expression variants

The kind tag makes each recursive node unambiguous:

kind Required content Meaning
comparison left, operator, right Compare two compatible values.
is_null operand Test whether one nullable scalar Field value is null.
and expressions True when every child is true.
or expressions True when at least one child is true.
not expression Negate one child using the same three-valued logic.
matches_predicate predicate Reuse a Predicate owned by the current Entity.
exists association, optional where Test whether any associated record satisfies a condition.

This table describes $defs/expression, the base recursive grammar used by Predicates, Validation conditions, and State Machine conditions. It contains no Policy reference. Policy.allow_when instead uses $defs/policyExpression, which recursively combines these condition semantics with the Policy-only matches_policy variant.

and and or each require at least two children. A one-child group adds no meaning, while an empty group forces a true-or-false identity choice that an author did not need to make.

Boolean groups use and and or rather than the former all and any. In mature query languages, any and all usually name collection quantifiers. Reserving those words avoids confusing Boolean grouping with relationship semantics.

Every variant rejects properties owned by another variant. Legacy equality shortcuts, record_equals, includes, policy_through, and the permissive untagged object are not alternate spellings.

Composition and Predicate reuse

Shinar defines active Memberships once and reuses that meaning for active owners:

{
  "subject_uuid": "019f9425-5412-77bc-b64c-1e7a4c6203f5",
  "key": "active_owner",
  "name": "Active owners",
  "expression": {
    "kind": "and",
    "expressions": [
      {
        "kind": "matches_predicate",
        "predicate": "membership.active"
      },
      {
        "kind": "comparison",
        "left": {
          "target": {
            "field": "membership.role"
          }
        },
        "operator": "equals",
        "right": {
          "kind": "literal",
          "value": "owner"
        }
      }
    ]
  }
}

matches_predicate keeps the current record and Entity scope. Semantic validation resolves the named Predicate, requires it to belong to that Entity, and rejects dependency cycles.

Cross-Entity reuse in a base Expression changes scope explicitly through exists. It does not silently reinterpret another Entity's Predicate against the current record. Policy-to-Policy delegation belongs to the separate Policy Expression described on the Policies page (repository-only).

Comparison targets

A comparison target identifies a value relative to the current Entity scope.

Direct Field

{
  "target": {
    "field": "rating.score"
  }
}

Field through singular Associations

{
  "through": [
    {
      "association": "membership.chat"
    }
  ],
  "target": {
    "field": "chat.kind"
  }
}

through contains typed Association locators. Every step is singular and begins where the previous step ended. The target Field belongs to the last Association result. A collection traversal uses exists instead.

This shape separates traversal from the terminal value. The former mixed path array could structurally accept a Field in the middle, several terminal Fields, or a collection hop even though none had scalar meaning.

Singular Association result

{
  "target": {
    "association": "bookmark.user"
  }
}

A singular Association result is a typed record identity. Equality can compare it with a compatible record value, such as current_account. Collection Associations are not scalar comparison targets.

Current scoped record

{
  "target": {
    "record": "current"
  }
}

The current-record target makes record identity explicit. At a Predicate or Policy root it means the owning Entity record. Inside exists.where it means the associated record brought into scope.

Comparison targets may also appear on the right. This admits compatible sibling-Field comparisons and bounded same-record paths without introducing arbitrary expressions or outer-scope references. Two Field operands need compatible value and comparison behavior.

Comparison values

The right operand is exactly one of:

in instead receives a nonempty, duplicate-free array of compatible tagged literals or reference-data records. It does not receive a runtime-generated collection. The tags distinguish value kinds structurally without duplicating the destination type:

{
  "kind": "literal",
  "value": "owner"
}

Dates, datetimes, decimals, enum values, and ordinary text may share a JSON representation. Structural validation therefore cannot prove compatibility from the tagged literal alone. Semantic validation resolves the left target and checks the literal payload against its Field type. A more elaborate type tag should enter only if authoring or target evidence shows that contextual typing is insufficient. Every current Foundation comparison already supplies a typed left target.

The first checked literal contract is:

These rules keep exact numeric meaning out of binary JSON floating point and reject permissive date/time casts that silently normalize invalid input. Fractional-second precision and the known numeric zero offset +00:00 remain valid RFC 3339 spellings; the checked binding carries semantic datetime type rather than treating source spelling as a distinct type.

Null is not a comparison value and cannot appear inside an in set. Null meaning belongs to is_null and three-valued evaluation.

Operators

The current operator vocabulary is:

Other negative meaning uses not around a positive Expression. The format does not add not_in, does_not_contain, or is_not_null. Both not_equals and not preserve three-valued logic: a null operand produces unknown rather than silently including nulls.

contains, starts_with, and ends_with treat the right text as literal content. SQL lowering needs to escape wildcard characters rather than reinterpret them as a LIKE pattern. These are deterministic string predicates, not tokenized or ranked full-text search.

Regular-expression matching is deliberately absent. Regex dialects, Unicode behavior, index support, and runtime cost differ across Ruby, PostgreSQL, and other targets. A stable whole-value pattern still belongs in a Field format Validation, where the target profile can validate one authored pattern and generate safety tests.

Field query profiles

The Field type determines which operators are meaningful. The Field's own properties refine that profile:

The Field catalog owns the current type matrix. In summary:

Reference and Association identities are not Field types. Compatible singular Association results support exact record equality. Relationship collections use exists.

The profile is derived from the Field type and its declared properties. The Plan does not carry an author-selected query_profile or repeat an operator list on every Field.

Null and missing values

The working semantic model uses SQL-style three-valued truth: true, false, and unknown.

is_null is a distinct unary variant:

{
  "kind": "is_null",
  "operand": {
    "target": {
      "field": "invite.expires_at"
    }
  }
}

Its operand is an optional queryable scalar Field, either direct or reached through singular Associations. A missing Association hop produces unknown; it is not reinterpreted as a null terminal Field. Association presence uses exists, which preserves that distinction in both SQL and record evaluation. A required terminal Field therefore remains ineligible for is_null even when an intermediate hop may be absent.

This model avoids the common trap where not equals unexpectedly admits nulls in one evaluator but not another. If a later application needs null-safe equality, it should add an explicit semantic operator comparable to SQL IS DISTINCT FROM rather than change equals.

Relationship predicates

exists brings each record returned by one Association into a nested Entity scope.

Association presence

Dunbar150 classifies root Comments by the absence of a parent:

{
  "kind": "not",
  "expression": {
    "kind": "exists",
    "association": "comment.parent"
  }
}

Without where, exists is true when the Association returns at least one record. It works for singular and collection Associations. not exists therefore means a missing singular result or an empty collection.

Associated record satisfies a condition

Shinar allows a Chat read when its active Members include the current Account:

{
  "kind": "exists",
  "association": "chat.members",
  "where": {
    "kind": "comparison",
    "left": {
      "target": {
        "record": "current"
      }
    },
    "operator": "equals",
    "right": {
      "kind": "environment",
      "name": "current_account"
    }
  }
}

The nested Expression runs against one Member at a time. Conditions inside the same where therefore apply to the same associated record rather than different rows in the collection. An Association's own Predicate remains part of that Association before exists evaluates it.

An empty Association makes exists false. A nested false or unknown does not satisfy it. Negating the node produces the ordinary “none satisfies” meaning.

In the reviewed three-Plan corpus, the current Rails lowering realizes five qualifying relationship representatives. The existing user.has_published_posts is a root exists with no where over the emitted predicated collection user.published_posts. It calls the emitted Post.published scope, selects the emitted author_id, and matches User id against that subquery. The outer relation is naturally deduplicated and completes in one SELECT. user.follows_anyone is the corresponding unpredicated form over the emitted user.outgoing_follows; it selects follower_id from all Follows and matches User id against that subquery. This collection target shape does not claim SQL EXISTS. The singular feed_occurrence.sourced_by_follow Predicate is instead a root unfiltered presence check on exact derived-forward feed_occurrence.source_follow. The Compiler emits:

scope :sourced_by_follow, -> { where.not(source_follow_id: nil) }

The optional immutable Reference requests nullification, and its integrity-checked non-null source_follow_id denotes an existing Follow. The ordinary non-deferrable foreign key rejects deletion while the referencing row remains because no admitted inverse can carry dependent: :nullify; the authored nullification is listed at the exact Reference pointer. This is an outer-row IS NOT NULL condition, not SQL EXISTS. For this unfiltered singular-presence projection, required, mutable, restricted, cascading, one-to-one, self, authored-inverse, indirect, filtered, nested, negated, grouped, reused, and every other singular shape remain valid authored meaning and listed gaps.

Two filtered forms compare an exact associated Account record with current_account. The generated scopes keep the runtime binding explicit:

scope :belongs_to_current_viewer, ->(current_account:) {
  current_account.is_a?(User) && current_account.persisted? && current_account.id.present? ?
    where(viewer_id: current_account.id) : none
}

scope :liked_by_current_account, ->(current_account:) {
  current_account.is_a?(User) && current_account.persisted? && current_account.id.present? ?
    where(id: Like.where(user_id: current_account.id).select(:post_id)) : none
}

The first is the exact cross-Entity required immutable non-one-to-one cascading derived-forward feed_occurrence.viewer Reference. The second is the exact cross-Entity unpredicated no-cardinality direct post.likes collection over a required immutable cascading Reference, followed by the same exact Account-Reference shape at like.user. Both Expressions are typed as current_record == current_account at the nested Account record. A nil, wrong-class, unpersisted, or destroyed argument returns an empty Relation. The generated model does not consult request, global, or thread-local state. Other filtered relationship shapes remain target gaps.

The current shape omits a universal every quantifier. Mature systems define every as true for an empty collection, which is often surprising. Add it only when a representative application needs that reviewed meaning often enough that not exists(not condition) is too indirect.

Nested Expressions cannot refer to the outer current record. A later outer-scope operand would need explicit syntax, type rules, SQL correlation, and instance-evaluation evidence.

Environment values

An environment value is supplied by the generated runtime and drawn from a closed, target-supported set:

Value Meaning
current_account The signed-in Account Entity record, or an absent value for a guest context.
current_date Today's date in the application's active time zone.
current_time The current timestamp in the application's active time zone.

current_account is compatible only with the Account Entity's record identity. Its absence produces unknown in an ordinary comparison and therefore denies a Policy that depends on it. The two generated relationship scopes above translate an absent or unusable runtime Account binding to an empty Relation.

params, arbitrary method calls, Ruby constants, JavaScript expressions, and arbitrary caller-selected Predicate arguments are not environment values. The two current-Account target scopes require a named current_account: keyword so request, job, task, test, and direct-model callers supply the runtime value explicitly. That target call shape is not a general authored Predicate-parameter feature. An environment_path Value may start from an admitted environment record such as current_account and follow the same typed path grammar used elsewhere. Semantic validation resolves its terminal type and treats a missing singular hop as an absent value.

Policy boundary

A base Expression cannot name a Policy. This preserves the dependency direction: Predicates, Validation conditions, and other ordinary consumers can use the Boolean algebra without depending on authorization operations.

Policy.allow_when points to the separate recursive $defs/policyExpression. Its ordinary nodes preserve the same comparison, null, Boolean, Predicate-reuse, and relationship semantics. Its recursion additionally admits matches_policy, allowing one Policy to consult a Policy on an associated record.

The Policies page (repository-only) owns the delegation syntax, dependency checks, and Action Policy lowering.

Authored and checked representations

The Foundation Plan should remain a small authored semantic tree. A Compiler can resolve it into a disposable checked representation:

authored Expression or Policy Expression
  → structural JSON Schema validation
  → semantic resolution and type checking
  → immutable checked semantic tree
       → SQL visitor
       → record visitor

The checked representation can carry resolved IDs, value types, current Entity scope, operator overload, terminal nullability, possible absence from a missing relationship or environment record, source JSON Pointer, and capabilities such as:

It is disposable Analyzer or Compiler working state, not a second approved Plan or another author-facing artifact. The Project graph's authored expression_data remains authoritative. Foundation Plan JSON is its external import/export and snapshot representation.

First Draft now persists and resolves all 25 Predicate roots, comprising 32 total Expression nodes in the six-Plan sketch/0.23 corpus. Its immutable resolver binds Field, Association, Predicate, nested-exists, current-record, and current-Account paths against the matching Association analysis; rejects missing or out-of-scope links and collection traversal in value paths; diagnoses transitive Predicate dependency cycles; and retains independent bindings when another root is blocked. A checked-in oracle compares all 20 bindings and 27 nodes in the original three-Plan corpus exactly with frozen Compiler revision f8334e3 after identity normalization.

First Draft now has the target-independent type foundation and pure single-root checker for that later phase. Six immutable Data values model scalar, text-comparison, fixed-currency money, closed-domain, record Entity-set, and opaque JSON types. Closed domains use stable Field subject UUIDs, and record sets use stable Entity subject UUIDs. This is an intentional sketch/0.23 adaptation of the frozen Compiler's readable semantic IDs.

The pure type system preserves the frozen Compiler's literal representations, comparison compatibility, directional assignment, and ordering rules. It receives an explicit, defensively copied map from Field subject UUIDs to their closed-domain local keys instead of querying Active Record. JSON compatibility expects a decoded value that has already crossed the strict Loader and immutable analysis-projection boundary. The type system does not repeat I-JSON parsing or finite-number checks.

An eleven-query REPEATABLE READ capture composes the resolved Predicate Expression graph with exact Field profiles, closed-value catalogs, and reference plus development Data Record headers. The checker combines that immutable value with complete same-generation Association facts, then returns immutable typed Value bindings, operator bindings, and source-addressed diagnostics without Active Record or SQL. A single resolved root carries no generation provenance of its own; its caller must supply a root resolved from this graph generation. The later phase-level orchestrator owns that cross-stage check. The checker keeps terminal nullability distinct from possible absence and exposes the broader standalone Value profile needed by defaults and later consumers. Unlike the query profile, the standalone profile admits opaque JSON, rich text, and encrypted-at-rest Fields; attachment and image Fields remain unsupported. Its allow_literal_null input controls whether an explicitly authored null is accepted; that null is the assignment-level absence marker even when the expected semantic type is opaque JSON, rather than a present JSON payload. Consumers still inspect the returned binding's nullable and may_be_absent facts for nonliteral Values. When a sketch/0.23 path terminates at any direct referencing Association for a Reference, the terminal retains that Reference's requiredness as nullability. An authored alias may additionally be absent when its Predicate filters the stored target; that qualifier-driven absence stays separate from the underlying Reference nullability. An intermediate traversal through the same Association instead contributes possible absence: an explicit Association target asks for the stored Reference value, while through establishes a nested evaluation scope that has no current record when the hop is missing. Unqualified aliases for one Reference on the same side identify the same record for impossible self-comparison analysis, while opposite sides and Predicate-qualified aliases remain distinct. This preserves the frozen model's distinction after Reference paths were unified with the Association catalog. The frozen three-Plan oracle contains no record-typed path, so focused tests based on the frozen source pin this adaptation rather than the oracle. Same-path equals, less_than_or_equal_to, and greater_than_or_equal_to remain a separate suspicious-rule case: SQL null semantics mean they are not universally true, so the current impossible-comparison diagnostic does not apply and tautology-oriented analysis remains open.

Validation comparison typing additionally treats an ordinary Reference terminal and its exact mechanically derived, same-key forward Association as one semantic record path. This lets not_equals reject an impossible comparison while equals remains coherent. Authored aliases, opposite-side traversals, and Predicate-qualified Associations remain distinct.

An environment_path inside an Expression contributes a binding for its inner Path and another for the enclosing Value. The standalone Value entry point returns only that enclosing binding. No Predicate in the frozen or current corpus exercises this shape; focused tests pin the frozen checker's convention.

A checked-in oracle exactly reproduces the frozen Compiler's three-Plan projection: 20 roots, 27 nodes, 41 Value bindings, and 19 operator bindings. The projection carries no diagnostics. All six design-and-parity Plans type-check cleanly, so diagnostic codes, source locations, related locations, and multi-diagnostic ordering are pinned by focused tests rather than frozen-Compiler parity. The six established design-and-parity Plans type-check as 25 roots, 32 nodes, 51 Value bindings, and 24 operator bindings. The Predicate-wide checker validates complete same-generation resolver evidence, checks every available root before propagating type failures through explicit and Association-implied Predicate dependencies, and returns unrelated bindings plus stage-local diagnostics. It treats the resolver's immutable resolved result as a process-owned receipt: provenance, coverage, readable paths, root pointers, and dependency shape are checked without re-resolving the authored Expression JSON. A shared evidence checker cross-checks direct matches_predicate leaves and Association-implied transitive dependencies against each available binding's declared dependency vector. Blocked roots carry no Expression or dependency vector, so their semantic provenance remains part of the trusted in-process resolver receipt rather than something a later stage can reconstruct. Like the resolver result, the type result omits a duplicate Predicate census; downstream stages must re-check its bindings plus blocked UUIDs against the same immutable graph.

The pure Policy-wide checker applies the same Value and operator semantics to every available resolved Policy root. It requires the Policy graph, shared Expression-type graph, whole-Policy resolution, Predicate-wide type result, and Association facts to describe the exact same Project generation. It validates complete prerequisite coverage and exact Predicate, Policy, and Association dependency evidence; checks all available roots before propagating local, Predicate, and Policy blocks; retains independently usable typed bindings; and reports only its own source-addressed diagnostics without Active Record or SQL. A checked-in six-Plan oracle matches the frozen Compiler's 21 Policies, 29 nodes, 28 Value bindings, and 14 operator bindings. The type result likewise carries no duplicate Policy census, so downstream stages must re-check its bindings plus blocked UUIDs against the same immutable graph.

The pure State Machine applicability checker applies those same Value and operator semantics to every available conditional applicability root while retaining explicit nil/empty typed evidence for required unconditional machines and the resolver's source pointer on every binding. It requires the applicability graph, shared Expression-type graph, applicability resolution, Predicate-wide type result, and Association facts to describe the same Project generation. Policy and State Machine typing share one structural Predicate-type evidence validator rather than independently interpreting that receipt. The applicability type result carries no duplicate machine census, so downstream stages must re-check its bindings plus blocked Field UUIDs against the same immutable applicability graph. Every available conditional root is checked before local and Predicate blocks are applied, so an otherwise doomed root still reports its own type diagnostics while unrelated bindings survive. A checked-in six-Plan oracle matches all 17 frozen bindings, including the sole conditional Expression, its 2 Value bindings, and its 1 operator binding. Sixteen of those machines are unconditional, so the oracle broadly proves only the nil/empty binding shape and provenance; its conditional parity covers one comparison shape. Focused applicability cases own block, dependency, and ordering behavior, while the shared single-root checker suite owns broader Value and operator semantics. The public Compiler now consumes the exact resolved and typed evidence for one bounded local recursive SQL projection. It admits supported direct-Field comparison, membership, and null-test leaves, Boolean composition, and same-Entity Predicate reuse only when every nested child lowers. One bounded direct Association reuses the admitted result-Entity scope from that same Compiler input. After that relationship catalog is fixed, one projection appends the two exact root unfiltered collection-existence scopes and one exact root unfiltered singular-reference existence scope that the Rails scopes page describes. After public Account qualification, another appends the two exact typed current-Account relationship scopes. Other filtered or composed relationship traversal, general record evaluation, and broader target lowering remain unimplemented. The public Policy lowering consumes the exact typed results for 17 bounded record roots and seven admitted relation demands: Bookmark and Rating manage, plus Notification, Case, Conversation, ConversationThread, and Message read. Generalized Web Scaffold consumers use those exact record and relation decisions. One private Policy-gate qualifier separately retains the historical typed current_record == current_account request-gate proof.

Each consumer asks for the projection it needs. A collection Predicate or item-level Policy needs set-based SQL. A Validation condition needs record evaluation. A Policy may need both. The Compiler omits a consumer and lists the gap when the selected target cannot preserve the same truth conditions in every required projection in the current strict renderer. A future pre-alpha renderer may instead retain a conventional unfiltered or unguarded scaffold consumer as starter code when the same reviewed gap remains explicit and the result does not claim the Predicate or Policy projection.

One semantic tree does not require one executable body. SQLAlchemy hybrids explicitly permit separate instance and SQL implementations. Foundation can likewise use separate visitors while testing them against the same semantic cases.

Why Relation membership is not record evaluation

An Active Record Relation answers which persisted database rows match a query. It is not a pure evaluator for the current in-memory state of one record.

Relation#include?(record) changes behavior with relation state. Rails searches loaded, limited, offset, or grouped records in memory. Otherwise it checks existence by the record's primary key. An unsaved record has no persisted identity, and a saved record may still contain unsaved changes that differ from the database row. Reloading would discard those changes rather than evaluate them.

Saving first can make an unloaded relation membership check appear to work, but the answer then describes the persisted row and may perform hidden I/O. It also makes conditional validation circular because the condition may need to run before persistence.

The idiomatic generated shape is therefore one checked semantic Expression with separate SQL and record evaluators. The Compiler can omit a record evaluator when no consumer requires one.

Separate filtering and ordering

The same qualifying records may appear in different sequences, so an Ordering remains its own subject. Case Chat defines newest-opened Cases with an explicit stable tie-breaker:

{
  "subject_uuid": "01a02b34-a965-7b37-96ed-9a15602535d2",
  "key": "recently_opened",
  "terms": [
    {
      "field": "case.opened_at",
      "direction": "descending"
    },
    {
      "system_field": "id",
      "direction": "descending"
    }
  ]
}

The current Compiler does not silently repair an incomplete order. Its explicit-tie-breaker subset requires the final authored term to be system id. Its all-Field alternative is admitted only when the exact term set already has an emitted unconditional uniqueness guarantee, so both routes derive stability from structured Plan meaning.

A Scaffold index or displayed collection may name both a Predicate and an Ordering. They remain independently reusable. An Association may name a Predicate because that changes which records the traversal returns, but it does not own an Ordering in the current format.

An Entity's optional implicit_order_column supplies an ascending fallback for ordered finder calls such as first and last. It does not order an ordinary relation or loaded Association. Omission requests the first profile's created_at, then id, default. An explicit tuple uses typed Field or system-Field locators and ends in id for deterministic ties.

The capture-bound Rails Compiler currently realizes one authored shape: Case Chat's Message selects required, stored sent_at followed by system id. It emits self.implicit_order_column = [:sent_at, :id] and an ordinary matching composite index. This does not create a named Ordering scope, widen general Ordering admission, or make the fallback available to Scaffold ordering.

Rails has merged a separate default_order relation and Association option that behaves as a fallback until an explicit order is supplied. That API is not in the current Rails 8.1 profile, so the Plan does not expose it. A Compiler should not approximate it with an ordinary Association scope whose order would compose with, rather than yield to, later orderings.

The current Ordering shape supports:

Every term requires a direction. Field and system-Field terms may place nulls last. Tree terms reject Field, system-Field, and null-placement keys instead of asking a Compiler which meaning wins.

The Rails profile needs to validate that the chosen Field is queryable and that the emitted expression is safe. in_order_of needs the complete enum value set when it should reorder without filtering rows.

First Draft's pure Ordering projection covers all 30 Orderings and 60 authored terms in the six established design-and-parity Plans: 28 Field, 30 system-Field, and two Tree-derived terms. It resolves readable Field and Tree locators to stable subject UUIDs and preserves owner, order, direction, null placement, Tree value, and source locations. Missing or cross-Entity sources reject the entire slice without touching Active Record. Repeated semantic sources and Rails-ineligible Fields remain representable for later whole-graph and target analysis. The live graph persists the same 30 roots and 60 terms with stable subject identities, resolved same-Entity Field and Tree links, closed system Fields, direction, null placement, Tree values, and authored positions. Deferred uniqueness covers owner-local root keys, root positions, and per-Ordering term positions; repeated sources remain deliberately non-unique.

The bounded importer now reconciles the Ordering roots whose source subjects survive service pruning. Replacement preserves stable root identities, removes stale terms before a source or owner changes, and retains deleted subject claims. The strict serializer reconstructs every retained Field, system-Field, or Tree term and reaches an exact byte fixpoint on a fully admitted representative graph.

The immutable Compiler projection retains every imported root and term with Project-generation provenance. It reports a source-addressed semantic error when one Ordering repeats the same source. Its first Rails lowering admits only a conservative indexed-scope subset:

An admitted Ordering emits one chainable scope whose order hash preserves every term and one ordinary composite B-tree index over the same columns. PostgreSQL can scan that index forward or backward for a homogeneous direction, so the schema does not encode DESC. When an admitted unique index already has the same ordered columns, it subsumes the redundant ordinary index while retaining both subjects in provenance. That general lowering does not claim mixed-direction, nullable, enum-rank, Tree-derived, or consumer-specific lowering.

The capture-bound composition additionally admits two structural patterns. Neither requires Oscar's Plan SHA, subject UUIDs, or readable paths; names still pass the collision checks below:

Both require realized Field storage without encryption, derivation, normalization, or comparison modifiers and safe model scope names without Predicate collisions. They retain the existing index-name collision rules: equivalent admitted claims share one index; incompatible claims withhold the affected Ordering. An existing ordinary Ordering, implicit-order, Reference, or uniqueness index takes precedence over a conflicting advanced index. No alternative index name is allocated. Structural scopes reserve their names in the same enum-helper and AASM fixed-API checks as ordinary scopes. An unsupported sibling cannot claim ownership of a shared admitted index.

Each result still consumes its exact central CompilationInput, submitted Head, and source coordinates. Analysis and Compile use the same structural admission; Compile preserves the reviewed GapSet unchanged. These identity checks establish provenance, while the Field and term rules determine feature support. Reformatting a Plan or changing unrelated metadata cannot change that support.

The independent-Plan qualification (repository-only) owns the new support evidence. Oscar Party still contributes four named scopes, while Case Chat and Photogram contribute five each, 14 total. The older exact-Oscar receipt remains historical proof of that application, not the structural-family qualification. Direct rendering without a captured input does not run this composition. Every retained but inadmissible Ordering receives a target-support gap. A service-pruned Field or Tree dependency keeps its source-addressed service gap. Scaffold selection of either added pattern still has its own consumer gap and deterministic id fallback.

Single-term and broader uniqueness-backed shapes, stable-term completion, general in_order_of, Ordering consumers, and broader target semantics remain open.

Per-owner top-N, latest-per-group, computed rankings, and timeline unions remain outside the current shape. Naively putting limit on a preloaded Association can change a per-owner limit into a global limit.

Predicate boundaries

Before adding a parameterized Predicate, inspect what the parameter represents:

Unsupported residue remains outside the Plan. This keeps Predicate smaller than a serialized query program without preventing a repeated closed pattern from entering later.

Rails realization and target checks

The Rails scopes page owns how admitted Predicates and Orderings become named scopes, how Associations and existence scopes reuse them, and which target checks run before emission.

Alternatives and rationale

Open questions

  1. How should the shared Value and type rules apply to remaining consumers such as State Machine effects, defaults, bindings, and initial data?
  2. What CQL2-derived semantic corpus, extended for Foundation relationships and reuse, should check the algebra?
  3. Is relative time needed in the first Compiler slice?
  4. Do the fixed Compiler safety ceilings need profile-level configuration or distinct diagnostics?
  5. Which Predicates receive record evaluators?
  6. Which query uses justify generated indexes?
  7. How should emitted Predicate reuse extend beyond the current direct Association and Scaffold shapes to other generated consumers?

References marked “repository-only” name implementation or internal material outside this public guide. They are intentionally not links. Publishing a design does not prove its implementation.