Searching entities

Canonical reference: cyoda help search

This page is the narrative; the binary is authoritative.

Canonical reference: cyoda help predicates

This page is the narrative; the binary is authoritative.

Cyoda exposes search over REST for any query that returns a set of entities. Use it when you need more than one entity back, when the filter goes beyond a single id lookup, or when you want to scope by workflow state. For single-entity reads, stay on the CRUD endpoints in working with entities; for cross-entity analytics, use SQL; for event-driven compute, use gRPC compute nodes. For counts and aggregates without the entity bodies, jump to grouped statistics.

Two query modes

Cyoda splits search into Immediate and Background modes. Pick by expected result size and urgency; the filter grammar is identical.

• Immediate (API term: direct) — synchronous. The request streams matching entities back as NDJSON. It is bounded-or-fail: limit caps the matched set, and a query matching more than that fails outright rather than returning part of the answer. Use it when you know the filter produces a bounded, small set — a UI list, a lookup, a small report.
• Background (API term: async) — queued. The request returns a job handle; poll it, then page the results. Result size is unbounded. On the Cassandra-backed tier (Cyoda Cloud, or a licensed Enterprise install), async runs distributed across the cluster: for a fixed query shape, throughput scales roughly linearly with the number of nodes.

Direct search fails rather than truncating

A direct search whose matched set exceeds the effective limit returns 400 SEARCH_RESULT_LIMIT. It does not return the first limit entities. This is deliberate: a truncated page was indistinguishable from a complete one, so callers silently processed partial data.

• limit defaults to 1000 when omitted, and caps at 10000.
• limit=0 — which once meant “unbounded” — is rejected with 400 BAD_REQUEST, as is any value below 1. gRPC rejects limit < 1 identically.
• A matched set exactly equal to limit succeeds.

When you hit it, narrow the condition, raise limit within the maximum, or move the query to async.

The decision tree is short:

• Small bounded result, UI-facing → direct.
• Might be large, can tolerate a second or two of queuing; exports, reports, batch jobs → async.
• Hitting SEARCH_RESULT_LIMIT or the request timeout → async.
• Ordered top-N over a large model → async (see sorting).

A direct search

Filter by a combination of entity fields and workflow state:

curl -X POST 'http://localhost:8080/api/search/direct/orders/1?limit=200' \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "group",
    "operator": "AND",
    "conditions": [
      { "type": "lifecycle", "field": "state",
        "operatorType": "EQUALS", "value": "submitted" },
      { "type": "simple", "jsonPath": "$.customerId",
        "operatorType": "EQUALS", "value": "CUST-7" }
    ]
  }'

The path is /api/search/direct/{entityName}/{modelVersion}, and the request body is the condition document itself — there is no wrapper object. limit, pointInTime and trackingRead are query parameters, not body fields.

The response is application/x-ndjson: one complete entity envelope per line, each carrying data plus a meta block with the id, model key, state, and timestamps.

{"type":"ENTITY","data":{"customerId":"CUST-7"},"meta":{"id":"74807f00-…","state":"submitted","creationDate":"2026-03-01T10:00:00.000000000Z"}}
{"type":"ENTITY","data":{"customerId":"CUST-7"},"meta":{"id":"89abc100-…","state":"submitted","creationDate":"2026-03-02T11:30:00.000000000Z"}}

Because it is a stream, parse it line by line rather than buffering the whole body as a single JSON document.

An async search

Submit the search to /api/search/async/{entityName}/{modelVersion} and capture the handle:

curl -X POST http://localhost:8080/api/search/async/orders/1 \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "lifecycle",
    "field": "state",
    "operatorType": "EQUALS",
    "value": "submitted"
  }'

The response is a bare UUID string — the job id. Poll its status until the job reports SUCCESSFUL, then page the results:

GET /api/search/async/{jobId}/status
GET /api/search/async/{jobId}?pageNumber=0&pageSize=1000
GET /api/search/async/{jobId}?pageNumber=1&pageSize=1000

The status response carries searchJobStatus (RUNNING, SUCCESSFUL, FAILED, CANCELLED, or NOT_FOUND), entitiesCount for the total match count, and expirationDate. Fetching results before the job is SUCCESSFUL returns 400 BAD_REQUEST.

Results come back as a paged JSON object — content plus a page block with number, size, totalElements and totalPages — not as a stream. Job results expire 24 hours after creation; a single jobId can be paged repeatedly until then.

Async search never returns SEARCH_RESULT_LIMIT: it snapshots the whole matched set and pages over it. An out-of-range pageSize or pageNumber is a plain 400 BAD_REQUEST.

Cancelling a job

If a job is no longer needed — the user navigated away, a replacement query was submitted, the deployment is shutting down — cancel it rather than letting it run to completion:

curl -X PUT http://localhost:8080/api/search/async/{jobId}/cancel \
  -H "Authorization: Bearer $TOKEN"

Cancellation is cooperative: in-flight work is stopped at the next safe point and any partial results for that jobId are discarded.

The condition DSL

The request body is a Condition document. Every condition carries a type discriminator. The same DSL is used by workflow and transition criteria, so what you learn here transfers directly.

simple — match a JSONPath into the entity payload:

{ "type": "simple", "jsonPath": "$.amount",
  "operatorType": "GREATER_OR_EQUAL", "value": 1000 }

lifecycle — match entity metadata rather than payload. field is one of state, creationDate, lastUpdateTime, transitionForLatestSave (alias previousTransition), transactionId, id:

{ "type": "lifecycle", "field": "state",
  "operatorType": "EQUALS", "value": "submitted" }

group — combine conditions. operator is AND or OR; these are the only two, and NOT is not supported. Groups nest to a maximum depth of 50:

{
  "type": "group",
  "operator": "AND",
  "conditions": [
    { "type": "lifecycle", "field": "state", "operatorType": "EQUALS", "value": "submitted" },
    { "type": "simple", "jsonPath": "$.amount", "operatorType": "GREATER_OR_EQUAL", "value": 1000 }
  ]
}

array — match positional values in a JSON array, where null matches any value at that index:

{ "type": "array", "jsonPath": "$.laureates",
  "values": ["John", null, "Hopfield"] }

function — delegate the predicate to a compute node. The condition is dispatched as an EntityCriteriaCalculationRequest, exactly like a workflow criterion:

{
  "type": "function",
  "function": {
    "name": "my-criteria-fn",
    "config": {
      "calculationNodesTags": "approval-service",
      "attachEntity": true,
      "responseTimeoutMs": 30000
    }
  }
}

function.name and config.calculationNodesTags are required for routing; attachEntity defaults to true and responseTimeoutMs to 30000.

A function condition cannot be pushed down

function is the one condition type that never translates to a storage-plugin filter. It always runs as a post-filter with in-memory entity loading, so every candidate entity is materialised and dispatched to your compute node. Combine it with a selective simple or lifecycle condition in an AND group so the pushdown narrows the population first — a bare function condition over a large model means one callout per entity.

operatorType also accepts the spellings operator and operation. The available operators are:

| Group | Operators | |---|---| | Comparison | EQUALS, NOT_EQUAL, GREATER_THAN, GREATER_OR_EQUAL, LESS_THAN, LESS_OR_EQUAL | | Range | BETWEEN, BETWEEN_INCLUSIVE — value must be a two-element [low, high] array | | Presence | IS_NULL, NOT_NULL | | String | CONTAINS, STARTS_WITH, ENDS_WITH and their NOT_* inverses | | Case-insensitive | IEQUALS, INOT_EQUAL, ICONTAINS, INOT_CONTAINS, ISTARTS_WITH, INOT_STARTS_WITH, IENDS_WITH, INOT_ENDS_WITH | | Pattern | LIKE (anchored glob), MATCHES_PATTERN (anchored RE2 regex) |

An operator outside this list is rejected with 400 BAD_REQUEST, and the error detail includes the canonical list. IS_CHANGED and IS_UNCHANGED are change-generation operators, not search predicates — cyoda-go does not implement them.

Matching everything

An empty body ({}), or any body without a type, is rejected with 400 BAD_REQUEST. To match every entity in the model, send a vacuous AND group:

{ "type": "group", "operator": "AND", "conditions": [] }

An AND group with no conditions is true; an OR group with no conditions is false.

Predicate semantics

Search and workflow criteria run on one shared evaluation kernel, so a predicate means the same thing wherever you write it. The kernel implements Cyoda Cloud’s model; cyoda help predicates is the exhaustive reference.

Comparison is type-directed and same-type. An operand is parse-tested against the field’s declared type(s), so the JSON number 30 and the string "30" behave identically. There is no cross-type coincidental matching: an operand that parses only as a number never matches a string-stored value. Numbers compare at arbitrary precision, so values beyond 2^53 are exact.

Absent and null fields never match a binary operator — including negatives. NOT_EQUAL, NOT_CONTAINS, INOT_* and the rest are null-guarded to non-match rather than evaluated as !positive. A missing field does not satisfy NOT_EQUAL just because it fails EQUALS. Use IS_NULL and NOT_NULL to test presence — they are the only operators that do.

LIKE is a real anchored glob. % matches any run of characters, _ matches exactly one, and \ escapes a literal %, _ or \. The match is whole-string and case-sensitive. MATCHES_PATTERN is a whole-string-anchored RE2 regex; RE2 diverges from Java’s dialect on constructs such as backreferences and some lookaround.

Temporal fields compare chronologically, not lexically, across LOCAL_DATE, LOCAL_DATE_TIME, LOCAL_TIME, ZONED_DATE_TIME, YEAR and YEAR_MONTH. The creationDate and lastUpdateTime meta fields accept a coarser operand that upscales — "2024" or "2024-09" are valid against a full timestamp.

Validation is parse-based, evaluated against the target model:

• 400 CONDITION_TYPE_MISMATCH — the operand parses into none of the field’s declared types. Comparison and range operators only; string operators and presence tests carry no operand-type constraint.
• 400 INVALID_FIELD_PATH — the path is unknown to the model, or it names a pure container (an object) and you used a scalar operator. Navigate to a scalar leaf instead. A path seen as both an object and a scalar across entities stays searchable via its scalar type. IS_NULL/NOT_NULL are exempt, since they test presence rather than a value.
• 400 INVALID_CONDITION — a null operand on a binary or range operator, a range operand that is not a two-element array, or an object operand.

There is no operator-versus-type rejection: CONTAINS on a numeric field is a valid request that simply evaluates to a non-match.

Your existing queries may return different results

cyoda-go v0.8.3 converged two predicate evaluators that had drifted apart, so the same query could previously return different results depending on which one ran. Aligning them changed observable behaviour:

• cross-type coincidental matches no longer occur;
• negatives no longer match null or absent fields;
• LIKE is a real glob on every backend, where SQL backends previously neutered the wildcards;
• BETWEEN_INCLUSIVE performs an inclusive range check, where Searcher-backed stores previously fell through to a regex evaluation;
• validation is parse-based, so some queries that used to fail now succeed, and some that used to pass are now rejected.

Re-check any query whose correctness you depend on, particularly negations and LIKE patterns.

Historical reads with pointInTime

Every search accepts a pointInTime parameter to run against the world as it existed at a given timestamp. Each entity maintains a history of revisions; point-in-time queries return results using the entity state that was current at the specified timestamp. The result is the set of entities that would have matched, using the revision active at that time.

The rule is canonical across every storage engine and read path: inclusive of the requested instant (<=), compared at native precision with no millisecond round-up.

pointInTime is a query parameter (RFC 3339), not a body field:

curl -X POST 'http://localhost:8080/api/search/direct/orders/1?pointInTime=2026-03-01T00:00:00Z' \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "lifecycle", "field": "state",
    "operatorType": "EQUALS", "value": "submitted"
  }'

This is the primary way to answer audit and regulatory questions from REST — what did this customer’s open orders look like at quarter close? The Trino surface exposes the same capability as a column named point_time (snake-case, matching SQL convention); for the analytical form, see point_time in analytics.

Sorting results

Both search endpoints (direct and async) accept sort keys. Over HTTP, add one or more sort query parameters using the grammar [@]path[:asc|desc]:

• a bare dotted path sorts on a scalar data field (amount, customer.tier);
• an @-prefix sorts on a meta field — one of state, creationDate, lastUpdateTime, transitionForLatestSave, transactionId, id;
• direction defaults to asc.

POST /api/search/direct/orders/1?sort=@state&sort=amount:desc

Ordering is canonical across every backend: text by byte order, numeric by IEEE-754 double, bool as false < true. Meta date fields order chronologically; temporal data paths are ordered as text, which for ISO-8601 values coincides with chronological order. Note that this is the sort contract — predicate comparison uses the arbitrary-precision, type-directed rules described under predicate semantics. Absent or null values sort last, and the entity id is always the final tiebreaker. An unsortable, array, or unknown path is rejected with 400 INVALID_FIELD_PATH, as is exceeding the sort-key cap set by CYODA_SEARCH_MAX_SORT_KEYS (default 16; see the configuration reference). Over gRPC the same capability is expressed as a structured orderBy array.

Ordered top-N belongs on the async path

Sorting a direct search does not let you take the top N of a large population. Because direct search is bounded-or-fail, the whole matched set must fit within limit before ordering is applied — so sort plus a small limit over a large model returns 400 SEARCH_RESULT_LIMIT rather than the first N rows.

Run these as async searches, which snapshot the full matched set and page over it in order.

Paging (async)

• pageSize and pageNumber are query parameters on /search/async/{jobId}; they apply at result-fetch time, not at job submission. pageNumber is zero-indexed.
• A completed jobId is stable for its 24-hour retention window — page reads are idempotent.

Searching inside a transaction

A search issued inside an active transaction is read-your-own-writes correct: it sees that transaction’s uncommitted writes, without falling back to a full-model scan. The memory and SQLite backends overlay the transaction buffer on the committed stream; PostgreSQL runs the query natively on the transaction’s own connection.

By default the search is a plain snapshot read that records nothing. Pass trackingRead=true to opt into recording the entities the search returns into the transaction’s read-set, so that a concurrent commit touching any of them aborts your transaction with 409 Conflict at commit time:

POST /api/search/direct/orders/1?trackingRead=true

The protection is entity-level, the same as a GetAll read. Neither setting protects against phantoms — an entity that starts matching your predicate after the snapshot was taken is not detected either way. The flag is ignored outside a transaction, and async search does not expose it at all, since it runs detached.

Performance notes

• Scope by state or a high-selectivity field first — the workflow state is indexed on every entity and is almost always the right first predicate.
• Prefer async as soon as the result set might be thousands of entities; the distributed execution on the Cassandra tier makes it cheaper per entity than a series of direct pages.
• Avoid open-ended pointInTime scans across every revision — anchor the query at a specific timestamp or a short window.
• On PostgreSQL, supported predicates push down into SQL — JSONB extraction plus numeric, range, and string comparisons run in the database. Non-pushable operators (regex, case-insensitive) are post-filtered as rows stream. This removes full-model scans and per-document decode; it is a constant-factor win, not a JSON-path index (indexing queried paths remains a separate operational step). SQLite already does this; the in-memory backend filters in memory by design.
• Pushdown is a narrowing optimization only — the in-process kernel remains authoritative for every match decision, so results never diverge between backends. Pushdown applies inside transactions too, so an in-transaction search no longer degrades into a full GetAll scan.
• Boolean conditions on PostgreSQL are pushed down correctly as of v0.8.3. Previously an EQUALS/NOT_EQUAL against a boolean field returned 500, because the planner bound a raw Go bool against a text-typed JSON extraction that the driver cannot encode.
• A condition the plugin cannot translate falls back to in-memory filtering after a full scan. If that residual scan examines more rows than the backend’s budget allows, it fails with 400 SCAN_BUDGET_EXHAUSTED — narrow the query or add an indexable predicate rather than retrying.

Grouped statistics

When you want counts and aggregates rather than the entities themselves — how many orders are in each state? what is the total amount per country? — use the grouped-statistics query instead of paging a search result and summing client-side. It returns one row per group and never the underlying entity bodies, so it stays cheap over large populations.

POST /entity/stats/{entityName}/{modelVersion}/query

curl -X POST http://localhost:8080/api/entity/stats/orders/1/query \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "groupBy": ["state", "$.country"],
    "aggregations": [
      { "op": "sum", "field": "$.amount", "as": "totalAmount" },
      { "op": "avg", "field": "$.amount" }
    ],
    "condition": {
      "type": "simple",
      "jsonPath": "$.channel",
      "operation": "EQUALS",
      "value": "web"
    }
  }'

The request has four parts:

• groupBy (required) — an ordered list of dimensions. Each entry is either the literal state (the workflow state) or a $.-prefixed JSONPath into the entity payload. The result groupKey is ordered to match.
• aggregations (optional) — per-group numeric aggregates. Each is { "op", "field", "as" } where op is one of sum, avg, min, max, stdev, field is a JSONPath, and as is an optional alias (defaults to op(field), e.g. sum($.amount)). Every group also carries a count whether or not you request aggregations.
• condition (optional) — a predicate that restricts the population, using the same condition DSL as search.
• limit (optional) — caps the number of buckets returned; must be ≤ the server’s CYODA_STATS_GROUP_MAX (default 10000).

The response is one bucket per group:

[
  {
    "groupKey": [
      { "field": "state", "value": "submitted" },
      { "field": "$.country", "value": "GB" }
    ],
    "count": 31,
    "aggregations": { "totalAmount": 48210.0, "avg($.amount)": 1555.16 }
  }
]

Aggregation values are numeric, or null when a group has no eligible inputs.

Point-in-time statistics

Like search, the query accepts a pointInTime to aggregate the world as it existed at a past instant — without standing up a derived counter entity that processors must maintain on every transition. Add it to the request body:

{
  "groupBy": ["state"],
  "pointInTime": "2026-03-31T23:59:59Z"
}

This answers end-of-quarter and audit roll-up questions directly — how many claims were pending at midnight on the last day of Q1? When omitted, the query runs against the current consistency time.

Where to go next

• REST API reference — authoritative search payload schema, operator grammar, status and result endpoints.
• Working with entities — single-entity CRUD and transitions; the CRUD page for reference on the same surface.
• Analytics with SQL — heavy analytical work, cross-entity joins, historical scans via point_time.
• Entities and lifecycle — the audit/history model behind pointInTime.