Skip to content

Value presentation

Purpose

How a client of the platform (the TUI, the web app app) shows platform data: state entries and events as one thing, a Value, whose kinds are decided from the instance, filtered with one expression language, grouped by name as a tree, followed as ValueStreams and shown in the representations a data file maps each kind to. The spec is client-neutral: it fixes the data model, the choice rules and the file formats, and shared conformance vectors make every client decide alike; how a client draws and which keys it uses belong to its own spec (Terminal UI).

Related specs: - Terminal UI — the TUI's filter line, keys, tree views and how it draws the representations. - Pub/Sub Topics — the CloudEvent envelope (type, source, subject, datacontenttype, dataschema, time) a Value's origin is read from, and the dotted topic and state-key names the Name Tree splits.

Requirements

Requirement: Values

A client SHALL show state entries and events as Values: decoded data plus its origin.

Field State entry Event
source state event
name the key the topic
service the state service –
etag the entry's etag –
ce_type, ce_source, subject – the CloudEvent's type, source, subject
content_type when known datacontenttype
dataschema – dataschema
time when the value was read the CloudEvent time, else when it was received

The data SHALL be decoded one way for both sources: text (a state value, an event's text data) that parses as JSON to a JSON value, other text to a string; binary data of a JSON content type, or with no content type, that parses as UTF-8 JSON to a JSON value, binary data of a text/* content type to a string, any other binary data to bytes. The one-line text of a Value SHALL be its data as compact JSON, a string as itself, bytes as <N bytes>.

Scenario: State entry and event alike

  • WHEN the same JSON document is a state value and the data of an event
  • THEN both SHALL decode to the same data, and differ only in their origin

Scenario: An order book image stays binary

  • WHEN an event carries an order book FlatBuffers image (application/x-virtufin-orderbook+flatbuffers)
  • THEN its data SHALL be the bytes, its one-line text <N bytes>

Requirement: Kinds by Instance

A Value's kinds SHALL be decided from the instance (its data and origin), never from a declared type of the client's language. A Value SHALL have every kind whose match it satisfies, in rule order, without duplicates, and always the catch-all kind * last.

A match SHALL be an object whose predicates all hold:

Predicate Holds when
json_type the data's JSON type is this one: number, string, boolean, null, object, array
has_keys the data is an object holding all these keys
dataschema the origin's dataschema equals it
content_type the origin's content type matches it (glob)
ce_type the origin's ce-type matches it (glob)
name the key or topic matches it (glob)
source the origin's source is state or event
decoder the named decoder accepts the data

A decoder SHALL be the only code in kind matching, for a binary format a match cannot describe; every client SHALL implement the decoders the shipped rules name, from the owning devkit's package in its language (orderbook.image: an order book FlatBuffers image, urn:virtufin:marketdata:orderbook:v1, read with market-data-devkit).

The rules SHALL be data: the kinds list of a JSON file representations.json ({"version": 1, "kinds": [{"kind": "<name>", "match": {...}}]}), validated by its JSON Schema. A client SHALL ship the default file with its code, never as a copy in the user's configuration, and SHALL match the rules of the user's file, when there is one, before the shipped ones. A rule with an unknown predicate, JSON type or decoder, and an unreadable user file, SHALL be reported once and ignored.

The shipped rules SHALL be, in order: orderbook (decoder orderbook.image; or an object with bids, asks, last_update_id), tradeflow (an object with units, buyunits, sellunits), number, text (json_type string), json.object, json.array.

Scenario: Same key, different kinds

  • WHEN a key held a number and later holds a JSON object
  • THEN the first value SHALL have the kinds number, * and the later one json.object, *

Scenario: The order book from either source

  • WHEN an event carries an order book image and a state entry holds the saved book as JSON
  • THEN both SHALL have the kind orderbook, the saved book also json.object

Scenario: A user kind

  • WHEN the user's file has a rule limit matching name: cfg.* and json_type: number
  • THEN the number at cfg.limit SHALL have the kinds limit, number, *

Requirement: Filter Expressions

A client SHALL filter Values with one expression language: whitespace-separated terms, and a Value SHALL pass when every term holds.

Term Holds when
word the Value's name (key or topic) or its one-line text contains it
field:pattern that field of the Value matches pattern
-term term does not hold
"two words" quoted: the spaces belong to the term, in a word or a pattern
Field State entry Event
name key topic
kind any of the Value's kinds same
type – ce-type
source state service ce-source
subject – ce-subject
value the Value's one-line text same

Matching SHALL be case-insensitive. A pattern with * or ? SHALL be a glob over the whole field; any other pattern SHALL be a substring. A field the Value does not have SHALL never match (so its negation holds). The empty expression SHALL pass every Value. An expression that does not parse (an unknown field, an unclosed quote, a - or field: with nothing after it) SHALL be reported with its position and SHALL filter nothing until it parses.

Scenario: Words and fields

  • WHEN the expression is name:cfg.* -name:*.index
  • THEN the state entries whose keys start with cfg. SHALL pass, except those ending in .index

Scenario: A field the Value lacks

  • WHEN the expression is type:*.gap
  • THEN no state entry SHALL pass, and the events whose ce-type ends in .gap SHALL

Scenario: Kinds

  • WHEN the expression is kind:orderbook
  • THEN a saved order book (JSON) and an order book event (image) SHALL both pass

Requirement: Worker Filter

A client SHALL filter workers with the Filter Expressions language (terms, matching, quoting and parse errors alike), over the workers' own fields:

Field Worker
name its name: the type tag, else the package of its code URL
id its id
type its MIME type
status its status
topic any of its topics
source the source tag, else its code URL
tag any of its tags, as key=value

A word SHALL hold when the worker's name, id, any topic or any tag contains it. A Value field (kind, subject, value) SHALL be an unknown field.

Scenario: A tag

  • WHEN the expression is tag:world=act_binance_top10
  • THEN the workers with the tag world = act_binance_top10 SHALL pass

Scenario: Having a tag or not

  • WHEN the expression is tag:world=*, or -tag:world=*
  • THEN the workers with a world tag SHALL pass, or those without one

Scenario: Combined

  • WHEN the expression is binance status:running -tag:type=*
  • THEN the running workers that mention binance and have no type tag SHALL pass

Requirement: Name Tree

A client SHALL be able to show the names (keys or topics) of a list of Values flat or as a tree. In the tree a name SHALL be split at . into segments: each node is a prefix, its children are the next segments in sorted order, and a name SHALL be a node of its own even when it is also the prefix of other names. Every node SHALL show how many Values are at or under it (a topic with several events counts each).

The filter SHALL apply before the tree is built: only the names of Values that pass appear, with their counts. Picking a node SHALL restrict the view to the Values whose name is the node's prefix or starts with it followed by ., on top of the filter; picking the root SHALL lift the restriction.

Scenario: Keys as a tree

  • WHEN the names are cfg.world.alpha, cfg.world.beta, cfg.world.index and sys.api.trigger.t1
  • THEN the tree SHALL have the nodes cfg (3) with world (3) and its leaves alpha, beta, index (1 each), and sys (1) down to t1

Scenario: A node narrows the view

  • WHEN the node act.exchange.binance is picked
  • THEN act.exchange.binance and the names starting with act.exchange.binance. SHALL be kept, act.exchange.binanceus.x SHALL not

Requirement: Value Streams

A ValueStream SHALL be an ordered series of Values, oldest first, with a bound on its length (default 500, oldest dropped), that a view can follow as values are appended. The stream of a topic SHALL append each event received on it; the stream of a state key SHALL append each value read for that key that differs from the stream's newest (etag or data), so reading an unchanged key again adds nothing. A stream SHALL hold only what the client received, never history from before. A stream SHALL have the kinds (Kinds by Instance) of its newest Value, and only * while it is empty.

Scenario: A key's history

  • WHEN a state key is read three times while it is shown, with three different values, and a fourth time unchanged
  • THEN its stream SHALL hold the three Values, oldest first, and have the kinds of the newest

Requirement: Representation Mapping

The representations of each kind SHALL be data, in the same representations.json whose kinds rules Kinds by Instance defines:

{
  "version": 1,
  "kinds": [{"kind": "<name>", "match": {"<predicate>": "..."}}],
  "values": {"<kind>": ["<representation>", {"name": "<representation>", "<parameter>": "..."}]},
  "streams": {"<kind>": ["<stream representation>"]}
}
  • values SHALL map a kind to one or more value representations, streams to one or more stream representations. A Value's representations SHALL be the lists of its kinds in kind order, without duplicates; the first SHALL be shown by default and the others SHALL be reachable by one user action.
  • An entry SHALL be a representation name or an object with name and that representation's parameters.
  • A user values or streams entry SHALL replace the shipped list of that kind.
  • A client SHALL skip a representation it does not implement, and SHALL report an unknown kind or representation name once and ignore it; a representation that fails on a value SHALL be skipped for that value, never ending the view.

The shipped mapping SHALL contain at least:

Kind Match Values Streams
orderbook decoder orderbook.image; or object with bids, asks, last_update_id orderbook.ladder, json.tree latest, log
tradeflow object with units, buyunits, sellunits indicator.table, json.tree plot.line (field: units), latest, log
bookimbalance object with mid, imbalance indicator.table, json.tree plot.line (field: mid), latest, log
indicator.value object with value indicator.table, json.tree plot.line (field: value), latest, log
number json_type number number.text plot.line, log
text json_type string text log, latest
json.object json_type object json.tree, json.text log, latest
json.array json_type array json.tree, json.text log, latest
* – text log, latest

Scenario: The mapping is data

  • WHEN the user's mapping adds a kind or maps a kind to other representations
  • THEN the client SHALL use it at its next start without a code change

Scenario: A representation the client lacks

  • WHEN a kind maps to a representation one client implements and another does not
  • THEN the other client SHALL show the kind's next representation it implements

Scenario: An indicator value

  • WHEN a Value's data is an object with value (e.g. a moving average) or with mid and imbalance (a book imbalance)
  • THEN it SHALL be shown as the indicator table by default, and its stream as a line plot of value or mid over time

Requirement: Representations

A representation name SHALL mean the same content in every client; how it is drawn belongs to the client's spec (e.g. terminal-ui).

Name Shows
number.text the number at full precision, no exponent
text the data as text, wrapped
json.text the data as indented JSON
json.tree the data as a tree whose objects and arrays expand and collapse
orderbook.ladder the book's top levels per side, asks above the spread, each level's size as a number and as a horizontal bar scaled to the largest size shown on either side, with each side's cumulative size, and the book's mid, spread and update id
indicator.table the indicator value's fields as the Indicators view shows them
stream plot.line a 2D line plot: x the Values' time, y the number, or the number at the JSON path field
stream log one entry per Value, newest last, each in its first value representation, shortened to one line
stream latest the newest Value in its first value representation, redrawn as Values are appended

Scenario: A number and a stream of numbers

  • WHEN a Value's data is a JSON number
  • THEN it SHALL be shown as its text, and a stream whose newest Value is a number SHALL be shown as a line plot over time by default

Scenario: Level sizes as bars

  • WHEN an order book is shown as orderbook.ladder
  • THEN each level SHALL carry a bar whose length is its size relative to the largest size shown on either side, the largest level's bar SHALL be full length, and a level with a size above zero SHALL never have an empty bar

Requirement: Conformance

Every client implementing this spec SHALL pass the shared conformance vectors in its CI: for given Values (data, binary as base64, and origin) the expected kinds under the shipped rules and the expected value and stream representation lists, in order; for an expression and given Values the Values that pass or a parse error; for given names the expected tree and the names kept under a picked node. A change to these rules or to the shipped mapping SHALL change the vectors with it. The vectors, the default representations.json and its schema live in the TUI repository (conformance/value-presentation/, src/virtufin_tui/values/) until they move to a shared home (tui#41).

Scenario: Two clients, one answer

  • WHEN the TUI and the web app are given the same Values and mapping, expression or names
  • THEN they SHALL find the same kinds, the same representation lists, the same passing Values and the same tree