Skip to content

Client Testing

Purpose

Standardized client test requirements across the client repos (api, workmanager) and all 3 languages (Python, TypeScript, .NET). Every repo SHALL have the same set of client tests in every language, ensuring SDK parity and preventing regressions.

Requirements

Requirement: Cross-Language Client Test Parity

Every test case listed in the per-service tables below SHALL exist in every language marked ✓ for that row. The tables are the source of truth for which tests are expected where.

common/scripts/check-client-test-parity.sh parses those tables directly (via awk, keyed on the ## <service> Client Test Cases headings and the | <ID> | rows) and compares them against ID comments found in each repo's test tree. It parses rather than restates them, so the two cannot drift.

Scenario: A test is added in one language

  • WHEN a hand-written client test is added to any of the client repos
  • THEN a corresponding row SHALL be added to the per-service table in this spec
  • AND the test SHALL be added to every other language the row marks ✓

Scenario: A parity gap exists

  • WHEN the checker finds an ID expected by the tables but absent from a language's test tree
  • THEN it SHALL exit non-zero and name the missing IDs
  • AND the scheduled test-parity-common workflow (weekly, plus workflow_dispatch) SHALL report the failure — note this is a periodic check, not a per-merge gate, so a gap can land and be caught later

Scenario: The tables and the checker disagree

  • WHEN the checker's expectations diverge from these tables
  • THEN the tables win, and the checker SHALL be corrected to parse them rather than restate them

Requirement: Test-Case ID Comments

Every test case ID in the per-service tables SHALL appear as a comment in each language's implementation of that test. This is what makes automated matching possible: the checker scans for the IDs, not for test names.

Scenario: Marking a test with its ID

  • WHEN a hand-written test implements a spec test case
  • THEN the ID SHALL appear as a comment immediately above the test, in that language's comment syntax (# M18, // M18)
  • AND it SHALL be present in every language implementing that case, not only the first

Requirement: Test Naming Conventions

Test names SHALL follow the per-language convention below, so that a human reading a gap report can locate the test the ID refers to.

Language Pattern Example
Python test_<snake_case_name> test_connect_delegates_to_stub
TypeScript it("<human readable name>") it("connect delegates to gRPC client")
.NET CamelCaseName ConnectDelegatesToStub

Scenario: Naming a new test

  • WHEN a test is added in any language
  • THEN its name SHALL match that language's pattern above

Requirement: Separate ID Namespaces for Generated and Hand-Written Tests

Generated structural tests SHALL use a G-prefixed ID namespace distinct from the hand-written spec IDs, so the two cannot collide and the checker validates only what the tables actually track.

Service Hand-written prefix Generated prefix
virtufin-api A GA
virtufin-workmanager M GM

Scenario: Generated tests are excluded from parity expectations

  • WHEN the checker scans a repo containing generated test files
  • THEN their GA* / GM* IDs SHALL NOT match the hand-written \b(A|M)[0-9]+\b pattern
  • AND only hand-written IDs SHALL be validated against the tables

Scenario: Regenerating structural tests

  • WHEN a .proto file changes and the generator is re-run
  • THEN generated test files SHALL be overwritten wholesale and SHALL NOT be hand-edited
  • AND hand-written tests SHALL be unaffected, because they live in separate files

Workflow

How to run the generator (common/scripts/generate-client-tests.py, after a .proto change) and the checker (common/scripts/check-client-test-parity.sh), and the weekly test-parity-common workflow, are described in common/AGENTS.md §Client test parity. Generated tests live in tests/python/test_generated.py, tests/typescript/generated.test.ts and tests/<Service>.Client.Tests/GeneratedTests.cs and are never hand-edited; every other test file is hand-written and ID-commented.

The ## virtufin-<service> Client Test Cases headings below are parsed by the checker; keep them verbatim.


virtufin-api Client Test Cases

Structural coverage of the API client comes from the generated GA-prefix tests (test_generated.py / generated.test.ts / GeneratedTests.cs); A11 is the only hand-written case.

# Category Test Case Python TypeScript .NET
A11 Integration Connectivity check ✓ -- --

virtufin-workmanager Client Test Cases

# Category Test Case Python TypeScript .NET
M1 Constructor Default host/port ✓ ✓ ✓
M2 Constructor Custom host/port ✓ -- --
M3 Constructor All expected methods present ✓ ✓ ✓
M4 Package Proto types importable ✓ ✓ ✓
M5 Package Proto enums accessible -- ✓ ✓
M6 Package gRPC client stub accessible ✓ ✓ ✓
M7 Package Transport dependency available -- ✓ ✓
M8 Package CreateWorkerRequest schema -- ✓ ✓
M9 Package ListWorkersRequest schema -- ✓ ✓
M10 Package DeleteWorkerRequest schema -- ✓ ✓
M11 Package StartWorkerRequest schema -- ✓ ✓
M12 Package StopWorkerRequest schema -- ✓ ✓
M13 Package RecoverWorkersRequest schema -- ✓ ✓
M14 Package LoadCodeRequest has an Id property -- ✓ ✓
M15 Package LoadCodeRequest has a CodeSource property -- ✓ ✓
M16 Package WorkerInfo schema -- ✓ ✓
M17 Package CodeSource schema -- ✓ ✓
M18 Package WorkerStatus enum -- ✓ ✓
M19 Method CreateWorker with URL ✓ ✓ --
M20 Method CreateWorker with content ✓ -- --
M21 Method CreateWorker without connect raises ✓ -- --
M22 Method ListWorkers delegates to stub ✓ ✓ --
M23 Method DeleteWorker delegates to stub ✓ ✓ --
M24 Method StartWorker delegates to stub ✓ ✓ --
M25 Method StopWorker delegates to stub ✓ ✓ --
M26 Method RecoverWorkers delegates to stub ✓ ✓ --
M27 Method GetWorkerHistory delegates to stub ✓ ✓ --
M28 Obsolete ~~LoadCodeFromContent delegates to stub~~ — LoadCode consolidates both variants behind one RPC (see M3); no separate LoadCodeFromContent stub method exists to delegate to. -- -- --
M29 Obsolete ~~LoadCodeFromUrl delegates to stub~~ — same consolidation as M28. -- -- --
M30 Error CreateWorker error propagates ✓ ✓ ✓
M31 Error ListWorkers error propagates -- ✓ --
M32 Error DeleteWorker error propagates -- ✓ --
M33 Lifecycle GrpcChannel construction -- -- ✓
M34 Lifecycle Transport creates successfully -- ✓ --
M35 Lifecycle Package exports index entry -- ✓ --
M36 Obsolete ~~SetEnvironmentVariable delegates to stub~~ — the custom worker environment-variable RPC was removed entirely (process-wide state, unsafe across co-loaded workers); config now flows through the CloudEvent payload instead. -- -- --
M37 Obsolete ~~GetEnvironmentVariable delegates to stub~~ — same removal as M36. -- -- --
M38 Obsolete ~~SetEnvironmentVariables delegates to stub~~ — same removal as M36. -- -- --
M39 Obsolete ~~GetEnvironmentVariables delegates to stub~~ — same removal as M36. -- -- --

Rules for Adding Tests

Moved to Requirements above, so the rules and the tables they govern cannot drift apart. See Cross-Language Client Test Parity, Test-Case ID Comments, Test Naming Conventions, and Separate ID Namespaces for Generated and Hand-Written Tests.