Client Testing
Purpose
Standardized client test requirements across the client repos (api, workmanager) and all 3 languages (Python, TypeScript, .NET). Every repo SHALL have the same set of client tests in every language, ensuring SDK parity and preventing regressions.
Requirements
Requirement: Cross-Language Client Test Parity
Every test case listed in the per-service tables below SHALL exist in every language marked ✓ for that row. The tables are the source of truth for which tests are expected where.
common/scripts/check-client-test-parity.sh parses those tables directly (via awk, keyed on the ## <service> Client Test Cases headings and the | <ID> | rows) and compares them against ID comments found in each repo's test tree. It parses rather than restates them, so the two cannot drift.
Scenario: A test is added in one language
- WHEN a hand-written client test is added to any of the client repos
- THEN a corresponding row SHALL be added to the per-service table in this spec
- AND the test SHALL be added to every other language the row marks
✓
Scenario: A parity gap exists
- WHEN the checker finds an ID expected by the tables but absent from a language's test tree
- THEN it SHALL exit non-zero and name the missing IDs
- AND the scheduled
test-parity-commonworkflow (weekly, plusworkflow_dispatch) SHALL report the failure — note this is a periodic check, not a per-merge gate, so a gap can land and be caught later
Scenario: The tables and the checker disagree
- WHEN the checker's expectations diverge from these tables
- THEN the tables win, and the checker SHALL be corrected to parse them rather than restate them
Requirement: Test-Case ID Comments
Every test case ID in the per-service tables SHALL appear as a comment in each language's implementation of that test. This is what makes automated matching possible: the checker scans for the IDs, not for test names.
Scenario: Marking a test with its ID
- WHEN a hand-written test implements a spec test case
- THEN the ID SHALL appear as a comment immediately above the test, in that language's comment syntax (
# M18,// M18) - AND it SHALL be present in every language implementing that case, not only the first
Requirement: Test Naming Conventions
Test names SHALL follow the per-language convention below, so that a human reading a gap report can locate the test the ID refers to.
| Language | Pattern | Example |
|---|---|---|
| Python | test_<snake_case_name> |
test_connect_delegates_to_stub |
| TypeScript | it("<human readable name>") |
it("connect delegates to gRPC client") |
| .NET | CamelCaseName |
ConnectDelegatesToStub |
Scenario: Naming a new test
- WHEN a test is added in any language
- THEN its name SHALL match that language's pattern above
Requirement: Separate ID Namespaces for Generated and Hand-Written Tests
Generated structural tests SHALL use a G-prefixed ID namespace distinct from the hand-written spec IDs, so the two cannot collide and the checker validates only what the tables actually track.
| Service | Hand-written prefix | Generated prefix |
|---|---|---|
| virtufin-api | A |
GA |
| virtufin-workmanager | M |
GM |
Scenario: Generated tests are excluded from parity expectations
- WHEN the checker scans a repo containing generated test files
- THEN their
GA*/GM*IDs SHALL NOT match the hand-written\b(A|M)[0-9]+\bpattern - AND only hand-written IDs SHALL be validated against the tables
Scenario: Regenerating structural tests
- WHEN a
.protofile changes and the generator is re-run - THEN generated test files SHALL be overwritten wholesale and SHALL NOT be hand-edited
- AND hand-written tests SHALL be unaffected, because they live in separate files
Workflow
How to run the generator (common/scripts/generate-client-tests.py, after a .proto change) and the checker (common/scripts/check-client-test-parity.sh), and the weekly test-parity-common workflow, are described in common/AGENTS.md §Client test parity. Generated tests live in tests/python/test_generated.py, tests/typescript/generated.test.ts and tests/<Service>.Client.Tests/GeneratedTests.cs and are never hand-edited; every other test file is hand-written and ID-commented.
The ## virtufin-<service> Client Test Cases headings below are parsed by the checker; keep them verbatim.
virtufin-api Client Test Cases
Structural coverage of the API client comes from the generated GA-prefix tests (test_generated.py / generated.test.ts / GeneratedTests.cs); A11 is the only hand-written case.
| # | Category | Test Case | Python | TypeScript | .NET |
|---|---|---|---|---|---|
| A11 | Integration | Connectivity check | ✓ | -- | -- |
virtufin-workmanager Client Test Cases
| # | Category | Test Case | Python | TypeScript | .NET |
|---|---|---|---|---|---|
| M1 | Constructor | Default host/port | ✓ | ✓ | ✓ |
| M2 | Constructor | Custom host/port | ✓ | -- | -- |
| M3 | Constructor | All expected methods present | ✓ | ✓ | ✓ |
| M4 | Package | Proto types importable | ✓ | ✓ | ✓ |
| M5 | Package | Proto enums accessible | -- | ✓ | ✓ |
| M6 | Package | gRPC client stub accessible | ✓ | ✓ | ✓ |
| M7 | Package | Transport dependency available | -- | ✓ | ✓ |
| M8 | Package | CreateWorkerRequest schema | -- | ✓ | ✓ |
| M9 | Package | ListWorkersRequest schema | -- | ✓ | ✓ |
| M10 | Package | DeleteWorkerRequest schema | -- | ✓ | ✓ |
| M11 | Package | StartWorkerRequest schema | -- | ✓ | ✓ |
| M12 | Package | StopWorkerRequest schema | -- | ✓ | ✓ |
| M13 | Package | RecoverWorkersRequest schema | -- | ✓ | ✓ |
| M14 | Package | LoadCodeRequest has an Id property | -- | ✓ | ✓ |
| M15 | Package | LoadCodeRequest has a CodeSource property | -- | ✓ | ✓ |
| M16 | Package | WorkerInfo schema | -- | ✓ | ✓ |
| M17 | Package | CodeSource schema | -- | ✓ | ✓ |
| M18 | Package | WorkerStatus enum | -- | ✓ | ✓ |
| M19 | Method | CreateWorker with URL | ✓ | ✓ | -- |
| M20 | Method | CreateWorker with content | ✓ | -- | -- |
| M21 | Method | CreateWorker without connect raises | ✓ | -- | -- |
| M22 | Method | ListWorkers delegates to stub | ✓ | ✓ | -- |
| M23 | Method | DeleteWorker delegates to stub | ✓ | ✓ | -- |
| M24 | Method | StartWorker delegates to stub | ✓ | ✓ | -- |
| M25 | Method | StopWorker delegates to stub | ✓ | ✓ | -- |
| M26 | Method | RecoverWorkers delegates to stub | ✓ | ✓ | -- |
| M27 | Method | GetWorkerHistory delegates to stub | ✓ | ✓ | -- |
| M28 | Obsolete | ~~LoadCodeFromContent delegates to stub~~ — LoadCode consolidates both variants behind one RPC (see M3); no separate LoadCodeFromContent stub method exists to delegate to. |
-- | -- | -- |
| M29 | Obsolete | ~~LoadCodeFromUrl delegates to stub~~ — same consolidation as M28. | -- | -- | -- |
| M30 | Error | CreateWorker error propagates | ✓ | ✓ | ✓ |
| M31 | Error | ListWorkers error propagates | -- | ✓ | -- |
| M32 | Error | DeleteWorker error propagates | -- | ✓ | -- |
| M33 | Lifecycle | GrpcChannel construction | -- | -- | ✓ |
| M34 | Lifecycle | Transport creates successfully | -- | ✓ | -- |
| M35 | Lifecycle | Package exports index entry | -- | ✓ | -- |
| M36 | Obsolete | ~~SetEnvironmentVariable delegates to stub~~ — the custom worker environment-variable RPC was removed entirely (process-wide state, unsafe across co-loaded workers); config now flows through the CloudEvent payload instead. | -- | -- | -- |
| M37 | Obsolete | ~~GetEnvironmentVariable delegates to stub~~ — same removal as M36. | -- | -- | -- |
| M38 | Obsolete | ~~SetEnvironmentVariables delegates to stub~~ — same removal as M36. | -- | -- | -- |
| M39 | Obsolete | ~~GetEnvironmentVariables delegates to stub~~ — same removal as M36. | -- | -- | -- |
Rules for Adding Tests
Moved to Requirements above, so the rules and the tables they govern cannot drift apart. See Cross-Language Client Test Parity, Test-Case ID Comments, Test Naming Conventions, and Separate ID Namespaces for Generated and Hand-Written Tests.