Test Plan
The Test Plan is split into 5 phases outlined as follows:
| Phase | Description |
|---|---|
| 1 | Baseline check of the Discovery document, JWKS, and the ability to perform Client Registrations with the Holder. The dynamic registrations are a critical component prior to moving forward as the FAPI Conformance suite requires two clients. |
| 2 | Deep dive check into the overall Information Security posture of the solution. Biza relies heavily upon the FAPI Conformance suite as this fundamentally drives many of the critical security decisions behind the CDR. After the FAPI Conformance Suite is passed additional information security tests are conducted with respect to the customisations CDR has applied. |
| 3 | Focuses primarily on the Information Security modifications that the CDR Standard has made to the underlying FAPI profile. |
| 4 | InfoSec CX focus with regards to expected behaviours, testing of Register behaviours when offline, Status/Outages endpoint checks |
| 5 | CDR Data API testing across all required endpoints. This includes data quality checks using the personas defined, query string behaviours, header verification and http error code and payload validation |
Phase Breakdown
Note
For Phase 5 Tests, please see the Verification Test Suite (VTS) Documentation
Phase 1
| Title | Name | Description | Execution method |
|---|---|---|---|
| CDR-DISCOVERY-END-POINT-VERIFICATION | Discovery End Point Verification | This test ensures that the server's configurations (including scopes, response_types, grant_types etc) is containing the required value in the specification | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-JWKS | Verify JWKS Document | In this particular test we verify that the JWKS contains properties which are expected for CDR | AUTOMATED (FAPI BIZA) |
| INFOSEC-CLIENT-REGISTRATION-HAPPY | Verifies that a Holder accepts dynamic registrations, allows deletes and updates and rejects invalid registrations | In this particular test the Data Recipient attempts a Dynamic Client Registration to the Data Holder using a valid SSA and content | SEMI AUTOMATED (PRAK) |
Phase 2
| Title | Name | Description | Execution method |
|---|---|---|---|
| FAPI-RW-ID2-ATTEMPT-AUTHORISATION-CODE-REUSE | Verifies the authorisation code expires within a period of time | This test case combines two FAPI tests regarding the reuse of a previous authorisation code. The first test verifies that an authorisation code is not usable after 30 seconds while the second does the same test with a 1 second delay. The 30 second version will result in a failure while the 1 second version will produce a warning. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-CLIENT-ASSERTION-INVALID-AUD-FAIL | Verifies that client assertions which are not intended for the Holder are rejected | This test passes client assertion where 'aud' is wrong (is for a different server). The server must be rejected at the token endpoint and returning an error message that the 'aud' claim of the client assertion is invalid. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-CLIENT-ASSERTION-WITH-NO-SUB-FAILS | Verifies that client assertions include a sub claim | This test passes a client assertion where 'sub' is missing to the token endpoint. The server must reject this, returning an 'invalid_client' error. It is vital that the server requires the 'sub' claim, as this is the primary reason a signed request object cannot be used as a client authentication assertion. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-CLIENT-ID-IN-TOKEN-ENDPOINT | Ensures that the Holder verifies the Certificate used matches the Client ID | This test establishes consent for Client 1 and then attempts to use Client 2’s certificate to renew tokens. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-DISCOVERY-END-POINT-VERIFICATION | Discovery End Point Verification | This test ensures that the server's configurations (including scopes, response_types, grant_types etc) is containing the required value in the specification | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-AUTH-CODE-IS-CLIENT-BOUND | Verifies that an authorisation code is binded to the client it was intended for | This test involves a first client establishing authorisation codes for token claiming but then Client 1’s valid code being used via Client 2’s client assertion and certificatecredentials. The Holder should reject requests which include a valid authorisation code but were issued to separate clients. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-AUTHORISATION-MATCHES-CLIENT-PAR | Ensures the holder verifies that request objects are signed by the active client | This test verifies that request objects submitted to the PAR endpoint are suitably verified based on their cryptographic signature. If an invalid signature is detected the server should reject the request during PAR setup | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-AUTHORISATION-REQUEST-MISSING-REQUEST-FAILS | Ensures the holder rejects requests where a request object or request uri is not supplied | This test calls the authorization endpoint without using a request object (i.e. with all the parameters passed in the url query), and should end with the authorization server showing an error message that the request is invalid (a screenshot of which should be uploaded) or with the user being redirected back to the conformance suite with an invalid_request error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-AUTHORIZATION-REQUEST-WITHOUT-STATE-SUCCESS | Verify Requests missing state are accepted | This test makes an authentication request that does not include 'state'. State is an optional parameter, so the authorization server must successfully authenticate and must not return state nor s_hash. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-CLEARTEXT-SIGNING-FAILS-PAR | Ensure the holder does not accept request objects which are not signed | This test verifies that request objects submitted to the PAR endpoint are not unsigned (ie. plain text json serialisations) and, if they are, the server rejects the request during PAR setup. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-DIFFERENT-NONCE | Verifies behaviour related to supplying a nonce on an endpoint different to a request | This test passes a different nonce in the authorization_endpoint parameters to the one inside the signed request object. The authorization server must either return an invalid_request error back to the client, and must show an error page (saying the request object is invalid as the 'nonce' value in the request object and outside it are different - upload a screenshot of the error page), or must successfully authenticate and return the nonce from inside the request object in the id_token. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-EXPIRED-CLIENT-ASSERTION-FAIL | Verifies that client assertions which are expired fail | This test passes client assertion where 'exp' is 5 minutes ago. The server must be rejected at the token endpoint and returning an error message that the expiration time ('exp') of the client assertion is invalid. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-EXPIRED-REQUEST-OBJECT-FAILS-PAR | Ensure the holder enforces request objects which are not expired | This test should end with the authorization server rejecting a PAR establishment when supplied a request object which is expired | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-INVALID-SIGNING-FAILS-PAR | Ensure the holder does not accept request objects which are not signed | This test verifies that request objects submitted to the PAR endpoint are suitably verified based on their cryptographic signature. If an invalid signature is detected the server should reject the request during PAR setup | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-MISSING-CLIENT-ASSERTION-FAILS | Verified that a client assertion is always supplied on the token endpoint | This test performs a normal authentication but when it calls the token endpoint to exchange the authorization code for tokens, it does not include a client authentication assertion. The token endpoint must return the error 'invalid_client' or 'invalid_request' as client authentication is mandatory at the token endpoint when using private_key_jwt client authentication. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-MTLS-HOK-REQUIRED | Verifies TLS version/cipher limitations and valid token endpoint error responses | This test does an iterative check on the TLS version/ciphers in use before attempting to POST /token and expecting that an OAuth2 error is returned | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-OTHER-SCOPE-ORDER-SUCCEEDS | Ensure a different scope ordering is still acceptable | This test makes a FAPI authorization request but with the order of the entries in the 'scope' reversed, which must succeed. As per RFC6749 section 3.3, 'If the value contains multiple space-delimited strings, their order does not matter'. The reason for this test is that some OAuth clients process scopes in a way that the order they are sent to the server is not under the control of a developer using that client, and as per the spec such requests must still be accepted. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REDIRECT-URI-INCLUDED-PAR | Ensure the holder enforces the requirement for the redirect uri to specified within the request object | This test omits the redirect uri from being specified and should end with the authorisation server rejecting the PAR request setup | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REGISTERED-REDIRECT-URI | Ensures the Holder is enforcing only registered Redirect URIs | This test uses an unregistered redirect uri. The authorization server should display an error saying the redirect uri is invalid, a screenshot of which should be uploaded. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITH-BAD-AUD-FAILS-PAR | Ensure the holders does not allow requests it is not the audience for | This test verifies that a Holder does not accept request objects which, while cryptographically acceptable, include an aud audience value that is not their issuer. This case handles the situation where only invalid aud entries are provided versus FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITH-MULTIPLE-AUD-SUCCEEDS which should succeed if one of a multiple number of aud are matching. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITH-LONG-NONCE | Ensures the Holder accepts long nonces or produces an error if not supported | This test passes a 384 character nonce in the request object. The authorization server must either successfully authenticate and return the nonce correctly, or if it does not support nonces that long, it may return an invalid_request error back to the client or show an error page (saying the server rejects long nonce - upload a screenshot of the error page). The server MUST NOT return the nonce corrupted or truncated. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITH-LONG-STATE | Ensures the Holder accepts long states or produces an error if not supported | This test passes a long state in request object. The authorization server must either return an invalid_request error back to the client, and must show an error page (saying the server rejects long state - upload a screenshot of the error page) or must successfully authenticate and return the state correctly. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITH-MULTIPLE-AUD-SUCCEEDS | Request Objects with multiple aud | This test passes an aud value as an array containing good and bad values then the server must accept it. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITHOUT-EXP-FAILS-PAR | Ensure the holder enforces expiration claim in request object | This test is designed to ensure that request objects with missing expiration are rejected by the authorization server. It should end with the authorization server rejecting the PAR setup request with an invalid_request_object error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITHOUT-NONCE-FAILS-PAR | Ensure the holder enforces nonce to exist within the request object | This test should end with the authorization server showing an error message that the request or request object is invalid (a screenshot of which should be uploaded) or with the user being redirected back to the conformance suite with a correct error response. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITHOUT-REDIRECT-URI-FAILS-PAR | Ensure the holder enforces redirect_uri to exist within the request object | This test should end with the authorization server rejecting the PAR request because the redirect_uri has not been specified. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-REQUEST-OBJECT-WITHOUT-SCOPE-FAILS-PAR | Ensure the holder enforces scopes to exist within the request object | This test passes scope only outside the request object. As FAPI-RW requires that all parameters be inside the signed request object, this test should end with the authorization server producing an invalid_request_uri response at the PAR endpoint. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-RESPONSE-MODE-QUERY-PAR | Ensure that a response_mode of query is disallowed | This test includes response_mode=query in the authorization request. The authorization server should reject the PAR request establishment with an OAuth2 error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-RS256-SIGNING-FAILS-PAR | Ensure the holder does not accept RS256 signed requests | This test authenticates normally except that the request object is signed using RS256. RS256 is not permitted by the FAPI-RW specification. The test must end with the authorization server showing an error message that the request object is invalid (a screenshot of which should be uploaded) or with the user being redirected back to the conformance suite with a correct error response. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-SIGNED-CLIENT-ASSERTION-WITH-RS256-FAILS | Verifies that the use of RS256 for client authentication is disallowed | This test authenticates normally except that the client assertion passed to the token endpoint when exchanging the authorization code for tokens is signed using RS256. RS256 is not permitted by the FAPI-RW specification. The test must end with the token endpoint returning an invalid_client error due to invalid client authentication. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-ENSURE-VALID-PKCE-SUCCEEDS | Verify that valid PKCE requests are ignored | This test makes a FAPI authorization request using valid PKCE (RFC7636), which must succeed. FAPI-RW-ID2 does not require servers to support PKCE, but as per [https://tools.ietf.org/html/rfc6749#section-3.1 ]'The authorization server MUST ignore unrecognized request parameters' - i.e. whether the server supports PKCE or not, a valid PKCE request must succeed. The reason for this test is that many OpenID Connect clients speculatively use PKCE, and the OAuth2 standard requires that requests from such clients must not fail. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR | Happy Flow Tests (PAR Enabled) | Tests primarily 'happy' flows, using two different OAuth2 clients (and hence authenticating the user twice), and uses different variations on request objects, registered redirect URI (both redirect URIs must be pre-registered as shown in the instructions). It also tests that TLS Certificate-Bound access tokens (required by the FAPI-RW spec) are correctly implemented. This version uses Pushed Authorization Requests to create authorization objects. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-ATTEMPT-REUSE-REQUEST_URI | Verifies that Request URI cannot be reused | This test tries to use a request_uri twice and expects the authorization server to either show an error or redirect back with an invalid_request_uri error. PAR section 7.3 states 'the AS SHOULD make the request URIs one-time use'. If the authentication succeeds, a warning will be issued. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-ATTEMPT-TO-USE-EXPIRED-REQUEST | PAR Request expiration verification | This test tries to use a request_uri that has expired and expects the authorization server to return an error. The test will call the PAR endpoint to create a new request_uri, then sleep until the expiry time for that request_uri before attempting to use it (this means this test may take some minutes to run, depending on the lifetime of the request_uri). | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-REJECT-AUTHORIZE-AUDIENCE | Verifies that the Holder rejects PAR with setup set to /authorize | This test tries to use the pushed authorization endpoint URL as the audience in the request object. The authorization server is expected to reject the request - depending on when the server chooses to verify the request object, the refusal may be an error from the pushed request object endpoint or may be returned from the authorization endpoint. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-REJECT-INVALID-METHOD | Verifies that a Holder rejects PAR requests which use an invalid method | This test tries to use an invalid HTTP method and expects the authorization server to return an error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-REJECT-INVALID-REDIRECT_URI | Verifies that the Holder rejects PAR that specify invalid Redirect URIs | This test tries to provide an invalid redirect_uri and expects the authorization server to return an error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-REJECT-REQUEST_URI-IN-FORM | Verifies that a Holder rejects request_uri form parameters | This test tries to use a request_uri (meant for client1) with client2 and expects the authorization server to return an error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-REJECT-REQUEST_URI-IN-OBJECT | Verifies that a Holder rejects request_uri within the request object | This test sends a random request_uri parameter in the authorization request object and expects the authorization server to return an error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-TEST-PAR-AUDIENCE-ASSERTION | Verifies that the Holder accepts a client assertion with a PAR audience | This test tries to use the pushed authorization endpoint URL as the audience for Client JWT Assertion. The authorization server is expected to accept it. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-PAR-TEST-REQUEST_URI-MIXUP | Verifies that a Holder binds PAR-generated Request URIs to Clients | This test tries to use a request_uri (meant for client1) with client2 and expects the authorization server to return an error. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-REFRESH-TOKEN | Refresh Token Tests and Binding | This test obtains refresh tokens and performs various checks, including checking that the refresh token is correctly bound to the client. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-RESPONSE-TYPE-CODE-FAILS-PAR | Ensure the holder enforces the code id_token flow by attempting to use a code only response type | This test uses response_type=code in the authorization request, which (as a JARM response has not been requested) is not permitted in FAPI-RW - only the hybrid flow (response_type=code id_token) is allowed. The authorization server should show an error message that the response type is unsupported or the request is invalid (a screenshot of which should be uploaded) or the user should be redirected back to the conformance suite with a correct error response, or an error could be returned from the PAR endpoint. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-STATE-ONLY-OUTSIDE-REQUEST-OBJECT-NOT-USED-PAR | Ensure the Holder ignores a state request parameter if it is not within the request object | This test uses a request object that does not contain state, but state is passed in the URL parameters to the authorization endpoint (hence state should be ignored, as FAPI-RW says only parameters inside the request object should be used). The expected result is a successful authentication that returns neither state nor s_hash. | AUTOMATED (FAPI BIZA) |
| FAPI-RW-ID2-USER-REJECTS-AUTHENTICATION | User Consent Flow Failure | This test requires the user to reject the authentication, for example by pressing the 'cancel' button on the login screen. It verifies the error is correctly notified back to the relying party. | AUTOMATED (FAPI BIZA) |
Phase 3
| Title | Name | Description | Execution method |
|---|---|---|---|
| INFOSEC-AUTHORIZATION-ENDPOINT-IDT-NO-PII | Verify that the ID Token returned during Authorisation redirect does not contain personal claims | In this particular test the Recipient initiates a consent flow that requests ongoing access. The Data Holder provides an ID Token during the initial authorisation redirect (post /authorize) which must not contain personally identifiable information | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-ID-REQUIRED | CDR Arrangement Identifier must be issued | In this particular test the Holder is required to issue CDR Arrangement ID’s on retrieval of the AT/RT components in /token and during /introspect token introspection calls | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-ID-UPDATE-UNCHANGED | Verify that Arrangement Identifiers do not change if provided by Recipient | In this particular test the recipient initiates a consent then attempts to update the consent using an existing arrangement id. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-NOT-RECYCLED | Softly verify that arrangement identifiers are not recycled | In this particular test the recipient consents a number of times without providing existing arrangement identifiers. The expectation is that each of these consent initiations will result in unique arrangement identifiers. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-ONCE-OFF-EXPIRED-NO-EXTENSION | Verify that Arrangements established as Once Off or already Expired cannot be extended | In this particular test an arrangement is established first as a once off and then as a short lived duration. For each scenario an attempt to reuse the arrangement is attempted (using the cdr_arrangement_id claim) and should be rejected. Refer to: https://cdr-support.zendesk.com/hc/en-us/articles/900004373763-Extending-an-arrangement-when-it-is-expired-or-revoked. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-SETUP-24HOUR-ONCE-OFF | CDR Arrangement Establishment (Once Off Arrangement for 24 hours) | In this particular test the Data Recipient attempts an arrangement establishment specifying either a zero value or missing sharing_duration to setup a once off consent. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-SETUP-HAPPY | CDR Arrangement Establishment (Happy Flow) | In this particular test the Data Recipient attempts an arrangement establishment for one month specifying a sharing_duration and expecting a sharing_expires_at . | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-SETUP-INVALID-DURATION | CDR Arrangement Establishment (Invalid Durations specified) | In this particular test the Data Recipient attempts an arrangement establishment for invalid durations and durations in excess of one year. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-SETUP-ONCE-OFF | CDR Arrangement Establishment (Once Off Arrangement) | In this particular test the Data Recipient attempts an arrangement establishment specifying either a zero value or missing sharing_duration to setup a once off consent. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-ARRANGEMENT-UPDATE-KILLS-OLD-TOKENS | Updates of CDR Arrangements must trigger Token revocations for previous arrangement definition | In this particular test the Holder is presented with an existing CDR Arrangement Identifier which the Recipient is currently using (coupled with active tokens). After completing the consent flow the Holder must revoke existing tokens associated with the previous state of the arrangement and replace them with new tokens obtained during the subsequent POST /token call. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-DISCOVERY-CAPABILITIES | Verify CDR Discovery Document | In this particular test we verify that the discovery document does not “over advertise” capability that the CDR alters further. | AUTOMATED (FAPI BIZA) |
| INFOSEC-CDR-REGISTER-INVALID-SSA | Ensure that SSA’s are only accepted if signed by the Register JWKS Endpoint | In this particular test we attempt a Dynamic Registration using a Software Statement which is not signed by the CDR Register. | SEMI AUTOMATED (PRAK) |
| INFOSEC-CLIENT-REGISTRATION-SAD | This executes a number of unhappy client registration requests | In this particular test the Data Recipient attempts to perform a registration using an expired SSA, a fake software product, invalid metadata, and invalid SSA and expects all such attempts to fail. | SEMI AUTOMATED (PRAK) |
| INFOSEC-CODE-ID_TOKEN-SUPPORT | Verify that the OIDC Hybrid Flow is in use and that the use of code id_token is supported | This scenario requires that a request to the authorization endpoint can be made with a response_type containing code id_token. | AUTOMATED (FAPI BASELINE) |
| INFOSEC-DIGITAL-CHANNEL-ID | Verify that the Holder uses the Holder’s digital channel user identifier | The scenario requires that the Holder presents on successful initiation from the Data Recipient a request for an identifier that matches the Holder's digital channels. | MANUAL (CX) |
| INFOSEC-DIGITAL-CHANNEL-OTP | Verify that the Holder utilises an OTP for authentication and not a password | This scenario simultaneously verifies that the Holder is using OTPs for authentication and that the Holder is not relying upon password-based authentication through the flows. | MANUAL (CX) |
| INFOSEC-INTROSPECTION-HAPPY | Verify that the introspection endpoint is available, contains minimal attributes, that only Refresh Tokens can be introspected and that Access and ID Token introspection requests are rejected. | In this particular test a series of introspection requests are made to verify that the introspection endpoint only accepts refresh tokens and that the attributes exposed via this endpoint are restricted. | AUTOMATED (FAPI BIZA) |
| INFOSEC-RECIPIENT-ENDPOINTS-PUBLIC-OR-ACCC-CA | Verify that actions issued by Holders to Recipients can occur using either ACCC CA signed or Public CA signed certificates | In this particular test case it is expected that a Holder will accept an ACCC CA or Public trusted CA for the revocation, JWKS, and arrangement revocation endpoints hosted by the Recipients. | SEMI AUTOMATED (PRAK) |
| INFOSEC-REVOCATION-ARRANGEMENT-ENDPOINT-EXISTS | Verify that the Holder is publishing a CDR Arrangement Revocation Endpoint | In this particular test the user has an established sharing arrangement and chooses to cancel it from the Recipient. In this scenario the Recipient submits a revocation request to the Holder’s CDR Arrangement Revocation endpoint specifying an arrangement identifier and secured using a private_key_jwt client assertion. | AUTOMATED (FAPI BIZA) |
| INFOSEC-REVOCATION-AT-RT-SUPPORT | Verify that the Holder supports both Access and Refresh Token revocation. | In this particular test case the Holder must support both access and refresh token revocation such that a Data Recipient who chooses to notify the revocation endpoint with either type of token loses access to the Holder with that token. | AUTOMATED (FAPI BIZA) |
| INFOSEC-REVOCATION-VIA-ARRANGEMENT-REVOKE-ENDPOINT | Verify that on consent cancellation from the Holder Dashboard a Revocation request to the Data Recipient's Arrangement Revocation Endpoint. | In this particular test the user has an established sharing arrangement and chooses to cancel it. In this scenario the Holder must contact the Recipient’s defined CDR Arrangement Revocation Endpoint specifying an arrangement identifier and secured using a private_key_jwt client assertion. | AUTOMATED (FAPI BIZA) |
| INFOSEC-TOKEN-RESPONSE-EXPIRES-IN | Verify that the Token Response contains the expires_in attribute | In this particular test the Recipient verifies that the Token Response has an expires_in attribute which outlines the lifetime of the access token. | AUTOMATED (FAPI BIZA) |
| INFOSEC-USER-INFO-VALID | Verify that the userinfo endpoint is valid and contains information about the authenticated user | In this particular test the Recipient establishes a consent agreement then performs a userinfo request to verify that the required claims are specified. | AUTOMATED (FAPI BIZA) |
Phase 4
| Title | Name | Description | Execution method |
|---|---|---|---|
| API-COMMON-GET-OUTAGES-V1 | Verify Get Outages V1 | In this scenario the Get Outages API is called and verified for compliance | AUTOMATED (FAPI BIZA) |
| API-COMMON-GET-STATUS-V1-OK | Verify Get Status V1 (OK) | In this scenario the Get Status API is called and verified for OK state | AUTOMATED (FAPI BIZA) |
| API-COMMON-GET-STATUS-V1-PARTIAL | Verify Get Status V1 (PARTIAL) | In this scenario the Get Status API is called and verified for PARTIAL state | AUTOMATED (FAPI BIZA) |
| API-COMMON-GET-STATUS-V1-SCHEDULED | Verify Get Status V1 (SCHEDULED_OUTAGE) | In this scenario the Get Status API is called and verified for SCHEDULED_OUTAGE state | AUTOMATED (FAPI BIZA) |
| API-COMMON-GET-STATUS-V1-UNAVAILABLE | Verify Get Status V1(Unavaiable) | In this scenario the Get Status API is called and verified for UNAVAILABLE state | AUTOMATED (FAPI BIZA) |
| API-ADMIN-METRICS | Verify the Get Metrics API (Admin) | In this scenario we verify that the Get Metrics functionality exists at a Data Holder. | SEMI AUTOMATED (PRAK) |
| INFOSEC-CX-CONSENT-FLOW | Verifies UX Compliance during Consent Flow | In this particular test careful attention is paid to the consent workflow and the graphical attributes required to meet the CX guidelines released by the DSB | MANUAL (CX) |
| INFOSEC-CX-DH-DASHBOARD | Verifies UX Compliance within Data Holder Dashboard | In this particular test careful attention is paid to the operations of the Data Holder’s Dashboard to verify the graphical attributes meet the CX guidelines released by the DSB. This is a subjective test case and reference should be made to the CX Guideline | MANUAL (CX) |
| INFOSEC-CX-OTP-EXPIRY | Verify that the Holder invalidates the OTP it has sent within a period of time | This scenario verifies that if the OTP is not input “within a reasonable period of time” which we define as 10 minutes | MANUAL (CX) |
| INFOSEC-CX-UNAVAILABLE-ACCOUNTS | Verify the Holder presents reasons for accounts not being available | This scenario involves a CX based verification of unavailable accounts being given a reason for their unavailability | MANUAL (CX) |
| INFOSEC-DIGITAL-CHANNEL-ID | Verify that the Holder uses the Holder’s digital channel user identifier | The scenario requires that the Holder presents on successful initiation from the Data Recipient a request for an identifier that matches the Holder's digital channels | MANUAL (CX) |
| INFOSEC-DIGITAL-CHANNEL-OTP | Verify that the Holder utilises an OTP for authentication and not a password | This scenario simultaneously verifies that the Holder is using OTPs for authentication and that the Holder is not relying upon password-based authentication through the flows. | MANUAL (CX) |
| REGISTER-CA-ENFORCEMENT | Verify that a Data Holder is not allowing comms to secure endpoints with certificates which are not signed by the Register or when Register CA is offline | This test case validates that Data Holders are validating the Register supplied software statement. | SEMI AUTOMATED (PRAK) |
| REGISTER-OFFLINE | Verify that a Data Holder continues to function as per normal when Register is broken | This test case validates that Data Holders continue to operate when the Register is offline. | SEMI AUTOMATED (PRAK) |
| REGISTER-SSA-VALIDATION | Verify that SSA’s are validated | This test case validates that Data Holders are validating the Register supplied software statement. | SEMI AUTOMATED (PRAK) |
| REGISTER-VERSIONING | Verifies that the Holder implementation is using endpoint versioning | This test case validates that a Data Holder is accessing the Register using the x-v header based version negotiation scheme. | SEMI AUTOMATED (PRAK) |
Phase 5 - Banking
Note
For Banking sector Phase 5 testing, please refer to the VTS documentation
Phase 5 - Energy
Note
For Energy sector Phase 5 testing, please refer to the VTS documentation