Skip to main content

Phase 4: Testing

This phase involves formalized, broad-scale testing to ensure the solution is robust, secure, and meets all specified requirements before being deployed to the stage environment. For Osborn, this process is heavily automated through the Bitbucket Pipelines CI/CD workflow.

Key Activities​

1. Environment and Data Setup​

  • Test Environment Provisioning: The stage environment is automatically deployed or updated when code is merged from develop. It is provisioned using Infrastructure as Code (IaC) to mirror the production environment as closely as possible.
  • Test Data Protection: The protection of testing data is crucial. The preferred practice is to use "dummy" (anonymized) data or data generated by factories (e.g., Factory Boy for Django).

2. Vulnerability and Compliance Testing​

  • Automated Security Testing: As part of the Pull Request pipeline in Bitbucket, solutions are automatically assessed for vulnerabilities using tools for:
    • SAST, SCA, and IAST.
    • Serverless Vulnerability Testing for any serverless components.
  • Vulnerability Management: Identified vulnerabilities are logged as issues in Jira and must be remediated according to the timeframes defined in the Omnicom Vulnerability and Patch Management Guidance before a PR can be merged.
  • Standards Validation: Testing must validate and document alignment with key standards:
    • Privacy-by-Design: ISO 27701
    • Digital Accessibility: Accessibility is tested in the frontend using automated tools and manually in Storybook. See ISO 30071-1.
    • Quality Management: ISO 9001.

3. Quality and User Acceptance Testing​

  • Testing Strategy: Osborn follows a testing pyramid strategy as defined in the Backend Best Practices, which prioritizes unit tests, followed by integration and end-to-end tests.
  • Quality Assurance (QA): QA testing is performed in the stage environment to ensure functionality and compatibility. Functional testing should exercise at least 70% of the application's data and logic routes.
  • User Acceptance Testing (UAT): Formal UAT is conducted by stakeholders in the stage environment to ensure the solution meets business needs.
  • Peer Code Reviews: The Pull Request process in Bitbucket requires at least two approvals, ensuring additional code review.

Control Point​

  • The Pull Request in Bitbucket serves as the formal Control Point for this phase.
  • CI/CD Pipeline Gate: The pipeline is configured to fail if there are any high or critical validated true-positive vulnerabilities, preventing the merge to develop and subsequent promotion to stage.
  • Clean Security Report: The successful completion of the Bitbucket pipeline, which includes a clean security scan (Attestation and SBOM reports), is required for approval.
  • This Control Point, inclusive of all testing, security reviews, and business approvals within the PR, must be successfully passed before the code is promoted to the stage environment via a merge to the develop branch.