eduvia/workavia-mail-front
11
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

docs: add ADRs for Patrol web integration test — setup, architecture, migration (#4452)

Browse Source
This commit is contained in:
Dat Vu
2026-04-17 17:38:35 +07:00
committed by GitHub
parent 51e3c218d0
commit bfe51749aa
4 changed files with 664 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/adr/0080-patrol-web-integration-test-setup.md
+123
View File
@@ -0,0 +1,123 @@
# 80. Patrol Web Integration Test — Setup & Execution
Date: 2026-04-15
## Status
Accepted
## Related ADRs
- [ADR-0053](./0053-patrol-integration-test.md) — Patrol for mobile integration testing (foundation)
- [ADR-0081](./0081-patrol-web-test-architecture.md) — Cross-platform test architecture
- [ADR-0082](./0082-patrol-web-test-migration-guide.md) — Migration guide & implementation example
- [ADR-0083](./0083-patrol-web-test-migration-plan.md) — PR-by-PR migration plan
## Context
[ADR-0053](./0053-patrol-integration-test.md) established Patrol for mobile (Android/iOS) integration testing. As Twake Mail is also deployed as a web app, the same level of E2E coverage is needed on the browser. Patrol 4.x added Chrome support via `--device=chrome`, making it feasible to extend the existing setup without introducing a second testing framework.
## Decision
Use Patrol with Chrome as the target device for web integration testing, sharing the same backend Docker infrastructure used for mobile tests.
### Prerequisites
- Flutter 3.38.9
- Patrol CLI 4.3.1: `dart pub global activate patrol_cli 4.3.1`
- `patrol` package **4.5.0** (and `patrol_finders 3.x`, `patrol_log 0.8.x`) — requires upgrading from the 3.x currently in `pubspec.yaml`
- Docker Desktop (running)
- `openssl` installed (used to generate JWT keys for the backend)
> Unlike mobile tests, web tests do **not** require `ngrok` or `jq`. The app runs at `localhost` and Chrome connects to it directly — no traffic forwarding needed.
### Environment Variables
All tests receive configuration via `--dart-define`. Store values in an env file and source it in the run scripts — this keeps credentials out of command-line history and git history (add the file to `.gitignore`):
```bash
# integration_test/test_config.env
BASIC_AUTH_URL=http://localhost/
USERNAME=test_user
PASSWORD=test_password
BASIC_AUTH_EMAIL=test@example.com
ADDITIONAL_MAIL_RECIPIENT=recipient@example.com
```
The run scripts source this file and forward each value:
```bash
source integration_test/test_config.env
patrol test \
--dart-define=BASIC_AUTH_URL=$BASIC_AUTH_URL \
--dart-define=USERNAME=$USERNAME \
--dart-define=PASSWORD=$PASSWORD \
--dart-define=BASIC_AUTH_EMAIL=$BASIC_AUTH_EMAIL \
--dart-define=ADDITIONAL_MAIL_RECIPIENT=$ADDITIONAL_MAIL_RECIPIENT \
...
```
| Variable | Description |
|----------|-------------|
| `BASIC_AUTH_URL` | App server URL (e.g. `http://localhost/`) |
| `USERNAME` | Test account username |
| `PASSWORD` | Test account password |
| `BASIC_AUTH_EMAIL` | Test account email address |
| `ADDITIONAL_MAIL_RECIPIENT` | Secondary email for multi-recipient tests |
> `--dart-define-from-file` only accepts JSON, not `.env` format, so sourcing in the script is the correct approach.
### Running Locally (with visible browser)
```bash
./scripts/patrol-web-local-integration-test-with-docker.sh
```
This starts the Docker backend, then runs patrol with Chrome visible. Useful during development.
To run a single test file, edit the script and replace `patrol test -v` with:
```bash
patrol test -v -t integration_test/tests/composer/send_email_test.dart
```
> After migrating to the architecture defined in [ADR-0081](./0081-patrol-web-test-architecture.md), test files no longer have a `web_` prefix — one file covers all platforms.
### Running in CI (headless)
```bash
./scripts/patrol-web-integration-test-with-docker.sh
```
Adds `--web-headless=true` — no display required. The GitHub Actions workflow (`.github/workflows/patrol-web-integration-test.yaml`) runs this script on pull requests.
### CI Orchestration — Running the Full Project
Mobile and web test suites run as **separate parallel jobs** in CI. Each job targets its own platform:
```text
PR opened / push
│
├── job: patrol-mobile-integration-test (ubuntu-latest + Android emulator)
│ patrol test --device=android
│
└── job: patrol-web-integration-test (ubuntu-latest + Chrome via Patrol)
./scripts/patrol-web-integration-test-with-docker.sh
# patrol test --device=chrome --web-headless=true
```
The two jobs are independent — they share no state and can run concurrently. A PR is gated on **both** jobs passing.
Platform filtering is handled automatically by `runPatrolTest()` (see [ADR-0082](./0082-patrol-web-test-migration-guide.md)). Tests that do not apply to the current platform skip themselves — no `--exclude-tags` flag needed in either job.
### Viewing Results
- **Terminal:** Patrol prints pass/fail per test with full stack traces on failure.
- **GitHub Actions:** Results appear in the workflow run summary under the "Test" step. Each platform job has its own step, making it easy to identify which platform a failure comes from.
## Consequences
- Web and mobile tests share the same test files; platform-only tests skip themselves in `runPatrolTest()` — no manual tagging required.
- The same Docker JMAP backend serves both platforms — no additional infrastructure needed.
- Data isolation limitations from [ADR-0053](./0053-patrol-integration-test.md) still apply: backend state is shared across all tests in a single run.
- Patrol automatically installs and manages Chromium — no manual Chrome installation is needed on local machines or CI runners.
docs/adr/0081-patrol-web-test-architecture.md
+174
View File
@@ -0,0 +1,174 @@
# 81. Patrol Web Integration Test — Architecture & Test Authoring
Date: 2026-04-15
## Status
Proposed
## Related ADRs
- [ADR-0053](./0053-patrol-integration-test.md) — Patrol for mobile integration testing (foundation)
- [ADR-0080](./0080-patrol-web-integration-test-setup.md) — Web test setup & execution
- [ADR-0082](./0082-patrol-web-test-migration-guide.md) — Migration guide & implementation example
- [ADR-0083](./0083-patrol-web-test-migration-plan.md) — PR-by-PR migration plan
## Context
[ADR-0053](./0053-patrol-integration-test.md) established a three-layer architecture — **Test → Scenario → Robot** — for mobile. Extending to web with the POC approach revealed three problems:
- **Robot classes mix platform logic** — `ComposerRobot` has both `addContent()` (mobile) and `addContentWeb()` (web), violating SRP.
- **Scenarios are duplicated** — same business flow written twice for each platform, violating DRY.
- **Closed to extension** — adding a new platform requires modifying existing robot classes, violating OCP.
## Decision
Apply **Abstract Factory** + **Interface Segregation** to separate:
- **WHAT** — Business flow (Scenario): shared across all platforms.
- **HOW** — UI interaction (Robot): implemented per platform.
Platform selection uses **Dart conditional exports** — no logic class ever contains a platform `if/else` branch.
### Architecture
```
Test File (one file per scenario)
│
▼
TestBase.runPatrolTest()
├── auto-applies platform tag (web / mobile)
└── calls createRobotFactory($)
↑ resolved at compile time via conditional export
├── MobileRobotFactory (default)
└── WebRobotFactory (dart.library.html)
│ injects into
▼
BaseTestScenario(PatrolIntegrationTester $, RobotFactory robots)
│ uses
▼
Abstract Robot Interfaces ← Scenarios depend only on these
├── AbstractComposerRobot
└── AbstractThreadRobot
│ implemented by
▼
Concrete Robots
├── mobile/MobileComposerRobot
└── web/WebComposerRobot
```
`TestBase` is the only class that calls `createRobotFactory()`. Scenarios and test files never reference any platform-specific class.
### Folder Structure
```
integration_test/
├── robots/
│ ├── abstract/ # Interfaces — one per feature area
│ ├── mobile/ # Mobile implementations
│ └── web/ # Web implementations (only where UI differs)
├── factories/
│ ├── robot_factory.dart
│ ├── robot_factory_provider.dart # Conditional export — never changes
│ ├── mobile_robot_factory.dart
│ ├── mobile_robot_factory_provider.dart
│ ├── web_robot_factory.dart
│ └── web_robot_factory_provider.dart
├── scenarios/ # All shared — no web_/mobile_ prefix
└── tests/ # One file per scenario — no platform subfolders
```
### Key Design Decisions
**1. Conditional export — the only platform-branching point**
```dart
// factories/robot_factory_provider.dart ← this file never changes
export 'mobile_robot_factory_provider.dart'
if (dart.library.html) 'web_robot_factory_provider.dart';
// Each provider exports one function with the same name
RobotFactory createRobotFactory(PatrolIntegrationTester $) => WebRobotFactory($);
```
Adding a new platform: create a provider file + factory + robot implementations. **No existing file is modified.**
**2. TestBase resolves and injects the factory**
```dart
// base/test_base.dart
import '../factories/robot_factory_provider.dart';
void runPatrolTest({
required String description,
required BaseTestScenario Function(PatrolIntegrationTester $, RobotFactory robots) scenarioBuilder,
List<TestTags> tags = const [],
}) {
final resolvedTags = [
...tags.map((t) => t.name),
];
patrolTest(description, tags: resolvedTags, ($) async {
await setupTest();
await scenarioBuilder($, createRobotFactory($)).execute();
});
}
```
**3. Scenario receives factory via DI — zero platform knowledge**
```dart
abstract class BaseTestScenario extends BaseScenario {
final RobotFactory robots;
const BaseTestScenario(super.$, this.robots);
@override
Future<void> execute() async {
await robots.loginRobot().login(...);
await runTestLogic();
}
}
```
**4. Test file — one file for all platforms**
```dart
void main() {
TestBase().runPatrolTest(
description: 'Send email',
scenarioBuilder: ($, robots) => SendEmailScenario($, robots),
);
}
```
**5. Robot reuse — when behavior is identical, both factories return the same class**
```dart
// WebRobotFactory — reuse mobile robot when no UI difference
@override AbstractThreadRobot threadRobot() => MobileThreadRobot($);
```
### How to Add a Test for a New Feature
1. Add `UiKey`s to all platform widget variants in `lib/features/base/model/ui_keys.dart`.
2. Define an abstract robot interface in `robots/abstract/` if one does not exist for the feature area.
3. Implement the robot in `robots/mobile/` and `robots/web/`. Use a shared base class for methods identical across platforms. Reuse the same class in both factories if behavior is entirely identical.
4. Register the robot in `MobileRobotFactory` and `WebRobotFactory`.
5. Write one scenario in `scenarios/` using abstract robots only.
6. Write one test file in `tests/`: `scenarioBuilder: ($, robots) => MyScenario($, robots)`.
### SOLID Alignment
| Principle | How it applies |
|-----------|---------------|
| **S** | Robot: UI interaction. Scenario: business flow. Factory: object creation. `TestBase`: lifecycle + wiring. |
| **O** | New platform: add provider + factory + robots. No existing class modified. |
| **L** | `WebRobotFactory` and `MobileRobotFactory` are substitutable through `RobotFactory`. |
| **I** | Each robot interface is scoped to its feature area. Split `RobotFactory` if it exceeds ~15 methods. |
| **D** | Scenarios depend on `RobotFactory` (abstraction). `TestBase` imports `createRobotFactory` from provider. |
## Consequences
- Scenarios and test files are written exactly once — no platform variants.
- No logic class contains a platform `if/else` — platform branching is a Dart export declaration.
- Adding a new platform touches zero existing files.
- Platform tags are auto-applied by `TestBase`.
- Migration is a one-time breaking change — see [ADR-0082](./0082-patrol-web-test-migration-guide.md) and [ADR-0083](./0083-patrol-web-test-migration-plan.md).
docs/adr/0082-patrol-web-test-migration-guide.md
+237
View File
@@ -0,0 +1,237 @@
# 82. Patrol Web Integration Test — Migration Guide & Implementation Example
Date: 2026-04-15
## Status
Proposed
## Related ADRs
- [ADR-0053](./0053-patrol-integration-test.md) — Patrol for mobile integration testing (foundation)
- [ADR-0080](./0080-patrol-web-integration-test-setup.md) — Web test setup & execution
- [ADR-0081](./0081-patrol-web-test-architecture.md) — Cross-platform test architecture (read first)
- [ADR-0083](./0083-patrol-web-test-migration-plan.md) — PR-by-PR migration plan
## Context
[ADR-0081](./0081-patrol-web-test-architecture.md) defines the target architecture. This ADR covers the impact on existing mobile tests and provides a concrete implementation example. For the PR breakdown and sequencing, see [ADR-0083](./0083-patrol-web-test-migration-plan.md).
---
## Impact on Existing Mobile Tests
The migration changes two public APIs in `base/`:
| Location | Before | After |
|----------|--------|-------|
| Test file | `scenarioBuilder: ($) => MyScenario($)` | `scenarioBuilder: ($, robots) => MyScenario($, robots)` |
| Scenario constructor | `MyScenario(super.$)` | `MyScenario(super.$, super.robots)` |
| Login | `RequiresLoginMixin` | `robots.loginRobot().login()` in `BaseTestScenario` |
### Estimated scope
| Category | Action | Count |
|----------|--------|-------|
| `scenarios/**/*.dart` | Update constructor | ~80 files |
| `tests/**/*.dart` | Update `scenarioBuilder` | ~120 files |
| `robots/*.dart` | Move to `robots/mobile/`, add interface | ~40 files |
| `robots/abstract/*.dart` | New interfaces | ~40 new files |
| `robots/web/*.dart` | New web implementations | ~15 new files |
| `factories/*.dart` | New factory + provider files | ~6 new files |
| `base/` | Update `TestBase` + `BaseTestScenario` | 2 files |
### What is NOT affected
- `ScenarioUtilsMixin` — provisioning helpers are unchanged.
- `base_scenario.dart`, `core_robot.dart` — unchanged.
- Robot interaction logic — moved, not rewritten.
- CI pipeline scripts — unchanged.
### Side effects
- Mobile CI breaks if the base API change ships without the scenario/test file updates. The final PR must be **atomic** — see [ADR-0083](./0083-patrol-web-test-migration-plan.md).
- `RequiresLoginMixin` is deleted once migration is complete.
- Robots identical across platforms (e.g. `SearchRobot`) need an abstract interface but no web variant.
---
## Implementation Example: Send Email
This example shows the full implementation of a cross-platform test under the new architecture.
### 1. UIKeys — applied to all platform widget variants
```dart
// lib/features/base/model/ui_keys.dart
class UiKeys {
static const String composeEmailButton = 'composeEmailButton';
static const String sendEmailButton = 'sendEmailButton';
}
```
Apply the same key to mobile toolbar, web toolbar, and tablet toolbar widgets.
### 2. Abstract robot interfaces
```dart
// robots/abstract/abstract_composer_robot.dart
abstract class AbstractComposerRobot {
Future<void> addRecipient(PrefixEmailAddress prefix, String email);
Future<void> addSubject(String subject);
Future<void> addContent(String content);
Future<void> send();
}
```
### 3. Shared base + platform implementations
Methods identical across platforms go in a shared base to avoid duplication. The base class can live in its own file (e.g. `robots/mobile/base_composer_robot.dart`) or alongside the mobile implementation:
```dart
// robots/mobile/base_composer_robot.dart
abstract class BaseComposerRobot extends CoreRobot implements AbstractComposerRobot {
BaseComposerRobot(super.$);
// Identical on all platforms — defined once here
@override Future<void> send() async =>
$(const ValueKey(UiKeys.sendEmailButton)).tap();
@override Future<void> addRecipient(...) async { ... }
@override Future<void> addSubject(...) async { ... }
}
class MobileComposerRobot extends BaseComposerRobot {
MobileComposerRobot(super.$);
@override
Future<void> addContent(String content) async {
// Navigate InAppWebView widget tree
ComposerController? controller;
await $(ComposerView).which<ComposerView>((w) {
controller = w.controller; return true;
}).$(MobileEditorView).$(HtmlEditor).$(InAppWebView).tap();
await controller!.htmlEditorApi!.insertHtml('$content <br><br>');
}
}
// robots/web/web_composer_robot.dart
class WebComposerRobot extends BaseComposerRobot {
WebComposerRobot(super.$);
@override
Future<void> addContent(String content) async {
await $(WebEditorWidget).tap();
await $.platformAutomator.web.enterText(
WebSelector(cssOrXpath: 'div.note-editable'),
iframeSelector: WebSelector(cssOrXpath: 'iframe'),
text: content,
);
}
}
```
`ThreadRobot` has no UI difference — one class reused by both factories:
```dart
class MobileThreadRobot extends CoreRobot implements AbstractThreadRobot {
@override Future<void> openComposer() async =>
$(const ValueKey(UiKeys.composeEmailButton)).$(InkWell).tap();
}
```
### 4. Factory registration
```dart
// factories/web_robot_factory.dart
class WebRobotFactory implements RobotFactory {
final PatrolIntegrationTester $;
WebRobotFactory(this.$);
@override AbstractComposerRobot composerRobot() => WebComposerRobot($);
@override AbstractThreadRobot threadRobot() => MobileThreadRobot($); // reused
@override AbstractLoginRobot loginRobot() => WebLoginRobot($);
}
```
### 5. Scenario — written once
```dart
// scenarios/composer/send_email_scenario.dart
class SendEmailScenario extends BaseTestScenario {
const SendEmailScenario(super.$, super.robots);
@override
Future<void> runTestLogic() async {
await robots.threadRobot().openComposer();
await robots.composerRobot().addRecipient(PrefixEmailAddress.to,
const String.fromEnvironment('BASIC_AUTH_EMAIL'));
await robots.composerRobot().addSubject('Send email test');
await robots.composerRobot().addContent('Hello from Patrol');
await robots.composerRobot().send();
await expectViewVisible(
$(find.text(AppLocalizations().message_has_been_sent_successfully)));
}
}
```
### 6. Test file — one file, all platforms
```dart
// tests/composer/send_email_test.dart
void main() {
TestBase().runPatrolTest(
description: 'Send email',
scenarioBuilder: ($, robots) => SendEmailScenario($, robots),
);
}
```
```bash
# Mobile
patrol test -t integration_test/tests/composer/send_email_test.dart \
--device=android
# Web
patrol test -t integration_test/tests/composer/send_email_test.dart \
--device=chrome --web-headless=true
```
Same file. Same scenario. Factory resolved automatically per platform via `kIsWeb` — `--device` already determines the target, so `--tags` is not required for routing.
---
## Handling Platform-Only Features
Some features exist only on web (e.g., a browser-specific upload flow) or only on mobile (e.g., deep links). Using `@Tags` annotations works but is error-prone — developers can easily forget to add them. Instead, platform constraints are encoded **inside `runPatrolTest()`** so the test skips itself automatically.
### CI commands (simplified)
```bash
# Mobile — runs all tests; platform-only tests skip themselves
patrol test --device=android
# Web — runs all tests; platform-only tests skip themselves
patrol test --device=chrome --web-headless=true
```
### Unsupported robot in the factory
As a second safety net, if a web-only robot is accidentally called on mobile (e.g. shared scenario logic), the mobile factory fails fast:
```dart
// factories/mobile_robot_factory.dart
@override
AbstractUploadRobot uploadRobot() =>
throw UnsupportedError('Web upload is not supported on mobile');
// factories/web_robot_factory.dart
@override
AbstractUploadRobot uploadRobot() => WebUploadRobot($);
```
## Consequences
- Migration touches ~200 files but changes are mechanical — suitable for scripted bulk update.
- Atomic PR (base + scenarios + test files) will be large; review it as a structural change, not a logic change.
- After migration, shared base robots keep web implementations minimal — only the genuinely different method is overridden.
docs/adr/0083-patrol-web-test-migration-plan.md
+130
View File
@@ -0,0 +1,130 @@
# 83. Patrol Web Integration Test — Migration Plan
Date: 2026-04-15
## Status
Proposed
## Related ADRs
- [ADR-0081](./0081-patrol-web-test-architecture.md) — Cross-platform test architecture (read first)
- [ADR-0082](./0082-patrol-web-test-migration-guide.md) — Impact analysis & implementation example
## Context
This ADR translates the migration steps from [ADR-0082](./0082-patrol-web-test-migration-guide.md) into a concrete PR plan. Each PR must leave the **mobile CI pipeline green**. Only the final PR is a breaking change.
---
## PR Plan
| PR | Type | Files changed | Files added | Mobile CI | Web CI |
|----|------|:---:|:---:|:---:|:---:|
| PR-1 | Addition | 0 | ~40 | ✅ unaffected | — |
| PR-2 | Refactor | ~82 | 0 | ✅ must pass | — |
| PR-3 | Addition | 0 | ~15 | ✅ unaffected | — |
| PR-4 | Addition | 0 | ~6 | ✅ unaffected | — |
| **PR-5** | **Breaking** | **~202** | 0 | ✅ must pass | ✅ first run |
PRs 1–4 are additive and can be merged in any order. **PR-5 must be a single atomic PR** — splitting it will break mobile CI.
---
## PR-1 — Abstract Robot Interfaces
Create `robots/abstract/` with one interface per existing robot class. No existing file is touched.
- **New files:** ~40 abstract interface files
- **Verify:** `flutter analyze` passes
- **Outcome:** Abstract contracts ready for implementations
---
## PR-2 — Move Robots to `robots/mobile/`
Rename all `robots/*.dart` → `robots/mobile/mobile_*.dart`. Add `implements AbstractXxxRobot` to each class. Extract shared base robots for methods identical across platforms. Update import paths in scenario files.
- **Moved files:** ~40 robots
- **Changed files:** ~80 scenarios + base files (import path updates only)
- **Verify:** `flutter analyze` passes + **mobile CI green** — confirms no logic was changed
- **Outcome:** Mobile robot layer clean; shared base robots ready for web robots to extend
---
## PR-3 — Web Robot Implementations
Create `robots/web/` with web-specific implementations for feature areas where the web UI genuinely differs. Robots with identical behavior across platforms are **not** duplicated here — both factories will return the same mobile class.
- **New files:** ~15 web robot files
- **Verify:** `flutter analyze` passes
- **Outcome:** Web interaction layer ready to be wired via factories
---
## PR-4 — Factories & Provider Files
Create the factory layer and the conditional export provider.
```
factories/
├── robot_factory.dart # Abstract factory interface
├── robot_factory_provider.dart # Conditional export — never changes
├── mobile_robot_factory.dart
├── mobile_robot_factory_provider.dart
├── web_robot_factory.dart
└── web_robot_factory_provider.dart
```
- **New files:** ~6
- **Verify:** `flutter analyze` passes
- **Outcome:** Full factory graph in place; ready for `TestBase` to consume
---
## PR-5 (Atomic) — Base API + All Scenarios + All Test Files
> ⚠️ Do not split. Merging base changes without the scenario/test file updates breaks mobile CI immediately.
**`base/base_test_scenario.dart`** — accept `RobotFactory` via constructor, remove `RequiresLoginMixin`.
**`base/test_base.dart`** — import `robot_factory_provider.dart`, update `runPatrolTest` signature.
**All ~80 scenarios** — update constructor:
```dart
// Before // After
MyScenario(super.$); → MyScenario(super.$, super.robots);
```
**All ~120 test files** — update `scenarioBuilder`:
```dart
// Before // After
scenarioBuilder: ($) => ... → scenarioBuilder: ($, robots) => ...
```
**Delete** `mixin/requires_login_mixin.dart`.
The scenario and test file changes are mechanical and repetitive — suitable for bulk scripted refactoring followed by careful diff review.
- **Changed files:** ~202
- **Deleted files:** 1
- **Verify:** `flutter analyze` + **mobile CI green** + **web CI green** (first successful web run)
- **Outcome:** Architecture fully in place. Both platforms run from the same test files.
---
## Flow Summary
```
PR-1 abstract interfaces → flutter analyze ✅
PR-2 move robots → mobile CI ✅
PR-3 web robots → flutter analyze ✅
PR-4 factories → flutter analyze ✅
PR-5 base + all call sites → mobile CI ✅ web CI ✅
```
## Consequences
- PRs 1–4 are low-risk, reviewable asynchronously, and do not affect the team's daily workflow.
- PR-5 is large but entirely mechanical. Prioritize CI verification over line-by-line review.
- After PR-5, the migration is complete and the new architecture is the baseline for all future test work.