QA Automation Journey
Chatoshi Mobile
E2E Automation Framework
From ad-hoc coordinate taps to a rock-solid, id-based test suite trusted for every release.
The numbers
Concrete wins from the foundation phase.
8
Accessibility MRs shipped to shared package
13/13
Smoke suite flows passing green
10 → 0
Coordinate based taps eliminated
2
Apps benefit (Chatoshi plus VIP via shared code)
Shipped: 8 accessibility merge requests
Each MR added stable identifiers to a widget. Shared package means both apps benefit.
!1306
Message controls bar
Overflow menu on every AI response
!1308
Chat reason status widget
Report submission success sheet
!1309
Chat text field send button
Primary send or stop action in composer
!1315
Custom bottom nav bar
Chat, History, Alerts, Learn, Menu tabs
!1318
Scroll to bottom button
Chat surface auto scroll arrow
!1319
History item
Chat history list row
!1326
Rendering issue reminder
First time educational popup
!1327
First session explanation
Post onboarding starter suggestions
Roadmap
Four phases showing where we are and where we're going.
Done
Phase 1
Foundation
Foundation
- Smoke suite green
- 8 MRs shipped
- Coord taps eliminated
- Dev QA pattern proven
Now
Phase 2
Improve
Improve
- Smoother test runs
- Fewer manual steps
- Faster feedback loop
- Small quality of life fixes
Next
Phase 3
Expand
Expand
- More coverage
- New feature areas
- Better selectors
- Continued cleanup
Future
Phase 4
Scale
Scale
- Broader reach
- Deeper insights
- Shared patterns
- Nice to have polish
Six pillars of the framework
What holds the whole thing up.
Selector strategy
Semantics ids first. Text second. Coordinates never.
Coverage strategy
Smoke gates releases. Regression catches drift. Feature MRs ship with tests.
Reliability
Deterministic sim state. Smart retry on infra failures. Fail loud on real bugs.
CI/CD integration
Tests run on every MR. Failures block merges. Green means truly green.
Cross app scale
Shared package pattern where every win benefits Chatoshi and VIP together.
Observability
Structured logs. Failure reports. Trends visible to the whole team.
Auto trigger modals, status
Designer flagged 5 modals for automation. Current progress:
1
Rate app request
Native OS modal, outside Flutter's reach
Out of scope
2
Rendering issue reminder
First time popup after chat response
Shipped
3
First session explanation
Post onboarding starter suggestions
Shipped
4
Update available
Full screen new version prompt
Pending dev
5
Critical update available
Forced update variant
Pending dev
How the team makes it work
Automation scales when everyone owns their piece.
QA (me)
Owns flows, selector strategy, runner infra, and coverage roadmap.
Frontend devs
Add Semantics ids to new widgets as part of Definition of Done.
Backend devs
Provide testable states or debug flags where features can't otherwise be triggered.
Reviewers
Confirm identifier naming and public const exposure on every MR.
Team lead
Owns CI resources, merge policies, and cross team alignment.
Designer
Flags auto trigger UX to QA proactively so tests plan around it.
The message in one line
We've built a stable foundation for mobile E2E automation. From here we keep improving it, step by step, based on what the team needs most.