Performance and benchmark testing
Performance And Benchmark Testing Deep Dive¶
Overview¶
This testing area improves confidence by focusing on macro vs micro benchmarks, regression thresholds, and device variance.
Core Concepts¶
- Prefer deterministic tests over broad but brittle coverage.
- Test behavior contracts rather than implementation details.
- Keep test ownership close to code ownership.
Test Pyramid and Strategy¶
- Place this topic at the correct layer in the pyramid.
- Reserve expensive tests for high-risk regressions.
- Use risk history to prioritize suite depth.
Tooling and Infrastructure¶
- Standardize fixture builders and dependency overrides.
- Keep environment setup hermetic and repeatable.
- Use CI sharding to maintain feedback speed.
Flaky Test Mitigation¶
- Remove hidden time and thread dependencies.
- Replace sleeps with explicit synchronization.
- Quarantine only as a temporary safety valve.
Code Examples¶
@Test
fun sample() = runTest {
// Arrange
// Act
// Assert
}
Common Interview Questions¶
- Q: Why does this test belong at this layer? A: Answer by test pyramid intent: unit for logic speed, integration for boundaries, and end-to-end for critical journeys with flakiness controls.
- Q: What failure mode does this suite catch best? A: Answer by test pyramid intent: unit for logic speed, integration for boundaries, and end-to-end for critical journeys with flakiness controls.
- Q: How do you keep signal high as suite size grows? A: Answer by test pyramid intent: unit for logic speed, integration for boundaries, and end-to-end for critical journeys with flakiness controls.
Production Considerations¶
- Tie test gates to release risk and incident history.
- Track regression escape rate as an outcome metric.
- Keep slow, low-signal tests under review.
Performance Insights¶
- Fast suites enable frequent refactoring and safer iteration.
- Deterministic async testing reduces rerun waste in CI.
Senior-Level Insights¶
- Great answers explain tradeoffs between confidence, cost, and team velocity.