Sr QA Automation Engineer

Role Overview

The Senior QA Automation Engineer (Performance, Database & Scale Testing) is responsible for designing, building, and operating the performance and scale testing capability across SmartShipHubs cloud-native platform. The role owns the full performance engineering lifecycle — from requirements and test design through execution, analysis, and actionable recommendations to engineering teams.

The successful candidate will architect reusable, maintainable performance test frameworks; build AI-augmented test generation pipelines leveraging Gen AI tooling; validate database query performance under realistic fleet-scale loads; and champion performance as a first-class quality gate in CI/CD pipelines.

JMeter / Gatling / k6

Locust / Artillery

Prometheus / Grafana

PostgreSQL / MongoDB

Cloud: GCP / AWS / Azure

Gen AI: Claude / ChatGPT

K8s Scale Testing

Database Benchmarking

CI/CD Integration

IoT / Maritime SaaS

Experience Requirements

Core Experience

• 7-12 years of total QA / quality engineering experience with meaningful progression
• 5+ years of hands-on performance, load, and scale testing experience in cloud-native, SaaS, or product-based environments
• Proven experience building performance test frameworks from scratch — not just running existing scripts
• Experience at a product-based company or MNC with global SLAs and high-availability commitments is strongly preferred
• Experience contributing to or owning performance engineering as a dedicated function (not combined QA generalist)

• Experience testing cloud-based products deployed on GCP, AWS, or Azure — Kubernetes-hosted microservices strongly preferred
• Experience with IoT, maritime, industrial, or high-volume telemetry platforms is a significant plus
• Comfort working in polyglot engineering environments (Java, Python, Node.js, Go) across test tooling
• Experience collaborating with SRE, platform, and database engineering teams on performance RCA
• Exposure to maritime vessel data, AIS feeds, or time-series telemetry data pipelines is a bonus.

• Performance & Load Testing Frameworks
• Expert-level proficiency with two or more: Apache JMeter, Gatling, k6, Locust, Artillery, Tsung — test plan design, parameterisation, distributed execution
• Test scenario design: steady-state load, ramp-up/ramp-down, spike tests, soak tests, breakpoint / stress tests — mapped to real user journey profiles
• Distributed load generation: JMeter master-slave clusters on Kubernetes; k6 operator on GKE/EKS; Artillery cloud generating realistic global load from multiple regions
• Protocol coverage: HTTP/HTTPS REST, gRPC, WebSocket, MQTT (IoT telemetry), GraphQL, Server-Sent Events (SSE) — performance testing beyond simple HTTP
• Realistic data generation: parameterised virtual user datasets, correlated parameters (login → session token → API call chains), data pool management
• Correlation and dynamic extraction: response token extraction, session correlation, JSON/XML/regex extractors — no hard-coded test values
• Think time and pacing: realistic inter-request delays, user pacing, constant throughput timers — preventing artificial test results
• Performance test environment management: traffic shaping, network throttling simulation, database seeding with production-scale data volumes
• Database Performance & Query Optimisation Testing
• PostgreSQL performance testing: EXPLAIN / EXPLAIN ANALYSE, query plan inspection, index usage validation, connection pool benchmarking (PgBouncer)
• MongoDB performance: aggregation pipeline performance analysis, index strategy validation, read/write throughput benchmarking under shard-scale data volumes
• Time-series databases: TimescaleDB / InfluxDB / BigQuery — benchmarking ingestion throughput for IoT telemetry (vessel sensor data at fleet scale)
• Database load simulation: realistic concurrent query load generation (pgbench, mongo-perf, custom Python/Java load drivers) replicating production query mix
• Slow query identification: automated slow query log analysis, long-running transaction detection, lock contention identification
• N+1 query detection: integration of database call tracing into API performance tests — identify ORM-generated N+1 patterns under load
• Connection pool exhaustion testing: simulate pool saturation, queue depth, connection timeout behaviour under peak concurrency
• Database backup performance: validate backup and restore duration under production data volumes against RTO/RPO targets
• Data volume scaling: populate test environments with statistically representative data volumes (millions of vessel events, port calls, voyage records)
• Cloud Scale Testing & Infrastructure Validation
• Kubernetes scale testing: HPA (Horizontal Pod Autoscaler) validation — trigger, scale-up latency, scale-down behaviour; PodDisruptionBudget compliance under load
• Cloud service limits: API rate limit validation, cloud quota headroom testing, throttling behaviour under burst traffic
• Multi-region load testing: latency profiling across GCP / AWS / Azure regions; CDN / global load balancer behaviour validation
• Chaos engineering for performance: Chaos Mesh / LitmusChaos — pod kill, network latency injection, disk I/O stress during load tests to validate graceful degradation
• Storage performance: GCS / S3 / Azure Blob throughput testing; PVC / persistent volume I/O benchmarking in Kubernetes
• Message queue throughput: Kafka / Pub/Sub / RabbitMQ consumer lag benchmarking; producer throughput at fleet-scale IoT message rates
• Cache performance validation: Redis / Memcached hit ratio, eviction rate, latency percentiles under production-representative cache workloads
• CDN and edge performance: cache-hit rate validation, origin offload percentage, global TTFB (Time to First Byte) benchmarking
• Autoscaling cost analysis: correlate scale events with cloud billing impact — performance vs cost trade-off reporting
• Gen AI — Augmented Test Design & Automation (Mandatory)
• Claude (Anthropic): using Claude API / claude.ai to generate realistic load test scripts from API specifications (OpenAPI / Swagger), user story descriptions, or production traffic samples
• ChatGPT / GPT-4o: prompt engineering for test scenario generation, edge case identification, performance test data synthesis, test result narrative generation
• AI-assisted test code generation: using LLM prompts to generate k6 / JMeter / Gatling scripts, database test queries, and data seed scripts — with human review and validation
• AI for anomaly analysis: using LLMs to analyse Grafana/Prometheus alert output, slow query logs, and JMeter result summaries — generate natural-language RCA narratives
• Prompt engineering discipline: structured prompts for consistent, reproducible test artefact generation; version-controlled prompt libraries; output validation pipelines
• AI-powered test maintenance: LLM-based script update generation when APIs change — diff-aware prompt templates that preserve existing scenario logic
• Gen AI evaluation: critical evaluation of LLM-generated test output for correctness, security (no credential leakage), and representativeness — AI assists, engineer decides
• Emerging tooling: staying current with AI-native testing tools (Testim AI, Mabl, Functionize, Applitools) and evaluating their applicability to performance use cases
• Performance Observability, Metrics & Analysis
• Prometheus: custom metrics instrumentation (client libraries for Java / Python / Node.js), performance-relevant alerting rules, histogram / percentile analysis
• Grafana: performance test result dashboards (live during test execution), SLO compliance dashboards, regression comparison across test runs
• Distributed tracing: OpenTelemetry / Jaeger / Zipkin — trace slow paths under load, identify latency contributions per microservice during test execution
• APM integration: Datadog / New Relic / Dynatrace — correlate load test activity with APM traces for root cause identification
• JMeter / k6 result analysis: response time percentile analysis (p50, p90, p95, p99), error rate analysis, throughput vs latency correlation
• Flame graph analysis: async-profiler / perf / py-spy — CPU and memory hotspot identification under load in JVM and Python services
• Statistical analysis: performance regression detection using Mann-Whitney U test or t-test — automated baseline comparison in CI pipeline
• Performance test reporting: executive summary generation (automated via Gen AI), trend reports, capacity planning recommendations
• CI/CD Pipeline Integration & Shift-Left Performance
• GitHub Actions / GitLab CI: integrate k6 / Gatling performance tests as CI pipeline stages — block merges on performance regression (p99 threshold breach)
• Performance baselines: automated baseline capture on main branch; PR-triggered comparative tests with pass/fail decision against baseline
• Performance budget enforcement: define and enforce response time, throughput, and error rate budgets per API endpoint — surfaced as PR status checks
• Staging environment performance gates: automated nightly performance regression suite in staging — failures block production deployments
• Artefact management: performance test scripts versioned in git; test result artefacts stored in GCS / S3 with retention policies; baseline database in Postgres
• Container-native test execution: k6 operator / JMeter Docker containers orchestrated in Kubernetes — ephemeral, reproducible test infrastructure
• Cloud-Native Product Testing Expertise
• REST API performance testing: complete CRUD lifecycle load testing, pagination performance, bulk operation throughput, concurrent user simulation
• WebSocket / real-time performance: sustained connection load testing (vessel tracking dashboards receiving live AIS updates), message delivery latency validation
• Multi-tenancy performance isolation: validate that one tenants load does not degrade other tenants performance — noisy-neighbour detection
• SLA / SLO validation: automated validation that p99 API response times meet contractual SLAs under defined load profiles
• Geographical latency testing: Cloud-native load generation from GCP / AWS regions closest to vessel fleet locations; latency profiling per maritime route
• Serverless cold-start benchmarking: Cloud Functions / Lambda cold-start latency measurement and optimisation impact validation

Performance Engineering Ownership

• Own the end-to-end performance test strategy for SmartShipHubs cloud platform — from SLA definition through test design, execution, analysis, and remediation tracking
• Build and maintain reusable, modular performance test frameworks covering API, database, messaging, and WebSocket layers
• Establish and maintain performance baselines per sprint; detect and report regressions before they reach production
• Define performance NFRs (Non-Functional Requirements) in collaboration with product and engineering — translate into testable acceptance criteria
• Lead performance RCA investigations — coordinate with platform, database, and backend engineering to resolve identified bottlenecks

• Design and execute database performance test suites for PostgreSQL, MongoDB, and time-series stores — query benchmarking, index validation, connection pool testing
• Populate test environments with statistically representative production-scale data volumes — automated data seeding pipelines
• Validate scale-out behaviour of Kubernetes-hosted services under fleet-scale IoT telemetry loads (millions of vessel events per hour)
• Validate BCP/DR database failover performance — measure RTO/RPO achievement under controlled failure scenarios

• Build and maintain Gen AI-augmented test generation pipelines (Claude, ChatGPT) — accelerating test script creation, data synthesis, and RCA reporting
• Develop prompt engineering standards and libraries for performance test artefact generation — version-controlled, reviewed, and auditable
• Evaluate and adopt emerging AI-native testing tools appropriate for SmartShipHubs platform context
• Advocate for responsible AI use in testing — ensuring LLM-generated output is validated, not blindly trusted

Performance & Testing Tools

• Apache JMeter, Gatling, k6, Locust, Artillery
• pgbench, mongo-perf, TimescaleDB bench
• Chaos Mesh, LitmusChaos
• OpenTelemetry, Jaeger, Zipkin
• Datadog / New Relic / Dynatrace APM
• async-profiler, py-spy (flame graphs)
• Prometheus, Grafana (dashboards)
• BlazeMeter / Flood.io (cloud load generation)

• Claude (Anthropic API / claude.ai)
• ChatGPT / GPT-4o (OpenAI API)
• GitHub Copilot (AI code completion)
• Testim AI, Mabl, Functionize (evaluation)
• Python / JavaScript / Java (test frameworks)
• GitHub Actions, GitLab CI (pipeline integration)
• Docker / Kubernetes (test infrastructure)
• GCP / AWS / Azure (cloud environments)