ROCm/TheRock · CI Analysis · Generated May 26, 2026
AWS Build Stage Time Analysis Report
Multi-Arch CI · Branch: main · May 19 – May 26, 2026 · 7-day window
6 AWS runs · 76 build-stage jobs 6 Azure runs · 74 build-stage jobs aws-linux-scale-rocm-prod azure-linux-scale-rocm

Scope: This report compares build-stage job durations between exactly two runner pools: aws-linux-scale-rocm-prod and azure-linux-scale-rocm (exact label match — heavy-ramdisk, Windows, and all other runner variants are excluded). Jobs are sourced from completed Multi-Arch CI workflow runs on main branch, May 19–26 2026. Only jobs whose name contains "Stage" are counted as build-stage jobs.

Data coverage note: AWS jobs were found in 6 runs (May 21–25). Azure exact-label jobs were found in 6 runs (May 22–23). Both pools run the same workflow; the difference in run timing reflects which runs happened to land on each pool during the weighted-routing window (PR #5318). All 14 components now have data on both pools. Both averages are based on n=6 runs for a fair comparison.
🔍 Key Findings

AWS shows a clear split in performance depending on architecture family — strong gains on Linux datacenter GPUs, meaningful regressions on Windows-targeted GPU builds.

✅ AWS Faster — Linux datacenter GPU builds
gfx94X-dcgpu (math-libs): 30% faster  ·  77.2 vs 109.7 min
gfx950-dcgpu (math-libs): 44% faster  ·  67.9 vs 122.1 min
AWS dcgpu instances appear better matched to MI300X / MI350 workloads. Likely larger memory bandwidth and higher core counts on the EC2 instance type.
⚠️ AWS Slower — Windows / consumer GPU builds
gfx110X-all (math-libs): 153% slower  ·  103.7 vs 41.0 min
gfx1151 (math-libs): 77% slower  ·  46.1 vs 26.1 min
gfx110X and gfx1151 are RDNA3/RDNA3.5 consumer GPU targets, often exercised in Windows CI paths. The AWS runner pool may not have equivalent GPU or CPU resources for these workloads — worth investigating instance type alignment.
Recommendation: AWS runners are delivering real value for Linux datacenter GPU builds (gfx94X, gfx950). The gfx110X-all and gfx1151 regressions suggest the AWS runner pool for aws-linux-scale-rocm-prod may need a different instance type or GPU assignment for Windows-targeted architectures. Consider separating the routing policy for these build types.
Summary — 7-Day Build Stage Coverage (May 19–26, 2026)
6
AWS runs sampled
76
AWS build-stage jobs
6
Azure runs sampled
74
Azure build-stage jobs
14
Unique components
14
Components with both pools

Component Build-Stage Comparison: AWS vs Azure

4 architectures · 4 component builds
Component / Build AWS avg, min (n=6) Azure avg, min (n=6) Faster/Slower
gfx94X-dcgpu
math-libs 77.2 109.7 AWS 30% faster
gfx950-dcgpu
math-libs 67.9 122.1 AWS 44% faster
gfx110X-all
math-libs 103.7 41.0 AWS 153% slower
gfx1151
math-libs 46.1 26.1 AWS 77% slower

Showing architecture-specific build stages only (gfx94X-dcgpu, gfx950-dcgpu, gfx110X-all, gfx1151). Cross-architecture components excluded.  Red = AWS slower than Azure.

Build-Stage Duration: AWS vs Azure per Component

Avg duration (min) · all 14 components
aws-linux-scale-rocm-prod  avg, n=6 runs
azure-linux-scale-rocm  avg, n=6 runs
comm-libs
AWS
39.3 min
Azure
99.0 min
compiler-runtime
AWS
20.6 min
Azure
29.1 min
dctools-core
AWS
3.4 min
Azure
2.9 min
debug-tools
AWS
4.7 min
Azure
4.8 min
foundation
AWS
2.5 min
Azure
5.0 min
fusilli-libs
AWS
6.2 min
Azure
4.6 min
iree-compiler
AWS
13.3 min
Azure
4.0 min
math-libs (gfx110X-all)
AWS
103.7 min
Azure
41.0 min
math-libs (gfx1151)
AWS
46.1 min
Azure
26.1 min
math-libs (gfx120X-all)
AWS
77.1 min
Azure
54.4 min
math-libs (gfx94X-dcgpu)
AWS
77.2 min
Azure
109.7 min
math-libs (gfx950-dcgpu)
AWS
67.9 min
Azure
122.1 min
media-libs
AWS
2.1 min
Azure
2.6 min
profiler-apps
AWS
22.6 min
Azure
5.7 min

Bar widths are proportional to average duration, scaled to the longest component (math-libs gfx950-dcgpu Azure = 122.1 min = 100%).

Build Stage Duration: AWS vs Azure (6-run averages)

Grouped · all 14 components
aws-linux-scale-rocm-prod  avg, n=6 runs
azure-linux-scale-rocm  avg, n=6 runs
0 20 40 60 80 100 120 Duration (min) comm-libs compiler-runtime dctools-core debug-tools foundation fusilli-libs iree-compiler math-libs 110X math-libs 1151 math-libs 120X math-libs 94X math-libs 950 media-libs profiler-apps

Y-axis ceiling: 130 min. Each component shows an AWS bar (amber) and an Azure bar (blue) side by side. Values are 6-run averages.

AWS Per-Run Detail — aws-linux-scale-rocm-prod

6 runs · 76 build-stage jobs
Run ID Date Component Duration (min) Conclusion
26418270338 2026-05-25 foundation 2.4 success
26418270338 2026-05-25 compiler-runtime 20.2 success
26418270338 2026-05-25 debug-tools 4.5 success
26418270338 2026-05-25 comm-libs 38.9 success
26418270338 2026-05-25 profiler-apps 22.3 success
26418270338 2026-05-25 dctools-core 3.3 success
26418270338 2026-05-25 media-libs 2.4 success
26418270338 2026-05-25 math-libs (gfx110X-all) 103.2 success
26418270338 2026-05-25 math-libs (gfx94X-dcgpu) 77.9 success
26418270338 2026-05-25 math-libs (gfx1151) 46.2 success
26418270338 2026-05-25 math-libs (gfx120X-all) 79.0 success
26418270338 2026-05-25 math-libs (gfx950-dcgpu) 68.0 success
26406494345 2026-05-25 foundation 2.7 success
26406494345 2026-05-25 compiler-runtime 20.5 success
26406494345 2026-05-25 profiler-apps 22.4 success
26406494345 2026-05-25 debug-tools 4.3 success
26406494345 2026-05-25 media-libs 2.5 success
26406494345 2026-05-25 comm-libs 39.4 success
26406494345 2026-05-25 math-libs (gfx110X-all) 103.8 success
26406494345 2026-05-25 math-libs (gfx94X-dcgpu) 77.7 success
26406494345 2026-05-25 math-libs (gfx120X-all) 77.5 success
26406494345 2026-05-25 math-libs (gfx1151) 46.9 success
26406494345 2026-05-25 math-libs (gfx950-dcgpu) 68.5 success
26406494345 2026-05-25 dctools-core 3.9 success
26275003776 2026-05-22 foundation 2.2 success
26275003776 2026-05-22 compiler-runtime 19.8 success
26275003776 2026-05-22 debug-tools 4.0 success
26275003776 2026-05-22 profiler-apps 22.5 success
26275003776 2026-05-22 dctools-core 2.8 success
26275003776 2026-05-22 media-libs 2.3 success
26275003776 2026-05-22 comm-libs 42.7 success
26275003776 2026-05-22 math-libs (gfx950-dcgpu) 69.7 success
26275003776 2026-05-22 math-libs (gfx120X-all) 77.4 success
26275003776 2026-05-22 math-libs (gfx110X-all) 103.9 success
26275003776 2026-05-22 math-libs (gfx1151) 47.8 success
26275003776 2026-05-22 math-libs (gfx94X-dcgpu) 78.4 success
26265292143 2026-05-22 foundation 2.2 success
26265292143 2026-05-22 compiler-runtime 20.6 success
26265292143 2026-05-22 comm-libs 38.2 success
26265292143 2026-05-22 profiler-apps 22.4 success
26265292143 2026-05-22 media-libs 1.4 success
26265292143 2026-05-22 debug-tools 5.1 success
26265292143 2026-05-22 dctools-core 2.7 success
26265292143 2026-05-22 math-libs (gfx94X-dcgpu) 76.7 success
26265292143 2026-05-22 math-libs (gfx110X-all) 103.6 success
26265292143 2026-05-22 math-libs (gfx120X-all) 77.1 success
26265292143 2026-05-22 math-libs (gfx950-dcgpu) 68.5 success
26265292143 2026-05-22 math-libs (gfx1151) 46.5 success
26250209937 2026-05-21 foundation 3.2 success
26250209937 2026-05-21 compiler-runtime 21.8 success
26250209937 2026-05-21 comm-libs 38.2 success
26250209937 2026-05-21 profiler-apps 23.3 success
26250209937 2026-05-21 debug-tools 5.2 success
26250209937 2026-05-21 media-libs 1.3 success
26250209937 2026-05-21 dctools-core 3.8 success
26250209937 2026-05-21 math-libs (gfx120X-all) 77.3 success
26250209937 2026-05-21 math-libs (gfx110X-all) 104.1 success
26250209937 2026-05-21 math-libs (gfx1151) 46.2 success
26250209937 2026-05-21 math-libs (gfx94X-dcgpu) 77.1 success
26250209937 2026-05-21 math-libs (gfx950-dcgpu) 68.1 success
26250209937 2026-05-21 iree-compiler 9.5 success
26250209937 2026-05-21 fusilli-libs 6.4 success
26250008888 2026-05-21 foundation 2.3 success
26250008888 2026-05-21 compiler-runtime 20.5 success
26250008888 2026-05-21 dctools-core 3.8 success
26250008888 2026-05-21 comm-libs 38.2 success
26250008888 2026-05-21 iree-compiler 17.0 success
26250008888 2026-05-21 profiler-apps 22.5 success
26250008888 2026-05-21 debug-tools 5.2 success
26250008888 2026-05-21 media-libs 2.5 success
26250008888 2026-05-21 math-libs (gfx1151) 43.0 success
26250008888 2026-05-21 math-libs (gfx110X-all) 103.6 success
26250008888 2026-05-21 math-libs (gfx950-dcgpu) 64.8 success
26250008888 2026-05-21 math-libs (gfx94X-dcgpu) 75.6 success
26250008888 2026-05-21 math-libs (gfx120X-all) 74.2 success
26250008888 2026-05-21 fusilli-libs 6.0 success

Azure Per-Run Detail — azure-linux-scale-rocm

6 runs · 74 build-stage jobs
Run ID Date Component Duration (min) Conclusion
26260296418 2026-05-22 foundation 5.4 success
26260296418 2026-05-22 compiler-runtime 36.0 success
26260296418 2026-05-22 comm-libs 99.3 success
26260296418 2026-05-22 debug-tools 4.8 success
26260296418 2026-05-22 dctools-core 2.9 success
26260296418 2026-05-22 profiler-apps 6.2 success
26260296418 2026-05-22 media-libs 2.8 success
26260296418 2026-05-22 iree-compiler 4.0 success
26260296418 2026-05-22 fusilli-libs 4.6 success
26260296418 2026-05-22 math-libs (gfx110X-all) 33.3 success
26260296418 2026-05-22 math-libs (gfx1151) 25.1 success
26260296418 2026-05-22 math-libs (gfx120X-all) 57.1 success
26260296418 2026-05-22 math-libs (gfx94X-dcgpu) 108.7 success
26260296418 2026-05-22 math-libs (gfx950-dcgpu) 91.5 success
26300643008 2026-05-22 foundation 5.3 success
26300643008 2026-05-22 compiler-runtime 22.7 success
26300643008 2026-05-22 dctools-core 2.8 success
26300643008 2026-05-22 debug-tools 5.1 success
26300643008 2026-05-22 media-libs 2.3 success
26300643008 2026-05-22 profiler-apps 5.9 success
26300643008 2026-05-22 comm-libs 99.0 success
26300643008 2026-05-22 math-libs (gfx110X-all) 27.4 success
26300643008 2026-05-22 math-libs (gfx1151) 20.7 success
26300643008 2026-05-22 math-libs (gfx120X-all) 44.6 success
26300643008 2026-05-22 math-libs (gfx94X-dcgpu) 101.7 success
26300643008 2026-05-22 math-libs (gfx950-dcgpu) 86.8 success
26301109870 2026-05-22 foundation 3.9 success
26301109870 2026-05-22 compiler-runtime 22.1 success
26301109870 2026-05-22 dctools-core 3.1 success
26301109870 2026-05-22 profiler-apps 5.5 success
26301109870 2026-05-22 debug-tools 5.0 success
26301109870 2026-05-22 media-libs 2.7 success
26301109870 2026-05-22 comm-libs 97.5 success
26301109870 2026-05-22 math-libs (gfx94X-dcgpu) 111.8 success
26301109870 2026-05-22 math-libs (gfx120X-all) 44.0 success
26301109870 2026-05-22 math-libs (gfx950-dcgpu) 193.7 success
26301109870 2026-05-22 math-libs (gfx110X-all) 33.1 success
26301109870 2026-05-22 math-libs (gfx1151) 26.1 success
26301150995 2026-05-22 foundation 5.4 success
26301150995 2026-05-22 compiler-runtime 36.0 success
26301150995 2026-05-22 comm-libs 99.0 success
26301150995 2026-05-22 profiler-apps 6.0 success
26301150995 2026-05-22 dctools-core 2.8 success
26301150995 2026-05-22 media-libs 2.3 success
26301150995 2026-05-22 debug-tools 4.8 success
26301150995 2026-05-22 math-libs (gfx1151) 40.8 success
26301150995 2026-05-22 math-libs (gfx94X-dcgpu) 123.5 success
26301150995 2026-05-22 math-libs (gfx120X-all) 83.5 success
26301150995 2026-05-22 math-libs (gfx950-dcgpu) 171.8 success
26301150995 2026-05-22 math-libs (gfx110X-all) 87.0 success
26305024550 2026-05-22 foundation 4.7 success
26305024550 2026-05-22 compiler-runtime 27.1 success
26305024550 2026-05-22 profiler-apps 6.0 success
26305024550 2026-05-22 media-libs 2.9 success
26305024550 2026-05-22 comm-libs 98.5 success
26305024550 2026-05-22 dctools-core 2.9 success
26305024550 2026-05-22 debug-tools 5.0 success
26305024550 2026-05-22 math-libs (gfx94X-dcgpu) 103.4 success
26305024550 2026-05-22 math-libs (gfx1151) 23.2 success
26305024550 2026-05-22 math-libs (gfx120X-all) 56.5 success
26305024550 2026-05-22 math-libs (gfx110X-all) 34.8 success
26305024550 2026-05-22 math-libs (gfx950-dcgpu) 105.3 success
26335612340 2026-05-23 foundation 5.2 success
26335612340 2026-05-23 compiler-runtime 30.7 success
26335612340 2026-05-23 media-libs 2.4 success
26335612340 2026-05-23 dctools-core 2.8 success
26335612340 2026-05-23 profiler-apps 4.6 success
26335612340 2026-05-23 comm-libs 100.5 success
26335612340 2026-05-23 debug-tools 4.2 success
26335612340 2026-05-23 math-libs (gfx110X-all) 30.2 success
26335612340 2026-05-23 math-libs (gfx950-dcgpu) 83.7 success
26335612340 2026-05-23 math-libs (gfx120X-all) 40.8 success
26335612340 2026-05-23 math-libs (gfx1151) 20.5 success
26335612340 2026-05-23 math-libs (gfx94X-dcgpu) 108.9 success

Data sourced from GitHub Actions API · ROCm/TheRock · Jobs filtered to exact runner label match only · Report generated 2026-05-27 · AWS data: 6 Multi-Arch CI runs (May 21–25) · Azure data: 6 Multi-Arch CI runs (May 22–23) · Run IDs: 26260296418, 26300643008, 26301109870, 26301150995, 26305024550, 26335612340