Set reduction thresholds for block size and block algo

[TedThemistokleous]

Fix issue with reductions causing kernel blow ups with groupnorm_silu seen in customer models

Was seeing blow up during compile of customer models when selecting block size and block_algo

Added thresholds to generate more reasonable bounds so we don't create large reductions that blow over memory bounds.

Iterated with cursor on this one to get the result. Putting this as draft to sanity check/get reviews.
Got to this concolusion by doing the following with cursor.

1. Repro fault with customer model on Navi3x
2. Found it was deterministic (simultatnous faults)
3. Added test case that reproduced the fault
4. Tracked with MIGRAPHX_TRACE_BENCHMARKING=3 + MIGRAPHX_TRACE_COMPILE=1
5. Identified first kernel fault (Block size 1024 which generated large kernel that went out of bounds)
6. Retested and iterated on this - Still failed
7. Found 2nd failure related to block algo size using step 3 logging on customer model during compile
8. Added change to picking reduction algorithm
9. Retested with all green
10. Retested on customer model

Potential relevant fix to - #5052

Original tested this on Navi3x where I was able to repro the isssue with the added unit tests and then verify fix works. @tamahedi @tperry-amd give this a try in your setup with a cherry-pick and see if this resolves your case on Strix workflow.

Motivation

Technical Details

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

• • Added: New functionality.
• • Changed: Changes to existing functionality.
• • Removed: Functionality or support that has been removed. (Compared to a previous release)
• • Optimized: Component performance that has been optimized or improved.
• • Resolved Issues: Known issues from a previous version that have been resolved.
• • Not Applicable: This PR is not to be included in the changelog.

Follow the LLVM AI Tool Use Policy for contributions using AI.

[gh-app-migraphx-bot-pr-write]

Test	Batch	New Rate (7033af)	Old Rate (3a503c)*	Diff	Status
torchvision-resnet50	64	3,320.81	3,264.92	1.71%	✅
torchvision-resnet50_fp16	64	7,873.91	7,548.67	4.31%	✅
torchvision-densenet121	32	2,490.62	2,483.99	0.27%	✅
torchvision-densenet121_fp16	32	5,024.39	5,004.24	0.40%	✅
torchvision-inceptionv3	32	2,071.21	2,058.51	0.62%	✅
torchvision-inceptionv3_fp16	32	4,453.33	4,416.99	0.82%	✅
cadene-inceptionv4	16	817.53	820.61	-0.37%	✅
cadene-resnext64x4	16	782.69	782.78	-0.01%	✅
slim-mobilenet	64	8,382.60	8,386.36	-0.04%	✅
slim-nasnetalarge	64	228.34	228.86	-0.23%	✅
slim-resnet50v2	64	3,238.10	3,180.91	1.80%	✅
bert-mrpc-onnx	8	1,171.26	1,168.84	0.21%	✅
bert-mrpc-tf	1	505.92	498.63	1.46%	✅
pytorch-examples-wlang-gru	1	493.53	473.35	4.26%	✅
pytorch-examples-wlang-lstm	1	456.13	384.83	18.53%	🔆
torchvision-resnet50_1	1	1,057.63	1,046.63	1.05%	✅
cadene-dpn92_1	1	455.09	437.32	4.06%	✅
cadene-resnext101_1	1	364.60	365.89	-0.35%	✅
onnx-taau-downsample	1	850.72	844.09	0.79%	✅
dlrm-criteoterabyte	1	32.23	32.42	-0.59%	✅
dlrm-criteoterabyte_fp16	1	51.63	51.80	-0.33%	✅
agentmodel	1	14,284.04	9,209.12	55.11%	🔆
unet_fp16	2	58.74	58.80	-0.10%	✅
resnet50v1_fp16	1	1,457.87	1,366.11	6.72%	🔆
resnet50v1_int8	1	1,812.98	1,883.96	-3.77%	✅
bert_base_cased_fp16	64	1,097.24	1,098.16	-0.08%	✅
bert_large_uncased_fp16	32	345.87	345.59	0.08%	✅
bert_large_fp16	1	206.57	206.59	-0.01%	✅
distilgpt2_fp16	16	2,092.89	2,092.89	0.00%	✅
yolov5s	1	566.22	558.33	1.41%	✅
tinyllama	1	45.84	45.83	0.01%	✅
vicuna-fastchat	1	44.19	44.20	-0.02%	✅
whisper-tiny-encoder	1	412.23	411.87	0.09%	✅
whisper-tiny-decoder	1	410.37	408.48	0.46%	✅
llama2_7b	1	20.86	20.84	0.10%	✅
qwen1.5-7b	1	23.67	23.58	0.40%	✅
phi3-3.8b	1	28.33	26.72	6.05%	🔆
llama3-8b	1	22.75	21.80	4.35%	✅
whisper-large-encoder	1	10.17	10.18	-0.09%	✅
whisper-large-decoder	1	106.52	105.30	1.16%	✅
mistral-7b	1	23.76	23.78	-0.09%	✅
FLUX.1-schnell	1	783.34	755.22	3.72%	✅

Check flagged results 🔆

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

[gh-app-migraphx-bot-pr-write]

Test	Status	Result
bert-mrpc-onnx	✅	PASSED: MIGraphX meets tolerance
bert-mrpc-tf	❌	ERROR - check error output traceback Traceback (most recent call last): File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in main() File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 313, in main import tensorflow as tf File "/usr/local/lib/python3.12/dist-packages/tensorflow/init.py", line 40, in from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow # pylint: disable=unused-import ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 37, in self_check.preload_check() File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/platform/self_check.py", line 63, in preload_check from tensorflow.python.platform import _pywrap_cpu_feature_guard ImportError: libnuma.so.1: cannot open shared object file: No such file or directory
pytorch-examples-wlang-gru	✅	PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm	✅	PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte	✅	PASSED: MIGraphX meets tolerance
agentmodel	✅	PASSED: MIGraphX meets tolerance
unet	✅	PASSED: MIGraphX meets tolerance
resnet50v1	✅	PASSED: MIGraphX meets tolerance
bert_base_cased_fp16	✅	PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16	🔴	FAILED: MIGraphX is not within tolerance - check verbose output
bert_large	✅	PASSED: MIGraphX meets tolerance
yolov5s	✅	PASSED: MIGraphX meets tolerance
tinyllama	✅	PASSED: MIGraphX meets tolerance
vicuna-fastchat	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-encoder	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-decoder	✅	PASSED: MIGraphX meets tolerance
distilgpt2_fp16	🔴	FAILED: MIGraphX is not within tolerance - check verbose output
llama2_7b	✅	PASSED: MIGraphX meets tolerance
qwen1.5-7b	✅	PASSED: MIGraphX meets tolerance
phi3-3.8b	✅	PASSED: MIGraphX meets tolerance
llama3-8b	✅	PASSED: MIGraphX meets tolerance
whisper-large-encoder	✅	PASSED: MIGraphX meets tolerance
whisper-large-decoder	✅	PASSED: MIGraphX meets tolerance
mistral-7b	✅	PASSED: MIGraphX meets tolerance
FLUX.1-schnell	✅	PASSED: MIGraphX meets tolerance

[tamahedi]

Built and tested on strix halo - it allows the model to compile and run as expected, also not causing reboots on modelbench.