Address review comments

Signed-off-by: Kai Xu <kaix@nvidia.com>

Author: kaix-nv

.claude/skills/evaluation/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: evaluation
-description: Evaluate accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Use when user says "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "accuracy drop", "run nel", or needs to measure how quantization affects model quality. Handles model deployment, config generation, and evaluation execution.
+description: Evaluate accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Use when user says "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "accuracy drop", "run nel", or needs to measure how quantization affects model quality. Handles model deployment, config generation, and evaluation execution. Do NOT use for quantizing models (use ptq) or deploying/serving models (use deployment).
 license: Apache-2.0
 # Based on nel-assistant skill from NeMo Evaluator Launcher (commit f1fa073)
 # https://github.com/NVIDIA-NeMo/Evaluator/tree/f1fa073/packages/nemo-evaluator-launcher/.claude/skills/nel-assistant
@@ -76,6 +76,8 @@ Prompt the user with "I'll ask you 5 questions to build the base config we'll ad
 
 DON'T ALLOW FOR ANY OTHER OPTIONS, only the ones listed above under each category (Execution, Deployment, Auto-export, Model type, Benchmarks). YOU HAVE TO GATHER THE ANSWERS for the 5 questions before you can build the base config.
 
+> **Note:** These categories come from NEL's `build-config` CLI. If `nel skills build-config --help` shows different options than listed above, use the CLI's current options instead.
+
 When you have all the answers, run the script to build the base config:
 
 ```bash
@@ -118,14 +120,7 @@ If no `hf_quant_config.json`, also check `config.json` for a `quantization_confi
 
 **Quantization-aware benchmark defaults:**
 
-When a quantized checkpoint is detected, recommend benchmarks sensitive to quantization accuracy loss:
-
-- **Always include**: MMLU (general knowledge — typically shows measurable accuracy loss from quantization)
-- **Recommended**: GSM8K (math reasoning — sensitive to precision loss), ARC-Challenge (reasoning)
-- **Good to add**: HumanEval (code generation — catches subtle degradation), Winogrande (commonsense)
-- **Less useful for quant comparison**: IFEval (instruction following — typically less affected, but worth including for aggressive quantization like FP4)
-
-Present these recommendations to the user and ask which to include. If the user already specified benchmarks, keep their choice but mention any accuracy-sensitive benchmarks they may have missed.
+When a quantized checkpoint is detected, read `references/quantization-benchmarks.md` for benchmark sensitivity rankings and recommended sets. Present recommendations to the user and ask which to include.
 
 Read `references/model-card-research.md` for the full extraction checklist (sampling params, reasoning config, ARM64 compatibility, pre_cmd, etc.). Use WebSearch to research the model card, present findings, and ask the user to confirm.
 
@@ -191,7 +186,7 @@ Print the following commands to the user. Propose to execute them in order to co
 **Important**: Export required environment variables based on your config. If any tokens or keys are missing (e.g. `HF_TOKEN`, `NGC_API_KEY`, `api_key_name` from the config), ask the user to put them in a `.env` file in the project root so you can run `set -a && source .env && set +a` (or equivalent) before executing `nel run` commands.
 
 ```bash
-# If using pre_cmd or post_cmd:
+# If using pre_cmd or post_cmd (review pre_cmd content before enabling — it runs arbitrary commands):
 export NEMO_EVALUATOR_TRUST_PRE_CMD=1
 
 # If using nemo_skills.* tasks with self-deployment:

.claude/skills/evaluation/references/quantization-benchmarks.md

@@ -0,0 +1,26 @@
+# Quantization-Aware Benchmark Recommendations
+
+When evaluating a quantized checkpoint, prioritize benchmarks that are sensitive to precision loss.
+
+## Sensitivity ranking
+
+| Priority | Benchmarks | Why |
+|----------|-----------|-----|
+| **Always include** | MMLU | General knowledge — typically shows measurable accuracy loss from quantization |
+| **Recommended** | GSM8K, ARC-Challenge | Math reasoning and general reasoning — sensitive to precision loss |
+| **Good to add** | HumanEval, Winogrande | Code generation and commonsense — catches subtle degradation |
+| **Less useful for quant comparison** | IFEval | Instruction following — typically less affected, but worth including for aggressive quantization like FP4 |
+
+## Recommended sets by use case
+
+| Use case | Benchmarks |
+|----------|-----------|
+| Quick sanity check | MMLU |
+| Standard quant validation | MMLU, GSM8K, ARC-Challenge |
+| Thorough evaluation | MMLU, GSM8K, ARC-Challenge, HumanEval, Winogrande |
+| Code-focused model | HumanEval, MBPP, MMLU |
+| Reasoning model | GSM8K, MATH-500, GPQA, MMLU |
+
+## How to use
+
+Present these recommendations to the user and ask which to include. If the user already specified benchmarks, keep their choice but mention any accuracy-sensitive benchmarks they may have missed.