integrating-a-use-case
GitHub将现有科学训练代码(脚本、笔记本等)转换为 itwinai 插件,支持 HPC 分布式训练、MLflow 日志记录、Ray 超参优化及性能分析。
触发场景
安装
npx skills add interTwin-eu/itwinai --skill integrating-a-use-case -g -y
SKILL.md
Frontmatter
{
"name": "integrating-a-use-case",
"description": "Use when adapting existing scientific training code (a standalone PyTorch script, a notebook, or a research repo) into an itwinai plugin - covers scaffolding from the plugin template, porting the training loop, distributed training on HPC, MLflow logging, Ray-based hyperparameter optimization, profiling, and scalability reports. Also use when adding any one of those capabilities to a plugin that already exists."
}
Integrating a use case into itwinai
Turn existing scientific training code into an installable itwinai plugin that runs distributed on HPC, logs to MLflow, tunes hyperparameters with Ray, and produces a scalability report.
This skill targets itwinai 0.5.0. Phase 0 checks the installed version against it.
Re-entry: already have a working plugin?
Do not restart from Phase 0. Jump straight to what you need.
| You want to | Go to |
|---|---|
| Start from a training script | Phase 0 |
| Add logging | Phase 4, step 1 |
| Add distributed training | Phase 4, step 2 |
| Add profiling / GPU monitoring | Phase 4, step 3 |
| Add hyperparameter optimization | Phase 4, step 4 |
| Produce a scalability report | Phase 5 |
| Publish the trained model to the Model Hub | references/model-hub.md |
| Port to another HPC system | references/slurm.md |
| Something failed | references/troubleshooting.md |
Rules
- Gates are hard. Each phase ends with a gate. If it fails, go to
references/troubleshooting.mdand fix it. Never advance past a failing gate, and never report a phase complete without running its gate command and seeing it pass. A gate the machine cannot run is not a failing gate: stop there and say so, see Rule 2. - Stop where the hardware stops. Gates 1 to 3 run anywhere, including a laptop. Gate 4 needs a GPU node and SLURM, Gate 5 needs several. Without them, finish Phase 3, report which gates could not be run and why, and stop. Write the later config only if the user asks for it, and label it unverified. Never present an unrun gate as passed.
- Read the source, don't trust this skill for field values. Every reference names the class that owns a set of fields. Read that class from the installed itwinai before generating config. Users are often on a skill version older than their itwinai.
- Load references lazily. Read a reference file when you reach the step that needs it, not before.
- Change behaviour and structure in separate steps. Porting must preserve the science. If the ported code trains differently from the original, that is a bug, not an improvement.
- Ask before consuming allocation. Any multi-node job, and anything in Phase 5, needs the user's explicit go-ahead.
Phase 0 - Assess
Check version skew.
itwinai --version
If it differs from the version stated above, say so plainly and recommend the user refresh this
plugin (/plugin menu) before continuing. Then continue anyway, obeying Rule 3 more strictly.
Inventory the source code. Read the training script and write a short table recording where each of these lives, or "none":
| Concern | Where it is now |
|---|---|
| Model construction | |
| Optimizer / LR scheduler | |
| Loss function | |
| Training loop | |
| Validation loop | |
| Dataset and DataLoader | |
| Metrics | |
| Checkpointing | |
| Hyperparameters (argparse, constants, YAML) | |
| Entry point |
Choose the fork. This decides Phase 2 and nothing else:
- Components-only - the model is a plain
nn.Module, the loss is one of itwinai's built-ins, and the loop is a conventional forward/backward/step. You write no trainer at all: aDataGettersupplies the data and the stockitwinai.torch.trainer.TorchTrainerruns it. This is the default. Choose it unless something below forces the other branch. - Custom trainer - a
TorchTrainersubclass. Required when the loss is not inTrainingConfiguration's allowed set, the optimizer or scheduler is constructed in a non-standard way, the training step is unusual (multiple optimizers, custom backward, physics residuals, rollouts), or batches need per-batch transformation.
Gate 0: the inventory table is filled in and the fork is chosen and justified in one sentence.
Phase 1 - Scaffold
Read references/scaffolding.md.
Create the plugin repository from itwinai-plugin-template, choose the package name, and fix
pyproject.toml.
Gate 1:
uv pip install -e .
python -c "import itwinai.plugins.<name>; print('ok')"
Phase 2 - Port
Read references/porting-training-code.md, then the branch you chose in Phase 0.
Move the science under src/itwinai/plugins/<name>/. Keep the original script around until
Phase 3's gate passes, so you can diff behaviour against it.
Gate 2: the import above still succeeds, and no module defines a class whose name collides with an itwinai class it also imports. Check the second half, do not eyeball it:
python -c "import itwinai.plugins.<name>; print('ok')"
grep -rhoE "^class [A-Za-z_][A-Za-z_0-9]*" src/itwinai/plugins/<name>/ | sort -u
grep -rhoE "from itwinai[A-Za-z_.0-9]* import .*" src/itwinai/plugins/<name>/ | sort -u
No name from the first list may appear in the second. An overlap means your class shadows the
itwinai class of the same name in that module: rename yours after the science (FNOTrainer).
Phase 3 - Wire
Read references/pipeline-config.md.
Write config.yaml: a Pipeline with steps, hyperparameters at the top, ${} interpolation to
avoid repeating them.
Gate 3 - the first rung that actually trains. Use a truncated dataset and one epoch:
itwinai exec-pipeline --config-name config +pipe_key=training_pipeline epochs=1
Run it with no launcher, so itwinai falls back to NonDistributedStrategy. The loss must move.
Compare it against a one-epoch run of the original script; they should be in the same range.
Phase 4 - Capabilities
Do these in order. They form a dependency chain, not a menu. generate-scalability-report
reads its data out of MLflow, so profiling without logging, or a scaling test without either,
produces an empty report and no error message.
- Logging -
references/logging.md. AddLoggersCollectionwithConsoleLoggerandMLFlowLogger. Re-run Gate 3 and confirm metrics appear undermllogs/mlflow. - Distributed -
references/distributed.md, thenreferences/slurm.md. Add theslurm_configblock toconfig.yamland submit a single-node, multi-GPU job. - Profiling -
references/profiling-and-scalability.md. Enablemeasure_epoch_timeandmeasure_gpu_data. - HPO -
references/hpo.md. Add theray_*blocks and a search space. Leavestrategyalone; itwinai swaps in the Ray strategy by itself when it detects a Ray cluster.
Gate 4: a single-node multi-GPU SLURM job completes, and MLflow contains a run with epoch times and GPU utilisation recorded.
No GPU node or no SLURM? sinfo or sbatch --version failing is the signal. Then Phase 3
is the end of the road here, by Rule 2: step 1 of this phase is still verifiable through Gate 3,
steps 2 to 4 are not. Report the plugin as working up to Gate 3, list Gate 4 as not run and
why, and offer to write the remaining config as unverified for someone with cluster access.
Publishing the model is not part of this chain and needs no cluster: it reads checkpoints,
not MLflow. Only when the user asks, read references/model-hub.md.
Phase 5 - Scale (opt-in)
Only on explicit request. This consumes real allocation and may outlive the session.
Read references/profiling-and-scalability.md. Set mode: scaling-test and scalability_nodes,
submit, wait for every job, then:
itwinai generate-scalability-report
Gate 5: the plots exist and are non-empty. Empty plots mean the chain in Phase 4 was broken - go back to step 1, do not reinterpret the plots.
Verification ladder
The gates above, as one list. Rungs 1-3 run anywhere. Rung 4 needs a GPU node and SLURM, rung 5 needs several nodes and is opt-in; where they cannot run, stop and say so (Rule 2).
- Plugin installs and imports (Gate 1, Gate 2)
itwinai sanity-check --torchpasses- One-epoch non-distributed run, loss decreases (Gate 3)
- Single-node multi-GPU SLURM job, metrics in MLflow (Gate 4)
- Multi-node scaling test, non-empty scalability report (Gate 5)
Never
- Never write a training loop from scratch. When the stock trainer does not fit, subclass
TorchTrainerand override the narrowest method that covers the difference: the base class is what keeps distributed training, logging and profiling working. - Never copy field lists out of a reference file into config without reading the owning class.
- Never submit a multi-node job without asking.
- Never claim a gate passed without running it.
版本历史
-
fc5b2a5
当前 2026-11-03 01:26
版本从 0.4.2 升级至 0.5.0
- 8ec704b 2026-10-19 08:24


