# When fine-tuning an LLM, how do you decide which layers/modules to train before actually running the experiment? — retained record

> Retained reddit record from reddit:deeplearning.

Canonical: https://highsignal.app/data/records/8232cb9b7336eb4f

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:deeplearning)  published 2026-09-21 04:20 ingested 2026-09-22 02:36

[original source ↗](https://www.reddit.com/r/deeplearning/comments/1wm2ilj/when_finetuning_an_llm_how_do_you_decide_which/)

## retained text



```
For example, how do you determine whether to fine-tune:only adapters/LoRA

* specific transformer layers
* input/projector layers
* deeper/middle layers
* or the full model?

Are there reliable diagnostics, probing methods, gradient analysis, ablations, or small-scale tests that can tell you **where the bottleneck is** before doing a full training run and only finding out afterward that the chosen layers didn’t help?

Curious how people make this decision in practice.
```



document fetched 2026-09-22 00:23 · canonical https://reddit.com/r/deeplearning/comments/1wm2ilj/when_finetuning_an_llm_how_do_you_decide_which
