LoRA is the biggest hype train

It can't teach the model anything new, it just helps it focus on what it already knows. I like to think of it as a function that can change the "form" of the output, but not add intelligence to it. Your base model has to already be good at the task you're trying to finetune it for.