You are right - most casual fans can't see the forrest for the trees. You are wrong - most fans see a QB struggling and link it 100% to the QB. That is the concept of that phrase, not the other way around. You are unable to see the bigger picture, because you are only focusing on what you see immediately in front of you.