Yosuga no Sora question!?

So I'm on episode 7 of "Yosuga no Sora" And I'm just SOOO CONFUSED! @ - @ From like episode 3 - 4 He's going out with Kazuha I think, 5 - 6 It's Akira, and 7 there are flashbacks of Haruka and Nao.....doing stuff, and Nao sewing on Sora's ons when Kazuha already did that for her in the earlier episodes. WHAT THE JEEZ IS GOING ON? D: < Is it something that I'll have to watch the entire anime to understand? PLEASE NO SPOILERS!!