Why would you walk away just because one sentence.
Don't give such great deal about one sentence to your whole relationship.
In my knowledge, guys don't give such big deal about stuff that are, in their opinion, commonly known.
My bf and I had once casual conversation (we started as friends and still have conversations about random stuff) and he said, that for him if he said that he loves somebody, he didn't feel like he needed to repeat it. He sometimes wouldn't say it, but if he loved a girl he would just want to make her happy and smile and didn't find some sentence important for his feelings, she already knows them. Even he wouldn't feel comfortable to repeat it all the time cause it doesn't makes sense to him and feels like he has to repeat it because of girl and it's not him. He would feel like she trapped him to do something he doesn't want to.
If you really want to tell him that you love him, tell it casual and don't expect him to tell you that he loves you back or make drama i he doesn't. Just be normal and yourself.
If you practically ask him to tell you if he loves you or not, he'll probably do it just to make you happy, not because he feels it.
And if you need to hear him that he loves you to affirm your love, that's a different problem.
For end, my advice: please, STOP OBSESSING!
Forget "love you", forget sex and just enjoy your time together.