Y sounds so much like my husband! I went through the same things, for several years: asking to be complimented, to have more attention, to get the kind of romance and flirtation from him that I could see would be easily available from other men in my life.
Honestly what finally got through to him, I think, was the Love Languages book. He really understood at that point that I didn't feel loved, no matter how much he thought he was loving me, because he wasn't showing it in the way I needed it. Like you, I needed words of affection and quality time. He thought being a good provider was an obvious demonstration of his love. Maybe Y thinks he's showing you love because in his own way he is, and your asking for it in other ways feels unnecessary and frivolous to him. It's really amazing the difference between someone waking up and leaving the room, and someone first rolling over and saying "Good morning," isn't there?
I'm in agreement with everyone here that honesty is the only way to go. I think the more important thing to talk about with Y is your relationship with each other. I would hesitate to start or end anything with E until you've worked things out with Y. Once your marriage is the best it could be, you'll have a better idea of whether you want E for E, or just to fill the gaps.
Married to a monogamous man 15 yrs, mother of 2, dating C 3 yrs, and in a romantic friendship with L more than 20 yrs