While jealousy and insecurity may indeed be an issue, I would say that regardless of anything else, the lack of quality time is the one that predominates. You two want very different things. He does not want a defined relationship and is satisfied with seeing you infrequently, whereas you want a guy who is there for you. Bottom line, he is not that guy.
As for the insecurity / jealousy issue, it might be something you could overcome if your relationship with him was not affected by his other pursuits - but it is in terms of time, if nothing else.
That said, in polyamory, you don't have to break up with him to date someone else. Find someone who can be there for you, someone who can meet the needs he is not meeting.