I agree with what has been said... You're all young and rapid change happens all the time, especially in the teen years and early 20s. Not that it doesn't continue to happen, but it's common to completely change opinions or outlooks when you're first emerging from your parents' home(s) and venturing out on your own. This is a perfect opportunity for you and F to figure out if your relationship really is a life-long (or as close to it as is possible) one or not.
As for your questions...
I think just letting J know what you're feeling, what F thinks/feels towards it all, and what YOU would like is important. Asking him to respond to these three things will help you know where he is at. You also have to make sure you are clear about what you would like to happen ideally as opposed to what can happen realistically - especially if it means the difference between you being with F and you not being with F. If F's opinion is going to affect what you do with J (as it sounds like it will), I would encourage you and F to figure stuff out as much as possible before delving too deep with J.
Making F more comfortable really depends on him. You are already discussing everything which is going to continue to be important. Before doing ANYTHING further, take some time to read, let him process, and talk about what you (either or both of you) perceive as potential issues. Discussing any boundaries the two of you feel you need would help make sure everyone is on the same page. He has already admitted that the idea of you being physical with someone bothers him, so obviously jumping into having sex with J would NOT be making F comfortable with the shift. Easing into it by having you and F going on a date and ending the evening with a kiss and seeing how everyone feels afterward would be a good first step, I think. It sounds like F is interested in having an open relationship to some extent but has some natural concerns. Help him work through those while you both learn about various types of relationships and figure out what works for you. J has been around for a year and a half, I'm sure he can handle a little bit more time - especially if there is progress being made.
If J doesn't want anything else from you, why are you concerned you would "go for more?" I would think he would also care about your relationship with F since they are friends and would not put you in the situation to cheat unless you led him to believe that it was okay for things to happen between you two. Do you not trust that J or you would have that self control? If you doubt your control or his, I would definitely say you would need to not be alone with J. And I agree with Anne that alcohol or other things that lower inhibition need to be avoided.
I don't think being tempted to cheat is a reason to dump J. I think it is a definite sign that some examination and possible renegotiation needs to happen, but that doesn't mean the relationship has to end unless you find that the relationship (and J) aren't able to adapt to your new desires/needs.
So far it sounds like you're doing great - getting everything out in the open and trying to figure out what it is you're wanting. Just avoid any hasty actions that could jeopardize any of the relationship in play here (you and F, you and J, F and J's friendship) so that the minimal amount of anxiety occurs.