For us it was 3. A nice triangle for the physical part. We have talked about the possibility of more but we never got the chance to really try it. We did have and still have many more that we are very close to but the relationships have never grown past the really close friendship stages. Maybe one day though. We feel that if you add to many people the lines of communication start to break down.
With 3 people you have 6 different relationships going on, add one other and you will have 8 or more depending on how you look at it and the more you add the more complicated it gets. So we just decided that 3 to 4 was the prefect number for the core group.