A House With Two Occupants Sometimes One Rarely Three

Riddle: A house with two occupants, sometimes one, rarely three. Break the walls, eat the borders, then throw away me. What am I?

Answer: Peanut