Historically referred to as the Gray-Guilbeau House, the building is named for James Gray and French consul Francois Guilbeau who bought the house from Gray in 1853. The house saw many owners, including the Henshaw family, and was the last of the private dwellings in La Villita.
The Phelps family purchased the home in December of 1970 as their personal residence. Visitors to the home were enthralled with its historical significance and ambiance. Prompted by this the family voted to share the house with all, and the Fig Tree Restaurant was born.Mr. and Mrs. Frank Phelps opened the Fig Tree Restaurant on March 17, 1971, and the family has continuously operated the restaurant ever since.