The original inhabitants of Gaither’s Plantation may be long in their graves, but if dozens of visitors are to be believed, the dead still walk the halls of this 159-year-old plantation home.

Mr. Gaither, who went on to play an important role in the 1961 Freedom Rides, died on Dec. 23 at his home in Prospect, Pa., north of Pittsburgh, his family said. He was 86. A little-sung catalyst ...