The catalpa tree in Ellwood's front yard was a witness to the Battle of the Wilderness in 1864. Chatham Manor, across the Rappahannock River from Fredericksburg, also has a catalpa tree which is present in photographs taken during the Civil War. The William Jones and J. Horace Lacy families owned both Ellwood and Chatham during two separate periods.