William Trousdale married Mary Bugg in 1827 and they had seven children, five of whom survived to adulthood --- Charles, Julius, Valeria, Maria Lousia, and Frances. Valeria married General James Lafferty of Bean Station (near Knoxville) in 1854. Lafferty served under William in the Mexican War. Valeria died in 1860, leaving three children. Maria Lousia married Benjamin Franklin Allen and they had five children. Frances (Fannie) married John Bell Peyton, the son of Congressman Balie Peyton. The Peyton home was located across the street.

Charles “Cappy” was the eldest son and he married Catherine Odom in 1860. He was a successful businessman in Gallatin and he served in the Confederate Army. At the Battle of Chickamauga in 1863, he was severely wounded and lost a leg. He and Catherine had a daughter, Kate, in 1866. Within a month of Kate’s birth, Catherine died. Charles devoted the rest of his life to his daughter, whom he sent to New Orleans and France to be educated. Kate is known to have had a beau, William Allen, for many years but she would not marry him until her childbearing years had passed. Charles died in 1900 and Kate built a home on Coles Ferry Pike that became known as Langley Hall. It was the largest frame house in the region, but it was torn down in 1965.

Julius also served in the Confederate Army and was wounded at Shiloh in 1862. He continued to serve in the Quartermaster Corps in Knoxville before he was captured in 1864 and taken to Camp Chase, Maryland. After the war, he served as a lawyer and a member of the Tennessee State Legislature. He married 26-year-old Annie Berry in 1880 at the age of 40. She was the daughter of William Wells Berry, who started the first pharmacy in Nashville. When Mary Bugg Trousdale died in 1880, Annie became the mistress of the house. Annie and Julius had one daughter, Mary, who was born in 1881 and named after her grandmother. Mary returned from school in New York in 1899 and died of fever that August. Three weeks later, Julius died as well. Upon Charles’ death in January of 1900, a grief-stricken Annie Berry deeded Trousdale Place to the Clark Chapter 13 of the UDC to be used as a museum to honor soldiers of all wars.