Fort Jefferson, the largest all-masonry fort in the United States, was built between 1846 and 1875 to protect the nation's gateway to the Gulf of Mexico. During the Civil War, it was used as a Federal prison primarily for Union deserters, though in 1865 three of the Lincoln conspirators were imprisoned within its walls.