Ayer was born near Barnwell, South Carolina, and educated in the common schools. He attended law school and passed his bar exam. After establishing a successful practice, he served in the state legislature from 1848 until 1856. He married Anna Patterson and raised a family.
An ardent supporter of states rights, Ayer signed the ordinance of secession and represented South Carolina in the Confederate Congress from 1862 until the war's end in 1865.