She was born Alice Elvira Freeman in Colesville, New York and brought up in Windsor, New York. Her parents both came from well-to-do families with interests in lumber, dairy farming and land. Alice was born a farmer's daughter but her father knew there was no future for him in farming, so he left the family to take care of the farm while he gained further education and became a doctor. He enrolled in medical school in 1861 and completed the M.D. in 1864.