Joe Levin, the son of a lawyer, was born in Montgomery in 1943. While studying at the University of Alabama he witnessed the persecution of Melvin Meyer, the editor of the student newspaper, for arguing in favour on integration. This eventually resulted in the Ku Klux Klan burning a 12-foot cross in front of where Meyer was living. This experience made Levin a supporter of civil rights.

After serving in the United States Army, Levin worked as a lawyer in Montgomery. In 1971 Levin joined with Morris Dees and Julian Bond to establish the Southern Poverty Law Center (SPLC) in Montgomery, Alabama. A non-profit organization, the SPLC attempts to "combat hate, intolerance, and discrimination through education and litigation." The SPLC initially provided legal representation in individual cases dealing with racial discrimination.

As the SPLC legal director (1971-76) Levin worked on more than 50 major civil rights cases. This included segregation in schools and in public parks. He also played a leading role in the landmark sex discrimination case that resulted in the Supreme Court ruling against men obtaining preferential treatment in the military.

In 1976 Levin left the Southern Poverty Law Center to serve as Special Assistant to the Attorney General and Chief Counsel to the National Highway Traffic Safety Administration. In 1979 Levin became a lawyer in Washington but continued as president of the SPLC.