Thompson was born in Columbia, South Carolina and moved to Detroit, Michigan during his childhood. Thompson had to raise his siblings after his mother died, and he practiced saxophone fingerings on a broom handle before acquiring his first instrument. He joined Erskine Hawkins' band in 1942 upon graduating from high school.
After playing with the swing orchestras of Lionel Hampton, Don Redman, Billy Eckstine (alongside Dizzy Gillespie and Charlie Parker), Lucky Millinder, and Count Basie, he worked in rhythm and blues and then established a career in bebop and hard bop, working with Kenny Clarke, Miles Davis, Dizzy Gillespie and Milt Jackson.
