By 1954, the Jordanaires were singing behind artists such as Elton Britt, Red Foley, and Jimmy Wakely. That year they appeared on Eddy Arnold's television show, but didn't get their big break until Elvis Presley, a longtime fan, invited the group to back him after receiving a major recording contract from RCA Victor. When Elvis became a star, he honored his promise to keep them as his backup singers, and they worked with him until 1970, appearing in many of his films and on his gospel recordings. In 1954, Culley Holt became ill and was replaced by Hugh Jarrett; he left in 1958 and was replaced by Ray Walker.

When not backing Elvis, the Jordanaires were busy making their own mark in country music. Neal Matthews was a talented arranger and was responsible for Jim Reeves' massive hit "Four Walls" (1957). In 1959, the Jordanaires began working with Patsy Cline, and also devised the Nashville number system of chords that is still widely used in recording studios and performances. They also recorded their own gospel and country albums. It was the Jordanaires who provided the main impetus for the formation of the American Federation of Television and Radio Artists/Screen Actors' Guild in Nashville. They also established Nashville's commercial jingle market, which helped singers like Janie Fricke and Judy Rodman get their start.