Ehrling was born in Malmö, Sweden. From the age of 18 he attended the Royal Swedish Academy of Music in Stockholm. At the academy he studied the violin, organ, and piano as well as conducting. During World War II, he studied under both Karl Böhm and Albert Wolff.
He made his public debut as a conductor with the Royal Stockholm Philharmonic in 1950, conducting Igor Stravinsky's "The Rite of Spring" from memory. In 1953 Ehrling was named the music director of the Royal Swedish Opera, a post he held until 1960. In the early 1950s Ehrling recorded the first complete set of Sibelius symphonies with the Stockholm orchestra. In 1959 Ehrling took the production of Aniara, composed by Karl-Birger Blomdahl, to the Edinburgh International Festival.
Ehrling's tenure with the Swedish Royal Opera ended in bitterness. He resigned his post and departed for the United States after he was asked to amend, and apologise for, his robust leadership style. In 1963 Ehrling replaced the departing Paul Paray as the principal conductor of the Detroit Symphony Orchestra.
Ehrling also taught at the Juilliard School of Music between 1973 and 1987. He died in New York, where he had lived since the 1970s. He was married to a former Stockholm opera ballerina, Gunnel Lindgren. They had two daughters.
Maestro Ehrling conducted nearly 700 works, including 24 world premieres, and helped inaugurate the Meadow Brook Summer Music Festival. In 1973, he made his Metropolitan Opera debut, where he conducted 12 different operas. He led 55 orchestras and ensembles in North and South America, and countless orchestras around the world, during his celebrated five decade career.
Ehrling was one of the last conductors to know both Stravinsky and Sibelius, personally. When he discovered mistakes in their manuscripts, they were immediately informed.
In 1987 Ehrling participated in the documentary A Woman Is a Risky Bet: Six Orchestra Conductors directed by Christina Olofson where he comments on the conservative attitudes towards women in the world of classical music.