Krauss was born in Vienna to Clementine Krauss, then a 15-year-old dancer in the Vienna Imperial Opera Ballet, later a leading actress and operetta singer.
Krauss sang in the Hofkapelle (Imperial Choir) as a Vienna Choir Boy. He graduated from the Vienna Conservatory in 1912. He was then appointed chorus master in the Brno Theater, where he made his conducting debut in 1913.
Krauss made the rounds of regional centers, conducting in Riga (1913-1914), Nuremberg (1915) and Stettin (1916-1921). He then returned to Austria as director of the opera and symphony concerts in Graz. In 1922, he joined the conducting staff of the Vienna State Opera.
He guest-conducted in the United States in 1929, with the Philadelphia Orchestra and New York Philharmonic. Also in 1929, he was appointed director of the Vienna State Opera. He was a regular conductor at the Salzburg Festival from 1926 to 1934.
Krauss gave up his Vienna positions in 1933-34 to direct the Berlin State Opera in 1935 after Erich Kleiber had resigned in protest against Nazi policies. Krauss's own position on Nazism was unclear, although he did enjoy a close relationship with Nazi official Alfred Frauenfeld and it has been claimed that he sought Nazi Party membership in 1933. However, throughout the 1930s he and his wife were involved in helping Jews escape from Germany.
In 1937, he was appointed Intendant of the National Theatre Munich. He became a close friend of Strauss, and even wrote the libretto for his opera Capriccio. He also conducted the premieres of Strauss's operas Friedenstag and Die Liebe der Danae.
After the Munich opera house had been destroyed by Allied bombing, Krauss returned to conduct the Vienna Philharmonic in 1944-45 until it ceased activities shortly before the end of World War II. After the war, Allied officials investigated his career and forbade him from appearing in public until 1947, when it emerged that he had helped Jews escape from the Third Reich. Krauss then resumed conducting many of the Vienna Philharmonic's concerts.
He conducted opera at Covent Garden in London starting in 1951, and at the 1953 Bayreuth Festival.
He died in 1954 while on tour with the Vienna Philharmonic in Mexico City, and is buried alongside his wife, singer Viorica Ursuleac, who died in 1985, in Ehrwald, Austria.