Songfacts®:

• In George Gershwin's final months of life in Hollywood, it was easy for his friends to see that he was unhappy. He had always had assorted health issues, so it was not unusual that he was now complaining of headaches. Friends speculated that it was his relentless work schedule. Others suggested it was due to his unrequited love for fashion model turned actress, Paulette Goddard. The true reason for his complaints, it would later be learned, is that he was dying from a brain tumor. However, that did not mean that Goddard was not on his mind.
  
  "They Can't Take That Away From Me" tells of lovers who cannot be separated in spirit, even if they are apart: "The way you wear your hat, the way you sip your tea. No, no, they can't take that away from me." It was no secret that Gershwin was infatuated with the beautiful actress, who would go on to become one of the glamour girls of the silver screen in the 1940s. The problem was, according to Gershwin, she was secretly married to one of the most popular entertainers in the world, Charlie Chaplin. The legality of that marriage is still debated today, but, nonetheless, she was taken. Reports are that Chaplin discovered the blossoming affair when he had Gershwin and Goddard followed. With his brother, Ira, supplying the lyrics, "They Can't Take That Away from Me" is Gershwin's ode to Goddard.
• The song made its debut in the 1937 film musical Shall We Dance, starring Fred Astaire and Ginger Rogers. Critics were not enamored with the movie's thin plot and Gershwin was even less thrilled with the way the song was presented in the film, which featured only one verse and one rendition of the chorus. Still, Astaire and Rogers had enough starpower to make the movie – and the song – a hit. Astaire and the Johnny Green Orchestra took it to #1 for 10 weeks in 1937 and Ozzie Nelson, Tommy Dorsey, and Billie Holliday also had hits with the song that year. Astaire and Rogers revived it 12 years later in MGM's Barkleys of Broadway.
  
  Despite the motivation for the song, Gershwin intended for it to be a light-hearted moment in the film when Astaire's character, Peter, and Rogers' character, Linda, are about to divorce and separate. Gershwin succeeds because, for a breakup song, it is far from somber; Peter sings to Linda, "Our romance won't end on a sorrowful note."
• Gershwin did not live to reap the rewards of the song's success. Two months after Shall We Dance was released, he died from the tumor that had been causing his headaches. "They Can't Take That Away from Me" became the only Gershwin song to be nominated for an Academy Award.
• The song was famously recorded in 1962 by Frank Sinatra with his trademark swing feel. His version can be found on the album Sinatra and Swingin' Brass.
• Other movies where the song has appeared include Mr. Holland's Opus (1995) where it featured as part of a high school Gershwin review and also as one of the Broadway tunes included by Kenneth Branagh in his 2000 film adaptation of Love's Labour's Lost.