Rick Hall, who produced dozens of hits as the owner of FAME Studios in Muscie Shoals, Ala., has died. He was 85.

Judy Hood, the wife of bassist David Hood and the chairperson of the Muscle Shoals Music Association, told Billboard that Hall died of cancer.

“I am sorry to hear about the passing of the ‘Father of Muscle Shoals Music,’ Rick Hall,” tweeted Alabama governor Kay Ivey. “He was truly a music legend and one of Alabama’s stars. He will surely be missed.”

As told in the 2013 documentary Muscle Shoals, Hall was born into poverty on Jan. 31, 1932 in Tishomingo County, Miss. He suffered a crushing blow when he was five when his three-year-old brother died. This split apart his family, with his mother becoming a prostitute.

“I carried that shame throughout my life,” he told No Depression. “It turned me into a rascal of sorts, and I became very hardened and determined. My determination made me a tough businessman and I was very hard to say no to. All of this helped me become a great record producer.”

According to Rolling Stone, he learned how to play guitar, mandolin and violin, playing in local country and R&B bands. But in the late ’50s he found himself as a songwriter, penning Roy Orbison’s “Sweet and Innocent” with Billy Sherrill. The duo, with Tom Stafford, formed the publishing house Florence Alabama Music Enterprises (FAME) in 1959. But they split with Hall, who retained the rights to the name, and he built FAME Studios in nearby Muscle Shoals.

He quickly had a hit in 1961 with Arthur Alexander’s “You Better Move On,” which was covered by the Rolling Stones. This allowed him to build a bigger studio and he soon put together a crack house band known as the Swampers — Hood, Barry Beckett (piano), Jimmy Johnson (guitar) and Roger Hawkins (drums) with Spooner Oldham (organ) and a horn section — all of whom were white, but fluent in soul music to the point where African American singers were surprised to discover who made the music when they arrived at the studio.

After Percy Sledge had a No. 1 hit in 1966 with “When a Man Loves a Woman,” cut at FAME, Jerry Wexler at Atlantic began bringing his artists to the studio. Wanting in on the action was a young guitarist, Duane Allman, who camped out in FAME’s parking lot until Hall gave him a shot, making his mark on Wilson Pickett’s cover of “Hey Jude” and Aretha Franklin’s cover of “The Weight.”

But by 1969, Hall and Wexler were feuding, and when Hall signed a distribution deal with Capitol, Wexler took the Swampers and helped fund the Muscle Shoals Sound Studio, which they owned, across town. The split prompted Hall to return to his country roots, and many country and crossover hits over the next few decades were tracked within its walls. Gregg Allman chose FAME for his final record, Southern Blood.

Hall published his memoir, The Man from Muscle Shoals: My Journey from Shame to Fame, in 2015. He is survived by his wife, Linda, and sons, Rick Jr., Mark and Rodney.

Rest in peace, Rick!