The label was founded in 1947 by Ahmet Ertegün and Herb Abramson. Upon its creation, Atlantic was principally a jazz and R&B label, signing Ray Charles from 1952-1959, though it also released some country western recordings as well. In the early fifties Ahmet was joined by Jerry Wexler and then Nesuhi Ertegün. From February 7, 1955 Nesuhi headed the label's jazz division and was responsible for major signings such as Charles Mingus and John Coltrane; later Joel Dorn filled this position. Although it began as an independent record company, it became a major player in the music business in the 1960s, with mainstream pop signings like Sonny and Cher. Competing record labels included Columbia Records and RCA Records.

The engineer, and later producer, Tom Dowd headed Atlantic's engineering department. Several sub-labels have been created or acquired since then. Atco Records was started in 1955 by Herb Abramson. Spark Records (the record label of Jerry Leiber and Mike Stoller) was purchased in November 1955. Others including Lava Records and 143 Records became part of the Atlantic group. In 1960, Wexler began a distribution relationship between Atlantic and Memphis-based Stax Records. The association with Stax ended in 1968. Atlantic was acquired by Warner Bros.-Seven Arts in 1967. It is currently a large part of the Warner Music Group, a former division of media conglomerate Time Warner that was sold to a group of investors in 2004 for $2.6 billion. From 1968-1973 rock band Led Zeppelin had a deal with Atlantic Records, then formed their own Atlantic distributed label, Swan Song Records.

In May of 1988, the label held a 40th Anniversary concert, broadcast on HBO. This 11-hour concert featured performances by a large number of their artists and included reunions of some rock legends like Led Zeppelin and Crosby, Stills, and Nash (being David Crosby's first full band performance since being released from prison).

In the early 1990s, Atlantic owned 50% of Interscope Records, which released notable gangsta rap titles — many in conjunction with Death Row Records. Pressure from activist groups opposed to gangsta rap, however, later led to parent company Time Warner's decision to sell Atlantic's stake in the label in 1995.

The label has also a number of deals with independents such as Must Destroy (which brought Goldie Lookin' Chain and The Darkness into the label) and VP Records in Jamaica, home to reggae artists such as Sean Paul.

Craig Kallman is currently Chairman of Atlantic Records. Ahmet Ertegün served as "Founding Chairman" until his death on December 14, 2006 at age 83.

In 2006, the label denied "Weird Al" Yankovic permission to release a parody of James Blunt's "You're Beautiful", claiming that it was "too early" in Blunt's career, and they didn't want Blunt to become a one-hit wonder. Later, in 2007, the label celebrated its 60th anniversary with the May 2 PBS broadcast of the American Masters documentary Atlantic Records: The House that Ahmet Built and the simultaneous Starbucks Entertainment CD release of Atlantic 60th Anniversary: R&B Classics Chosen By Ahmet Ertegun.