Whitehead joined Goldman Sachs in 1947 and worked his way to the highest rung of its corporate ladder before leaving after 38 years to become a deputy secretary of state under U.S. President Ronald Reagan.
He was a chairman of the Federal Reserve Bank of New York and a member of the board of the New York Stock Exchange. Active in political and philanthropic circles, he also served as chairman of the Lower Manhattan Development Corp after the World Trade Center was destroyed during the Sept. 11, 2001 attacks.
"We grieve the loss of John Whitehead and honor his achievements and contributions in service to his country and Goldman Sachs," Goldman chief executive Lloyd Blankfein said in a statement.
"He was a man of enormous grace and integrity and his legacy will endure in the institutions he led and in the lives of those he cared for and mentored."
Whitehead joined Goldman with a starting salary of $3,600 a year when it had fewer than 300 employees and its service offerings were almost exclusively in commercial paper. The firm today has 34,000 workers and $869 billion in assets.
Over time he identified new business lines including such things as mergers and acquisitions and initial public offerings, according to a biographical account provided to Harvard Business School.
"I remember assigning one young fellow, who later became an important partner, to keep records about companies that might want to merge with a larger company or might be interested in acquisitions," he said.
"And then we tried to match them up. We were doing this before anybody thought that there might be business for investment bankers in the merger field."
Goldman pioneered two other innovation in investment banking: the idea of soliciting business and the handling of public offerings, Whitehead said in the Harvard biography.
He was born in Evanston, Illinois on April 2, 1922 and grew up in Montclair, New Jersey, during the Great Depression, when he remembered "scrimping and saving" and eating a lot of macaroni and cheese, fish cakes, but not much meat, he said in the Harvard account.