FORTUNE — When I reviewed Steve Ballmer’s umpteenth structural reorganization of Microsoft last month — was it only last month? — I noted that the key missing ingredient to making the construct work was leadership. A re-jiggered Apple without Steve Jobs doing the jiggering never would have been successful. It took the man who understood the culture of the organization, was willing to make the painful cuts as well as to promote what was working, and who commanded the respect of the employees, to succeed at Apple.
Such a man exists for Microsoft (MSFT), of course, but according to conventional wisdom as well as comments the man has made, Bill Gates simply isn’t interested.
So who could run Microsoft? Let’s start with the names being tossed about. The first bunch is ex-Microsoft executives who lead other organizations. Paul Maritz was CEO of VMware (VMW) and currently runs a company called Pivotal. Steve Sinofsky has been kicking around academia and recently signed on for a stint with Andreessen Horowitz. Jeff Raikes runs the Bill & Melinda Gates Foundation. I’d rule all of them out. They ran portions of Microsoft, an organization that needs a cultural re-boot, in other eras. They are too tied to the old Microsoft, despite that old Microsoft having been in its glory days. (As a problematic founder, Jobs missed most of Apple’s (AAPL) decline; he arguably was around only for all but its best days.)
There’s a current crop of Microsoft names being mentioned too. Tony Bates is the most credible of this list. A legitimate technologist and manager from his Cisco (CSCO) days, he presided over Skype’s rejuvenation and has stuck around for a couple of years at Microsoft. Bates counts as an outsider, but he hasn’t run anything nearly the size and scope of Microsoft. Julie Larson-Green and Satya Nadella are top Microsoft executives in important roles. They have 41 years of Microsoft experience between them, and that just won’t cut it for the future. Next.
MORE: America’s most hated device: The cable box
The real fun is in predicting outsiders. Marc Benioff and Jeff Bezos would be inspired choices if Microsoft were to buy their companies. Don’t count on it. An ex-Microsoftie suggests a quasi-outsider, Yammer CEO David Sacks, a card-carrying member of the PayPal mafia. It’s a fine idea, but for the absence of any evidence the entrepreneurial Sacks could handle the bureaucratic nightmare of running such massive organization.
So who is the executive with software and hardware chops and the experience with fractious, global, diverse organizations? An executive with a proven track record managing technologists and sales people? That person, surprisingly, is 62-year-old Sam Palmisano, the recently retired CEO of IBM (IBM). Think about it. Palmisano presided over the stunningly successful marketing gambit called “Smarter Planet,” a campaign that unified all of IBM behind a single sales concept. He accelerated IBM’s move into software, notably emphasizing the company’s embrace of the Linux open-source software platform. That’s a move that infuriated Microsoft. What better credentials could an outsider CEO have than having humiliated and beaten his new charges?
Yes, Palmisano is a sales guy, like Ballmer. But he also is a tech-industry jack-of-all-trades, having run all of IBM’s major divisions, as described in this 2011 profile by Fortune’s Jessi Hempel.
For a company like Microsoft, fixing the culture is everything, as the Wall Street Journal astutely asserted Monday. Palmisano adroitly enhanced the IBM culture Lou Gerstner left him. He also unsentimentally jettisoned the parts of the company that no longer mattered to IBM, including its PC unit, now owned by Lenovo. Such cool calculation could benefit Microsoft’s disparate parts, many of which are begging to be split off from the mother ship.
Microsoft’s stock jumped on news that Steve Ballmer is stepping down. Were Sam Palmisano to get the nod, it’d jump more.