By Scott Wong - 11-03-16 17:17 PM EDT
Chatter is growing louder on Capitol Hill that Paul Ryan's days as Speaker are numbered.
Four House Republicans, including a senior lawmaker close to leadership, told The Hill they expect Ryan to step down after Tuesday's elections, arguing that he faces a daunting path to the 218 votes he needs to win a full two-year term leading the House GOP.
Aides to the Wisconsin Republican insist he isn't going anywhere and that he's completely focused right now on protecting the GOP's majority in the lower chamber.
"He is running. The Speaker's only focus until Election Day is defeating Democrats and protecting our majority, and nothing else," Ryan spokeswoman AshLee Strong said.
But even some Ryan allies are conceding that the Speaker now finds himself in an untenable position after just a year on the job.
It's not just the usual Freedom Caucus members who are pushing for change at the top; some more mainstream Republicans from safe GOP districts could pull their support over Ryan's handling of GOP presidential nominee Donald Trump, lawmakers said.
The brash New York real estate mogul is enormously popular in many of these districts, but Ryan stopped actively supporting Trump after a recording surfaced of him talking in 2005 about groping and kissing women.
"Speculation is growing that Paul will not return," said one senior GOP lawmaker close to leadership.
The lawmaker noted that if the GOP majority shrinks, Ryan could lose next year's floor vote to remain Speaker with a relatively small number of GOP defections.
"Why would he put himself in a situation where as few as 10 dissident members or one ill-timed quote from Trump could put his future in jeopardy?"
The outcome of the presidential race is a major factor for Ryan.
If Trump loses narrowly to Democratic nominee Hillary Clinton, he and his die-hard loyalists will almost certainly try to pin some of the blame on Ryan.
"Those who talk to Paul say he is all in to stay Speaker," the lawmaker continued. "But if you talk to members from the South, many will struggle to vote for him - even though they like him - because their constituents are furious" over his treatment of Trump.
One conservative GOP lawmaker told The Hill last week that a "fresh start" might be needed in the House and that Ryan would probably "step aside."
"I'm still solidly in the camp that thinks Ryan will not run," the same lawmaker reiterated on Thursday.
GOP sources said Ryan helped himself this week when he announced that he cast an early ballot for Trump.
But many Republicans are still fuming over a House GOP conference call last month during which Ryan told his colleagues he could no longer defend or campaign with Trump after news outlets published the recording of his lewd comments.
Trump already took to Twitter to call Ryan a disloyal, "weak and ineffective" leader. On Thursday, Trump's running mate, Mike Pence, repeatedly declined to say whether he thought his close friend, Ryan, deserved another term as Speaker.
"If Trump wins, Ryan wins," said one Southern GOP lawmaker who is still supporting Ryan and believes he will run again. "I have been hit hard in my district by folks who know Paul and I are friends. He is probably more unfavorable in my district than [former Speaker] John Boehner was."
A Freedom Caucus member predicted that there is "a 25 percent chance Ryan is Speaker in the 115th Congress."
"His unfavorability among Republicans is around 68 percent," the Freedom Caucus member said of Ryan. "If Hillary wins, he will surely take a good share of the blame among Trump supporters.
"I personally just sense he really didn't want the job in the first place and won't want to go to the floor and risk a second vote, which I think would be more likely than not."
House Republicans have scheduled their closed-door internal leadership elections for one week after the general election, on Nov. 15. Ryan is expected to coast to the nomination for Speaker in that vote because he only would need a simple majority of the GOP conference.
It's the second vote, on Jan. 3, that really matters. Ryan will need 218 votes on the House floor - half plus one of the House's 435 members - to retain the Speaker's gavel.
Ten Republicans didn't support Ryan last time.
If Republicans lose between 10 and 20 seats they are predicted to this cycle, Ryan's margin for error will be even smaller compared with last year's vote, and he won't be able to survive many more defections.
The conventional wisdom has been that no one other than Ryan - a former Ways and Means Committee chairman and Mitt Romney's 2012 running mate - can even amass 218 votes in the GOP conference. But chatter about a Ryan resignation has sparked wild speculation on Capitol Hill about who might throw a hat in the ring if the unexpected occurs and the Speaker's race becomes a free-for-all.
Rep. Bill Flores (R-Texas), who is wrapping up his term as chairman of the 178-member Republican Study Committee (RSC), is looking for his next gig and flirted with a bid for the leadership post last year after then-Speaker John Boehner (R-Ohio) resigned mid-term and Majority Leader Kevin McCarthy (R-Calif.) unexpectedly dropped his bid to succeed Boehner.
In a brief interview, Flores predicted Ryan would run again and win reelection. But asked if he would look at another run for Speaker in the event Ryan quits, Flores replied: "I would definitely consider doing so."
Ryan's top three lieutenants - McCarthy, Majority Whip Steve Scalise (R-La.) and Conference Chair Cathy McMorris Rodgers (R-Wash.) - all would be flawed candidates. McCarthy quit the Speaker's race last year after failing to nail down 218 votes. Scalise gave a speech to a white supremacist group with ties to former Ku Klux Klan leader David Duke, a controversy that would be resurface if Scalise ran for Speaker.
And McMorris Rodgers, the highest-ranking GOP woman in Congress, failed to get much traction last year in her bid for majority leader, the No. 2 post. She dropped out of her race against Scalise and Budget Chairman Tom Price (R-Ga.).
Other members of Ryan's leadership team, including GOP policy committee chairman Luke Messer (R-Ind.) or GOP campaign chairman Greg Walden (R-Ore.), have won conference-wide elections and could be seen as compromise candidates, similar to Denny Hastert, the chief deputy whip who rose to Speaker in the wake of Newt Gingrich's resignation in 1998. Messer was freshman class president and has pushed for rules changes to empower rank-and-file members. As the House GOP's campaigns chief, Walden has campaigned and raised cash for many of his colleagues, and built up a lot of political capital in the process.
Other names being tossed around include Rep. Trey Gowdy (R-S.C.), the enormously popular chairman of the special committee that investigated the Benghazi attacks; Rep. Marsha Blackburn (R-Tenn.), who leads the special panel investigating Planned Parenthood; and Rep. Peter Roskam (R-Ill.), a former member of leadership.
A number of committee chairmen, including several from Texas, could take a look at a possible Speaker's race as well. Agriculture Committee Chairman Mike Conaway (R-Texas) weighed a bid for Speaker last October, as did Homeland Security Committee Chairman Michael McCaul (R-Texas). The ambitious McCaul is eyeing a possible primary challenge to Sen. Ted Cruz (R-Texas), who lost the presidential nomination to Trump this year.
Two former RSC chairmen, Tom Price and Financial Services Committee Chairman Jeb Hensarling (R-Texas), are other popular lawmakers who have demonstrated that they have broad support in the 246-member conference.
"As is well known, Chairman Hensarling is a good friend and strong supporter of Speaker Ryan. He looks forward to continuing to work with Speaker Ryan next Congress to advance conservative Republican policies," said Hensarling spokeswoman Sarah Rozier.
Ryan allies have pointed out that the wonky, 46-year-old father of three didn't want the Speaker's job in the first place. And he's certainly not going to submit to a bunch of demands just to hold on to power.
That's prompted one GOP lawmaker to come up with an unorthodox solution: have Ryan swap roles with current Ways and Means Committee Chairman Kevin Brady (R-Texas). After all, Ryan has made no secret that chairing the Ways and Means panel is his dream job, and Brady would have a hard time turning down the Speakership.
With no obvious successor waiting in the wings, the next Speaker will have to put together a broad coalition of supporters. That could include groups like big-state delegations, the freshman and sophomore classes, committee alliances, and political caucuses like the centrist Tuesday Group.
"No one is positioned to plant a flag and go get it," said the Republican lawmaker close to leadership. "It would not be solved quickly."