NEW ORLEANS - On the cusp of trial over the catastrophic 2010 oil spill in the Gulf of Mexico, phalanxes of lawyers, executives and public officials have spent the waning days in settlement talks. Holed up in small groups inside law offices, war rooms and hotel suites in New Orleans and Washington, they are trying to put a number on what BP and its partners in the doomed Macondo well project should pay to make up for the worst offshore spill in U.S. history.

It is a complex equation, and the answer is proving elusive.

The federal government, Gulf states, plaintiffs' attorneys, BP PLC, rig owner Transocean Ltd. and cementer Halliburton Energy Services Inc. have been in simultaneous and separate negotiations in New Orleans, according to a person with direct knowledge of the talks and others who had been briefed on them.

Trial is set for Monday, and by Friday, no deal had been reached, several people familiar with the negotiations told The Associated Press on condition of anonymity. The biggest stumbling block appeared to be the sheer size and sprawling uncertainty over the unprecedented dollar amounts at stake.

Financial analysts estimate BP's potential settlement payout at $15 billion to roughly $30 billion. The company itself estimated it would cost about $41 billion in the weeks after the explosion to account for all of its costs, including cleanup, compensating businesses, and paying fines and ecological damage.

"This one is off the charts in terms of size and significance," said Eric Schaeffer, the director of the Environmental Integrity Project in Washington and former head of the Environmental Protection Agency's Office of Regulatory Enforcement.

BP has to weigh its chances of getting off cheaper by piecing together a sweeping settlement or put its fate in the hands of one man, a federal judge who will hear testimony in lieu of a jury.

If the judge sides with plaintiffs on the amount of oil spilled and determines BP was grossly negligent, the company conceivably could face up to $52 billion in environmental fines and compensation alone, according to an AP analysis.

While such a scenario is unlikely, it illustrates the broad range and staggering sums at play.

No matter what, the case is all but guaranteed to set records as the most expensive environmental disaster in history, far surpassing the Exxon Valdez disaster in 1989. Exxon ultimately settled with the U.S. government for $1 billion, which would be about $1.8 billion today.

If BP settles, it's almost certain to dwarf previous deals the U.S. has reached with corporate offenders in any industry. That record now stands at $2.3 billion against Pfizer Inc. in 2009 to settle claims over the painkiller Bextra, according to the Justice Department.

And once the civil case is resolved, depending on the scope of any settlement, BP still could face criminal fines; penalties for violations of oil pollution, clean water and wildlife protection laws; and still-pending economic losses due to the partial shutdown of the Gulf. Morgan Stanley analysts estimated criminal fines would come in between $5 billion and $15 billion in any eventual settlement.

Robert Wiygul, an environmental lawyer in New Orleans who represents spill plaintiffs but is not involved in the settlement talks, said putting a dollar figure on what is the right sum for BP to pay is extremely difficult.

"There is going to be a lot of voodoo there," he said.

The bill will be commensurate to the magnitude of the disaster: An epic engineering failure that highlighted the dangers of drilling in extreme conditions miles from shore and miles under water.

The April 20, 2010, blowout of BP's deepwater Macondo well killed 11 workers and injured 17. The burning drilling rig Deepwater Horizon toppled and sank to the Gulf floor, where it sits today.

It took engineers 85 days to permanently cap the well. By then, more than 200 million gallons of oil leaked from the well and had covered much of the northern half of the Gulf of Mexico - endangering fisheries, killing marine life and shutting down offshore oil drilling operations.

About 900 miles of shoreline were fouled and beaches were closed for months. The spill forced President Barack Obama in June 2010 to make his first Oval Office speech, in which he called the BP spill "the worst environmental disaster the nation has ever faced."

Under the Clean Water Act, which is designed to punish companies and prevent future spills, a polluter pays a minimum of $1,100 per barrel of spilled oil; the fines nearly quadruple for companies found guilty of grossly negligent behavior. Under this statute, BP could owe $5 billion to $21 billion. Transocean and Anadarko Petroleum Corp., a minority owner of the Macondo well, also face paying hefty fines.

One of the biggest questions facing U.S. District Judge Carl Barbier, a maritime law expert presiding over the trial, will be to determine if BP was guilty of gross negligence.

Under the Oil Pollution Act, companies must pay to restore what they fouled. Based on criteria from what Exxon paid after the 1989 Exxon Valdez spill in Alaska, BP could pay about $31 billion, or $148 per gallon, to cover the ecosystem damage to the Gulf. Exxon paid $900 million for 11 million gallons of spilled oil, or about $81 per gallon. Adjusted for inflation, that's $148 per gallon.

Experts said Barbier will weigh a number of factors in determining what BP should pay to restore damaged natural resources, and BP's liability under the Oil Pollution Act could be much higher or much lower than what Exxon paid per gallon.

There are several arguments that BP is likely to make. The company could say the amount it pays should be much lower because it has spent billions on cleanup already and provided $1 billion for early ecosystem restoration. BP may say the spill's effects were minimized by the Gulf's warm waters, oil-eating bacteria and other factors.

The Gulf has been soiled by past spills and natural oil seeps, so the oil giant could say it's too hard to pinpoint what is BP damage and what isn't, said Mark Davis, a Tulane University law professor who specializes in water resources.

State and federal lawyers are likely to argue that the damage was extensive and that the Gulf's marine environment is more varied and rich than even that of Prince William Sound, where the Exxon Valdez went aground.

Beyond that, there are more than 110,000 people and businesses - among them large fishing and hotel operations - who have not settled with BP and have outstanding claims against the company. Technically, people have until April 20, 2013, to file claims against BP, which committed to pay $20 billion to cover damage claims and so far has spent about $7 billion.

What makes this trial so good for plaintiffs - and a nightmare for BP, Halliburton and Transocean - is that the spill was a chronicle of corporate failures. Federal investigators have concluded cost-cutting by BP and shoddy work by all three companies caused the blowout.

"It's the perfect case for plaintiffs' lawyers," said Blaine LeCesne, a tort law specialist at Loyola University New Orleans who's analyzed the case. "They have everything to gain by going to trial."

While the settlement haggling stretches through the weekend, the hundreds of lawyers who have come to New Orleans are primed for battle.

Garret Graves, an aide to Louisiana Gov. Bobby Jindal and a member of a federal and state council assessing damage from the spill, was adamant that any last-minute settlement in the price range of $20 billion would let BP off too easily.

"We're not going to sell short the citizens and we're not going to let BP walk away," Graves said.

Mike Brock, a BP trial lawyer, said BP was ready to prove "that no single action, person or party was the sole cause of the blowout."