SINJAR, Iraq (AP) — One by one, the flags belonging to a patchwork of armed forces were lowered in a northern Iraqi town once brutalized by the Islamic State group. The territorial claims symbolized by each were replaced by the fluttering of just one: The Iraqi state’s.

The hoisting of the national flag in Sinjar, home to Iraq’s Yazidi religious minority, is the result of a deal months in the making for the federal government to restore order from a tangled web of paramilitaries, who sowed chaos in the district during the bedlam following liberation from IS three years ago.

This month, Iraq’s army deployed there for the first time since the 2003 fall of Saddam Hussein.

Lt. Imad Hasan hiked up a rocky ascent overlooking the deserted ruins of Sinjar’s old town, vacant since IS was dislodged. His gaze fell on a lookout on the other side of the mountain — the last, he said, that belongs to a local affiliate of an outlawed Kurdish guerrilla group, known as the PKK.

“We have problems with them,” he said. “Their leaders have agreed to withdraw, but some of their fighters have not.”

Sealing the deal was hard enough. Implementing it brings new problems. Critics say it will take more than a change of flags to cement rule of law in Sinjar.

The Yazidis, traumatized by the mass killing and enslavement that IS unleashed against them, have no trust in the Iraqi authorities they say abandoned them to the militants’ brutality. With the central government weak, they fear militias — including Iranian-backed Shiite factions — will gain sway over them.

The militias policing Sinjar the past three years are a mix. They include peshmerga fighters from Iraq’s Kurdish autonomy zone, as well as the PKK and its affiliate made up of local Yazidi fighters, called the Sinjar Resistance Units or YBS. There are also Yazidi units belonging to the Popular Mobilization Forces, an umbrella group of state-sanctioned paramilitaries created in 2014 to defeat IS.

There are signs of recovery of Sinjar. Its city center hummed with shoppers, merchants — and the odd Iraqi army tank. More of the 200,000 Yazidis displaced by the 2014 IS onslaught are coming back — some 21,600 returning between June to September, many times the rate of previous years.

But scratch the surface, and almost everyone harbors raw, unresolved trauma. Everyone vividly recalls the IS attack that murdered fathers and sons, enslaved thousands of women and sent survivors fleeing up Sinjar mountain.

In Sinjar’s market, a farmer, Zaidan Khalaf, introduced himself first by telling The Associated Press how many relatives he lost under IS: 18. Others in the market did the same.

“We lost our dignity,” he said.

Communities remain deeply divided and bitterly resentful of one another.

“What agreement?” scoffed Farzo Mato Sabo, an 86-year-old in the predominantly Yazidi village of Tal Binat, south of Sinjar. She and her three daughters were taken by IS militants and later saved by smugglers. Eleven of her family members are still unaccounted for.

“I lost everyone,” she sobbed. “Will it bring them back?”

Neighboring Tal Binat is the Sunni Arab village of Khailo.

“We used to be like brothers, but now the Yazidis stay away from us,” said a tribal elder, Sheikh Naif Ibrahim. “They can’t distinguish between civilians and IS members.”

Many Yazidis accuse local Sunni Arabs of supporting IS. Since the militants’ fall, Sunni Arabs have had frictions with Yazidi militias — and a number of Sunnis have been killed. At the same time, many Yazidis reject the Kurdish peshmerga, who consider the Sinjar area part of their domain.

“Seven flags ruled over us, you never knew who had power over you which day,” said Khalaf, the farmer.

The U.N. has focused on the return of displaced Yazidis, but this is not the only criterion for success, said Sajad Jiyad, a fellow at The Century Foundation. “It’s about services, schools, security and the ability to move around without being shaken down by various groups,” he said.

“This is a test for the effectiveness of post-war governance and post-war liberation,” he said. “Is the government prepared enough to allow the return to normalcy?”

The Iraqi military will secure the area for now, with other factions leaving their positions, although many remain in the Sinjar area. Under the plan, the Kurdish authority is to appoint a mayor — a prospect many Yazidis oppose — and local police are eventually to take over security, working under the government’s intelligence agency and National Security Adviser. The plan calls for 2,500 new security personnel to be hired locally.

Most Yazidi leaders and residents interviewed said they were irate the community was not consulted by the government in the making of the plan.

“We are the ones who sacrificed, lost our lives,” said Fahed Hamed, Sinjar’s district mayor. “We should have been the main interlocutors.”

“We want a force from our own. We don’t trust anyone.”

The force most trusted by locals is a faction the plan seeks to eject — the YBS, whose fighters are largely Sinjar Yazidis. While other forces retreated from the IS onslaught in 2014, many recall it was the YBS that fought to secure a safe route for civilians.

“They were the only ones who stayed to protect us,” said Sherko Khalaf, a Yazidi village mukhtar.

Despite protests by locals, negotiations led to the withdrawal of YBS from Sinjar’s city center.

YBS fighters interviewed said they expected to be subsumed as a unit of the Popular Mobilization Forces, providing them with much-needed political legitimacy. A portion of the 2,500-3,500 YBS fighters are already on the PMF payroll.

In theory, the plan calls for the PMF to end its presence in the city as well. To date, they are supporting forces and securing Sinjar's peripheries. But Khan Ali, the commander of the Laslish Brigades, a Yazidi unit of the group, told the AP, “The (PMF) will stay forever, we are kings over the heads of the security forces in Sinjar.”

That prospect has divided Yazidis. Some want Yazidi PMF factions included in the security arrangement. Others fear it will bring Sinjar under the influence of the Shiite Arab factions close to Iran that dominate the umbrella group.

“If the international community and central government don’t care about Sinjar, the PMF will take control,” one prominent Yazidi leader said, requesting anonymity to speak freely. “This is clear.”