WASHINGTON (AP) — Stepping up pressure, the nation's six largest veterans groups on Wednesday called on President Donald Trump to immediately fire Veterans Affairs Secretary Robert Wilkie over the mishandling of a congressional aide's allegation of a sexual assault at a VA hospital.

In a letter to the White House, the groups said they had lost all confidence in Wilkie's ability to lead the government's second-largest department following a blistering audit last week that found he had acted unprofessionally if not unethically in disparaging the aide. The groups pressed Trump to act in the last weeks of his administration since Wilkie had refused to accept responsibility and was refusing to resign.

“Sexual assault and harassment have no place in the VA or our society,” according to the letter from the American Legion, Veterans of Foreign Wars, Disabled American Veterans, AMVETS, Paralyzed Veterans of America and Vietnam Veterans of America.

“This is a tremendous breach of trust among veterans," they wrote.

The groups are adding pressure on Trump after lawmakers from both parties expressed concern about VA's leadership and at least a dozen veterans organizations — as well as House Speaker Nancy Pelosi, D-Calif., and House Majority Leader Steny Hoyer, D-Md. — last week called for Wilkie's resignation.

Wilkie has denied wrongdoing. In response to calls for his resignation, the VA says Wilkie “will continue to lead the department.”

The investigation by the Veterans Affairs’ inspector general concluded that Wilkie repeatedly sought to discredit Andrea Goldstein, a senior policy adviser to Democratic Rep. Mark Takano, who is chair of the House Veterans Affairs Committee, after she alleged in September 2019 that a man at the VA medical center in Washington, D.C., had physically assaulted her.

The inspector general found that Wilkie’s disparaging comments about Goldstein, a Navy veteran, as a repeat complainer as well as the overall “tone” he set influenced his staff to spread negative information about her while ignoring known problems of harassment at the facility.

Wilkie and other senior officials had declined to fully cooperate with the investigation by VA Inspector General Michael Missal. For that reason, Missal said he could not conclude whether Wilkie had violated government policies or laws, allegedly by personally digging into the woman’s past.

The case of Goldstein, who agreed to be publicly identified, was ultimately closed by the inspector general’s office and Justice Department earlier this year due to a lack of enough evidence to bring charges.

Wilkie is Trump’s second VA secretary after David Shulkin was fired in 2018.