Yongah brawlers not boat people: Dutton

Immigration Minister Peter Dutton says violence won't be tolerated in detention centres after a brawl at Western Australia's Yongah Hill facility.

The federal government has confirmed detainees forced their way into a compound on Tuesday, with union United Voice saying the fracas was sparked by a "group bashing of one targeted detainee".

It's understood four guards needed to be assessed in hospital for injuries, including a dislocated finger and a concussion, but they were all released by Wednesday morning and continue to have their health monitored.

"Keeping people safe is our first priority," a Serco spokesman said.

"We are supporting our staff, who acted immediately to de-escalate this incident.

"They do an excellent job in what can often be difficult and challenging circumstances."

About 50 detainees were involved in the incident.

The minister said an investigation was underway and it was up to police to press charges.

"We've been very clear we won't tolerate this sort of behaviour within the detention centre network," Mr Dutton.

"We do have a number of people who are there on 501 cancellations so they're not boat people, they're not refugees or asylum seekers.

"These are people who in many cases are hardened criminals and they are on a pathway to return to their country of origin."

United Voice officials will meet members at the centre for a full debriefing at midday after their scheduled appointment on Tuesday was cancelled because of the incident.

WA opposition corrective services spokesman Paul Papalia said there was clearly a "real problem" at Yongah Hill - and probably other detention centres - with criminals, including visa overstayers, living alongside genuine asylum seekers.

"That doesn't appear to be going well," Mr Papalia told AAP.

"It's a dangerous, volatile situation and the staff are suffering.

"The safety of the staff should be paramount."