Howard yelled at by angry union protesters

Former prime minister John Howard has overcome the shock of stumbling across a rally of angry union members protesting penalty cuts in Sydney.

Mr Howard needed to be "rescued by police" after inadvertently coming across the Sydney march protesting cuts to penalty rates and the return of the Australian Building and Construction Commission on Thursday

The former Liberal leader had been attending a conference in the city when he walked out on to the street to find dozens of people shouting, booing him, holding up flags and making profane gestures.

Footage on social media shows him having to walk several hundred metres along Pitt St, flanked by a number of NSW policemen, as noisy protesters repeatedly chant "the workers, united, will never be defeated".

"I condemn CFMEU's thuggish harassment of John Howard today. A reminder of why ABCC is needed," Prime Minister Malcolm Turnbull tweeted after the run-in.

"No, of course not," Opposition Leader Bill Shorten said when asked if he thought the protesters' behaviour was an appropriate way to treat a former prime minister.

A spokesman for Mr Howard said the 77-year-old had initially been shocked before NSW police "came to the rescue" but was "fine".

The CFMEU issued a statement arguing Mr Howard had "walked into the Sydney rally", which it described as "peaceful but loud".

Rallies organised by the CFMEU were held in every mainland capital city except Perth on Thursday with thousands marching in Sydney, Melbourne and Brisbane.