The Environment Protection Authority is investigating after the substance was found at parks in the Hobsons Bay City and Merri-bek council areas.
Early investigations raised concerns that the asbestos was being introduced to the parks through mulch deliveries, like the contaminated mulch crisis in NSW.
But on Monday the authority said there was no evidence of that, and it is more likely the contamination came from illegal dumping.
Sites have been fenced off as investigations continue into asbestos discovered at a number of parks. (James Ross/AAP PHOTOS)
The authority says they are confident Victoria doesn't have the same widespread problem as NSW, but conceded asbestos would be found across the state because of its widespread use before it was banned.
It has confirmed contamination in two parks and is awaiting results for another four, which are expected on Tuesday.
A seventh park, the PA Burns Reserve dog park at Altona, remains under investigation after it was flagged by a community member late on Saturday.
Victoria's chief environmental scientist Mark Patrick Taylor described the contamination in those parks as "very, very low".
"We've only found a handful of asbestos pieces in combing through quite a number of parks in the council area," he told reporters on Monday.
Hobsons Bay City Council, where five of the six parks are located, on Sunday received two remedial notices from the EPA requesting proactive inspections of more parks and gardens.
The notices require the council to commission a hygienist to inspect all council managed parks and gardens that have received mulch in the past 18 months.
The council on Monday said those works are already under way.
The EPA will also review Hobsons Bay City Council's procedures for sourcing and handling mulch, and has requested all reports dating back to July 2021 regarding asbestos-containing materials found in mulch in or near council infrastructure.
In a statement, the council said it would provide a list of all suppliers used to source mulch, as well as a list of contractors and sub-contractors used, from the same time period.
Chief environmental scientist Mark Patrick Taylor described the contamination as "very, very low". (Diego Fedele/AAP PHOTOS)
Merri-bek Council was remediating Hosken Reserve at North Coburg after asbestos was found during the construction of a park at the reserve in January.
The asbestos there was believed to be from historic building material.
Merri-bek Council confirmed the asbestos-contaminated soil was found within the area closed for the major upgrade works.
It was professionally removed but more asbestos-contaminated soil was discovered recently, again within the construction site area closed for upgrade works.
The newly located contaminated soil is being removed.
Energy Minister Lily D'Ambrosio brushed away a question on whether a task force was needed to investigate the issue, saying the EPA was "all over this".
"They've not lost a minute in getting on the ground working with local councils, who own these parks and these facilities, to hold them to account," she said.
So far, the material found has contained bonded asbestos rather than the more dangerous friable asbestos.