Two Victorian men who died from coronavirus were cancer patients in the oncology ward at the Alfred Hospital.
The men who died on Thursday among patients in the ward diagnosed with the virus.
The other two patients infected are in a stable condition, an Alfred Hospital spokesman confirmed.
Three staff members have since tested positive for coronavirus, with others now self isolating.
Victoria had its first three virus deaths on Thursday, all men in their 70s.
The state's cases of coronavirus rose on Friday to 574, an increase of 54 from Thursday.
That included 16 cases acquired through community transmission, an increase of nine within a day.
The state's total also includes 363 cases acquired through overseas travel and 163 people who became infected after contact with a confirmed case, with the source of the remaining 32 cases under investigation.
As Victoria's coronavirus cases continue to climb, Port Phillip City Council clamped down on beach access as of 6pm on Friday.
Police were seen at the popular St Kilda beach, moving people on, and are expected to patrol the area over the coming days.
Scores of people were at St Kilda beach to enjoy the warmer weather, in clear breach of health advice to keep apart.
The council said in a statement that its beaches must not be used as gathering places, with visitors having to keep their distance from each other.
Premier Daniel Andrews also issued a stern warning on Friday afternoon and threatened to close all beaches if people continue to visit them, as the number of the COVID-19 cases in the state continues to rise.
"If you can stay home, you must stay home," Mr Andrews told reporters.
"You don't need to be at the beach. If you choose to be at the beach then that is a choice that may cost someone their life.
"I've closed the pubs because no one has to go to the pubs. If I have to close the beaches, I will. Be in no doubt about that."
Hundreds of Victorian police are cracking down on people flouting coronavirus restrictions, including by checking on returned travellers required to self-isolate at home for two weeks.
The premier said the 500 officers would have even more scope to catch people when Australians returning from overseas were quarantined in hotels and other facilities from Saturday evening.
"If you're doing the wrong thing, you will be caught," Mr Andrews said.
Between 1300 and 2000 Australians are returning to Melbourne daily at the moment but the figure is expected to drop off in the coming days.
The state already has about 5000 rooms on stand-by for travellers forced to self-isolate from Saturday night after the measure was agreed to at a meeting of federal, state and territory leaders on Friday.
The premier stressed more severe restrictions were on their way.
"Stage three will come. It is not here yet," he said.
Department of Health and Human Services data showing the spread of cases across the state indicate some of Melbourne's most affluent council areas have the highest number of infections.
There are 58 cases in Stonnington, which includes the suburbs of Toorak and South Yarra, make it the most-afflicted area.
The Mornington Peninsula has 39 cases and central Melbourne 33.
The Boroondara council, which includes Hawthorn and Kew, has 28 cases, while Port Phillip has 26.
Australian Associated Press