The Beulah Historical, Learning and Progress Association hosted a community barbecue on Saturday to show that the true nature of the community was "kind, generous and welcoming".

Organisers also said they wanted to lend their voice to "the growing number of calls for the Victorian Government to ban the public display of the Nazi Swastika in Victoria".

Attendees brought a range of flags to the event including Aboriginal flags, a Torres Strait Islander flag and a Pride flag. Flags from various countries including Australia, Japan and South Korea were also on display.

Member for Mallee Anne Webster and Member for Mildura Ali Cupper spoke at the event.

In a social media post to Facebook, Dr Webster said she was standing in solidarity with Beulah.

"This community is warm, friendly and welcoming and I was proud to stand up with them and be counted today," the post said.

"We must stand in solidarity against the flying of the swastika (a symbol of hate).

"As Australians we must remember the horror of World War Two and our fight against the tyrannical regime represented by this symbol.

"We must not allow a poor memory and slow creep toward apathy. If we do not actively remember and learn from the past, and remain vigilant, then we are doomed to repeat it.

"As Australians we value the hard won freedoms for all to live side by side in this our great democracy."

The display of the flag gained international media attention and was condemned by Premier Daniel Andrews.

Speaking to the Mail-Times last week, the flag's owner Bill, who wished not to have his full name printed, said he'd had the flag in his garden for a month and received it as a gift.

He said he hadn't received any complaints from neighbours, the council or police prior to a story running in a Melbourne newspaper on Tuesday.

He originally said he would not take the flag down, however it was removed by Wednesday.

Victoria's Equal Opportunity and Human Rights Commission said in a statement on Tuesday that the symbols on the flag caused harm to the Jewish community.

"(They) are more than just lines and colours - they represent hate and trauma for so many people around the world, including the members of Victoria's Jewish community," the statement said.

"To display that flag publicly is reprehensible - it runs counter to community values and the importance Victorians place on inclusion and diversity. It has no place in this state."