All Michael Noble wanted was a home for his family not too far from work.

At $690 per week, the 89-square metre shoebox in inner-city Waterloo was no bargain but it was within his price range and conveniently located to his job in healthcare and his daughter's day care centre.

Finances became tighter when the landlord wanted to raise the rent to $710, followed by another increase of $35 only five months later.

"We were just at the mercy of the landlord," he said. "We felt powerless."

Mr Noble and his wife successfully challenged the increase in the NSW Civil and Administrative Tribunal but were later hit with an eviction notice despite having a perfect tenancy record.

Now the couple are desperately trying to find something affordable and close to their jobs, the holy grail of Sydney real estate for middle-income workers.

"Anywhere decent is gone in an instant," he said. "People are so desperate, they apply online without even inspecting the property and then offer to pay a higher deposit. It's so competitive."

The cut-throat character of Sydney's rental market was laid bare in research released by peak housing advocate National Shelter last week which found that great swathes of the city were beyond the reach of average income households.

Analysis by SGS Economics & Planning found that affordable suburbs closest to the city were Liverpool and Blacktown, both about 40 kilometres away from Sydney's CBD. The data showed rents in suburbs nearer to the city such as Paddington would suck up almost two-thirds of an average income.

"Housing is the biggest household cost and renters don't get any relief," National Shelter's executive officer Adrian Pisarski​ said. "They face the choice between long and expensive commutes or living somewhere they cannot afford. Whatever choice they make they end up with not enough income to live a reasonable life."

The cost of housing is the source of growing anxiety across the Sydney basin. A survey of Sydneysiders published by McCrindle Research earlier this month found 95 per cent of respondents rated housing affordability as a challenge for their children and the next generation. Nearly 60 per cent described it as a "massive challenge."

One factor helping to drive up housing costs has been rapid changes in Sydney's economy, including its deepening global integration. Robust employment growth in knowledge-based industries, which have tended to cluster in well-located areas near the CBD, has boosted housing demand for inner and middle-ring suburbs.

Sydney's far-reaching global links would might surprise some locals. The city is ranked about the 12th most important city in the international trading network, a recent report by the federal government's Bureau of Infrastructure, Transport and Regional Development said. Sydney is also highly ranked on indexes that measure the global connectivity of cities. The one prepared by the British-based Globalisation and World Cities Research Network groups Sydney in its second-highest category alongside Hong Kong, Paris, Singapore, Shanghai, Tokyo, Beijing and Dubai.

Terry Rawnsley, an economist with SGS Economics and Planning, says Sydney has been casting off its 1950s car-based character dominated by cottages on quarter-acre blocks for some time, but the housing market has been slow to respond.

"The housing market hasn't really picked up on some of these changes until the past couple of years," he said. "Until quite recently the focus for new housing was on greenfields development out west. Its only in the past couple of years that large number of apartments have started to pop up close to the jobs."

With the housing market lagging changes in the labour market, the supply of inner and middle-ring housing has not kept pace with demand. The inevitable result is higher prices.

In recent years Sydney has also become more popular with international investors adding a new dimension to the property market.

Diversity in danger

There's widespread debate among planners, policymakers and developers about how to manage Sydney's ever escalating housing costs as it develops into a global city.

As rising property prices have forced would-be home buyers to spend more time saving for a deposit, they stay longer in the rental market with the proportion of tenants growing from 18 per cent to 25 per cent over the past two decades, according to figures from the Australian Institute of Health and Welfare.

There's concern high housing costs are resulting in an unhealthy lack of diversity in many neighbourhoods. For example, there's evidence that Sydney's essential workers like nurses, teachers and police are being forced to live long distances from where they work because of high housing costs. The 2011 census revealed the highest concentration of essential workers like nurses, policemen, teachers and tradesmen were in outer suburbs where housing is more affordable. Glenmore Park (near Penrith), Engadine and Cronulla had more police officers than any other suburb – all of them a long distance from the CBD. The largest clusters of educational professionals – an occupation category that includes teachers – were in Castle Hill and Baulkham Hills. In contrast, the biggest clusters of well-paid professionals including lawyers and doctors were in high-cost suburbs near Sydney Harbour.

The rental squeeze is not unique to Sydney, according to Hal Pawson, associate director of the City Futures Research Centre at the University of NSW, but other big global cities manage it much better.

He points out that London has a long-running planning policy of inclusionary zoning, in which a certain portion of new residential development is reserved for social and affordable housing for low to middle income earners.

"It's a way of hard-wiring the development of affordable housing into the expansion of a city," he said. "That's significant for a city such as Sydney which is going to expand dramatically over the next few decades. It creates an automatic supply of affordable housing which keeps pace with the growth of the population."

New York has a long history of rent control which has kept costs down for tens of thousands of tenants. The US government also has a system of providing tax credits to developers of low income housing, an initiative introduced under Ronald Reagan in the 1980s which still endures to this day.

"Some of the measures used in the US are quite unexpected," Pawson said. "The impression is that everything runs by markets in the US and government doesn't intervene. Well, government does intervene to quite a surprising degree. It does it through tax incentives which enable developers to finance affordable housing schemes."

Australia's National Rental Affordability Scheme, introduced under the Labor government in 2008, took a similar approach but was scrapped in 2014.

Pawson believes Australian developers and the state government are open to the idea of setting affordable housing goals.

"Among developers there is a growing head of steam behind the idea of having mandatory housing targets," he said. "They recognise, as policy makers do, that this is a common approach in many other comparable cities."

Until recently, London had an affordable housing target of 50 per cent of properties in new developments. In Sydney, that figure is closer to 3 per cent.

Pawson accepts that setting mandatory targets would not be painless for private developers or the state government.

"It does reduce the value of land that's being developed because the profit that's being made from land is being constrained by regulation," he said. "There is a limit on the ability to sell everything on the site at the highest profit. That said, the value of land has increased so drastically as a result of the housing boom, it should not be too much of a hard pill to swallow."

In a recent issues paper, the Committee for Sydney identified housing affordability as a threat to productivity, noting: "There are few policy objectives as fundamental as ensuring Sydney has the housing supply it needs. Cities that cannot meet this need will lose the competition for talent with cities which can and struggle to sustain economic growth."

Pawson agrees: "There is an economic price as well as a social price paid for a dysfunctional housing market."

Shelter NSW executive officer Mary Perkins also supports the concept of inclusionary zoning to create more affordable homes in Sydney, which she says is increasingly a city of haves and have-nots.

"Sydney has become unequal," she said. "If you are in an average income household, you cannot comfortably rent anywhere close to the Sydney CBD. That's creating a divided city where inner areas are for the affluent, and those on average incomes have to move further and further out."

NSW Federation of Housing Associations chief executive officer Wendy Hayhurst agrees the state government could be more supportive of inclusionary zoning, which has only been adopted by one council in NSW, the City of Sydney.

"Other big cities have shown it can be done," she said. "Their politicians looked around and realised that in order to have a city which functions properly, you need affordable housing and they did it."

As the senior policy officer with the Tenants' Union of NSW, Ned Cutcher is used to hearing tales of woe from renters faced with spiralling costs.

"They tell us they can't afford meals, they can't afford to see the doctor, they put off paying bills for as long as possible, all because so much of their income is going on rent," he said.

He hopes the increasing proportion of renters in Sydney will lend the sort of political clout tenants have in many European cities where they often outnumber owner occupiers.

"In Sydney and Australia more broadly there hasn't been a lot of political will to look after the rights of tenants," he said. "We've been a nation of home owners. That age old assumption is starting to buckle under the pressure of affordability."

The Grattan Institute's John Daley said the "dominant culprit" driving up housing costs are planning rules that limit the supply of housing in well-located areas.

"In theory, if there is extra demand in any particular area you would see extra supply and the impact on prices would be marginal," he said "But we are not seeing enough of that supply-side response."

Terry Rawnsley points out that many cities with a growing population – like Sydney – struggle with housing affordability. "The ones that have dealt with it best are those which have invested in transport infrastructure," he said

As for Michael Noble and his family, they are now facing the prospect of moving further out from the Sydney CBD to avoid paying crippling rents.

"But that has a butterfly effect," he said. "What we save in money we'll spend on extra commuting and less time together as a family."