The former Bega mayor is set to become the first female MP to represent the southern NSW seat since Federation, after narrowly edging out Liberal candidate Fiona Kotvojs.

While ABC chief election analyst Antony Green called the seat for Labor around 1am on Sunday, the party did not claim victory until around 1pm on Sunday, when Ms McBain was around 1400 votes ahead with approximately 4000 postal votes to be counted.

While Dr Kotvojs was ahead on primary votes, Ms McBain nosed in front of her on a two-party preferred basis.

"On all metrics it looks like the Labor party will secure the seat again," Ms McBain said.

However the Liberals were reluctant to concede until all the votes were counted.

By 5pm the gap had narrowed to around 900 votes and by 5.40pm it was down to around 740.

Several errors were detected in the count which advantaged the Liberal candidate. One error meant Dr Kotvojs picked up an extra 300 votes.

However Labor sources indicated they were still confident the seat would fall their way.

Ms McBain said Dr Kotvojs would have required 63 per cent of the remaining postal votes on a two-party preferred basis to snatch victory.

Just before 6pm, Labor said the Liberals would need to get around 70 per cent of the 1700 votes left to count if it were to win the seat.

However a senior Labor figure says it was an "ugly win", with the party's primary vote going backwards and the win largely relying on preferences.

"It's a bit of an ugly win for us I can see, but it's a win just the same. It was a difficult election for us, the advantages of incumbency, for example, was an enormous hurdle to overcome," Hunter MP Joel Fitzgibbon told the ABC's Insiders program.

"We, of course, lost a bit of our primary vote again, coming off a fairly low base in 2019, but as you pointed out, we lost Mike Kelly, we have a Prime Minister who, with the Opposition's full support, has been literally spending hundreds of billions of dollars, providing support to people.

"He has had, of course, saturation media now for weeks. I have never seen the advantages of incumbency so large. Of course, as I said, his approval ratings were high, but he wasn't able to improve his primary vote by much last night."

Ms McBain told The Canberra Times she would work hard to gain back ground lost by Dr Kelly's resignation.

"We knew that Mike Kelly had a personal vote of around 3 per cent, that's probably exactly what we saw. It's going to take a lot of hard work right across the electorate to win some of Mike Kelly's personal voters back but I'm up for that challenge," she said.

Labor leader Anthony Albanese described it as an "against the odds victory", given the government's record spending to stave off the full impacts of the coronavirus downturn.

"Kristy McBain will be a champion for the people of Eden-Monaro. She is passionate about her local community, she is articulate, she has an extraordinary capacity and she is someone who will look for solutions, not look for argument," Mr Albanese said.

He rejected claims the tight race left his leadership of the party vulnerable.

"What this campaign was about was not people with power, the Prime Minister or the leader of the Labor Party. This campaign was about people that don't have power. Those people that we visited just outside Cobargo on Friday who are living in a caravan. They have been living there for six months. They still have debris on their property. This is about them. It is about people who cannot get assistance for their small businesses, who don't know what is going to happen to them after September in terms of JobKeeper. This was about all of those issues - it was not about leadership," Mr Albanese said.

However Coalition minister Angus Taylor labelled the result as "devastating" for Labor.

"On average by-elections like this have a swing against the government of 3.8 per cent and the government hasn't won one for 100 years," Mr Taylor told Sky News.

"Albanese practically moved to the electorate, 23 different visits. He threw the book at this and you could see that on the polling booths on the day, you could see it on the advertising on television. Absolutely threw the book at it."

Ms McBain said it had been an extraordinarily tough campaign, with Labor forced to find new ways of reaching out to voters due to the ongoing pandemic.

"We had to find new ways of engaging the electorate, we made thousands of phone calls, I did a whole range of Zoom meetings so things weren't as they would normally be during a general election," Ms McBain said.

Labor's campaign centred on the region's ongoing bushfire recovery, and the need for jobs coming out of the coronavirus pandemic.

Ms McBain - who made her mark during the bushfire crisis as mayor of the fire-ravaged Bega Valley - will push for a local jobs plan, for roads and black spots to be fixed and more money to be released from disaster recovery programs.

She pledged to work with state MPs John Barilaro and Andrew Constance, Joe McGirr and Wendy Tuckerman to keep the spotlight on the region's struggles.

"As Mike [Kelly] said when he gave his press conference for retiring from the position, he hoped the byelection would bring a spotlight back to Eden-Monaro especially on the back of drought, bushfires and COVID-19 and that's exactly what it did," Ms McBain said.

"I don't think people realised that most of Eden-Monaro was still in drought. I don't think people realised that some of that bushfire recovery money hadn't been flowing out as quickly as it should have been and I don't think people realised the impact of cumulative disasters on regional economies such as ours across Eden-Monaro.

"We know that the cameras will leave and the spotlights will fade now the byelection's over so it's up to me to continue to amplify the voices of those people around me to make sure that we are fighting hard to make sure these issues won't be forgotten by this government."

The byelection was sparked by the resignation of Labor MP Mike Kelly for health reasons.

Dr Kelly congratulated Ms McBain on her win.

"Delighted at the attention we drew to our issues. As painful as leaving has been this has made it all a little bit easier," Dr Kelly tweeted.