Inspections of what is reputed to be one of the country's most haunted houses have revealed the extent of decay is worse than feared.
The 400-year-old Wymering Manor in Portsmouth has suffered damage by death watch beetle on top of known existing structural problems.
Volunteers hoping to save the building say they are "very confident" it can still be restored for community use.
Portsmouth University students are being enlisted to work on the project.
The Grade II* listed building was handed to a trust in 2012 after owners Portsmouth City Council failed to sell it.
A recent survey unearthed evidence of death watch beetle damaging old oak beams in its frame.
A complete restoration of the house will cost about £2.5m
Andy Mason, of Wymering Manor Trust, said: "It was worse than we were expecting and this has brought us down to earth a bit. It doesn't knock us back - we knew we had a lot of work to do here.
"Even in a relatively short time, I've noticed the house's condition is starting to slide further."
Nevertheless, the trust has started work to develop it as community facility - with ideas including creating a hub for social enterprises, arts performance areas or making it available for wedding receptions or corporate events.
A £50,000 Big Lottery grant is being spent on bringing the Victorian parts of the house back in to use with new glazing and removal of asbestos.
Mr Mason said: "We're starting to deliver solutions to the problems we've got here."
Built around 1581, with various add-ons since, it now surrounded by modern suburbs
Reputed to be haunted by more than 20 ghosts, including a choir of nuns and Sir Roderick of Portchester
Used as a vicarage, home to a Catholic religious order, a family house and most recently was a youth hostel
Unused and in a state of disrepair since 2006. Featured on the UK's Most Haunted TV show
With parts of ceilings in a few rooms held up with scaffolding, the full restoration cost is estimated at £2.5m. The trust is currently investigating possible sources of funding.
Portsmouth University is giving architecture students the opportunity to work on the building as a unique case study.
Lecturer Karen Fielder said she was "very optimistic" the restoration project would come to fruition.
"It's ideal - students get the chance to experience a real life project, trying to find viable uses for historic buildings that are financially sustainable."
"The manor still has a lot of secrets - there is a lot to learn about the history and structure.
"It's such a treasure for Portsmouth, the city can't see it collapse."
Further reading with video: HAUNTED WYMERING MANOR NOW HANDED TO A TRUST