The property, bequeathed to the district by a physician friend of the artist for use as an arts centre, has been neglected for decades