The late artist revealed he was too frightened to go out during the pandemic and was "a bit relieved" that his latest project in Paris was delayed