The composer explains the thinking behind his latest film, and reminisces about the parties held by a Frieze founder’s dad