Some residents object to exhibition space, arguing that the artist’s work is pornographic