In two decades, Wally Swett's sanctuary has become a place of peace for primates - it's the humans that have him going bananas