“Back in the ’80s, people used to tell me I had the best restaurant downtown,” says Clive Berkman, owner of Charley’s 517. “I would laugh and mutter under my breath, ‘I have the only restaurant downtown.’ ” Charley’s 517, which opened in 1971, was originally an old-school dinner house, with…