Inquisitor debuffs are quite good. My main is a Swashbuckler, too. Granted they aren't as good as Swashbuckler's debuffs, but Rogues are the debuff class in the game. Comparitively among Priests, the damage isn't that bad. Easily out does Templar and Mystic (never played a Defiler, so can't comment on their damage), and with debuffs already attached to their skills they do even more naturally.
Besides check the AA for Clerics, there are a whole lot of ways to improve damage in them. Practically every line has some damage boosting ability or another. The Exorcist ones are quite massive for a Priest (though they only work on undead).