I know this sounds crazy, but hear me out.
If you only allow the participants themselves vote on their own debate, then they will focus on convincing their opponent.
[LINK]
This solves every possible concern you've mentioned in this post with basically zero policy changes.
It makes duplicate accounts pointless.
It makes vote bombs impossible.
It makes (tedious) vote modding unnecessary.
The other solution would be to have appointed judges (or mods) that have well established records that would be required to vote on all debates (it would be less work than all the tedious vote modding).
My key frustrations with the debate and voting system are that,
if you choose a judge, the debate doesn't count.
you can spend days working on a debate and then nobody votes on it.
when people do vote on your debate their RFD makes no sense.
when mods review RFD's their opinions are too vague.