By Wilson's theorem, we have $22!\equiv -1\pmod{23}$. Therefore $$1\equiv 22! \equiv 22\times 21 \times 20!\equiv (-1)\times (-2)\times 20!\equiv 2\times 20!\pmod{23}$$

By noting $12$ is the inverse of $2$ modulo $23$, we find $$20!\equiv 12\times (-1) \equiv \boxed{11}\pmod{23}$$