Like every plot twist in Doctor Who , it seems the cast and crew knew the departure of Matt Smith was coming far in advance of when we did.