Because that (οτ). Causal use of οτ.

By reason of him (δι' αυτον). "Because of him," regular idiom, accusative case with δια.

Went away (υπηγον). Cf. John 6:67 for this verb. Inchoative imperfect active of υπαγω, "began to withdraw" as happened at the time of the raising of Lazarus (John 11:45) and the secession was still going on.

And believed on Jesus (κα επιστευον εις τον Ιησουν). Imperfect active of πιστευω (note aorist in John 11:45). There was danger of a mass movement of the people to Jesus.