See, I don't think I would agree that thought is a purely internal experience. Insofar as thought is communication to oneself, it's still communication, and thus necessarily based in language as a system of symbols. Experience qua experience is experiencable, but meaningless. For it to have a meaning (e.g., "get on the bus") there must be language.
.