All of the stuff borat says in "kazakh" is actually Hebrew, its just a bunch of random Hebrew words. Sasha Cohen is Orthodox Jewish...
Also, if you watch his sketches, you'll notice he uses a lot of Cyrillic captions (which are then overlayed with "translated" English ones). In reality, the Cyrillic characters are randomly arranged, and don't form any real words, Kazakh or otherwise.

But then, that's the whole reason he chose Kazakhstan as Borat's country in the first place, most of the people he lampoons know little (if anything) about the country, so they just assume it's authentic.