Ye should not be banking at all when taxiing, lassie

! Yes, you can use aileron with small planes to keep the wind from tipping you over, but always steer with rudder, unless it's the kind of older large plane that requires differential braking (the Ford Trimotor is one- I wouldn't recommend that beast for a beginner, although it's pretty docile in the air). Either way, if the plane is actually tipping over while you're taxiing, you're going way too fast. And- if you have the autorudder on (setting in the Airplane/Realism section), I'm not sure you'll be able to use the rudder properly for this.
Taxiing is probably the hardest skill to master in flying. The key is to nudge the throttle until it moves, then pull back a little and coast a bit. As it slows down, add a little more. Basically, you don't need to go much above idle RPM for taxiing, except when you're starting from a dead stop. Even then, as soon as she starts to move, reduce power right away. In the real world, at least in smaller planes at small airports, about the speed of a brisk walk is just right. Any faster and you can go off the taxiway easily, or mess up while turning.
When you want to slow down, first pull back the throttle, then use brief but full brake applications. Don't ride the brakes. If you are stopped and need to turn tightly right away, use differential braking (F11 for left, F12 for right) in addition to rudder. Just hold that brake down while also applying rudder in that direction,, and the plane will pivot on that wheel.
I've also found that I had to lower the rudder sensitivity a bit , or the planes veer all over the place. They're not usually so squirrely in real life. That's my preference, so experiment until you like it.
Anyway, take it slow for now and get the feel of it. Good luck!