I'm sorry to hear you have to give up on martial arts. I have studied and practiced martial arts for many years and find that bike riding is generally pretty uncomfortable, especially for back, neck, and hands. I have a couple bikes including a short wheelbase recumbent with below seat steering and that is the only type of bike I can recommend if you're having back problems. The low steering allows me to ride with my hands at my side- no effort whatsoever goes into keeping my hands on the steering, and none of my weight is on my hands, so they don't get numb like they do in about 1/2 hour on my road bike. My particular bike has a hard seat with a foam pad. On long rides I get a bit butt-numb, and occasionally get some numbness in my toes, but that is cured easily by walking around for a minute or two, which also allows some of my sweat to evaporate. I believe the recumbents with mesh type seats are far better in that regard.
A pair of gloves is a good thing to have when riding. Their main purpose is to protect your hands in the event of a crash. I have some full fingered gloves for MTB riding and some 1/2 finger gloves for road riding, but I mostly use the full fingered gloves now (I have scraped fingers up in crashes).
The main difference between a $20 helmet and a $50 one is the in the way the fitting is done- i.e. the strap hardware. A $50 helmet will generally have hardware that allows you to fit the thing to your head a little better. I can't say if that translates to improved protection, but it improves comfort a little, and bikes are generally so uncomfortable that any way that you can get comfort is a justifiable expense.
Toe clips are OK, but you'll find that you really won't pull much when you have them. They mainly keep your foot centered properly on the pedals. There are add-on toe clips that fit most platform pedals. You just use the reflector mounting holes on the pedals to mount them. It will take some practice to get in and out of toe clips at stop lights, but you'll get used to them. Later, if you switch to clipless pedals, you'll wonder why you ever bothered with toe clips.
Make sure you have a water bottle and cage to mount on the bike. You need to drink a lot when you ride.
bikepartsusa.com has all sorts of little doodads for biking if you're in the US.
TD