you can get a meh horse for a few hundred bucks at auction but you are going to spend a couple hundred a month in feed and you could get a cheap stable for around 250 a month that you would have to clean yourself daily. i think my aunt stables her horse for around 300 a month but its at a family friends place so that is a super steal.
realistically you are going to spend about a grand on the horse then another 500+ a month to maintain unless you have facilities and pasture already...have fun
edit: that is not even considering vet fees, training, etc. horses are super, super delicate animals and can go from perfectly fine to near death instantly if you are not careful with what they eat and ****
double edit: our old horse died when i was like 10 from colic because it ate some corn and we didn't notice before it was too late
. like i said, they are a ****ing pain in the *** to care for