Been there... done that. My son is now 8 and I am re-married. But, there were a few years in there where it was just him and I. It's not bad at all. It's actually wonderful. Sure... it cuts into your social life big time... but you won't even care. The joy of spending all that time with your child is well worth whatever sacrifices you have to make.
Don't get me wrong... it can be lonely, frustrating and at times depressing doing it alone but when your kid is laying on your shoulder you'll smile.
Kudos to you man for stepping up and taking on the responsibility. Not all guys are true dads.