Yes, the crate should indeed be a "happy" place.
I've found, and been told by others when my dog was just a pup..that when you take the dog out to do his business, use a command. Such as "go potty" or "go pee!" Then when your pup does, make a really big deal out of it, say "good boy!" a few times and give a treat. You want to make it as good of an experience for the pup. If he knows he is doing something that makes you happy, he should do it more often.
From what I am reading, you take him outside, and he goes, then you feed him and take him outside to go again? Puppies don't digest their food that fast!

hehe
Let us know how it goes with him.