I think he'll work if they give him time.
Colbert, I think, can pick up the audience that Conan would have had if NBC had just given him due time. (Honestly, while Conan occasionally has a good bit or two, I think he's lost something in the transition to basic cable.) Colbert's quirky, but he can grow on you if you play along and willing to listen to his side of the story. Many conservatives won't be willing, but so be it.
I think it will be a good mix for the Letterman/Ferguson folks who are used to turning on CBS and I think he'll draw a healthy crowd from the folks who used to watch Comedy Central, but haven't quite turned onto Larry Wilmore's panel format and would rather follow Colbert than try and pick up the new Daily Show.
I get the impression Colbert is going to be kind of sly, smart, witty and intelligent, in New York sort of way. There's a place for that now. Fallon is kind of more giggly, light, harmless fare, Kimmel is a little lowbrow and Conan is uneven at best. Colbert will put out a solid, smart, snappy show that you want to see every night. I'm kind of looking forward to it.