The background image is defined in the css for class infoIcon, and it works for IE7, FF2 and FF3, but just not for IE6. After digging around, I finally found a solution to this damn simple problem, thanks to Kapil’s article. The solution is to add a html comment inside the div:
Kapil mentioned to use will appear in FF. The best way to do it is to use the complete html comment, and put any explanation inside. The reason that it works is because IE6 tends to leave spaces for text content in div or span, even though there is nothing. If you insert comments between them, it will force IE6 to render div or span without any content at all. Hope this helps to other people who need it.