All web pages should contain valid HTML code. This goal is straightforward to achieve (because it can be checked with any number of different tools) yet often ignored, and exacerbated by the forgiving nature of some browsers, notably Internet Explorer, which let you "get away" with poorly written web pages. The problem is that, while your page may display properly in your browser, it may not in a different browser, on a different platform, or even a different version of the same browser. Having valid code doesn't guarantee that a page will display properly; but not having valid code is flirting with unpredictable display rendering.
At the time of writing, the latest Hypertext Markup Language specification is XHTML 1.0 (superseding HTML 4.01) from the World Wide Web Consortium (W3C). (XHTML is, according to the W3C, "a reformulation of HTML 4.0 as an application of the Extensible Markup Language (XML)." Its purpose is to add extensibility and portability to HTML. In practice, an XHTML file looks like an HTML file with all the "I"s dotted, "T"s crossed, and just a bit more rigid structure.) See the HTML reference for more.
Some older browsers will not display XHTML properly, so you may choose to use HTML 4.01 as your standard. But even so, you should write well-formed HTML, making sure, for example, that elements are properly nested. Most browsers are forgiving when it comes to poorly written HTML, but you should never count on that. I further suggest that you get as close to XHTML as you can. By that I mean use the XHTML rules which do not conflict with HTML rules, such as always use lowercase for elements and attributes, always put attribute values in quotation marks, and so forth. For guidelines on XHTML and on converting your existing HTML to XHTML see the W3C's official HTML vs XHTML differences, W3School.com's How To article, New York Public Library's Guidelines, or Better Living Through XHTML, among others.
Once you've created your web pages, you should always validate them to make sure they conform to the standard you've selected. There are web-based tools, such as the W3C validator. A handy way to invoke this W3C validator in Internet Explorer is by dragging this button -- Check HTML -- onto your IE toolbar. Then you can browse any page and just click your new button to validate the HTML. Many webpage editors also provide HTML validation. HTML-Kit is one example, a free utility which includes the popular HTML-Tidy validator. Once validated, you may add the W3C seal of approval to your page, an image indicating that your webpage conforms to web standards. This image is available through the W3C validator once your page successfully validates.
Part of getting your page ready for validation is deciding what standard you wish to conform to, as discussed above. You need to specify this in each web page via the DOCTYPE element. Besides choosing between HTML and XHTML, you also must choose strict, loose, or transitional. One web-based tool to help you generate the DOCTYPE, among other base elements is this web page builder, where you specify your choices via pull-down menus, and it generates an HTML framework for you.