Randall has some disparaging things to say about most "professional"-looking RPG products, and I completely agree. In particular, I don't think the White Wolf books, the WotC D&D books, or Nobilis were all that good-looking. It's like people are blinded by the glitz and didn't think about the usability.
Specific points Randall raises that I'd like to comment on:
Page Backgrounds should hardly ever be used. If you are going to use them at all, use a simple design, one to four large symbols, at about 5% grayscale, or a very pale blue, red, green, or yellow. And only use it intermittently, like on the first page of a chapter. Use it to help readers navigate, not because you think it's pretty. Solid colors that are much lighter than the color of the text may be reasonable, as well. Busy designs are crap, because then you have trouble reading overprinted text; for this reason, don't use them unless the busy part of the background is in the top half or bottom half of a page that's only half-full of text.
Borders, as Randall says, should be functional. One function he doesn't mention is as a divider. If, for example, you have a border on the top and bottom of the page, and page headers above the top border, page numbers below the bottom border, then the borders are separating the main text from the page headers and page numbers. That's useful. A border between the sidebar and the main text is also a reasonable idea, as is a section divider or border for a pull quote/optional rule/chapter summary.
The more often a stylistic element is used, the simpler it should be. Stuff that appears on every page, like the margin borders, should be very simple. Complicated designs should be reserved for chapter head, chapter end, and things like that. You should not have an illustration on every single page, because sparser illustrations help to distinguish one section of a chapter or book from another. Think of it this way: you shouldn't have more than four pages or so in a row that all have the same design. After two to four pages of plain text, throw in an illustration, a pull-quote, or a table. Likewise, after two to four pages of tables, throw something in to break up the monotony. Don't use the same limit for the number of pages in a single style, either. Sometimes throw in an illustration after two pages, sometimes four, sometimes a run of three pages with illustrations. It's all about a careful balance between a uniform appearance and a monotonous one.