1. Ideally they should be at least 5 or 6 inches tall to ensure visibility. Smaller numbers may not be visible from the street if you have a large front yard. 
2. The numbers should be of a color that contrasts with their background. Reflective numbers are great because they are easier to see at night.   Make sure the numbers are lighted by your exterior lighting.
3. Check to ensure the numbers are not blocked by any trees, shrubs, or anything else that may obscure their visibility from the street.  Double check this in the summer, as shrubs may fill out or flowers may cover the numbers.  In winter, make sure snow doesn't pile up or otherwise obstruct the numbers.
4. Make sure that the numbers face the street that is listed in the house's address. It does emergency workers no good if the house number faces a different street than the one the workers are traveling on.
5. If your house not visible from the road, place the numbers at the entrance to your driveway.

  ... And 5 Fun Facts About Street Addresses   

1. Who's on First?  The most common street name in the US is Second St.  It is more common than First Street, since First is often called Main Street, Center Street, etc.  Third is the second most common street name.
2. City numbering usually starts at a base point, with numbers reflecting the distance from that point and usually the direction (north, south, east, west)
3. City addresses also often correlate with a street-numbering system.  For example, you may know that in Philly 1610 Walnut Street would be on Walnut between 16th and 17th streets. 
4. Rural addresses, that used to be along the lines of RD #2, were changed with the advent of 9-1-1 to make them more easily located.  Now rural numbering usually reflects distance from a town center, usually 1000 for each mile.
5. Want to know the nearest cross-street for a New York City address?  Find the formula here