As we mentioned above, you can never eliminate all problems, so you’ll want to set a target level for declaring that the site is adequately bug-free for launch, called the test-completion criteria. A common target is to eliminate all mission-critical problems, most of the moderate problems, and as many minor problems as your budget allows.

Mission-Critical Errors
Mission-critical errors are also known as fatal errors. These are the errors you must catch or the web site and final product will fail. Mission-critical errors prevent users from achieving their goal.

While fatal errors are traditionally thought of as errors that crash the system (or, in this case, the web browser), in web site design this term should be extended. For example, missing a submission button on an order from would be considered a fatal error because a critical task cannot be completed.

Moderate Errors
Moderate errors are those problems that are irritating and may make you look stupid, but don’t prevent the users from continuing to do what they need to do. An example of this is a user who is uncertain of how to submit an order request because you’ve decided to label the button “OK,” Moderate problems should be ranked in order of severity and based on the cost involved in making the changes.

Minor Errors
Minor errors are those problems that pose little or no obstacle to the user achieving a task but can be distracting. Typos are often considered minor problems when they appear in the middle of a paragraph of text. Color dithering problems, alignment problems, and duplicate links can all be considered minor problems. They usually do not impede users in performing their tasks but can reflect poorly on the level of professionalism associated with your site.

Types of errors
In addition to ranking the severity levels of the problems you encounter, it’s also important to document and be aware of the types of errors that are cropping up. Classifying the type of error will help determine who on the design team needs to make the corrections and also helps to determine the severity and priority of the fix. There are four broad categories of errors.

Cosmetic Errors
Cosmetic errors commonly refer to image-loading problems, alignment flaws, readability and legibility problems, color and palette issues, typos, and basic page layout inconsistencies. These errors are usually only minor disruptions to the user, but you need to be aware that some cosmetic errors can keep the user from achieving desired tasks. For example, a missing graphic might cause navigation problems.

Structural Errors
Structural errors manifest themselves as structural design problems. They are often the result of poor information architecture or overlooked user paths through the web site, for example, a site architecture with a dead-end page requiring the user to hit the browser Back button.

Worse yet would be a site that didn’t allow a user to get from the product page to the order page. User testing can be extremely useful in identifying structural design errors and should be integrated early in the process to avoid major changes toward the end of the project.

Platform Errors
Platform errors are errors that appear only in specific hardware, operating system, or browser configurations. These appear when testing on multiple platforms. For example, when exploring page layout on different machines, the text may overflow the display area on a PC but not on a Mac (conversely, text that looks fine on a PC may be too small to read on a Mac).

Coding Errors
Coding errors are frequently the most difficult to catch and also the most devastating. Imagine miscalculating a loan or overcharging the user for services (or systematically undercharging)!

Dealing with Errors
Some of the most common problems have to do with layout differences on different platforms. The best way to combat the variations in layouts that occur due to browser differences is to test, test, and retest. You’ll want to test early, often and thoroughly. Knowing of the problems ahead of time will keep post-QA testing fixes to a minimum.

Load testing will give you an idea of the number of users or transactions you can expect before experiencing trouble. This will help you understand how your site might handle a spike in traffic, or how well it will stand up to service limitations due to limited back-end or ISP capacity.

When performing these tests, you’ll want to be sure that the number of users you can handle is sufficiently higher than your average number of users. Estimating such numbers is an art form. At the very least, you should be able to handle spikes of three to four times your normal traffic during peak hours.

Again, this also depends on the type of site you have. If you are selling products, then you may expect even larger spikes that coincide with product releases on your web site, and so you may want to set your threshold for much larger spikes (e.g., the spike that comes when a new Harry Potter book is released).

In evaluating what level support you need for various loads, ask yourself the following questions: How critical is it that people can access your site? Do you lose money if people can’t get there or perform transactions?

What kind of users do you have: a more dynamic user base or a static, consistent user base? (A dynamic user base, one with more turnover in users, may be more highly variable in load, while a more static user base may provide more predictable numbers.) What kind of content do you have? Do you have time-dependent features, such as found on a news site?

Outsourcing
One of the approaches to testing is is outsource the work. Specialized testing companies can perform a wide variety of tests that are often difficult and expensive to perform in-house. In addition, they often have access to a wide diversity of users and testing environments.

For example, they may have several sites around the world that they can use to test global bandwidth or cross cultural issues. If you require a good deal of testing and don’t necessarily have the expertise and equipment in house, you may want to consider out sourcing as a possible solution.

One method involves writing scripts that behave like a user but send all of the possible paths and elements to the computer. This can be very useful for regression testing. Once a test has been created, it is easy to test updates to the pages because the same script can be run again to ensure that nothing has been broken when making updates.

Another method of automated testing is provided by automated link-testing programs available both in the public domain and on the retail market.

These are systems that will automatically check all the links on a given page. Such programs are not perfect, however, and should not replace other forms of testing.

Similarly, with mission-critical systems, such as e-commerce or transactional systems, you should perform a through review of the code to ensure proper calculations. For this, a set of code-testing procedures must be developed.

As mentioned earlier, the level to which this needs to occur depends on the nature of the web site you are developing. Mission-critical sites require much more thorough testing than sites designed primarily for entertainment.

There are several methods for code testing that have been established through years of software engineering experience that can be applied to web site design. The following is provided as an overview of the possible methods. You may select the most appropriate for your given needs.

Code Review
Code should be reviewed by another programmer or set of programmers. This means having programmers actually read through the code line by line to verify that it implements the site’s requirements and procedures.

This should be an established practice for all mission-critical code. The more eyes that review the code, the more likely it is that you’ll catch simple programming mistakes such as one-off errors or rounding errors in calculations. This is essential for any transactional system in order to avoid costly mistakes.

Unit Testing
Unit testing refers to taking a functional subset of the system (a unit) and verifying that the outputs are correct – for example, examining a particular subset of a task such as a piece of code that processes an individual form. In unit testing, you want to ensure that a single piece of code can take the proper inputs or parameters and produce the relevant outputs.

Task-Based Testing
Examining the processes that users will most likely be interested in and ensuring that they occur flawlessly is the goal of this type of testing. This is not a completely comprehensive approach to testing, but rather a matter of placing emphasis on exploring the most common paths of the web site.

You should focus on testing the possible paths a user might take through the web site for the most common or critical tasks. Develop some specific questions that the user might have and then evaluate whether or not users could achieve these goals.

For example, a library site may expect the users to find directions to the site, find the database of documents and successfully search and find an item, and perhaps find contact information in order to call the library.

Draft a formal of the process tasks and results tasks that users should be able to achieve. You can then test that each of the processes can be achieved in a reasonable way, note those that are problematic, and use your formalized feedback mechanisms to alert the whole design team of problems.

This method is not guaranteed to catch every problem with your web site. However, it will help to identify the larger issues and will let you know if your major tasks can be achieved.

It will not test every possible behavioral path through the site. Task-based testing often catches the most glaring problems – that is, those related to the subset of tasks you’ve chosen to investigate – and is more useful in the early stages of design. The best testing will occur with a suite of tests that includes both task-based testing and amore comprehensive testing process.

You need to develop a method for informing earlier stages of design based on testing experiences. This feedback loop allows you to reevaluate and correct earlier assumptions when they may have catastrophic implications.

You want to minimize the probability of having to correct earlier assumptions, and having a well-planned design process will help, but you also need a way to inform people of problems at the development and requirements levels. This means establishing processes for informing your design team about how they need to change things and what the critical differences need to be.

Why Bother With Quality Assurance?
QA may seem painful at times; however, its benefits greatly outweigh the growing pains involved in establishing quality assurance processes. QA serves several purposes:

• Providing design guidance early in the design process
• Eliminating errors as early as possible
• Achieving an overall cost savings by catching errors early
• Providing guidelines for changes, additions, versioning, and so forth
• Verifying that the original site requirements are still meaningful

A major benefit of developing pervasive QA measures is the cost savings from instituting error-saving devices early on in the design process. In software development, the earlier you catch an error, the less expensive it is to fix.

In addition to catching errors at the earliest stage possible, properly recorded QA measures provide a reference manual for the design. This can help to eliminate repeated errors, minimize the chances of breaking something when performing a fix (a much more common occurrence than any of us would like to admit), and provide a rationale for the course of the design.

Integrating Quality Assurance Measures
There’s no limit to the level at which you can integrate quality assurance measures into your process – you can always keep improving your site. However, 100 percent quality requires infinite cost. You’ll never develop a product that is perfect. Even if you test the product thoroughly, your test or tester might be wrong, in which case you will need to test the test. This leads to infinite regress. Thus, you need to designate a level of quality that can be targeted in a cost-effective way.

The level of quality often depends on the cost of failure. For example, an interface that is used to ensure that people receive life saving medicine in time will require much more testing and a higher level of error free operation than an interface for a game that has no life-threatening costs to the users.

Thus, determine a critical level of error-free workmanship and set this as your goal. Do this up front and by weighing the costs of any mistakes that may make it into the product. This can then be used as a guide for your quality assurance measures.

Regression Testing
Regression testing is retesting after you’ve made a fix. Regression testing verifies to things: that the fix you made actually fixed the problem that had been identified, and that the fix you made didn’t break anything else in the process.

A vast majority of problems are caused by designers and coders trying to fix things. Too often this happens because, when making minor changes or “obvious” fixes, no thought is given to testing the site after the changes have been made.

The introduction of new problems can be minimized by documenting the changes made and the tests performed and making sure retesting is part of the process. This documentation allows you to understand the implications of changes and any important dependencies that may exist, thus facilitating later changes as well.

The following section reviews many areas that often turn up as last-minute problems. By scheduling time for the following processes, you can ensure that you’ll have the extra time at the end of a project to deal with truly unexpected events that will undoubtedly transpire.

Pre-Launch Planning
Testing aside, some preparation is necessary for the final launch date. Technical planning ensures that your server will be available, your domain name and security certificates are working (and payments are up to date), your server has been tested with all the software services that will be required at launch, and capacity and security plans are in place.

You also need to coordinate with the business end of your site, making sure that marketing doesn’t announce it when it is ready, that business partners and customers are ready to conduct business through the site, and that internal support staff and support processes are ready to go.

Registering Your Domain Name
One of the crucial planning steps is to make sure you’ve selected and registered your domain name before launch. This is something it’s wise to do very early in your project, and it must be done before going live.

Choose a domain name that is easy for your users to guess, to remember well, to type quickly, and to type without spelling problems. Whenever possible, most businesses will want to get a .com address, because browsers will default to .com when the user omits the extension.

In addition to your primary domain name, consider getting alternatives that users might enter when they guess alternative spellings or abbreviations, when they choose to hyphenate or not hyphenate multiword names, and when they misspell your name.

You can find out what these alternatives might be by asking your prospective users what they think the domain name for your business is, and asking them to type it in so you can see what misspellings they make (or ask them to type it over and over again as quickly as they can, and you can count how commonly each misspelling occurs).

Also consider whether you need to register .net and .org alternatives for your name and abusive misspellings of your name) to prevent other companies who use a domain name similar to yours from creating a parody of your site or competing with you.).

• Does this technology provide real value to the user?
• Is it cross-platform?
• Is it standard (do people need to download a plug-in?
• How much of the user’s time will be saved?
• How will users use it?
• How much learning will it require?
• What are the overall benefits to the user and provider?
• What are the development and maintenance costs?
• Will the extra complexity add significant risk to the project?

Example: Usability of Frames

Frames are sometimes used in web pages to enable convenient navigation. However, they also introduce a variety of problems, many of which have to do with how the browser supports frames.

As an example of weighing the tradeoffs in applying a certain technology, here is a list of pros and cons for frames. While our experience is that frames are most often a problem for users, consider which of these issues applies to your situation and choose accordingly.

Advantages of Using Frames
Frames can provide a technical solution that facilitates a successful user interaction. Frames can improve quality by making software development easier, primarily by facilitating the separation of content and navigation. Following are a few of the most common advantages.

Nonscrolling page components: Certain items, such as ads and navigation, can remain on the screen at all times.

Uncluttered printing: Printing is more content-specific, since usually only a single frame is printed. This can be useful, for instance, if users want to print pages from an online manual for inclusion in a preexisting document and they want the printed format to match that of the current document. The surplus navigation will not be printed.

Mandatory home page traversal: Because bookmarks only capture the URL of the frameset, you can make sure that all bookmarks will lead to your home page. This can be useful for very dynamic sites where the content changes often (no “Document Not Found” errors). It’s also useful if you want to force the user to come in through the front page (so they see current events, new postings, etc.).

Easy maintenance: Frames make it easier to change navigation, if it’s only in one frame (but if pages are dynamically generated this shouldn’t be a big deal). Other maintenance may be easier because page components can be separated.

Easier development: Frames can decrease development time because content can easily be developed independently of navigation. Checking for consistency across pages is also simplified (although frames themselves can sometimes be tricky to implement correctly).
Embedded external sites: Frames can be used to keep people from leaving your site when they click external links, by displaying external pages within a frame.

Increased speed: Pages load faster, although there isn’t a big difference as long as graphics are reused between pages of the nonframe version. The main speed advantage of frames is the reduction of screen refresh time, generally not download time.

Disadvantages of Using Frames
Despite these desirable advantages, frames create a more complicated and confusing user interface, and they introduce new design problems. The frequency with which we observe these usability problems generally leads us to avoid the use of frames.

Confused printing: The print command usually prints the most recently selected frame. In many cases this is a navigation frame, not the desired content frame, which can be extremely confusing to users.

Inefficient use of screen real estate: Frame layout is extremely difficult for small window sizes (anyone with a smaller monitor). For smaller windows, a logo, a navigation bar, and ads can use up half the window or more.

Inability to bookmark subpages: Bookmarks don’t work as users expect, so it’s not possible to bookmark a subpage within a frame. Only the home page can be bookmarked.
Confusing Back button and History behavior: The Back button in the browser does not always behave as users expect (should you return to the previous frame or the previous frameset?), and the History does not show subpages within a frame, so browsing back to previous pages is extremely confusing.

Impracticality for navbar changes: Changing the appearance of the current page in the navbars not practical with frames. This can be done by reloading the whole frameset, but this means you have less of the speed and maintenance advantage of frames.
Incompatibility with redundant text links: We generally recommend including a text version of links so that people with graphics turned off can still navigate.

If you use frames, you will need to do one of the following: only use text links for navigation (less interesting, less customizable); only use graphics links (in which case users can’t easily navigate with images turned off); include text and graphics in your navigation frame (confusing and ugly); or include text navigation at the bottom of your content frame (and thus you don’t get the ease-of-development advantage of frames).

Unpredictable search engine behavior: Sites built with frames don’t interact well with search engines, since an individual frame can be indexed as a page, so that when a user visits the page, they’re missing the other frames in the frameset (such as the navigation).
Complicated keyboard controls: Users can normally scroll through a page with the scroll buttons on the keyboard (arrows, page up/down, home/end).

With frames, this only works when the user explicitly chooses the frame first. In most cases, this is only an inconvenience. For users with motor disabilities, this can make the page significantly less accessible.

Potential problems when scrolling is turned off: If you turn off scrolling in a frame, make sure all the content correctly displays on small monitors and small window sizes.
Problems on small screens: While frames can work on desktop machines, they don’t translate well onto small screens, such as mobile phones and PDAs.

In many cases, the benefits of frames can be duplicated in HTML without frames. This example illustrates how we go about evaluating the use of a new technology – we systematically weigh the tradeoffs of determine whether use of the technology is justified in each situation.

Database Interaction Guidelines
Database and enterprise integration is about providing web access to what are often huge repositories of information. Care should be taken to provide sufficient capabilities in the interface for the users to achieve their goals without being overwhelmed. Some suggested ways to structure a user’s interaction with databases include the following:

Provide Users with Just Enough Search Capability
Databases often contain much more information than we can usefully search. Unless the user really needs advanced search capabilities, Boolean arguments on every data field make the search task overly complicated.

Limit the Number of Returned Records
Don’t just dump out all 500 records in response to a query. Give the user a choice about how many records to return, if possible. Chances are, if the return list is too large, the user will want to refine the search anyway.

Provide Sufficient Information in the Return List
Display search results in a logical order and with enough detail so that the user can distinguish one from another without viewing their details.

Database Development
As discussed, databases are an extremely powerful tool for web development. For lightweight applications (those with relatively few records or infrequent use), a lightweight database engine such as Microsoft Access or Filemaker Pro is just fine.

Databases can be rapidly created in those environments and web applications rapidly prototyped. For heavyweight applications like an e-commerce site serving thousands of requests per second, an industrial strength database management system is required.

High-performance database management systems such as DB2 or Oracle, however, have significant development overhead and typically require highly trained professionals to set up and maintain.