Final prototype

In conducting the usability evaluation for the stated scenario of an organizational website, insuring that all groups needs and planning are needed so that the process can go smoothly. The first factors of information to be addressed in the premeeting stages are to get a high level overview of what is important for each of the groups in addressing usage of the website.

What premeeting information will you include in the invitation you send to the participants?

The first order of business during the premeeting stage is to get an idea of the factors that are the most important to each group, along with sticking points to watch for. Since the premeeting invitation should address a high level overview of planning, during this stage we are not concerned with specific problems. Instead, questions should focus on fundamental information architecture such as navigational overviews, how the website’s information will be organized and the organization of any online help systems (Rubin & Chisnell, 2008).

What plans will you make to ensure that the discussions are fully reported?

            The careful organization of results from the discussions should be a priority. One possibility for keeping things in order and as a way in order to focus on the participant’s responses, having another person take notes is a good idea as well. Insuring that the video equipment is in good working order should be done before the testing so that there are no interruptions from a systems standpoint. In recording the session, making use of time stamps with dates should be incorporated; there should be identifiable numbers as well as dates set up on any videos or DVD created from the sessions (Rubin & Chisnell, 2008).

 What do you intend to cover in the storming session for the meeting?

The process that the participant goes through in interacting with the test prototype should be carefully reviewed. By highlighting areas where the participants were confused or found missing information during the storming session, steps can be taken later to identify and clear them up.

How will you control negative inputs?

During the sessions, the moderator will find participants all have different inclinations which include negative attitudes in regards to the new website. To combat negative inputs, it is explained by Rubin & Chisnell that it is important to have “an open and non-judgmental environment…where the participants feel completely at ease, even if they make mistakes” (p. 219).The moderator should be cautious not to give off a certain vibe that they are judging participants during the evaluations. Any questions asked should be neutrally crafted so as not to lead to any specific conclusions (Rubin & Chisnell, 2008).

Which key topics will you ask the participants to cover and how will you introduce these topics?

The indication of having accessibility issues is the main reason in conducting the evaluation in the first place, so determining if the organization of information and how easy the software is to use is important. The design of graphics, along with clarity of actions while using interfaces as well as insuring that the technical information is correct in the application are also necessary qualifiers for covering with the participants. The use of a pilot test to gauge how effective and correctly applied that the topics are will be a good way to introduce and get an idea of the topics to use (Rubin & Chisnell, 2008).

Which techniques will you use to debrief the participants?

The process of debriefing is used to analyze what actions that the participants took during the test and can be helpful in determining any problems occurred as well as how to correct them. The techniques that will make this process fruitful include making the participants feel as if they are in a comfortable environment in stating what they were thinking as they progressed. Further, any type of reactions from moderators or interrogators should not lead them to a specific answer. Neutrality is key here (Rubin & Chisnell, 2008).

What routines will you follow to ensure that the participants receive a brief yet informative report on the outcomes of the meeting?

After the end of testing, you want to utilize the participants as helpful reviewers so going over the results is an excellent way to review what the findings were and to see if they have any other suggestions on improvement. Leading off with casual questions about how the process went could lead into an open dialog between members of the study and increase the chances of uncovering something that you as the facilitator may have missed. Reiterating on what the participants found surprising as well as go over task singularly are all helpful routines that will yield the best results for this post review (Rubin & Chisnell, 2008).

There are a lot of steps involved in a usability survey, but by following the steps of preplanning, making plans to cover all the necessary information, and careful debriefing processes, having a successful launch of new software will be much easier.

Reference

Rubin, J. & Chisnell, D. (2008). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (2^nd Edition). Indianapolis: Wiley Publishing.

Comparative Analysis Using Inferential Statistics

The process of testing and compiling data is necessary so that the evaluator can see trends in compared data. As the testing is completed each day, the data discovered should be recorded along with audio recordings for easier review later in the process – this is a good facilitation since all the days details will still be easy to recall ((Rubin & Chisnell, 2008). The compilation of data is important so that we can compare the quality of them. No process does this more effectively than an actual comparison of websites and usability factors that go into their design of each. As a result, the purpose of this paper will be to compare two specific websites and by recording timings, accuracy and other usability factors, we can identify benchmarks that could be improved in order to improve our own designs in the future.
The comparisons between two websites that are closely related provide excellent examples of factors that are different in each even though we can select websites for entities in the same classification. By comparing two websites that are in the same industry – for our example we will use hotel websites – we see how immature that the industry is. For example, since web design, accessibility and usability is still a young industry without many standardizations across the board, we will likely see many difference between websites even in the same industry.
To highlight these discrepancies, I wanted to select two hotels that are in the same price range with similar quality ratings and reside in the same city for the most even comparisons. I selected the Fairmont in San Francisco (http://www.fairmont.com/san-francisco/) and the Ritz Carlton hotels (http://www.ritzcarlton.com/en/Properties/SanFrancisco/Default.htm).
There were some differences, mainly in the fact that certain sections of the websites where under different classifications or simply not mentioned at all. For instance, there were no Special services or guest services area of either website, but there was a spa section and a gift shop for both. In addition, I could not locate disabled areas on the websites of either hotel. Overall, I found the Fairmont Hotel to be much faster in fetching pages. In fact I don’t think I have ever seen a website so fast before. On the Fairmont hotel site, most pages opened within 1 second and all the others within 2 seconds. Comparatively, the Ritz-Carlton site took a few moments longer for each section page. In every category, the Fairmont website opened faster.

In order to get equal comparisons for the booking of rooms, I selected the date of September 1st leaving Sept 2nd. Both hotels dealt with the booking procedure similarly with the first click of accommodations off the homepage to go into a rates page where the prices were displayed. The next page of each website allowed guests to enter their names into a forms area. In the actual booking of the rooms, the Ritz-Carlton hotel loaded quicker. It took about 5 seconds for the room prices to come up for the Ritz, while it took 7 seconds for the Fairmont. To move into the next guest registration and credit card page, the Ritz again came out ahead by loading in only 1 second, while the Fairmont loaded the guest page in 3 seconds.
Overall, I would say that the Ritz-Carlton might want to make some changes to their website to capture more of the intrinsic charm of an older stately luxury experience. While both sites were elegant, the Fairmont took the lead in the visual design category due to their color choices and web aesthetics. Both websites were very fast in loading pages; while I noticed the Fairmont’s general, home and services pages loaded quickly, I found that the Ritz got users to their reservation pages faster. As a result, I think that the Ritz has shaped their web experience in getting customers the fastest and most convenient way to be able to reserve a room. One other factor in their design is the interface design of the reservations form on the front homepage. The overall design seems to be a bit easier to comprehend when a user quickly scans it. It is very simple with one field for the location and the travel dates. In addition, I noticed that once the user clicked the “Select a Hotel” field, it changed to a form that is a bit more complex but the initial view makes the user feel more at ease due to its simplicity. Here is the first interface for booking a room, followed by the form choice once the hotel location has been clicked:

First View – Ritz Carlton

Second click at the Ritz brings you to this screen:

The next screenshot shows the form from the Fairmont Hotel:

The additional fields on the Fairmont reservation form make the interface a bit more of a clumsy initial view. It is a busier interface and makes the user cognitively have more to process. As Steve Krug (2005) points out in “Don’t Make Me Think,” the idea of mindless choices are more important, especially to the user that is in a hurry. As an example of simplicity, we find that the first interface of the Ritz is the simpler one and that translates to a better user experience.

By utilizing the concept of inferential statistics, the data suggests that the web host for the Fairmont Hotel is faster in serving pages. Whether this is due to the operating system it uses or other reasons is unclear for the purposes of this report, we will assume this to be the reason. For actual booking of the rooms however, the Ritz-Carlton loaded faster. The backend of the Ritz website is likely a contributing factor as well in presenting the reservation details, however due to the unknown factor of what database is being used it is not possible to use this as a comparison. Both hotels provided a great user experience with easy to use navigation and search capabilities. As far as the visual components, both websites looked elegant, however I found that the Fairmont webpage seemed to have a greater charm to it which lead me to feel that it was a slightly more exclusive hotel. This is likely due to the fact that the Ritz Hotel had a very common user interface pattern and more common color variations which did not suit the feel of luxury as well as at the Fairmont. Both sites loaded quickly however and I do not believe in the long run, either website would make me choose to go with the other hotel if I were shopping them out for pricing. They are very evenly matched.

Ritz Carlton Both sites were quick to serve important details. Ritz had a simpler interface initially and made the booking process easier with a very fast load time.
Fairmont Both sites were quick to serve important details. Fairmont served general pages faster but the reservations were a bit slower and the booking interface a bit too busy.

Reference:

Rubin, J. & Chisnell, D. (2008). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (2nd Edition). Indianapolis: Wiley Publishing.

Krug, S. (2005). Don’t Make Me Think! : A Common Sense Approach to Web usability. Berkeley, Calif : New Riders Pub., ©2006.