Deliverable F-Usability Results
Team
Yellow: Deliverable F-Usability Testing Results
Mykle Brossette,
Brian Green, Satyam Kaneria & Jessica LoPresti
ECT 455 Autumn 2011
_______________________________________________________________________
PERSONAS
Primary
Persona 1
Name: “Amanda Hoel”
Age: 26
Profession: e-Marketing
Manager
Type of
Persona: Primary
Date and
Time of Testing: 11/5/2011 at 3 p.m.
Duration
of Test: 25 minutes
Primary
Persona 2
Name: “Viral
Patel”
Age: 25
Profession: Graduate
Research Assistant
Type of
Persona: Primary
Date and
Time of Testing: 11/5/2011 at 2:34pm
Duration
of Test: 17 minutes
Secondary
Persona
Name: “Sharon
Day”
Age: 38
Profession: Unemployed
Architect and Mother of Two
Type of
Persona: Secondary
Date and
Time of Testing: 11/4/2011
Duration
of Test: 30 minutes
Negative
Persona
Name: “Jack
Bishop”
Age: 62
Profession: Artist
and Author
Type of
Persona: Negative
Date and
Time of Testing: 11/4/2011 at 9:30am
Duration
of Test: 25 minutes
PRE-TEST
QUESTIONNAIRE RESULTS
1.
Do you currently have any pets?
Primary 1:
YES
Primary 2:
YES
Secondary: NO
Negative: NO
2. Are you or is someone you know looking to adopt a pet?
Primary 1:
YES
Primary 2:
YES
Secondary: NO
Negative: NO
3. Do you use a computer for work and/or personal use?
Primary 1:
YES
Primary 2:
YES
Secondary: YES
Negative: YES
4. If yes, what type of activities do you use the computer for?
Primary 1: Web
Copywriting, Attending Virtual Meetings, Emails, Searching the Web, Social
Networking, Entertainment, Word Processing, Online Banking, Video Chat, and
more.
Primary 2:
Word
Processing, Email, Banking, Web Search, and College Projects, Work
Secondary: Word
Processing, Emails, Searching the Web, Social Networking and Drafting
Negative: Word Processing, Email, Searching the Web, Social
Networking and Illustration
5. Rate your level of experience in using the Internet:
Primary 1: Advanced
Primary 2:
Advanced
Secondary: Advanced
Negative: Intermediate
6. How many hours a week
do you spend viewing websites online?
Primary 1: 30+ hours
– “my entire life is spent viewing websites.”
Primary 2:
5+
hours
Secondary: 3-4
hours
Negative: 5+ hours
7. Do you own a smartphone?
Primary 1: YES – iPhone
Primary 2:
YES
Secondary: YES
Negative: NO
8. If yes, what do you typically use your smartphone for (besides
placing calls)?
Primary 1: Emails,
Texting, Web Search, Social Networking, Taking Pictures, Streaming Video
Primary: Email, Web
Search, and Camera
Secondary: Emails,
Searching the Web, Social Networking, Directions, Toddler watch, To-Do List,
Appointments, Music, Camera, Voice Memos
Negative: N/A
9. What age group do you belong to?
Primary 1:
18-29
Primary 2:
18-29
Secondary: 30- 39
Negative: 60-69
10. What is the highest level of education you completed?
Primary 1:
Bachelor’s Degree in Communication, Master’s in Management and Organizations,
Master’s in New Media Studies
Primary 2:
Graduate
Degree in Mechanical Engineering
Secondary:
Bachelor’s Degree in Architecture
Negative: Graduate Degree in International Business Management
INITIAL
PROTOTYPE REACTION
WEBSITE:
1. What is your first reaction to the site?
Primary 1:
“Very
clean and I like the way it looks.”
Primary 2:
“Looks like it has information that I may
need if I wanted to adopt a pet.”
Secondary: “It’s
cute. Like the pictures and logo. Kids on the logo look like
they’re turning into a dog; evolutionary chart.”
Negative: “Instead
of JOIN NOW, I’d rather see BEGIN NOW.” “Quick Search section
too small.”
2. What is appealing to you and why?
Primary 1:
“It’s
user-friendly and simple to use.”
Primary 2:
“Personality quiz is interesting as it
will help me find the right pet.”
Secondary: “Colors
are nice and cheery.”
Negative: “Pleasant
design, friendly colors…”
3. Who is this website built for?
Primary 1: “People
who are looking for an animal.”
Primary 2:
“Cat and Dog lovers and owners.”
Secondary: “People
looking for pets.”
Negative: “Women
and children.”
4. Does this seem like a website designed for someone like you?
Primary 1: “Yes.”
Primary 2:
“Yes. It does.”
Secondary: “If I
was looking for a dog or cat. I guess so.”
Negative: “If I
was looking for a pet…sure, but I’m not.”
MOBILE
1. What is your first reaction to the site?
Primary 1:
“I
thought it was very clean but the advertisements and navigation buttons need to
be moved so you can use the site.”
Primary 2:
“It is good because it is easy to
navigate and has few and only required options.”
Secondary: “Cute.”
Negative: “I
don’t understand the difference between ADOPT/FIND A PET and MATCH ME. They
both seem like a way to search for a pet.”
2. What is appealing to you and why?
Primary 1: “It’s an
easy way to find what you’re looking for.”
Primary 2:
“It saves me time and is faster on my
phone.”
Secondary: “The
Dogs and Cats. I like them.”
Negative: “Logo…Inviting
design”
3. Who is this website built for?
Primary 1: “People
who are looking for a pet and don’t own an iPhone.”
Primary 2:
“For people who are looking for Cats
or Dogs on the go.”
Secondary: “Someone looking for a pet.”
Negative: “Anyone
interested in pets with a smartphone.”
4. Does this seem like a website designed for someone like you?
Primary 1:
“Yes.”
Primary 2:
“Yes.”
Secondary: “Yes.”
Negative: “No, I
don’t own a smartphone. Also, design seems geared toward women and kids.”
TEST
RESULTS
WEBSITE
TASK 1: Create a user account
Time of Completion:
Primary 1: 16 secs
Primary 2: 40 secs
Secondary:
47.7 secs
Negative: 1min 12 secs
Correct Sequence of Actions taken to complete task?
Primary 1: YES
Primary 2: YES
Secondary: NO. Technical Difficulties with site.
Negative: YES
NOTES:
Primary 1: Everything was easy to find
and the form was very brief. I’m
surprised it didn’t ask me to confirm my password via email.
Primary 2: It just took a moment to
understand the flow of menus and buttons
Secondary: There were
computer problems. System timed out or something.
Negative: User
could not see the field boxes at first. “Too light of an outline”.
TASK 2: Conduct a Quick Search for a
pet
Time of Completion:
Primary 1: 10 seconds
Primary 2: 19 secs
Secondary: 1 minute
and 58 seconds
Negative: 21 secs
Correct Sequence of Actions taken to complete task?
Primary 1: YES
Primary 2: YES
Secondary: NO
Negative: YES
NOTES:
Primary 1: User searched for a
medium-sized male cat and got a dog. And the dog described in the breed section
is wrong. User notes the task is easy to perform, though.
Primary 2: Got 1 result. User liked the
overall idea.
Secondary: Why is
“Match Me” a button on search? Not happy about the errors when trying to use
the Adopt/Find link.
Negative: Task seemed
straightforward and easy
TASK 3: Fill out pet personality
matching quiz
Time of Completion:
Primary 1: 22 seconds
Primary 2: 50 secs
Secondary: 1 minute
74 seconds
Negative: 40 secs
Correct Sequence of Actions taken to complete task?
Primary 1: YES
Primary 2: YES
Secondary: NO
Negative: YES
NOTES:
Primary 1: The results display should
be reorganized, it’s just a long list. The process is
pretty easy. Again, the results should match the search parameters.
Primary 2: User was slow in typing that
slowed him compared to other tasks. The user was injured.
Secondary: The “Join
Now” button is very confusing. She thought she needed to sign-in again in order
to start the quiz. Said aloud, “This is very confusing.
I don’t know what to do.” The “Match Me” button the Quick Search confused her
and she thought she would match herself to a pet using quick search.
Negative: User
noticed the photo of the dog did not match the accompanying description.
MOBILE
TASK 1: Create a user account
Time of Completion:
Primary 1: 1 min
Primary 2: 34 secs
Secondary:
Negative: 1min 16 secs
Correct Sequence of Actions taken to complete task?
Primary 1: YES, 4th time
Primary 2: YES
Secondary: YES
Negative: YES
NOTES:
Primary 1: It’s confusing how navigation
buttons are floating over the entry form, had to click
several times to get access to the forms. Also kept receiving
error messages that passwords didn’t match due to input errors – very
frustrating. “I hate typing on my phone.” Also when I enter a password
on the web, I expect to have it blocked out.
Primary 2:
Secondary: Participant
found it easy.
Negative: Standard form
TASK 2: Sign-in to mobile site and take
Pet Match quiz
Time of Completion:
Primary 1: 1 min
Primary 2: 20 secs
Secondary: 23.6
seconds
Negative: 1min 34 secs
Correct Sequence of Actions taken to complete task?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: YES
NOTES:
Primary 1: User able to sign in with no
trouble. User notes that this time her password entry was obscured. The
navigation buttons were blocking some of the fields on the Pet Match quiz, so
she couldn’t choose different options. The search results page only displayed a
large picture of a cat.
Primary: User received the “Hello, ‘email’”
that he liked as it acknowledged the login.
Secondary: Easy
Negative: User received message ‘Login Failed’ but
received a welcome message acknowledging his user name after he took the quiz.
User noticed there was no option for ‘BOTH’ or ‘ALL’ in the dropdown menus when
choosing gender, breed, age and size.
TASK 3: Search for a Siamese cat in zip
code 60660
Time of Completion:
Primary 1: 10 secs
Primary 2: 28 secs
Secondary: 74 secs
Negative: 53 secs
Correct Sequence of Actions taken to complete task?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: YES
NOTES:
Primary 1: “Can’t select gender or the
size because of Avoderm ad. Nobody will know what
“Location” refers to – if it’s a zip code you want, it should say zip code.”
The search results are just a picture, nothing to tell me it’s a Siamese cat or
its location.
Primary 2: User was able to find a
Female cat in 60660. User selected age as 2.
Secondary: Entered in
steps to take. Technical difficulties otherwise.
Negative: User only had to enter the 60660 location.
Siamese was the default option. There was no search result photo.
USER
SATISFACTION QUESTIONNAIRE
WEBSITE
TASK 1: Create a user account
1) Did you complete the task quickly and easily?
Primary 1: YES
Primary 2: YES
Secondary: No
Negative: YES
2) Was it the information you expected?
Primary 1: YES
Primary 2: YES
Secondary: NO
Negative: YES
3) Are you pleased with the steps taken?
Primary 1: YES
Primary 2: YES
Secondary: No
Negative: YES
TASK 2: Conduct a Quick Search for a
pet
1) Did you complete the task quickly and easily?
Primary 1: YES
Primary 2: YES
Secondary: Yes
Negative: YES
2) Was it the information you expected?
Primary 1: NO
Primary 2: YES
Secondary: YES?
Negative: YES
3) Are you pleased with the steps taken?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: YES
TASK 3: Fill out pet personality
matching quiz
1) Did you complete the task quickly and easily?
Primary 1: YES
Primary 2: YES
Secondary: NO
Negative: YES
2) Was it the information you expected?
Primary 1: YES
Primary 2: NO – Expected search result
but did not get any pets listed.
Secondary: NO
Negative: YES
3) Are you pleased with the steps taken?
Primary 1: YES
Primary 2: YES
Secondary: NO
Negative: YES
MOBILE
TASK 1: Create a user account
1) Did you complete the task quickly and easily?
Primary 1: YES, but had a hard time
entering the information because the images covered one of the password
entries.
Primary 2: YES
Secondary: YES
Negative: YES
2) Was it the information you expected?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: YES
3) Are you pleased with the steps taken?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: YES
TASK 2: Sign-in to mobile site and take
Pet Match quiz
1) Did you complete the task quickly and easily?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: NO-Login Failed
2) Was it the information you expected?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: Not enough matching questions to be a valid
feature
3) Are you pleased with the steps taken?
Primary 1: YES
Primary 2: YES
Secondary: YES
Negative: NO
TASK 3: Search for a Siamese cat in zip
code 60660
1) Did you complete the task quickly and easily?
Primary 1: YES
Primary 2: YES
Secondary:
Negative: YES
2) Was it the information you expected?
Primary 1: Somewhat – it was a picture
of a cat, but not a Siamese cat, and there was no information about the cat.
Primary 2: YES
Secondary:
Negative: YES
3) Are you pleased with the steps taken?
Primary 1: YES
Primary 2: YES
Secondary:
Negative: YES
DEBRIEFING
WEBSITE/MOBILE
1) What do you think of your first experience with our site?
Primary 1: I’m happy with it overall.
Once the information that displays in search results is correct, it will be
great.
Primary 2: User friendly and easy to navigate.
Secondary: Confusing.
Mobile was easier to use.
Negative: I prefer the website over the mobile because I
don’t own a smartphone.
2) What was the most useful aspect of the site?
Primary 1: The personality quiz.
Primary 2: Pet matching by taking a quiz.
Secondary: Matching
on Mobile. Creating an account.
Negative: The search function
on the mobile and website.
3) What aspects of the site were ineffective?
Primary 1: The incorrect information
displayed throughout the search results and the images covering the mobile
entry fields.
Primary 2: Pet Match quiz did not give results on Website and Mobile site did not
protect password.
Secondary: Didn’t
know where to go. Asked for specifics but not given what asked for in return.
Negative: On the website, the pictures and text were
not big enough…hard to see/read. On the mobile site, main menu options seemed
the same and the drop down menus did not list all available search options.
4) What improvements do you suggest for the site?
Primary 1: See answer to question 3.
Primary 2: Removing the ineffective pieces of the sites mentioned above would make
it a pretty decent site.
Secondary: Not Sure.
Negative: See answer to last question. Also, add motion
to logo (i.e. have the kids and dog run across the top of the screen.
SUMMARY
Primary
Persona: The user was very happy with how the sites were organized and the
visual look and feel. The tasks were easy to complete and once the correct
information displays and more pets are added to the database, it will be a
great site.
Primary
Persona: Viral is a hard working and passively
critical about the sites that he visits. His perception of the sites for pets
was very positive just because of the fact that people are putting in effort.
Overall, he really liked the design, layout and simplicity of sites. He
completed all the tasks very easily; however, felt that the some of the site
features are still in development at the time. Both the website and mobile site
took him just a moment to figure out how it would work.
Secondary
Persona: The mobile site testing was more satisfactory then the website.
This could have been because the participant ran through similar tests on the
website, or the website had more technical issues than the mobile. Overall, the
participant was confused on how to complete the “Match Me” task and the
“Search” task due to ambiguous copywriting on the site.
Negative
Persona: “Jack” completed each task of the website easily and quickly. He
thought creating an account was standard, although, he didn’t see the data
field boxes at first. Filling out the Pet Match Personality Quiz and conducting
a Quick Search to find a pet was simple and clear. His only recommendations for
improvement were increasing the size of text and photos, particularly for
people in his age group and older.
For the mobile site, “Jack” was unclear at the difference between
the two menu options: Adopt/Find a Pet and Match Me. He thought ‘Match Me’ did
not indicate that was how to take the Pet Match quiz. He recommended “Pet Match
quiz” as a more appropriate title. As with the website, creating an account was
simple and standard. He had some trouble initially with the sign-in…he received
a login error even thought he entered the same info
he just created the account with. Also, he thought there were not enough
questions to the Pet Match quiz to truly result in valid and customized
matches. Finally, searching for a Siamese cat in 60660 was a simple task,
although, there was no photo or description provided in the search results.
LIST
OF CHANGES
Website:
·
Larger text and pictures on search results
page
·
Increase size of Quick Search section
·
Make create acct field boxes more prominent
·
Add dynamic content such as animated logo,
video, etc…
·
Remove the “Join Now” button on Pet
Personality and have a “Start” button.
·
Change the “Match Me” button on quick search
to say “Search”.
·
Remove the drop-down options on quick search.
User feels they need to be specific in order to search.
Mobile:
·
Ensure drop down menu options are appropriate
for each question
·
Add more questions to matching feature
·
Ensure the navigation buttons and
advertisements don’t display over the entry fields
MEMBER
CONTRIBUTIONS:
Mykle Brossette
·
Tested the Secondary Persona for Web and
Mobile
Brian Green
·
Tested the Primary Persona 1 for Web and
Mobile
Satyam Kaneria
·
Tested the Primary Persona 2 for Web and
Mobile
Jessica LoPresti
·
Tested the Negative Persona for Web and Mobile
·
Created Template for Final Document
URL To
Deliverable F:
http://ectweb2.cs.depaul.edu/yellow46/deliverables/deliverableF.html