4.5  Data analysis

 

A thematic analysis was chosen to analyze the data. A thematic analysis usually involves the researcher preparing transcripts, reading and re-reading the transcripts the a few times, to identify recurring ideas or potential omissions, a process known as ‘coding’ after which the extracted codes are abstracted into themes (Rosenthal, 2016). Thematic analysis ideally involves uncovering three to five themes and so triangulation may be necessary to make a decision on the final set of themes that will be used (Rosenthal, 2016). To complement the independent thematic analysis, the Usability Problem Taxonomy model is used to classify the data into categories and subcategories which will be compared with the independent thematic analysis in order to come up with a finalized, reasonable and trustworthy themes.  The first step in analyzing the data is transcribing it as without transcriptions, an in-depth analysis is hard to achieve (Rosenthal, 2016) so transcriptions of all interviews and observations were prepared by the researcher in a word-processing program. The next step involves reading and re-reading the transcripts in order to locate concepts that repeat themselves and frequently has to do with the researcher marking comments made by the participants and taking notes (Rosenthal, 2016). I have strived to follow this step and some of the extracted interviewee’ comments and ideas/concepts are revealed into the findings section where they were grouped into themes. The final step involves the researcher abstracting the codes that he or she has found into themes (Rosenthal, 2016). The codes were sifted through and were reduced to only six themes. Due to the qualitative nature of the research, a discovery process is strived for and no predefined measures or hypotheses are initially presented (Schutt, 2001) although previous literature is reviewed in order to set the proper background and compare and contrast the resulting findings.

 

 

The Usability Problem Taxonomy model is used to organize and/or classify different usability issues in order to facilitate analysis (Keenan et al, 1999). UPT allows for classification using direct observations, which makes it a suitable addition to the research (Keenan et al, 1999). Its structure is based on the notion that usability issues should be divided into artifact perspective and task perspective. The former perspective focuses on the ways in which the user examines, views, reads, understands, or manipulates the different parts of the user interface while the latter focuses on issues emerging when the user progresses through different tasks that involve the user interface (Keenan et al, 1999). An assumed contribution of UPT in terms of this research is its ability to give a new perspective on what the problems surrounding the usability of the chosen design patterns are and its ability to present a different approach to identifying problem clusters (Keenan et al, 1999). UPT was used to evaluate the usability of a diabetes mHealth system and initially 50% of the usability problems were detected, followed by an addition of 29% during the post-interviews with the participants (Georgsson and Staggers, 2016). This makes the framework suitable for this research as it can bring a new perspective on the horizon and help identify usability issues stemming from the design patterns and the observation of the users.         

 

4.6  Validity and Reliability

To maintain the reliability of the research, the interviews and the observations were recorded and transcribed verbatim. Afterwards, the transcriptions were checked for possible mistakes. Due to a personal connection of the researcher with the participants some additional measures to ensure validity were also employed - two separate information gathering techniques were utilized to identify themes (interviews and observations) and the extracted themes were shown to the participants so that they can confirm that the analysis correctly represents what they have said (Creswel, 2009) 

4.6.1 Internal Validity/Credibility

 

Internal validity tackles the issue of how compatible the research findings and reality are. In other words, it examines how accurately the results depict reality. However, the qualitative, interpretive viewpoint of this study assumes a plurality of world views which makes the original understanding of internal validity unsuitable. Therefore, it is best to talk about “credibility” or how credible the research findings are considering the gathered data (Meriam and Tisdell, 2015, p.242).

 

To ensure the credibility of the research, a strategy for adequate engagement in data collection will be maintained and differing worldviews would be examined.  Despite the commonalities amongst the participants (nationality, age group, education) the participants’ differ on many levels, in differing degrees, such as worldviews, interests, type of higher degree/specialty, place of residence, Internet usage time, experience with technology, age. This increases the credibility of the research as the findings would not be coming from a single source/perspective (Meriam and Tisdell, 2015, p. 245)

 

 

Another manner in which the credibility of the research is by deciding how many people need to be interviewed in order to grasp fully the understanding of the participants of the phenomena under examination (Meriam and Tisdell, 2015, p. 246). To achieve this, I collected data until the data felt saturated in the sense that I kept landing upon the same information when a new participant was included.

4.6.2 External Validity/Transferability

 

The application of a research findings in different situations is known as external validity, or generalizability. In the case of qualitative studies, it is more appropriate to talk about transferability (Meriam and Tisdell, 2015, p. 256) where the application of the study to different situations is in the hands of the reader. The way to ensure that a level of transferability is maintained in a qualitative study is to provide readers with detailed, contextual information so that they can decide upon the study’s transferability (Meriam and Tisdell, 2015, p. 256). This has been strived for in this research by elaborating about the research setting, the participants involved in the study and by backing up the study’s findings with quotes from the participants’ responses (Meriam and Tisdell, 2015, p. 257). 

 

4.7  Ethical considerations

 

A research should be undertaken with attention to certain rules of ethical conduct (Lichtman, 2010) which are followed in this study. The behavior that is strived in this study can be summed into “do good and avoid evil” (Lichtman, 2010, p.54). The following points have been paid attention to in terms of ethical conduct:

1. Do not cause harm

 

Should any potential harms emerging from the research be envisaged, the participants reasonably expect to be informed of them (Lichtman, 2006). I envisage no harms to the participants from the research but I will remain vigilant and stop the interview or observation without undue delay should an adverse reaction emerges (Lichtman, 2010, p.54).

 

2.  Provide informed consent                         

 

Participants in such a research have a reasonable expectation that they will be informed about the nature of the research so that they can decide whether to hop in or not (Lichtman, 2006).  That is why the participants chosen for the interviews were provided with a consent form which they could read and sign before the actual interview and observation (the consent form can be seen in Appendix A).

 

They were informed that the interview and the observation are entirely voluntary and that they can withdraw from participation before or during the actual interview. The participants were also informed that the audio contents of the interviews are going to be recorded and that the recordings are not going to be shared with third-parties but will only be used for the purposes of the current research.

 

3.  Provide a sufficient degree of confidentiality and privacy

 

No names and personally-identifiable data of the participants were shared with someone other than the researcher, his supervisor and his examiner. Keeping the privacy and the confidentiality of the participants by removing potentially identifying information is naturally expected by the participants and an explicit permission from the participants is typically necessary should such information be publicly available (Lichtman, 2006).

 

4. Avoid unsuitable behavior

 

 

The participants’ personal lives, space or time would be respected. Participants would be informed of the time that the study could take and their personal space would not be intruded as the interviews and observations would take place entirely online with no requirement for their physical location.  Special attention would be paid not to touch upon subjects which could be sensitive for the participants (Lichtman, 2010, p.57).

 

5 Findings

 

In this chapter, I will focus on the findings of the research resulting from the thematic analysis and I will briefly cover some of the specific usability problems that were found with the chosen website user interface design patterns

 

 

 

 

5.1           Themes

 

The extracted themes will be mentioned below, backed by excerpts from the interviews of the participants, reflections from the observations and author comments giving the context and the reason behind the extracted theme. The names of the participants have been changed in order to preserve their confidentiality. 

5.1.1 Usability issues surrounding the “Home” text

 

Four of the six participants stated that the text “Home” for returning to the homepage is not the best option, and three of them were not explicitly asked to compare “Home” and “Homepage” but were asked what kind of text they would like to see for that action.

 

Penyo stated that ‘homepage’ is the most suitable text as ‘home’ is vague:

Interviewee: I think the best would be if it states homepage because home is a little bit strange, I mean it should say exactly homepage, so you should be sure that you go to the homepage because home you can go to a special section for home interior, for example, and you may not want to do so.   

 

Pesho believes that the button Home may be hard to relate for some users and it should be known as Start page or Starting page:

“Because in computers it’s homepage but usually It should be called the starting page, the start page, but we call it homepage which is different so maybe they may not relate the button home with the starting page...”

In line with the previous participant who thought ‘Home’ is confusing, Pesho thinks that the text label is vague and is not satisfied with the given naming which could lessen the usability due to low “information scent” or efficiency/accuracy (as Penyo stated about the Home button -  is it the “Home interior” page?)

 

Mariah believes that the text “Homepage” is a better alternative than “Home” as the former “sounds better” to her:

“…. I mean, maybe ‘homepage’ would be better.

Researcher: Can you think of why you think so?

Interviewee: Well, maybe because websites have so many different menus and each menu goes to a different thing and at the end you might find yourself a little bit confused and if you want to go to the homepage you would press the button that says so.

Researcher: Yes, but I mean why ‘homepage’ is better than ‘home’, can you…

Interviewee: Well, actually, there isn’t a reason why that I could explain. It just sounds better to me.”

 

Thus, using the text ‘Home’ to pinpoint a link to the homepage may bring forward information scent and recognition issues.

5.1.2 Substitute ‘the hamburger’ icon with text for greater usability

 

Two (2) of the six (6) participants think that the ‘hamburger’ icon is confusing and makes no sense while all six (6) agree that the icon, or action, could be made better.

Four (4) out of the six (6) participants stated that a text-based alternative for the ‘hamburger’ icon would be the way to go while 3 of them explicitly stated that ‘Menu’ is the clearest and the most user-friendly way to inform the user about the presence of the hidden menu.

 

Penyo is of the opinion that a text Menu would be much more suitable for opening up the navigation. He further believes that the picture does not imply and is not related to the activity that is performed upon a click on the icon which is:

Interviewee: I think the best solution is to use a picture that implies the activity related to the icon. The picture should be clear and should not be confusing.

 

The participant understands implicitly the need for recognizable, easy to learn icons with good information scent and believes the icon is not doing its job.

 

According to him, the ‘hamburger’ icon does not look like a part of the website:

Interviewee: And if you put the text Menu – a small text Menu – or something like this so that the people can see also text because sometimes it should have only these lines and people can’t think that it is a part of the site but it is not special menu.

 

Hence, according to him, the icon may not only have problems with its recognition easiness and its information scent but it is also hard to find and locate because the icon, as placed in a website, may look like it is nothing of interest.

 

Pesho states that he has no idea why they designed the icon like that and that he did not understand it at first:

“...because I really do not know the origin of the three lines and the navigation just do not ring a bell when I look at it. I did not understand it at first.”

“I really do not know why did they design it like that.”

 

 He believes a much better alternative would be an icon that is relatable (we can put this to the recognition part of the icon’s usability) to the user such as the gear icon (shown for Settings), a ship or a compass which would “ring a bell”:

“Yeah, if the icon was not three lines but it was the settings icon. For example, the gear wheel it would have been a lot easier.”

The participant implicitly states that the ‘hamburger icon’ has low learnability because it is not recognizable (Bedford, 2016):

“I have understood it by experiencing it but I have not understood it by its looks.”

This pinpoints that the icon is not only hard to recognize but also has issues with the “information scent” (Bedford, 2016) or the user’s awareness of what happens upon the icon’s click, according to this particular participant.

 

Alex believes that a Menu text would be more convenient, easier and logical but he also believes that the ‘hamburger’ icon is now widely used and has turned simple for him, most likely due to his extensive usage of the icon.

“If it is written “Menu” it will be convenient, of course, but that’s something which is now very common and it’s a simple menu”

“Probably if a menu button is … it would be easier and it would be in written form so you can just know where the menu is and click on it.”

 

The ‘Menu’ text seems like the first enhancement to the icon that came into his mind. It is possible that several of the participants specifically focused on the “Menu text” because they believe that such a text would increase the learnability, recognition and information scent of the task because whenever a user sees it - he would know what he can do with it.

 

 

Smith first stated that the ‘hamburger’ icon is very recognizable (and so usable) but then thought and came to the opinion that it is not evident what it does as there are no “clues” that it is a menu icon. Thus, his experience with the icon has lead him to initially believe that the icon is self-explanatory but after reflecting on this - he realized that it was really not.  The experience with it had just given the icon a status of a tool that he uses in his daily browsing and was transformed to be perfectly usable. 

“Researcher: I see. Do you have something else that you would prefer, that you think would be, that you would like more?

Interviewee: I don’t know, I think it’s very recognizable.

Researcher: I see, you think it’s recognizable. Do you think it’s easy to learn and why do you think so. I mean to learn what the icon does?

Interviewee: If you look it like that, if you haven’t seen what it is doing before, normally I wouldn’t understand what it does. Like, it doesn’t have any clues that it’s a menu icon.”

 

Mariah thinks the ‘hamburger’ icon is easy once you get the hang of it but initially it is not very recognizable. She also believes that using words is more user-friendly and adding a text “Menu” would make it more recognizable for her and others.

 

“Researcher: Yeah. How easy do you think it is to learn what the icon does and why?

Interviewee: Uhm, it might be actually pretty easy because once you see it many times on different sites and you click on it – you usually get the concept. I don’t know, maybe at first I didn’t know what it did but when I started using more and more Internet and more websites I saw that the button pretty much does the same thing in every site.

Researcher: Yeah, but you think at first is difficult or? What do you think?

Interviewee: Yeah, maybe at first, maybe the first couple of times that you see it but after that you get used to it, after that it’s pretty recognizable but at first not so much.

Researcher: Yeah, I think so. So, what changes you have in mind for the icon to be easier to understand and use, at least in the beginning. To be more recognizable.

Interviewee:  To be more recognizable, well, I think that maybe just, for example, if it is in a little square and inside the square it says Menu or something like this, usually it’s more user-friendly if people click on words that they understand. So maybe if it is not just the symbol or an icon and it says an actual word – then it would be more recognizable.”

 

Angelina believes that the clearest way to trigger a navigation would just be to write what the button will do with text although she pinpoints text other than “Menu”:

“The most clear way to mark an additional settings menu or whatever menu is to just write it - other options or additional or whatever. The written is the most clear way for the user to find out what they are searching for”.

 

Otherwise, they do not have problems with using the icon and find it usable enough:

 

Pesho stated:

“it’s easy to get it but I do not know if people who are not used to smartphone usage, who do not use smartphones a lot, will get it because getting the menu could be kind of difficult.”, Alex stated “...If it is written “Menu” it will be convenient, of course, but that’s something which is now very common and it’s a simple menu. “and this process repeated itself in a slightly different way across the participants.

 

5.1.3 Dissatisfaction with CAPTCHAs

 

Three (3) of the six (6) participants expressed very negative wording towards CAPTCHAs in general while all six (6) participants were dissatisfied to different extents and were of the opinion that they slowed them down and had a negative effect on their experience.

 

Penyo has seen both CAPTCHAs which are okay to handle and CAPTCHAs which has forced him to abandon a website entirely:

“Researcher: And how do you think that a CAPTCHA affects your speed when dealing with websites?

Interviewee: Sometimes you can find CAPTCHA that is extremely complex to understand, to see the signs. I mean and because of this you may need to spend a lot of time trying to guess the combination and as a result you may simply decide to skip the site and go to another site – and not use the site. Because I have several similar cases, I saw a CAPTCHA – it was complex. I, perhaps, tried to find the right number for 5 times or 6 times – it was not possible to do so and at the end I said “sorry, goodbye, I am not going to stay in this site”. So, it is very important to have good CAPTCHA.”

 

 

Pesho also thinks that some CAPTCHAs are difficult (those with noise in the picture):

“Interviewee: But there are some CAPTCHAs which have noise in the picture and  sometimes they are very difficult to discern.

Researcher: So, you think CAPTCHAs affect your speed negatively, yes?

Interviewee: Yes, for sure.”

 

 

Smith  finds the CAPTCHAs with letters and numbers to be often “very hard to recognize” and he even stated that he hates them:

“Researcher: And how do you think it affects your speed when dealing with websites?

Interviewee: I hate it.

Researcher: You hate it. So it’s not easy to complete the different CAPTCHAs?

Interviewee: Sometimes it’s easy but there are some very, very difficult ones. You can easily mistake the words or numbers.”

 

Maria admits that CAPTCHAs can not only slow her down but also distract her from her tasks:

“And when you are at work, you have an agenda that you have to follow so these kinds of things may slow you down or may even distract you?”

 

Angelina finds them boring and confusing and finds them quite negative:

“Interviewee: They’re just boring (CAPTCHAs).

Researcher: You find them boring?

Interviewee: Yes, and frustrating, when you have to work… Usually, it happens to me at work.”

 

On CAPTCHAs in general, Alex states that he finds them “...disturbing, annoying and I do not like it [them]. ” (speaking about CAPTCHA).

 

Thus, the participants perceive CAPTCHAs as having a mostly negative effect on their usability due to their low efficiency and low perceived satisfaction.

 

5.1.4 The usable type of CAPTCHA is the shortest to complete

 

Three (3) of the six (6) participants explicitly stated that they preferred the quickest CAPTCHAs while the rest did not mention why exactly they found certain types of CAPTCHAs easy.

 

Mariah explicitly states that she would prefer the CAPTCHAs which require less time to complete:

“I don’t know, maybe the thing that you have to type and click the least, that’s what you need.”

 

Alex explicitly states that wants a CAPTCHA that takes minimal amounts of time:

“Researcher: So, what do you think is annoying with pictures, is hard?

Interviewee: Because you are losing so much time to focus on the picture and to figure out which one exactly you have to select. Because sometimes they are asking about 3 photos or from 9 photos you have to select the photos with water in them and you have to look in all 9 photos in order to figure out which ones you should neglect.

Researcher: Yes, that’s interesting. And, can you think of some ways that would ease the completion of such CAPTCHAs?

Interviewee: Say it again?

Researcher: Can you think of some ways that would ease the completion of such CAPTCHAs?

Interviewee: Well, I think that by clicking on something only it would be very effective in terms of time.  It would be a faster way.

Researcher: Yeah. And which CAPTCHAs did you find easy, satisfying and quick to complete. Like the easiest?

Interviewee: The easiest is by clicking on something like “Do you agree with the terms” and then press submit and continue.”

 

 

Smith also declared that he prefers the CAPTCHAs which take less time:

“Researcher: So, why did you like it the most?

Interviewee: Because it’s quicker and I don’t have to repeat it. Sometimes I enter a normal CAPTCHA, I make a mistake and after that I have to retype the password or some other information.

Researcher: And, it takes more time, yeah?

Interviewee: Yeah.”

 

CAPTCHAs which entertain such as the ones where you have to select a particular photos from a larger set of photos were only explicitly preferred by Penyo (one (1) of the six (6) participants) and Mariah stated that they are entertaining but would prefer the shortest ones, in most cases (for example, when she is at work).

 

Hence, users associate the satisfaction (and usability) of CAPTCHAs mainly with one factor - efficiency or ease of use/completion which itself depends on their recognizability.  There are many different types of CAPTCHAs and each differs from the other so learnability is not something to strive for.

 

5.1.5  A lack of characteristics of CAPTCHAs that are equally usable

 

One (1) of the participants preferred having a simple arithmetic to solve, three (3) prefer the letter and numbers but (1) one of those three (3) prefers real-life objects to be shown while the second stated that real-life objects are very hard complete and two (2) prefer the “innovative” CAPTCHAs  - the first prefers ticking a box while the latter prefers sliding down or left. Thus, it can easily be seen that at the moment there is no single type of CAPTCHA which could be shown to users and satisfy a diverse user base.

 

 

Penyo preferred CAPTCHAs with letters and numbers which are not “signs” like objects from real-life and are not stretched/blurred or rotated:

“Researcher: Yes, I see. And so, this is the easier is where you only have to reverse, or, the easiest CAPTCHA that you have done?

Interviewee: It is important that the CAPTCHA shouldn’t be a photo. It should be simply a sign. It shouldn‘t be a photo

Researcher: But, what do you mean a sign. If it’s not a photo…

Interviewee: A sign, for example, you can see here. There is a CAPTCHA. Okay, I am going to show, yeah?

Researcher: Yes, okay.

Interviewee: I am going to show you, so you can understand  the thing I would like to say.

(The interviewee shows photos of CAPTCHAs (which are photos) but do not look like a photo – just letters/numbers that are only slightly rotated on a clear background)

 

Figure 4: Type of CAPTCHA that is preferred by the participant Penyo

(Figure 4 shows the CAPTCHA which Penyo has shown as an example of a CAPTCHA which he personally prefers)

Interviewee: It should be, I think, like this…

Interviewee: You see, you can see it. It shouldn’t be. Let me show you…

Researcher: Yes, I understood completely.

Interviewee: Yes, something like this, or like this.

(Interviewee shows CAPTCHAs which look like photos taken from a street/house number)

Figure 5: Type of CAPTCHA that is difficult for the partcipant Penyo

(Figure 5 shows the CAPTCHA that Penyo has shown as an example of a CAPTCHA which is difficult to complete)

Researcher: It shouldn’t be something like this, yeah?

Interviewee: Yes, because sometimes you cannot see it. And it also shouldn’t be too complex. I mean, the signs shouldn’t be too complex, like this for example. You can see here. This is now a problem. You spend a lot of time and sometimes you can make mistake because it is misleading.”

 

Pesho also believes that numbers are a better choice for CAPTCHAs in terms of users, that CAPTCHAs with noise are difficult.  However, the easiest CAPTCHA should contain a real-life alternative (like a car’s number, a picture of dogs and so on):

“but it’s very easy for a person because we have gotten used to discerning them (CAPTCHAs with pictures from real-life) in real-life and it’s just easy. You just see a number, a car’s number and you can say what the numbers are. I really like them.”

 

On the other hand,  Alex is of the opinion that the CAPTCHA where you have to tick a box is the best choice followed by the letters and numbers:

Interviewee: Well, I think that by clicking on something only it would be very effective in terms of time.  It would be a faster way.

Researcher: Yeah. And which CAPTCHAs did you find easy, satisfying and quick to complete. Like the easiest?

Interviewee: The easiest is by clicking on something like “Do you agree with the terms” and then press submit and continue.

 

 

Smith also likes the “innovative” CAPTCHAs which he explains as CAPTCHAs which require the user to slide left or right and perform such simple tasks. He finds them better because they are quicker to complete (and thus have higher efficiency) and because in ordinary CAPTCHAs (with letters and numbers which you have to fill) you can easily make a mistake and often in such case you would have to retype some information (such as a password in a form). He finds the CAPTCHAs with letters and numbers to be often “very hard to recognize” and he even stated that he hates them. He thus deems the CAPTCHAs with letters and numbers  as very unsatisfactory as satisfaction is explained in  ISO 9241-11  as consisting of a freedom from discomfort for the user and a positive attitude of the user towards the specified system (Jokela et al., 2003) and both are lacking with Smith:

“Researcher: And how do you think it affects your speed when dealing with websites? (CAPTCHAs)

Interviewee: I hate it.

Researcher: You hate it. So it’s not easy to complete the different CAPTCHAs?

Interviewee: Sometimes it’s easy but there are some very, very difficult ones. You can easily mistake the words or numbers.

Researcher: Yes. And can you think of some ways that would ease the completion of such CAPTCHAs?

Interviewee: I have seen some other innovative methods like just to drop or slide with your mouse.

Researcher:  And.. Can you tell me about the CAPTCHA that you saw and found very good for you. I mean you liked it a lot, like what characteristics did it have, what did you have to do and why did you like it?

Interviewee: Okay. When you have just numbers you can easily mistake. Because sometimes there’s like a background noise and numbers – you cannot see what is the number – it’s very hard to recognize. But if it’s clear numbers – it’s much easier.

Researcher: So, about the innovative methods – you have pictures and you have to select only a couple of them.

Interviewee: Yeah, it’s alright. But I like the most the thing that you have to just slide it to the left or to the right. Have you seen it?”

 

Mariah sometimes like the ones in which you have to select photos but usually prefers the letters and numbers:

“Interviewee: Well, for entertaining the pictures are better but usually there’s more of them and there are couple of questions in a row so I don’t like that one. It depends if I am looking to be entertained, I would prefer the pictures but if I am in a hurry, I am at work and I am doing something very important I would rather take the numbers and letters.”

 

 

Angelina, on the other hand, prefers CAPTCHAs which ask for simple arithmetic:

“Researcher: So which type would you prefer if you have to choose (type of CAPTCHA)?

Interviewee: I think where I have  to calculate some math.”

 

5.1.6 Lower objective usability due to numerous ways of returning to the homepage

 

Although in the interviews, the participants state that the current ways of returning to the homepage are satisfactorily and usable some problems emerged during the observations reliant upon the Think Aloud method.

 

In the interviews, Pesho stated that clicking on the logo is very easy:

“Researcher: I see, that is awesome. And what actions do you ideally want to take to return to the homepage of a website. What would the most suitable for you that you would like the most?

Interviewee: Well, clicking the label is just very simple. It does not require any other action than simply dragging the mouse and clicking this on any given website.

Researcher: It’s about clicking the logo,  yes?

Interviewee: Yes, it’s easy.”

 

Alex stated that no changes are necessary and the clicking on the Home button is usable enough for him but he made a mistake and did not realize it during the observation:

“Interviewee: But usually I am clicking on the home button.

Researcher: And what actions do you ideally want to take to return to the homepage of a website? Like what would have been the most suitable, interesting, satisfying and easiest for you?

Interviewee: I am satisfied like that”

 

Smith also stated that no changes are necessary and the current patterns are usable (satisfying) but he made the same mistake in the observation:

“Researcher: I get it. And what actions do you ideally want to take to return to the homepage of a website? What would be perfect for you?

Interviewee: I’ll just click on the icon.

 

During the observation, Penyo tried to return to the homepage of PayPal by clicking on the browser’s back button. However, this did not return him to the homepage as it was not the previous website that he visited. Two (2) of the participants (Alex and Smith) clicked on the Home button instead of the logo in the last observation task (the BBC website) and this brought them to the homepage of the news section of BBC and not on the homepage of the entire website and they did not realize that without help. Upon realization, in line with the Think Aloud instructions they followed, some of the participants were really surprised that they did not actually go to the website’s homepage. So, it appears like “Home” does not always mean that you will be redirected to the website’s homepage like if you click on the website’s logo but even experienced users are not aware of this. This pinpoints a potential difference between the actual usability and the perceived usability and shows that the usability of returning to the homepage may be lower than expected due to the differences between the different methods for returning to the homepage.

 

Therefore, the mixture of ways to return to the homepage of a website may lead to lower usability than the perceived one.

5.2 Themes summary

 

The inappropriateness of the “Home” text was a recurring idea within the data analysis.  Some of the participants explicitly declared their unhappiness with the naming of the button, while others implicitly brought this up by talking about the ‘Homepage’ instead of the ‘Home’. The most repeated pattern was for it to be named ‘Homepage’ as ‘home’ can mean something related to one’s physical home such as home interior though other alternatives were presented by the computer science major who proposed the concept of a ‘start’ or ‘starting page’.

 

The participants had no trouble interacting with the ‘hamburger’ icon which some of them themselves attribute to their experience with technology and websites in general. This pinpoints that the ‘hamburger’ icon is perfectly usable amongst experienced Internet users and contradicts past studies exploring the lack of usability of the ‘hamburger’ icon which is most likely due to the nature of the participants – young and experienced Internet users who are online every day. Though, when the participants were probed and reflected upon the issues – they unanimously got to the conclusion that the ‘hamburger’ icon does not represent what it does and that better alternatives exist such as a simple “Menu” text. They implicitly pinpointed learnability and ‘information scent’ issues with the icon as it is used nowadays and one of the participants unknowingly came up with a findability issue of the icon by stating that it may not look like it is a part of the website, even less a special menu (Bedford, 2016). The main idea is that a better solution exists out there that is not the ‘hamburger’ icon.

 

Another recurring idea that participants share is that CAPTCHAs are a pain for their usability. Some of the participants find them boring, others distracting and most hard to complete. Though the participants expressed their desired characteristics of CAPTCHAs in different ways and even pointed to different and conflicting forms of CAPTCHAs it emerged from the thematic analysis that the participants associate their usability with the efficiency of completing the CAPTCHA. That is, the desired CAPTCHA is the one that takes the least to complete.

 

Returning to the homepage was perceived as perfectly usable by the participants but the observation revealed a flaw and contradiction between their actual and perceived usability. The multiple ways to return to the homepage confused three of the six participants which took the wrong action and did not complete the task without guidance. Thus, the slight differences between the ways to the return to the homepage are unknown, to some extent, to even experienced Internet users and further study in the field may be carried out to explore this phenomenon.

 

 

5.3 Usability problems of the chosen design patterns according to UPT

 

The UPT classification brought forward helped to come up and analyze most of the issues with the examined design patterns. The issues mentioned below emerged from the analysis.

 

Two (2) issues were identified with the ‘hamburger’ icon during the UPT classification. The first issue is related to the naming of the pattern which is part of the artifact component and its language subcategory and the second is related to the appearance of the object which is part of the artifact component and is related to the subcategory of visual appearance.

1.      The analysis of the participants’ viewpoints pinpoints that the ‘hamburger’ icon may need a text naming it ‘Menu’ or something similar. Thus, there probably is an issue related to the Artifact component, the Language category and the Naming/labeling field.

2.      Four of the six participants have stated that the icon is not relatable in one way or another. Namely, Pesho, Penyo, Smith and Mariah. Thus, The icon should imply the activity related to the icon and it should be more recognizable. Therefore, there is a strong reason to believe that there is an issue that has to do with the Artifact component, the Visualness category, and related to the Object appearance.

 

Two (2) issues were also identified for the current ways of returning to the homepage:

1.      The perspectives of the participants pinpoint that the text ‘Home’ may be confusing as it can mean a lot of things such as home interior and participants who mentioned the text button as a way of returning to the homepage preferred “Homepage”. Therefore, the naming of the pattern may also be ambiguous. Thus, the issue has to do with the Artifact component, the Language category and is related to the Naming/labeling of the pattern.

2.      One (1) of the participants explicitly stated that there should be a link that says homepage which will take you to the homepage while other three (3) took this as granted and Angelina did not think of clicking on the logo as a way to return to the homepage, she thought of just searching for the “Home” button. Though, too many alternatives also confused the users as revealed in the observation. Therefore, second issue has to do with the abundance of alternatives to return to the homepage and the subtle differences between them. Thus, it is related to the Task component, it affects the category/goal of  Task-facilitation, and is related to the  Alternatives field

 

Three (3) issues were identified with the different CAPTCHAs:

1.      The participants have strong beliefs that some CAPTCHAs are hard to understand. This has to do with the Artifact component, the Visualness category and is related to the  Object appearance.

2.      Some of the participants have brought up the issue that some forms of CAPTCHAs are boring to interact with, they could be made more entertaining through the use of pictures. This can be arranged into an issue dealing with the Task component, the Task-mapping category and is related to the  Interaction with the pattern.

3.      The participants are brought up the viewpoint that some CAPTCHAs are distracting and confusing and the users can lose focus on what they are actually doing when completing the CAPTCHA. This can be grouped into the Task Component, the Task-facilitation category and is a detriment to the Keeping the user on track goal.

 

6 Discussion

 

In this chapter, I discuss the resulting findings in line with the theoretical basis of the research, I pinpoint how and why they answer the research questions of the research and I compare the findings with some of the existing literature on the topic

 

 

 

6.1 Answers to the research questions

 

The research questions posed at the beginning of the research were:

1.      How do users perceive the current usage of (user interface) website design patterns in regards to their usability?

2.      How do users describe their desired characteristics of design patterns to enhance usability?

 

The research questions were answered individually for each of the three (3) chosen design patterns in the interviews and the observations.

 

In terms of the first research question, It became evident that the experienced participants perceive the ‘hamburger’ icon as relatively usable/easy but had in mind ideas to increase that usability for them and for less experienced users because the ‘hamburger’ icon has low learnability as it does not represent properly what it stands for (its information scent is lacking besides learnability). Thus, their usability accrued with the help of experience which negated the effects of the low learnability. Their usability was confirmed during the observation where they were aware of the process that they have to undergo to complete the tasks with the icon and did not hesitate. In terms of the second research question, the majority of the participants confirmed that a text-based alternative would be the most desired alternative and the majority pointed towards a text named “Menu”

 

In terms of the CAPTCHAs, the first research question was answered in themes 3 and 4. Generally, the participants found CAPTCHAs to have low satisfaction and associated satisfaction and usability with the efficiency that they could achieve in the task which varies from CAPTCHA to CAPTCHA. As regards to the second research question, each participant described the desired characteristics of a CAPTCHA in a different or slightly different way which is revealed in the 5th theme. 

 

Concerning returning to the homepage, although participants perceived the current ways usable (satisfying, easy and efficient) as shown in theme 6 they struggled with all the ways that they could return to the homepage and three (3) of them took the wrong path without realizing it (in terms of the first research question). Concerning the second research question, the majority of the participants have suggested that ‘homepage’ is a better alternative than ‘home’ (theme 1) as a characteristic of the design pattern for returning to the homepage (The home button). Some other ideas were presented but were not shared amongst the majority of the different participants.

 

6.2 Discussion of perceived usability

 

In recent literature, returning to the homepage has been depicted as relatively successful with 77% of users in a recent test being able to return to the homepage using the logo and 98% managing if there is a “Home” option (Fichter and Wisniewski, 2016). This research has shown that the abundance of methods to return to the homepage may confuse users because clicking “Home” does not always return to the real homepage of the website like the logo does.  The participants were not aware that the two methods could provide different results prior to the observation and three (3) out of the six (6) participants did not return to the real homepage successfully. Thus, although all of the participants perceived returning to the homepage as usable they ended up with unexpected results which surprised them and have shown possible gaps. The difference between perceived/subjective and objective usability is only expected (Holden and Rada, 2011). That is why the inclusion of a method to gather the subjective usability of the users (interview) and their objective usability (observation) was undertaken during the study. Objective usability can differ and does not always coincide with the subjectively perceived usability of the users (Holden and Rada, 2011).

 

 

Concerning the ‘hamburger’ icon, a recent research has shown that 71 out of 75 participants failed to achieve tasks that required the use of the ‘hamburger’ icon (Fichter and Wisniewski, 2016) but all six (6) participants managed to complete the given tasks and perceive it usable, likely due to their extensive experience with the Internet and the use of technology. The majority have pinpointed though that the icon has low information scent, recognition and learnability due to its non-relatable looks and have thought of ways that would make it more usable when a user is first meeting the icon. The ‘hamburger’ icon seems to be thought of as efficient, effective and satisfying once you get the hang of it but hard in the beginning which can be attributed to its low learnability. The decrease in user experience after an introduction of a hidden navigation accessible with a button such as the ‘hamburger’ icon was also confirmed by Pernice and Budiu (2016). In their tests, desktop content discoverability from a hidden menu was 23% lower than having a visible menu. They also argue that this drop in discoverability is due to low information scent, low salience (the icon is hard to notice at times and on larger displays), low familiarity with using the icon, amongst other reasons. The participants in this research also touched upon these issues and a probability for their success with using and knowing the icon is their high familiarity with the icon. A thing that was omitted from the discussion with the participants is their mobile usage as desktop-only users theoretically may be less experienced and familiar with the ‘hamburger’ icon despite their experience with the World Wide Web as a whole.

 

Regarding the CAPTCHAs, the participants find them a detriment to their usability of the website in general. They pinpoint different characteristics/types of the CAPTCHAs that are more usable for them but what those characteristics have in common is the desire to get done with the CAPTCHA quicker. A CAPTCHA may not have high learnability at this point as there are numerous types of CAPTCHAs and because every CAPTCHA shows a different thing due to its purpose so the participants have associated their satisfaction from that task with their efficiency in completing it and vice-versa. This confirms other literature on CAPTCHAs like Pogue (2012, p.23) who stated that CAPTCHAs have merely substituted one public nuisance (the ‘bad’ guys who abuse websites which CAPTCHAs try to stop) for another. Thus, in his words, CAPTCHAs have become a nuisance for the users.  He also states that the words are often too distorted and humans struggle reading them (Pogue, 2012) which was also mentioned by some of the participants.

 

 

6.3 Discussion of the desired characteristics of the chosen patterns

Most of the inputs from the participants regarding their desired characteristics match with what has been concluded in recent literature. For example, Fichter and Wisniewski (2016) state that if somebody that a web developer works require the usage of the ‘hamburger’ icon, then the least that can be done to increase the usability is to add the text “Menu” which was proposed by the majority of the participants. Other research has also concluded that the ‘hamburger’ icon has issues (though all 6 participants managed to cope with using it), for example, after removing it Spotify had a 30% increase in navigation hits (Archer, 2015) and recommending the ‘hamburger’ icon to be replaced or complemented with the text “Menu” (Archer, 2015) is not a unique result of this research.

 

The participant’s desired text of “Homepage” and the one input of “Starting page” over “Home” is interesting as the current standard is to have a button with the text “Home”, if there is such a button at all. This is exemplified in a few of the ‘big’ websites out there below:

Figure 6: A ‘Home’ button in the BBC website.

 

 

Figure 7: A ‘Home’ button in Twitter

 

Figure 8: A “Home” button in Snapchat.

 

Figure 7 and Figure 8 above show that the popular websites Snapchat, Twitter and BBC all use a “Home” button. Nonetheless, they are actually referring to what is seemingly more clearly expressed and known as “Homepage” (Dinet et al, 2013, p.81).          

As it was mentioned above, culture can be thought of as a factor that affects usability (Dinet et al., 2013) and the main difference lies in cultures which are holistic and those which are analytical (Dinet et al., 2013). The perspective of the participants is expected to match the style of the web designers that crafted the websites as the analytical perspective is present in the United States of America and most parts of Europe while the holistic perspective is mostly present in countries located in the Far East such as China, Japan and Korea (Dinet et al., 2013).  Therefore, the participants are expected to be in an area with greater usability due to cultural alikeness (usability is generally greater if the cognitive style of the web designer and the user are the same) (Dinet et al., 2013).   

 

It is true that “Home” saves some of the screen real estate, especially on mobile devices, and this is a valid reason for simply “Home” to be preferred so further research may be necessary to weigh between these two contradicting factors.  Still, “Home” either does not sound right, or seems like a source of confusion that may have a wrong information scent/misrecognition. The BBC website exemplifies this by confusing the users that it would link to the homepage of the website but it also leads to the “home” of the section that the user is in (like the news section). The existence of a problem with returning to the homepage has been discussed in a case of traveling websites with users over 45 years old (Finn and Johnson, 2013). In that case, returning to the homepage turned out difficult even when there was a visible Home button that returns users to the homepage and some websites did not have a Home button at all, assuming that the users can find a way (like changing the URL, going back in the browser’s history, etc.)  (Finn and Johnson, 2013). However, this study has shown that the multitude of ways to return to the homepage may be tricky as some young, experienced Internet users may only know/use specific ways to return to the homepage which can have unexpected results.

 

The research shows that a single CAPTCHA cannot satisfy the usability requirements of every single user and so a further research may be necessary to pinpoint what the different majorities prefer. Though it is evident that the usability of CAPTCHAs, as perceived by the participants, is associated with their efficiency. A reason why the participants preferred different types could be that they are efficient at performing different tasks and not at the others which could be attributed to fine details such as background, gender, culture and personality (Chowdhury and Chowdhury, 2011).  A recent research (Bursztein et al., 2014) has shown that the users’ efficiency (accuracy and solving time) do not necessarily mean that the user would prefer the given CAPTCHA. The researchers found people to complete CAPTCHAs using Amazon’s mechanical turk website which allows people to carry out certain tasks and get paid and also found people for several surveys which indicated that users perceive digits as faster, easier and likable although the research found out that English words were the ones completed with the highest accuracy and speed (Bursztein et al., 2014). Therefore, quantitative analysis which grasps the actual efficiency of the users when entrusted with the task of completing different CAPTCHAs may be a better shot at understanding the objective user preference as clearly there is a difference between what the users perceive as quick and easy and what actually is quick and easy. Our current study can be seen as an illustration of this ambivalence of perception and reality – as participants gave varying and even contradicting types of CAPTCHAs as examples of quick, easy and likable which clearly were not the easiest and quickest to get around.

 

7 Conclusion

 

 

In this chapter, a few remarks are mentioned to conclude the research, the potential benefit and impact of the research is discussed along with the possibilities for future research on the topic.

 

 

This study sought to qualitatively research how experienced, young (aged 23-32) Web users perceive three commonly used website design patterns (CAPTCHAs, returning to the homepage, and the ‘hamburger’ icon) in terms of their usability and sought to find out in what ways those website design patterns could be made better by exploring what the participants desire for the actions the website design patterns represent. Attention was paid to ensuring that the research is performed in an ethical way.

 

The ‘hamburger’ icon was generally found unsatisfying and the majority of the participants proposed a text-based alternative, mainly pinpointing the text “Menu” which has already been proposed in past researches (Fichter and Wisniewsk, 2016). Nonetheless, the objective usability of the users as depicted in the observation turned out high in the observation where none of the participants struggled with accessing the menu hidden by a ‘hamburger’ icon.

 

The preference of CAPTCHAs revolved mainly around the ones that are the quickest to complete although users pinpointed different types as being easy and quick. As Bursztein et al. (2014) have determined in another research – users seem to have a false perception about which CAPTCHAs they complete the quickest and a quantitative approach may be better suited for such a case. Nonetheless, it appears from the research that the preference of CAPTCHAs and the subjective usability may depend on other factors such as the familiarity with different types of CAPTCHAs and possibly age/gender/education and so on as the participants gave differing answers although they all had ease and quickness of completion in mind.

 

In terms of returning to the homepage, a text/button “Home” appears to be undesired due to the possibility of confusion/low information scent and “Homepage” was proposed by some of the participants as a way to enhance the usability of that action although all of the users knew a way to return to the homepage. During the observation, the objective usability also seemed to be lower than expected as three out of the six participants confused the proper way to return to the homepage which can be attributed to the multiple ways of returning to the homepage and the fact that they have subtle differences in terms of functionality. For example, you can return to the homepage of a website by clicking on the browser’s back button, by going to the domain’s root in the URL address bar, by finding the website’s logo, by finding a Home button, etc.

 

Generally, the research indicates that a text like “Menu” should be added should the ‘hamburger’ icon be used, ‘Homepage’ should be used instead of ‘Home’ with a clear indication of where the ‘Homepage’ would bring the user and CAPTCHAs need to evolve to be quick and effortless to complete, hopefully without sacrificing the level of security they provide. In terms of CAPTCHAs, the reCAPTCHA which only asks the user to tick a checkbox might be the way to go, although the users did not pick that as the preferred source, possibly due to their lack of experience with it and this must be further explored.