Оcenka-BEL.com - безплатни онлайн тестове, пробни изпити, видео уроци и помощ при писане на задания от литературно и хуманитарно естество

Degree project, IVAN DIMOV

Публикувано: 2017-09-11 11:37:52


Дипломната теза по информатика е защитена от Иван Димов в Швеция

Degree project


Свързано изображение



The users’ perspective and preference on the usability of three user interface website design patterns

A qualitative study

























Author: Ivan Dimov

Supervisor:  Despina Fyntanoglou and Elissavet Kartaloglou    


Date: 2016/05/03

Course Code:4IK50E, 15 credits

Subject: Degree project at Master Level

Level: Postgraduate

Department of Informatics















This study is qualitative and interpretive in nature. It examines the perception of 6 people aged 23-32 with decent experience in using the Web on the usability of[u1]  the ‘hamburger’ icon (an icon used primarily in mobile websites and apps that shows a hidden navigation when clicked), CAPTCHAs (a task that users have to complete to continue browsing a webpage to prevent automated software operating on the webpage) and returning to the homepage and searches for the characteristics that they desire to see in those three user interface design patterns and the actions that those patterns represent.[u2]  The participants are reached through interviews and observations and the research pinpoints that although experienced Internet users find the user interface elements relatively usable some usability factors can be worked upon in the chosen design elements and pinpoints what users would want to see changed, the actual changes they want and the problems they actually encounter with the current status of the three (3) design patterns and their usability. More noticeably,  the research pinpoints that a “Homepage” button would be more usable than “Home” button which is the de facto standard as of this moment and it shows that the ‘hamburger’ icon is usable enough amongst experienced users, contradicting the research pinpointing that 71 out of 76 fail using the icon (Fichter and Wisniewski, 2016) probably due to the participants’ experience with technology, but other, preferable alternatives to the ‘hamburger’ icon are revealed from the participants which are in line with the current literature. CAPTCHAs are confirmed as a ‘nuisance’ (Pogue, 2012) and the need for CAPTCHAs which are quick to solve emerges which is what forms the perception of usability of the participants.





usability, captcha, hamburger, icon, homepage, website design pattern, qualitative, user interface
























I would like to express my gratitude to the people who helped me with this endeavor. They are:


My supervisors, Despina Fyntanoglou and Elissavet Kartaloglou, who were patient as I proceeded slowly with the research. Without their precious and professional help this endeavor would have not been realized.

Of course, I am also thankful to all of my professors at Linnaeus University for the path to knowledge to which they guided me.


I also want to express my gratitude to all colleagues in Linnaeus University for their cooperation throughout the year and to Neringa Daniulaityte, in particular.


Finally, I would like to thank all of the participants in the research for taking some of their time and for the information that they shared without who the research would have not been possible either.
































List of abbreviations


CAPTCHA - Completely Automated Public Turing test to tell Computers and Humans Apart


UI/ui – user interface

HTML – Hypertext Markup Language


MVC – Model/View/Controller (a software architecture)


UPT – Usability Problem Taxonomy





































The users’ perspective and preference on three user interface website design patterns and their usability  1

1 Introduction. 7

1.1 Background. 7

1.2 Purpose of the study. 9

1.3 Topic justification. 9

1.4 Aim and research questions. 10

1.5 Scope and Limitations. 10

1.6 Structure of the Thesis. 11

2 Literature Review.. 12

3 Theory. 15

3.1 Human-computer interaction. 15

3.2 Usability. 16

3.3 Theoretical framework. 17

4 Methodology. 19

4.2  Research setting. 20

4.3  Data collection methods. 20

4.4  Data analysis. 21

4.5  Validity and Reliability. 22

4.6  Ethical considerations. 22

4 Findings. 24

4.1        Themes. 24

4.1.1 Usability issues surrounding the “Home” text 24

4.1.2 Substitute ‘the hamburger’ icon with text for greater usability. 25

4.1.3 Dissatisfaction with CAPTCHAs. 27

4.1.4 The usable type of CAPTCHA is the shortest to complete. 28

4.1.5  A lack of characteristics of CAPTCHAs that are equally usable. 29

4.1.6 Lower objective usability due to numerous ways of returning to the homepage. 32

4.2        Themes summary. 33

4.3 Usability problems of the chosen design patterns according to UPT. 34

5 Discussion. 35

5.1 Answers to the research questions. 35

5.2 Discussion of perceived usability. 36

5.3 Discussion of the desired characteristics of the chosen patterns. 37

6 Conclusion. 40

6.1 Contribution. 41

6.2 Reflections. 41

6.3 Further research. 42

7 References. 43

Appendix A: Consent form.. 48

Appendix B: Interview Questions. 50

Demographic questions. 50

Research questions. 50

Appendix C: Observation Questions. 52

List of figures. 53






List of figures




Figure 1: Icon examples. 7

Figure 2: The hamburger icon in popular website TechCrunch. 8

Figure 3: An adaptation of  the Usability Problem Taxonomy (UTP) (Keenan et al., 1999, p.74) 16

Figure 4: Type of CAPTCHA that is preferred by the participant Penyo. 27

Figure 5: Type of CAPTCHA that is difficult for the partcipant Penyo. 27

Figure 6: A ‘Home’ button in the BBC website. 33

Figure 7: A ‘Home’ button in Twitter 33

Figure 8: A “Home” button in Snapchat. 33










1 Introduction 



This chapter provides an overview of the topic, its purpose, structure, objects of examination and highlights some reasons for conducting it in the first place



1.1 Background


The Web contains more than 1 billion websites with nearly 3 billion users who “surf” through them (Internetlivestats.com, 2016). It is only expected that users want to find what they are looking for quickly and effortlessly (Montero et al., 2002, p.1).


To help the users, user interface design patterns have emerged which are recurring solutions that solve common design problems (Toxboe, 2016) and the recurring principle in design patterns is to achieve a usability boost for the users (Montero et al., 2002, p.1)


Usability itself has been historically defined in different ways. It is defined in ISO 9241-11 as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (Jokela et al., 2003, p.54). A usable system essentially includes the following components: high user satisfaction and high efficiency and effectiveness (Jokela et al., 2003). However, the system’s usability would depend on the context in which it is used, the goal of the user, the task at hand and the user’s environment of use (the hardware, software and materials that the user disposes with) (Jokela et al., 2003).


Examples of patterns are the ‘hamburger’ icons that we use to reveal navigation in most websites today, the magnifier icon which indicates to us that we can search the website for something specific or the ‘breadcrumbs’ that we see in information-heavy websites which indicate to us where we are in terms of the website’s structure/hierarchy.




Figure 1[u4] : The website design patterns examined in this study

As shown in figure 1, Circle 1 shows a "hamburger" icon; the icon indicates that clicking on it will show the website's navigation. It is typically shown on small screens such as smartphones and tablets due to their low amounts of screen real estate and the need to save space that comes from this.

Circle 2 shows the logo of Amazon which when clicked would return the users to the homepage. Circle 3 shows a popular CAPTCHA in which the users have to type the words from the image to proceed with accessing the web resource.


According to Johansson et al. (2015, p.1), website design patterns help websites to become accessible to all computer users and pleasing them all. However, Montero et al. (2002, p.1) asserts that design patterns in website originate from intuitions and not from research.  Should this be the case today, the usability boost of website design patterns may be ambiguous. This lack of research can be confirmed by looking at the history of the hamburger icon. Its earliest use can be traced to the interface design for the Xerox Star personal workstation in 1981 by Norm Cox and research was not undertaken on its usability qualities (Alday, 2014) (Quora.com, 2014) (Vimeo.com, 2013)   - “the hamburger, which looks like a list, seemed like a good way to remind users of a menu list” (Campbell-Dollaghan, 2014).


In line with this trend, we believe that researching the ways users feel about the design patterns currently in use in websites, the effects of those patterns on their usability and examining what users actually desire can be beneficial to the research community and all people involved with websites such as users or developers in general. Such a research can pinpoint if current design patterns are serving their purpose of enhancing usability and it can pinpoint if, how and in what ways the design patterns can be ameliorated.


In this study, I will focus on the hamburger icon mentioned above, CAPTCHAs which are user interface patterns meant to prevent bots from posting content such as comments on websites by requesting the user to type a text from an image and the ‘return to the homepage’ pattern. The latter does not have any icon itself nowadays as users are expected to click on the website’s logo to get returned to the homepage or they are presented with a button which states “Home”.

Recent research (Fichter and Wisniewski, 2016) (Pogue, 2012), has shown that those three design patterns probably have poor usability but yet are used in highly popular websites and apps such as Amazon and TechCrunch. This research can serve to confirm/disconfirm this discovery, expand the available knowledge on the reasons of their poor usability and can pinpoint ways to create more usable design patterns in the future.



Figure 2[u5] [ID6] : The hamburger icon in popular website TechCrunch



This research will focus on user interface design patterns used in websites (like the ‘hamburger’ icon). Essentially, this relates to design patterns that have a user interface component which is presented to the users to solve a particular interaction problem or to make interacting with the website more usable/easier. Furthermore, only three (3) of those user interface design patterns are chosen for the research.

1.2 Purpose of the study


User interface (UI) design patterns in websites are very common.  Twitter Bootstrap is a UI library which is used by millions of website out there. It allows technology designers to create websites with pre-made UI components. They just pick the UI component that they want through an HTML element and class and the UI component appears on the page. It contains a lot of components such as slideshows, styled tables, panels, buttons, forms, icons and so on. It essentially allows easy and rapid inclusion of design patterns in any website.  Twitter Bootstrap is used by 11% of the global 10,000 most visited websites and it is used by 15.3% of the million most visited websites with the trend being that more and more websites are using Bootstrap for their user interface (Trends.builtwith.com, 2016). Though, Twitter Bootstrap is not the only source of UI components for websites, there are many other alternatives available out there such as Zurb Foundation, UI kit and so on.

The usage of website design patterns will most likely continue to rise (Trends.builtwith.com, 2016) so research into the users’ desires and their viewpoint regarding the actual usability effects of the current design patterns should come naturally. Involving the users’ when building a system (collecting user requirements) is seen as very important and evidence exists to the usefulness of that practice (Shneiderman, 2006). This research seeks to find how users perceive the current design patterns in terms of their effects on their usability and what kind of characteristics users want the design patterns to have. This can have the effect of determining whether design patterns are regarded positively by users which can potentially reduce the stage of gathering user requirements due to the awareness that users find them satisfactorily. It can also pinpoint characteristics that users find lacking in current design patterns and pave the way for changes in that field or find characteristics that were neglected in the past and influence the future generation of user interface design patterns in websites.


The web involves a diverse user base focusing on different aspects of it. Some access the web using screen readers (visually impaired), some use it to find information, connect with others, others use it to shop, work, learn new skills and so on. This research cannot possibly capture the diverse user base of the World Wide Web and focuses mostly on young (between their 20s and 30s) long-time Web users who use the web in the most common way – visually through a web browser such as Mozilla Firefox or Chrome.

1.3 Topic justification


The topic has been somewhat neglected in the past and this research can potentially reach unexplored conclusions. When searching Google Scholar for articles that contain both the words design patterns and usability in their title (intitle:"design pattern" intitle:"usability"), Google only returns 2 pages of results while most of them do not focus on websites at all but on other specific types of applications. The study may create new knowledge to guide further research in fields such as website design, usability, user experience and human-computer interaction design by analyzing the user’s standpoint on the usage of user interface design patterns in websites.  Despite the wide acceptance and usage of website design patterns (mentioned above) many users still experience trouble using and navigating websites, such as older adults (Barros, Leitão, & Ribeiro, 2014) so involving the users with the design patterns (of which every website is composed) can be beneficial for enhancing the patterns’ usability and enhancing the existing design patterns.

1.4 Aim and research questions


This research focuses on two questions:

  1. How do users perceive the current usage of (user interface) website design patterns in regards to their usability?
  2. How do users describe their desired characteristics of design patterns to enhance usability?


Therefore, the study aims to find out the answers to questions such as how the user interface design patterns in-use in websites affect usability, in the perception of the users. The research further aims to understand what characteristics of user interface design patterns in websites the users want to see more to achieve better usability. To exemplify the goals of the second question, the research can pinpoint how the users actually want the icon that opens up the website’s navigation to look like, how it can be made more user-friendly and what exactly needs to be ameliorated. The research focuses on the so-called ‘hamburger’ icon, the ‘return to homepage’ pattern and CAPTCHAs incorporating experienced Internet users of slightly varying age.


1.5 Scope and Limitations[u7] 


Due to a distance between the researcher and some of the potential participants in the study - the observations will occur online with the help of a video conferencing and screen sharing tools. Extracting and interpreting as accurately as possible the usability perspectives and characteristics of design patterns of users are crucial for this research. 
The interviews with the participants will be semi-structured with some predetermined questions and a topic to discuss that can fluctuate based on the participants’ responses. The interviews are going to be recorded to be used in the analysis later on. Due to the lack of a natural setting in the interview, the users’ direct experience would not be accessible as what people say often differs from what people actually do. This is why an observation will also take place in which the users’ direct experience would be analyzed.

 The research maintains an interpretive view. It is assumed that understanding the effects of the design patterns depends on the meanings and worldviews attached to them by the participants.

The research will include a heterogeneous group of participants in terms of different/slightly different individual qualities - such as technological experience, occupation and age. This will have an impact on the answers and perspectives of users which will be accounted for.[u8] [ID9] 

1.6 Structure of the Thesis

This thesis starts with an introduction. The introduction gives an overview of the research questions, aims and objectives of the study. After the introduction, I am going to present a glimpse of the previous research in the field of design patterns, usability and users (Section 2). Users’ experience with design patterns is going to be shown and the research on the usability of various user interface design patterns of websites is going to be examined[u10] [ID11] . In Section 3, the theories underlying this research are going to be revealed and focused upon, as well as the data collection methods and the ethical considerations of the research.  Section 4 presents the results of the research and shows the findings of the research. Section 5 discusses and analyzes the findings and strives to discover new and useful patterns in the results of the observations and the interviews. Finally, Section 6 concludes the research and focuses on possible future works in the area.




























2 Literature Review




This chapter reviews literature on what CAPTCHAs, the ‘hamburger’ icon, returning to the homepage represent, on usability and the factors affecting it and the chapter summarizes recent research on the usability of those three user interface design patterns.




CAPTCHAs are words (sometimes actual words and sometimes just a collection of random letters) that are visually distorted in a graphic (Pogue, 2012). Their purpose is to stop automated bots from registering in websites, posting comments and so on. In other words, their goal is to keep the “Good guys in, bad guys out…” (Pogue, 2012, p.23).  According to Pogue (2012), CAPTCHAs have merely replaced one public nuisance with a second one. The words are often too distorted and humans struggle reading them (Pogue, 2012). Furthermore, blind people are incapable of achieving the required task. (Pogue, 2012). Alternative forms of CAPTCHAs were invented but Pogue (2012) points out that they always exclude certain parts of people. For example, listening to a garbled audio is not suitable for people with hearing issues and CAPTCHAs which require users to answer simple questions may seem impossible to complete by non-English speakers which use a translator to view a website (Pogue, 2012).


The hamburger icon’s purpose is to open sliders when clicked (Fichter and Wisniewski, 2016). It is used in a number of popular apps and web designers are under the impression that if people are not aware of it - they will learn it (Fichter and Wisniewski, 2016).  It’s main purpose of use is to free up some of the screen real estate on mobile devices and it is presumed to provide “a clean, uncluttered look” (Fichter and Wisniewski, 2016, p. 76) of websites.


Returning to the homepage using the website’s logo has become a widely accepted practice used by an infinite number of popular websites such as Amazon and PayPal. It is documented as a design pattern that enables users to quickly navigate to the main page (Toxboe, 2016). However, a recent study has shown contradictory results regarding the pattern’s usability (Fichter and Wisniewski, 2016).              


Website design patterns may sometimes emerge from the intuition of the one building the system (Montero et al. 2002). This is sometimes confirmed by research that comes after the establishment of the pattern which concludes that the pattern is not usable enough. Fichter and Wisniewski (2016) discuss the hamburger icon and the homepage return patterns and pinpoint existing research which shows their bad usability as presented at the San Diego UXPA conference by Danielle Coole and Mike Ryan[u12] [ID13] [ID14] . To illustrate, many pages disregard homepage icons and use the website’s logo as a link to the homepage (Nielsen, 1999) (Fichter and Wisniewski, 2016) (Ux.stackexchange.com, 2010). However, research showed that only 77% of users managed to successfully return to the homepage using the website’s logo while 98% managed to do it if there is a ‘Home’ option in the website’s navigation (Fichter and Wisniewski, 2016). Furthermore, results from six usability tests lead to the conclusion that users largely fail to complete browsing tasks when the task involves a step with a ‘hamburger’ icon (Fichter and Wisniewski, 2016). In fact, 71 out of 76 participants did not manage to complete the tasks that involved the ‘hamburger’ icon (Fichter and Wisniewski, 2016). This reveals how important it is to get the users’ perception on the current design patterns and retrieve information from them regarding the ways to optimize current and future design patterns.


Usability (along with usefulness) can be considered essential to the main goals of any computer system (Harms & Adams, 2008). However, usability is difficult to define due to the fact that a person’s technological usability depends on a myriad of factors (Shultz and Hand, 2015).  To start with, the capacity of the device that the user is using and the software and hardware on it may affect the usability of an application as perceived by the user (Shultz and Hand, 2015).  Along with that, the specific setting, user and task may affect the overall usability (Shultz and Hand, 2015). Factors are even fine user details such as age, nationality and culture (Shultz and Hand, 2015). There have been numerous attributes that a usable system must have proposed in previous research.  Shultz and Hand (2015, p.66) propose the following definition: “the degree to which the user perceives acceptable learnability, efficiency, and satisfaction when using the [website] technology”. Learnability is measured by the degree of easiness involved when a user completes a task for the first time (Chowdhury and Chowdhury, 2011). Efficiency refers to the speed with which users can accomplish certain tasks in a technological product (Chowdhury and Chowdhury, 2011) and satisfaction refers to the user’s satisfaction with the given technological product (Chowdhury and Chowdhury, 2011). One can measure the satisfaction of users’ by asking them about their opinion on the quality and uniqueness of the system or asking how its different features feel compared to its market competitors (Chowdhury and Chowdhury, 2011).   Other attributes, not mentioned in the previous definition, have also emerged in different studies such as the ones presented by Alonso-Rios et al. (2010, p.65): “knowability, operability, efficiency, robustness, safety and subjective satisfaction”. Evidently, usability is subjective and the perceptions of those attributes depend on the user, situation and task (Shultz and Hand, 2015) which is why a qualitative, interpretive research is particularly suitable.


When retrieving the users’ perception regarding the chosen design patterns in the interview, special attention has to be paid to their context of use and goals (Chowdhury and Chowdhury, 2011) and the user, in particular. It is important to stress that the usability attributes differ between users due to various factors such as age, gender, capabilities, background, personality, work, technological skills and so on (Chowdhury and Chowdhury, 2011). An interpretive research may be particularly suitable to capturing the subjective factors involved in forming the user’s usability perspective.   


The intertwining of two research methods: interviews and observations is purposeful.  It aims to capture both the users’ subjective and objective usability. A website, or a design pattern, might be perfectly usable in terms of a set of objective criteria but a user may still experience it unusable in his/her subjective point of view for a myriad of reasons such as the appearance of distracting elements or problematic wording of links (Thielsch, Engel and Hirschfeld, 2015). Therefore, a distinction has to be made between the experienced usability by the users and the objective/real usability of a website. This is further revealed in a recent study on CAPTCHAs which shows that users preferred CAPTCHAs with digits for being the quickest and effortless to solve although the objective measurements pinpointed that they were not (Bursztein et al., 2014)


Details about the participants needs to be also taken into account when analyzing the usability. This research focuses on young (23-32 years old), experienced Web users. On average, speed and ability to accomplish a task in the Web declines with age, with estimates of 0.8% drop per year from age 25 to 60 (Nielsen, 2008). After 65 years of age, users need drastic changes for websites to be usable (Nielsen, 2008). However, this research does not focus on such users who may experience special needs in terms of usability and the participants appear to be all in an age where their capabilities are similar in terms of that factor.  Interestingly enough, income is also a factor for the speed and ability to use a website, with the higher the income – the less time it takes to accomplish a task (Nielsen, 2008). Nonetheless, the participants were not asked about their income in this research to take this factor into account.


Another fact worth considering is the education of the intended users. Websites involve reading and understanding text which can be an issue depending on the user comprehension and literacy level (Badre, 2002). Therefore, participants with lower level of education need a different text/form of presentation that is appropriate to their level of education (Badre, 2002). For example, non-technical users visiting a technical website may need to see content with technical jargons omitted and a simpler vocabulary (Badre, 2002). In this study, all participants either have some form of higher education or are working on it.


There are a range of studies covering the examined website design patterns in different contexts. Griffen (2015) carried out a research on the ‘hamburger’ icon when combined with long pages and older adults.  A number of issues emerged from that particular set of participants – such as the need to scroll back to the top of the website where the ‘hamburger’ icon is quicker, a number of the participants confused the option to go deeper into the website (the ‘hamburger’ icon) with other buttons and icons present on the assigned website which had rather different purposes and a number of users were unfamiliar with the icon (Griffen, 2015). The older adults give several recommendations for the icon -  the introduction of a visual hint for it, the possibility for the icon to be always visible at the top part of the webpage or entirely removing it and having an always visible menu (Griffen, 2015).


Finn and Johnson (2013) have found out that returning to the homepage might be a challenge even if there is a clear “Home” button on the page for people above 45 years old and it would be beneficial to determine how experienced and young Web users cope with the task.


Furthermore, the culture of the user and the designer may have an effect on the usability of a given website as users that have the same cognitive style which the web designer who crafted the website has are more likely to find the website likable and usable (Dinet et al., 2013). A major way that cultures differ is their perceptual processes (Nisbett and Miyamoto, 2005). Westerners are inclined to regard things in a context-independent analytical perspective while Asians regard things from a context-dependent, holistic perspective (Nisbett and Miyamoto, 2005). This is a variable that may be taken into account when examining website design patterns from a multi-cultural level though the participants in the research are expected to be culturally alike due to their birth and life in a single country (Bulgaria) and for the most part in the same city.


Importantly, the young age of the participants in this research will certainly have a positive effect on their usability. It has been estimated that satisfaction and usability decline amongst older adults along with the users’ spatial ability, performance and disorientation levels (Wagner, Hassanein and Head, 2014). Therefore, a decline in perceived/objective usability is expected should older users are included, possibly with the addition of different desired characteristics.










































3 Theory[u15] 



 In this chapter, the theories underlying this research, related to usability, human-computer interaction and methods for acquainting oneself with a user’s usability are going to be revealed and focused upon, the two data collection methods - interviews and observations are going to be discussed. Besides these, the methodology used in the research and the ethical considerations of the research are also focused upon.



3.1 Human-computer interaction


Most people carry out computerized tasks with ease, relying solely on their intuition but there is still a lot of people who experience trouble using computers (McKay, 2008). Using computers is actually complicated, it often requires all fingers of both of our hands and using them properly depends on several of our senses (McKay, 2008). There are people who think with the help of visual images while others think through speech or words (McKay, 2008). This basically pinpoints that some users will remember better if they are shown visual material while other will remember better if they are shown textual material (McKay, 2008). In terms of user interface patterns in websites, this would mean that the usability of the patterns would somewhat depend on the cognitive preference of the user. Not only does the term “good website” differs between user groups but an efficient and effective website may not be a “good website” if it does not please the aesthetic preferences of the given user (Sørum, Andersen and Vatrapu, 2012, p.698). The visual appearance has an important role in usability as websites may have around 50 milliseconds to impress the users and make them stay (Lindgaard et al., 2006). Therefore, the satisfaction of the users’ from the examined design patterns has to take into account their opinion about the design pattern’s current visual appearance.


Human-computer interaction principles require software to be “functional, direct and easy to use” (Bian, Jin and Zhang, 2010, p. 718).  The system itself should bring the user emotional happiness upon interaction and it should be “reasonable, efficient and consistent” (Bian, Jin and Zhang, 2010, p. 718). For example, a consistent website does not change its visual appearance sporadically across webpages (Bian, Jin and Zhang, 2010). Finally, the given system should be safe and stable (Bian, Jin and Zhang, 2010). A safe and stable system would allow the user to reverse his/her action and attempt to reduce the costs of an error (Bian, Jin and Zhang, 2010). For example, the WordPress platform for website creation allows users to bring back deleted pages and posts. In our context, when a user clicks on the hamburger icon on his/her smartphone and a lengthy navigation pops on the screen - it is a good idea to be able to hide the navigation again, possibly by clicking on the same icon again. Those are factors that can be taken advantage of when evaluating the usability of the chosen design patterns.


3.2 Usability


Traditional usability testing’s purpose is to get to know the users and how they are interacting with a given website (Sørum, Andersen and Vatrapu, 2012) and this research has such elements (performing observation, getting to understand users) in it, although it does not represent a usability test in its essence. Tests have shown that users which possess prior experience with a given system typically give higher usability scores when asked to rate the systems - with a difference from first-timers from 6% to 15% (Konstantina[u16] [ID17]  et al., 2015). Therefore, we may expect that users who have little knowledge and experience of the chosen patterns find them unsatisfying and possibly in need of change. That is why we take into account the user’s experience with technology and the actual knowledge of the user of the patterns - so we would be able to interpret their viewpoint better and in the proper context of their worldly experience.


The ISO standard 9241-11 notes efficacy, effectiveness and satisfaction as the most important factors that affect the usability of a system (Khan et al., 2013). Different usability models include different number of factors but in most models efficiency, effectiveness, satisfaction and learnability are considered crucial (Khan et al., 2013). To assess those major factors, the models typically rely on sub-factors which have different metrics and which contribute to the factor which they represent usability and in that way to the overall usability of the system (Khan et al., 2013).


In ISO 9241-11, satisfaction is presented as consisting of a freedom from discomfort for the user and a positive attitude of the user towards the specified system (Jokela et al., 2003) and qualitative methods are particularly useful to retrieve this factor.  In it, efficiency is described as measurable by a comparison of the resources that the user inputs with the accuracy and completeness with which specific goals are achieved within the system (Jokela et al., 2003) for the analysis of which the observation is particularly relevant. Essentially, the satisfaction of the user can be understood simply by asking the user what he thinks about the system’s ease of use (Usabilitynet.org, 2016) which is one of the aims of the interviews. Effectiveness can be analyzed by checking if the user can actually use the target system (or design pattern) to complete their desired task - or in other words - if they can “do what they want to do” (Usabilitynet.org, 2016) which our observation is supposed to reveal.

Testing the usability of icons generally requires knowledge of how findable they are or whether users can find them on a webpage, how recognizable they are or whether users can understand what the icon does by looking at it, how attractive they are or whether users find the icon aesthetically pleasing and whether users know what will happen once they click on the desired icon (Bedford, 2016).


According to the Technology Acceptance Model, the perceived usefulness and ease-of-use of a system (and websites) affects users in deciding whether to use that system (Chuttur, 2009). If the website is useful, users may still use the website if it is not easy to use as long as the perceived usefulness outweighs the handicap in ease of use (Chuttur, 2009). The model is relevant because usability is essentially this – how easy it is for a person to use a system or a website and because users can use a website even if they find its usability unsatisfactorily (at least in the initial uses). Therefore, users may have initially found a certain website/design pattern easy to cope with but its usefulness may have forced them to work with it initially.

3.3 Theoretical framework


To come up with meaningful conclusions from the results of the interviews and the observations we need to be able to identify the global usability problems of the chosen design patterns, the problems that share common characteristics and we need to be able to identify trends and patterns in the resulting data. The usability problem taxonomy can help us to achieve the goals mentioned above (Keenan et al., 1999). The Usability Problem Taxonomy (UPT) is “a taxonomic model for classification of usability problems detected on graphical user interfaces with textual components” (Keenan et al., 1999, p.73).  UPT lets us classify the usability issues into two components; those related with the task to be achieved by the user (the functionality/purpose of the design pattern) and those related to the artifact (the design pattern) (Keenan et al., 1999). The artifact component itself contains three categories - language, manipulation and visual looks while the task component is split into two categories - task-mapping and task-facilitation (Keenan et al., 1999). Each of these categories consists of subcategories as shown in the figure 3[u18] [ID19]  below. The categories are mutually exclusive and a problem may only be assigned to one at a time (Keenan et al., 1999).


Figure 3: An adaptation of  the Usability Problem Taxonomy (UTP) (Keenan et al., 1999, p.74)


UTP can help us classify and analyze the qualitative data collected from the interviews and the observations. On the one hand, it will be used to detect, classify and help in analyzing the existing usability problems of the three (3) chosen design patterns. On the other hand, it will help us in classifying and analyzing the user’s preferences when it comes to the desired characteristics of design patterns.


Think Aloud is a popular usability assessment method which is used to find out what the users are thinking and experiencing when they are in the process of completing a task on a system (Georgsson and Staggers, 2016[u21] [ID22] ). The method requires users to talk aloud during their interaction with the given system sharing their opinion and stating what they are trying to achieve when doing the pre-determined tasks (Georgsson and Staggers, 2016). The interference with the user as he/she is performing the tasks is kept minimal (Georgsson and Staggers, 2016). The goal of the method is to get an insight on how users actually experience the given system and on their decision-making process (Georgsson and Staggers, 2016). Think Aloud provides highly detailed data which enables researchers to only involve a few subjects - it has been determined that five (5) to eight (8) users can pinpoint 80 to 85% of the existing usability problems of the system. (Georgsson and Staggers, 2016). The method will be used along with the interview to grasp an insight on the actual usability problems when the users are required to take advantage of the chosen design patterns.

The interview questions themselves would be based on the most popular division of usability factors (the questions can be seen in Appendix B). Namely, those factors are ease of learning or learnability which specifies the extent to which the system is easy to learn both for beginners and users with experience, task efficiency which is the extent to which the system is efficient for the regular user, ease of remembering which is the extent to which the system is easy to remember, understandability which is the extent to which the user understands the nature and purpose of the system and the subjective satisfaction of the user with the given system (S. Lauesen and H. Younessi, 1998[u23] [ID24] ). The questions will try to extrapolate different usability problems related to those factors and later classify them with UPT and extract themes in the end from the resulting classification and the resulting categories.


Qualitative researches typically suffer from low standardization of strategies for data analysis which leads to difficulties in replicating results across studies and UPT can be considered as a means for combatting this flaw (Georgsson and Staggers, 2016) a