System Usability Scale (SUS) – An Improved German Translation of the Questionnaire
written by Kris Lohmann and Jörg Schäffer
Usability professionals have a variety of choices when it comes to picking a questionnaire to evaluate and measure the usability of a software product or a website in a quantitative manner.
A particularly often-used scale is the System Usability Scale (SUS). This scale was developed and made available by Brooke (1). The SUS consists of ten items. Ratings are performed on a 5-point scale on which 1 corresponds to “strongly disagree” and 5 to “strongly agree”. The evaluation results in an easy-to-interpret score from 0 – 100, similar to a percentage score. This intuitive score is a definitive plus when communicating the results to stakeholder and the management.
Aside from being a de-facto standard, the SUS has some advantages over other scales. It is scientifically validated and has been shown to outperform other scales in the sense of validity (it measures what it is supposed to measure) and sensitivity (the result shows small changes) (2, 3, 10). The SUS has been shown to be reliable (that is, if a study is repeated, the same outcome is can be expected) (10).
The SUS is free, and has been used in a lot of usability evaluations. In the words of the original author: “… [B]ecause it was made freely available has been picked up and used in many, many usability evaluations. (Cheapskates! But I’m glad so many people have found it useful).” (4)
Meta-Studies allow for comparison with other systems, the average SUS score measured by Tullis and Albert is 66 (3). Furthermore, a study by Bangor and colleagues has shown which semantic label people associate to a given SUS score. For example, a score around 20 is associated to an “awful” system and about “90” to the “best imaginable” system (9). Due to the fact that it only has ten items, the SUS is lightweight and, thus, participants are quickly done with it.
Yet, the SUS has some drawbacks to be considered when it is used. To start with, the SUS focuses just on pragmatic quality. The hedonistic quality (5) is not really in the scope of the evaluation (though, we are not saying that hedonistic aspects do not influence the result!). Furthermore, there is no tested German translation of the scale available – this issue is addressed in this post.
German Translations of the System Usability Scale (SUS)
The original author made the scale available in English. It was not officially translated to German. Recently there have been approaches towards translation (6,7). The most notable result was based on crowdsourcing the translation (8). We at CoreMedia reviewed the results and decided to develop our own version based on these approaches as we were not completely happy with the wording. This is the solution that we came up with:
- Ich denke, dass ich dieses System gerne regelmäßig nutzen würde.
- Ich fand das System unnötig komplex.
- Ich denke, das System war leicht zu benutzen.
- Ich denke, ich würde die Unterstützung einer fachkundigen Person benötigen, um das System benutzen zu können.
- Ich fand, die verschiedenen Funktionen des Systems waren gut integriert.
- Ich halte das System für zu inkonsistent.
- Ich glaube, dass die meisten Menschen sehr schnell lernen würden, mit dem System umzugehen.
- Ich fand das System sehr umständlich zu benutzen.
- Ich fühlte mich bei der Nutzung des Systems sehr sicher.
- Ich musste viele Dinge lernen, bevor ich mit dem System arbeiten konnte.
Why should you prefer the suggested German translation of SUS over other ones? Our translated version of the SUS has been used in a comparably large on-site study with 89 participants. Overall, there were no indicators of problems.
Furthermore, there is an important difference to the version proposed in the crowdsourced translation (8). Item 10, which is formulated “I needed to learn a lot of things before I could get going with this system” in the original SUS, is translated to “Ich musste eine Menge lernen, bevor ich anfangen konnte das System zu verwenden.” in the crowdsourced translation.
From our point of view, this solution is suboptimal: In contrast to the original English version, the translation does not put any focus on the users’ goal. Therefore, we suggest a slightly different translation of this item, namely “Ich musste viele Dinge lernen, bevor ich mit dem System arbeiten konnte.” This puts more focus on the user being able to accomplish a goal with the system. Thus, the semantics of the translation of this item is closer to the original item.
Some Practical Hints for Augmenting the SUS
We augmented the SUS in two respects:
First, we used two free text fields in which the participants were asked to indicate what they liked about the software and what they disliked about the software to obtain qualitative input in addition to the quantitative measurement. The results were really insightful, hence, if a study has a formative aspect (finding problems) in addition to the summative aspect (evaluating an existing solution), augmenting the SUS with free text items should be considered.
Second, we added some in-detail items to evaluate certain aspects of the the software that are not directly covered by the SUS (e.g., quality of icons, microcopy, and visual appeal). This augmentation provided additional valuable information, as well.
The System Usability Scale (SUS) is a lightweight and scientifically validated questionnaire and an effective tool to assess perceived usability. On the basis of existing attempts for a translation, we created an improved translation, which we successfully used in an on-site study with 89 participants.
As the SUS is a lightweight questionnaire with only 10 items, augmenting it with other items is unproblematic with respect to the time participants need to complete the questionnaire. Such an augmentation is suggested if a study has formative aspects and in circumstances when specific hypothesis need to be tested.
(1) Brooke, J.: SUS: A “quick and dirty” usability scale. In: Jordan, P. W., Thomas, B., Weerdmeester, B. A., McClelland (eds.) Usability Evaluation in Industry pp. 189—194. Taylor & Francis, London, UK (1996)
(2) A Comparison of Questionnaires for Assessing Website Usability
(3) Tullis and Albert: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann (2008)
(9) A Bangor, P Kortum, J Miller – Journal of usability studies (2009)
(10) Brooke: SUS: A Retrospective Journal of Usability Studies, Volume 8, Issue 2, pp. 29 – 40 (2013)