Automation of the calculation of usability attributes in user tests using RPA and sentiment analysis
DOI:
https://doi.org/10.15665/rp.v22i2.3431Keywords:
Análisis de sentimientos, automatización de procesos robóticos, pruebas de usabilidad, test con usuarios, usabilidadAbstract
With the large number of apps deployed in cloud application stores, usability has become a fundamental attribute of software quality to enable user-centered de-sign. One of the challenges in usability tests is the automation of the process of determining the level of usability, based on the attributes of effectiveness, efficiency and satisfaction according to ISO 9241-11. This paper proposes as a contribution the use of robotic process automation (RPA) in the automatic determination of the attributes of effectiveness, efficiency and satisfaction in end-user tests, in order to estimate the level of usability of the software evaluated in these tests. For the development of this research, an adaptation of the itera-tive research pattern proposed by Pratt was used, so that 4 phases were de-fined: characterization of the tests, definition of the process, implementation of the process through RPA and evaluation of the automated process. From the metrics associated with effectiveness (number of subtasks per task), efficiency (times per task) and satisfaction (post-test questionnaire ratings and opinions), an automated process was designed and implemented using the UiPath tool, which in an Excel spreadsheet determines the percentage of effectiveness, the percentage and the percentage of satisfaction, incorporating the sentiment analysis approach in the calculation of the last attribute. The proof of concept performed by using the automated process was correctly developed, allowing to demonstrate its usefulness and contribution both in the optimization of the analysis times of a test with users, as well as in the incorporation of the sentiment analysis approach in the conventional analysis of this type of tests.
References
D. Hering, T. Schwartz, A. Boden, and V. Wulf, “Integrating Usability-Engineering into the Software Developing Processes of SME: A Case Study of Software Developing SME in Germany,” in 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering, May 2015, pp. 121–122. doi: 10.1109/CHASE.2015.22.
K. Puri and S. K. Dubey, “Analytical and critical approach for usability measurement method,” in 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), 2016, pp. 4045–4050.
J. Nielsen, “Usability 101: Introduction to Usability,” 2012. [Online]. Available: https://www.nngroup.com/articles/usability-101-introduction-to-usability/
L. Cheikhi, A. Abran, and W. Suryn, “Harmonization of usability measurements in ISO9126 software engineering standards,” in 2006 IEEE International Symposium on Industrial Electronics, Jul. 2006, pp. 3246–3251. doi: 10.1109/ISIE.2006.296137.
W. Sanchez, “La usabilidad en ingeniería de software: definición y características,” Ing-novación, no. 2, pp. 7–21, 2011, Accessed: Oct. 20, 2020. [Online]. Available: https://core.ac.uk/download/pdf/47264961.pdf
I. Feroz, N. Ahmad, and M. W. Iqbal, “Usability Based Rating Scale for Mobile Health Applications,” in 2019 International Conference on Engineering and Emerging Technologies (ICEET), Feb. 2019, pp. 1–7. doi: 10.1109/CEET1.2019.8711845.
D. Fathiyyah, M. D. Sulthon Diani, Z. Ayuning Saputri, and Sunardi, “Usability Evaluation on Life Insurance Application Using System Usability Scale and ISO 9241-11,” in 2022 8th International HCI and UX Conference in Indonesia (CHIuXiD), Nov. 2022, pp. 94–99. doi: 10.1109/CHIuXiD57244.2022.10009774.
F. H. Nakagawa, A. S. Felinto, and M. T. Omori, “Inclusion of Teaching Slides in Games: Analysis of the Efficiency, Effectiveness and Satisfaction,” IEEE Lat. Am. Trans., vol. 11, no. 6, pp. 1372–1377, Dec. 2013, doi: 10.1109/TLA.2013.6710386.
D. A. Albornoz, S. A. Moncayo, S. Ruano-Hoyos, G. E. Chanchí-Golondrino, and K. Márceles-Villalba, “Sistema software para la ejecución de pruebas de usabilidad bajo el enfoque de mouse tracking,” TecnoLógicas, vol. 22, pp. 19–31, Dec. 2019, doi: 10.22430/22565337.1511.
C. Santos, T. Novais, M. Ferreira, C. Albuquerque, I. H. de Farias, and A. P. C. Furtado, “Metrics focused on usability ISO 9126 based,” in 2016 11th Iberian Conference on Information Systems and Technologies (CISTI), Jun. 2016, pp. 1–3. doi: 10.1109/CISTI.2016.7521437.
J. L. Gozález-Sánchez, F. Montero-Simarro, and F. L. Gutiérrez-Vela, “Evolución del concepto de usabilidad como indicador de calidad del software. José-Luis González-Sánchez, Francisco Montero-Simarro, Francisco-Luis Gutiérrez-Vela,” Rev. Int. Inf. Doc. Bibl. y Comun., vol. 21, no. 5, pp. 529–536, 2012, Accessed: Oct. 17, 2020. [Online]. Available: http://www.elprofesionaldelainformacion.com/contenidos/2012/septiembre/13.html
A. Leon-Montano and L. Barba-Guaman, “Design of the architecture for text recognition and reading in an online assessment applied to visually impaired students,” in 2020 International Conference of Digital Transformation and Innovation Technology (Incodtrin), Oct. 2020, pp. 59–65. doi: 10.1109/Incodtrin51881.2020.00023.
P. Weichbroth, “Usability of Mobile Applications: A Systematic Literature Study,” IEEE Access, vol. 8, pp. 55563–55577, 2020, doi: 10.1109/ACCESS.2020.2981892.
AENOR, “UNE-EN ISO 9241-11,” 1998.
G. E. Chanchí-Golondrino, L. F. Muñoz-Sanabria, and L. M. Sierra-Martínez, “Estimación del atributo de satisfacción en test con usuarios mediante técnicas de análisis de sentimientos,” Prospectiva, vol. 21, no. 2, pp. 40–50, 2023, doi: https://doi.org/10.15665/rp.v21i2.3248.
G. E. Chanchí, D. M. Delgado, D. F. Girón Timaná, and K. Márceles Villalba, “Estimación del atributo satisfacción en test de usuarios a partir del análisis de la expresión facial,” Rev. Ing. Univ. Medellín, vol. 19, no. 36, pp. 13–28, Jun. 2019, doi: 10.22395/rium.v19n36a1.
C. J. Mueller, D. Tamir, O. V. Komogortsev, and L. Feldman, “An Economical Approach to Usability Testing,” in 2009 33rd Annual IEEE International Computer Software and Applications Conference, 2009, pp. 124–129. doi: 10.1109/COMPSAC.2009.26.
G. Enriquez and S. Casas, “Usabilidad en aplicaciones móviles,” Inf. Científico Técnico UNPA, vol. 5, no. 2, pp. 25–47, 2013, Accessed: Oct. 20, 2020. [Online]. Available: https://dialnet.unirioja.es/servlet/articulo?codigo=5123524&info=resumen&idioma=SPA
N. Zhang and B. Liu, “Alignment of business in robotic process automation,” Int. J. Crowd Sci., vol. 3, no. 1, pp. 26–35, May 2019, doi: 10.1108/IJCS-09-2018-0018.
J. G. Enriquez, A. Jimenez-Ramirez, F. J. Dominguez-Mayo, and J. A. Garcia-Garcia, “Robotic Process Automation: A Scientific and Industrial Systematic Mapping Study,” IEEE Access, vol. 8, pp. 39113–39129, 2020, doi: 10.1109/ACCESS.2020.2974934.
D. Choi, H. R’bigui, and C. Cho, “Enabling the Gab Between RPA and Process Mining: User Interface Interactions Recorder,” IEEE Access, vol. 10, pp. 39604–39612, 2022, doi: 10.1109/ACCESS.2022.3165797.
B. Vajgel et al., “Development of Intelligent Robotic Process Automation: A Utility Case Study in Brazil,” IEEE Access, vol. 9, pp. 71222–71235, 2021, doi: 10.1109/ACCESS.2021.3075693.
I. E. Nielsen, A. Piyatilake, A. Thibbotuwawa, M. M. De Silva, G. Bocewicz, and Z. A. Banaszak, “Benefits Realization of Robotic Process Automation (RPA) Initiatives in Supply Chains,” IEEE Access, vol. 11, pp. 37623–37636, 2023, doi: 10.1109/ACCESS.2023.3266293.
E. Hartikainen, V. Hotti, and M. Tukiainen, “Improving Software Robot Maintenance in Large-Scale Environments–is Center of Excellence a Solution?,” IEEE Access, vol. 10, pp. 96760–96773, 2022, doi: 10.1109/ACCESS.2022.3205420.
G.-E. Chanchí-Golondrino, M.-A. Ospina-Alarcón, and F. Rico-Rodríguez, “Propuesta de un conjunto de recomendaciones de accesibilidad para mejorar el posicionamiento de portales web empresariales,” Rev. Científica, vol. 45, no. 3, pp. 390–401, Sep. 2022, doi: 10.14483/23448350.19374.
K. Pratt, “Design Patterns for Research Methods: Iterative Field Research,” 2009. [Online]. Available: http://kpratt.net/wp-content/uploads/2009/01/research_methods.pdf
G. E. Chanchí-Golondrino, L. M. Sierra-Martínez, and W. Y. Campo-Muñoz, “Fuzzy Logic-Based System for the Estimation of the Usability Level in User Tests,” Int. J. Comput. Commun. Control, vol. 17, no. 2, 2022.
Downloads
Published
License
Copyright (c) 2024 Gabriel Elías Chanchí Golondrino, Manuel Alejandro Ospina Alarcón, Mónica Esther Ospino Pinedo
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The authors to publish in this journal agree to the following conditions:
- The authors transfer the copyright and give the the journal first publication right of the work registered with Creative Commons Attribution License, which allows third parties to use the published work on the condition of always mentioning the authorship and first publication in this journal.
- The authors may perform other independent and additional contractual arrangements for the non-exclusive distribution of the version of the article published in this issue (E.g., Inclusion in an institutional repository or publication in a book), it must be indicated clearly that the work was first published in this journal.
- It allows and encourages the authors to publish their work online (eg institutional or personal pages) before and during the review and publication process. It can lead to productive exchanges and greater and faster dissemination of the published work (see The Effect of Open Access)
Instructions to fill out Certificate of Originality and Copyright Assignment
- Click here and get the forms of Certificate of Originality and Copyright Assignment .
- In each field to fill out, click and complete the corresponding information.
- Once the fields are filled out, at the end of the form copy your scanned signature or digital signature. Please adjust the size of the signature on the form.
- Finally, you can save them as pdf files and send them through the OJS platform as an attachment.
