Multidimensional model to measure quality in e-commerce websites using item response theory

Tezza, Rafael; Bornia, Antonio Cezar; Andrade, Dalton Francisco de; Barbetta, Pedro Alberto

doi:10.1590/0104-530X1875-18

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Original Article • Gest. Prod. 25 (4) • Oct-Dec 2018 • https://doi.org/10.1590/0104-530X1875-18 copy

Multidimensional model to measure quality in e-commerce websites using item response theory

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract

The goal of this article is to propose a multidimensional modeling to measure the quality of commercial websites with the use of Item Response Theory (IRT). The quality of a website encompasses technical characteristics (usability-navigability, presentation of information and interactivity) and non-technical characteristics (design, aesthetics, visual appeal, reliability, hedonism, image), theoretically configuring a multidimensional context. The initial hypothesis of the dimensions and the elaboration of the items were based on a bibliographic analysis about the theme of e-commerce website quality. A set of 75 items was prepared and submitted to a sample of 441 e-commerce websites from a wide variety of sectors. The treatment and analysis of data was conducted using IRT. In this step, questions related to dimensionality and the choice of the most suitable model was discussed. Finally, a multidimensional model with four dimensions was adjusted.

Keywords:
Quality in websites; E-commerce; Modeling; Multidimensional Item Response Theory

Models	Difference χ2			AIC	BIC
Models	χ²	Degrees of liberty	p-valor	AIC	BIC
Mod1 x Mod2	525.50	74	<0.01	377.50	74.91
Mod2x Mod3	283.32	73	<0.01	137.32	-161.18
Mod3 x Mod4	83.88	72	0.16	-60.12	-354.53

Dimension	1	2	3	4	5	6	7	8	9	10
Eigenvalue	8.95	4.44	3.89	3.15	2.55	2.12	1.79	1.61	1.60	1.47
Accumulated proportion of explained variation	20.34	30.42	39.26	46.41	52.22	57.03	61.09	64.74	68.38	71.72

Model	-2 Log da máxima verossimilhança	AIC	BIC
Mod1	-6849.03	14068.05	14824.53
Mod2	-6617.70	13691.87	14623.69
Mod3	-6531.43	13602.87	14706.91
Mod4	-6493.57	13609.14	14880.84

Item	Item Description	F_1	F_2	F_3	F_4	Communality
03	Does the homepage make clear what the site does (deomsntrate the main products and or a brief description of its objective and or benefits that it offers), without needing the roll-bar?	0.130	0.660	0.030	-0.020	0.454
05	Does the site have moving images that can distract the user?	-0.160	0.140	-0.200	-0.440	0.279
06	Do the links for sales go directly to the sale announced?	0.390	0.680	0.110	-0.090	0.635
08	Does the site have a cascade menu?	-0.060	0.070	-0.190	-0.570	0.370
10	Are sub-categories grouped?	-0.050	0.300	0.030	0.530	0.374
12	Are there information for telephone contact or an address?	-0.060	0.450	0.380	-0.300	0.441
19	Are apparently clickable words in fact clickable?	0.290	0.390	0.220	0.400	0.445
21	Do all the pages have a search field?	0.450	0.280	0.590	0.300	0.719
22	When there is rolling, are there design elements (in the intial screen) thaat appear with end of page markers?	0.260	0.380	0.270	-0.280	0.363
23	Does the company logo in the upper left corner on allthe site pages?	0.240	0.480	0.260	0.270	0.429
24	Is there a link with a single click that leads to the homepage?	0.520	0.250	0.750	0.320	0.998
25	Does the site allow navigating its pages in only one window, that is, there is not opening of new windows in amid the navigation?	0.110	-0.030	0.820	-0.280	0.764
27	Is there a list of frequently asked questions - FAQs?	0.030	0.300	-0.080	0.480	0.328
28	When entering search terms in the search field does the search engine offer suggestions?	0.060	0.020	0.180	0.660	0.472
29	Is the search system flexible in relation to the terms used by the user, that is, if the user types a term incorrectly, does the search system suggest a correction?	0.220	0.170	-0.170	0.530	0.387
30	Do the search results allow classification by other criteria in addition to cost?	0.290	0.390	0.300	0.500	0.576
32	Are page continuation items visible?	0.290	0.310	0.310	0.290	0.360
33	Is the price of a product next to the image or link for the product?	0.280	0.710	0.380	0.000	0.727
35	In products in which there is more than one perspective, is it possible to visualize all the perspectives?	0.050	0.330	0.050	0.330	0.223
37	Is there sufficient information about the products (size, basic characteristics)?	0.270	0.620	-0.040	0.110	0.471
38	Is there a way for consumers to provide to insert feedback about the products?	0.400	0.200	0.290	0.460	0.496
40	Does the site present related products at the end of the page?	0.070	0.450	0.030	0.240	0.266
43	Does the site have multimedia for product presentation?	0.000	0.520	-0.090	0.080	0.285
45	Is there an indication that the site is safe at the time of making the purchase	0.130	0.520	0.020	0.330	0.397
46	When filling in the forms, can the user visualize the next steps in the interface?	0.210	0.200	0.530	0.410	0.533
47	Does the site have other payment forms in addition to a credit card?	0.240	0.430	0.470	-0.240	0.521
48	Is it possible to know the total cost before registering (including shipping costs)?	0.330	0.600	0.290	0.250	0.616
52	Are required data differentiated from optional data in a visually clear manner?	0.280	0.220	0.440	0.220	0.369
55	Are the error messages free of abbreviations and or codes generated by the operating system?	0.210	0.100	0.540	-0.200	0.386
56	Can any user action be taken back with the UNDO or BACK option?	0.580	0.550	0.280	0.100	0.727
57	Does all the non-textual content that is presented to the user have an alternative in text form that serves an equivalent purpose?	-0.710	-0.050	0.400	-0.010	0.667
58	Can the information, structure and relations broadcast through the presentation be determined in a programmatic form or are they available in the text?	-0.960	0.160	0.130	0.190	1.000
59	Is there another visual form of presenting information, beside color, to indicate an action, request a response or distinguish a visual element?	0.150	-0.460	0.520	0.270	0.577
60	Does the visual presentation of text and images have a relation of contrast of at least 4.5:1?	-0.720	-0.120	0.010	0.510	0.793
61	Except for captions and text images, can the text be resized up to 200 percent without support technology, and without losing content or functionality?	-0.340	-0.180	0.430	0.280	0.411
64	For each time limit defined by the content, is there control by the user?	0.220	-0.600	0.550	0.100	0.721
65	For information in movement, in an intermittent mode, in shifting or automatic updating, is there an option for user control?	0.060	-0.590	0.670	0.230	0.854
66	Is a mechanism available to ignore blockages of content that are repeated on various Web pages?	0.440	-0.060	0.320	0.400	0.460
69	Do the headers and the tags describe the topic or the finality?	-0.430	0.160	-0.230	-0.200	0.303
70	Can the pre-defined human language for each Web page be determined in a programmatic manner?	-0.180	-0.050	0.420	0.220	0.260
71	Does changing the definition of a component of the user interface automatically provoke a change of context, at least when the user has been warned about this situation before using the component?	-0.490	0.220	0.220	0.120	0.351
73	If an input error is automatically detected, is the item that has the error identified and is the error described to the user in text?	0.180	-0.740	0.520	-0.050	0.853
74	Are labels or instructions provided when the content requires inputting data by the part of the user?	-0.880	0.120	0.260	0.130	0.873
75	In the content implemented using languages for marking, do the elements have complete marks at the beginning and end, are the elements fit according to the respective specifications, do the elements have duplicated attributes, and are all the IDs exclusive?	-0.410	0.150	0.170	-0.120	0.234

Dimension	Items
System accessibility and reliability	56 ^* * Items loaded opposite most dimension items. , 57, 58, 60, 66*, 69, 71, 74, 75
Information presentation	03, 06, 12, 22, 23, 32, 33, 35, 37, 40, 43, 45, 48, 56, 64, 73
Interactivity	21, 24, 25, 32, 46, 47, 52, 55, 59, 61, 64, 65, 70
Navigability/ user guidance	05, 08, 10, 19, 27, 28, 29, 30, 35, 38, 66

Item	a 1	s.e.	a 2	s.e.	a 3	s.e.	a 4	s.e.	MDISC	d	s.e.	MDIFF
3	1.43	0.78	0.65	0.76	-0.34	0.64	0.66	0.74	1.74	4.23	0.82	-2.43
6	2.29	1.18	0.98	0.89	-0.03	0.70	0.39	0.93	2.52	4.88	1.39	-1.94
8	-0.02	0.35	0.29	0.40	-0.99	0.33	-0.70	0.39	1.25	0.79	0.19	-0.63
10	0.51	0.42	-0.15	0.43	0.39	0.42	1.20	0.46	1.37	2.09	0.36	-1.53
12	0.66	0.55	1.47	0.58	-0.06	0.57	-0.02	0.50	1.61	2.53	0.44	-1.57
19	1.20	0.39	0.14	0.45	0.75	0.38	0.92	0.40	1.69	1.50	0.25	-0.89
21	1.73	1.03	1.26	1.00	2.35	1.24	0.74	0.99	3.26	5.57	1.46	-1.71
22	1.11	0.79	0.94	0.71	0.33	0.63	-0.48	0.67	1.57	4.02	0.90	-2.57
23	1.22	0.50	0.42	0.62	0.44	0.59	0.91	0.63	1.64	3.61	0.52	-2.20
25	0.27	0.93	2.39	1.30	1.54	0.91	-0.61	0.99	2.92	5.34	1.50	-1.83
27	0.61	0.38	-0.39	0.41	0.24	0.40	1.13	0.40	1.36	-0.62	0.21	0.45
28	0.24	0.43	-0.31	0.43	1.17	0.40	1.01	0.45	1.59	-1.33	0.26	0.83
29	0.70	0.43	-0.71	0.46	0.39	0.47	0.92	0.43	1.41	-2.07	0.32	1.47
30	1.40	0.50	0.30	0.50	1.31	0.47	0.90	0.48	2.14	1.38	0.27	-0.65
32	0.96	0.36	0.38	0.41	0.86	0.40	0.47	0.39	1.42	1.62	0.24	-1.14
33	2.50	1.33	1.91	1.21	0.49	0.92	1.00	0.91	3.34	6.07	1.78	-1.82
35	1.60	0.47	0.20	0.47	-0.13	0.42	0.61	0.49	1.73	2.43	0.37	-1.41
37	0.62	0.30	0.05	0.31	0.22	0.28	0.77	0.28	1.01	-0.36	0.17	0.36
38	1.16	0.44	0.04	0.49	1.60	0.47	0.39	0.48	2.01	0.73	0.24	-0.36
40	0.82	0.29	0.11	0.34	0.06	0.28	0.68	0.29	1.07	-0.15	0.18	0.14
43	0.84	0.53	0.42	0.69	-0.35	0.78	0.75	0.84	1.25	-4.36	0.91	3.48
45	0.68	0.75	1.11	0.76	0.60	0.70	-0.21	0.74	1.45	4.44	0.72	-3.06
46	0.70	0.40	0.59	0.46	1.40	0.45	0.83	0.41	1.87	1.23	0.25	-0.66
47	1.37	1.28	1.51	1.01	0.21	0.86	0.14	1.09	2.05	5.40	1.38	-2.63
48	1.89	0.58	0.73	0.59	0.81	0.54	0.72	0.62	2.30	3.12	0.50	-1.36
52	0.82	0.38	0.63	0.44	1.14	0.43	0.24	0.39	1.56	1.82	0.27	-1.17
55	1.09	0.41	0.14	0.42	0.14	0.38	0.99	0.40	1.49	1.52	0.27	-1.02
56	2.57	1.28	1.03	1.35	0.66	1.14	0.84	1.16	2.97	6.78	2.46	-2.28
57	-1.98	0.97	2.04	1.07	-0.26	0.81	1.63	0.98	3.29	-3.05	0.91	0.93
59	-0.55	0.59	0.27	0.73	2.00	0.91	-0.18	0.65	2.10	3.94	0.99	-1.88
60	-1.30	1.00	0.20	1.19	0.22	1.00	2.03	1.53	2.43	5.74	2.17	-2.36
61	-0.86	0.41	0.62	0.49	0.79	0.42	0.90	0.43	1.60	0.40	0.20	-0.25
64	-0.51	0.82	0.47	1.10	2.08	1.01	-0.34	1.29	2.22	5.59	1.34	-2.52
65	-1.00	1.05	0.86	1.46	2.58	1.92	0.19	1.28	2.90	6.71	2.89	-2.31
66	0.73	0.79	0.06	1.13	1.56	1.05	0.31	0.72	1.75	5.02	1.28	-2.87
69	-0.25	0.34	0.19	0.40	-0.97	0.31	0.30	0.34	1.06	1.10	0.24	-1.04
70	-0.29	0.30	0.48	0.35	0.63	0.31	0.51	0.32	0.99	-0.57	0.20	0.58
71	-0.31	0.45	0.79	0.47	-0.15	0.44	1.01	0.45	1.33	2.28	0.32	-1.72
74	-1.14	0.56	1.18	0.65	-0.36	0.54	1.29	0.65	2.12	-1.77	0.36	0.84
75	-0.35	0.31	0.79	0.35	-0.33	0.31	0.43	0.35	1.02	0.53	0.17	-0.52

Dimension	Items
Navigability/ user guidance	03, 06, 19, 22, 23, 30, 32, 33, 35, 40, 43, 48, 55, 56
System accessibility and reliability	12, 25, 45, 47, 57, 75
Interactivity	08 ^* * Items loaded opposite most dimension items. , 21, 28, 38, 46, 52, 59, 64, 65, 66, 69*, 70
Information presentation	10, 27, 29, 37, 60, 61, 71, 74

*Website*	Navigability/ user guidance	System accessibility and reliability	Interactivity	Information presentation
001	0.105	0.130	-0.864	0.677
002	0.367	0.524	-0.753	0.040
003	-0.887	-0.328	-0.557	0.525
004	0.782	0.324	-0.879	0.314

*Website*	Navigability/ user guidance	System accessibility and reliability	Interactivity	Information presentation
001	0.105	0.130	-0.864	0.677
002	0.367	0.524	-0.753	0.040
003	-0.887	-0.328	-0.557	0.525
004	0.782	0.324	-0.879	0.314

Universidade Federal de São Carlos Departamento de Engenharia de Produção , Caixa Postal 676 , 13.565-905 São Carlos SP Brazil, Tel.: +55 16 3351 8471 - São Carlos - SP - Brazil
E-mail: gp@dep.ufscar.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] AIC - Akaike Information Criterion; BIC - Bayesian Information Criterion.