This paper reports on an investigation in which usability testing was applied to a computer-mediated reading comprehension task. The concept "usability" was divided into design usability - focusing ont he surface of materials - and instructional usahility - focusing on the presentation of feedback and reading strategies. Five beginning EFL students were selected and asked to read a text and answer comprehension questions using the CALL activity. This activity provided the subjects with assisted performance through an online dictionary, open-answer comprehension questions, and different types of feedback, automatically activated by the system. Data were collected through think aloud protocol, researcher's notes and after-task questionnaire. The results showed that students, through assisted performance, were able to construct meaning from a text that was beyond their linguistic competence. The implications of the study for the preparation of CALL materials are also presented and discussed.