Usage of Content in Web-Supported Academic Courses
Dr. Rafi Nachmias
is a Faculty member at the
is a graduate student at the
The use of the Internet as an instructional tool in higher education is rapidly increasing. Most universities, including leading academic institutes, are implementing advanced technologies as a part of existing teaching frameworks (Mioduser & Nachmias, 2002). Some models were developed for characterizing online courses (for example, Harasim et al., 1995; Bonk et al., 1999). Harasim (2000) claims that the invention of CMC on one hand and the Internet (specifically the WWW) on the other, led to the development of two main models for online learning: one based on cooperative learning and interaction, while the other relies on publication of information on the Internet. Indeed, we are currently witnessing the development of huge amounts of web-based learning materials (Bork, 2001) and web-based contents has become a major component in many academic courses (Nachmias, 2002). Cummings, Bonk & Jacobs (2002) believe that presentation of educational contents on the Internet is highly valuable for students, who enjoy visual presentation of information and supplements to materials taught in lectures. However, the authors worry that presentation of learning materials does not necessarily lead to their utilization by students in order to enhance their learning or course understanding. Another source of concern derives from the fact that the development of online learning materials requires investment of considerable amounts of resources, financially and in time and effort (Nachmias, 2002). Therefore, the extent to which these resources have added value must be examined (Soong, Chan & Chua, 2001).
The dominance of contents in course websites, the investment in preparing this material, and the lack of certainty regarding their utilization, emphasize the need to examine whether and to what extent students make use of learning materials presented in course websites. However, there are only few studies dealing with this issue (for example, Rafaeli & Ravid, 1997). Moreover, these studies have examined consumption of contents in specific courses and do not deal with fundamental questions relating to content usage in course websites. One of the few studies attempting to identify patterns of content usage in course websites was conducted by Sasson and Nachmias (1999). They analyzed the connection between the structure of information pages in online courses and the way in which these pages were viewed by students. Results showed that viewing of information pages in the course website decreases as a function of the location of the online chapter in the learning process and the location of the lesson within the chapter. In addition, it was found that many information pages were not being viewed at all. In general, then, the observation is that content at deeper levels is less likely to be viewed and used in the learning process.
One of the unique tools that can be used for evaluating content use in course websites is the analysis of the computer log derived while students access the contents. There are number of reasons why analysis of computer log files may be worthwhile (Peled & Rashty, 1999). First, such an analysis gives us insight into students navigate in a course website. Secondly, one can trace the usage of the course website, by looking at usage of its different components. Evaluation of online courses by means of analysis of computer logs also has methodological advantages: it does not suffer from biases due to self-report methods (McLaughlin, Goldberg, Ellison, & Lucas, 1999). In addition, the information regarding students’ learning is accumulated automatically and the data collection does not interfere with everyday learning activities.
The main goal of the current study is to evaluate content usage in websites of academic courses, applying the potential of computer log analysis for this purpose. The evaluation of content utilization involves three main questions.
1. To what extent are contents presented in course websites?
2. What is the usage rate of content items in course websites?
3. What are the individual differences among students regarding content consumption?
“Content item”: Content item is a general name for all types of information uploaded into a database of course websites, i.e. presentations, lesson summaries, exercises, etc. Content items vary also in their presentation format (Word, Acrobat, PowerPoint etc.) and in their size. Each student could access and see only the learning materials of the courses he or she were signed to.
“Viewing a content item”: By the word "viewing" we refer to each time a student clicked on a link to a content item and opened it. We shall have to assume she or he looked at that item and read it.
The study was conducted on 117 course websites which are part of the Virtual TAU project, during 2001. Over 5,000 undergraduate, graduate and Ph.D. students from all faculties took part in these courses.
websites that were included in the study were developed as part of the Virtual TAU project -
Data for this study were gathered from the automatic documentation of students’ viewings of content items in course websites. The following steps were taken with relation to raw data: The number of content items was counted for each course; the number of content items viewed by each student was documented, and the number of students viewing each content item was examined. The data were analyzed by quantitative means, using descriptive statistics.
The results are presented according to the research questions.
Amount of content items presented in course websites:
In 117 course websites examined over 3,300 content items were presented. The average number of items presented in each site was 28 (SD 24.9), ranging between 2 to 118 items for each website. For most courses (88 courses, comprising 75% of the courses in the study) up to 35 items were in existence. Only in one course over 100 items were presented. There were no differences in the number of content items presented in websites of Exact Sciences faculties in comparison to Social Sciences and Humanities faculties, nor between websites for undergraduate vs. graduate courses. A positive correlation was found between the number of students in a course and the number of content items presented in the website (R=0.21; p<0.05).
Characteristics of content item usage:
Examination of contents presented on the website reveals that students viewed a large proportion of this content. Of the 3,301 items available online, students viewed 2,926 items, 89% of all items presented. An average of 25 items in each course website was viewed (SD 21.9). Over 35,000 content items were accessed by the students in all courses together. Items that were not viewed were found in 59 of the 117 course websites (about half of the studied courses), i.e. in about half of the courses all content items were viewed by at least one student.
In addition, results show that each item was viewed by an average of 28% of the students allowed to access it (SD 21%). It was found that 55% of the items were visited by less than a quarter of the students listed in the course, and only 16% of the items gained the attention of over half of the students. Moreover, only 2 items were used by more than 90% of the students, and no content item was found to be used by all students listed in a course.
Individual differences among students:
One of the central findings of this study is that 62% of the students viewed the content items, while 38% of the students listed in the courses did not do so even once. The study shows that the rate of students that did not use the items at all was higher among undergraduate than graduate students (c2=0.09; p<0.01), and again among Social Sciences and Humanities students, as opposed to Exact Sciences students (c2=-0.08; p<0.01).
According to findings, among students using content items there is variance with regard to the number of items viewed. Most students viewed only a small percentage of the available items, whilst few students viewed all of them. About 20% of the students have viewed no more than one tenth of the items available, using only a small part of the information presented in their course website. Other students, however, viewed a greater percentage of the items presented, with 5% consuming all content items presented on the course website. Each student viewed an average of 38% of the items available (SD 30%).
Examination of individual differences among students has shown that students in the Social Sciences and Humanities faculties viewed a larger number of items compared to students in Exact Sciences faculties (t (3165)=-13.5; p<0.01). In addition, graduate students viewed a larger number of items compared to undergraduates (t (3165)=-16.8; p<0.01).
Universities nowadays are offering a lot of distance learning courses via the Internet to potential new students. The more courses are offered this way, the greater the need to understand how students learn from course web-sites. Hence, it is important to outline and study learning patterns of students in course websites.
The present study shows that lecturers in their first year of Internet integration in instruction rely on the course website in presenting large amounts of learning materials. One may conclude from this finding that the lecturers view course web-sites as a useful means for improving teaching. A similar usage pattern of course websites by lecturers was described by Bonk (2001), who found that most lecturers upload files to websites and use means of presenting questions, queries and case studies. Harasim (2000) also recognizes the fact that the Internet has created an unprecedented emphasis on the publication of information. She claims that this derives from the preference of lecturers for traditional teaching models, mainly frontal lectures. Peffers and Bloom (1999) ascribe this emphasis to the fact that these activities are easy to implement and do not oblige a great investment in resources. For many of the lecturers in the current study it was a first and innovative experience in implementing the web in academic teaching. It is possible that they not ready yet to change their instruction philosophy and habits.
One of the remarkable findings in the current study is that most of the content items were viewed by the students. In this respect it seems that the references and resources were useful to the students. This finding regarding the relevance of contents contradicts the descriptions in Sassson and Nachmias (1999), who found great variance in viewing rates of web pages in course websites, with some pages not being entered at all. This contradiction in findings can be explained, among other things, by the fact that the web pages examined by Sassson and Nachmias (1999) were linear in nature. The researchers found that this linearity bore a clear positive relation to the viewing of content within the website. In the current study the materials presented in the course websites were arranged in a variety of ways. Possibly, the fact that the content were not arranged linearly caused students to expose themselves to greater amounts of content items during the search for specific contents.
The study found individual differences among students in viewing the content items in course web-sites. Nearly 40% of the students did not view content items at all. Furthermore, there was a variance among students who did view the contents, with many of them viewing only a few items, and some viewing a lot of them. A similar finding regarding interpersonal differences among students learning with courses web-sites is presented by Nachmias and Shany (2002), who found a large variance in students' tendency to succeed in a virtual course. The researchers found that one third of the students were left outside the learning circle, while the online course suited about 40% of the students. Findings of the two studies match the AFT report (2001), according to which a certain type of student tends to succeed more than others in distance learning courses on the Internet.
Findings related to variance among students with regard to usage of contents and characteristics of item viewing demonstrate the potential of computer log as a tool for evaluating ICT implementation in academic teaching. Furthermore, analysis of a computer log can be a valuable tool for developing a set of measures that will be part of online evaluation tools, which are easy to apply and understand. This kind of tool can be of importance on three different levels: the lecturer level, the project administration level and the decision-maker level:
1. For the lecturer, measures regarding the degree and manner in which students view content items can be a powerful tool for feedback relating to the implementation of the course website. The lecturer can learn about the rate of content item usage by students and make decision on whether certain actions should be taken in order to increase students' viewing of the items.
2. A report including measures regarding students' behavior and content utilization for all courses in a certain project of ICT implementation can be a useful tool for project managers. The measures can be used for instructing the lecturers in developing and operating course websites, as well as for directing implementation efficiently.
3. Measures based on an analysis of computer log can be a unique and powerful tool for comparing the goals of developing course websites with their realization de facto. The measures can be used for comparison between faculties within a university, as well as for follow-up of a project in the course of its operation.
“We are in a period of history when new tools based on information and communications technologies have the potential to advance learning dramatically and make it more accessible” (The Advisory Committee for Online Learning, 2001, p. 28). There is no doubt that all those involved in learning processes are on the verge of an exciting era, that can dramatically change the manner in which students learn, lecturers teach and academic systems are operated. This is an era full of promise also for researchers focusing on different aspects of teaching and learning in academic institutes. For these researchers many questions regarding the implementation of ICT in academia are still a waiting answers.
1. AFT (2001). A Virtual revolution: Trends in the expansion of distance education. American Federation of Teachers, May 2001 [On-line]. Available at: http://www.aft.org/higher_ed/downloadable/VirtualRevolution.pdf
2. Bonk, C. J., Cummings, J. A., Hara, N., Fischler, R. B., & Lee, S. M. (1999). A ten level web integration vontinuum for higher education: New rsources, partners, courses, and markets [On-line]. Available at:
3. Bonk, C. J (2001). Online teacing in an online world [On-line]. Available at:
4. Bork, A. (2001). What is needed for effective learning on the Internet? Educational Technology and Society, 4 (3) [On-line]. Available at:
5. Cummings, J. A., Bonk, C. J., & Jacobs, F. R. (2002). Twenty-first century college syllabi options for online communication and interactivity. Internet and Higher Education, 4, 1-19.
Harasim, L., Hiltz, S. R., Teles, L., & Turoff, L. (1995). Learning networks.
7. Harasim, L. (2000). Shift happens: Online education as a new paradigm in learning. The Internet and Higher Education, 3, 41-61.
M, Goldberg, S. B., Ellison, N., & Lucas, J. (1999). Measuring Internet
audiences. In: S. Jones (Ed.): Doing Internet research (pp.
& Nachmias, R. (2002). WWW in education. In H. Adelsberger, B. Collis, & M. Pawlowski (Eds.), Handbook on Information Technologies
for education & Training,
10. Nachmias, R. (2002). Research framework for the study of a campus-wide web-based academic instruction project. Internet and Higher Education, 5 (3), 213-229.
11. Nachmias, R., & Shany, N. (2002). Learning in virtual courses and its relationship to thinking styles. Journal of Educational Computing Research, 27 (3). 313-327.
12. Peffers, K., & Bloom, S. (1999). Internet-based innovation for teaching IS courses: The state of adoption, 1998-2000. Journal of Information Technology Theory & Application, 1 (1) [On-line]. Available at:
13. Peled, A., & Rashty, D. (1999). Logging for success: Advancing the use of WWW logs to improve computer mediated distance learning. Journal of Educational Computing Research, 21(3) [On-line]. Available at: http://www.rashty.com/art2.html.
14. Rafaeli, S., & Ravid, G. (1997). Online, web-based learning environment for an information systems course: Access logs, linearity and performance [On-line]. Available at: http://mis.huji.ac.il/papers/mmm.paper.htm
15. Sasson, E., & Nachmias, R.
(1999). What makes them click? Paper presented in the ED-MEDIA 99. World
conference on educational multimedia, hypermedia & telecommunications,
16. Soong, M. H. B., Chan, H. C., & Chua, B. C. (2001). Critical success factors for on-line course resources. Computers & Education, 36, 101-120.
17. The Advisory committee for Online Learning (2001). The E-learning E-volution in colleges and universities. A pan-Canadian challenge [On-line]. Available at: