Software Agents in Interactive Multimedia Learning Environments

How can we foster the process of knowledge construction in learners using an agent-based multimedia environment? One of my lines of research has consisted of examining how students learn in various agent-based multimedia environments designed to promote an understanding of environmental science. The major challenge in this research area is to create a computer-based environment that learners will interpret as involving a social-agency relationship with the computer. The central feature of the social agency environment is an animated pedagogical agent--a likable cartoon figure who talks to the learner and who responds to the learner's input. In our study, the aspects of the social agency environment that are examined include: (a) presenting a visual image of the agent's body, especially the agent's face, (b) presenting an auditory image of the agent's voice, using speech rather than on- screen text, and (c) allowing the learner to interact with the agent by providing input and receiving a contingent response. These are some of the major aspects that can help the learner accept his or her relation with the computer as a social one. My ultimate goal is to determine whether techniques aimed at priming a social interpretation in the learner will encourage the learner to make a stronger effort to understand the material.

The program used is based on a multimedia educational game called Design-A-Plant designed by James Lester at the Multimedia Laboratory at the Department of Computer Science of North Carolina State University.

In the Design-A-Plant microworld, students travel to an alien planet that has certain environmental conditions (e.g., low rainfall, light sunlight) and must design a plant that would flourish there (e.g., including designing the characteristics of the leaves, stem, and roots). The program uses a pedagogic agent who offers individualized advice concerning the relation between plant features and environmental features by providing students with feedback on the choices that they make in the process of designing plants.

Related Publications

Moreno, R. (in press). Constructing knowledge with an agent-based instructional program: A comparison of cooperative and individual meaning making. Learning and Instruction.

Moreno, R. (2007). Animated software pedagogical agents: How do they help students construct knowledge from interactive multimedia games? In R. Lowe & W. Schnotz (Eds), Learning with animation (pp. 183-207). New York: Cambridge University Press.

Moreno, R. & Flowerday, T. (2006). Students’ choice of animated pedagogical agents in science learning: A test of the similarity attraction hypothesis on gender and ethnicity. Contemporary Educational Psychology, 31, 186-207. View Abstract

Moreno, R. (2005). Multimedia learning with animated pedagogical agents. In R. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (pp. 507-524). Cambridge University Press. View Abstract

Moreno, R. & Mayer, R. E. (2005). Role of guidance, reflection, and interactivity in an agent-based multimedia game. Journal of Educational Psychology, 97, 117-128. View Abstract

Moreno, R. (2004). Animated pedagogical agents in educational technology. Educational Technology, 44 (6), 23-30. View Abstract

Moreno, R., Mayer, R. E., Spires, H. & Lester, J. (2001).The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19, 177-213. View Abstract

Moreno, R. (2001). Software agents in multimedia: An experimental study of their contributions to students' learning. Human-Computer Interaction Proceedings (pp. 275-277), Mahwah, NJ: Lawrence Erlbaum Associates. View Abstract

Moreno, R., Mayer, R. E., & Lester, J. C. (2000). Life-like pedagogical agents in constructivist multimedia environments: Cognitive consequences of their interaction. ED-MEDIA 2000 Proceedings (pp. 741-746). Charlottesville, VA: AACE Press View Abstract

Virtual Reality Environments for Science Learning

I am currently associated as a research scientist in the Research Center for Virtual Environments and Behavior, located within the Department of Psychology at the University of California, Santa Barbara. I am working in collaboration with Professor Richard Mayer

Immersive virtual environments offer immense potential as educational tools. By applying learning principles developed by cognitive and educational psychologists, my goal is to study how humans learn in virtual environments and how to design virtual environments that maximize learning potential.

Building on previous cognitive research and theory in desktop multimedia, I am currently targeting vocal instruction, for example, as being more helpful to constructivist learning than textual instruction. Furthermore, I am investigating the personalization effect, which suggests that "dialogue" or self-referenced instructions are processed more efficiently than "monologue" type instructions. To test this theory, we have developed an immersive environment in which students will learn, across varying modalities, a lesson in environmental science, and then are tested on what they have learned. The program is based on following is a frame taken from the environment.

A second issue is to determine the cognitive effects of presenting highly visible pedagogical agents who share the virtual world with the student. A third issue, consists of investigating the role of agents' guidance in constructivist virtual environments by comparing different levels of exploration ranging from pure discovery, where the learner is free to explore the VR environment on his own, to guided discovery, where the learner's exploration is guided by the computerized mentor. More generally, I am interested in whether the feeling of being in the simulated environment (presence) can result in a qualitatively different learning outcome than learning in non-immersive environments. test if the set of cognitive principles that have arisen from their prior studies in desktop multimedia learning, such as the contiguity and coherence principles, can be extended to virtual environments.

Related Publications

Moreno, R. (in press). Animated software pedagogical agents: How do they help students construct knowledge from interactive multimedia games? In R. Lowe & W. Schnotz (Eds), Learning with Animation (pp. XX-XX). Cambridge University Press.

Moreno, R. & Flowerday, T. (2006). Students’ choice of animated pedagogical agents in science learning: A test of the similarity attraction hypothesis on gender and ethnicity. Contemporary Educational Psychology, 31, 186-207. View Abstract

Moreno, R. & Mayer, R. E. (2004). Personalized messages that promote science learning in virtual environments. Journal of Educational Psychology, 96, 165-173. View Abstract

Moreno, R. (2004). Immersive agent-based multimedia environments: Identifying social features for enhanced learning. In H. M. Niegemann, F. D. Leutner, & R. Brünken (Eds.), Instructional design for multimedia learning (pp. 9-18). Münster: Waxmann. View Abstract

Moreno, R. & Mayer, R. E. (2002). Learning science in virtual reality multimedia environments: Role of methods and media. Journal of Educational Psychology, 94, 598-610. View Abstract

Deriving Instructional Design Principles from a Cognitive Theory: The Case of Multimedia Explanations

How can we help students understand scientific systems? One promising approach involves using a multimedia explanation: the combination of visual and verbal representations of the system to-be-learned. For example, below, you can see selected frames from a computer-generated animation about the process of lightning formation synchronized with an on-screen text explanation of the phenomenon.

In this line of research, I proposed a cognitive-affective theory of learning with media (CATLM; Moreno, 2005) from which principles of instructional design can be derived and tested. Some of the principles include:

Visual Split-Attention Principle
Students learn better when the instructional material does not require them to split their visual working memory between multiple sources of mutually referring information.

Auditory Split-Attention Principle
Students learn better when extraneous auditory material (such as music and irrelevant sounds) is excluded rather than included in multimedia explanations.

Modality Principle
Students learn better when the verbal information is presented auditorily as speech rather than visually as on-screen text both for concurrent and sequential presentations.

Spatial Contiguity Principle
Students learn better when on-screen text and visual materials are physically integrated rather than separated.

Temporal Contiguity Principle
Students learn better when verbal and visual materials are temporally synchronized rather than separated in time.

Self-Reference Principle
Students learn better when instructional explanations are provided in a personalized, dialogue style rather than a non-personalized, monologue style.

Related Publications

Moreno, R., & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309-326. View Abstract

Moreno, R. (2006). Learning with high tech and multimedia environments. Current Directions in Psychological Science, 15, 63-67. View Abstract

Moreno, R. (2006). Does the modality principle hold for different media? A test of the method affects learning hypothesis. Journal of Computer Assisted Learning, 22, 149-158. View Abstract

Moreno, R., & Valdez, F. (2005). Cognitive load and learning effects of having students organize pictures and words in multimedia environments: The role of student interactivity and feedback. Educational Technology Research and Development, 53, 35-45. View Abstract

Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback on discovery-based multimedia. Instructional Science, 32, 99-113. View Abstract

Moreno, R. (2003). Assessing the effectiveness of multimedia environments: Empirically-based guidelines for teachers. Society for Information Technology and Teacher Education (SITE) 2003 Proceedings (pp. 3716-3719). Charlottesville, VA: AACE Press. View Abstract

Mayer, R. E. & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. In Bruning, R., Horn, C. A., & PytlikZillig, L. M. (Eds.), Web-based learning: What do we know? Where do we go? (pp. 23-44). Greenwich, CT: Information Age Publishing. View Abstract

Mayer, R. E. & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43-52. View Abstract

Moreno, R. & Mayer, R. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94, 156-163. View Abstract

Mayer, R. E. & Moreno, R. (2002). Aids to computer-based multimedia learning. Learning and Instruction, 12, 107-119. View Abstract

Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14, 87-99. View Abstract

Moreno, R. & Mayer, R. E. (2000). A learner-centered approach to multimedia explanations: Deriving instructional design principles from cognitive theory. Interactive Multimedia Electronic Journal of Computer Enhanced Learning, http://imej.wfu.edu. View Abstract

Moreno, R. & Mayer, R. E. (2000). Engaging students in active learning: The case for personalized multimedia messages. Journal of Educational Psychology, 92, 724-733. View Abstract

Moreno, R. & Mayer, R. E. (2000). Meaningful design for meaningful learning: Applying cognitive theory to multimedia explanations. ED-MEDIA 2000 Proceedings (pp. 747-752) Charlottesville, VA: AACE Press View Abstract

Moreno, R. & Mayer, R. E. (2000). A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 97, 117-125. View Abstract

Moreno, R. & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358-368. View Abstract

Mayer, R. E. & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual information processing systems In working memory. Journal of Educational Psychology, 90, 312-320. View Abstract

Visual Metaphors for Meaning Making in Mathematics

How can we use multimedia environments to help students build connections between mathematical procedures and their existing conceptual knowledge? For example, what can be done to help students make sense out of procedures for addition and subtraction of signed numbers? This line of research starts with a theory of learning as model-building, according to which learning is a constructive process of sense making in which learners make connections between new information and existing conceptual models. More specifically, I am interested in using visual aids, such as the number line metaphor, to help students build connections between the formal computational procedures that use symbols and the informal conceptual knowledge about moving along a path. The following figure depicts one implementation of the number-line metaphor within a computer-supported multimedia environment.

Some of the questions that my research in this area intends to answer are: Would multiple representations of an arithmetic procedure help students' understanding? What is the role that verbal representations play in the learning process? Do students who are in the transition to learning English use their first language to guide them in the understanding of the mathematical process if they are given the option to do so?

Related Publications

Moreno, R. & Durán, R. (2004). Do multiple representations need explanations? The role of verbal guidance and individual differences in multimedia mathematics learning. Journal of Educational Psychology, 96, 492-503. View Abstract

Moreno, R. (2002). Who learns best with multiple representations? Cognitive theory implications for individual differences in multimedia learning. ED-MEDIA 2002 Proceedings (pp. 1380-1385). Charlottesville, VA: AACE Press. View Abstract

Moreno, R. & Duran, R. (2001). Interactive visual metaphors in multimedia: Aids to math learning among English language learners. Paper presented at the annual meeting of the American Educational Research Association. Seattle, WA

Moreno, R. & Mayer, R. E. (1999). Multimedia-supported metaphors for meaning making in mathematics. Cognition and Instruction, 17, 215-248. View Abstract

Learning with Instructional Animations

How do animations need to be designed to promote learning? Past research has shown that the combination of simultaneous narrated explanations with pictures or animations is most effective in promoting learning as measured by verbal transfer tests, a finding that has been called the multimedia principle (Mayer, 2001; Moreno, 2006). Despite this evidence, still open are the following questions: Do visual representations of complex systems need to be animated to help students’ visualize the relationship between the elements of the system? Can animations help students change their scientific misconceptions? Who learns best from animations? In this line of research, my colleagues and I have investigated the conditions for fostering effective learning with animated displays and individual differences in learning from such displays.

Related Publications

Moreno, R. (2007). Optimizing learning from animations by minimizing cognitive load: Cognitive and affective consequences of signaling and segmentation methods. Applied Cognitive Psychology, 21, 1-17. doi: 10.1002/acp.1348. View Abstract

Moreno, R., & Marley, S. (2007, August). Do students’ verbal and visual abilities and preferences affect their learning and perceptions about learning astronomy with static and animated graphics? Paper presented at the European Association for Research on Learning and Instruction Annual Conference, Budapest, Hungary.

Moreno, R., Marley, S., & Helak, J. (2007, April). Cognitive and affective consequences of learning astronomy with and without static and dynamic visual representations Paper presented at the 2007 annual meeting of the American Educational Research Association (AERA), Chicago, IL. View Abstract

Moreno, R., & Plass, J. (2007, April). Relationship of cognitive abilities, cognitive styles, and learning preferences and their effect on multimedia learning. Paper presented at the 2007 annual meeting of the American Educational Research Association (AERA), Chicago, IL.

Moreno, R. & Plass, J. (2006). Individual Differences in Learning with Verbal and Visual Representations. Proceedings of the 2006 Steinhardt Technology and Learning Symposium. http://create.alt.ed.nyu.edu/symposium2006/proceedings.html

Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14, 87-99. View Abstract

Cognitive Load Theory, Research, and Applications

Cognitive Load Theory (CLT) allows us to predict learning outcomes by taking into consideration the capabilities and limitations of the human cognitive architecture. The theory can be applied to a broad range of learning environments because it links the design characteristics of learning materials to principles of human information processing. In this line of research, we have investigated how instructional design may affect students' cognitive demands and learning and have proposed a set of instructional principles aimed at reducing cognitive load.

Related Publications

Moreno, R., Brünken, R., & Plass, J. (in review). Bridging theory and application of CL theory. In J. Plass, R. Moreno, & R. Brüenken (Eds), Cognitive load: Theory and application. New York: Cambridge University Press.

Moreno, R., & Mayer, R. E. (in review). Techniques that increase germane cognitive load in multimedia learning. In J. Plass, R. Moreno, & R. Brüenken (Eds), Cognitive load: Theory and application. New York: Cambridge University Press.

Mayer, R., & Moreno, R. (in review). Techniques that reduce extraneous cognitive load and manage intrinsic cognitive load during online multimedia learning. In J. Plass, R. Moreno, & R. Brüenken (Eds), Cognitive load: Theory and application. New York: Cambridge University Press.

Moreno, R. (2007). Optimizing learning from animations by minimizing cognitive load: Cognitive and affective consequences of signaling and segmentation methods. Applied Cognitive Psychology, 21, 1-17. doi: 10.1002/acp.1348. View Abstract

Moreno, R. (2006). When worked examples don’t work: Is cognitive load theory at an impasse? Learning and Instruction, 16, 170-181. View Abstract

Moreno, R., & Valdez, F. (2005). Cognitive load and learning effects of having students organize pictures and words in multimedia environments: The role of student interactivity and feedback. Educational Technology Research and Development, 53, 35-45. View Abstract

Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback on discovery-based multimedia. Instructional Science, 32, 99-113. View Abstract

Empirically-based Instructional Tools for Engineering Education

Related Publications

Reisslein, M. & Moreno, R. (2007). Instructional strategies for pre-college engineering education. Proceedings of IEEE/ASEE Frontiers in Education (FIE) Conference (pp. F1B-1--F1B-2). Piscataway, NJ: IEEE Press.

Moreno, R., Reisslein, M., & Delgoda, G. M. (2006). Toward a fundamental understanding of worked example instruction: Impact of means-ends practice, backward/forward fading, and adaptivity. Proceedings of IEEE/ASEE Frontiers in Education (FIE) Conference (pp. S3D-5–S3D-10). Piscataway, NJ: IEEE Press.

Reisslein, M., Moreno, R., & Reisslein, J. (2005). WIP: Bridging Cognitive and Motivational Psychology to Combat Shortage of Engineers. Proceedings of IEEE/ASEE Frontiers in Education (FIE) Conference (pp. F1E-30-F1E-31). Piscataway, NJ: IEEE Press.