Research&Innovation Actions
Title of Proposal
Multisensory-enhanced
Innovative Learning Network
(MultILearNet)
List
of participants
Participant No * |
Participant organisation
name |
Country |
1 (Coordinator) |
Friedrich-Alexander-Universität |
Germany |
2 |
National Technical University of Ukraine
“Kyiv Polytechnic Institute” |
Ukraine |
3 |
Athlone Institute of Technology |
Ireland |
4 |
Holografika |
Hungary |
5 |
Imaginary |
Italy |
6 |
LiveU |
Israel |
7 |
FORTH |
Greece |
8 |
Brunel University |
UK |
9 |
Dublin City University |
Ireland |
10 |
RGB Screen Networks Ltd. |
Ireland |
11 |
Slovak University of Technology |
Slovakia |
12 |
Middle East Technical University |
Turkey |
The MultILearNet
proposal will address the need for robust and fit-for-purpose digital
technologies for learning. The aim is to create multisensory learning
environments that represent realistically the natural environment and evaluate
its technical and pedagogical effectiveness. The consortium combines industry
and academia cooperation to develop an innovative approach to learning by using
multisensory environment to mimic the real life environment and create adaptive learning, use augmented cognition technologies, virtual
environments/virtual worlds to real-life learning situations.
Traditional learning was based on students in a passive mode with
the tutor or teacher presenting information. This has currently evolved to
support various blended learning approaches. The next logical step is the
creation of multisensorial multimodal media learning
environments and this is the key aim of the MultiLearNet
project.
The importance of
using naturalistic environments to study human perception and information
systems has been emphasized by many researchers (Findlay & Gilchrist, 2003;
Hayhoe & Ballard, 2005; Land, 2003). Naturalistic
environments contain complex sensory patterns, and our perceptual systems
evolved and developed to allow us to sense and interpret these patterns. Until
recently, it has been difficult for scientists to conduct carefully controlled
experiments with naturalistic stimuli. However, new technologies, such as
advanced computer graphics chips and virtual reality environments, are allowing
researchers to investigate human perception in more realistic settings than has
been possible in the past.
Multimedia systems try to take
advantage of human senses to facilitate human communication and improve
educational delivery and learning. Multimedia systems have the potential to
make more efficient use of the human perceptual and cognitive capabilities
because the human brain has evolved in a multisensory environment. Therefore
the use of multiple media can make the interaction and learning more natural.
However, existing multimedia systems address mostly only two senses by using
two communication channels (visual and audio) of the five human senses (sight,
hearing, taste, smell, and touch), limiting its potential efficiency. Though
the current progress in technology enables to register and reproduce more types
of information. Examples of new types of devices are electronic nose,
electronic skin, etc. The task of multimodal data representation, compression,
security, transfer, storage, retrieval, and reproducing is topical and its
solving creates new amazing opportunities for learning / teaching,
entertainment and telecommunications. The range of potential applications of MultILearNet is quite wide. One of the most exiting areas
of application lies within education and training where MultILearNet
technology will be able to recreate real like environments and especially
environments not easily accessible by the majority of people like, volcanos,
the artic, mountains, deep see etc. More
precisely, educational topics that can be addressed are: geography,
environmental education, physics etc. Hardware necessary for such applications
include 3D scanners, 3D projectors, surround sound systems, Olfactory
devices and high-speed network connections which are becoming wider available
and its costs decreases considerably. To strengthen the impression of a
presence a mechanism for tactual information registration and reproduction is
necessary.
The use of smell detectors and
reproducers is also necessary, since the environment can be characterized by
specific smell (e.g. sea, forest, etc.) as well as the interlocutor can also be
a bearer of specific smell – e.g. he/she can use some perfume. Thus,
information about the real object or person and its or his/her environment
consists of visual information, audio information, tactile information,
information about smell and environmental conditions (temperature, humidity, pressure).
MultILearNet will
achieve its goal by creating 4 such installations where real environments will
be reconstructed using either data obtained by the users or data freely
available to the public through research institutes. The installations will be
created in different partner locations and in different types of places to
achieve accessibility and sustainability. One installation will feature the
highest form of all considered peripheries, including a full-scale light-field
3D display for multi-user glasses-free visualization, while the other
installations will use lower cost alternatives (such as desktop 3D displays,
3DTVs or head-mounted displays) to maintain an acceptable level of overall
project cost.
One installation will be developed
in Ireland at Cinema environment at RGB Screen Networks in Ireland. RGB Screen
Networks currently have a commercially operating multisensory theatre that can
support up to nine simultaneous users. It can provide effects such as 2D &
3D video, audio, a moveable platform (e.g. for earthquake simulation), as well
as multiple wind, mist, snow and smoke devices in addition to strobe lighting
effects. Such installations will guarantee two things, first, accessibility due
to the number of people using the cinemas (their current estimated annual
footfall is approx. 25000 per annum) and second sustainability of the project
developments because RGB will sustain the installation well beyond the end of
the project. The second installation will be developed at the technological
museum in Kiev which will also secure access to a large number of visitors and
users and its wide range of pedagogical activities. The third installation will
be created at the media lab of ILI where can be used to test the effectiveness
of multisensory learning beyond the duration of the project. Finally, the place
of the last installation will be decided during the project based on the
requirements analysis and contact with museums to achieve as wider
accessibility and success as possible. One such possible candidate will be the Deutsches Museum in Munich which is one of the largest in
Europe and very active in educational programmes. Alternatively, MultILearNet will use Oculus Rift-like head mount device
technology to create more cost effective and adaptive environment without
losing in the other main sensory modes.
The programme will validate the
technology and the educational benefits of the multisensory learning approach
in real-life pilots in the consortium countries were the installations will be
based and in the rest of the countries the head mount devices will be tested
with at least 20-30 users evaluating the technical and pedagogical aspects of
the devices. Additionally the sustainability of the installations beyond the
end of the project will be guaranteed because some of those installations will
be created by private companies and museums which in turn will sustain them
since they will be part of their normal businesses.
One of the tasks to be solved in
the proposed project consists in the development of new data formats for
optimal representation of complex data about a real object. The list of issues
related to this project:
1. Complex data representation,
including data compression and data security
2. Registration and reproducing of
3D images
3. Registration and reproducing of
3D sound
4. Registration and reproducing of
olfactory data (smells)
5. Registration and reproducing of
touches
6. Registration and reproducing of
environmental conditions
7. Multisensory learning
In order for MultILearNet to realise its
goals the project sets the following objectives presented in the section below.