International Journal

on Marine Navigation

and Safety of Sea Transportation

Volume 6

Number 1

March 2012

1 INTRODUCTION

According to STCW section A-I/12”..... the simulator

shall be capable of simulating the operating

capabilities of shipboard equipment , to a level of

physical realism appropriate to the training and

assessment objectives.....”.

In trying to identify the capability of maritime

simulator course, we can say that is multifunction’s

tool through which special techniques in ships han-

dling either to deck or to engine department can be

promote. More specifically, we can say that maritime

simulator can work a knowledge accelerator for the

seafarer in order to protect the human life at sea and

the environment protection.

Actually the simulator course is an interactive

course among the machinery, the instructor and the

students. But how do those three ingredients interact

with each other, under which pedagologic model?

And finally who makes the evaluation and the as-

sessment of this course?

Those are some of the main questions which we

will try to answer through our research.

The main goal in most of the pedagologic theories

is what finally the students take from the course in

long term future and not by the end of the course.

From literature view the closer pedagologic theories

to the virtual learning like maritime simulation are:

Problem based learning, Discovering Learning,

Learning by exploration, just in time teaching, Case

based learning. The main guide line in all this theory

is that the evaluation of the course stems from the in-

teraction between the student and the instructor.

Based on this acceptance, in our research we will

try to apply the Concern Based Model in Full Mis-

sion Engine Room Simulator which has steadily been

introduced and corresponded to a fraction of the aca-

demic syllabus in the merchant marine academy of

Chios. This Study is a part of an expanded research

according to the Concerned Based Adoption Model

(CBAM) which includes the:

− Stage of Concerned

A Methodological Framework for Evaluating

Maritime Simulation

P. Vasilakis & N. Nikitakos

University of Aegean Department of Shipping Trade & Transportation, Chios, Greece

ABSTRACT: The application of simulation courses according to STCW conversion is addressed to the edu-

cation of marine deck and engine officers in order to familiarize with the working environment, emergency

contingency training and trouble shooting. This paper presents a framework which evaluates the participants

in the courses of simulator, according to their concerns and their level of use. Actually this framework is an

innovative concept which tries to identify how the contributors think and work in this virtual environment.

The results from the application of this framework are presented in this paper, based on student’s concerns,

reactions and level of use with respect to the exercise and efficiency of simulation training courses taken place

at the Merchant Marine Academy of Engine Officers on Chios Island. The main goal of our research is to

promote a general framework which can be easily applied in any marine simulation curses and will be very

useful to the instructor for reorganizing, redesigning and finally configurating the Simulation Courses accord-

ing to their participant.

− Level of Use

Within this paper, the value of simulation along

the stage of concern and the Level of Use of simula-

tor taught courses will be examined. Data are ex-

tracted from student reactions that have participated

and experienced simulator applications. The analysis

was based on the theoretical model C.B.A.M. Results

are extracted and anticipated to present the percep-

tion skills of participating students as well as their

reactions and feedback.

The scope of the analysis is to obtain results that

will enable the academic staff to ascertain the needs

and requirements for improvements in order to de-

velop more resourceful and efficient methods for

simulation course education. Thus the application of

simulators will be far more productive for the partic-

ipating students enabling them also to be adapted in

the real working environment.

The objective goal of application of simulator

courses is the reproduction of virtual reality cas-

es/problems that converge to the real operation tech-

niques and troubleshooting that the officers might

encounter during real operation at the ships engine

room.

By evaluating the performance of simulator

courses it is anticipated and expected to improve the

offered education; thus the candidate officers will

make a success when they face the “real world”. It

will be a virtue and success of the academic schools

that are associated with the maritime education to

graduate qualified officers who are not only quali-

fied graduates but they also bare the knowledge and

ability to undertake and effectively execute their du-

ties without any doubt and with the expected profes-

sionalism and responsibility.

2 PARTICIPANTS

The participants who took part in the CBAM ques-

tionnaire are students of Merchant Marine Academy

on Chios Island. The Academy was founded in 1955

and currently the supported by the Greek Ministry of

Economy, Competitiveness & Shipping. The dura-

tion of studies is 4 years amongst which 12 months

are practice on merchant marine vessels. The rest

three years consist of a syllabus of theoretical and la-

boratory courses. The Orientation of Students is both

theoretical and technical education according to

Sandwich Courses and STCW. Graduates are granted

with the diploma of 3rd Engineer of Merchant Ma-

rine.

By total of sixty five students that participated in

the course of simulation during spring (half-year) pe-

riod 2008-2009, fifty questionnaires were collected

in total. Nine questionnaires were not fully complet-

ed thus they were excluded from the analysis.

In order to present a complete picture for the

standards and knowledge of the participating students

the following are noted:

− The students are in a percentage of 60% of 20-22

years if age.

− Most students came from high school with general

education, in a percentage of 46, 3%. This fact de-

termines that they are individuals with high level

of theoretical knowledge.

− Those that emanate from high school with tech-

nical education represent a percentage of 43, 90%.

− All of them declare that they have been trained in

the use of the English language and they evaluate

their knowledge as on average level in a percent-

age of 60%.

− Only a percentage that reaches the 17% is aware

of a second foreign language, mainly German also

reported to be used at an average level.

− The students that participated in the research have

completed the compulsory educational travels,

thus they have completed their practical educa-

tion, therefore, it is agreed that they have a com-

pleted aspect with regard to their education as

well as the profession they will follow upon suc-

cessful graduation.

− In the submitted questions to the students referring

to the level of education, a percentage of 78% de-

clared satisfied while a percentage of 48% evalu-

ated the level of education as very good.

3 CONCERNED BASED ADOPTION MODEL,

(C.B.A.M)

The Concerns-Based Adoption Model- (CBAM)

mainly deals with the effort of description, measure-

ment, and explanation of process of change in educa-

tion.

Figure 1. Concern Based Adoption Model

The CBAM, is experienced both the instructors

and their students that try to apply and follow new

procedures in teaching and studying as well as the

implemented educational material and practices.

With the assistance of figure 1, it is shown that

CBAM is a framework designed to help change facil-

itators indentify the special needs of individuals in-

volved in the change process and address those needs

appropriately based on the information gathered

through the model diagnostic dimensions.

With those diagnostic data, the instructor could

then develop a policy for any required interventions

in order to facilitate the change effort. Together the

SoC and LoU provide a powerful description of the

dynamics of an individual involved in change; one

dimension focusing on feelings, the other on behav-

iorural patterns.

4 STAGE OF CONCERN

Stages of Concern provide a framework from which

to understand the personal side of the change pro-

cess. According to this we assign different priorities

and level of interest the things we perceive, individu-

ally and in various combinations, but most of the

time we have little or no interest in most stimuli.

Concerns are an important dimension in working

with individuals involved in a change process like the

simulation course in the maritime education.

Figure 2 : Cycle Diagram of understanding

Seven Stages of Concern about an innovation

have been identified (see figure 2).They are called

stages because usually there is a development

movement through them. The participants of a simu-

lator course may experience a certain type of concern

although in the process they may experience another

kind of concern. Another type of concern may

emerge. The Stage of Concern about a simulator

course appears progressively from little or no con-

cern, to concern about the task of adopting by simu-

lator, and finally to concern about the whole impact

of the simulator course. The Stage of Concern Ques-

tionnaire (SoCQ) is the primary tool for determining

in which stage the individual is.

5 THE LEVEL OF USE

The levels of use represent models of behavior which

are classified in eight separate categories. The focus

is on what the individual is doing or not doing. Those

behavior models mainly focus on the actions of indi-

viduals which have completed the course regardless

if the outcome is successful or not (see table 2). Each

model is recorded and analyzed as well as a series of

individual reactions that are also connected with the

particular behavior. The result is drawn on a table 5

that consists of the levels of use along with the seven

categories which we are examining and the prede-

termined behaviors

6 METHODOLOGY-ANALYSIS

The methodology utilized in this research is the fo-

cused interview. A typical questionnaire contains

questions in which we usually anticipate answers that

are related to questions asked during each structure

of object.

6.1 MEASURING THE STAGE OF CONCERN

The Stage of Concern Questionnaire (SoCQ) are de-

veloped in order to measure the concern of the indi-

vidual. The data analyses were based on the follow-

ing:

1 Determine the highest perceived concern for each

participant, along with one or two also high con-

cerns. The remaining stages are characterized by

default, of lower concern.

2 SoCQ percentile stage scores (seven plus total)

The validity of the stage of concern questionnaire

is investigated by examining the highest and the low-

est scores at each stage separately and related to one

or another stage. In the duration of the investigation

all the other variables as concerns theory seizes are

taken into consideration.

The participants in this study used a 0-7 scale to

respond each item. The highest response indicated

that the person considered items in its scale; the sum

of the scale scores constituted the total score. Exam-

ining both the highest and the second highest a more

detailed interpretation is possible. Analyzing the

complete profiles allows us the most sensitive inter-

pretations of responders.

6.2 MEASURING LEVEL OF USE

As a definition, the Levels of Use are the sequence a

user which follows during his progression while us-

ing an innovation. It is addressed to the adapted

methodology by the user while he gains his confi-

dence with his developed skills in order to ascertain

the use of the educational innovation. Following the

same logic, however, a different individual may re-

main unchangeable during the duration of process of

change.

Table 1. STAGE OF CONCERN

Table 2: The level of use of the innovation

Table 3: Result Table of stage of concern

___________________________________________________

Stage Percent % Group

___________________________________________________

0 Awareness 12.75% SELF 39,25%

1 Informational 13.79%

2 Personal 12.71%

___________________________________________________

3 Management 11.42% TASK 23,92%

4 Consequence 12.50%

___________________________________________________

5 Collaboration 13.93% PERFORMING 27,43%

6 Refocusing 13.50%

___________________________________________________

The adopted methodology is outlined in the fol-

lowing 3 points:

− The questionnaire developers investigate the va-

lidity of the LoU by examining the relation of

scores on the seven stages scales amongst each

other as well as amongst variables suggested by

theory of Level of use.

− The behaviors that are described at each intersec-

tion in the LoU Chart are derived from combining

the described of a category of this level. Overall

the Lou chart is describe and reliably measured.

Each level of use represents a different approach

in using a simulator.

− Each LoU is described in terms of the types of be-

haviors represented in the intersections in the

chart of each category with a particular Level.

Table No 1 shows the levels of use according to

model C.B.A.M. The individuals that participate in

the educational innovation take for granted that they

function in higher level than the level of routine IVA,

so that the innovation is maintained and its use is

adopted.

In the case of Levels of Use it is assumed that the

inquired person has a good comprehension of the

theoretical frame Level of Use and then direct him-

self accordingly and proceeds by answering the ques-

tions. Even if the questionnaire is drawn with various

types of discriminations so that it facilitates the user

but also us, a lot of points exist where discrimination

between the formal responses is sensitive, thus the

inquired person will have to reply in a consistent and

reliable way in order to be effectively evaluated.

In order to access and mark the usefulness of En-

gine Room Simulation, always according to the theo-

ry of Level of Use the behaviors of individuals were

developed and delimited in the seven categories (see

Table 4 ).

7 RESULT ACCORDING THE CONCERN

Towards classified the percentile result which ob-

tained and following the theory of high and low stage

of concern table 3 coming up. The results which can

be extracted about the concern of participants accord-

ing to the highest percentiles is that 13.95% indicates

a great concern about collaboration between the par-

ticipant and the relation which has been developed

during the simulation courses. The interpretation for

this result can be considered very logical, if we com-

pare it with the real work environment in the engine

room of a ship. The second highest percentage is

13,79% and indicates concern about the collecting of

information and knowledge for the simulation course

and the definition of all this process.

In whole process of simulator courses the way

which cope the 13.50% present of participant is very

interesting. That indicates participants who are inter-

ested in learning more from the whole procedure of

the task. They focus on exploring ways to reap more

universal benefits from the simulation courses, in-

cluding the possibility of making major changes to it.

8 RESULT ACCORDING THE LEVEL OF USE

Based on the theory of Level of use and the response

of participants the result of our research is classified

between level of use and categories coordinates at

the table No4. The highest percentage is in the cate-

gory of sharing and at the category of assessing

which is classified at the level of use V (integration).

Integration indicates that our participants combin-

ing their own efforts to use the simulation course

with the related activities of colleagues to achieve a

collective impact for their own common sphere of

influence.

As for the category of assessing, participants in-

dicate that they appraise collaborative use of the in-

novation in terms to increasing his own outcomes

and strengths and weakness as the integrated effort.

At the categories of Status Reporting and at the

Acquiring information the users indicated Level of

Use IVA Routine: that means they determine the use

of simulator and few changes take place in ongoing

use.

The weaknesses of the users indentified on the

knowledge, planning and performing category. Ac-

tually in this category the instructor must be given

more attention in order to enhance the output of

simulator course.

9 PEDAGOGICAL VIEW

From pedagogical view the simulator courses can be

used under cooperation of cognitive theories based

on virtual reality learning. As mentioned at the be-

ginning of our research many theories can be

adapted in order to enrich the whole procedure. The

final choice of correct theory must be a combination

of the simulation task and the following correct the-

ory. The correct approach of pedagogical theory dur-

ing the course can be defined from the category of

knowledge and performing at level of use. During

our research we investigate and compare the closest

pedagological theory which can be adapted to simu-

lation course the result can be found in table 5. The

main purpose of adoption of the pedagogical theory

in simulation course is to engage the participant in

the role of transferring and generating cognitive in-

side from the hall procedure of simulation curses as

much as possible. In an effort to indentify in which

stage of learning our participant are; we adapted the

theory learning theory of Kolb. According to this

theory typically expressed as four-stage cycle of learning,

1 Concrete Experience - (CE)

2 Reflective Observation - (RO)

3 Abstract Conceptualization - (AC)

4 Active Experimentation - (AE)

Furthermore we are trying to assign the results

from the stage of concern with the ones from the

stage of learning of Kolb theory. As the theory of

concern says we categorized the seven stages of

concern into three groups ( see table 4). Those

groups are: from stage 0 to stage 2 called Self

Group. Stage 3 and 4 called Target group. Stage 5

and 6 called Performance group. We summarized

the percentile score for each group and the result is

for Self concern group 39.25%, for Target concern

group 23.92%, and for group of performance

27.43%.

Finally we pair the data from stage of concern

with the stage of learning (see figure 2) and the find-

ings are that our participants in 39.25% are between

the concrete experience and the observations. The

23,92% percent is between reflective observation

and abstract conceptualization. Respectively the

27,43% are between the abstract conceptualization

and active experimentation.

10 CONCLUSIONS

The conclusion of the research is the fact that the

surveyed students experienced and accepted the in-

troduction of simulation in training positively by ex-

pressing an interest to learn more about the applica-

tion. One of the most important outputs, is the ex-

pression of strong concern on the basis of the devel-

oped collaboration amongst students, during the

simulation.

This findings probably are the main outcome of

our research given that:

1 The simulator course works like knowledge ac-

celerator and transfer knowledge.

2 The purpose of the engine simulator is to dupli-

cate activities carried out in the engine rooms of

ships.

3 In the engine rooms of ships there should be es-

tablished and imposed conditions of teamwork

and cooperation, rather than surface level and

should actually include exchange of views and

knowledge amongst the crew.

4 27.43% participants fell comfortable with the

simulation course and they are ready to apply

their knowledge in order to learn by trial and er-

ror and ensure their thoughts about the operation

of engine room.

It seems that the students are fully aware of the

role and importance of simulation and begin to seek

partnerships and relationships that would assist them

to cope with the theoretical exercises that might also

experience the simulator in practice. In conclusion,

the findings that the instructors received on the reac-

tions and concerns of students on the receptivity of

the implementation of the simulator as part of their

basic education could be positively described.

Also encouraging data for the full and smooth ac-

ceptance of simulation in education process have

been addressed.

− At the end of the course the 39,25% of partici-

pants are between of concrete experience and

observation which means they don’t realize what

exactly happened or they don’t have enough time

to react .

− Not given or there are not enough incentives from

the instructor to the participants for planning or

to performing during the course.

The participants indicate a low concern to organ-

izing, managing and scheduling the simulation

course.

11 THE EFFECT OF OUR RESULT

− The instructor decides to corporate more closely

with the manufacture company to readjust the

simulation course in the specific needs of student.

− The syllabus of Academy reorganized according

to the needs of simulation course.

− The instructor decide to devote much more time

at the stage of debriefing and more general to in-

form the participant for the main purpose of

simulation course.

It is the first time that the C.B.A.M. “framework”

is used as a tool to measure the outcome of the ap-

plication of marine simulation in a merchant marine

Academy. It was anticipated to provide an output re-

lated to the concerns and aptitude not only from the

users but also the instructors. It could be used as a

tool to measure the effectiveness between the in-

vestment of simulation and the knowledge which the

students acquire.

12 FUTURE RESEARCH

This paper is an initial part of our research. It is in

our future plan to apply the CBAM model in as

much as possible maritime simulators applications,

in Europe but further more in Asia. Our main pro-

posal is to create a flexible tool which can be

adapted in each maritime simulations course in order

to determine the effects of the courses to the seafar-

ers. From the other point of view, according to the

response of the seafarers, it will determine the eval-

uation of the teaching and the pedagogical method

which can be applied.

REFERENCES

Wickens C.D., Kramer A. (1985). Engineering Psychology.

Annual Review of Psychology. Vol. 36: 307-348.

Newhouse C.P. (2001). Applying the Concerns-based Adop-

tion Model to research on computers in classrooms. Journal

of Research on Technology in Education, 33(5).

George A.A., Hall G.E., Stiegelbauer S.M. (2006). Measuring

Implementation in schools: The Stage of Concern Ques-

tionnaire. Austin, Texas, SEDL.

Hall G.E., Dirksen D.J., George A.A. (2006). Measuring Im-

plementation in scholls: Level of Use. Austin, Texas:

SEDL.

Willis J. (1992). Technology diffusion in the soft disciplines:

Using Social Technology to support information technolo-

gy. Texas.

McKinnon D.H., Nolan P.C.J. (1989). Using computers in edu-

cation: A concerns based approach to professional devel-

opment for teacher. Australian Journal of Education Tech-

nology, No. 5, May.

Petruzella F.D. (1995). Industrial Electronics. McGraw-Hill.

Anderson S.E. (1997). Understanding Teacher Change: Revis-

iting The Concerns Based Adoption Model. Curriculum In-

quiry, 27(3), 331-367.

Hord S.M., Stiegelbauer S.M., Hall G.E., George A.A. (2006).

Measuring Implementation in scools: Innoviation Configu-

rations. Austin, Texas: SEDL.

STCW Convention Resolutions of the 1995 conference.

(1996). International Maritime Organization, London.

Avouris N., K. C. Issues Evaluation of Images collaborative

environment. Wesley Publications

Kokotos D.H. (2007). Virtual environments. Stamoulis Publi-

cations, Athens.