Reliability, Practicality, and Validity level of the entrance exam

Abstract

The reason for this review is to expose the consistency, practicality, and validity degree of the entrance exam for the teacher training program at National Institute of Education in Cambodia also to measure the knowledge of the learner and test when they did the examination this year 2010. This analysis utilized documentary interviews from both teachers and both students at NIE. Due to these findings, the existing study contributes to the understanding of the practicality, trustworthiness, and validity level of test items and providing ideas towards improvement of the checks design for the entrance exam at NIE in the foreseeable future.

Introduction

National Institute of Education is well-known school in Cambodian along with the Country wide Institute of Education is incurred with responsibility for working out of Cambodia's university teachers and university administrators. It comprises two departments - the training Division, which trains lower and top secondary school instructors in the sciences and cultural sciences, and the Planning and Management Office, which trains university principals, inspectors, supervisors and office administrators to plan and measure the quality of education throughout the country. And it has an enrollment of about 900 students annually (MoEYS). Which means this newspaper will consider dependability, practicality, and validity level in a representatives English language exams that was an integral part of the entrance evaluation for the National Institute of Education in 2010 2010 and to be able to understand the type of English tests and the examination put forward the context where the entrance evaluation was taken by the students or trainees. Moreover, all students or trainees must take the entrance examination whether they will have a chance to examine there and if indeed they pass the assessment, they will have a chance to study there. If they fail the exam, they must wait until next time and take the exam again.

Literature review

Reliability

According to Clark (1975) said that trustworthiness is in fact a prerequisite to validity in performance evaluation in the sense that the test must provide consistent, replicable information about individuals' vocabulary performance. And Jones (1979) said that there surely is no test can perform its intended purpose if the test outcomes are unreliable. Reliability in a performance test will depend on two significant variables: (1) the simulation of the test duties, and the consistency of the scores, and four types of dependability have attracted serious attention: (1) inter-examiner stability, (2) intra-examine reliability, (3) inter-rater trustworthiness, and (4) intra-rater stability.

Jones also talked about that because the supervision of performance checks may vary in several contexts at differing times, it may cause inconsistent rankings for the same examinee on different performance assessments. Therefore, attention should be devoted to inter-examiner and intra-examiner consistency, which concern persistence in eliciting test performance from the test takers (Jones, 1979).

In addition, performance tests require individuals or mechanical raters' judgments. The stability issue is generally more difficult when tests involve human being raters because real human judgments involve subjective interpretation for the rater and may thus lead to disagreement (McNamara, 1996). Inter-rater and intra-rater trustworthiness are the key considerations when looking into the problem of rater disagreement. Inter-rater consistency has to do with the regularity between several raters who evaluate the same test performance (Jones, 1979). For inter-rater trustworthiness, it is of key interest to examine if the observations over raters are steady or not, which might be estimated through the application of generalizability (Crocker & Algina, 1986). Intra-rater trustworthiness concerns the steadiness of 1 rater for the same test performance at different times (Jones, 1979). Both inter- and intra-rater trustworthiness deserve close attention in that test scores are likely to vary from rater to rater or even from the same rater (Clark, 1979).

Practicality

It identifies the economy of your energy, money and effort in testing. Quite simply, a test should be easy to create, easy to manage, easy to tag, and easy to interpret the results (Bachman and Palmer, 1996). Moreover, according to Dark brown (2004) said that the test that is sensible it needs to be within the method of financial restrictions, appropriate time constraints, easy to administrator, credit score, and interpret.

Validity

The term validity refers to if a test actions what it intends to measure. On the test with high validity the items will be tightly linked to the test's intended focus. For many certification and licensure checks which means that the items will be highly related to a specific job or job. In case a test has poor validity then it generally does not gauge the job-related content and competencies it ought to (Bachman and Palmer, 1996).

Content Validity

Content validity refers to the connections between your test items and the subject-related tasks. The test should assess only this content related to the field of review in a way sufficiently representative, relevant, and comprehensible (Bachman and Palmer, 1996).

Hughes (1989) said a test is to get content validity if its content constitutes a representative test of the terminology skills, structures, etc. with which it is meant to be concerned. Hughes (1989) also said that to be able to judge whether or not a test has content validity, we desire a specification of the skills or structures. That it's designed to cover a principled selection of elements for addition in the test. The higher a test's content validity, the much more likely it is usually to be an accurate way of measuring what it is supposed to assess.

Construct Validity

According to Bachman and Palmer (1996) said that It suggests using the construct correctly (principles, ideas, notions). Build validity seeks arrangement between a theoretical theory and a particular measuring device or treatment. And Derewianka (1999) said that Build validity is concerned with the scope to that your assessment task reflects the theoretical assumptions underpinning its construction. In addition, Brow (2004) also said that construct validity is any theory, hypothesis, or model that tries to explain recognized phenomena inside our universe of understanding.

Research questions:

What is the consistency level of the entry exam for teacher training curriculum at NIE?

What is the practicality level of the access exam for tutor training curriculum at NIE?

What is the validity degree of the access exam for teacher training curriculum at NIE?

Method

Participant

The two instructors who British at NIE and both students who study at NIE were preferred to participate in one-on-one interviews. These people were invited to take part in this study because they're more encounters in designing the lab tests and rating the lab tests at NIE and other two students are the trainee there plus they experienced experience in doing access exam there. More importantly, they really would like to share their demonstration experiences with the interviewer. Furthermore, as a result, enough information must be obtained through in-depth interviews with the four members about the practicality, trustworthiness, and validity degree of the entry exam for the educator training curriculum at NIE.

Procedure

The two professors and two students were asked in one-on-one interview at different time. And I've made the appointment at least two times before the operating the interview was occurred because I'd like to give more time to allow them to prepare for the interview process. While I am interviewing, the process of noting and recording will be exposed and the key objective of the study will find out to the four participants. After saving and noting the participant conversation will be transcribed in to the writing script because it is simpler for writing the effect and conversation which need to be done at the end of the research. The result from the participants and the findings from books review will be compared to find the similarities and dissimilarities because the goal of this research is just to further understand the practicality, trustworthiness, and validity of the access exam for tutor training program at Country wide Institute of Education (NIE).

Instrument

The equipment were decided on to collect the data needed for the research. Those kinds of tool gave information about specific factors to evaluate in the study such as personal preferences, practicality, stability, validity degree of the access exam at for the tutor training at NIE. At the end of the treatment, the information accumulated by those equipment was the necessary to find the key purpose of the research.

Scope and Limitation

It is not possible to increase more result to stand for the practicality, dependability, and validity level of this research because the number of participants participating in this research is small and the data from interviewees is very limited. Therefore, this research effect aims to confirm the prior research cause the books review

Results

The following is the result of an interview with the four people who act as the participant or interviewee in this research. The interview's consequence is shown the next paragraph

The two participant educators seem to really have the same the same ideas about the exams for doing the entry exam at NIE. However, they still some things that will vary from one another. Moreover, both students seem to have the same ideas when they answer the questions of the interview. However, they have got a little bit different ideas in answering the questions. I must say i appreciate for his or her ideas because they are good and useful for this research.

Reliability degree of the entrance exam for instructor training curriculum at NIE

The two instructors said that almost all of the test item there is merely one answer such as section A, B, C, but except section D because in the section D is the writing skill and the students could have a different answer using their own views. Additionally, the first instructor said that he didn't think he was properly trained because the content just like copied and pasted, the launch had not been clear and the format and design was messy. The second professor said that the test artist was well-educated because he previously gotten a expert degree from Japan, major in educational management. However, he thought that the test designer had not been well-trained in making the test. Both of them seem to have the same ideas about the test designer.

In addition, when I asked them if there have been any subjective test items in the test, both of these replied "Yes" and there is only 1 "section D writing skill" and also said that writing skill the learner would have a different answer from the own opinion to write the answer. When I asked them about their criterion for rating the students' writing, they seem to be to have a little bit different idea to make that subjective item. The first teacher said that for marking the writing section first he viewed the format such as introduction, body, and bottom line, grammar, punctuation, coherence whether the ideas were well-linked, and yet another thing was about acceptability. And the second tutor said that he previously some criterion differed from other lectures here and those criterion were company, content and ideas, sentence structure, sentence constructions, and the previous one is crucial thinking. However, if the two criterions are compared to each other, there are a few points which are the same such as: format and group, grammar, and there are a few points which are also different such as: critical thinking, acceptability, and syntax.

Moreover, both of them said that there was only one rater for marking one test, but there were two examiners in the class room while students were doing the testing because they didn't want students cheat through the exam.

Practicality degree of the entrance exam for professor training curriculum at NIE

Both of the teachers said that the test was administrated in the class at NIE and everything the trainees who does the tests. In addition, they said that they didn't know exactly about the expense of exam paper, but the y said that the test may be cheap there have been three exam documents like the answer sheet. In addition they said that the financial for replicating the test was supported by the NIE and the Ministry of Education and everything the examinees were allowed only one hour to do the assessments.

Validity level of the entrance exam for educator training curriculum at NIE

The first instructor said that from his own view it was beyond a few of the students' capability. However, he said that the test could measure the actual students' capacity since it was the access exam and the school had to find the best one. He also gave example that if the institution needed sixty and the institution had to choose the top sixty students. The next educator said that the students may have faced several problems with doing this one because almost all of the test items had been copied from the TOEFL booklet. However, He still thought that there have been some things (grammar or vocabulary) students had studied when they had been the students at the university or college, so he also said that they could prosper.

Moreover, the first scholar said that some sentence structure and vocabulary test items he had been seen such as 2, 7, 8but beside those he had never seen before, and he also said that test could evaluate his actual capacity because he previously done all details on the exam paper and also handed down the exam there. He also talked about that this test was ideal for students had done the bachelor degree. The second college student said that there have been some factors which she acquired seen when she have been studying British such such as sentence structure and vocabulary, but she also that it was difficulty with doing reading skill because there have been some key works she couldn't understand. However, she also said that she experienced done all test items. Then she said that the test could evaluate her actual capacity because she handed the exam and analyzed there.

The improvement of the test

The firs professor said that in his point of view, selecting the test custom made should be properly preferred, especially, makes certain that the test artist should be properly been trained in designing the checks. The second instructor said that the test should be clear enough to see especially the layout, one more thing was that the test artist should be add additional time to the test because the time was a bit short and that meant the test should be 1 hour and 30 mins because I hour is so brief including providing the exam paper to students and also students signed on the test and attendant sheet.

Moreover, the first learner said that enough time was a little bit short for all the students and the test artist should add more time to the test and he also advised that the test should change from one hour to 1 hour-thirty minutes. And the next university student said that the test items are long if she set alongside the time arrangement in the test. She thought it was easier to cut down one reading in the test or test creator should add additional time to 90 minutes because there have been only 60 minutes.

Discussion

Overall, the finding of trustworthiness level of the entry exam for tutor training curriculum at NIE demonstrates the majority of the test items are easy to indicate to the students because they are the objective testing and it is multiple choice questions, but except writing section since it is the subjective tests which is hard to guage or evaluate the students' writing and there is only one rater for marking the test items, but there are examiners in the classroom while students are doing the exam. However, the test raters are well-trained and also have more experience in marking writing; especially they have a clear criterion for marking the writing section, so I feel that test is more reliable. This finding is also much like Jones, 1979; McNamara, 1996; Clark, 1979.

Another finding is similar to Bachman and Palmer, 1996; Dark brown, 2004 because the effect demonstrates the test is cheap, especially cost for copying the exam paper is backed by NIE and the ministry of education. As well as the test is not hard to tag, the test was administrated in the school room at NIE and they are comfortable class room all, but the time-allowed is a little bit short. However, I really do think that the test is still practicality because both participants surface finish the test promptly and it is the access exam, so the school must select the top students.

The last consequence demonstrates it is hard to judge if it is just a content validity since it is the entrance exam, and it focuses on the general English, so I think that the majority of the test items are develop validity because the students do it with regards to the instructions in the exams. And almost all of the test items are multiple choices and the pupil would familiar with these ones because they experienced in the university they would know the idea to do it. This means that the majority of test items similar to the TOEFL questions. This finding is comparable to Bachman and Palmer, 1996; Brown, 2004 because they have mentioned that construct validity implies using theory, ideas, notions, theory, hypothesis, and model.

However, I feel that some test items are the content validity because when I did so the interview with the students they said there were some items in the sentence structure and vocabulary test item they have seen and done them before such as question 2, 7, 8, 12. because these were at the university they took the TOEFL course. That is why they have seen some test items before which is comparable to Bachman and Palmer, 1996; Hughes, 1989 because these authors said that content validity identifies the connections between your test items and the subject-related activity and it is representative test of the vocabulary skills, buildings.

In conclusion the result above show the practicality, dependability, and validity level of the entrance assessment at NIE in 2010 2010. However, there's also a little bit suggestions about the test items from students and lecturers there because they would like the test more functional, reliable, and valid able both present and the near future.

Also We Can Offer!

Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)