quantitative research

DOING QUANTITATIVE RESEARCH IN APPLIED LINGUISTICS

Yudi Juniardi
Staf Pengajar FKIP Untirta dan STKIP Setia Budhi

Abstrak
Tujuan penulisan ini adalah untuk menggambarkan bagaimana melakukan penelitian kuantitatif dalam pengajaran Bahasa. Pendekatan digunakan dalam dalam penulisan ini adalah studi literatur. Setidaknya ada beberapa hal yang dibahas dalam makalah ini: penelitian kuantitatif, design penelitian, validitas dan reliabilitas, serta format laporan dalam penelitian. Output penulisan ini, diharapkan dengan adanya pemahaman tentang desain penelitian serta validitas dan reliabilitas instrument yang baik, pegiat bahasa dapat menghasilkan penelitian yang baik dan memberikan kontribusi bagi pengembangan Bahasa, khususnya dalam pengajaran Bahasa Inggris

Kata Kunci: Research, Quantitative, Applied Linguistics

1. Introduction

Doing a research is a compulsory for students of university, since it is as one of the requirement of graduation. However we still find many students having difficulties in doing a research especially when they will conduct a quantitative research. Some difficulties are how to design a research, how to decide a research method, how to analyze the data, and how to prepare a research report.
To assists students in doing a research, sharing and discussion is needed. One can do a research if she or he understands the systematic way of doing a research. Furthermore he needs to know the way in formulating instruments of the research and interpreting collected data.
We should understand how to conduct a quantitative research. By knowing the sequence, the procedure, and the practice, it will make the students found of conducting researches. Students will acquire many advantages if they often do researches because doing a research gives a good implication for developing knowledge. To share practically my ideas about quantitative research in applied linguistics, on this paper, I shall only discuss a quantitative research in a glance, which covers research design, methods and research report.

2. Quantitative research
Quantitative research put emphasis on the use of quantitative methods (E.g. Descriptive statistics research, correlational research, etc.). The following are some characteristics of quantitative research:
- Quantitative research looks for facts and causes of situation from an over all view rather than from individual viewpoint.
- Quantitative research prefers controlled setting
- Quantitative research is objective
- Quantitative research is undertaken by a researcher outside the situation under study.
- Quantitative research tends to be hypothesis led and deductive
- Looks for reliable, replicable data
- Is concerned that results should be generalisable.
- Assumes a reality that is stable.

3. Research design, methods, and strategies
When we attempt to conduct a research, design decisions will depend on the purpose of the research, the nature of the problem, and the alternative appropriate for its investigation.
Once the purposes have been decided, the study should have explicit scope and direction, and attention can be focused on a delimited target area. The nature of the problem then plays the major role in determining what approaches are suitable.
According to Issac and Michael (1981) designs alternatives can be organized into nine functional categories based on these differing problem characteristics:
historical
descriptive
developmental
case or field
correlational
causal comparative
true experimental
quasi experimental, and
action

In this paper, I attempt to explain only correlational, causal comparative, true experimental, and quasi-experimental since these designs most common used in applied linguistic research.

Correlational research
Purpose of this research is to study the extent to which variations in one factor correspond with variation in one or more other factors based on correlation coefficients. Correlation studies permit us to determine the extent to which scores on one test are associated with scores on another test.
Examples:
A study investigating the relationship between grade point average as the criterion variable and a number of other variables interest or;
Relationship between reading speed and reading comprehension scores,
Relationship between Ss’ number of turn talk in the classroom and their final grades.

Characteristics of this research:
Appropriate where variables are very complex and or do not lend themselves to the experimental method and controlled manipulation.
Permits the measurement of several variables and their interrelationship simultaneously and in a realistic setting.
Gets at the degree of relationship rather than the all-or-nothing question posed by experimental design: “is an effect present or absent?”
The limitation of the correlational studies as follow:
a. it only identify what goes with what- it does not necessarily identify cause and effect relationship.
b. It is less rigorous than the experimental approach because it exercises less control over the independent variables.
c. It is prone to identify spurious relational patterns or elements, which have little or no reliability or validity.
d. The relational patterns are often arbitrary and ambiguous.
Steps in doing correlation:
a. Define the problem
b. Review the literature
c. Design the approach:
- Identify the relevant variables
- Select appropriate subjects
- Select or develop appropriate measuring instruments
- Select the correlational approach that fits the problem.
d. Collect the data
e. Analyze and interpret the results

Causal-comparative research
The Purpose of this research is to investigate possible cause and effect relationship by observing some existing consequence and searching back through the data for plausible causal factors. This is in contrast to the experimental method which collects its data under controlled condition in the present.
Examples:
1. To identify factors characterizing person having either high or low accident rates, using data in insurance company records, or
2. To look for patterns of behavior and achievement associated with age differences at the time of school entrance using descriptive data on behavior and achievement test scores in the cumulative pupil records of children currently in the sixth grade.

Principal characteristics of this research:
This research is ex post facto in nature, which means the data are collected after all events of interest have occurred. The investigator then takes one or more effects (dependent variables) and examines the data by going back through time, seeking out causes, relationship, and their meanings.
Steps:
- Define the problem
- Survey the literature.
- State the hypotheses
- List the assumption upon which the hypothesis and procedures will be based.
- Design the approach:
- Validate the data – gathering techniques
- Describe, analyze, and interpret the findings in clear, precise terms.

True experimental Research
The purpose of this research is to investigate possible cause and effect relationship by exposing one or more experimental groups to one or more treatment conditions and comparing the result to one or more control groups not receiving the treatment.

Characteristics of experimental designs:
Requires rigorous management of experimental variables and conditions either by direct control/manipulation or through randomization.
Typically uses a control group as a baseline against which to compare the groups receiving the experimental treatment.
Internal validity is the sine qua non of research design and the first objective of experimental methodology. It asks the question: did the experimental manipulation in this particular study really make a difference?
External validity is the second objective of experimental methodology. It asks the question: How representative are the findings can the results be generalized to similar circumstances and subjects?

Seven steps in experimental research:
survey the literature relating t the problem
identify and define the problem
formulate a problem hypothesis, deducing the consequences and defining basic terms and variables.
construct an experimental plan:
(1) identify all non experimental variables that might contaminate the experiment, and determine how to control them.
(2) Select a research design
(3) Select a sample of subject to represent a given population, assign subjects to groups, and assign experimental treatments to groups
(4) Select or construct and validate instruments to measure the out come of the experiment.
(5) Outline procedures for collecting the data and possibly conduct a pilot or “trial run” test to perfect the instrument or design.
(6) State the statistical or null hypothesis.

Conduct the experiments.
Reduce the raw data in a manner that will produce the best appraisal of the effect which is presumed to exist.
Apply an appropriate test of significance to determine the confidence one can place on the results of the study.

Quasi experimental Research

The purpose of this research is to approximate the conditions of the true experiment in a setting which does not allow the control and/or manipulation of all relevant variables. The researcher must clearly understand what compromise exist in the internal and external validity of his design and proceed within these limitations.
Characteristics of this research, Quasi experimental research typically involves applied setting where it is not possible to control all the relevant variables but only some of them. The researcher gets as close to the true experimental rigor as condition allow, carefully qualifying the importance exception and limitations. Step in doing quasi experimental the same with true experimental research, carefully recognizing each limitation to the internal and external validity of the design.
Quasi experimental designs are practical compromises between true experimentation and the nature of human language behaviour which we wish to investigate.
By using Quasi experimental design, we control as many variables as we can and also limit the kinds of interpretations we make about cause- effect relationship and hedge the power of our generalization statements. Time series design allows us to do this.
T1 T2 T3 X T4 T5 T6


For example, suppose that we have developed a set of programmed materials to teach English tenses for the class of STKIP students. We haven’t been able to find similar groups to serve as control and experimental groups. We have only STKIP students who learn English. What we can carry out as follow:

a. We start administering tests on articles at the end of each week for the first three weeks of the class. By week for, you can see the improvement that your students have made in article usage.
b. Then we give them the programmed materials to work through.
c. Following this we start giving posttests on articles at the end of each week following the treatment.
d. A comparison of the curves for the first three weeks with those after the treatment will give you an idea whether your innovative materials worked or not. Your result of the research will come up with any of the following possibilities:

1

2
31
2
3


The picture above explains us if we obtained a line similar to line 1 it would show that there was no effect from our special treatment. The students continued the same developmental pattern that we saw during the first three weeks of the study. If the result were like those in line 2, it would indicate that our materials had a negative effect. Since after the treatment their scores declined consistently. Line 3 is the line we would hope to get, since it shows that our treatment was effective
Another version of the time series design is the equivalent time-sample design. It works like this: the treatment is introduced and re introduced between every other pretest and posttest. Then after the second pretest, an alternative treatment is introduced and that is followed by posttest. This procedure is followed for two or three times and the results following the experimental treatment are compared with the alternative treatment scores.
T1XT2 ---T3 0 T4 …T5 X T6 ….T7 0 T8, etc.



Ex Post facto designs
When researchers control the threats to internal and external validity, they are trying to find a direct relationship between the independent and dependent variables. In other words, they select the population, sample, treatment, and variables in order to find a cause and effect relationship between the variable.
Ex post facto designs are often used when the researcher does not have control over the selection and manipulation of the independent variable. This is why researchers look at the type of relationship between the two variables rather than at a cause and effect relationship.
Suppose we wanted to know about the performances of two groups of students (say one group from Banten and another one is out of Banten) on an entrance exam given to students of STKIP SB. Any relationship between the scores of the groups would not be related to any instructional program we had given them before test. That is why the designs are called ex post facto. The researcher has no control over what has already happened to the Ss. The treatment has been given prior to the research project.
Correllational designs are the most commonly used subset of ex post facto design. In correlational design a group of Ss may give us data on two different take the TOEFL. Many universities also have their own entrance exam which they administer to students. We can then look at the relationship of Ss’ scores on one test to their scores on the other. Or for example students have two test TOEFL and GRE (Graduate Record Exam) then the score for each S on one test can be compared with the score on the other, allowing us to see whether those students whose score high on one test also score high on the other.




The schematic representation of this design would be:

T1 T2



Another ex post facto design is called criterion group design. In this design two groups of Ss are compared on one measure. For example measure the reading speed of Indonesian and Malaysian students, assuming we want to see how related or different they might be. The design would look like this:

G1 T1
G2 T2




4. Validity and reliability
A good research has valid and reliable measurement. Validity refers to the extent to which an instrument measures what it is supposed to measure. On the other hand, reliability is the extent to which a measuring device is consistent in measuring whatever it measures.

Content validity
Content validity refers to the degree to which a test sample the content area which is to be measured. Ghiselle (1964) pointed out this is the type of validity educators are most concerned with.
Content validity is essentially and necessity based on judgment. The test makers may ask a number of experts to examine the items systematically and indicate whether or not they represent sufficiently well the theoretical universe from which they were drawn. It means that all major aspect of the content area must be adequately covered by the test items, and they must be in the correct proportions. All the major objectives of instruction must be included as well as the subject matter.

Predictive validity
Predictive validity refers to the extent to which a test can predict the future performance of individuals. A test has predictive validity when it can effectively indicate how a person will do on a later task.
Predictive validity is established empirically by correlating the test scores with subsequent measures on a criterion. The correlation coefficient become validity coefficient.

Construct validity
Construct validity refers to the extent to which a test reflect construct presumed to underlie the test performance and also the extent to which it is based on theories regarding these construct.
Construct validation utilizes both a logical and an empirical approach. In the construct validation of a test, one usually makes a number of inferences from theory and experiences regarding the trait to be tested, then attempts to find experimentally how well the test corresponds with these inferences.

Reliability
There are four commonly used methods for estimating the reliability of a test. Three of them: test – retest, equivalent forms, and split –half are based on correlational procedures; the other Kuder Richardson formula 21, is based on the proportion of subjects who get the items right or wrong.
Test – retest reliability is an obvious way to estimate the reliability of a test by administering it to the same group on two occasion and correlated the paired scores. The coefficient of correlation (Pearson r) between the scores of the subjects, obtained on the two administration of the test will give us an indication of its reliability.
The equivalent forms technique of estimating, Which is also referred to as the alternate or parallel form technique, is used when it is probable that subjects will recall a their responses to the test items. It correlates the results of equivalent forms of the test administered to the same individuals. If the two forms are administered at essentially the same time, the resulting reliability coefficient is called the coefficient of equivalence.
Split half reliability. It is possible to get a measure of reliability from a single administration of one form of a test by using split half procedures. The test is administered to a group of subjects, and later the items are divided into two comparable halves and a coefficient of correlation calculated between these two scores
Kuder Richardson Reliability. This method is by far the least time consuming of all the reliability estimation procedures. It involves only one administration of a test and employs only easily available information. The formula as follow:

r11 = Ksx2 – X (K-X)
sx2 (K-1)
r11= the reliability of the whole test
K = the number of item in the test
sx2= the variance of the scores
X = the mean of the scores

In interpreting of reliability coefficient, most test makers and researchers are satisfied if they obtain reliability coefficient of 0.90 and above but dissatisfied if they got coefficients below 0.70.

5. Report
The last step in doing a research is writing a research report. Dugdale (1967) gives the following format in writing a thesis report.

I. Preliminary pages
A. Title page
B. Acceptance page
C. Acknowledgement page
D. Table of content
E. List of tables
F. List of figures

II. Text
A. Introduction
1. Statement of the problem and justification for the study
2. Statement of the objectives
3. Definition of the term used (if necessary)
4. Review of the research relating to the topic
5. A brief account of the sources of the data and the methods and procedures used
B. Body of the report
1. detailed account of the sources of data and the methods and procedures used
2. presentation and analysis of the data
3. interpretation of the findings
A. Summary and conclusion
1. summary of the study
2. general conclusions drawn from the findings
3. recommendations
II. Supplementary pages
A. Bibliography
B. Appendix
C. Index
D. Vita (if required)

6. Conclusion
A quantitative research is different from a qualitative one. In doing this research we propose the research design and the method based on the purpose of the study and the problem of the research.
To make a good result in a quantitative research, the instrument should be valid and reliable. It means that the instrument measures what should be measured and has stability. It has appropriateness of the statistical treatment and data processing procedures.
Finally, the conclusion of the research should precise and accurate statement of the problem, the methodology followed, and the findings without the introduction of new or irrelevant information.