The PMS will be a member of the Psychometric group within the Center for Educational Measurement (CEM) at Excelsior College, and will work with another Measurement Statistician, a Data Analyst, the Director of Psychometric Services, and staff from the Test Administration, Test Development, Information Technology, and Editorial groups within the CEM. The person will help implement the objectives and procedures for the psychometric analysis and evaluation of the College's international program of examinations. These examinations are taken by students to gain credit for prior learning at colleges and universities in the U.S. and at international locations.
The Psychometrician / Measurement Statistician will assist in planning and coordinating the daily activities and responsibilities of the Psychometric group and will conduct various aspects of statistical and psychometric programming, analysis, and interpretation. This work will involve both classical and item response theory (IRT) methods of evaluating test items, building comparable test forms, and standard setting. The PMS will be able to prioritize various projects and ensure that all work is accurately completed within defined time frames. On a daily basis, this position will deal with secure, highly confidential information.
The PMS will be responsible for a wide range of statistical and psychometric activities related to test construction, validity, reliability, standard setting, equating, analysis, and scoring for a battery of over 50 examinations. Responsibilities include, but are not limited to the following:
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions.
- Conduct classical item analysis and IRT calibrations to support test development activities for multiple choice, extended response, mixed format (multiple choice & extended response), and examinations featuring alternate item types.
- Conduct standard setting and rater training for various exams.
- Collaborate with staff from the AU test development group to construct parallel test forms.
- Automate processes and keep automated processes current for ongoing psychometric activities.
- Assist in the publication of an annual technical manual documenting psychometric characteristics of all AU assessments offered.
- Support the development of psychometrically sound assessments for online courses.
- Conduct studies required to address issues of examination and item exposure, reports of testing irregularities, differential item functioning, speediness, equating, scaling, subscore reporting, and decision consistency and accuracy.
- Write statistical computer programs (using SPSS or other programming languages) for the purpose of cleaning data files and conducting statistical and psychometric analyses.
- Conduct testing on both internal and external systems used for exam administration and student scoring and grading.
- Create, modify, and maintain databases for item banking.
- Carry out research in measurement theory and practice to improve Excelsior College testing program and contribute to the measurement profession.
- Some travel may be required.
Education and/or Experience
- Demonstrate high level of knowledge and proficiency in statistical and psychometric theory, analysis, and interpretation.
- Display a high degree of accuracy with excellent attention to detail.
- Demonstrate excellent decision making and organization skills and productive time management.
- Provide quality assistance to staff.
- Demonstrate professional commitment to responsibly secure and maintain confidential information.
- Write, debug, and run statistical computer programs.
- Demonstrate proficiency in the utilization of database, spreadsheet, and word processing applications.
- Effectively work both independently and cooperatively in a team environment.
- Demonstrate excellent written, oral, and interpersonal communication skills.
- Work flexible hours as needed.
- The minimum requirements for the position of Psychometrician/Measurement Statistician are an MA or MS degree in measurement or related field from a regionally accredited institution with strong relevant experience and/or training in large scale standardized assessment.
- A PhD in measurement or related field is preferred.
- Coursework and/or practical training in measurement is required.
- Knowledge of computerized adaptive testing (CAT) is desirable.
- The ability to use technology to automate repetitive tasks and processes.
- Hands on experience programming in Visual Basic, C Sharp, Java, or some other computer language is highly desirable.
Founded as Regents College in 1971 and chartered as a private, nonprofit institution in 1998, the College was renamed Excelsior College in...