Test Development Manager

Develop multi-media English language assessments in reading, listening, speaking, writing; administer assessments to international students learning English; author/review Test Content; develop specifications for test questions; write/revise test questions based on specifications; work with graphic designers to supplement test questions with graphics/animations; supervise external vendors; contractors to record audio for test questions; conceptualize avatars for tests; write scripts for avatars; work with graphic designers/external vendors to create avatars; review test questions on Test Platform to ensure rendered accurately; perform hypotheses testing (t-tests, Chi-squared tests, One-way/Factorial ANOVA), correlation; conduct simple/multiple linear regression; apply educational research applications; use logistic/multinomial logistic regression, advanced modeling methods (Logistic Regression, Multilevel/HLM (Hierarchical Linear Modeling), SEM (Structural Equation Modeling); design/implement data-based case studies, correlational studies, quasi-experimental studies with Lexile Framework for Reading and Quantile Framework for Mathematics; develop documents (research protocols, literature reviews, white papers) that communicate information about studies, products, assessments according to Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014); design data collection instruments (survey questionnaires, interview protocols, other quantitative or qualitative collection instruments); execute appropriate statistical analyses to test hypotheses/yield insights from internal/external data sources according to various research designs; communicate results/insights to diverse technical/non-technical audiences (internal stakeholders, MetaMetrics partners, professional conferences or publications); collaborate with key stakeholders/subject matter experts on research design/feedback into MetaMetrics products/services; participate on planning of test question trials (thousands of students); develop/revise scoring criteria for speaking/writing performances; create rater training materials; train raters, score test responses; participate in test question review/screening; compile standard-setting materials to link test scores to established English language standards; assist in facilitating standard-setting workshops; write/update item specifications; write/review Trial test administrator manuals; write/review Student and Educator Information Flyers for Trial; document rater training/scoring processes; write relevant sections in Trial analysis documentation; write relevant sections in overall validity framework documentation; write relevant sections in Standard Setting documentation; author research articles; participate in psychometrics/AI Scoring; assist with analysis of test score data; check for test questions (ones that do not fit with the others); assist with score scaling/score aggregation for domain scores, overall scores; provide input into development of AI scoring models based on rating criteria/ expert rater judgments; conduct validation research; plan, conduct, analyze, write reports for research studies to demonstrate validity/efficacy of developed English language assessment products; perform internal/external communications/presentations; participate in meetings (multiple teams, departments) within MetaMetrics; present test questions to team members/solicit feedback; provide input on data analysis/modeling from test development’s perspective; liaise with partners/clients to communicate projects’ needs, status, outcomes; represent MetaMetrics in customer meetings (professional/academic contexts); and present MetaMetrics’ work at academic conferences. This is a telecommuting position with remote benefit from various unanticipated locations throughout the U.S. Occasional travel to employer’s headquarters.

Job Requirements:

Master’s degree or foreign equivalent in Education, Curriculum and Instruction, or Teaching of English as a Second Language plus coursework, internships, or experience to include performing hypotheses testing (t-tests, Chi-squared tests, One-way/Factorial ANOVA), correlation; conducting simple/multiple linear regression; and applying educational research applications using logistic/multinomial logistic regression, advanced modeling methods (Logistic Regression, Multilevel/HLM, SEM). Experience and skills may be gained through academic coursework and concurrently while pursuing academic studies. This is a telecommuting position with remote benefit from various unanticipated locations throughout the U.S. Occasional travel to employer’s headquarters.

Mail resumes to 800 Taylor Street, Suite 102, Durham, NC 27701.