Frequently Asked Questions: FAMU’s Assessment Process
What is assessment?
Many good definitions of assessment abound in the literature. Florida Agricultural and Mechanical University (FAMU) define assessment as “a systematic process of documenting and analyzing the effectiveness of the teaching and learning processes to ensure that the expectations and standards are met in fulfilling the mission of FAMU. The process includes monitoring and enhancing the administrative and educational support structure that leads to the continuous quality improvement of FAMU's faculty, staff and student programs and services."
Why is assessment done?
Improving student learning -- for both current and future students -- is the best reason to do assessment. However, assessment is often done to satisfy some mandate from within an institution or from some governing or accrediting unit outside the institution.
Assessment at FAMU serves three major purposes. The first purpose is program and service improvement aimed at making programs/units (academic majors, general education, certification programs, functional units etc.) more effective. The second purpose is for accountability aimed at demonstrating institutional responsiveness to external constituencies by ensuring that students demonstrate basic academic competencies and skills mandated by state and federal legislators. The third purpose is for institutional effectiveness aimed at meeting requirements of accrediting agencies.
What is the purpose of assessment plans?
Since one of the purposes of assessment at FAMU is accountability to state and federally mandated policies, the Office of University Assessment requires that assessments be documented in order to effectively meet the needs of both external and internal constituents. An effective and efficient system of developing and implementing assessment plans and reporting the results for each academic program and administrative unit is critical to the success of FAMU. Assessment plans enable the Office of University Assessment to clarify student learning expectations as agreed to by all parties concerned, as well as monitor and document on:
Improvements in student learning outcomes, both inside and outside the classroom, that is, the outcomes established for our academic and administrative units,
Performance measurements as required by the institutional budget planning process,
Progress in the implementation of the University’s Strategic Plan, and
Compliance with the requirements of the Southern Association of Colleges and Schools Commission on Colleges(SACSCOC) and other specialized accrediting agencies.
Is there a specific approach and/or guidelines for completing assessment planning forms?
Yes. Assessment at FAMU is based upon the “FAMOUS” assessment approach, which involves six sequential and precise steps. Each letter of the acronym “FAMOUS” represents an important step that is connected to the next step in a chain that ultimately comes together to contribute to the goal of successfully developing and implementing an effective assessment plan. FAMU’s assessment planning forms are developed to facilitate the documentation of the six steps.
The FAMOUS assessment planning and implementation model includes the following steps:
Step 1: Formulating statements of outcomes/objectives aligned to the institutional mission/goals;
Step 2: Ascertaining criteria for success;
Step 3: Measuring student/service performance using direct and indirect methods;
Step 4: Observing and analyzing results for congruence between expected and actual outcomes;
Step 5: Using the results to effect improvement of instructional programs and administrative and educational support services; and
Step 6: Strengthening programs and services by continuously evaluating, planning, allocating resources and implementing new approaches to ensure congruence between expected and actual outcomes
What is "criteria for success?"
Criteria for success require stating "how well" something is done (e.g. how well students are performsing , how well a function is carried out.) Criteria for success serve as the benchmarks for judging the results of the assessment activity. Criteria should neither be set unrealistically high nor so modestly low that anyone can meet them. Withou specifying the criteria for success, it would be difficult to make use of the data to omprove the program or service.
When should assessment plans be submitted?
The Assessment Cycle runs from September till October of the following year. Assessment plans are normally due in the Office of University Assessment by September. Each plan has to be submitted to the college/school or divisional assessment committees for review and approval before submission to the Office of University Assessment.
How many outcomes or functional objectives need to be identified?
Each academic program and administrative unit should identify at least five outcomes. However, academic programs that undergo specialized accreditation (e.g. engineering, architecture, nursing, pharmacy etc), and whose accrediting agencies may require them to document on more than five outcomes are encouraged to do so.
What is the difference between a learning objective and a learning outcome?
When used in the context of assessment, learning goal and learning outcome are usually synonymous, but more general than learning objective. A learning goal may be very general. For example, a learning goal of the mathematics major might be that graduates will be able to apply mathematics to solve real world problems. In order to measure progress toward that goal and to design curricular strategies for achieving the goal, objectives need to be developed. One such objective might be to use definite integrals to model real world problems. That objective might be reached early in the mathematics major, and would be a step along the way to reaching the general goal.
Goal, objective, and outcome are used in different ways in different contexts and by different people. Consequently, making precise distinctions is difficult.
How does one establish learning outcomes?
The faculty who has stewardship of an academic program should articulate the goals and outcomes of the program. Since at FAMU, assessment plans are normally required at the program level, it is expected that program chairs in conjunction with their faculty, prepare outcomes for their respective programs.
What data should be gathered and what kinds of records should be kept?
The results of the application of any assessment tool should be recorded. Examples include test scores, faculty judgments of student portfolios, and student self-assessments. All data should be carefully identified as to purpose, date, and populations involved. Interpretations of data and any actions taken as a result of those interpretations should be recorded. Complete and thorough records that can be used over time are essential.
What are assessment methods and measures?
Also known as means of assessment, assessment methods/measures are strategies with which information will be collected in order to validate each intended outcome or objective. These may include such techniques as comprehensive exams; student portfolios, senior projects, theses, or dissertations evaluated by a committee; exit interviews, alumni surveys, graduating student surveys, employer surveys; results of licensing exams; evaluations by practitioners; or student satisfaction levels. Academic programs should consider including one instance of value-added assessment and one instance of external review in each assessment plan. Three to five methods of measurement (including alternate strategies) are suggested for each outcome or objective.
What do you do with the data from assessment?
You interpret assessment data in the context of learning outcomes and objectives and make curricular decisions about changes that the interpretations imply.
What are common uses of assessment data?
The most important use is for improving academic programs and enhancing student learning. Other uses are in evaluation of students, programs, and institutions, often for accountability or accreditation (See also question 3).
What should students know about assessment programs?
A major goal of assessment at FAMU is to support the university’s mission of “excellence with caring” and accountability to internal and external stakeholders by emphasizing the importance of systematically planning, implementing, analyzing, documenting and reflecting on the results to improve student outcomes and support services effectiveness.
It is vital, therefore, that students and faculty be involved in and informed about the assessment process, from the planning stages throughout implementation. This implies that students should know the purposes, the processes, and how the results will be used.
What is a scoring rubric?
A rubric is a printed set of scoring guidelines (criteria) for evaluating work (a performance or a product) and giving feedback. A scoring rubric refers to the established criteria, including rules, principles and illustrations, used in scoring responses to individual and clusters of items. (AERA/APA/NCME 1999 p. 182) For more information on rubrics, please visit www.rubrics.com.
What is a portfolio?
A portfolio is a collection of documents representative of an individual’s best work over a period of time. This may include a variety of other kinds of process information (e.g., drafts of student work, student's self assessment of their work.) In the teaching and learning context, a portfolio is a purposeful, meaningful collection of student work that tells a story about the student's developmental growth, achievements and progress over time. The principle behind portfolios is that the onus is on the owner/participant to collect, select, and reflect upon his/her work.
ACADEMIC LEARNING COMPACTS
What are Academic Learning Compacts?
Academic Learning Compacts (ALC) are written agreements that satisfy guidelines issued in 2004 by the State University System/Division of Colleges and Universities. The Academic Learning Compacts require each State University in Florida to identify, by academic program, what students will have learned by the end of their baccalaureate degree programs, and how that learning will be measured above and beyond course grades. ALCs answer three basic questions: What will students learn by the end of their academic programs? Have they learned what they have been taught by their professors? How do we measure these quantities?
Each Academic Learning Compact will include statements of the core student learning outcomes in the areas of content/discipline knowledge and skills, communication skills, and critical thinking skills that program graduates will have adequately demonstrated prior to being awarded the baccalaureate degree.
What is the purpose of ALCs?
Academic Learning Compacts are meant to account for student achievement in baccalaureate degree programs in the State University System.
When do ALCs begin?
According to the proposed time-line for the Academic Learning Outcomes here at FAMU, the implementation phase begun in the Fall of 2005.
How will ALC be incorporated in courses?
Each Academic Learning Compact is required to identify the corresponding assessment processes used to measure student achievement on each of the core student learning outcomes for the program. These assessment processes are meant to specify:
the required courses or other academic equivalents through which all students pursuing the baccalaureate degree are assessed on each outcome, Reviews of the Academic Learning Compacts and related results will be done by teams of faculty at the program level and the college/school assessment committees in conjunction with the Office of University Assessment.
the assessments used in those courses or academic equivalents that correspond to each outcome, and
the standards used during the assessments to determine if student work matches the expectations articulated for each outcome.
Will the ALCs be available to students?
Yes. The State University System mandated that the ALCs be made available for the enrolled and prospective students by fall 2005. Also, all academic programs are expected to distribute their Academic Learning Compacts to students through the University website and course syllabi. The ALC for FAMU’s baccalaureate programs are available at the Office of University Assessment webpage.
Who will be responsible for evaluation of the ALCs?
Each department or program through the college/school assessment committees is expected to conduct evaluation and review processes sufficient to corroborate that the assessments in the Academic Learning Compact measure student achievement on each of the expected core learning outcomes. All evaluation and review processes will be consistent with the Guidelines for Academic Learning Compacts. The ALCs evaluation results will be made an integral part of the Annual Institutional Assessment Report prepared by the Office of University Assessment.