The purpose of peer review is to insure that:

(a) Those activities supported by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) are of the highest scientific, administrative, and technical quality; and

(b) Activity results may be widely applied to appropriate target populations and rehabilitation problems.

(a) The Director refers each application for an award governed by these regulations in this part to a peer review panel established by the Director.

(b) Peer review panels review applications on the basis of the applicable selection criteria in § 1330.23.

(a) The Director selects as members of a peer review panel scientists and other experts in disability, independent living, rehabilitation or related fields who are qualified, on the basis of training, knowledge, or experience, to give expert advice on the merit of the applications under review.

(b) The scientific peer review process shall be conducted by individuals who are not Department of Health and Human Services employees.

(c) In selecting members to serve on a peer review panel, the Director may take into account the following factors:

(1) The level of formal scientific or technical education completed by potential panel members.

(2) The extent to which potential panel members have engaged in scientific, technical, or administrative activities appropriate to the category of applications that the panel will consider; the roles of potential panel members in those activities; and the quality of those activities.

(3) The recognition received by potential panel members as reflected by awards and other honors from scientific and professional agencies and organizations outside the Department.

(4) Whether the panel includes knowledgeable individuals with disabilities, or parents, family members, guardians, advocates, or authorized representatives of individuals with disabilities.

(5) Whether the panel includes individuals from diverse populations.

(a) The Director selects one or more of the selection criteria to evaluate an application:

(1) The Director establishes selection criteria based on statutory provisions that apply to the Program which may include, but are not limited to:

(i) Specific statutory selection criteria;

(ii) Allowable activities;

(iii) Application content requirements; or

(iv) Other pre-award and post-award conditions; or

(2) The Director may use a combination of selection criteria established under paragraph (a)(1) of this section and selection criteria from § 1330.24 to evaluate a competition.

(3) For Field-Initiated Projects, the Director does not consider § 1330.24(b) (Responsiveness to the Absolute or Competitive Priority) in evaluating an application.

(b) In considering selection criteria in § 1330.24, the Director selects one or more of the factors listed in the criteria, but always considers the factors in § 1330.24(n) regarding people with disabilities, and members of groups that have traditionally been underrepresented based on race, ethnicity, national origin, sex (including sexual orientation and gender identity), or age.

(c) The maximum possible score for an application is 100 points.

(d) In the application package or a notice published in the Federal Register, the Director informs applicants of:

(1) The selection criteria chosen and the maximum possible score for each of the selection criteria; and

(2) The factors selected for considering the selection criteria and if points are assigned to each factor, the maximum possible score for each factor under each criterion. If no points are assigned to each factor, the Director evaluates each factor equally.

(e) For all instances in which the Director chooses to allow field-initiated research and development, the selection criteria in § 1330.25 will apply, including the requirement that the applicant must achieve a score of 85 percent or more of maximum possible points.

[81 FR 29159, May 11, 2016, as amended at 87 FR 50003, Aug. 15, 2022]

In addition to criteria established under § 1330.23(a)(1), the Director may select one or more of the following criteria in evaluating an application:

(a) Importance of the problem. In determining the importance of the problem, the Director considers one or more of the following factors:

(1) The extent to which the applicant clearly describes the need and target population.

(2) The extent to which the proposed activities further the purposes of the Rehabilitation Act.

(3) The extent to which the proposed activities address a significant need of individuals with disabilities.

(4) The extent to which the proposed activities address a significant need of rehabilitation service providers.

(5) The extent to which the proposed activities address a significant need of those who provide services to individuals with disabilities.

(6) The extent to which the applicant proposes to provide training in a rehabilitation discipline or area of study in which there is a shortage of qualified researchers, or to a trainee population in which there is a need for more qualified researchers.

(7) The extent to which the proposed project will have beneficial impact on the target population.

(b) Responsiveness to an absolute or competitive priority. In determining the application's responsiveness to the application package or the absolute or competitive priority published in the Federal Register, the Director considers one or more of the following factors:

(1) The extent to which the applicant addresses all requirements of the absolute or competitive priority.

(2) The extent to which the applicant's proposed activities are likely to achieve the purposes of the absolute or competitive priority.

(c) Design of research activities. In determining the extent to which the design is likely to be effective in accomplishing the objectives of the project, the Director considers one or more of the following factors:

(1) The extent to which the research activities constitute a coherent, sustained approach to research in the field, including a substantial addition to the state-of-the-art.

(2) The extent to which the methodology of each proposed research activity is meritorious, including consideration of the extent to which:

(i) The proposed design includes a comprehensive and informed review of the current literature, demonstrating knowledge of the state-of-the-art;

(ii) Each research hypothesis or research question, as appropriate, is theoretically sound and based on current knowledge;

(iii) Each sample is drawn from an appropriate, specified population and is of sufficient size to address the proposed hypotheses or research questions, as appropriate, and to support the proposed data analysis methods;

(iv) The source or sources of the data and the data collection methods are appropriate to address the proposed hypotheses or research questions and to support the proposed data analysis methods;

(v) The data analysis methods are appropriate;

(vi) Implementation of the proposed research design is feasible, given the current state of the science and the time and resources available;

(vii) Input of individuals with disabilities and other key stakeholders is used to shape the proposed research activities; and

(viii) The applicant identifies and justifies the stage of research being proposed and the research methods associated with the stage.

(3) The extent to which anticipated research results are likely to satisfy the original hypotheses or answer the original research questions, as appropriate, and could be used for planning additional research, including generation of new hypotheses or research questions, where applicable.

(4) The extent to which the stage of research is identified and justified in the description of the research project(s) being proposed.

(5) The extent to which research activities use appropriate engineering knowledge and techniques to collect, analyze, or synthesize research data.

(d) Design of development activities. In determining the extent to which the project design is likely to be effective in accomplishing project objectives, the Director considers one or more of the following factors:

(1) The extent to which the proposed project identifies a significant need and a well-defined target population for the new or improved product;

(2) The extent to which the proposed project methodology is meritorious, including consideration of the extent to which:

(i) The proposed project shows awareness of the state-of-the-art for current, related products;

(ii) The proposed project employs appropriate concepts, components, or systems to develop the new or improved product;

(iii) The proposed project employs appropriate samples in tests, trials, and other development activities;

(iv) The proposed project conducts development activities in appropriate environment(s);

(v) Input from individuals with disabilities and other key stakeholders is obtained to establish and guide proposed development activities; and

(vi) The applicant identifies and justifies the stage(s) of development for the proposed project; and activities associated with each stage.

(3) The new product will be developed and tested in an appropriate environment.

(4) The extent to which development activities apply appropriate engineering knowledge and techniques to achieve development objectives.

(e) Design of demonstration activities. In determining the extent to which the design of demonstration activities is likely to be effective in accomplishing the objectives of the project, the Director considers one or more of the following factors:

(1) The extent to which the proposed demonstration activities build on previous research, testing, or practices.

(2) The extent to which the proposed demonstration activities include the use of proper methodological tools and theoretically sound procedures to determine the effectiveness of the strategy or approach.

(3) The extent to which the proposed demonstration activities include innovative and effective strategies or approaches.

(4) The extent to which the proposed demonstration activities are likely to contribute to current knowledge and practice and be a substantial addition to the state-of-the-art.

(5) The extent to which the proposed demonstration activities can be applied and replicated in other settings.

(f) Design of training activities. In determining the extent to which the design is likely to be effective in accomplishing the objectives of the project, the Director considers one or more of the following factors:

(1) The extent to which the proposed training materials are likely to be effective, including consideration of their quality, clarity, and variety.

(2) The extent to which the proposed training methods are of sufficient quality, intensity, and duration.

(3) The extent to which the proposed training content:

(i) Covers all of the relevant aspects of the subject matter; and

(ii) If relevant, is based on new knowledge derived from research activities of the proposed project.

(4) The extent to which the proposed training materials, methods, and content are appropriate to the trainees, including consideration of the skill level of the trainees and the subject matter of the materials.

(5) The extent to which the proposed training materials and methods are accessible to individuals with disabilities.

(6) The extent to which the applicant's proposed recruitment program is likely to be effective in recruiting highly qualified trainees, including those who are individuals with disabilities.

(7) The extent to which the applicant is able to carry out the training activities, either directly or through another entity.

(8) The extent to which the proposed didactic and classroom training programs emphasize scientific methodology and are likely to develop highly qualified researchers.

(9) The extent to which the quality and extent of the academic mentorship, guidance, and supervision to be provided to each individual trainee are of a high level and are likely to develop highly qualified researchers.

(10) The extent to which the type, extent, and quality of the proposed research experience, including the opportunity to participate in advanced-level research, are likely to develop highly qualified researchers.

(11) The extent to which the opportunities for collegial and collaborative activities, exposure to outstanding scientists in the field, and opportunities to participate in the preparation of scholarly or scientific publications and presentations are extensive and appropriate.

(g) Design of dissemination activities. In determining the extent to which the design is likely to be effective in accomplishing the objectives of the project, the Director considers one or more of the following factors:

(1) The extent to which the content of the information to be disseminated:

(i) Covers all of the relevant aspects of the subject matter; and

(ii) If appropriate, is based on new knowledge derived from research activities of the project.

(2) The extent to which the materials to be disseminated are likely to be effective and usable, including consideration of their quality, clarity, variety, and format.

(3) The extent to which the methods for dissemination are of sufficient quality, intensity, and duration.

(4) The extent to which the materials and information to be disseminated and the methods for dissemination are appropriate to the target population, including consideration of the familiarity of the target population with the subject matter, format of the information, and subject matter.

(5) The extent to which the information to be disseminated will be accessible to individuals with disabilities.

(h) Design of utilization activities. In determining the extent to which the design of utilization activities is likely to be effective in accomplishing the objectives of the project, the Director considers one or more of the following factors:

(1) The extent to which the potential new users of the information or technology have a practical use for the information and are likely to adopt the practices or use the information or technology, including new devices.

(2) The extent to which the utilization strategies are likely to be effective.

(3) The extent to which the information or technology is likely to be of use in other settings.

(i) Design of technical assistance activities. In determining the extent to which the design of technical assistance activities is likely to be effective in accomplishing the objectives of the project, the Director considers one or more of the following factors:

(1) The extent to which the methods for providing technical assistance are of sufficient quality, intensity, and duration.

(2) The extent to which the information to be provided through technical assistance covers all of the relevant aspects of the subject matter.

(3) The extent to which the technical assistance is appropriate to the target population, including consideration of the knowledge level of the target population, needs of the target population, and format for providing information.

(4) The extent to which the technical assistance is accessible to individuals with disabilities.

(j) Plan of operation. In determining the quality of the plan of operation, the Director considers one or more of the following factors:

(1) The adequacy of the plan of operation to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, and timelines for accomplishing project tasks.

(2) The adequacy of the plan of operation to provide for using resources, equipment, and personnel to achieve each objective.

(k) Collaboration. In determining the quality of collaboration, the Director considers one or more of the following factors:

(1) The extent to which the applicant's proposed collaboration with one or more agencies, organizations, or institutions is likely to be effective in achieving the relevant proposed activities of the project.

(2) The extent to which agencies, organizations, or institutions demonstrate a commitment to collaborate with the applicant.

(3) The extent to which agencies, organizations, or institutions that commit to collaborate with the applicant have the capacity to carry out collaborative activities.

(l) Adequacy and reasonableness of the budget. In determining the adequacy and the reasonableness of the proposed budget, the Director considers one or more of the following factors:

(1) The extent to which the costs are reasonable in relation to the proposed project activities.

(2) The extent to which the budget for the project, including any subcontracts, is adequately justified to support the proposed project activities.

(3) The extent to which the applicant is of sufficient size, scope, and quality to effectively carry out the activities in an efficient manner.

(m) Plan of evaluation. In determining the quality of the plan of evaluation, the Director considers one or more of the following factors:

(1) The extent to which the plan of evaluation provides for periodic assessment of progress toward:

(i) Implementing the plan of operation; and

(ii) Achieving the project's intended outcomes and expected impacts.

(2) The extent to which the plan of evaluation will be used to improve the performance of the project through the feedback generated by its periodic assessments.

(3) The extent to which the plan of evaluation provides for periodic assessment of a project's progress that is based on identified performance measures that:

(i) Are clearly related to the intended outcomes of the project and expected impacts on the target population; and

(ii) Are objective, and quantifiable or qualitative, as appropriate.

(n) Project staff. In determining the quality of the applicant's project staff, the Director considers one or more of the following factors:

(1) The extent to which the applicant encourages applications for employment from people with disabilities, who may include but are not limited to people with disabilities who have the greatest support needs.

(2) The extent to which the applicant encourages applications for employment from people who are members of other groups that have traditionally been underrepresented in research professions based on race, ethnicity, national origin, sex (including sexual orientation and gender identity), or age.

(3) The extent to which the key personnel and other key staff have appropriate training and experience in disciplines required to conduct all proposed activities.

(4) The extent to which the commitment of staff time is adequate to accomplish all the proposed activities of the project.

(5) The extent to which the key personnel are knowledgeable about the methodology and literature of pertinent subject areas.

(6) The extent to which the project staff includes outstanding scientists in the field.

(7) The extent to which key personnel have up-to-date knowledge from research or effective practice in the subject area covered in the priority.

(o) Adequacy and accessibility of resources. In determining the adequacy and accessibility of the applicant's resources to implement the proposed project, the Director considers one or more of the following factors:

(1) The extent to which the applicant is committed to provide adequate facilities, equipment, other resources, including administrative support, and laboratories, if appropriate.

(2) The quality of an applicant's past performance in carrying out a grant.

(3) The extent to which the applicant has appropriate access to populations and organizations representing individuals with disabilities to support advanced disability, independent living and clinical rehabilitation research.

(4) The extent to which the facilities, equipment, and other resources are appropriately accessible to individuals with disabilities who may use the facilities, equipment, and other resources of the project.

(p) Quality of the project design. In determining the quality of the design of the proposed project, the Director considers one or more of the following factors:

(1) The extent to which the goals, objectives, and outcomes to be achieved by the proposed project are clearly specified and measurable.

(2) The quality of the methodology to be employed in the proposed project.

(3) The extent to which the design of the proposed project includes a thorough, high-quality review of the relevant literature, a high-quality plan for project implementation, and the use of appropriate methodological tools to ensure successful achievement of project objectives.

(4) The extent to which the design of the proposed project is appropriate to, and will successfully address, the needs of the target population or other identified needs.

(5) The extent to which the proposed development efforts include adequate quality controls and, as appropriate, repeated testing of products.

(6) The extent to which the proposed project will be coordinated with similar or related efforts, and with other appropriate community, State, and Federal resources.

(7) The extent to which the design of the proposed project reflects up-to-date knowledge from research and effective practice.

(8) The extent to which the proposed project represents an exceptional approach to the priority or priorities established for the competition.

[81 FR 29159, May 11, 2016, as amended at 87 FR 50003, Aug. 15, 2022]

(a) The Director reserves funds to support field-initiated applications funded under this part when those applications have been awarded points totaling 85 percent or more of the maximum possible points under the procedures described in § 1330.23.

(b) In making a final selection from applications received when NIDILRR uses field-initiated priorities, the Director may consider whether one of the following conditions is met and, if so, use this information to fund an application out of rank order:

(1) The proposed project represents a unique opportunity to advance rehabilitation and other knowledge to improve the lives of individual with disabilities.

(2) The proposed project complements or balances research activity already planned or funded by NIDILRR through its annual priorities or addresses the research in a new and promising way.

(c) If the Director funds an application out of rank order under paragraph (b) of this section, the public will be notified through a notice on the NIDILRR Web site or through other means deemed appropriate by the Director.