This subpart sets forth the regulations governing review and assessment of State Workforce Agency (SWA) compliance with the ES regulations at this part and parts 651, 652, 653, and 654 of this chapter. All recordkeeping and reporting requirements contained in this part and part 653 of this chapter have been approved by the Office of Management and Budget as required by the Paperwork Reduction Act of 1980.

(a) Each SWA must establish and maintain a self-appraisal system for ES operations to determine success in reaching goals and to correct deficiencies in performance. The self-appraisal system must include numerical (quantitative) appraisal and non-numerical (qualitative) appraisal.

(1) Numerical appraisal at the ES office level must be conducted as follows:

(i) Performance must be measured on a quarterly-basis against planned service levels as stated in the Unified or Combined State Plan (“State Plan”). The State Plan must be consistent with numerical goals contained in ES office plans.

(ii) To appraise numerical activities/indicators, actual results as shown on the Department's ETA Form 9172, or any successor report required by the Department must be compared to planned levels. Differences between achievement and plan levels must be identified.

(iii) When the numerical appraisal of required activities/indicators identifies significant differences from planned levels, additional analysis must be conducted to isolate possible contributing factors. This data analysis must include, as appropriate, comparisons to past performance, attainment of State Plan goals and consideration of pertinent non-numerical factors.

(iv) Results of ES office numerical reviews must be documented and significant deficiencies identified. A corrective action plan as described in paragraph (a)(6) of this section must be developed to address these deficiencies.

(v) The result of ES office appraisal, including corrective action plans, must be communicated in writing to the next higher level of authority for review. This review must cover adequacy of analysis, appropriateness of corrective actions, and need for higher level involvement. When this review is conducted at an area or district office, a report describing ES office performance within the area or district jurisdiction must be communicated to the SWA on a quarterly basis.

(2) Numerical appraisal at the SWA level must be conducted as follows:

(i) Performance must be measured on a quarterly basis against planned service levels as stated in the State Plan. The State Plan must be consistent with numerical goals contained in ES office plans.

(ii) To appraise these key numerical activities/indicators, actual results as shown on ETA Form 9172, or any successor report required by the Department must be compared to planned levels. Differences between achievement and plan levels must be identified.

(iii) The SWA must review statewide data and performance against planned service levels as stated in the State Plan on at least a quarterly basis to identify significant statewide deficiencies and to determine the need for additional analysis, including identification of trends, comparisons to past performance, and attainment of State Plan goals.

(iv) Results of numerical reviews must be documented and significant deficiencies identified. A corrective action plan as described in paragraph (a)(5) of this section must be developed to address these deficiencies. These plans must be submitted to the ETA Regional Office as part of the periodic performance process described at § 658.603(d)(2).

(3) Non-numerical (qualitative) appraisal of ES office activities must be conducted at least annually as follows:

(i) Each ES office must assess the quality of its services to applicants, employers, and the community and its compliance with Federal regulations.

(ii) At a minimum, non-numerical review must include an assessment of the following factors:

(A) Appropriateness of services provided to participants and employers;

(B) Timely delivery of services to participants and employers;

(C) Staff responsiveness to individual participants and employer needs;

(D) Thoroughness and accuracy of documents prepared in the course of service delivery; and

(E) Effectiveness of ES interface with external organizations, such as other ETA-funded programs, community groups, etc.

(iii) Non-numerical review methods must include:

(A) Observation of processes;

(B) Review of documents used in service provisions; and

(C) Solicitation of input from applicants, employers, and the community.

(iv) The result of non-numerical reviews must be documented and deficiencies identified. A corrective action plan addressing these deficiencies as described in paragraph (a)(6) of this section must be developed.

(v) The result of ES office non-numerical appraisal, including corrective actions, must be communicated in writing to the next higher level of authority for review. This review must cover thoroughness and adequacy of ES office appraisal, appropriateness of corrective actions, and need for higher level involvement. When this review is conducted at an area or district level, a report summarizing local ES office performance within that jurisdiction must be communicated to the SWA on an annual basis.

(4) As part of its oversight responsibilities, the SWA must conduct onsite reviews in those ES offices which show continuing internal problems or deficiencies in performance as indicated by such sources as data analysis, non-numerical appraisal, or other sources of information.

(5) Non-numerical (qualitative) review of SWA ES activities must be conducted as follows:

(i) SWA operations must be assessed annually to determine compliance with Federal regulations.

(ii) Results of non-numerical reviews must be documented and deficiencies identified. A corrective action plan addressing these deficiencies must be developed.

(6) Corrective action plans developed to address deficiencies uncovered at any administrative level within the State as a result of the self-appraisal process must include:

(i) Specific descriptions of the type of action to be taken, the time frame involved, and the assignment of responsibility.

(ii) Provision for the delivery of technical assistance as needed.

(iii) A plan to conduct follow-up on a timely basis to determine if action taken to correct the deficiencies has been effective.

(7)(i) The provisions of the ES regulations which require numerical and non-numerical assessment of service to special applicant groups (e.g., services to veterans at 20 CFR part 1001—Services for Veterans and services to MSFWs at this part and part 653 of this chapter), are supplementary to the provisions of this section.

(ii) Each State Administrator and ES office manager must ensure their staff know and carry out ES regulations, including regulations on performance standards and program emphases, and any corrective action plans imposed by the SWA or by the Department.

(iii) Each State Administrator must ensure the SWA complies with its approved State Plan.

(iv) Each State Administrator must ensure to the maximum extent feasible the accuracy of data entered by the SWA into Department-required management information systems. Each SWA must establish and maintain a data validation system pursuant to Department instructions. The system must review every local ES office at least once every 4 years. The system must include the validation of time distribution reports and the review of data gathering procedures.

(b) [Reserved]

The ETA National Office must:

(a) Monitor ETA Regional Offices' operations under ES regulations;

(b) From time to time, conduct such special reviews and audits as necessary to monitor ETA regional office and SWA compliance with ES regulations;

(c) Offer technical assistance to the ETA regional offices and SWAs in carrying out ES regulations and programs;

(d) Have report validation surveys conducted in support of resource allocations; and

(e) Develop tools and techniques for reviewing and assessing SWA performance and compliance with ES regulations.

(f) ETA must appoint a National Monitor Advocate (NMA), who must devote full time to the duties set forth in this subpart. The NMA must:

(1) Review the effective functioning of the Regional Monitor Advocates (RMAs) and SMAs;

(2) Review the performance of SWAs in providing the full range of ES services to MSFWs;

(3) Take steps to resolve or refer ES-related problems of MSFWs which come to their attention;

(4) Take steps to refer non-ES-related problems of MSFWs which come to their attention;

(5) Recommend to the Administrator changes in policy toward MSFWs; and

(6) Serve as an advocate to improve services for MSFWs within the ES system. The NMA must be a member of the National Farm Labor Coordinated Enforcement Staff Level Working Committee and other Occupational Safety and Health Administration (OSHA) and Wage and Hour Division (WHD) task forces, and other committees as appropriate.

(g) The NMA must be appointed by the Office of Workforce Investment Administrator (Administrator) after informing farmworker organizations and other organizations with expertise concerning MSFWs of the opening and encouraging them to refer qualified applicants to apply through the Federal merit system. Among qualified candidates, determined through merit systems procedures, individuals must be sought who meet the criteria used in the selection of the SMAs, as provided in SWA self-monitoring requirements at § 653.108(a) of this chapter.

(h) The NMA must be assigned staff necessary to fulfill effectively all the responsibilities set forth in this subpart.

(i) The NMA must submit the Annual Report to the OWI Administrator, the ETA Assistant Secretary, and the National Farm Labor Coordinated Enforcement Committee covering the matters set forth in this subpart.

(j) The NMA must monitor and assess SWA compliance with ES regulations affecting MSFWs on a continuing basis. Their assessment must consider:

(1) Information from RMAs and SMAs;

(2) Program performance data, including the service indicators;

(3) Periodic reports from regional offices;

(4) All Federal on-site reviews;

(5) Selected State on-site reviews;

(6) Other relevant reports prepared by the ES;

(7) Information received from farmworker organizations and employers; and

(8) Their personal observations from visits to SWAs, ES offices, agricultural work sites, and migrant camps. In the Annual Report, the NMA must include both a quantitative and qualitative analysis of their findings and the implementation of their recommendations by State and Federal officials, and must address the information obtained from all of the foregoing sources.

(k) The NMA must review the activities of the State/Federal monitoring system as it applies to services to MSFWs and the Complaint System including the effectiveness of the regional monitoring function in each region and must recommend any appropriate changes in the operation of the system. The NMA's findings and recommendations must be fully set forth in the Annual Report.

(l) If the NMA finds the effectiveness of any RMA has been substantially impeded by the Regional Administrator or other regional office official, they must, if unable to resolve such problems informally, report and recommend appropriate actions directly to the OWI Administrator. If the NMA receives information that the effectiveness of any SMA has been substantially impeded by the State Administrator, a State or Federal ES official, or other ES staff, they must, in the absence of a satisfactory informal resolution at the regional level, report and recommend appropriate actions directly to the OWI Administrator.

(m) The NMA must be informed of all proposed changes in policy and practice within the ES, including ES regulations, which may affect the delivery of services to MSFWs. The NMA must advise the OWI Administrator concerning all such proposed changes which may adversely affect MSFWs. The NMA must propose directly to the OWI Administrator changes in ES policy and administration which may substantially improve the delivery of services to MSFWs. They also must recommend changes in the funding of SWAs and/or adjustment or reallocation of the discretionary portions of funding formulae.

(n) The NMA must participate in the review and assessment activities required in this section and §§ 658.700 through 658.711. As part of such participation, the NMA, or if they are unable to participate, an RMA must accompany the National Office review team on National Office on-site reviews. The NMA must engage in the following activities during each State on-site review:

(1) They must accompany selected outreach staff on their field visits.

(2) They must participate in field check(s) of migrant camps or work site(s) where MSFWs have been placed on inter or intrastate clearance orders.

(3) They must contact local WIOA sec. 167 National Farmworker Jobs Program grantees or other farmworker organizations as part of the on-site review and discuss with representatives of these organizations current trends and any other pertinent information concerning MSFWs.

(4) They must meet with the SMA and discuss the full range of the ES services to MSFWs, including monitoring and the Complaint System.

(o) In addition to the duties specified in paragraph (f) of this section, the NMA each year during the harvest season must visit the four States with the highest level of MSFW activity during the prior fiscal year, if they are not scheduled for a National Office on-site review during the current fiscal year, and must:

(1) Meet with the SMA and other ES staff to discuss MSFW service delivery; and

(2) Contact representatives of MSFW organizations and interested employer organizations to obtain information concerning ES delivery and coordination with other agencies.

(p) The NMA must perform duties specified in §§ 658.700 through 765.711. As part of this function, they must monitor the performance of regional offices in imposing corrective action. The NMA must report any deficiencies in performance to the Administrator.

(q) The NMA must establish routine and regular contacts with WIOA sec. 167 National Farmworker Jobs Program grantees, other farmworker organizations and agricultural employers and/or employer organizations. The NMA must attend conferences or meetings of these groups wherever possible and must report to the Administrator and the National Farm Labor Coordinated Enforcement Committee on these contacts when appropriate. The NMA must include in the Annual Report recommendations about how the Department might better coordinate ES and WIOA sec. 167 National Farmworker Jobs Program services as they pertain to MSFWs.

(r) In the event that any SMA or RMA, enforcement agency, or MSFW group refers a matter to the NMA which requires emergency action, the NMA must assist them in obtaining action by appropriate agencies and staff, inform the originating party of the action taken, and, upon request, provide written confirmation.

(s) Through all the mechanisms provided in this subpart, the NMA must aggressively seek to ascertain and remedy, if possible, systemic deficiencies in the provisions of ES services and protections afforded by these regulations to MSFWs. The NMA must:

(1) Use the regular reports on complaints submitted by SWAs and ETA regional offices to assess the adequacy of these systems and to determine the existence of systemic deficiencies.

(2) Provide technical assistance to ETA regional office and ES staff for administering the Complaint System, and any other ES services as appropriate.

(3) Recommend to the Regional Administrator specific instructions for action by regional office staff to correct any ES-related systemic deficiencies. Prior to any ETA review of regional office operations concerning ES services to MSFWs, the NMA must provide to the Regional Administrator a brief summary of ES-related services to MSFWs in that region and their recommendations for incorporation in the regional review materials as the Regional Administrator and ETA reviewing organization deem appropriate.

(4) Recommend to the National Farm Labor Coordinated Enforcement Committee specific instructions for action by WHD and OSHA regional office staff to correct any non-ES-related systemic deficiencies of which he/she is aware.

(a) The Regional Administrator must have responsibility for the regular review and assessment of SWA performance and compliance with ES regulations.

(b) The Regional Administrator must participate with the National Office staff in reviewing and approving the State Plan for the SWAs within the region. In reviewing the State Plans the Regional Administrator and appropriate National Office staff must consider relevant factors including the following:

(1) State Workforce Agency compliance with ES regulations;

(2) State Workforce Agency performance against the goals and objectives established in the previous State Plan;

(3) The effect which economic conditions and other external factors considered by the ETA in the resource allocation process may have had or are expected to have on the SWA's performance;

(4) SWA adherence to national program emphasis; and

(5) The adequacy and appropriateness of the State Plan for carrying out ES programs.

(c) The Regional Administrator must assess the overall performance of SWAs on an ongoing basis through desk reviews and the use of required reporting systems and other available information.

(d) As appropriate, Regional Administrators must conduct or have conducted:

(1) Comprehensive on-site reviews of SWAs and their offices to review SWA organization, management, and program operations;

(2) Periodic performance reviews of SWA operation of ES programs to measure actual performance against the State Plan, past performance, the performance of other SWAs, etc.;

(3) Audits of SWA programs to review their program activity and to assess whether the expenditure of grant funds has been in accordance with the approved budget. Regional Administrators also may conduct audits through other agencies or organizations or may require the SWA to have audits conducted;

(4) Validations of data entered into management information systems to assess:

(i) The accuracy of data entered by the SWAs into the management information system;

(ii) Whether the SWAs' data validating and reviewing procedures conform to Department instructions; and

(iii) Whether SWAs have implemented any corrective action plans required by the Department to remedy deficiencies in their validation programs;

(5) Technical assistance programs to assist SWAs in carrying out ES regulations and programs;

(6) Reviews to assess whether the SWA has complied with corrective action plans imposed by the Department or by the SWA itself; and

(7) Unannounced field checks of a sample of agricultural work sites to which ES placements have been made through the clearance system to determine and document whether wages, hours, and working and housing conditions are as specified on the clearance order. If regional office staff find reason to believe that conditions vary from clearance order specifications, findings must be documented on the Complaint/Apparent Violation Referral Form and provided to the State Workforce Agency to be processed as an apparent violation under § 658.419.

(e) The Regional Administrator must provide technical assistance to SWAs to assist them in carrying out ES regulations and programs.

(f) The Regional Administrator must appoint a RMA who must carry out the duties set forth in this subpart. The RMA must:

(1) Review the effective functioning of the SMAs in their region;

(2) Review the performance of SWAs in providing the full range of ES services to MSFWs;

(3) Take steps to resolve ES-related problems of MSFWs which come to their attention;

(4) Recommend to the Regional Administrator changes in policy towards MSFWs;

(5) Review the operation of the Complaint System; and

(6) Serve as an advocate to improve service for MSFWs within the ES. The RMA must be a member of the Regional Farm Labor Coordinated Enforcement Committee.

(g) The RMA must be appointed by the Regional Administrator after informing farmworker organizations and other organizations in the region with expertise concerning MSFWs of the opening and encouraging them to refer qualified applicants to apply through the Federal merit system. The RMA must have direct personal access to the Regional Administrator wherever they find it necessary. Among qualified candidates, individuals must be sought who meet the criteria used in the selection of the SMAs, as provided in § 653.108(b) of this chapter.

(h) The Regional Administrator must ensure that staff necessary to fulfill effectively all the regional office responsibilities set forth in this section are assigned.

(i) The RMA must participate in training sessions including those offered by the National Office and those necessary to maintain competency and enhance their understanding of issues farmworkers face (including trainings offered by OSHA, WHD, EEOC, CRC, and other organizations offering farmworker-related information).

(j) At the regional level, the RMA must have primary responsibility for:

(1) Monitoring the effectiveness of the Complaint System set forth at subpart E of this part;

(2) Apprising appropriate State and ETA officials of deficiencies in the Complaint System; and

(3) Providing technical assistance to SMAs in the region.

(k) At the ETA regional level, the RMA must have primary responsibility for ensuring SWA compliance with ES regulations as it pertains to services to MSFWs is monitored by the regional office. They must independently assess on a continuing basis the provision of ES services to MSFWs, seeking out and using:

(1) Information from SMAs, including all reports and other documents;

(2) Program performance data;

(3) The periodic and other required reports from SWAs;

(4) Federal on-site reviews;

(5) Other reports prepared by the National Office;

(6) Information received from farmworker organizations and employers; and

(7) Any other pertinent information which comes to their attention from any possible source.

(8) In addition, the RMA must consider their personal observations from visits to ES offices, agricultural work sites, and migrant camps.

(l) The RMA must assist the Regional Administrator and other line officials in applying appropriate corrective and remedial actions to State agencies.

(m) The Regional Administrator's quarterly report to the National Office must include the RMA's summary of their independent assessment as required in paragraph (f)(5) of this section. The fourth quarter summary must include an Annual Summary from the region. The summary also must include both a quantitative and a qualitative analysis of their reviews and must address all the matters with respect to which they have responsibilities under these regulations.

(n) The RMA must review the activities and performance of the SMAs and the State monitoring system in the region, and must recommend any appropriate changes in the operation of the system to the Regional Administrator. The RMA's review must include a determination whether the SMA:

(1) Does not have adequate access to information;

(2) Is being impeded in fulfilling their duties; or

(3) Is making recommendations that are being consistently ignored by SWA officials. If the RMA believes that the effectiveness of any SMA has been substantially impeded by the State Administrator, other State agency officials, any Federal officials, or other ES staff, the RMA must report and recommend appropriate actions to the Regional Administrator. Copies of the recommendations must be provided to the NMA electronically or in hard copy.

(o)(1) The RMA must be informed of all proposed changes in policy and practice within the ES, including ES regulations, which may affect the delivery of services to MSFWs. They must advise the Regional Administrator on all such proposed changes which, in their opinion, may adversely affect MSFWs or which may substantially improve the delivery of services to MSFWs.

(2) The RMA also may recommend changes in ES policy or regulations, as well as changes in the funding of State Workforce Agencies and/or adjustments of reallocation of the discretionary portions of funding formulae as they pertain to MSFWs.

(p) The RMA must participate in the review and assessment activities required in this section and §§ 658.700 through 658.711. The RMA, an assistant, or another RMA must participate in National Office and regional office on-site statewide reviews of ES services to MSFWs in States in the region. The RMA must engage in the following activities in the course of participating in an on-site SWA review:

(1) Accompany selected outreach staff on their field visits;

(2) Participate in a field check of migrant camps or work sites where MSFWs have been placed on intrastate or interstate clearance orders;

(3) Contact local WIOA sec. 167 National Farmworker Jobs Program grantees or other farmworker organizations as part of the on-site review, and must discuss with representatives of these organizations perceived trends, and/or other relevant information concerning MSFWs in the area; and

(4) Meet with the SMA and discuss the full range of the ES services to MSFWs, including monitoring and the Complaint System.

(q) During the calendar quarter preceding the time of peak MSFW activity in each State, the RMA must meet with the SMA and must review in detail the State Workforce Agency's capability for providing the full range of services to MSFWs as required by ES regulations, during the upcoming harvest season. The RMA must offer technical assistance and recommend to the SWA and/or the Regional Administrator any changes in State policy or practice that the RMA finds necessary.

(r) As appropriate, each year during the peak harvest season, the RMA must visit each State in the region not scheduled for an onsite review during that fiscal year and must:

(1) Meet with the SMA and other ES staff to discuss MSFW service delivery; and

(2) Contact representatives of MSFW organizations to obtain information concerning ES delivery and coordination with other agencies and interested employer organizations.

(s) The RMA must initiate and maintain regular and personal contacts, including informal contacts in addition to those specifically required by these regulations, with SMAs in the region. In addition, the RMA must have personal and regular contact with the NMA. The RMA also must establish routine and regular contacts with WIOA sec. 167 National Farmworker Jobs Program grantees, other farmworker organizations and agricultural employers and/or employer organizations in the RMA's region. The RMA must attend conferences or meetings of these groups wherever possible and must report to the Regional Administrator and the Regional Farm Labor Coordinated Enforcement Committee on these contacts when appropriate. The RMA also must make recommendations as to how the Department might better coordinate ES and WIOA sec. 167 National Farmworker Jobs Program services to MSFWs.

(t) The RMA must attend MSFW-related public meeting(s) conducted in the region, as appropriate. Following such meetings or hearings, the RMA must take such steps or make such recommendations to the Regional Administrator, as the RMA deems necessary to remedy problem(s) or condition(s) identified or described therein.

(u) The RMA must attempt to achieve regional solutions to any problems, deficiencies, or improper practices concerning services to MSFWs which are regional in scope. Further, the RMA must recommend policies, offer technical assistance, or take any other necessary steps as they deem desirable or appropriate on a regional, rather than State-by-State, basis to promote region-wide improvement in the delivery of ES services to MSFWs. The RMA must facilitate region-wide coordination and communication regarding provision of ES services to MSFWs among SMAs, State Administrators, and Federal ETA officials to the greatest extent possible. In the event that any SWA or other RMA, enforcement agency, or MSFW group refers a matter to the RMA which requires emergency action, the RMA must assist them in obtaining action by appropriate agencies and staff, inform the originating party of the action taken, and, upon request, provide written confirmation.

(v) The RMA must initiate and maintain such contacts as they deem necessary with RMAs in other regions to seek to resolve problems concerning MSFWs who work, live, or travel through the region. The RMA must recommend to the Regional Administrator and/or the National Office inter-regional cooperation on any particular matter, problem, or policy with respect to which inter-regional action is desirable.

(w) The RMA must establish regular contacts with the regional agricultural coordinators from WHD and OSHA and any other regional staff from other Federal enforcement agencies and must establish contacts with the staff of other Department agencies represented on the Regional Farm Labor Coordinated Enforcement Committee and to the extent necessary, on other pertinent task forces or committees.

(x) The RMA must participate in the regional reviews of the State Plans, and must comment to the Regional Administrator as to the SWA compliance with the ES regulations as they pertain to services to MSFWs, including the staffing of ES offices.

(a) State Workforce Agencies must compile program performance data required by the Department, including statistical information on program operations.

(b) The Department must use the program performance data in assessing and evaluating whether each SWA has complied with ES regulations and its State Plan.

(c) In assessing and evaluating program performance data, the Department must act in accordance with the following general principles:

(1) The fact that the program performance data from a SWA, whether overall or relative to a particular program activity, indicate poor program performance does not by itself constitute a violation of ES regulations or of the State Workforce Agency's responsibilities under its State Plan;

(2) Program performance data, however, may so strongly indicate that a SWA's performance is so poor that the data may raise a presumption (prima facie case) that a SWA is violating ES regulations or the State Plan. A SWA's failure to meet the operational objectives set forth in the State Plan raises a presumption that the agency is violating ES regulations and/or obligations under its State Plan. In such cases, the Department must afford the SWA an opportunity to rebut the presumption of a violation pursuant to the procedures at subpart H of this part.

(3) The Department must take into account that certain program performance data may measure items over which SWAs have direct or substantial control while other data may measure items over which the SWA has indirect or minimal control.

(i) Generally, for example, a SWA has direct and substantial control over the delivery of ES services such as referrals to jobs, job development contacts, counseling, referrals to career and supportive services, and the conduct of field checks.

(ii) State Workforce Agencies, however, have only indirect control over the outcome of services. For example, SWAs cannot guarantee that an employer will hire a referred applicant, nor can they guarantee that the terms and conditions of employment will be as stated on a job order.

(iii) Outside forces, such as a sudden heavy increase in unemployment rates, a strike by SWA employees, or a severe drought or flood, may skew the results measured by program performance data.

(4) The Department must consider a SWA's failure to keep accurate and complete program performance data required by ES regulations as a violation of the ES regulations.

(a) The Regional Administrator must inform SWAs in writing of the results of review and assessment activities and, as appropriate, must discuss with the State Administrator the impact or action required by the Department as a result of review and assessment activities.

(b) The ETA National Office must transmit the results of any review and assessment activities it conducted to the Regional Administrator who must send the information to the SWA.

(c) Whenever the review and assessment indicates a SWA violation of ES regulations or its State Plan, the Regional Administrator must follow the procedures set forth at subpart H of this part.

(d) Regional Administrators must follow-up any corrective action plan imposed on a SWA under subpart H of this part by further review and assessment of the State Workforce Agency pursuant to this subpart.