GME Policy Manual

POLICY#:     IX. 5
SECTION:     INSTITUTIONAL RESPONSIBILITIES
SUBJECT:     INTERNAL REVIEW OF GME PROGRAMS WITH NO RESIDENTS

I. PURPOSE

To conduct internal reviews of all residency/fellowship programs to assure compliance with ACGME regulations.

II. PROCESS/PROTOCOL

A. Internal reviews are conducted on all residency/fellowship programs. A scheduled review takes place approximately midpoint between the date of the meeting at which the final accreditation action was taken and the time of the next site visit. When a program has no residents enrolled at the mid-point of the review cycle, the following circumstances apply:

1. A modified internal review that ensures the program has maintained adequate faculty and staff resources, clinical volume, and other necessary curricular elements required to be in substantial compliance with the Institutional, Common and specialty-specific Program Requirements prior to the program enrolling a resident.

2. After enrolling a resident, an internal review must be completed within the second six-month period of the resident's first year in the program.

B. Active participation in the internal review process is one of the most important responsibilities for members of the GMEC. The process for conducting an effective review is as follows:

1. Approximately three months before the date of the review, Program Directors and reviewers are notified by the Office of Graduate Medical Education of the date of the review.

2. The GME office will forward the following documents to the review team: 1) A copy of the institutional requirements, 2) a copy of that program's RRC requirements, including the common program requirements in effect at the time of the review, 3) accreditation letters of notification from previous ACGME reviews, 4) progress reports sent to the RRC, 5) reports from previous GMEC internal reviews of the program, and 6) results from internal or external resident surveys. The team is instructed to utilize this material and interviews (see below) to answer the questions listed below in this document.

The director of the program being reviewed completes the departmental self-study by answering the GME questions in Section Three, and forwards a copy to each member of the review team. The Director of the program being reviewed also provides a copy of the document that meets the ACGME requirement for the program's annual formal systematic evaluation of the curriculum.

3. In addition to written material provided by the Office of GME, reviewers should collect supporting documents from program directors that address questions.

4. Reviewers must interview the program director, key faculty members from every site/institution to where residents rotate, and other individuals deemed appropriate by the committee, addressing the questions in the written protocol, to validate written responses and obtain a variety of perspectives regarding the program. The report must note all individuals interviewed and how they were selected.

5. The Internal Review Committee will consist of at least one faculty member and at least one resident from within RWJMS, but not from within the GME program being reviewed.

6. The internal review assesses each program's:

a. Compliance with the Common, specialty/subspecialty-specific Program, and Institutional Requirements;

b. Educational Objectives and effectiveness in meeting those objectives;

c. Educational and financial resources;

d. Effectiveness in addressing areas of non-compliance and concerns in previous ACGME accreditation letters of notification and previous internal reviews;

e. Effectiveness of educational outcomes in the ACGME general competencies;

f. Effectiveness in using evaluation tools and outcome measures to assess a resident's level of competence in each of the ACGME general competencies; and,

g. Annual program improvement efforts in:

(1) resident performance using aggregated resident data;

(2) faculty development;

(3) graduate performance including performance of program graduates on the certification examination; and,

(4) program quality.

7. A written copy of the review should be shared with the Program Director of the program being reviewed prior to submission to the Office of GME. The report should be sent to the GME Office one week before the meeting at which the review will be presented to the GMEC to allow dissemination of the review with the agenda packet to Committee members. The written report of the internal review contains, at a minimum, the following and follows the format of the report template in Attachment II:

a. the name of the program or subspecialty program reviewed and the date the written report is initially presented to and reviewed by the GMEC;

b. the names and titles of the internal review committee members;

c. a brief description of how the internal review process was conducted, including the list of the groups/individuals interviewed, their titles and the documents reviewed;

d. sufficient documentation to demonstrate that a comprehensive review followed the GMEC's internal review protocol;

e. a list of the citations and areas of noncompliance or any concerns or comments from the previous ACGME accreditation letter of notification with a summary of how the program and/or institution subsequently addressed each item.

8. The Program Director must be present for his/her program to be reviewed. In addition to responding to the review, he/she is asked to comment regarding corrective actions on any citation from the most recent RRC review.

9. A copy of the internal review and summary of the discussion by the GMEC is included in the minutes of the GMEC.

10. Each program director will receive correspondence from the chair of the GMEC as a follow-up on the concerns and recommendations of the committee. The program director must respond directly to the chair in writing as to how the program will address concerns and report back to the GMEC at the next scheduled meeting.

III. PROGRAM REVIEW QUESTIONS

A. Give an overview of the program utilizing the most recent ACGME Institutional and Program requirements from the Essentials of Accredited Residency Programs as your guide.

1. Name the Program Director and Associate Program Directors. If more than one hospital participates in the residency program, describe how the program is supervised at each hospital.

2. Describe the role each participating hospital plays in the residency, mentioning both the strengths and weaknesses.

3. How many full and part-time faculty are there? What is the ratio of full-time faculty to residents? How many voluntary faculty are there? Describe the roles of paid and voluntary faculty.

4. Does the program have written policies on Supervision, Back-up, Selection, Evaluation, Promotion & Dismissal, Duty Hours, Effect of Leave of Absence on Completion of Program, Resident Responsibilities for each level and Goals & Objectives for each level? Please obtain copies and incorporate into this review document.

5. How many hours per week will residents work at each PGY level? Will they have 24 consecutive hours out of the hospital each week? How frequently will they take call? Are the on-call facilities adequate? How many consecutive hours will they work?

6. What system is planned for evaluating resident performance and giving them feedback? Is it written? (If so, include sample form.) How often will evaluations be performed?

7. Describe the mechanism in place for evaluating program faculty? When is the evaluation performed?

8. How will the educational effectiveness of your program be evaluated in a systematic manner? Please describe annual program improvement efforts in:

a. resident performance using aggregated resident data;

b. faculty development;

c. graduate performance including performance of program graduates on the certification examination; and

d. program quality.

9. The Program Director must provide:

a. evidence of a curriculum, complete with goals and objectives for teaching the six competencies (1-6 below), and their involvement in the curriculum.

(1) Patient Care

(2) Medical Knowledge

(3) Practice-based Learning and Improvement

(4) Interpersonal and Communication Skills

(5) Professionalism

(6) Systems-based Practice

b. a list of the evaluation tools used by the program for evaluating the competencies (Please complete the chart in Attachment I).

c. the dependable measure to access the residents' competence in each of the above areas.

d. the process developed to link educational outcomes with program improvement.

10. Describe the system for resident supervision and increasing responsibilities.

11. Are there written educational goals and objectives for each rotation within the program? (Include a copy.) Will the housestaff be aware of them? Are the educational and financial resources available adequate to meet these objectives?

12. Is there a required ambulatory care component? Describe it, its strengths, and its weaknesses.

13. Describe the educational conferences offered. Is there a curriculum? Is there a system for evaluating them? Will residents be able to attend?

14. How will resident procedural experiences or operative statistics be documented and evaluated?

15. Will a research experience be required or available?

16. How will residents formally evaluate the program, faculty and rotations? (include sample copies).

17. Will residents take an in-training exam? How will the results be used?

B. Please give a summary of your assessment of the strengths and weaknesses of the program.

Approved by GMEC on 11/13/07

Attachment I

AN INTERNAL REVIEW CHECKLIST FOR THE GENERAL COMPETENCIES

List the evaluation tools used by the program for the following General Competencies:

General Competencies	List Evaluation Tools Used or In Development by the Program
		Other tools being designed by Program?
Patient Care
Medical Knowledge
Interpersonal and Communication Skills
Professionalism
Practice Based Learning
Systems Based Practice

Attachment II

(Internal Review Report Template)

INTERNAL REVIEW

Program Reviewed

Midpoint of ACGME

Review Cycle

Time period over which

review was conducted &

report was written

Date Presented to GMEC

Review Team: (Name of Program Director)

Program Director, (Residency or Fellowship Program)

UMDNJ-Robert Wood Johnson Medical School

(Resident Name)

PGY (level), (Residency or Fellowship Program)

UMDNJ-Robert Wood Johnson Medical School

(Name of additional team member)

(Title)

(Institution)

I. Process Description

1. Interviewed:

(List Program Director Name & Title)

(List key faculty member name, title and site – must be at least one from each rotation site)

No Residents Currently In Program

2. Documentation Reviewed:

Responses to Internal Review Questions

ACGME Common Requirements

ACGME Specialty/Subspecialty Requirements

Institutional Requirements

Goals and Objectives

Duty Hours Tracking

Six General Competencies and Evaluations

Curriculum

Previous ACGME reports and responses (list specific correspondence) with concerns or comments and how the program addressed each one

Previous internal review (list review date) with concerns or comments and how the program addressed each one

(Additional Information Reviewed)

3. Description of Review Process

(How was the review conducted?

II. Report

1.        Introduction

(Provide a brief description of the program.)

2.       Questions and Answers

(Provide the answers to the following questions from the program director whose program is being reviewed:)

A. Give an overview of the program utilizing the most recent ACGME Institutional and Program requirements from the Essentials of Accredited Residency Programs as your guide.

1. Name the Program Director and Associate Program Directors. If more than one hospital participates in the residency program, describe how the program is supervised at each hospital.

2. Describe the role each participating hospital plays in the residency, mentioning both the strengths and weaknesses.

3. How many full and part-time faculty are there? What do you anticipate the ratio of full-time faculty to residents to be? How many voluntary faculty are there? Describe the roles of paid and voluntary faculty.

4. Does the program have written policies on Supervision, Back-up, Selection, Evaluation, Promotion & Dismissal, Duty Hours, Effect of Leave of Absence on Completion of Program, Resident Responsibilities for each level and Goals & Objectives for each level? Please obtain copies and incorporate into this review document.

5. How many hours per week will residents work at each PGY level? Will they have 24 consecutive hours out of the hospital each week? How frequently will they take call? Are the on-call facilities adequate? How many consecutive hours will they work?

6. What system is planned for evaluating resident performance and giving them feedback? Is it written? (If so, include sample form.) How often will evaluations be performed?

7. Describe the mechanism planned for evaluating program faculty? When is the evaluation performed?

8. Who will the educational effectiveness of your program be evaluated in a systematic manner? Please describe annual program improvement efforts in:

• resident performance using aggregated resident data;

• faculty development;

• graduate performance including performance of program graduates on the certification examination; and

• program quality.

9. The Program Director must provide:

a. evidence of a curriculum, complete with goals and objectives for teaching the six competencies (1-6 below), and their involvement in the curriculum.

(1) Patient Care

(2) Medical Knowledge

(3) Practice-based Learning and Improvement

(4) Interpersonal and Communication Skills

(5) Professionalism

(6) Systems-based Practice

b. a list of the evaluation tools used by the program for evaluating the competencies ( Please complete the chart in Attachment I).

c. the dependable measures to access the residents' competence in each of the above areas.

d. the process developed to link educational outcomes with program improvement.

10. Describe the system for resident supervision and increasing responsibilities.

11. Are there written educational goals and objectives for each rotation within the program? (Include a copy.) Are the housestaff be made aware of them? Are the educational and financial resources available adequate to meet these objectives?

12. Is there a required ambulatory care component? Describe it, its strengths, and its weaknesses.

13. Describe the educational conferences offered. Is there a curriculum? Is there a system for evaluating them? Are residents able to attend?

14. How will resident procedural experiences or operative statistics be documented and evaluated?

15. Will a research experience be required or available?

16. How will residents formally evaluate the program, faculty and rotations? (include sample copies)

17. Will residents take an in-training exam? How will the results be used?

B. Please give a summary of your assessment of the strengths and weaknesses of the program.

3.        Summary of Interview with Faculty

4. Review Summary

(Provide a brief overview with sufficient documentation or discussion of the specialty's/subspecialty's Program Requirements and the Institutional Requirements to demonstrate that a comprehensive review was conducted and was based on the GMEC's internal review protocol. Also include a list of areas of noncompliance or any concerns or comments from the previous ACGME accreditation letter with a summary of how the program and/or institution addressed each one.)

(List strengths and weaknesses or concerns.)

(Include the General Competency Checklist grid from the program in the electronic version of the report that is emailed to the GME Office (Brenda.Rambo@umdnj.edu) and forward any additional documentation requested of the program director in hard copy format to the GME Office.)

Attachment III

INTERNAL REVIEW CHECKLIST

Program Reviewed

Program Review Team

Midpoint Date

Date submitted to GME Office

Date Presented to GMEC

Did all assigned review team members participate in review? Yes O No O

Interview conducted with Program Director? Yes O No O

Interview conducted with key faculty from all sites/institutions? Yes O No O

Were reviewed documents listed in the report? Yes O No O

Was report template format followed? Yes O No O

Was Attachment I ( An Internal Review Checklist for the General

Competencies ) included with the Internal Review Report Yes O No O

Was review presented to GMEC at the midpoint of the review cycle as

determined by the ACGME? Yes O No O

If no, why not? _