Frequently Asked Questions - Clinical Competency Portfolio Review and I-OCA

These Frequently Asked Questions relate to the Clinical Competency Portfolio Review and the Independent Observed Clinical Activity (IOCA), which is a new component of this assessment.

Changes to timelines: It was previously communicated that trainees will be able to apply to sit an IOCA starting in November 2024.  The Education Committee, at their meeting on 6 September 2024, has extended the timelines for implementing IOCAs to August 2025, and the CCPR will be introduced in September 2026.  These revised timelines will provide adequate time for the development of new IOCA training resources and to commence progressive training for supervisors’ training in early 2025, prior to the launch of IOCA in August 2025.

1. What are the assessments that will be considered for the CCPR?

• Stage 2 and Stage 3 end-of-rotation In-Training Assessments (ITAs)
• Observed Clinical Activity (OCAs)
• Independent Observed Clinical activity (IOCAs)

Qualitative data such as supervisor feedback and narrative comments for the above assessments will also be considered.

2. What is the purpose of incorporating IOCAs into the Portfolio Review process?

IOCAs serve as an additional data point alongside other assessments to ensure a comprehensive, holistic evaluation of competencies. By providing independent assessments, IOCAs offer candidates constructive feedback on their skills in a clinical setting without potential bias from their usual supervisors. This helps trainees refine their clinical practice and build competence in essential skills for psychiatry.

3. What are the benefits of IOCAs for Trainees/SIMGs?

IOCAs offer candidates the opportunity to receive objective and independent feedback on their clinical assessment skills, free from any supervisory bias. This helps them improve their practice in day-to-day patient interactions and ensures that their progress is assessed fairly.

4. What are the eligibility criteria for the CCPR?

The CCPR eligibility criteria, based on the current Clinical Competency Assessment – Modified Portfolio Review (CCA – MPR) eligibility, are as follows (transition time exemptions are listed in further questions below).

5. Will completion of a Stage 2 IOCA be required for Trainees to progress to Stage 3, and will special consideration be given to the cohort starting Stage 3 training in August 2025?

With the implementation of the IOCA, trainees will be required to complete a Stage 2 IOCA before progressing to Stage 3.  

However, the requirement to complete a Stage 2 IOCA will not apply for trainees commencing Stage 3 in August 2025 or earlier.

6. Will there be any changes to assessment requirements for the substantially comparable SIMGs?

No, these changes are only applicable for the partially comparable SIMGs. 

7. What provisions are there for trainees who complete their final Stage 2 rotation in 2025 and take an approved Break In Training (BIT) in 2025?

Trainees who are on an approved Break in Training (BIT) in 2025 after completing their final Stage 2 rotation will be exempted from the Stage 2 IOCA requirement when applying for the CCPR later. They can progress to Stage 3 without fulfilling their Stage 2 IOCA requirement.

8. How does an IOCA differ from a standard OCA?

An IOCA is consistent with the protocols and delivery of the standard OCAs but there are three specific differences for IOCA: 

• it must be completed in a single session. 
• it is assessed by an external supervisor/independent assessor (who is not the candidate’s current Principal Supervisor or has not been the candidate’s supervisor in the previous 12 calendar months, has no personal or professional conflict of interest with the candidate they are assessing and has undertaken the IOCA calibration training).
• written patient consent is required prior to the IOCA assessment. 

9. What changes have been made to the Stage 2 IOCA requirement for September 2026 CCPR applicants?

The eligibility criteria for candidates applying for the September 2026 CCPR will exclude the requirement of a Stage 2 IOCA. The rest of the eligibility criteria will remain consistent with those outlined in Q4. This change aims to accommodate applicants in the initial cohort of the CCPR who may not have completed the Stage 2 IOCA by that time.

For SIMGs applying for the CCPR in September 2026, completing one Stage 3 IOCA and one Stage 3 OCA will be acceptable.

The special considerations for CCPR will also consider the trainees’ trajectory, so that no trainee is disadvantaged during the transition.

10. How will trainees be affected if there are any delays in the implementation of the IOCA assessment?

Trainees will be informed in advance of any delays in the implementation of the IOCA assessment. If a trainee misses the Stage 2 IOCA deadlines due to such delays, it will not affect their progression to Stage 3 training.

11. Where will the IOCAs be conducted?

The IOCA will be conducted face-to-face at the candidate’s workplace if this is a feasible option with the availability of an external supervisor/independent assessor. The IOCA must be completed in a single session.

If the external supervisor/independent assessor is not from the same jurisdiction, and the IOCA is not able to be conducted face-to-face due to the availability/location of the external supervisor/independent assessor, the IOCA will be conducted over Zoom/Microsoft Teams.  

12. When can candidates choose to schedule their IOCA during their Stage 2 and Stage 3 rotations?

Candidates will complete one IOCA during Stage 2 and one during Stage 3. Candidates are permitted to take their respective IOCAs at any time, without the necessity of waiting for a specific time within Stage 2 and Stage 3. The IOCAs can replace the mandatory OCA for that rotation.

13. Can the candidate re-sit the IOCA if they do not perform well?

Candidates who do not meet the required standard for their IOCAs are not required to re-sit the IOCA at that time.  Those who do not perform well in Stage 2 IOCA will need to be supported to improve their skills and address any identified deficits. A Stage 2 IOCA could be viewed as a reference point for subsequent improvement and development over time.

14. How many IOCAs are candidates required to complete for the CCPR?

Candidates are required to sit one IOCA in Stage 2 and one in Stage 3. Candidates must complete a Stage 2 IOCA before they can progress to Stage 3. The IOCAs can replace the mandatory OCA for that rotation.

15. Will supervisors receive training for conducting IOCAs/OCAs?

Yes, supervisor training will be provided to enhance the consistency of assessments across IOCAs and OCAs. This training aims to establish a shared understanding of the expected standards and ensure reliability in the evaluation process.

16. What are the criteria for being an external supervisor/independent assessor?

The external supervisor/independent assessor must meet the following criteria: 

• Has not been a supervisor of the candidate in the previous 12 months. 
• Has no personal or professional conflict of interest with the candidate. 
• Has undertaken the IOCA training and calibration (while this may not be possible in the first six months of IOCA implementation, it will become a mandatory requirement for all supervisors, as the same criteria will be applied for assessing OCAs as well).

17. Are supervisors expected to make pass/fail decisions for IOCAs/OCAs and ITAs?

Supervisors are not expected to make any pass/fail decisions. Their main focus is to provide meaningful feedback on the candidate’s performance, knowledge, and competency. 

18. Who selects the external supervisor/independent assessor (IA) for the IOCA, and how are they selected?

The Principal Supervisor is responsible for selecting an IA for conducting the IOCA. The Principal Supervisor will have access to a list of accredited IA database (across jurisdictions) via the InTrain IOCA form, which will automatically exclude any supervisors from the previous 12 months for the candidate who is applying to sit the IOCA.

The Principal Supervisor will also have an option to submit the IOCA form to the pool of accredited IA in the InTrain database. Individual IAs will be able to see the requests and select any of the IOCAs where they are available to conduct the assessment. 

If, after a reasonable time, an IA has not been assigned, the Principal Supervisor will email the College at clinicals@ranzcp.org for assistance. 

19. What procedures are followed if the available supervisors in smaller jurisdictions do not meet the 12-month criteria for external supervisors/independent assessors?

If the supervisors within a smaller jurisdiction do not meet the 12-month criteria, the Principal Supervisor may email the College at clinicals@ranzcp.org for assistance. 

Please note the candidate will not have any knowledge of the identity of the external supervisor/independent assessor who will be conducting their IOCA. 

20. How is conflict of interest managed for the IOCA assessment?

External supervisors/independent assessors are required to declare any personal or professional conflict of interest with the candidate to the Principal Supervisor before the assessment. They will receive the candidate's details via InTrain to review and declare any conflict of interest prior to the assessment. 

External supervisors/independent assessors who identify conflict of interest are expected not to participate in the IOCA assessment and must declare their conflict of interest via InTrain. 

Trainees will be able to submit an incident report up to 5 business days after completing an IOCA if they encounter an incident that impacted their performance in the assessment or experienced conflict of interest issues. 

21. Who is responsible for identifying suitable patient for a trainee’s IOCA?

Based on the feasibility, either the Principal Supervisor or the External Supervisor/Independent Assessor (or their delegate, such as the site coordinator) organizes the patient for the specified date and time and communicates with them about seeking their written consent before the assessment. 

They identify and arrange a suitable patient not well known to the trainee so that the trainee is able to do an initial clinical assessment, which is consistent with the existing OCA processes.

The candidate will not be informed about the identity of the specific patient who has been selected for their IOCA assessment. 

22. Is patient consent required before participating in an IOCA assessment?

Yes, patient consent is required before participating in an IOCA assessment. The Principal Supervisor or their delegate must provide the patient with a consent form, which is available for download from the IOCA webpage documentation. 

Patients should read the ‘Information to Patients’ document and sign the consent form prior to the assessment. The signed consent form will be kept by the health services and must be available for presentation to the College upon request.

23. Will the IOCA forms be available in InTrain?

Yes, the IOCA forms will be available in InTrain and completed forms will be included in their end-of-rotation ITAs. They will be clearly differentiated from the standard OCA forms.

24. I am currently employed in a setting that has difficulty holding online meetings and obtaining authorisation for external personnel to attend. How will I go about completing my IOCA?

In the event that appropriate facilities are unavailable, alternative arrangements will need to be considered. Please contact the College by email at clinicals@ranzcp.org to discuss options. 

25.Will trainees completing an Advanced Certificate (such as Child and Adolescent Psychiatry) be able to see a patient under the age of 18 for the purposes of the IOCA?

Generally, the College assessments are set at the standard expected of generalist psychiatrists, however any process that is currently followed for OCAs in sub-specialty areas may be applied to IOCAs. The consent process will ensure that the consent from families or carers is incorporated where required.

26. Can IOCAs be filmed in case there is an appeal at a later stage?

Currently there is no consideration being given to filming or recording IOCAs. This is consistent with the process of conducting OCAs in the workplace.

27. What is the governance structure for IOCA?

The IOCA Oversight Panel has been set up to provide guidance and governance oversight for the delivery and implementation of IOCAs including the calibration of supervisors and independent assessors. The Panel reports to the Committee for Training (CFT).

The Progression Competence Panel (PCP) oversees the holistic progression and determines overall competence and standards of candidates undertaking CCPR. The PCP reviews both qualitative and quantitative data from various assessments comprising the CCPR to determine whether the candidate has met the required standard.

28. How do IOCAs contribute to better decision-making in CCPR?

By providing additional independent assessment data, IOCAs allow for a more rigorous and accurate evaluation of a candidate’s performance over time. This data, along with other assessment data points, enhances the quality of decision-making during portfolio reviews.

29. Who will make the decision about the outcome of the CCPR?

The Progression Competence Panel (PCP) will comprise representatives from the relevant committees and independent Fellows will be responsible for making progression decisions based on the CCPR. The Panel will consider assessment elements of the CCPR covering assessments over a period of time from multiple sources in a holistic manner.

The assessment criteria, Terms of Reference and the business rules for the PCP are currently under development.