The Intelligent Tutoring System
An online tool to enhance your course curriculum in the field of behavioral and mental health service. Provide your students with essential practice in a variety of skills, along with tools to analyze their performance.
Learn more about ITS and how to include it into your next course below!
Introduction to ITS
What is ITS?
The Intelligent Tutoring System (ITS) is an online program designed to teach basic clinical skills relevant to behavioral interventions and other structured psychotherapies. It was developed based on the ITS creators’ own original research, as well as previous research in the fields of learning science, educational technology, machine learning, and knowledge representation. Several studies show that when students are exposed to an intelligent tutoring system as part of their training, they perform as well as students who are trained by seasoned professionals and they demonstrate better problem-solving skills and higher self-efficacy in clinical situations.
The ITS utilizes a framework called “productive struggle”, which is a scaffolded learning approach that involves practice with clinical concepts and goes beyond passive reading, listening, or watching. In productive struggle, students are given novel tasks that are challenging; the tasks must be challenging enough that the student learns something new in the process, but not so difficult that the student becomes frustrated and discouraged. Recognizing that students learn at different rates, the ITS is able to adapt to each student’s pace of learning. The program gradually increases in difficulty as the student better understands the topics, keeping an effective level of productive struggle at all times.
Topics/Skills Covered
Each clinical skill is taught via an interactive, case-based online module. Module options include: therapeutic alliance skills (empathy, reflective listening, cultural responsiveness), session organizational skills (agenda setting, redirection), intervention skills (enhancing motivation, assigning homework, reviewing homework), and suicide risk skills (assessment and management of suicide risk). As an instructor, you have the freedom to choose the modules relevant to the course you are teaching.
See below for more details about each module offered.
Relationship Skills
Empathy
EMPATHY
Empathy is a core component of the therapeutic alliance. When patients express negative or distressing feelings or experiences, we want to communicate acceptance and concern both verbally and non-verbally. This skill is particularly relevant when we are reviewing distressing symptoms or problems in living, but empathy should be a component throughout the encounter
An empathic response includes the following components:
- Name, describe, or communicate that you understand the patient’s emotional experience
- Communicate concern for the patient’s feelings or experiences, or
- Express that a patient’s emotions or experiences are normal, expected or understandable
A poor demonstration of empathy is conveyed by the following elements of a response:
- Treat the patient’s feelings as insignificant
- Give advice on how to view the problem or what to do about it, or
- Communicate negative judgement about a patient’s feelings or behavior
Reflection
REFLECTION
Reflection is a core component of the therapeutic alliance. We restate what patients have said on those occasions where we want to highlight their experience, and in particular ensure their awareness of the link between distressing behavioral health symptoms or behavior and a particular life circumstance or problem in living.
A good reflection includes the following components:
- Repeat or restate the patient’s feeling or experience
- Repeat or restate life context issues, problems, or stressors, or
- Draw a connection between life circumstances and the patient’s behavioral health symptoms
A poor demonstration of reflection might contain the following elements:
- Misinterpret or make incorrect assumptions about the patient’s experience
- Refocus attention away from the patient’s emotional experience by asking concrete questions
- Refocus attention away from the patient’s emotional experience by probing past experiences or origins of the problem
Cultural Responsiveness
CULTURAL RESPONSIVENESS
Cultural responsiveness is a core component of therapeutic alliance. We want to interact with patients and deliver our interventions in a way that is responsive to and congruent with the patient’s cultural identity, context, and preferences. One aspect of cultural responsiveness is recognizing collectivist orientation and the important of family (and extended family) for many cultures in our conceptualization and treatment, in contrast to the individualistic culture of middle-class, White America.
A culturally responsive response may include the following components:
- Maintain a curiosity and openness to the aspects of cultural identity that are most important to the patient
- Have a knowledge base of a patient’s cultural identity and is responsive and sensitive to this identity
- Incorporate a strengths-based approach by reframing distressing emotions and experiences as opportunities for resilience and building positive coping skills
A response that is less culturally responsive may include the following components:
- Assume an overgeneralized (“one-size-fits-all”) understanding of culture
- Adopt an (over)reliance on the values of the surrounding dominate culture
- Interact with patient from a place of disrespect or ignorance, which includes minimizing the patient’s feelings or experience
Session Organizational Skills
Agenda Setting and Symptom Review
AGENDA SETTING
The purpose of agenda-setting at all encounters is to collaboratively structure how time and effort gets spent. Agenda-setting involves patient input in determining ahead of time how to both focus the intervention and ensure that patient’s priorities are addressed. Setting an agenda at the start of each encounter helps to manage time throughout the encounter. Agendas typically include behavioral health symptom review, homework review, and new goal setting and homework.
Elements of agenda setting include the following components:
- Set agenda collaboratively with the patient as a first topic of the meeting, or as clearly as possible
- Ask patient if they feel they have improved since the beginning of treatment, and
- Briefly review the symptom questionnaire or behavior checklist, and ask the patient which symptoms have bothered them the most
Less helpful ways to start a session in the absence of agenda setting include the following examples:
- Ask unfocused, open-ended questions about the past week
- Review symptoms or behavior checklists before setting an agenda, and
- Review homework before setting an agenda
Redirection
REDIRECTION
At times during our encounters with patients, we may need to redirect them back to the task at hand. We use redirection skills to ensure that patients focus productively on one issue at a time. Good time management starts with agenda-setting at the beginning of an encounter and is maintained by ongoing redirection and focus throughout the encounter.
A helpful redirection will include the following elements:
- Reflect or briefly summarize the patient’s concern
- Talk about the connection, or lack or connection, between the patient’s new concern and the task at hand, and
- Suggest returning to the task at hand, or prioritizing the current concern as more important
Qualities of responses that do not redirect the patient include:
- Ask follow-up questions about the new concern
- Let the patient keep talking without interrupting them, or
- Ignore the patient’s concern and abruptly return to the task at hand
Interventions
Enhancing Motivation
ENHANCING MOTIVATION
Patients are often ambivalent about making changes in their lives. We use a variety of strategies to enhance patient motivation and “elicit change talk” to promote positive behavior change. These strategies are particularly helpful before assigning homework in order to ensure patient readiness and agreement.
Strategies that can enhance patient motivation include:
- Ask permission to provide psychoeducation or feedback about patient’s behavioral health symptoms
- Discuss the treatment rationale or ensure patient input in treatment
- Ask questions about the pros and cons of behavior from the patient’s perspective
- Ask questions about patient readiness for change
Ineffective strategies to attempt to enhance patient motivation include:
- Decide on treatment course without asking for patient input or preferences
- State the benefits of changing a behavior or the negatives of maintaining a behavior only from a paternalistic perspective, and
- Drop the focus on a distressing symptom or behavior and move on prematurely to a new concern
Assigning Homework
ASSIGNING HOMEWORK
After providing adequate psychoeducation to the patient about their behavioral health symptoms and about the intervention, we then need to structure the encounter in a way that will be most beneficial to helping the patient learn new skills, all while staying within our time limit. To do this, we focus on setting specific goals, teaching and illustrating therapeutic skills, detailing homework, and anticipating obstacles.
To adequately assign homework, we should work with patients/clients to sequentially:
- Set specific goals and action plans in collaboration with the patient
- Ensure the patient will be successful by predicting obstacles and planning realistic strategies to overcome them, and
- Explain how to evaluate whether the plan was successful or not
Less helpful ways to assign homework include:
- Identify a goal or activity without creating a specific action plan that covers the necessary steps the patient will need to do so, or
- Set an unrealistic goal or action plan
Reviewing Homework
REVIEWING HOMEWORK
Patients will be more likely to complete their action plans if the provider reviews them at the start of each encounter, discusses the patient’s satisfaction with their efforts, and troubleshoots any obstacles. This discussion of successes helps patients acknowledge and feel good about their accomplishments, even if they are still not functioning as well as they were since their behavioral health symptoms began. Reviewing homework also reinforces for the patient the link between their efforts and improvements in behavioral health symptoms and functioning.
A thorough homework review will do the following sequence:
- Ask the patient what they have accomplished, or how satisfied they were with themselves and their efforts
- Give appropriate praise for patient efforts
- Emphasize the relationship between the patient’s efforts and their symptoms improving
- Ask patient about challenges with the previous homework or activities, and
- Help the patient consider what they could do differently next time
An ineffective review of homework will:
- Ask open-ended questions about the past week
- Move on too quickly to schedule new plans without addressing any difficulties from last homework
- Encourage patient to drop an activity if they didn’t accomplish, or
- Encourage patient to redo homework that they didn’t accomplish without addressing any difficulties
Assessing and Managing Suicide Risk
Assessing Suicide Risk
ASSESSING SUICIDE RISK
When patients endorse item 9 on the PHQ, or when they report any thoughts of death or suicide, the provider must follow up with a series of questions around thoughts patients may be having in the past month. Risk levels range from:
- None
- Mild (wishes to be dead)
- Moderate (thoughts of suicide without a specific, detailed plan or intention; or currently wishes to be dead with lifetime history of a suicide attempt)
- High (specific, detailed plan or intention to end one’s life; or any suicidal behavior in the past 3 months).
A thorough and complete assessment of suicidal ideation involves inquiring into each of the below areas in sequential order:
- Ask about wishes to be dead in the past month
- Ask about thoughts of suicide or ending one’s life in the past month
- If reports of any thoughts of suicide in the past month, ask about:
- Specific detailed plan and intention
- Suicidal behavior or preparation (over patient’s lifetime and also specifically in the past 3 months)
- Context and stresses
- Reasons for living and protective factors
- Accurately categorize the suicide risk level of the patient
Ineffective or poor strategies to assess suicide risk include:
- Skip ahead, or move on prematurely
- Minimize patient feelings or experiences
- Convey disapproval or negative judgement about patient experience
- Ask questions that lead patient toward denying suicidal thoughts
- Do not inquire specifically about suicidal thoughts in the past month
- Do not ask about suicidal or preparation for both lifetime and past 3 months
- State that will seek out supervisor without completing full suicide risk assessment
Managing Mild to Moderate Suicide Risk
MANAGING MILD TO MODERATE SUICIDE RISK
Providers must take the appropriate actions after they determine suicide risk level to be mild to moderate
A thorough and complete action plan involves addressing each of the below areas:
- Convey that provider will inform Behavioral Health supervisor or PCP to discuss treatment
- State follow up appointment date and importance of regular sessions and ongoing monitoring of risk level
- Help patient identify self-management strategies in response to suicidal thoughts
- Help patient identify family or friends for support, including possible participation in conjoint sessions
- Provide care team numbers, hotline, 911, and 988 if thoughts become more severe
- Discuss importance of keeping environment safe by limiting access to firearms and medications
Ineffective or poor strategies to manage mild to moderate suicide risk include:
- Fail to communicate action plan or reasons for action plan to patient
- Call family member without patient permission
- Convey that patient is at high risk for suicide and will arrange emergency services
Managing Severe Suicide Risk
MANAGING SEVERE SUICIDE RISK
Providers must take the appropriate actions after they determine suicide risk level to be high.
A thorough and complete action plan involves addressing each of the below areas:
- Communicate care and concern
- Convey that provider will consult with the Behavioral Health supervisor or PCP, or arrange emergency services to ensure patient safety
- Convey that a care team member will remain with the patient until emergency services arrive
Ineffective or poor strategies to manage severe suicide risk include:
- Convey disapproval or disrespect toward the patient
- Communicate unclear emergency services plan to patient
- Recommend any other action than immediately arranging consultation from the Behavioral Health supervisor or PCP, or emergency services
Navigating ITS
- Close: Close Current Module (only for non-Canvas/standalone version)
- Module Info: Get description of current module
- Progress Bar: Percentage progress until module completion
- Report Bug/Issue: Click to report issues like technical bugs, typos, or confusion with the question content.
- Help: Click to review the student help and FAQ page
- Question Area: Typically will have a scenario and question based on that scenario
- Answer Input Area: Typically will have multiple choice options that you must select from. Some questions require typing in a response.
- Review: If you submit an incorrect response, click the ‘Review’ button to get more detailed feedback on why a selected response was incorrect and/or why an unselected response should have been selected
- Submit: Click to submit an answer and continue to the next question
Watch the following video for an overview of the user experience of ITS
Try it out for yourself
Use this link to freely test all the available modules in ITS. We recommend you go through the question sets for each module to determine whether it is appropriate for your course. In the drop down that says Choose Skill you can change the current module. Select the drop down that says Choose Problem Type to quickly skip to different question types, otherwise it will progress in the same sequence that a student will see.
Use this link to view sample data in the Instructor Dashboard. Email the ITS team for the credentials to login. We recommend that you review the Instructor Dashboard section below first to get a better idea of how the dashboard works.
Using it in your course
Canvas (recommended)
The most convenient method to access ITS is via Canvas, a popular Learning Management System used by many institutions.
Currently we are integrated with the following institutions:
- University of Washington
- Eastern Washington University
- Lake Washington Institute of Technology
If your institution is not on the list, please contact the ITS team and we will work on integrating it. To use ITS in Canvas, simply create an assignment that uses an External Tool. Then pick the skill you want the assignment to use. Once the assignment is created, students will be able to access the question set tied to that skill. Follow along in the video below to see the fill progress.
Standalone (without Canvas)
If your institution does not provide Canvas or a similar online course management system, or your course simply does not use such a system, we also offer a standalone option for using ITS.
When using the standalone version, each of your students, as well as yourself, will need to create an ITS account. View the links below to see the setup instructions:
Incorporating into Curriculum
We have found, through previous instructors and student feedback, that ITS has higher levels of satisfaction and engagement when it is more closely integrated with the course material. A good initial step is to simply schedule the module assignment at the same time you have a lecture similar to the module’s topic. For example, if you have a lecture related to Empathy, have an ITS assignment on Empathy be opened on the same day.
When using the Assessing Suicide Risk module, we recommend that you review this document first. It goes over the standards and practices that the module content is based on and might vary with your curriculum content. If you are able to spare more time, we encourage you to use the information instructor dashboard as starting points for discussions either during class or via email. (See the section on the Instructor Dashboard below to gain deeper insight on the type of information it presents). Here are some examples of how to use the dashboard:
- If several cards have the same skill component, discuss why that skill is challenging to interpret
- Go over an example problem that many students had issues with in class
On your course webpage, when introducing ITS you can include this file that gives a brief overview of ITS.
Instructor Dashboard
What is it?
The instructor dashboard is a key component of the ITS application as it provides you feedback on the most important aspects of your students’ performance for a module. It provides general metrics like how many questions each student answered, whether they were able to complete the module, and how much time they spent. More importantly it presents aggregate information about common mistakes or confusions that students encountered.
Watch this video for a walkthrough of the dashboard:
Navigating the Dashboard
- Module Selector: The module to view student performance on
- Report Card Type: Change the category of mistake/misunderstanding that is currently displayed as report cards. The types are Common mistakes by question types and Common confusions between components
- Hide Names: Check to anonymize the student names that are displayed (useful when sharing your screen with the class). Uncheck to view the real student names.
- Settings:
- Refresh Results: Manually reload student result data
- Card Settings: Adjust thresholds for cards
- Act as Student: Change to student viewpoint to view the module questions
- Open Tour: View a guided tutorial on elements of the dashboard
- Create Student Link: If using the standalone/non-canvas version of ITS, this creates a link to ITS that can be included in emails or course website
- Light Mode: Toggle background color between light and dark
- Report Cards: Collection of cards where each one is a summary of a common mistake or confusion
- Student Statistics Table: View metrics for individual students
- Click the down arrow to view full question response history
- Time student spent on module with idle time pruned out
- Number of questions attempted
- Percentage of the module completed
FAQ
How am I supposed to interpret the information in the dashboard?
Click the Show Summary Report button to get an overview of the student results. It will also provide an example of how to interpret the report card. Additionally, you can walk through the tutorial tour that appears the first time you enter the dashboard.
Why is nothing showing up on the dashboard?
Generally, this means not enough students have started working on the module. You may also want to go into Settings – Card Settings and lower the threshold for display of cards.
How is the active time per student calculated?
The active time is an estimate of the time that the student is engaged and interacting with the assignment. Idle time where the student has clicked on the screen or moved their mouse after a short while is trimmed out.