JUNE 17, 2010 VCU WEBCAST Services Provided By: Caption First, Inc. >> GARY BOND: Hello. My name is Gary Bond. I am a researcher at the Dartmouth psychiatric research center, and professor in the department of psychiatry at Dartmouth medical center. I'm going to tell you about a study that Marina Kukla and I did while I was on the faculty at Indiana university, Purdue University, Indianapolis. This was a study that was funded by the Virginia Commonwealth University research, Rehabilitation Research and Training Center, workplace supports and job retention headed by Paul Wehman with funding from the National Institute of Disability and Rehabilitation Research. As you can see here, the title of this presentation is the impact of follow along support on job tenure and evidence- based supported employment. I want to start by thanking VCU and in particular Paul Wehman for giving us the opportunity to do what is, I think, a rather interesting and unique study looking at one aspect of evidence-based supported employment that has been neglected, namely, the whole process of follow along after a client with severe mental illness gets a job. It was really Paul Wehman who back in the '80s took the lead in helping the vocational rehabilitation field think about the importance of long-term supports, instead of only thinking about the issue of helping people get jobs, but rather what happens after they get the jobs, an individual gets a job. So that is the focus of this presentation. I'm going to be talking about this study in a moment. But before starting, my outline includes some background on evidence-based supported employment, a model that we call individual placement support. I'm going to say a few things about what we know about evidence-based supported employments and some things that we don't know. And then the bulk of the presentation is devoted to the current study, which is a longitudinal study of clients with severe mental illness who have obtained their first competitive job after being enrolled in evidence-based supported employment. The program model I'm going to be talking about is called individual placement and support, or the initials IPS, acronym IPS, which I use throughout the presentation. This is a model that was developed right here in Dartmouth by Bob Strake and Debbie Becker. It is a model of supported employment that is intended and aimed for people with severe persistent mental illness such as schizophrenia. IPS has been established as an evidence-based practice based on a rather large body of research which I will only briefly touch on today. But I'm going to use the two terms, IPS and evidence- based supported employment, interchangeably in this talk. The IPS model is based on 7 principles, the first of which is anybody who is interested in working is eligible for IPS. This is the zero exclusion principle that Paul Wehman actually talked about in the 1980s and it is one of the cornerstones of the IPS approach, and it includes the fact that people who may have very severe psychiatric symptoms are able to work, people with pro occurring substance abuse disorders are capable of working, that there are no clinical factors or personal factors that would exclude somebody from working, and this is both a value statement about IPS, but it's also an empirically based finding that IPS seems to benefit people across the spectrum. The second principle of IPS is that the model focuses on competitive employment. So in IPS, there isn't an intermediate step of a sheltered workshop or a temporary job or anything of that nature. The main focus, the exclusive focus in most cases is on helping an individual find a job in the open labor market, taking jobs that anybody could work, working alongside people without disabilities, working, being paid the commensurate wage rate. A third principle is that of rapid job search, which says that once a client enrolls in an IPS program, the employment specialist, that is the term we use for people who work in IPS programs, begins within the first month to first identify what it is a client would like to do in terms of employment, and then start working towards developing jobs, finding jobs that are congruent with the client's aspirations and strengths, prior experiences, sometimes working with clients together, going to job sites, sometimes working behind the scenes. But the rapid job search principle says that there is not a requirement for skills training or extensive job preparations that precede the search for the job. The fourth principle of IPS is that clients are the primary focus of the decision process in terms of preferences for what kinds of jobs, where they work, what time of day, as well as the client's preferences are important for how they receive help, whether they disclose to the employer when going out for job interviews, whether the employment specialist meets with them in community settings or at the agency or at the workplace. So client preferences are critical within this model, and I would add within evidence-based practices in general. The fifth principle which is in fact the focus of this presentation is that clients are helped over the long term with individualized job supports, and this is an area that has long been part of the model, and I should add part of the supported employment in general, but it has not been very well described in the research literature. The next principle of IPS is that services are integrated with mental health treatment, which means that the employment specialist is going to treatment team meetings, meeting with the psychiatrist if appropriate to make sure that the medications are suitable for the individual who is going out on the job, so for example, the medications are not overly sedating, and working very closely with case managers and other treatment teams and staff, to make sure that everybody is working together towards the goal of helping people in their recovery process and in particular in the process of getting and keeping jobs. The treatment team in fact is very much involved in providing ongoing support after an individual gets a job. I'm not going to be talking much about that aspect today, and that was not part of our study. The final critical ingredient of the IPS supported employment model is careful attention to benefits planning, helping clients understand what taking a job will mean in terms of the Social Security and other benefits. And so the benefits counseling aspect is very important. We know that fear of losing benefits is one of the main worries that clients have that leads many to avoid seeking out work. So the benefits counseling piece is very important as well. So that very briefly is an overview of the IPS model. Let's start by asking what we know about IPS. Overwhelming findings from numerous studies is that IPS has better competitive employment outcomes than a whole range of other vocational approaches. In fact, there are no other vocational approaches that have anywhere near the evidence supporting the success that IPS has. This means that overall, more clients involved in IPS get competitive jobs. If you look at the monthly employment rates for IPS clients, compared to clients in other kinds of programs, the rate is consistently higher. We also know that IPS clients are successful in getting jobs much sooner than those in other kinds of vocational programs. Finally, and this is a very important and recent finding, we also know that looking in the long term, ten years or more, clients who have obtained help through an IPS program often become steady workers, meaning that they work consistently over that time period defined as at least 50 percent of the time. So this is a really crucial finding to understand the impact of IPS, that the outcomes from IPS are not short term, but they help clients in the recovery process start on a life-long trajectory, lifelong career pattern of working, in at least half of the cases. So just to give you a little thumbnail sketch of some of the research that is suggested by that last slide, I'm going to show you some graphs. This first graph shows the results from eleven randomized control trials of individual placement support. Randomized control trials of course, the gold standard for research in any kind of interventions field. There are now eleven studies of IPS where IPS has been measured to be high fidelity which means that it has been well implemented and implemented according to those principles I just outlined, and these eleven studies were done throughout the U.S. and abroad in New Hampshire, Illinois, Connecticut, Hong Kong, Australia, South Carolina, District of Columbia, California, Europe, Quebec and Maryland. In this graph, we see the employment rates for clients enrolled in IPS in the black bar, and the employment rate for the control group in red or in orange also. The control group receives a variety of vocational services ranging from self- help programs, career counseling programs, field training programs. There were a number of different program models including brokered supported employment, brokered approaches meaning that the supported employment services were not located within the mental health center, and having the feature of integration with treatment. So, this slide shows that the employment rate is more than double in every program and every study, the employment rate is more than double for IPS compared to the control groups. And overall, the average employment rate in these eleven studies was 60 percent for IPS and about 23 percent for the control groups. So this slide shows that IPS clients have a higher probability of getting a job than clients receiving other types of vocational services, even services that are highly regarded. This next slide looks at monthly competitive employment rates In one of the eleven studies conducted by Mueser. Here you see the monthly competitive employment rates, the pink lines, is hovering around 30 percent after three or four months, 30 to 35 percent, compared to the two comparison groups which include a supported employment approach that didn't follow all of those evidence-based principles I outlined, and a rehabilitation program using the (inaudible) ideas. So the monthly rates are much higher for IPS clients as well. This third data display looks at the days to first competitive job and several of these studies I mentioned earlier, and here you can see that the average time to the first competitive job is about 20 weeks in IPS, compared to 260 days or nine months for the comparison groups. This also suggests that time to the first job is more rapid for IPS. Fourthly, I mentioned that one of the larger goals of IPS is to help clients establish an identity as a worker and to become steady workers over the long term, and we now have two small follow-up studies looking at IPS clients ten years after they left or they first enrolled in an IPS program. And from this we can see that over half were able to achieve this steady worker pattern. So this is a key concept that I want to point out here, because I'm going to come back to it, I would propose it's not how long you work in a single job, in other words sometimes called job tenure. What is more important in terms of the recovery process for people with mental illness, and I would say for workers in general, is the proportion of time working overall. So we see that IPS has a deservedly wonderful reputation for helping people get jobs. And it has enjoyed phenomenal success. The results have been replicated many times. That being said, there are some things we don't know about IPS. I'm going to concentrate on three interconnected research questions that we really have not fully answered. There are other research questions of course, but these are three key questions. They are as follows. First of all, once a client starts the job, after the program helps an individual get a job, what happens next in terms of employment. What is the employment pattern? How long do they work in that initial job? Secondly, what are the typical patterns of employment specialist support. What does the IPS program do? We talk about individualized support, but what does that really mean? What does that look like. Thirdly, what is the relationship between the support provided by the IPS program, and job tenure. So does it matter how many times a month an employment specialist in contact with the client, so these are three questions that are not fully answered in the literature. So to elaborate on each of these points, after the first job, supported employment, job tenure, then what happens, let's say a client loses a job. A common criticism of IPS has been in fact that job is short term, and in one study, one review, we found that the months worked longest job is about five months. These are studies that add a short term follow-up, they were 18 months, two years, studies in terms of follow-ups. So these were not the best evidence for months worked in longest jobs. But five months does I think correctly seem to be somewhat short term. Okay. So in interpreting that short-term finding, if that is a fair characterization, there are a couple things you can say. First of all, in the IPS model, losing a job is not conceptualized as a failure. It is seen as a transition. In fact, it's normative. It is expected that clients will lose jobs or quit jobs. And that early in the employment process, one will discover things about themselves and perhaps find their job doesn't work out, even though the employment specialist and the client together have worked hard to make a good job match. As I mentioned earlier, it is not so much the tenure in any single job, but it's the cumulative duration of employment, the total time worked across all jobs within a time period which we think is the most valued indicator. So multiple jobs are common within IPS programs, and they are acceptable. They are seen as part of the process. In terms of the typical pattern of ongoing support, there are a couple issues here. One is, how much is the right amount of support, and where should the support be provided? Should it be in the community? If you look at the standards for IPS, there is a strong emphasis on community contact. When does the support end? In the IPS fidelity scale, the standard says that employment specialists should help clients through monthly support for a year or more after they found a job that they are satisfied with. And at that point, clients are stepped down, that is transitioned to support through other sources, through natural supports and/or through the mental health treatment team. But these standards are based on clinical impressions. They are not based on empirical literature. When does the support end? And a kind of related question that hasn't been well answered is, when is the appropriate time to close a client from IPS programs? The third question had to do with the relationship between amount of support, and job retention. There are very few studies that have been published over the last 20 years, to that I might briefly mention, one was done by C Leff, who found that number of employer contacts was positively related to helping clients obtain new jobs. So that's the front end of the equation. We know that working with employers is critical to job placement. In the Leff study, the group was unable to find any relationships between the number of follow-on contacts and length of time in jobs. So the Leff study concluded that intensity of job support and employment specialist support was unrelated to job tenure. Another study by Alan McGuire did find a positive relationship between number of contacts by the IPS team and length of time in the job or job tenure. So the literature is at the present time inconclusive and with mixed results. The current study is a longitudinal study looking at the relationship of employment specialist contact to duration of employment after the first job starts. Okay. So let me outline the methods and then give an overview, a brief description of the results of the study, and say a little bit about the implications. This was a prospective observational design. In other words, we started tracking clients at a particular point, namely when they had obtained a competitive job. And so we started by identifying IPS programs that were well implemented programs. We determined which program to choose based upon a fidelity scale. Fidelity is measuring how well a program meets the standards of the practice being studied. And we have a fidelity scale that has been well validated. So we used that information to choose our size. I said it was a prospective study which means that clients are recruited at a particular point right after they obtain competitive work. So we didn't, as many studies have done, recruit people when they entered IPS, but rather, after they obtained their first job. We had a two-year follow-up period, so clients were tracked monthly during this time. To collect the data, we asked the employment specialist to record on a monthly basis the number of service contacts that they made, and the employment states of the clients in each month. We found four agencies in the Midwest. All four agencies were located in cities, large and small cities. The agencies had all previously participated in various studies that we had done. As I said, they were all assessed on the IPS fidelity scale. Three of the types were rated as very high fidelity. One of the types was slightly below the standard for high fidelity. But we had a good relationship with these sites, so we went ahead and included them in the study. I should mention that in this particular study, the level of fidelity was not associated with employment outcomes. One site rated moderate fidelity did have the lowest outcomes of the four sites. In terms of data collection, we used a web-based survey, at three of the sites. At one site, the data requested through paper and pencil. We knew that this was an additional burden on the employment specialists, and so the 35 employment specialists who participated in the study were given a small token for helping out, $5 per month. Marina Kukla, who did the study with me, as I mentioned earlier, was wonderful about contacting the sites and keeping employment specialists energized to complete the data. And she was very successful and the sites were very successful in terms of completion rates. You can see there was a 98 percent completion rate overall for clients while they were still enrolled in the IPS programs. So this was a pretty positive data collection rate. Okay. In terms of the samples, there were 142 overall, and 70 percent of the clients remained in IPS for the entire 24- month period. And you can see here that the retention rate in IPS services varied greatly. At site 1, over 90 percent remained active for the entire two-year per. Site 1 has a very strong community treatment team model throughout the agency and the peer community treatment model emphasizes long- term support. So I think that is the reason for the high rate of retention at site 1. In the other three sites, you had varying rates of retention over the two-year period. That in itself leads to a rather interesting question: How long does a client remain active at a site? We think that this may be in part related to the policies of the agencies which may in turn have to do with the funding. Anyway, that's an issue that cropped up in this study, that we really didn't fully explore in this particular study. So, the basic sample, 142 clients, we had substantial data on all of those 142, with the exception of a handful who dropped out within the first six months. Then we had a sample of 99 that we had 24-month data for, and that second subset was used for some of our analyses. Okay. In terms of sample characteristics, you can see that this was a fairly typical sample of a study of clients receiving mental health services in public mental health, evenly divided between men and women, mean age, 40, about half the sample had schizophrenia, schizo-affective disorder. The majority of the clients had at least a high school diploma. Most of the clients were receiving Social Security. And over half had worked at competitive jobs prior to their enrollment in IPS. But 40 percent had not. Okay. So, moving now to the results. The first research question had to do with the employment outcomes. What are the patterns after they obtain their first job? First of all, to give you a thumbnail sketch of what kinds of jobs people were getting, the average wage was almost $8 an hour, which was a couple dollars above minimum wage. On average, clients worked about 23 hours a week, so that again, both of these are fairly typical for supported employment studies. In terms of the kinds of jobs they obtained, if you look at the specific jobs, titles and look at the details of the job, there really were a wide variety of kinds of jobs. But in terms of broad categories we see that food service was by a large margin the most common type of job. Retail and janitorial were rated, were second and third in terms of percentage of clients, followed by professional. So in our sample, we found that 15 percent of clients two years later were working in the same job that they were when they enrolled in the study. And these clients were still active in the IPS programs. Another 34 percent worked the same job, but they were not active at two years. So for this group, they held one job, and they didn't obtain a second job in the two-year period. And most of these, not all of these actually were not active. They had been closed before the two-year period. So that leaves almost half, over half who had worked two or more jobs during the two-year period. So, to answer the question, what happens after the first jobs, the answer would be that there are a variety of patterns, but for about half, the pattern is to have at least one other job within a two- year period. Okay. So this slide looks at the months worked before termination by IPS. And that first looked at the rate of termination. There were four clients who were, dropped out within six months after obtaining that first job. There were another 15 who were closed from the IPS program within 12 months, 13 within 18 months, eleven between 18 and 24 months. In terms of months worked, we have data only for clients while they were active in the IPS program. So understandably, we can see that the months worked overall climbed, depending how long they were active in the program. Interestingly enough, if you look at the clients who were closed within 24 months, between 18 and 24 months, the eleven clients in that category, the average time in the first job which is the light blue line, was substantially higher than for those who were active for, at 24 months. This graph also suggests that in terms of months worked, most of the time was spent in the first job. So, tenure in the first job, total duration of employment for the total sample, we see that the average overall is about ten months for the first job. Earlier I mentioned that in the literature, the figure for time in longest job was about five months. I think this figure is a better statistic to quote, because it's based on a longer follow-up period than that earlier set of studies. So my answer now to people asking how long do people work in IPS programs, would be that on average, about ten months. Of those who had more than two jobs, there was about three months in between the first and second jobs. That is a shorter time than time to the first job. So I think that figure of 2.9 months between first and second jobs is very encouraging, and something we tell clients that if they lose a job, that their prospects of getting another job in a relatively brief period of time is pretty good. Overall, clients work an average of 12.9 months in all jobs over the two-year period. There were site differences in this four sites in months competitively employed overall during follow-up. I would point out that site D had a phenomenal rate, nearly 18 months on average months competitively employed for clients in the programs. So in terms of having a high standard to meet, I think site D is really an impressive site. Let's go on now and take a look at the patterns of employment specialist support. So what is the frequency of contact, where is the contact, and how is the contact conducted? Here is the first big picture snapshot of contact, and the contact here includes both face-to-face and telephone contact. I should mention here that nearly all of the contact over, well, I should say 80 percent of the contact was face-to-face, rather than by telephone. And what this graph says is that in the first month after obtaining a job, the number of contacts average about three. And it tapers off rather dramatically over the next six months. And this fits with what common sense would say as well as IPS standards, which is that you would have intense contact around the time of a job start, and that as the client, acclimates to the job, you can taper off the support. Some of the ongoing support might be provided through family members and through the employer. But what is also important to note is that over the two-year period, the contact doesn't taper off to zero. It stays about one contact per month. So that is an intriguing finding, and an important finding. We looked at site differences. And here, there were some differences. The pink line is site 3. It had higher rate of contacts. Site 2, which you recall had the, you may recall had the lowest employment outcomes, tapered off to a low, the lowest rate of the four sites. We found that the mean number of contacts for clients who remained active, and the only count of those rates of contact were the rates at when the clients were active in the program, we see that there were more contacts for active clients than for dropouts which is not surprising. And whether or not it means that more active intervention with dropouts would have prevented the dropout, that would be speculation. In terms of the number of jobs held, this slide is harder to interpret. But it says that for someone who has a single job for the entire two-year period, the rate of contact is about the same as if, as for clients who had two or more jobs. So in the case of multiple jobs clients, you would expect there would be an increase of employer, employment specialist contact before and after obtaining the second job. We do see that the client who had one job loses it and doesn't get another one, the rate of contact drops very dramatically. Whether that is an opportunity to intervene, again, this would be speculation. Face-to-face contact, versus telephone contact, we see that the lines are pretty parallel. So it is not the case that after a client stabilizes in the job, that the employment specialist switches to telephone contact. By and large that is not true. The face-to-face contact remains the main mode of contact. Now, this slide on locus of contact is a little bit a surprise to me. What it says is that job site contacts were fairly high in the first six months, so around the time a person gets the job and is acclimating to the job, the employment specialist may be at the job site, if the client chooses to disclose. Those that did not, there were no contact at the job site. But after the first six months, there were practically no job site contacts. For the entire two-year period, the rate of contact in the community and in the agencies were about equal, slightly more for agency office contacts. We hope that this agency office contacts are a reflection of client preferences, and that would be something to investigate and further research. Let's move on to the third question, and that is: Does frequency of employment specialist contact make any difference? Is there any association between the intensity of contact and employment outcomes? Well, first let's look at site level data, and we only have four sites, and so we really can't draw any conclusion. What we find is that the two sites with the lowest employment rates also had the lowest rate of monthly contact. The site with the best outcomes did not have the highest frequency of contact. But there is a hint here in the data that frequency of contact averages at site level may be associated with duration of employment. It is really inconclusive would be the more statistically defensible answer. Now, in terms of frequency of contact with duration of employment, we limited this analysis to active clients. If we would have included the dropouts, it would have inflated the correlations because we didn't have contacts with dropouts after they dropped out. So looking at duration of employment in the first job only, we see that the correlations are not significant, except for job site contacts. So in other words, the employment specialist who had contact at the job site, and we can speculate that this included contact with the supervisor, facilitated longer job tenure for clients in that job. The correlation .21, .2 is not a huge difference, but there is a definite signal that something is going on there. Our second indicator was total number of months worked in any job over the two-year period. And as I have indicated early on, this is the indicator which I think is the most critical. Here there was a significant correlation between number of contacts and total months worked of .27, which is a fairly small correlation, but still is a clear signal that there is an association between frequency of contact and duration of employment. So based on that, I would advocate for continuing contact with clients in the interest of fostering a longer time on the job. Of the types of contact, face-to-face contact was statistically significant. And in terms of location, we find that contact at the agency office was correlated with total months of worked, whereas contacts in the community or the job site were not. Okay. Just to summarize, first of all, one nugget that I think is important is that the time worked at the first job in this study averaged ten months, which is higher, greater than what has been reported in the literature. So that I think is an important finding that needs to be replicated. But it does address some of the concerns that people have that IPS gets people short-term jobs. The second finding I would highlight is that the norm was about one contact per month, once the client stabilized in the job. So in some states, this is the expectation. It may be built into the policies. So maybe we are simply replicating what the policy, but nonetheless, that one contact a month was a pretty striking finding. A third finding which was a little bit of a surprise to me, was that over two-thirds of the clients received IPS contacts for a two-year period, which is longer than you would think from the IPS fidelity scale standard. So if this is replicated, it suggests to me that we need to take a look at our IPS fidelity standards. And the final finding highlighted here is that among the clients who lost a job, two-thirds of them continue to receive support after they lost that initial job, and many of them as we are seeing went on to get a second job, which is consistent with the IPS model to help people get not just a single job, but to help them with employment as a life choice. In terms of the site-level findings, we did see differences in both employment outcomes and frequency of IPS contacts, and also the point in time when clients typically were closed. So, the study was too small to really allow to say much about site differences, but we know from areas of healthcare that variations within sites often suggest that some sites are implementing the fastest with greater quality than others. So I think we need to look at site variations on a larger scale. In terms of correlational findings, we did find that frequency of IPS contacts predicted cumulative duration of employment across all jobs, which was the main point of the study. So the finding is encouraging. It should be studied further. And we found some intriguing sub findings regarding job site contact and face-to-face contact, which we don't know will replicate in future studies. I should mention that there are study limitations. We did study successful clients who obtained the first job, so the study did not examine if 30 to 40 percent of clients who enroll in IPS who often don't get a job at all. And another limitation was that we gave incentive to employment specialists to track clients' behavior, and their own clients' employment outcomes, and their own contacts with clients. And those payments may have influenced their behavior by keeping clients open longer, for example. A third limitation was we didn't track other types of service contacts, most notably the treatment team. A fourth limitation is that our method of data collection did not permit us easily to look at the frequency of contact around job start, for second and third jobs. That would have been an interesting thing to look at and needs to be looked at. And finally, our study looked at the simple fact of contact, and not what actually went on in the service contact, another important thing to study in future research. So, to summarize, I would leave you with three take-home messages. That long-term contact by the employment specialist is the norm in high fidelity supported employment programs. That this ongoing IPS contact does seem to be correlated with better employment outcomes, and that the intensity of support, we don't know what the optimal goal should be but monthly face-to-face check-ins may be sufficient, at least in this study that was the norm. And finally, I would indicate the importance of ongoing contact and outreach as a way to promote better employment outcomes. The data suggest that retention in IPS services may be especially important for clients who lose an initial job, rather than closing them out when clients become discouraged when the first job doesn't work out. It seems to me that it's important in many cases to encourage clients to persevere, and finally, I do believe it's important that IPS staff, increase their contacts with clients at the time of job loss and job starts. I thank you and I hope this generates some interesting discussion. ******