VIRGINIA COMMONWEALTH UNIVERSITY February 22, 2007 Webcast on Indicator 14 Captioning Provided By: Caption First, Inc. >> ELIZABETH GETZEL: Good afternoon. We will be talking to a federally required indicator, indicator number 14 that all states are required to submit data on concerning secondary education exiters. I will describe later in this training who we are targeting as school leavers or exiters. This is a Web-based form and the training will focus on this form that has been developed with great deal of input from stakeholders across the state. We have gone to small school divisions, large school divisions, schools have provided input as well as community agency personnel and family members. We will be looking at how to complete this form and how to conduct interviews so we have consistency across the state in gathering information. We ensured that we looked at what other states were developing as well as a national center funded by the federal government so that Virginia can align with what the federal government is requiring in terms of submitting post-school outcome to them. Indicator 14 reads it is the percent of youths who had IEPs, are no longer in secondary school and who have been competitively employed, enrolled in some type of postsecondary school or both within one year of leaving high school. What's important to note about the indicator is the percent of youths that had IEP, they are no longer in secondary school. That means they have separated completely from secondary education and we are looking at what they have been doing in the community either competitively employed, enrolled in some type of postsecondary education or both within one year of leaving school. We will be collecting data, information. You will be conducting telephone interviews on school leavers who exited during the 2005-2006 school year. We have a particular time line who has been given to us by the federal government that the data needs to be collected between April and August of this year 2007 inclusive. So we will be preparing you to begin these interviews by April 1st. This is one of a series of information that you will be receiving from myself or from the Department of Ed concerning the post-school outcome indicator. You will be looking at getting a CD of this particular training so that you have it on hand. Off-school divisions will receive multiple copies of the CD. We will be posting this training and all information that's forthcoming, frequently asked questions. We will be giving you a script that you are welcome to follow when conducting the interviews. All that information will be on the RTC Web site, VCU -- excuse me, the TTAC online Web site and we will also be on the Virginia Department of Education Web site. We want to make sure that you feel that you are getting the complete information so that you can successfully conduct these interviews. The survey is designed for follow-up contact by telephone. I can't emphasize enough how important it is that when conducting the interviews as much as possible, that it is done on the Web and you are on the telephone using it. You will see later the design of the form is very straightforward. Why we emphasize that we are strongly encouraging you to complete the form during the interview is that we cannot afford to have a number of missing data or missing items on the forms. Our concern is that by completing it by paper, pencil and then putting it on the Web form that during the interview you may inadvertently skip a question, that someone else may be entering it for you and they can't read your handwriting if they can't read your comments. We are strongly encouraging school personnel who will be conducting these interviews to please use the Web-based form. That way we can guarantee that all accurate information is being entered. The survey is developed so that once you complete a section, it automatically skips you to the next question that needs to be asked. If you're doing it by paper and pencil, I'm afraid that you will get involved in trying to flip the pages to find out what is the next appropriate question, which could lead you to missing a whole section. So, again, I can't emphasize enough that we really would like you to use the Web-based form while conducting the interview. We are looking to primarily conduct interviews with students, if that is possible, and feasible, with former students, I should say, school leavers or exiters or a designee. We will talk a little bit more about who a student designee might be. I want to emphasize who leavers are, who we talk about are exiting schools. You will be interviewing students who have earned an advanced studies diploma, a standard diploma, modify standard diplomas, special diplomas, certificate of program completion. Perhaps they have completed a GED or a general achievement diploma or they have dropped out or they have aged out of eligibility for free appropriate public education. So we are not looking at individual who is have just earned a diploma or who have graduated. School leavers or school exiters are all these individuals in these particular categories. Again, the Web-based survey that we will be talking about today is designed to enable you to complete it while talking with the individual or their designee. Information will be stored on my Web site here at VCU. School provisions will be provided a passcode and no one else in any other school division will be able to access data except their own information. We have a very secured Web site. We are used to doing multiple survey, data entry programs and so rest assured that the information on students or individuals that you are gathering will be secured. However, you will be able to access your own particular data at any time so you can print out information and get a sense of how many students are being interviewed or their designee as well as looking at how your data is looking in terms of post-school outcomes at any point in time during the process. You have two of my phone contact numbers. If you experience any problem while entering data, we hope that that's not the case, but we are doing a Web-based survey so sometimes things do happen. However, if there is any problem at all, please contact me immediately. Also, you will get information with my email address. I will give it to you now. However, it will be available. It's lgetzel@vcu.edu. You can email or call me with any questions or concerns and you will have multiple opportunities to have contact with me over the course of time in terms of conducting these surveys. We're now going to move to the actual data form. I want to walk through that and talk a little bit about some of the things that you need to consider while conducting these interviews. I have now pulled up the Web site form and we will walk through this I do apologize that this can't be a little bit more interactive, but I want to -- for this particular training, give you an idea what the form looks like, how it is laid out, some of the thinking behind the question and then at a later time you will be given two Web addresses. One is a training address that you can access at any time and practice and look how the form is laid out, how the questions do skip to the next depending on how individuals respond. And I would encourage you to go to the training site and become very familiar with the form. I think the ease of conducting this interview while on the telephone and using a Web-based form comes from being very familiar with the form. I think starting out perhaps cold, if you will, not really familiar with the form and trying to do an interview may prove to be a little bit difficult, even though the form is very easily laid out. But training can really help you. That does not go into your, as we call it, real data set. You will receive another address that will be used when you are actually conducting the interviews. Information about your passcode, as you can see, this is the very front of the form and you must enter a passcode. Passcode information and Web addresses will be sent to your special education director or transition coordinator, whoever has been designated in your school division for this information to be sent to. What we would like you to do is to practice and then once you are ready, you will be able to see that you can enter these forms, the actual data forms, at several points in time in case you have to update information or start a new conversation with an individual or their designee. So you will have multiple access or easy access at any time to the forms that you are working on. So what we will do first is, I will type in the passcode. What you see before you now is, you have two options here. It says add survey data for a new student. So once you begin a form, if you're completing it from beginning to end, including all contact information, and that type of information, you can hit add survey data for a new student. What you see below that is your second option. Let's say that you have decided to go in prior to contacting Mary Evans as you see on this form and put in all of Mary Evans' information, her student I.D. number, her telephone numbers, that are all set up with background information before actually picking up the phone and contacting to see if you can interview Mary Evans. That way you have got that all completed prior to the interview. So you can either -- or let's say you contact Mary Evans and you are unable to reach her at that point in time. Then you have got that information in. You have documented that you have attempted to contact her one time and you don't have to start from the beginning when you are trying to contact Mary Evans if it takes you more than one time to reach here or someone to interview. On the front sheet you have add survey data for someone other than Mary Evans. Or if you need to go back to Mary Evans to complete the survey, you just click on that and that particular survey is pulled up. So for purposes of the training, we're going to look at entering for the first time the information for an individual. We ask for the individual's name, student identifier number. This is the state testing number, the student number, the student I.D. that each student should have received while in school. It must be this state I.D. number. If there is any questions about that, please contact me or Mary Ann Moore at the Virginia Department of Education. But we will be running various data reports and I must have that student I.D. number to be able to do that. So you will be given a list, the way we have talked to special ed directors, or transition coordinators, has been that they will use a particular report. It is called the end-of-year report and they will identify a pool of students that you will be contacting. Depending on that number, then how you split those interviews up, we're hopeful on that end-of-year report will be listed the student identifier number which is that state number. The information below that is if you are unable to complete the survey after multiple attempts and those give you choices. You must for every student that you are attempting to contact, let's say your school division is attempting to contact 150 school leavers. We need to have some type of information about what happened to those 150 individuals. It may be that the student you learn is incarcerated or declined to be interviewed. If that's the case, you would indicate that. Check what option in front if you are unable to complete the survey and submit the survey. And that survey is then done. But we must have information on each of the students -- or individuals, school leavers, school exiters that you are attempting to contact. You will notice on the survey there are a number of comment boxes. Comment boxes will be very important because you want to give -- it does not have to be lengthy but if there is particular information that you think will shed light or further information about a student or an individual that you contacted about either their situation or some important information that you feel needs to be conveyed, then please freely use the comment boxes. The comment boxes aren't to be used, let's say the student is incarcerated and you put in the comment box unable to interview. Those are not the type of comments that we're looking for. Perhaps, though, a comment would be the individual has attempted it several times to obtain employment but there has been a problem with lack of training, skills or information that's needed for particular jobs. The reason that we put the comment boxes in is that the state will issue a report. Remember, we're looking at the percent of youths with IEPs that have been competitively employed in postsecondary education or both. And they need to report to the federal government the status, if you will, of Virginia concerning what happens to students. What happened to those students in the 2005-2006 school year after they exited secondary education? Then the report -- a report will also be issued by each school division. So it is very important that if a report is issued -- when a report, I should say, is issued by a division, perhaps there are circumstances in your area that has led to students becoming competitively employed, perhaps a new industry came in and that needs to be noted or perhaps you have a large number or significant number of students who are not employed. Then you would want to indicate maybe there was something happening in your community. That gives it a richer information in terms of a better picture of what's happening in your school division or in your community as well. We are going to go through the form as if we are going to attempt to contact. So we go to the next page. In front of you then are several options for telephone numbers. Right now I am just going to do the home number. If we have cell numbers or work numbers, that would be great. Below that then are the dates and times of contacts. This becomes your record of contact so that you don't need to really keep a separate log of your contacts unless your school division determines that a separate log needs to be kept, but this can be your log of your attempts. And then the time of day that you attempted to contact. Let's say we tried at 4:00. I am going to say we were successful in completing the survey. You will only indicate the date obviously that the survey was completed when you are ready to submit and you are not going back and forth into the form. Let's say you attempted to contact the student. Let's say that you attempted to contact and you were unable to. Then you can exit out of this and you have a record now of this -- of Mary -- of myself actually, of Liz Getzel, and it will show you that the next time you are going to try to contact that individual, it will show up on your front sheet you click on and then you can get to this page and, again, record the date and the time that you attempted another contact. Let's say we were able to get a hold of the former student and we move to the next page. Again, there is a reference to what indicator 14 is about so that you constantly have that reference and that then the time frame in terms of contacting individuals. This information is completed by all respondents, all the people that you are interviewing. We have found in research that has been done and in some of my experience with post-school surveys that it is helpful to know what classes students found to be the most helpful when they were in school. Oftentimes it is information that can be used to inform school boards or other administrators about what were important classes, especially during this time when sometimes a particular class is being cut because of resources in terms of emphasis on further academics. So we like to ask these questions. We'll say that -- and you can check as many as the individual indicates, that they found most helpful. Then we've asked what the individual -- what they had wished they had taken more classes in. And in previous surveys that I have been involved in, I found that we had a pretty strong return rate that individuals wish they had taken more math or science, vocational technical classes or more computer classes. We then asked, since leaving high school, have you received services? Now, as you well know, in terms of working with individuals, students or family members, we are more familiar with acronyms about what the D.R.S., Department of Rehabilitative Services means. Oftentimes family members or individuals relate perhaps more to an individual's name, perhaps they saw Mrs. Brown quite a bit from the Department of Rehabilitative Services. We are suggesting that either pulling maybe some background information prior to doing the survey, certainly we are only asking what is reasonable for you to do. But perhaps there were some referrals or some things that students were provided prior to exiting that they may not recall. So if you have a summary of performance or if you have some exit information available, perhaps the student may say -- the individual may say, no, I didn't get any services. I don't know about these agencies and you may be able to jog their memory and say, remember, during the transition planning meeting you met with Mrs. Brown and then you get a better picture of what's happened. So let's say that they have received services or have gone to the Department of Rehabilitative Services and are looking at some involvement in the Centers for Independent Living. We also like to get an idea of where students are, individuals are in terms of their particular feelings about where they are in their life, if you will. We are asking about satisfaction. We have given the option of neutral in terms if the individual says, the former student, I really don't know how I feel. You can check neutral. Of course, you will be giving them what options are available going down through the menu of choices that they have or you may be interviewing a family member and they really don't feel they could respond appropriately in terms of where the individual -- the former student is at this point in their lives. So you would use "don't know" in that instance. Say we are talking to the student, and the student says they are very satisfied right now where they are. We now move into really what is the critical, if you will, part of the information that we are gathering for the federal government. The federal government wants to know about employment, postsecondary education or whether both is happening, the student involvement is happening in those two areas. The first question says right now at this point in time, at that point in time when you are talking with the student, former student or family member, are you working? At this point, I will say yes. But I do want to talk about what happens if an individual says no. Right now we will say yes, they are currently working. We have now clicked on the page to get further employment-related information about the former student. As you can see, the first question asks them to describe their current job and to check only one option. So you will need to go through carefully with the individual what the employment options are so that you are able to capture the most appropriate work setting. Further technical assistance information that will be sent to you will define many of the items in this particular question as well as some of the other questions so that if the individual you're interviewing says I'm not quite sure what you mean by a particular term, you will be able to have that readily available. So we will be defining almost all, if not most of the terms on the survey so that you can feel comfortable with what each term means. That also is very important to establish consistency for all the interviews that are going to be happening across the state. We want everyone to be using the same information about what we mean by a competitive work setting and that kind of thing. So we will say at this point that the individual is in a competitive work setting and at this point is working part-time. There are some questions about if they are earning minimum wage or above. We're not asking for specific salaries or anything like that, but just sort of generally we're saying at this point that -- I will click "minimum wage" and the current position that the individual is in does not provide any benefits. We then ask who helped the individual the most in finding their current job. So once again, indicating what the choices are for the individual and at this point, we will say that the individual selected the Department of Rehabilitative Services with some assistance of school personnel in terms of helping them find their current job. You may at this point, if there are any comments that you find particularly important about the position, perhaps the student has said I have been working since the day I graduated from high school or I just recently found this position, anything that you feel might give some background information, feel free to use the comment boxes. We now go to the page on postsecondary ed. Again, there is just enough information about employment to give anyone looking at the data results an idea of who's -- how many would be competitively employed, if they're full or part-time. Just enough information to get an idea where students are right after high school. The question is, again, then right now, are you enrolled in any type of school or training program? So I'm going to check no, that right now when interviewing this particular student, she is not enrolled in any type of school or training program. The form then takes you to the next question. Since leaving high school, have you ever been enrolled in any type of school or training program. Let's say that this particular individual has left for a semester of school to earn more money to go back to school. So since leaving high school, yes, she has been involved in some type of training program and you may want to put in your comments that this individual has left for a semester to work, just to once again give a little bit of background information. So, yes, since leaving she has been in some type of postsecondary education experience. It will then take you to questions about what was that schooling? What kind of training or postsecondary education experience was the individual involved in? I am going to say that she was in a college, a four-year college, and has left to work. And at the time they were enrolled, I'm going to say that this student was going to school part-time and had been working part-time. That is the information that we're looking at in terms of post-secondary education engagement, what type of training or schooling and was it full-time or part-time. You're now at the end of the survey. What you will see is a summary of all the responses that you have typed in or checked. This can be printed off, if you want a hard-copy record to put into an individual's file. You can then also just scroll down to see -- to look back in terms of the comments that you've made or what kinds of questions have been answered. So this is a summary of what was completed, and then you hit "submit." Of course, that would be the end of the interview. You would thank the individual for taking time, to see if they had any further questions or concerns and that type of thing. Now you have complete add data entry form on an individual who has at that point in time of the interview, was currently working, and at the time of the interview was not engaged in postsecondary ed but had been engaged in postsecondary ed during that time period. What happens then, once you submit, it takes you back then to the front page and you type in your log-in, your passcode that you all will receive. All school divisions will receive their own unique passcode. Let's say you needed to go back to edit a particular form. You have been trying to contact this particular individual and was not successful during the first time of contact. And the reason we have four contacts listed is that it has been determined by the state that when possible, that four attempts be made to gather data on a particular individual. Let's say that I'm trying again now today. I was unsuccessful previously, but today I was able to complete the information. And I was unable to get the former student but this time, in terms of my contact, I was able to get a hold of a relative. We would then go through again. I am going quickly through these pages because I want to show you what happens if someone, since leaving high school, has not been employed and has not been engaged in postsecondary education. It will take you to a different section of the form if they have been completely unengaged, if you will, during the time period since high school. We felt it was important to get some information about what was happening to the individual with particular experiences or issues that they were facing that has prohibited them from being engaged in the employment or postsecondary opportunity. And we will say that the relative wasn't sure how satisfied they were with what was happening in life, so we then now go to that question right now at this time, is this individual working. We are going to put no. The form then will move you to the question at any time since leaving high school, has this individual ever worked? Again, we are going to say no. It would put you on the page for postsecondary education question. Right now are you enrolled in any kind of school or training program. We will put no. It moves you then to the question, since leaving high school, have you ever been involved in any type of training or program. Again, we will say no. And now it takes you to the very back of the form where it asks very specific questions about what's happening with that particular individual. It you have never been employed, it asks you if you want to work. We will say that the individual says yes. Have you attempted to find a job? The individual says yes. And then it gives a list of specific responses about issues that may be happening. Perhaps there is no transportation available in that individual's area, that being able to get to work and back has been problematic. That they lack specific support services that are needed to help them become fully engaged in employment or there is a lack of affordable child care. This may be all issues that individual faces. So all of those could be checked. If there are other ones that are indicated by the individual that somehow does not fit into the menu of responses we've given you, you always have "other" in every question so that you can type in the appropriate response. Then we ask about postsecondary ed. If they have never been enrolled, do they want to be. They may say no because right now they are not interested in pursuing education. We want to be able to show accurate information about the individual. The individual wants to work but doesn't necessarily mean that they want to be involved in postsecondary education at this point in time. Then go ahead, you do not need to answer that question if they say they are not interested. If they say yes, they are interested, let's say they said yes, they are not enrolled but they certainly are interested, then you have a menu of questions to respond to in terms of whether there's lack of financial support or other issues that are facing that particular individual. Again, at the end of the survey where you are gathering information on someone who is not engaged in employment, not engaged in competitive employment, not engaged in postsecondary ed, you get a response -- a summary of your responses to those particular questions. And you submit. And then you are back in. Now, you will always be submitting a form, even if you are going back in to edit because the database will know that you are back in to edit it and it is not a completed form. We wanted to make sure that everyone had access, multiple access, two forms, because you are attempting to contact someone four times. The reason that it's important to give information about your attempted contacts is, again, as I had said earlier, we need to have some accounting for every individual that you are attempting to reach. Let's say that in your contact, the phone is disconnected. Your first try, you don't have an accurate phone number. We need to document that there is an inaccurate phone number, but then you need to make a reasonable attempt, and I do emphasize "reasonable attempt" to try to secure information on how to contact that individual. I know that dropouts may be particularly challenging because they have separated from high school and may feel that they want no further contact with the high school. Again, in the beginning of the interview, we will have a script where you will emphasize that participation in this survey is voluntary and that they may stop the interview at any point in time. We will also have information about why the survey is being conducted. I think probably a couple of things that each of you will come across is that people will want to know how this information is going to be used, would it be used so that someone would know who it was that was answering the questions so we will have some information for you in some of our further technical assistance packets about dealing with confidentiality questions. We do encourage that if an individual wants to verify that you are from the school, that this is an activity being conducted by the school, that you have a name and number of someone you could give to that individual if they wanted to verify, if they felt they wanted to do that. So we need to, again, account for each individual and we -- submitting on the first go-round that you have an inaccurate phone number is probably not your best course of action. You need to make a reasonable attempt, either talking to maybe someone in the school that may be a friend or relative of the individual and that kind of thing, just to see if -- and then let's say that you do try to track down a friend or a relative to get information or to find out how to contact an individual and that didn't work, then go back on the survey and document that information in the comment boxes. That's why we gave you a lot of comment boxes on this Web form to indicate that other attempts were made to try to secure a number if you aren't able to contact the individual. We also encourage that the four contacts be at varying times. Perhaps the individual is not available during the day, but if you call in the early-evening hours that the individual is at home and available or someone is at the particular number that you have that would be able to respond to a survey. We do encourage that attempts that are made with numbers that are working and that you know are working, that you -- that you attempt these numbers at varying times during the day, if possible. Leaving messages on voice mail is an option, and that will be something that you as a local team making these calls will determine whether you will. I would caution that if you are going to leave a message on a voice mail about contacting that individual, I would do so only if you know that you're going to be available during a particular point in time that that individual could call you back. You don't want the individual to continually try to call you and become frustrated that they can't reach you because you are involved in other activities. So you will be receiving a script that gives you some background information about confidentiality. Also, probably one of the areas that you will deal with is that people will really want to talk with you and talk with you at great lengths about ways happening in their life. We are looking at, perhaps, 15 to 20 minutes or so to complete the surveys. You want to make sure you have ways to move through the survey, documenting perhaps what individuals have to say. This may be particularly important that individuals have a great deal to say if they are not employed or they just lost their job or they're not engaged in postsecondary ed and we are rooming that school divisions have, perhaps, a list of resources available that have you contacted this particular agency and maybe have that number to sort of reconnect individuals. Some school divisions who have been doing post-school surveys for a number of years actually have a school individual designated to say -- and they will say someone from the school will contact them just to talk further about what's happening with them so that the individual is reconnected. It is a very good way to sometimes get individuals who are not engaged in any service delivery system to reconnect. Our primary aim is to try to talk with the former student. However, if that is not possible either because of the student not being able to complete a survey by telephone, to understand sort of what the questions are about, or not being able to contact the former student, then, of course, trying to then talk with someone who knows the student and knows what's happening with them in their lives certainly is, as we say, an individual or student designee is perfectly fine. We will be providing further information about some of the interview tips. You will receive sort of interview tips to review, just to highlight. We are asking that the interview be done sort of in a natural kind of conversational way and to try to avoid acronyms as much as possible. Again, that is helpful if you have contact information names or some kind of source, either the summary performance or some of the transition planning that you knew that an individual was connected with a particular individual in the community agency that that can help trigger potential contacts that the individual has had. That concludes the training on this particular form. I hope that by going through it that it has answered a number of your questions. Again, we will be sending out Web addresses so that you can train on that. I would encourage you to get familiar with the form. The form is definitely set up for ease of moving through as you answer particular questions that I think will be a lot easier than trying to do it by paper or pencil. And we will be continually putting and posting information up for you. This particular Webcast will be archived tomorrow after we complete all a the question and answer in the chatroom. The CDs will be made again. On the training, and the training will be made available again on our Web site, TTAC online and the Virginia Department of Education. We will be posting and alerting individuals that maybe frequently asked questions have been posted. I know as more and more individuals get into perhaps conducting the surveys or looking at the form, there are some questions about it and we will post that to clarify any questions that you may have. I wish you the best of luck in interviewing. I think this can be and will turn out to be a very positive activity because we get a chance to see all the good work that you've been doing in high school and seeing how students are being engaged in competitive employment and postsecondary ed and/or postsecondary ed as well as looking at what courses may have made an influence in terms of how students are doing after high school after helping those who are not engaged to reconnect back into the system. So please contact me with any questions and, again, I wish you the best of luck in terms of contacting students and I look forward to working closely with you during this activity. What we will do now, we will move to the chatroom where if you have particular questions now, I would be happy to answer them. If you wouldn't mind completing the evaluation form, we would greatly appreciate that to get feedback for further trainings that we will be doing in the future. So thank you and have a good day. ***