TAACCCT Research Questions

In addition to the quantitative data required by the Department of Labor for Project IMPACT, they are also wanting a response to several research questions:

•What service delivery and/or system reform innovations resulted in improved impacts for participants?

•Under what conditions can these innovations most effectively be replicated?

•What are the types of emerging ideas for service delivery change and/or system reform that seem the most promising for further research? Under what conditions are these ideas most effective?

•What directions for future research on the country’s public workforce system, and workforce development in general, were learned?

As you know, I conduct quarterly site visits to gather ongoing information about the status of Project IMPACT at each of the individual community colleges. I will continue to do that through the duration of the grant. In addition, I will ask for responses on the four research questions and add that information to the case studies. 

As always, I have appreciated your willingness to share your many successes and your limited concerns. I look forward to visiting with all you in the new year.   

Coordinating Internal and External Evaluation

The following article is transcribed from the EvaluATE Newsletter (Lori Wingate, Fall, 2014).

All ATE (like all TAACCCT) projects are required to allocate funds for external evaluation services. So, when it comes to internal and external evaluation, the one certain thing is that you must have an external evaluator. . . . There are many options for coordinating internal and external evaluation functions. Over the years, I have noted four basic approaches:

(1) External Evaluator as Coach: The external evaluator provides guidance and feedback to the internal project team through the life of the grant. This is a good approach when there is already some evaluation competence among team members. The external evaluator's involvement enhances the credibility of the evaluation and helps the team continue to build their evaluation knowledge and skills.

(2) External Evaluator as Heavy-Lifter: The external evaluator takes the lead in planning the evaluation, designing instruments, analyzing results, and writing reports. The internal team mainly gathers data and provides it to the external evaluator for processing. In this approach, the external evaluator should provide clear-cut data collection protocols to ensure systematic collection and handling of data by the internal team before they turn the information over to the external evaluator.

(3) External Evaluator as Architect; The external evaluator designs the overall evaluation and develops data collection instruments. The project team executes the plan, with technical assistance from the eternal evaluator as needed - particularly at critical junctures in the evaluation such as analysis and reporting. With this approach, it is important to front-load the evaluation budget in the first year of the project to allow for intensive involvement by the external evaluator.

(4) Divide-and-Conquer: The internal team is responsible for evaluating project implementation and immediate results. The external evaluator handles the evaluation of longer-term outcomes. . . . The external evaluator is responsible for determining and assessing the impact of the work in terms of application of the content and changes in evaluation practice.

Taking on part of an evaluation internally is often seen as a means of conserving project resources, and it can have that effect. But do not make the mistake of thinking internal evaluation is cost-free. At minimum, it takes time, which is sometimes a rarer commodity than money. In short, there is no one best way to coordinate internal and external evaluation. Your approach should make sense for your project in light of available resources (including staff time and expertise) and what you need your evaluation to do for your project.