March 21, 2011--Notes from Ben on Informed Consent
Colleagues--Several of you have requested updated Informed Consent forms to be able to count your work as research. Of course, the normal course of programming you do not need consent. If there is a chance that you may want to present or publish a report, then you need to ask for consent. For those of you who are Learning Circle conveners, please pass on the forms and explanations to your colleagues. Feel free to refer any questions to me.

There are two types of consent forms for most 4-H programs. The Youth and Parent form is for parents to consent to their child's participation--in the study, not the program--and for youth (under 18 yrs) to add their "Assent." Parents should be able to retain a copy for themselves and agents/programs should retain a copy in their files. The Professional and Volunteer form parallels the Y/P form (at least on the same projects), with the descriptions targeted to adult leader roles/responsibilities. In many cases (e.g., research with camp staff, shooting sports leaders, Learning Circle teams), the research is fully focused on these adult leaders. The form should be retained by the program leader (e.g., camp director or E-LC specialist). Research project approval is for ONE YEAR (see the top of the form), so in most cases a new consent form must be signed each year. For ongoing projects involving 4-H programming (e.g., clubs, curricula) there are some cases where the first form is OK for multiple years. By a quirk of timing with IRB, the Youth and Paren form is renewed in early January and the Prof/Vol form is renewed in early June (or usually late May to accommodate camps and summer programs).

Keep in mind that the above comments relate ONLY to Consent forms. The project description itself must be presented in a proposal (or for ongoing programs, an addendum) that includes description of procedures and samples of instruments. Several of you have been through this process yourselves and any who have an idea and need help, just call me.

Finally, note that the Consent forms apply to research on a range of typical 4-H programming and training. NCSU IRB has agreed to put all these together under a single consent form as long as I supply project descriptions and tools for their review. "Stand alone" or "special interest" programs such as PYD, GPS, and Health Rocks usually get a separate review.


February 3, 2011-Notes from Ben Silliman
Colleagues--We are about a week away from the Conveners' Meeting at Millstone! I still need to hear from Patti, Ken, and Travis as to whether you are joining us. The tentative agenda is attached. Also, I am assuming that you all know how to find Millstone. Let me know if you need directions. We'll start at 9AM and go to 3PM. I am not big on professional dress, so wear what's comfortable in a drafty Lenz Building or Dining Hall : ) You will be receiving a registration form from James soon ($12) that will enable us to reimburse all your day's expenses.

There are several items of interest for your own programs and your Learning Circle teams:
1) WIKI. The http://nc4-heval.wikispaces.com site has a growing list of resources and descriptions. This week I've posted some notes to the "Designing Your Project Notes" section and posted a new plan reflecting Bill Stone and Linda Gore's efforts to refine the SC Club Quality and Impact Logic Model. You all are welcomed to look those over. MANY teams may have a Program Quality and Life Skills line that looks pretty similar. I am trying, wherever possible, to keep this aligned with your normal reporting requirements as well.

2) GPS. In a similar vein, we have a NEW opportunity to partner with Tufts on the next stage of assessment and programming on Goal Setting. Starting right now (but hopefully no later than March), the online GPS assessment (see link on our wikispaces site) will be "live." This round will use the same (well, slightly improved) tools as last Summer. However, they also want to monitor progress, so youth and mentors will have opportunities to go online three times (say March, June, and September) before the end of 2011 to: 1) complete the GPS/PYD tools (youth on themselves, mentors on 5 youth), 2) answer a few questions about the tools, process, and their group's progress, and 3) access goal setting/achieving skills training tools that you can incorporate in your programs.
Many of you already have Goal Setting skills implied or included in your Logic Model. In particular, I can see this as relevant for:

NW/Record Books-STEM: Do record books teach any goal setting skills?
SC/Club Quality & Impact: This looks like a good opp. to frame individual and club goals, keep a close coaching relationship, even compare results for Teen Leaders that agents coach and club members that volunteers coach
SW/Public Speaking: This would be an excellent opportunity to target at least one club in each county that wanted to do the GPS part and compare their more intensive focus on goals to kids who did the routine preparation for presentation contest.
SW/WC Science Fair: Ditto the above groups-link with groups, since you may not know individuals who will sign up for Sci Fair

Check out the research and resources at http://stepitup2thrive.org/tufts/
Please indicate your interest and feel free to pass on the opportunity to your learning circle peers.

3) E-LC Basic Training. A tentative plan for our online Evaluation Basic Training (the "second half" of Dr. Stewart's training grant) is attached. I plan to create a Camtasia (PPT + audio) series that covers the bulk of evaluation planning skills so that your team members can become familiar with the basic vocabulary and thinking behind program evaluation. Assessment tests would be included to check understanding. That way we can focus on -applying- skills to program plans/evaluations rather than spending too much time explaining the basic logic and vocabulary. I don't have a lot of funding, but would like to make these as interesting as possible. One idea I've tried is telling a story. I tried this approach with webinars last year with some positive feedback, but would like to see what you think, so will post the PPT under "Training Ideas" below the "Designing Your Project Notes" item on the Wiki site. If you all have stories from your county that we can use for illustration, that's even better.

4) NEI Self-Assess. Around Monday I will send out NEI Self-Assessment forms to any of you who started in Jan/Feb 2009 or 2010 and to any of your Learning Circle members. I will appreciate your encouraging peers to complete this (with the Consent form) since this is my best way to understand where to target training and our best way to document an incredibly innovative professional development effort for our 4-H colleagues nationally. Thanks for your support.

Well, that's more than enough to get in an e-mail after 5PM on a Thursday. Hope you have a good end of the week. Thank you for writing the history of 4-H and Extension through your innovative efforts!--ben

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
January 28, 2011-Notes from Ben Silliman
Lori--I have posted a new Logic Model for the SE/Presentation Contest Evaluation.
As you'll notice the "base" common to all the logic models is Club & Program Quality--If you're not doing your club and/or your teaching/curriculum right, outcomes are "accidents" or "corrections" of the status quo :)
Note that some items on the youth PRESENTATIONQs relate to evaluating the climate of the Presentation Contest preparation and event.
Beginning with Outcomes, you'll notice that on the Club row you might look for evidence of how 4-Hers are understanding and leading clubs (which may or may not impact Presentation Contest).
On the Curriculum/Programming row, the Presentation Contest Judging Rubric(s) seems appropriate to the subject matter and delivery skills that Presentation Contestants KNOW (e.g., what can they talk about); and the subject matter and delivery skills that they DO (e.g., they demonstrate the application of subject matter knowledge OR delivery skills).
Likewise, some of the items on the youth PRESENTATIONQs and PRESENTATIONQsParents are KASA Outcomes and some PRACTICE Outcomes. My bias is that if youth can DO it, it's a practice.
INTERVIEWQs pretty much asks about Practices. If the Practice has been applied in a community setting for more than six months (e.g., leading a Youth-Adult Partnership), then it might have a profound enough impact to change the social and cultural climate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
January 25, 2011-Notes from Ben Silliman
Designing Your Project: Presentation Contest example:

Presentation Contest entails some knowledge gains and skill building...
1) about the oral communication process (how to give an illustrated talk or demonstration).
This should track knowledge gains from review of the judging rubric and topic categories; informational or instructional workshops, websites, or videos of skills in action. Not all your team has to use the same "inputs," but each should keep track of what they use and all should use at least 1-2 of the same materials. If each of you does a workshop in your county, it should be about the same. The same applies to skill development. Describe the individual (coaching) or group (practice sessions) work so we can determine "what works" after judging is over. If club leaders do that, have each of them describe what they worked on and for how long. And for goodness sake, use the judging rubric as a reference point!
...this perhaps illustrates the need for a "Presentation Life Skills" curriculum :)

2) about the content (what was learned and applied). Since learning about the topic is pretty wide-ranging, a brief but very specific annotated references page might be required to submit to judges at the end of the presentation.

Those of you working with Record Books, Science Fair, or whatever, probably see the key point: Unless you document "what you've done," and it's pretty consistent for all kids/counties, it's pretty tough to design an outcome instrument or pool data on "what difference you made." So one of your first tasks will be to describe your program.

I hope to develop a measure of program quality that you can all use. I've attached a copy of the generic instrument I use at camps and conferences, but (like the SC Club team) each of you may want to design questions that fit your particular context.

As for measures of outcomes, I would suggest that 2-4 would be plenty. For Presentation Contest--Oral Communication (following the Judging Rubric, but we might want to add more detail), and Research (to be developed) come to mind.

Record Books--Goal Setting (GPS), Written Communication/Records (again, following or improving the judging rubric).

Science Fair--Visual Communication (following a judging rubric), and probably Research (could share with other programs).

...a wider range of skills and applications could be examined via interviews with a sample of participants, like we did on the first Presentation Contest research project in 2007.
...and we still need to target how to capture gains in content knowledge, attitudes, or aspirations. Jay's generic instruments designed to capture LFRA outcomes for STEM is a starting-point, but we can probably design something more precise...perhaps with the help of school teachers.