A Learning Goal

I’m requiring students in my technology course to keep a blog.  I think it’s an excellent way for them to learn about the technology and hopefully develop habits of reflective practice.  One of my course goals is to teach students to use Internet tools to develop their personal learning networks.  To help them focus their networks, I want them have a learning goal, so for their 1st post, I am asking them to answer the following:

1.  What do you think is the most important thing you need to learn right now in order to be an effective teacher?  Why?  What actions are you pursuing to learn it?

Since I’m asking students to create learning goals and develop habits of reflective practice, I think it is only fair that I set the example and create my own learning goal.  The students that I will be teaching are primarily pre-service elementary education candidates while my experience is primarily in secondary and post-secondary settings.  To best serve my students, I feel it is important for me to improve my understanding of teaching in the elementary school setting.  I am teaching in a pre-school co-op and am also teaching a Sunday School class for older pre-school aged children, which does give me some insight into working with early elementary aged children, but not as much with older elementary aged children.  I’ve done quite a bit of reading on child development, which also helps some, but I think I need to spend time talking with, and possibly observing, some of my friends who are elementary school teachers.  Also, I may be able to contact some additional teachers when I attend the local elementary school’s School Community Council (I’m a community representative).  I’d love to hear from any elementary school teachers out there–what do you wish you had known before you stepped in front of your 1st classroom?

What a Difference a Year Makes

Tomorrow, I’ll be in front of a classroom again!  I’ll be teaching 1 section of a 1-credit technology course for teachers and later in the week, I’ll be teaching 2 sections of a 3-credit science methods course for elementary school teachers.  I’m excited and nervous.  It’s going to be a busy week for me, as I’ll also be teaching my older son’s pre-school group on Tuesday and Thursday.  We’ll see how quickly I can change gears.

 

An End

Six years ago, I applied for a PhD program.  I wanted to further my ability to make a difference in the lives of children.  Also, I was comfortable in the university environment and didn’t want to leave.   I loved my coursework, I loved teaching college courses, and I loved working at the university’s Center for Teaching and Learning.  I was eager to do research.

Things changed a little when my son was born.  As I spent more time with him, I became less eager to do research, but I was still interested.  I plugged along and made slow, steady progress.  Then, when my daughter was born and died shortly after; I lost all interest in research.  Now it’s a year later and my interest in the kind of research I was doing for my dissertation hasn’t come back.  I’m interested in plenty of other things, but I no longer care about being a university professor.  I have finally admitted this to myself and officially ended my career as a doctoral student.  This decision is not without regret, but on the whole, I feel better having made it.

On Topic

I am fascinated by the Internet’s ability to expand personal learning networks and create opportunities for self-directed learning on an almost infinite number of topics.  One of my original motivations for blogging was to observe the impact of blogging on my personal PLN firsthand.  My passionate interest in PLN made it a natural choice for my dissertation topic, but as it turns out PLN won’t make even the slightest appearance in my dissertation.  Instead, my dissertation deals with pre-service teachers’ ability plan technology-enhanced lessons.

Over the past few months, I’ve occasionally wondered if I would have found it easier to work on my dissertation if I hadn’t changed my topic.  During the past week, I’ve decided that it really doesn’t matter.  I’m interested in how people learn and in the efforts people make to help learning happen; from that perspective, either topic provides plenty of interest.

This week’s progress

In the past week, I’ve read and outlined seven articles related to my dissertation.  Two of the articles related to a specific problem I am trying to solve.  Specifically, my conditionally approved prospectus states that I will use a rubric to score student responses on a pre- and post- assessment.  One of the conditions of approval is producing the rubric that I will use.  Two of the articles I read this week will help me craft my justification for a particular rubric.  (At the time of my defense, I was waiting for the release of the results of the study containing this rubric.)

Two of the articles were literature reviews that I wish I had read before designing my study.  I feel that these literature reviews provide perspective on the larger field of teacher education related to technology integration.  Someday when I am advising graduate students, I will require them to read literature reviews related to their area of interest before selecting their topic.  If they can’t find one, then I believe they have prematurely narrowed their topic.  In my case, I feel that (as of December 2009–I’m currently playing catch up) I had read most of the literature  related to technological pedagogical content knowledge (also known as TPACK or TPCK), but I should have looked for literature reviews on the broader topic of teacher education and technology integration in order to better understand the context of my study.  I may have made some adjustments to the design of my study or framed my questions differently.  If nothing else, I would have had a stronger defense.

The remaining three articles were reports on various attempts people have made to teach teachers how to teach with technology.  Some articles provide useful citations for my literature review, some are of note for methodological or contextual similarities to my study.

As I’ve read, I’ve remembered that I like reading articles, which has been helpful for regaining motivation and momentum.  I met my dissertation progress goals last week and got to enjoy my reward.  I’m currently waiting on four specific books from the library, so while I’m waiting, I’ll start transcribing the interviews from my study.  I expect transcription take most of the next two weeks.

Coming back to work

Some important facts:

1. The data for my dissertation have been collected.
2. My son (pictured above) is now 5 1/2 months, takes regular naps, and sometimes sleeps through the night.

My conclusions based on the above facts:

1. I need to devote time to analyzing my dissertation data.
2. I have time to devote to analyzing my dissertation data.

For the past couple of months, I’ve been half-heartedly trying to re-dedicate myself to the work of being a PhD student.  It’s been tough-going as half-heartedness and dedication are antagonistic concepts.  (Besides, my whole heart is already devoted to the pint-sized person pictured above.  He really is a sweet baby.)  Still, I’ve recently decided that finishing my dissertation and being a good mother are not mutually exclusive goals.  I just need to manage time differently than I have in the past.

To help myself reach my dissertation goals, I mapped out deadlines for the completion of important dissertation steps and placed them on my Calendar.   I brainstormed a list of all of the required tasks and sub-tasks related to my dissertation, and I decided that I will devote two hours each weekday to dissertation tasks and will read and outline at least one article daily. Each Saturday I will summarize my progress in a blog post, to try to hold myself accountable for meeting my goals.  As a reward for meeting my process goals, I will earn time that I can spend on my hobbies once my son is in bed for the night.  When I complete tasks from my to do list, I will “earn” a small financial reward (really, give myself permission to spend money previously earned).  The rewards for this upcoming week–knitting time and the registration fee for an event in which I want to participate.

Check back next week to see how the plan is working.

A Quick Update

My son, day 1

Last year, blogging took a back burner to other priorities in my life.  I was working at multiple jobs for most of the year, I was PhD student, and I was pregnant.  I am no longer pregnant, and I’ve stopped working outside the home, but blogging will still be on the back burner.  The little guy pictured above is priority #1 and will be for the foreseeable future.  I managed to defend my dissertation prospectus the week before he was born, and am still plugging away at collecting and analyzing my data, but I have to admit that educational technology isn’t as relevant in my life as it was even a few months ago.  My son is nearly seven weeks old.  He’s learning some important things right now, like how to hold up his head, smile, and use his hands.  The world of bits and bytes has very little to offer him right now, and I’m OK with that.

A Scientific Approach to Teaching

Yesterday, at an educational training, I sat down to lunch with a woman in charge of a large Head Start program in the Pacific Northwest.  Through our small talk, she learned that I had been a secondary science teacher for six years, and then informed me that her program was going to focus on science during the next school year.  I was excited to think of the possibilities of an early-childhood education program with a science focus and listened as she expressed concern that she hadn’t yet managed to convince her staff of the value of a science-focus.  Lunch ended before we could finish our conversation, but I’m still thinking about it a day later.

Obviously I am biased, but I am a strong advocate of a scientific (in the sense of “occupied or concerned with science”) approach to teaching at all levels.  My reasoning for a scientific approach is not because I have complete faith in the primacy of the scientific method as a way knowing, but because I believe science (in its broadest sense as “systematized knowledge in general” or “knowledge gained by systematic study”) to be an interdisciplinary study.  Art, Music, Language, Literature, the various Social Sciences, and Mathematics would benefit from what I would term, a scientific approach to teaching, or perhaps, a scientific approach to curriculum.

Below I share what I believe are the crucial components to a “scientific approach to teaching”.

1. Foster a sense of wonder

I would challenge all educators to become familiar with Rachel Carson’s short work, The Sense of Wonder.  All science, all true knowledge and understanding, begins with a sense of wonder, an intense curiosity about “why” and “how”.  Too often, our teaching at all levels focuses on the “who, what, and where” of the world around us and treats the potentially awe-inspiring matters of “why” and “how” as simply another “what” to memorize.    A scientific approach to teaching encourages learners to wonder about “why” and “how”.

2. Encourage observation

A sense of wonder is grounded in a keen awareness of one’s surroundings.  Scientific knowledge is based in empirical data, data gathered from experience and observation.  Encourage learners to attend to details and use their senses of sight, hearing, touch, taste, and smell whenever appropriate.  Challenge learners to record these details qualitatively and quantitatively.  A scientific approach to teaching gives learners many appropriate opportunities to use their senses and record their observations.

3.  Push for analysis

Wonder grows with understanding and understanding comes of analysis.  Quantitative analysis is the heart of much of what we traditionally consider the realm of science and mathematics.  Quantative analysis also reaches deep into the realms of the social sciences, of economics, psychology, sociology, and many others.  But qualitative analysis is also valid; it has a place in the sciences, the social sciences, and is especially important in the arts and literature.  Provide time for learners to revisit their observations and search for patterns, in whatever form they may take.  Push learners classify, connect, and count.  A scientific approach to teaching pushes learners to seek for patterns.

4. Require communication

Like most things in life, wonder is best appreciated when shared.  Our languages provide us with means of communication, but human language is far from our sole source of expression.  Prod learners toward precision in their descriptions of the world around them and the findings of their analyses.  But also instruct learners on the use of ambiguity in the creative expression of emotion and personal experiences.  A scientific approach to teacher requires learners to communicate their learning.

I see a scientific approach to teaching as a way to guide learners in the process of transforming the concrete objects and artifacts in the world around them into the abstract concepts that inhabit the human mind.  Each step in the process is crucial.  Without a sense of wonder, the learner will have no desire to continue the journey.  If a teacher fails to take the time to sufficiently ground the learner in concrete observation, the learner will be lost in unrecognizable terrain.  The skills of analysis are what will enable learners to chart future courses for independent learning, while the powers of clear communication and creative expression are what will bind the learner to the rest of the human race.

So how about it?  Do you agree, disagree?  What would a scientific approach to teaching look like in a Pre-K science lesson or  a college art history class?  I’d love to know what others think.

Evaluating E-learning

Right now I’m supposed to be writing an article about evaluating e-learning.  It is the only assignment in a seminar course on e-learning evaluation.  The spring term ends on June 18th, and with barely over two weeks before my deadline and all I’ve produced is an outline with which I am incredibly dissatisfied.  I could blame writer’s block.  I could claim that my brain hasn’t yet returned from DisneyWorld, where I celebrated my birthday last week.  I could point fingers at my 60+ hour a week work schedule that I’ve only just recently pared back. I could easily name many other distractions.   The simple truth that I am forced to admit to myself is that I haven’t consistently wanted to think long and hard about evaluating e-learning.  At least not until this week.    

As the instructor of an online course and a student who is attempting to complete the final stages of a doctoral degree from a distance, I feel that I have a unique perspective on e-learning and its evaluation.  Certainly, my perspective differs from my classmates; I am the only student in the e-learning evaluation seminar for whom the seminar itself is an e-learning experience.  I want my article to reflect what I have learned as I’ve participated in e-learning experiences, both formal and informal, and in both of my roles, instructor and student.

At the same time, my professor is expecting an academic article and I’m struggling with how best to weave my personal narrative into the evaluation framework that Dr. Williams has provided for us.  The e-learning evaluation framework given by David Williams and Charles Graham in their soon to be published article hangs on the following (rather generic) questions:

1. What is the context/background?
2. Who are the stakeholders?
3. What is the evaluand?
4. What are the criteria for judging the evaluand?
5.  What questions will answer how well the evaluand meets the criteria?
6. What methods should be used to answer the questions?
7. What do you get when you collect and analyze the data?
8. How does ‘what is’ compare to ‘what should be’?
9. What recommendations does the study yield?
10. How well was the evaluation conducted?

As I review these questions yet again, I realize that this framework matches the logic that I used to design and evaluate the online course that I taught.  At the same time, this methodology does not quite reflect the informal process that I, as a student, use to evaluate my e-learning experiences.  I know that I have found some of my e-learning experiences more valuable than others, which indicates that I evaluated those experiences on some level, but how?

Some of the questions from the framework seem to apply to both perspectives.   I can describe the context for each of my varied e-learning experiences, and in all my informal evaluations of e-learning there has only ever been but one stakeholder: me.  (Even now, in the evaluating e-learning seminar, I am relatively unconcerned with the learning experiences of my classmates.  I hope they are learning, but my evaluation of the experience is independent of what they feel they are gaining from the experience.  I am somewhat concerned about how Dr. Williams will evaluate me, but my evaluation of the seminar will not be impacted by whether or not Dr. Williams feels that it has been a success.)  I have never explicitly stated my criteria for evaluating e-learning experiences as a student and I’m not sure that I could articulate each criterion now, or even if I’ve applied the same criteria to each experience.  I have definitely never framed evaluation questions based on my criteria, contemplated methods of data collection, or analyzed data that I’ve collected during my e-learning experiences.  Still, I know that as a student in each of my e-learning experiences I’ve come to conclusions about how ‘what is’ compares to ‘what should be’ and have used these conclusions to inform the design of e-learning opportunities for the courses that I teach.  I’ve done this  without ever meta-evaluating the process that resulted in my conclusions.  

I find myself wondering: (1) how do most students evaluate e-learning experiences? and (2) if students were to consciously apply the Williams-Graham framework would their evaluations of their e-learning experiences change?

What do you think?  What process do you use to evaluate e-learning experiences as either an instructor or a student?

Dissertation Time

Tomorrow I have a meeting with my dissertation chair.  I am supposed to come to the meeting prepared with a timeline for completing my dissertation prospectus.  The timeline is meant to help me manage my time as well as provide a means for my professor to hold me accountable for the work that I should be doing.  I figure that I might as well make the timeline public.  Regular readers, feel free to help hold me accountable!

First, I know that my graduation goal is April 2010 and that I need to have my data collected during the Fall 2009 semester in order to meet this deadline.  Ideally, I should complete my prospectus this summer.  However, professors like to take vacations during the summer, which means that I will not be able to gather my committee in one place until the Fall.  As a result, I need to plan on defending my prospectus at the very beginning of the Fall 2009 semester.  The first day of Fall classes is August 31st.  I need to schedule my defense no later than September 4, 2009.

My committee will need time to read and review my prospectus prior to my defense.  Really, 2 weeks is plenty of time for them to review one prospectus.  However, they will be very busy at the beginning of the semester, so I plan to give them 4 weeks.  My prospectus must be completed by August 7, 2009.

I have approximately 12 weeks to complete my prospectus.  

I could say that I will work on the introduction for 4 weeks, then the literature review for 4 weeks, and then finish with the methodology for final 4 weeks.  This approach seems artificial to me.  I think a better, more organic approach would be to allow the prospectus to emerge from my reading.  I think I should start by intensely focusing on the literature.  I also need to follow up on the contacts that I made at the SITE conference.  To be realistic, I need to take into account that I will be on vacation with my husband for at least two weeks this summer.  As a result, I am proposing the following time line:

 

• May 21 – Prepare reading list from references of articles already read, and the recommendations of those in the field.  Begin reading the articles/books and take notes on each reading.  Also, begin a concept map connecting the themes from all of the readings.  The purpose of my reading between today and June 4 will be to better understand the terminology.
• June 4 – Prepare an outline based on my synthesis of my readings to date.  Frame broad research questions.  Edit (most likely augment) the original reading list.  The purpose of my reading between June 5 and June 11 will be to identify the types of research studies that have been conducted previously.
• June 11 – Revise the outline to include specific research questions.  Identify the types of study most appropriate for the research questions.  The purpose of my reading between June 11 and June 18 will be to identify specific research methodologies that are appropriate for my study.
• June 18 – Revise the outline to include a description of my proposed research methodology.  Contact any available committee members for feedback at this point.  The purpose of my reading between June 18 and June 24 will be to search for possible inspiration from tangential areas of research.
• June 24 – Revise outline to reflect any feedback received from the committee and any inspiration from reading.   Honestly, I’ll be attending a family reunion in late June/early July, so I’m not likely to do much academic reading during the weeks that immediately follow June 24.  Still, I will take my articles with me for the plane rides.  My purpose for reading between June 24 and July 9 is to re-read the literature with fresh eyes, looking for insights I may have missed previously.
• July 9 – Prepare my first prose draft of the literature review of my prospectus.   Provide copies of the draft to peers and members of my committee for feedback.
• July 16 – Prepare my first prose draft of the methodology section of my prospectus.  Provide copies of the draft to peers and members of my committee for feedback.
• July 23 – Prepare my first prose draft of the introduction section of my prospectus.  Provide copies of the draft to peers and members of my committee for feedback.  Between July 24 and August 6th, my purpose for reading will be to discover solutions to issues raised during the feedback process.
• July 30 – Prepare a list of constructive criticisms offered by peers and committee members.  
• August 6 -Revise all sections of my draft, incorporating feedback from my peers and committee members.
• August 7 – Provide all committee members with a copy of the prospectus in preparation for the defense on September 4.
• September 4 (or before) – Defend the prospectus.  I welcome feedback.

Am I being realistic?  Any suggestions?

Thanks!

Assessing Affective Characteristics in Schools

Another book summary in partial fulfillment of my independent reading assignment for graduate school.

Brief Review

I was assigned to read Assessing Affective Characteristics in Schools by Lorin Anderson and Sid Bourke.  I found the text to be less technical than Summated Rating Scale Construction, but often more detailed in its advice.  (This shouldn’t be particularly surprising, since Anderson and Bourke used far more pages than Paul Spector.)  Anderson and Bourke also dedicated far more pages to convincing the reader of the necessity of assessing affective characteristics than Spector did trying to convince the reader of the necessity of constructing summated rating scales.  Over the past few years, I’ve become increasingly convinced of the importance of affective characteristics in learning, particularly in the role of motivation.  As a result, I sometimes felt that Anderson and Bourke were preaching to the choir, and wished I could read a less evangelical version of the text that would simply tell me what I needed to know to get the job done.  

Summary of Content

In the first chapter, Anderson and Bourke define the terms that comprise their title. They enumerate five features that they claim define affective characteristics, specifically, that affective characteristics are typical ways of feeling that are directed toward some target with some intensity. Anderson and Bourke define assessment as “the gathering of information about a human characteristic for a stated purpose.” The authors choose to focus on affective characteristics of students in the context of school settings. According to Anderson and Bourke, affective characteristics have value as means to ends and as ends in themselves. In the latter sections of the first chapter, Anderson and Bourke address common beliefs that sometimes impede the assessment of affective characteristics in schools. According to Anderson and Bourke, affective can and should be assessed in school settings.

Chapter two of Assessing Affective Characteristics in Schools focuses further on definitions, detailing the importance of clearly defining the specific affective characteristic or characteristics that one intends to assess. Anderson and Bourke also point out the importance of carefully defining the target to which the affective characteristic is directed. Conceptual definitions provide an understanding of abstract meaning while operational definitions specify behaviors that allow observers to make inferences about affective characteristics. The authors believe that conceptual and operational definitions must be closely aligned in order to provide useful information about a particular affective characteristic. The chapter provides a description of two major approaches for developing operational definitions of affective characteristics, the mapping sentence approach and the domain-reference approach. Whether one is creating a new assessment instrument or selecting a previously created assessment instrument, one should begin with a precise definition of the affective characteristic in question.

The third chapter discusses the major methods for collecting data about human characteristics, the observational method and the self-report method. Both methods have strengths and weaknesses. The observational method is limited by the observer’s powers of observation as well as their powers of interpretation. The self-report method is limited by respondent’s memory and/or integrity as well as the questioner’s ability to ask the right questions. Some studies have shown that observational and self-report methods that claim to assess the same characteristic provide dissimilar results. Anderson and Bourke believe that, at least in the context of schools, self-report methods are generally superior. However, the authors also state that they do not intend the chapter to be interpreted as a complete rejection of observational methods.

Good affective scales must have communication value, objectivity, validity, reliability, and interpretability. A questionnaire has communication value if the respondent can easily understand what the questionnaire is asking them. A scale has objectivity when it has minimized scorer or coder bias. An instrument has validity when it actually measures what it purports to measure. Scales are considered reliable when they have internal consistency, stability , and equivalence. Internal consistency is often measured by Cronbach’s alpha, stability may be measured using test-retest results, and equivalence may involve a comparison of multiple measures of the same affective characteristic. Questionnaires are considered to have interpretability when the results are reported in such a way that primary audience of the data can understand the results. Anderson and Bourke describe a number of common practices in the assessment of affective characteristics included the use of several varieties of Likert scales.

Anderson and Bourke provide advice for either selecting or designing assessment instruments for affective characteristics. When possible, they recommend selecting an existing an instrument over designing one. They enumerate several potential sources for locating existing assessment instruments,

• electronic databases,
• commercial publishing houses,
• professional associations,
• research institutes and laboratories, and
• compendiums.

 They also provide a list of six steps for designing a new instrument:

• preparing a blueprint,
• writing the items,
• writing directions,
• having the draft instrument reviewed,
• pilot testing the instrument, and
• readying the instrument for administration.

 However, whether an individual will select an existing instrument or design a new one, Anderson and Bourke emphasize that the first steps are to determine the purpose of the assessment, identify the target population, and define the affective characteristics and targets. The authors list four common categories of purposes for affective assessment,

• enhancing student learning,
• improving the quality of educational programs,
• evaluating the quality of educational programs, and
• conforming to administrative or legislative mandates.

 Data analysis is the main focus of chapter six. The authors provide a list of five steps for developing and analyzing scale scores:

• coding,
• entering and checking data,
• dealing with missing data,
• recoding items as necessary,
• checking scale validity and reliability, and
• creating and reporting scale scores.

Anderson and Bourke address the importance of good data and provide advice for error checking, such as dual coding, as well as methods for dealing with small amounts of missing data. The authors also discuss using factor analysis to address empirical validity in multiscale instruments.

 The authors describe the process of interpreting assessment data for affective characteristics in chapter seven. They suggest using absolute and/or relative comparisons to assist in the interpretation of the data. Absolute comparisons require the identification of a neutral point and the creation of a neutral range as well as a range above the neutral range and a range below the neutral range. Relative comparisons may involve a normative sample or it may involve comparisons between known groups whose scale scores are expected to differ. Interpretations will depend on the comparison method used.

 Anderson and Bourke use chapter 8 to argue the importance of affective assessment in finding solutions to common education problems including student motivation, the design of effective learning environments, and character building.