Another book summary in partial fulfillment of my independent reading assignment for graduate school.

Brief Review

I was assigned to read Assessing Affective Characteristics in Schools by Lorin Anderson and Sid Bourke.  I found the text to be less technical than Summated Rating Scale Construction, but often more detailed in its advice.  (This shouldn’t be particularly surprising, since Anderson and Bourke used far more pages than Paul Spector.)  Anderson and Bourke also dedicated far more pages to convincing the reader of the necessity of assessing affective characteristics than Spector did trying to convince the reader of the necessity of constructing summated rating scales.  Over the past few years, I’ve become increasingly convinced of the importance of affective characteristics in learning, particularly in the role of motivation.  As a result, I sometimes felt that Anderson and Bourke were preaching to the choir, and wished I could read a less evangelical version of the text that would simply tell me what I needed to know to get the job done.  

Summary of Content

In the first chapter, Anderson and Bourke define the terms that comprise their title. They enumerate five features that they claim define affective characteristics, specifically, that affective characteristics are typical ways of feeling that are directed toward some target with some intensity. Anderson and Bourke define assessment as “the gathering of information about a human characteristic for a stated purpose.” The authors choose to focus on affective characteristics of students in the context of school settings. According to Anderson and Bourke, affective characteristics have value as means to ends and as ends in themselves. In the latter sections of the first chapter, Anderson and Bourke address common beliefs that sometimes impede the assessment of affective characteristics in schools. According to Anderson and Bourke, affective can and should be assessed in school settings.

Chapter two of Assessing Affective Characteristics in Schools focuses further on definitions, detailing the importance of clearly defining the specific affective characteristic or characteristics that one intends to assess. Anderson and Bourke also point out the importance of carefully defining the target to which the affective characteristic is directed. Conceptual definitions provide an understanding of abstract meaning while operational definitions specify behaviors that allow observers to make inferences about affective characteristics. The authors believe that conceptual and operational definitions must be closely aligned in order to provide useful information about a particular affective characteristic. The chapter provides a description of two major approaches for developing operational definitions of affective characteristics, the mapping sentence approach and the domain-reference approach. Whether one is creating a new assessment instrument or selecting a previously created assessment instrument, one should begin with a precise definition of the affective characteristic in question.

The third chapter discusses the major methods for collecting data about human characteristics, the observational method and the self-report method. Both methods have strengths and weaknesses. The observational method is limited by the observer’s powers of observation as well as their powers of interpretation. The self-report method is limited by respondent’s memory and/or integrity as well as the questioner’s ability to ask the right questions. Some studies have shown that observational and self-report methods that claim to assess the same characteristic provide dissimilar results. Anderson and Bourke believe that, at least in the context of schools, self-report methods are generally superior. However, the authors also state that they do not intend the chapter to be interpreted as a complete rejection of observational methods.

Good affective scales must have communication value, objectivity, validity, reliability, and interpretability. A questionnaire has communication value if the respondent can easily understand what the questionnaire is asking them. A scale has objectivity when it has minimized scorer or coder bias. An instrument has validity when it actually measures what it purports to measure. Scales are considered reliable when they have internal consistency, stability , and equivalence. Internal consistency is often measured by Cronbach’s alpha, stability may be measured using test-retest results, and equivalence may involve a comparison of multiple measures of the same affective characteristic. Questionnaires are considered to have interpretability when the results are reported in such a way that primary audience of the data can understand the results. Anderson and Bourke describe a number of common practices in the assessment of affective characteristics included the use of several varieties of Likert scales.

Anderson and Bourke provide advice for either selecting or designing assessment instruments for affective characteristics. When possible, they recommend selecting an existing an instrument over designing one. They enumerate several potential sources for locating existing assessment instruments,

• electronic databases,
• commercial publishing houses,
• professional associations,
• research institutes and laboratories, and
• compendiums.

 They also provide a list of six steps for designing a new instrument:

• preparing a blueprint,
• writing the items,
• writing directions,
• having the draft instrument reviewed,
• pilot testing the instrument, and
• readying the instrument for administration.

 However, whether an individual will select an existing instrument or design a new one, Anderson and Bourke emphasize that the first steps are to determine the purpose of the assessment, identify the target population, and define the affective characteristics and targets. The authors list four common categories of purposes for affective assessment,

• enhancing student learning,
• improving the quality of educational programs,
• evaluating the quality of educational programs, and
• conforming to administrative or legislative mandates.

 Data analysis is the main focus of chapter six. The authors provide a list of five steps for developing and analyzing scale scores:

• coding,
• entering and checking data,
• dealing with missing data,
• recoding items as necessary,
• checking scale validity and reliability, and
• creating and reporting scale scores.

Anderson and Bourke address the importance of good data and provide advice for error checking, such as dual coding, as well as methods for dealing with small amounts of missing data. The authors also discuss using factor analysis to address empirical validity in multiscale instruments.

 The authors describe the process of interpreting assessment data for affective characteristics in chapter seven. They suggest using absolute and/or relative comparisons to assist in the interpretation of the data. Absolute comparisons require the identification of a neutral point and the creation of a neutral range as well as a range above the neutral range and a range below the neutral range. Relative comparisons may involve a normative sample or it may involve comparisons between known groups whose scale scores are expected to differ. Interpretations will depend on the comparison method used.

 Anderson and Bourke use chapter 8 to argue the importance of affective assessment in finding solutions to common education problems including student motivation, the design of effective learning environments, and character building.   

A summary of Summated Rating Scale Construction: An Introduction by Paul E. Spector.  This summary is provided in partial fulfillment of the requirements for my independent reading course this semester.  

Brief Review

Spector uses the Work Locus of Control Survey throughout this work to exemplify the process of constructing summated rating scales. I found it more useful to consider how the advice given applies to the instrument that Dr. Graham has developed to assess pre-service teachers’ assessment of their own Technological Pedagogical Content Knowledge. Also, since Stata, not SPSS, is my preferred statistical package (and because this text was published in 1992) I found the information on computer software irrelevant or obsolete. Still, I think the text helped me to better understand information that I had previously read in survey methodology texts.

Summary of Content

One of the defining characteristic of a summated rating scale is the presence of multiple items. Multiple items provide reliability and precision. Additionally, the individual items that comprise a summated rating scale must be measured using a continuum and written so that there is no single answer. Individuals responding to a summated rating scale must answer each item with its own rating.

The process of developing a summated rating scale is iterative. The primary step involves defining the construct. Only after construct definition, can a researcher hope to design and then pilot a scale. Once a scale has been piloted, the next step is to administer the instrument and conduct a thorough item analysis. The results of the analysis may lead the refine his or her original construct definition. Once the researcher is satisfied with the construct definition, he or she may begin to validate and norm the assessment.

Three common categories of response categories include agreement, evaluation, and frequency. According to Spector, the optimum number of responses for an item ranges between five and nine. Negative responses should be re-scaled before the data is analyzed. The formula for re-scaling negative data is R = (H + L) – I where H is the largest number, L is the lowest number, I is the response to an item, and R is the score for the reversed item.

Spector shares several rules of thumb for item writing:

1. Items should express single ideas.
2. Some items should be worded positively, others negatively.
3. Items should avoid the use of colloquialisms, expressions, and jargon.
4. Item-writers should remember the reading level of the target audience for the scale.

A main purpose of item analysis is to determine the items that contribute to the internal consistency of the instrument. Coefficient alpha is a common measure for describing internal consistency and 0.70 is a minimum target. Coefficient alpha is used in tandem with item-remainder coefficients to identify potentially troublesome items. One strategy for selecting items for inclusion are to decide on a number, for example, m, and then select the m items with the highest item-remainder coefficients. Alternatively, you can set an item-remainder coefficient criterion and include all items that meet the set criterion. A researcher may consider other, external criteria, such as social desirability, hen selecting items. The Spearman-Brown prophesy formula can provide a useful estimate of the number of items needed to reach internal consistency.

There are many different ways to study the validity of an instrument. Criterion-related validity includes concurrent, predictive, and known-groups validity. Each of these criterion-related validity techniques involves a comparison between the scores from the summated rating scale in question and a set of other variables. In concurrent validity studies, the scale scores are collected at the same time, from the same individuals, as the other variables. In predictive validity, the scale scores are collected and then used to predict the value of a variable in the future. In known-groups validity, the researcher tests one or more hypotheses about differences between the scores to two or more groups.

Convergent and divergent validity studies are based on the principle that measures of the same construct will correlate strongly while measures of different constructs will correlate less strongly. Researchers use the Multitrait-Multimodal Matrix (MTMM) in order to explore convergent and divergent validity.

Factor analysis is another tool that researchers use to explore the validity of instruments. Exploratory factor analysis helps to determine the number of constructs that might describe a particular data set. Confirmatory factor analysis can help determine if a set of constructs in a theoretical framework fits the empirical data.

Spector suggests that researcher validate instruments by collecting as many different types of evidence as possible . Spector also addresses the importance of determining the reliability of the instrument, not only internally, but across time, as in test-retest reliability. Additionally, Spector points out that instruments should be normed with samples from the appropriate target population, not simply with samples of convenience found on college campuses. When calculating norms, mean and standard deviation are of primary importance, as is the overall shape of the distribution.

Finally, since scale construction is a recursive, iterative process, it is never-ending. The goal is not perfection, but to get a scale that behaves consistently within its own theoretical framework.

Another of the books that I finished toward the end of last year was A Whole New Mind: Why Right-Brainers Will Rule the Future byDaniel Pink.  The central argument of A Whole New Mind was that our global society has transformed from a society in which left-brain skills were dominant to one where right-brain skills are, at the very least, equally useful.  Pressure from automation, Asia, and abundance is forcing individuals to adapt their way of thinking in order to stay competitive.

After presenting his argument, Pink suggests a skill set for this new society:

• Design
• Story
• Symphony
• Empathy
• Play
• Meaning

I think the most interesting feature of Pink’s book is that each of the skill chapters ends with a set of suggested portfolio activities.  For example, to read Victor Frankl’s Man’s Search for Meaning: An Introduction to Logotherapy as one of the activities under “Meaning” or to attend a session of laughter yoga for “Play”.  Attempting at least some of the portfolio projects is something that I want to do when I stop “wanting” to work 60 hours a week.

My Mind Map of A Whole New Mind

It’s Friday night and my poor, sick husband fell asleep before 9:00 PM.  Not wanting to waste time, but also not wanting to violate my long-standing (dating back to high school) personal rule of no homework (or work-work) on Friday night, I decided that I might tackle an unfinished blog post.  So . . . .

Several months ago, I finished reading Long Tail, The, Revised and Updated Edition: Why the Future of Business is Selling Less of More by Chris Anderson.  My husband, when he saw the book on my bedside table, gave me a puzzled look.  A book about selling things?  He didn’t think it fit with my usual reading interests, and in many ways it didn’t;  I’m not particularly interested in selling less (or more) of anything . . . at least not right now.   However, at least some of the book’s themes did fit in with my interests in networks.

Fundamentally, the long tail is another expression of the power rule so common in networks.  The long tail phenomenon is fueled by network effects.  The existence of long tails (and even tails within tails) in music, movies, etc. allow for creation of niches of interest and lead to the troublesome possibilities of echo chambers, tribes, and ego-casting.

My fellow student, John Hilton, posted his (more thorough) review of the book over a month ago.   You can also learn more about the book by visiting  The Long Tail blog.  As usual, I’ve included my mind map notes, mainly for my own benefit.

My mind map for The Long Tail

Image by pumicehead via Flickr

I recently read the  The Social Life of Information by Brown and Duguid.  Compared to the technology-related books that I have read lately, Brown and Duguid are less enthusiastic about the promised blessings of technology.  They are skeptical of the tendency, which they name “endism”, to predict the end of everything from paper to traditional workplace and they caution against “infologic”, the tendency to believe that every problem can be solved simply by putting it in terms of information, or more information.  Brown and Duguid encourage a cautious examination of the social and psychological issues that put problems in their proper context.

The book is set up as a collection of essays touching on topics as varied as who should be held accountable for the actions of autonomous bots to a proposal for the redesign of higher education.  A recurring theme involves the unintended consequences of eliminating traditional social interactions from the workplace.  Technology is often implemented in ways that remove social support systems, causing isolated individuals to bear the burden of responsibilities once shared by a group.

I found the latter chapters to be the most interesting.  “Learning in Theory and Practice” pointed out that the presence of a knower is what distinguishes knowledge from information.  Brown and Duguid refer to definitions of knowledge by Bruner, Kyle and Polanyi that lay the framework for their argument that learning is a social process as well as a process that helps shape individual identities.  People with similar knowledge and resources have similar identities.  For example, people who identify as engineers identify as engineers largely because of what they have learned in the process of becoming engineers.  Similarities in identity help create ties between people occupying similar roles within and across different organizations.  Each tie is part of a larger network of practice. The concept of networks of practice is related to Lave and Wenger’s communities of practice.  However, I believe that networks of practice is a better concept for describing informal learning in online environments than communities of practice.

I’m now looking for articles that examine the networks of practice idea more closely.  Any suggestions?

My mindmap for The Social Life of Information

The Art of Focused Conversation for Schools describes a questioning method for facilitating classroom (or faculty meeting) discussions.  I found the book pretty light-weight. It only takes a few pages to describe the method, which revolves around four levels of questioning: objective, reflective, interpretive, and decisional.  Most of the book’s pages provide detailed templates of “focused conversations”.  I found these semi-scripted “conversations” of dubious value.   The questioning method itself seems sound.  I think that it’s simplicity (only 4 levels of questioning to remember) gives it some advantage over other questioning methods that I’ve been exposed to.

My mindmap of Focused Conversations for Schools

During the winter semester of this year, I took a course on distance education and discovered Learning Networks: A Field Guide to Teaching and Learning On-Line by by Linda Harasim, Starr Roxanne Hiltz, Lucio Teles, and Murray Turoff.  I finally finished reading the book earlier this month.

I was excited by the title, “Learning Networks”.  The authors’ state based their work in the book on foundational values such as (p.241):

• “The goal of making it possible for anyone, anywhere, at any time, at any age to engage in the learning process.”
• “The value to the learning process of active and collaborative learning.”
• “The ability of computer mediated communication systems to support the full range of human and social relationships.”
• “The belief that improvement of society is tied to a concept of lifelong learning.”

These beliefs are consistent with the concept of personal learning networks, but the book the didn’t provide much insight on informal learning.  Instead, it’s advice was more relevant for the use of learning management systems and the design of distance education courses.

Harasim et. al. identified three models of networked learning in higher education, training and informal learning.  Adjunct mode was the most common use of learning networks in 1994, the date of the book’s publication, and based on my experience studying the use of course management systems, it probably still is.  In adjunct mode, learning networks are typically optional and are used to extend classroom debate, increase access to the instructor, and submit assignments electronically.  In mixed mode, learning networks are fully integrated into the curriculum and are a regular part of course activities and the course grade.  In online mode, computer mediated communication provides the primary environment for course dicscussion and interaction.  The online course that I am teaching this semester provides an example of using learning networks in online mode.

According to Harasim et. al., there are seven models of learning approaches that one can take in the design of learning networks:

1. Electure (Each of my lessons includes a 3-6 minute video, presentation, and/or podcast orienting students to the week’s topic and activities)
2. Ask-an-expert (I’m not explicitly using in my course, though I am going to encourage my students to contact practicing educators by commenting on blogs.)
3. Mentorship (I have not assigned my students to mentors or asked them to find mentors on their own.)
4. Tutor support (I’m providing my students with online office hours through instant messaging)
5. Access to relevant information (I’ve provided links to help guides and to reference materials for each topic.  I’m also trying to suggest search terms for individual students to pursue on their own.)
6. Informal peer interaction (I have provided student discussion groups through a Ning network to encourage informal peer interaction.)
7. Structured group activity (I am requiring my students to participate in several structured group activites before they start working on thier individual projects.)

The authors take time to elaborate on several types of structured group activities including

• seminars
• small group discussions (I’m using this one)
• learning partnerships or dyads
• student work groups and learning circles (I’m using this one)
• team presentations and teaching by the learners (kind of using this one–might try to use it more in future semesters)
• simulation or role play
• debating teams

I found the advice in this book to be very useful for planning distance education lessons.  I also think that the book provides sound advice for instructors looking for ways to improve their use of course management systems.

On page 273, the authors predict that “Network learners of the future will have access to formal and informal education of their choice, wherever they are located, wherever they are able to participate – early morning, during the day, or late at night.”  It’s been 14 years since they wrote that statement and I think their prediction has proven correct.  However, the quality and quantity of these opportunities is still somewhat limited.  The authors’ lament on page 238 that “These activities . . . have not yet been incoporated into a strategic plan to restructure schooling.  They occur as a result of the initiative of some teachers, librarians, and staff,” is still true.  I’d really like to be part of a strategic plan to restructure schooling.  Where would we start?

My mindmap of Learning Networks

Image via Wikipedia

I was hoping to get my hands on a copy of The Cognitive Style of PowerPoint: Pitching Out Corrupts Within, Second Edition because I wanted to read his perspective on presentations before designing a lesson on teaching with presentations.  When I discovered that the copy at my local library was in the reference section and not available for checkout, I decided to return another day to read it and selected Tufte’s Envisioning Information in the meantime.

Envisioning Information examines the effectiveness of various solutions to the problem of displaying multivariate data in 2 dimensions.  Tufte suggests that some of the more successful solutions involve one or more of the following

• small, multiple images
• layering and separation of data
• micro/macro displays

Tufte also sprinkles advice and warnings throughout the book.  For example, in a vertical list in a tabular display, you should put high impact information at the top and bottom.  He lectures against “chartjunk” and the posterization of data, which he describes as data-thin graphs hiding behind cute concepts.

Envisioning Information is a quick read filled, as you might expect, with numerous images.  Still, I found the book thought-provoking, especially in the early chapters.  On page 34, Tufte asks rhetorically, “Who would trust a chart that looks like a video game?”  He raises this question in relation to a discussion of chartjunk and data posterization, and I expect he meant readers to think “No one!” when they came to the question.  However, I found myself thinking, “I might . . . depending on which video game you’re talking about.”

On page 31 Tufte states, “Unlike speech, visual displays are simultaneously a wideband and a perceiver-controllable channel.”  In contrast, Tufte describes speech as linear, nonreversible and one dimensional.  Would Tufte argue that written or recorded speech is still linear, nonreversible and one dimensional?  I’m not sure that I would.  Perceivers can rewind recorded speech, or leaf back through a book.  Hypertext and hypermedia do not necessitate a linear flow of information.

On page 32, Tufte praises an example of a compact display of 1,826 days of weather history and then states  “Emaciated data-thin designs, in contrast, provoke suspicions and rightfully so about the quality of measurement and analysis.”  Do “emaciated data-thin designs” really “provoke suspicion”.  I think perhaps they should, but I believe that the majority of the data displays that most people experience are data-thin.  Additionally, many data-thin displays are housed in familiar display templates such as pie charts, tables, and bar graphs.  At first glance, I found some of the data-rich designs that Tufte presented to be unsettling because of their unfamiliar format.  On further inspection, I was able to unlock the keys to the information contained, but I found myself wondering; how many people have a greater suspicion of an unfamiliar data-rich format than of a familiar but data-thin format?

I also noticed that many of the designs that earned Tufte’s harshest critiques bore a marked similarity to my memory of the charts, tables, and bus schedules on the standardized exams for basic literacy and numeracy that I took in ninth grade.  According to Tufte, these information displays were difficult to interpret because of poor design.  Is it fair to test children on their ability to interpret poorly designed information displays?  I’m not sure, but I fear that the ability to interpret poorly designed information displays may be a vital skill in the “real world”.

It may seem like hubris to mindmap a book entitled Envisioning Information , but I attempted it anyway.  My scribblings violate some, perhaps even many, of the principles that Tufte was trying to communicate in his book.  However, the purpose of my mindmap differs from the purpose of the information displays that Tufte critiques in his book.  The primary purpose of my mindmap is to help me organize my thoughts and impressions, while the primary purpose of the information displays discussed in Envisioning Information was to communicate data.  I post this here mostly for my own future reference.

When I was traveling to and from Asia, I had a lot of time on planes and trains and I used most of that time to read.  I’ve already posted reviews of some of the other books that I read and I decided to give Blink: The Power of Thinking Without Thinking by Malcolm Gladwell, the same treatment.  My notes for Blink are less detailed than my notes for some of the other books that I have read, notably Diffusion of Innovations. This is partially because I waited longer between finishing the book and organizing my notes, but mostly because Blink is a lighter read than Diffusion of Innovations.

In Blink, Gladwell described some of the processes involved in rapid cognition and proposed that much of our thinking is a result of such rapid cognition.  Whether we want to admit it or not, in a blink of an eye, we make quick judgments about people and situations.  Gladwell begins the book by describing situations where the “gut feelings” of experts proved to be more accurate than seemingly thorough investigations by skilled individuals.  In later chapters he delves deeper into mechanisms by which rapid cognition functions.  One such mechanism is thin-slicing.  Thin slicing involves recognition of only the key bits of information that matter when making a judgement.  Gladwell uses the example of a researcher who can predict the likelihood of divorce after viewing a married couple’s interactions for five minutes.  The researcher is able to thin-slice by looking for signs of contempt, stonewalling, defensiveness, etc. in the couple’s interaction.  Gladwell himself attempted to make predictions based on the same data and failed, which leads to one of the key points in Blink.  While people make many rapid judgments, only experts are consistently capabable of  coming up with accurate judgments.  The dark side of rapid cognition is predjudice.  We can be fooled more easily than we like to admit.

I’ve heard of the book Think!: Why Crucial Decisions Can’t Be Made in the Blink of an Eye by Michael LeGault and have not yet read it.  However, from the product description on Amazon, it seems clear that it was written in response to Blink.  From what I can gather, LeGault is offering a counter argument to an argument that Gladwell never makes in Blink.  Think seems to claim that Gladwell encourages people not to think when making decisions.  Instead, Gladwell encourages people to be aware of the power rapid cognition or gut reactions in their decision-making process.  This awareness requires people to acknowledge that rapid cognition can interfere with good decision making at least as often as it can aid.

My mindmap of Blink

My take on the networked nature of information

In today’s world, the documents will likely be electronic, they may even be recordings in video or audio format, and the people may be on Twitter, but it is still the same process, just sped up.

I worked for a year as a Research Assistant at the Utah Office of Legislative Research and General Counsel.  I was frequently asked to collect seemingly obscure data to help shed light on one policy issue or another.  Through a combination of Google, email, and telephone calls, I usually found what I was looking for.  I was good enough at my job that as I was leaving, my immediate supervisor suggested that I might return and offer a training workshop to the rest of the office.  I shied away from the opportunity partially because I knew I’d be busy in my new responsibilities and partially because I thought my advice would be so simple as to be potentially insulting.  I don’t think I had reflected enough on my research process to understand what I was doing beyond a tacit level.  At that time, I don’t think I could have offered advice other than “Just Google It.”  If given the same opportunity today, I think I would base my training on Bardach’s process.

This is a long way of saying that I don’t think that championing how to use Twitter or social networking or any of the new technologies really makes much of a difference.  I believe that many individuals lack the understanding of the complex relationship between people, documents (or recording), and information.  This is where the message needs to be.  If people truly understood the networked nature of data, the networked technologies would almost sell themselves.