Belief Structures, Common Policy Space and Health Care Reform: A Q Methodology Study

doi:10.4236/psych.2011.29143

Paper Menu >>

Journal Menu >>

Psychology

2011. Vol.2, No.9, 948-952

Belief Structures, Common Policy Space and Health Care

Reform: A Q Methodology Study

Charles Wilf

School of Business, Duquesne University, Pittsburgh, USA.

Email: [email protected]

Received

Debate on the merits of health care reform continues even after passage of the Affordable Care Act of 2010. Poll

results confirm a split along political party and associated ideological lines with democrats more supportive and

republicans generally opposed to the law. As parts of the law are now subject to increasing scrutiny, it may be

instructive to question whether a party-centered or surrogate liberal/conservative dichotomy is the best repre-

sentation of positions in the health care debate. Q Methodology reveals a more complex set of belief structures,

suggesting that a simple dichotomy is misleading in terms of the values that underlie the role of health care in

society. Five distinct belief structures were found, each with different concerns as to the purpose and potential

benefits of various health care initiatives. In addition, Q Methodology allows for the formation of a common

policy space within which all belief structures are independently in agreement in four specific areas. It is argued

that this empirically derived consensus can serve as a basis for effective political engagement and policy imple-

mentation.

Keywords: Belief Structures, Common Policy Space, Q Factor Analysis, Obamacare, Subjectivity

Introduction

Designed to increase health care coverage for much of the

United States population, improve accountability of insurance

providers and decrease costs, the Affordable Care Act became

law in March, 2010. Some provisions of the law were imple-

mented immediately, and provisions to be implemented through-

out the next decade are intended to save more than $500 billion

in Medicare beneficiary savings alone (Medicare Beneficiary

Savings and the Affordable Care A ct, 2011)1.

Yet the debate over the law’s anticipated effectiveness con-

tinues, and parts of the law, often referred to as Obamacare,

have been challenged by states and are subject to review and

possible invalidation. The political landscape that drives these

challenges is usually framed as a conflict between liberal and

conservative points of view for which we usually substitute the

democratic and republican political parties respectively. Recent

poll results show the democrats in principle remain in favor the

law which expands health care coverage, while the republicans

continue to believe that the law will neither improve health care

delivery nor reduce costs (Kaiser Health Tracking Poll, 2011).

The seemingly intractable conflict between opposing ideolo-

gies as reflected in the polls presupposes a categorization of

beliefs that may not reflect more complex differences in the

political continuum. Rather than assume that health care beliefs

are solely defined and limited to the positions espoused by the

major political parties, a systematic investigation of subjective

health care priorities may reveal more meaningful points of

view. The ability to identify alternative conceptualizations of

the health care debate may in turn lead to more appropriate

service delivery and cost solutions.

This paper argues that a liberal/conservative dichotomy is

overly simplistic and masks a more complex system of belief

structures. Through a subjectivity-based analysis of health care

opinions that uses Q Methodology, a more incisive array of

belief structures can not only pinpoint important differences

between these structures, but also suggest a common policy

space within which useful and politically feasible health care

solutions might occur.

Q Methodology

Q Methodology uses a specific experimental design to create

rank orderings of opinion statements, whereupon factor analysis

is performed to ascertain belief structures (Stephenson, 1953;

Brown, 1986, 1980; McKeown & Thomas, 1988). The theory

and technique of Q Methodology may be considered to fall

within the more general framework of social cognition (Fiske &

Taylor, 1984). In social cognition and particularly in Q Meth-

odology, a belief structure i s a subjective point of view that is 1)

communicable, 2) always advanced from a position of self-

reference, and 3) amenable to objective analysis since they do

have structure and form. Moreover, individuals have a struc-

tured set of beliefs that remain stable over time (Cook, Scioli,

& Brown, 1975).

Policy analysis consists of the identification of narratives

which describe policy dilemmas (Hampton, 2009; Dryzek,

1990). Durning (1999) and Durning & Osuna (1994) hold that

Q Methodology adds specific value by going beyond a strictly

positivist orientation so that analysts can derive deeper under-

standings. Among these is the creation of a policy space to

uncover areas of agreement in the policy debate (Hurd &

Brown, 2004). Wolf (2004), in addition to identifying unique

belief structures as a guide to pinpointing policy issues, like-

wise emphasizes similarities among belief structures as a pre-

ferred way to create a more comprehensive understanding of

potential common policy options.

1Medicare is the United States government provided health insurance pro-

gram which covers 1) persons over age 65; 2) persons under age 65 with

certain disabilities; and 3) individuals of any age with specific transplant

needs (Med icare Ben efit s, 2011). As the US p opulat ion ages and ev en unde

the new law, Medicare will become increasingly important in the provision

of health care services.

C. WILF 949

In the current Q Methodology study, a set of 36 health care

opinion statements, selected from newspaper articles and

magazine articles and letters to the editor, was evaluated by 40

subjects2 according to a specific criterion of instruction. Ac-

cording to Nunnally (1978), comparative responses only make

sense if the stimuli are from some common domain. Since pol-

icy analysts view reasoned debate as the legitimizing basis of

the democratic process (Albaek, 1995), published statements in

widely available sources are deemed to be representative of this

domain. The sample of statements was selected over a period of

several weeks from online and print versions of national and

local publications.3

The selection of sample statements is based on two theoreti-

cal criteria. The first is theoretical completeness. The model

requires that a comprehensive and representative selection of a

particular statement population be included such that the range

of possibilities is accounted for (McKeown & Thomas 1988).

The study uses three statement types which are given the labels:

“free market”-oriented statements, “socialist” or more collec-

tivist-oriented statements; and more “complex” statements that

relate health care issues to larger societal concerns. The free

market statements as they appear in the sources are generally

less favorable to a proactive health care stance, while the so-

cialist statement subset is more favorably inclined to health care

reform. The complex statement subset was included to ac-

knowledge that the public debate about health care is not al-

ways independent of other short and long term social, economic

and political concerns.

The second theoretical criterion in experiments of this type is

the degree of definiteness of the sample statements. Thompson

(1966 and discussed in Brown, 1980, and Coke & Brown, 1976)

studied the concept of definiteness in public opinion research.

He postulated a hierarchy of definiteness, consisting of three

levels-bias, wish and policy—that indicate the intensity of feel-

ing regarding opinion. A bias is a particular interpretation of

facts. A wish is a desire for a specified end or course of action.

A policy is a belief as to the best means for achieving valued

outcomes. In terms of intensity, a wish, or desired end state,

implies a bias, or a given state of affairs. Yet the reverse is not

true. Likewise, a policy implies a wish, but it represents a more

thought-out position; it is more intense, and hence more defi-

nite.

Table 1 presents the 3 × 3 factorial design indicating the nine

statement types to be included.

The model includes the nine opinion statement types each

replicated four times for a total of 36 statements in a balanced

factorial design. Four statements of each type are included as to

facilitate reliability (Brown, 1980). Three examples of state-

ment types are:

This is not about health care. It’s about government control.

(free market bias statement—fb)

The American health care system is in bad need of reforms

that will eliminate the tragedy of 46 million uninsured people.

(socialist wish statement—sw)

If we can enac t tort reform that protects p atients fr om reckless

Table 1.

Q Method opinion statement a r r a y .

Statement Type

Free-Market Socialistic Complex

Intensity Bias FB SB CB

Level Wish FW SW CW

Policy FP SP CP

doctors and doctors from reckless lawyers and juries, we reduce

the cost of healthcare tremendously. (complex policy statement—

cp)

In a 9-point “Most Agree-to—Most Disagree” continuum,

the methodology requires subjects to place two statements un-

der the Most Agree (value +4), and Most Disagree marker (–4),

three statements each under the +3 and –3 markers, four state-

ments each under the +2 and –2 markers, and six statements

each under the +1, 0, and –1 markers. Advantages of the forced

normal distribution are twofold. First, the same rating scale is

used by all respondents, and facilitates intersubjective com-

parisons. Secondly, the format forces frequent statement com-

parisons, especially at the extremes (Tetlock et al., 1992). And,

it is the extreme cases that most determine the belief structures

(Brown, 1980).

It is hypothesized that Q factor analysis of preference order-

ings will reveal otherwise nonverbalized belief structures.

Moreover, specification of the main effects (free market, so-

cialist and co mpl ex statem ent t ypes) in a m odel of this type do es

not guarantee that a set of conclusions will be revealed (Ste-

phenson, 1963). The interest lies in the interpretation of factors

(Stephenson, 19 64). There are no m eanings externall y imposed a

priori. Rather, the explanation of factors reaches into latent

belief systems (Stephenson, 1965), which, in the present case

concerns health care belief structures as subjectively construed.

Each statement has a factor score (a unique value on a scale

of –4 to +4, paralleling the original response scale) which indi-

cates the strength of belief of the officials who collectively load

on the respective factor. Defining statements for each belief

structure (factor) are those whose factor scores differ signifi-

cantly from the scores of the other belief structures on the same

statement. This indicates which statements are not only most

important to those within a given factor, but simultaneously

whether these statements are unique in importance (either in a

positive, negative, or neutral way) to that factor group. A factor

score wh ich differs fro m all other s by 2 is significa nt at the .05

level, and a factor score which differs by three is significant at

the .01 level (Brown , 1980) .

Findings

Five unique belief structures were found from the Q factor

analysis, consisting of 30 of the 40 respondents and represent-

ing 63% of the explained variance.4 The emergence of these

unique structures underscores the difficulty in formulating and

implementing acceptable health care policy.

2Since the interest of Q methodology is in the nature of the segments and the

extent to which they are similar or dissimilar, the issue of large numbers, so

fundamental to most social research, is rendered relatively unimportant. In

rinciple as well as practice, single cases can be the focus of significant

research (Brown, 1993).

3Main national sources for statements were the Wall Street Journal, The

ew York Times and Business Week magazine. As the study was con-

ducted in the Pittsburgh region, some statements were extracted from the

Pittsburgh Post-Gazette and the Pittsburgh Tribune-Review, the areas two

main newspapers.

4The remaining 10 respondents were spread over 5 additional factors. How-

ever, they contribute no unique factor scores and were they included they

would remove significant factor scores from belief structures 2 and 4. De-

mographically there are no dominant characteristics. As such their elimina-

tion fr om the model d oes n ot af fect t he f undament al 5- facto r bel ief st ructu re

derivation.

C. WILF

950

Belief Structure 1—Welfare State Activists

Ten respondents loaded on this factor which is defined by

four statements. Two significant factor scores are positive and

two are negative:

Statement 18.

Society should guarantee health care like we guarantee the

right to think and pray as you like (sw).

Factor scores: 3 –1 –2 0 –2

Statement 23.

A public option is the only option for people who have lost

jobs due to the economy and do not have affordable health

insurance (sp).

Factor scores: 1 –2 –3 –1 –1

Positive factor scores here indicate an entitlement position,

implying that government should take a proactive role in health

care. Even though the public option was only mildly supported,

all other belief structures viewed it negatively, suggesting that

even a program that was not well delineated still had support as

a matter of principle from this group.

Statement 12.

It is the responsibility of each person to pay for theirs and

their families own health care (fp).

Factor scores: –4 –1 1 –2 1

Statement 11.

The answer to the health care crisis is for people to take re-

sponsibility for their own health (fp).

Factor scores: –1 1 1 2 3

Statement 12 strongly reinforces the collectivist approach to

health care costs, and even the mild rejection of personal re-

sponsibility suggests that only activist government intervention

can provide adequate protection for the population. Note that

three of the four defining statements for the Welfare State Ac-

tivists are higher order policy level statements, representing

well-thought out positions. There are no republicans in this

group, and all educational levels are represented (high school

graduate through graduate degree).5

Belief Structure 2—Future Cost Alarmists

Eight respondents loaded on this factor, defined by three

statements.

Statement 7.

If we don’t reduce the growth in health care costs, we’ll

leave our children with a crushing tax burden (fw).

Factor scores: 0 3 0 –1 0

Statement 16.

Forty seven million Americans without health care is a fail-

ure of our society. (sb)

Factor scores: 3 1 –2 3 –1

Statement 25.

As things stand, the health care industry finds it more prof-

itable to treat chronic diseases than to prevent them (cb).

Factor scores: 2 0 2 3 –3

Statement 7 largely defines the Future Cost Alarmists. Yet

Statements 16 and 25 reinforce this belief structure. They are

concerned with the present state of affairs and the factor scores

for the Alarmists are significant in their neutrality with factor

scores of 1 and 0 compared to the more definitive positions of

the other belief structures. The Alarmists are singularly focused

on financial hardships to come. While all categories of state-

ments are represented, two of the three statements are bias-type,

or the most basic reactive class of expression. No political party

dominates this group and they are older.

Belief Structure 3—Status Quo Advocates

Seven respondents comprise this factor, defined by five

statements, two positive and three negative.

Statement 8.

Because a government-run health insurance company doesn’t

need to make a profit, privately run companies will be squeezed

out of business, and the government would have a monopoly

(fw).

Factor scores: –2 –3 3 –2 0

Statement 6.

Universal coverage undercuts the whole notion of cost con-

tainment (fw).

Factor scores: –2 –3 2 –2 –2

Statement 21.

To compete and win in a global world, no one needs the

burden of health insurance shifted from business to government

more than American business (sp).

Factor scores: 0 –2 –4 –1 0

Statement 29.

We need to address the economic inequity that underlies our

ability to achieve health care outcomes that other societies

have achieved (cw).

Factor scores: 1 1 –4 2 2

Statement 17.

The American health care system is in bad need of reforms

that will eliminate the tragedy of 46 million uninsured people

(sw).

Factor scores: 2 2 –3 2 2

Taken together, these statements suggest a distrust of sys-

temic changes in the health care system. Significant factor

scores for statements 8 and 16 reflect cost concerns, not unlike

those of the Alarmists. However, the negative factor scores for

statements 21, 29 and 17 reveal a more universal rejection of

change. If costs were the only concern of this group, it is

unlikely that statement 21 would be rated so negatively. With

one exception, the defining statements are wish, or desired end

state type of statements. Status Quo Advocates know what they

want and they do not want change. No political party dominates

this group, and all respondents have at least a college degree.

Belief Structure 4—Uncertain Interventionists

Three respondents load on this belief structure which is de-

fined by three statements.

Statement 20.

Health care should not be regarded primarily as a business,

with a main goal of increasing shareholder value (sw).

Factor scores: –1 0 1 4 –3

Statement 34.

We are going to have to invest in some kind of a system that

is walled off from politics (cp).

Factor scores: –1 2 1 –3 1

Statement 2.

This is not about health care. It’s about government control

(fb).

Factor scores: 3 –3 4 0 4

This group acknowledges the potential conflict between

business interests and the political process, but is decidedly

noncommittal regarding the role of government. While other

belief structures maintain extreme positions regarding govern-

ment and its place in health care, the Uncertain Interventionists

have yet to reconcile politics and government. And while they

5For all belief structures, males and females are almost equally represented.

C. WILF 951

reject the notion of health care as strictly a business, any alter-

native vision remains undefined. All categories and all levels of

statements are represented, reflecting this comprehensive yet

unfocused belief structure. This group has the youngest mean

age.

Belief Structure 5—Anti-Government Libertarians

While only two respondents are represented in this belief

structure, three statements with significant positive/non-nega-

tive factor scores, and six with significant negative factor scores

render it instructive in defining the health care reform parame-

ters. Especially as the debate over rescinding many provisions

of the Affordable Care Act intensifies, the quantity of defining

statements suggests that this belief structure should not be ig-

nored.

Statement 32.

What the public wants in universal health CARE, not uni-

versal health INSURANCE (cw).

Factor scores: 0 0 1 0 3

Statement 4.

Medicine is a commodity to be bought and sold (fb).

Factor scores: –2 –2 –1 –3 1

Statement 8.

Because a government-run health insurance company

doesn’t need to make a profit, privately run companies will be

squeezed out of business, and the government would have a

monopoly (fw).

Factor scores: –2 –3 3 –2 0

Statement 24.

The public option is the answer for middle class people who

are denied affordable coverage due to pre-existing conditions

(sp).

Factor scores: 1 0 –1 –1 –4

Statement 35.

Nobody should have to pay more than a fixed percentage of

their income for health insurance premiums (cp).

Factor scores: 2 2 –1 –2 –4

Statement 20.

Health care should not be regarded primarily as a business,

with a main goal of increasing shareholder value (sw).

Factor scores: –1 0 1 4 –3

Statement 22.

The federal government should provide assistance to states

to make their own plans cheaper and cover more people (sp).

Factor scores: –1 –1 0 1 –3

Statement 25.

As things stand, the health care industry finds it more prof-

itable to treat chronic diseases than to prevent them (sp).

Factor scores: 2 0 2 3 –3

Statement 9.

In healthcare what needs to happen is an emphasis on

healthy living (fp).

Factor scores: 2 3 2 1 –1

Taken together, this set of defining statements points to dis-

trust and even hostility toward government intervention in gen-

eral and health care in particular. The two statements with sig-

nificant positive factor scores suggest a muted if basic skepti-

cism, and there are no policy level statements here. Conversely,

two of the six negative statements are at the scale extreme, and

three are at the next highest. Additionally, five of the six are

policy-level statements, indicating a well-developed belief

structure. Anti-government Libertarians are more certain in

their rejection of interventionist ideas; even the seemingly non-

controversial statement 9 cannot be supported by this group, as

it suggests a course of action that in principle could be seen to

inhibit personal freedom.

Creating the Common Policy Space

The emergence of discrete belief structures via Q Methodol-

ogy highlights the breadth of diverse and often competing posi-

tions in the health care debate. This is particularly at issue when

different belief structures have significant factor scores on the

same statement. Three such statements are found here:

Statement 8.

Because a go ver nment-run health ins urance compa ny doesn’t

need to make a profit, privately run companies will be squeezed

out of business, and the government would have a monopoly

(fw).

Factor scores: –2 –3 3 –2 0

Statement 20.

Health care should not be regarded primarily as a business,

with a main goal of increasing shareholder value (sw).

Factor scores: –1 0 1 4 –3

Statement 25.

As things stand, the health care industry finds it more prof-

itable to treat chronic diseases than to prevent them (sp).

Factor scores: 2 0 2 3 –3

In each case the Anti-government Libertarians are at odds

with another belief structure. For statement 8, Libertarians dis-

agree with the Status Quo Advocates as the latter resist change

while for the Libertarians this is a non-issue. For statement 20,

the Libertarians and the Uncertain Interventionists have dia-

metrically opposite positions, while for statement 25 the Liber-

tarians and Future Cost Alarmists are most different in their

beliefs. As expected in both cases Libertarians infer that self

interest drives health care and the medical establishment. And,

as these statements are wish or policy level types, the respective

positions are more developed and thus more likely to be de-

fended by those belief structure s holding them .

While it may therefore appear that consensus on health care

reform is unattainable, there is equal empirical support for

agreement by all belief structures among a set of four state-

ments. It is these commonalities that form the basis for a com-

mon policy space and subsequent political engagement. There

are two statements with which all belief structures either agree

or are neutral:

Statement 19.

No American family should be bankrupted by catastrophic

health care bills (sw).

Factor scores: 4 4 1 3 1

Statement 31.

Business cannot keep absorbing soaring health care costs

and keep passing them on to their employees (cw).

Factor scores: 0 1 2 1 1

Statement 19 can be interpreted as a fundamental baseline in

the health care debate, as all five belief structures either

strongly or mildly agree. While statement 31 does not com-

mand the same intensity of support, no belief structure dis-

agrees with this concept. As both statements are wish or desired

end state types, there is genuine concern across the subjectivity

spectrum regarding an outcome that is sensitive to both poten-

tial individual and organizational hardship. A common policy

space begins with an awareness of this fundamental agreed-

upon position.

There are two additional statements with which all belief

C. WILF

952

structures eith er di sagree or are neutral.

Statement 3.

Health care is a luxury, just like living in a nice neighbor-

hood or driving a nice car (fb).

Factor scores: –4 –1 –3 –4 0

Statement 1.

Health care is socialism, and socialism is not an American

value (fb).

Factor scores: –3 –4 0 –4 0

Statement 3 emphasizes the universal prioritization of health

care common to all belief structures, and even the Libertarians

do not reject this position. The statement on socialism, included

so as to make the domain as comprehensive as possible, is also

either strongly rejected or ignored. The health care debate has

often been infused with inflammatory ideology to advance po-

litical agendas. However the analysis reveals an inverse asso-

ciation or indeed no association between health care and an

alternative societal paradigm. It is also significant that both

statements are fact types or lower order statements. All belief

structures show a common position at this most basic level, and

do not involve higher order goals or methods for their attain-

ment.

Conclusion

Q Methodology reveals a set of distinct belief structures that

go beyond differences found in political polls. Five belief

structures were derived from the analysis and indicate that un-

derlying ideas and concerns about health care reform cover a

more complex range than previously indicated. While labels

and hence ideologies such as liberal or conservative have been

repeatedly applied to the health care debate, Q Methodology

shows that these labels are simplistic at best, and potentially

misleading. It may seem, for example, that the conservative

label might equally apply to the Status Quo Advocates and the

Anti-Government Libertarians. But a review of their defining

statements indicates important differences between them. Con-

sider Statement 29:

We need to address the economic inequity that underlies our

ability to achieve health care outcomes that other societies

have achieved (cw).

Factor scores: 1 1 –4 2 2

The Status Quo Advocates are adamantly opposed since this

involves change, while the Libertarians take a position more

like the Welfare State Activists, the Future Cost Alarmists and

the Uncertain Interventionists, who could hardly be considered

conservatives. Yet one could argue that the Libertarians react

positively to this statement because in their belief structure it

represents an imperative for individual self-sufficiency on this

issue, just as self-sufficiency drives their other responses.

Equally important, Q Methodology has shown that despite

deep differences in underlying belief structures, there is a

common policy space that can serve as a practical starting point

for discussing health care service delivery. Protection from

catastrophic financial loss, a concern for accelerating business

health care expenditures, and a belief that access to health care

is a basic necessity are fundamental premises that every belief

structure, from Welfare State Activists to Anti-Government

Libertarians can support. As the health care debate continues

and likely intensifies as challenges to the Affordable Care Act

are presented, policy prescriptions that begin with these few

reliminary yet politically acceptable ideas may have a better

likelihood of generating meaningful and permanent health care

reform than more comprehensive programs that put opposing

belief structures against each other.

References

Albaek, E. (1995). Between knowledge and power: Utilization of social

science in public policy maki n g . Policy Sciences, 28, 79-100.

doi:10.1007/BF01000821

Brown, S. R. (1993). A primer on Q Methodology. Operant Subjec tivity,

16, 91-138.

Brown, S. R. (1986). Q technique and method. In W. D. Berry, & M. S.

Lewis-Beck (Eds.), New tools for social scientists (pp. 57-76).

Beverly Hills: Sage.

Brown, S. (1980). Political subjectivity: Applications of Q Methodology

in political science. New Haven: Yale University Press.

Cook, T. J., Scioli, F. P., & Brown, S. R. (1975). Experimental design

and Q Methodology: Improving the analysis of attitude change. Po-

litical Methodology, 2, 51-69.

Dryzek, J. S. (1990). Discursive democracy: Politics, policy, and po-

litical science. Cambridge: Cambridge University Press.

Durning, D. (1999). The transition from traditional to postpositivist

policy analysis: A role for Q-methodology. Journa l of Policy An alys is

and Management, 18, 389-41 0.

doi:10.1002/(SICI)1520-6688(199922)18:3<389::AID-PAM4>3.0.C

O;2-S

Durning, D., & Osuna, W. (1994). Policy analysts’ roles and value

orientations: An empirical investigation using Q Methodology.

Journal of Policy Analysis and Man a ge me nt , 13, 629-657.

doi:10.2307/3325491

Fiske, S., & Taylor S. (1984). Social cognition. New York: Random

House.

Hampton, G. (2009). Narrative policy analysis and the integration of

public involvement in decision making. Policy Sciences, 42, 227-242.

doi:10.1007/s11077-009-9087-1

Hurd, R. C., & Brown, S. R. (2004/2005). The future of the Q Method-

ology movement. Operant Subjectivity Journal of the International

Society for the Scientific Study of Subjectivity, 28, 58-75.

Kaiser Health Tracking Poll (July 2011). URL (last checked 17 October

2011) http://www.kff.org/kaiserpolls/8209.cfm

McKeown, B. B., & Thomas, D. (1988). Q Methodology—Quantitative

applications in the social sciences. Beverly Hills: Sage.

Medicare Beneficiary Savings and the Affordable Care Act (2011).

URL (last checked 17 October 2011)

http://www.healthcare.gov/law/resources/reports/affordablecareact.ht

Medicare Benefits (2011). URL (last checked 17 October 2011)

http://www.medicare.gov/navigation/medicare-basics/medicare-bene

fits/medicare-benefits-overview.aspx

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York:

McGraw Hill.

Stephenson, W. (1953). The study of behavior: Q-technique and its

methodology. Chicago: University of Chicago Press.

Stephenson, W. (1963). Independency and operationism in Q-sorting.

Psychological Record, 13, 269-272.

Stephenson, W. (1964). Application of Q method to measurement of

public opinion. Psycholog i ca l Record, 14, 265-273.

Stephenson, W. (1965). Definition of opinion, attitude and belief. Psy-

chological Record, 15, 281-288.

Tetlock, P. E., Peterson, R., McGuire, C., Chang, S., & Feld, P. (1992).

Assessing political and group dynamics: A test of the groupthink

model. Journal of Political and Social Psychology, 63, 403-425.

doi:10.1037/0022-3514.63.3.403

Thompson, G. C. (1966). The evaluation of public opinion. In B.

Berelson, & M. Janowitz (Eds.), Reader in public opinion and com-

munication, (2nd ed. , pp. 7-12). New York: The Free Press.

Wolf, A. (2004). The bones of a concourse. Operant Subjectivity Jour-

nal of the International Society for the Scientific Study of Subjectivity,

27, 145-165.