Coding Issues in Cognitive Mapping of Games

doi:10.4236/ojps.2013.31003

Paper Menu >>

Journal Menu >>

Open Journal of Political Science

2013. Vol.3, No.1, 16-23

Published Online January 2013 in SciRes (http://www.scirp.org/journal/ojps) http://dx.doi.org/10.4236/ojps.2013.31003

Coding Issues in Cognitive Mapping of Games

Roel Popping

Department of Sociology, University of Groningen, Groningen, The Netherlands

Email: [email protected]

Received September 17th, 2012; revised October 28th, 2012; accepted November 13th, 2012

In text analysis studies coders have to make qualitative decisions. These decisions are based on interpreta-

tions of the texts under study. In such situations it is very helpful to have coding rules. These do not only

help as an aid to the coder, but are also useful for readers of the research report that will follow. The rules

make visible in considerable extent how the coding task has been performed, they take care of transpar-

ency. This contribution focuses on motions that have been treated in the Dutch House of Representatives.

Motions usually contain information on why they are needed, the proposing member usually also tells

about it. There is a discussion with the secretary, who is supposed to put the motion into effect if it is ac-

cepted. The secretary even has to give an advice. It is assumed that under these discussion(s) a cognitive

map containing some game theoretic representation can be found. Rules are discussed that are used to

code the types of maps that might be found.

Keywords: Coding; Text Analysis; Content Analysis; Motions

Introduction

Many types of text are available that are not exploited yet.

Looking at political debates we have among others party mani-

festos, speeches, debates. Party manifestos allow investigating

parties with respect to their place on a left-right continuum

(Laver & Garry, 2000) or on a scale denoting a relevant char-

acteristic. This in its turn allows discovering how a party

changes position over time. Slapin and Proksch (2008) demon-

strate changes in economic, societal and foreign policy for sev-

eral political parties in Germany. Speeches might be speeches

by party officials, but also speeches by members of the parlia-

ment or members of the government. McLean and Patterson

(2007) analyzed speeches on Iraq by the British Prime minister,

who was at that time also party leader. Debates might be de-

bates at a party meeting, but also a debate in democratic gov-

ernment’s legislature like a parliament between members of

different parties or between such members and the government.

Bara, Weale and Bicquelet (2007), who actually compare the

results found by using two different computer programs, con-

sider the parliamentary debate. They look for dominant themes

in the debates and try to find out how far speakers who take

different positions follow a distinct pattern of discourse. Very

often, certainly in governmental bodies, transcripts of such

speeches or debates are available. Harris (1996) summarizes

studies in which content analysis was used to understand nego-

tiations. She showed that in these studies there has been a lot of

attention for positive and negative behaviors and that also style

and affect received attention. A disadvantage of discussions in

general however is that it is unclear when a party will come

with a reaction on an issue or statement; it is even possible that

there will be no reaction at all.

In debates in parliament sometimes typical formats are used

for discussing issues. In many countries parliamentarians have

the right to ask questions, to propose amendments to laws that

are under construction or to propose motions. A motion is an

official statement in which one or more members of the parlia-

ment (usually as representative(s) of one or more parties) ask

something (usually a change in policy) from the government. In

the Netherlands the motion is pre-structured; it consists of a

consideration which results in a request. In addition, before the

parliament will vote upon the motion, the government is by law

obliged to give an advice.

In order to really exploit the value of motions one can look at

what it is about. Traditional content analysis can be useful here.

Now one gets information on themes that are discussed, more

interesting might be the possibility to get information on how

themes are discussed. The study on how they are discussed

gives insight in negotiation and decision processes.

However, there is more. The motion hides stations of game

theoretic models, which inform about the structure of the nego-

tiations and decisions. Traditional content analysis does not

show this directly, but can help in finding these models. Game

theoretic models can be represented by using cognitive maps,

mathematical models of belief systems.

The purpose of this text is to indicate that such representa-

tions can be found in a structured way by applying text analysis.

Actor’s utilities with respect to two situations have to be coded

as positive or negative. Based on these codings the game can be

determined. The remaining part of the text is on the coding

process. How should one look at a text, what should be taken

into account to come to a judgment. The decisions made by the

coders in the investigation have to be clear and transparent

(Popping & Roberts, 2009). The text does not deal with a con-

crete analysis of representations of models found. This will

come at some other place.

Cognitive Maps of Games

A cognitive map is “designed to capture the structure of the

causal assertions of a person with respect to a particular policy

domain” (Axelrod, 1979, p. 58). The map consists on the one

hand of concepts that are treated as variables. These variables

refer to entities that can take different values (e.g. “education”

R. POPPING

can be high or low; a campaign can be successful or unsuc-

cessful, etc.). On the other hand the map consists of causal be-

liefs, expressed as assertions. These are found in the relations

between the concept variables.

For each concept, it can be determined whether it affects the

utility of an interdependent actor negatively or positively. The

concepts are pictured as nodes in a network and the relation

between the concepts as a line. The actors can also be presented

as nodes, and the lines from concept to actor are used to express

the utility the concepts have for the actor. Depending on the

values the relations take, it is possible to identify which game is

going on. Figure 1 contains the representation of a pure com-

petition game. Here Uh and Ug refer to the actors. Later on in

the text two actors will be used: the member of the House of

Representatives and the member of the government.

The type of game that is at hand depends on the values the

lines take. Types relevant for analyses of the data are the pure

competition, the social dilemma, the coordination, and the con-

sensus game. However, as this text concerns the coding and not

the analysis, neither treatment of the games is given, nor further

background regarding the substantial research questions. Possi-

bilities of the type of research have been discussed by Holsti

(1976). Among others Anthony, Heckathorn and Maser (1994)

performed research in which the course of negotiations is stud-

ied by using these game theoretic representations. The inter-

ested reader is referred to these publications.

Looking at legislative bodies there are several parties taking

a position with respect to a certain concept. In this study mo-

tions as they play a role in the Dutch House of Representatives

are used. The greater part of the motions is directed to the gov-

ernment. Therefore there are two actors: the House and the gov-

ernment. Within the House however, many subgroups might be

distinguished. A member of the House proposes a motion. Usu-

ally the motion pictures a perceived reality that is judged as

being wrong, and next a solution is proposed that is judged as

positive (or at least as a first step in a positive direction). By

law the government represented by the secretary under whose

responsibility the issue falls has to reflect on the motion. This

can be read as judging the perceived reality position. This re-

flection takes place in a debate in the House. The secretary also

has to give a recommendation, has to advice the members of the

House to be against or in favor. Besides, the secretary has to

present some argumentation for the advice. This advice is given

during the debate.

Studying motions is a way to investigate the structure of ne-

gotiation processes. A motion is proposed by a member of a

coalition or an opposition party. Different game theoretic rep-

−

Figure 1.

Cognitive map of a pure competition game (Ac

= Anarchy Node for the consideration part, Ar =

Anarchy Node for the request part, Uh = Utility

of the member of the House, Ug = Utility of the

government).

resentations are possible given who is proposing and that what

is asked for in combination with the reply by the government.

Motions might also be proposed by several members of the

House. These can even be members of both coalition and oppo-

sition parties. These members will be united in their view on

the issue under discussion. This might among others be related

to the ideology the parties have. Motions might also be used to

investigate other types of research questions.

The task of the investigator is to find documents in which the

motion is presented and (usually other) documents containing

the transcript of the debate about this motion. In the debate the

members have a certain amount of time to speak, during this

time or in the general debate afterwards they propose their mo-

tion. When all speakers have finished the secretary gets the

opportunity to answer. In the answer the government’s position

is presented and at the end also an advice is given.

When the motion is addressed to the government one might

say that the member of the House first is negative (there is

something to complain about) and comes with a solution that is

positive. For the secretary, representing the government, this is

not that simple. A secretary might have opinions different from

the ones of the member of the House, but the secretary might

also be limited in possible positions due to the coalition agree-

ment. What is documented in this agreement might be different

from the position the secretary takes as a private person or as a

member of a specific political party. The secretary has to an-

swer in the role of member of the government.

Motions

In the Netherlands a motion usually contains four elements:

an introduction, a consideration, a request or judgment and a

formal closure. An opinion or an observation is treated as a

special form of a consideration. Several considerations and

requests are allowed in one motion. Below follow two exam-

ples of a motion. These motions will be used as examples in the

text.

The House, having heard the deliberation, considering that

the National Influenza Center collaborating with WHO

(World Health Organization) has to be an independent

agency, requests the government to accommodate the Na-

tional Influenza Center at the National Institute for Public

Health and the Environment (RIVM), and returns to the

order of the day (Motion 22894, nr. 244).

The second motion:

The House, having heard de deliberations, considering

that in the master plan prison system 2009-2014 has also

been recorded that some institutions having limited guard-

ing (ILG) and some having very limited guarding (IVLG)

will be closed, considering that from several (international)

researches it appears that the chances on repetition by

prisoners after dismissal from open and half-open, less

protected institutions are lower than after dismissal from

more closed and more protected institutions, moreover

considering that places in ILG and IVLG are cheaper than

places in closed institutions with a regular guard level,

requests the government to postpone the planned closing

of ILG and IVLG, and to investigate how these valuable

places for the prison system can be remained preserved,

and returns to the order of the day (Motion 24587, nr.

R. POPPING

361).

Motions are directed towards the government or towards the

House itself. In the second case the motion contains a call to

have the House investigate something itself, a word of gratitude

to a committee that did such or a call containing a judgment

about the government or a secretary (normally the government

or the secretary has to go away). This type of motions is not

relevant for the present study. Motions are needed that are ad-

dressed towards the government.

A motion might be framed by the proposer, a specific con-

sideration or the request can be over-emphasized in order to

attach the attention of the media.

Data

The present data set contains all motions that have been pro-

posed between September 2009 and February 2010 or between

September 2010 and February 2011. These time intervals are

the ones in which the House deals especially with the budget

for each department. If the House wants to influence the gov-

ernment’s policy, then it should be done here.

In these intervals 2719 motions have been proposed, 2149 of

these motions have been put to the vote. A stratified random

sample of 729 motions from the ones that have been voted on

are coded with respect to the positions as have been discussed

before. Stratification is based on dividing the motions in 10

equally sized groups based on time of proposal.

Finding the motions and the debates is not that difficult. The

Dutch government has a good website (officielebekendmakin-

gen.nl) where all information can be found. Motions are in se-

parate documents that can be recognized based on the heading.

One has to learn however how to recognize, based on the date

and heading, whether a document (probably) will contain the

transcription of the debate preceding the proposing of the mo-

tion that is looked for and the reaction by the secretary after-

wards. Usually this all is reported in one document. Transcrip-

tions of all debates are presented on the website. Here content

analysis comes in. The problem is how to code the position the

secretary takes first, the advice and the reason of the advice.

For this a number of guidelines can be formulated. With respect

to the position taken first however, one must be careful, often

the decision is based on an interpretation of how things are said.

The guidelines are very important.

Wijnja (2010) investigated qualitative characteristics of the

motions proposed in the Dutch House of Representatives be-

tween September 2009 and February 2010. Out of the 181 mo-

tions (20% of the total) that have been investigated, only two

contained no request part. In 135 motions both the logical and

causal argumentation to come to what was requested seemed

correct on face value (note, motions do not have an “if … then”

structure), in 29 motions one of these two were weak, and in 15

motions both were weak. The fact that 75% of the motions were

judged as having a correct formulation gives a good basis for

assuming that it should be possible to find the suggested cogni-

tive maps in the motions. The formulation of the consideration

part in many cases demands that knowledge regarding the de-

bate in which the motion is presented is a necessity.

Content Analysis

The dominant social scientific methodologies for the quanti-

tative analysis of texts are ones in which statistics are applied to

data consisting of word-counts within thematic categories (Krip-

pendorff, 2004). These text analyses are part of a broader class

of methodologies called “content analysis”—a generic term for

statistical analyses of qualitative data (e.g., words, gestures, art

forms, etc.).

Instead of thematic categories one might also use concepts. A

concept is “a single idea, or ideational kernel, regardless [of

whether] it is represented by a single word or a phrase” (Carley,

1993, p. 81). Our research situation is close to this one. The

filling out of the concepts however is complex. Motions start

from a present manifestation of an issue as is found in reality

according to the proposer; it refers to the issue that needs im-

provement. The request part refers to the same issue but after

improvement again according to the proposer, therefore this is a

future manifestation. To be coded is whether proposer and gov-

ernment are positive or negative about the present and the fu-

ture view.

Roberts (1989) distinguished between four types on inten-

tions a clause can have: a description or a judgment about a

state of affairs or a process. The consideration part in a motion

contains one or more descriptions of states of affairs and proc-

esses necessary to build up an argument, and judgments about

both in order to express an opinion. The request part contains a

judgment concerning the improved issue. In order to get a mo-

tion completely clear it might be necessary to rephrase is, espe-

cially to fill in the implicit statements. See later.

roposer's present

Theattitude towards the

government's improved

negative

view ispossitive









 











Motions are a direct source of the information to be analyzed,

they contain the text produced by the sender (the proposer), and

there is no interpretation, transformation or whatever in be-

tween. Motions contain requests. This invites to ally modality

analysis as for example in Roberts et al. (2009). In this type of

analysis one looks at what in our case the government can (pos-

sible), must (inevitable) or must not (impossible) do. Due to the

required composition of motions however it is not possible to

distinguish between possible and inevitable. This would have

been possible when transcripts of the debates are studied.

The investigator also has to decide on whether concrete con-

cepts are to be recorded or not as in Shapiro and Markoff

(1998). On the one hand it is not necessary for finding the cog-

nitive maps. On the other hand each motion is part of a discus-

sion, in the times under investigation this usually was a discus-

sion on a department’s budget. The discussions are already

labeled by the departments responsible for organizing the

House’s meetings.

Lines Representing the Utilities

First one needs to recognize the concept nodes in a motion

and their relation. As soon as these are specified, one can look

at the positions taken by the actors in the motion. One has to

look at the links between the two concept nodes and the two

actor nodes separately. These will be valued and in that way

indicate the positions taken by the member of the House and by

the government with respect to the consideration part concept,

referring to the present situation, and to the request part concept,

referring to the desired situation. Each time context information

R. POPPING

might be necessary to decide on the position taken.

Relating the Member of the House to the

Consideration Part Concept Node

The position taken by the party or the Member of the House

of Representatives responsible for the motion is easiest to grasp.

This is so because the core of the necessary information is to be

found in the motion; usually the perceived present situation is

depicted as bad. In the first motion presented before the pro-

posing member states the influenza center has to be an inde-

pendent institution. The way the motion is formulated shows

that the proposer has doubts on whether this will really be the

case in the actual situation. Therefore there is a negative atti-

tude with respect to the present situation.

As far as the second motion is concerned, the member has

stated that the available knowledge regarding people in prison

is to be judged in a positive way and the closing in a negative

one. The closing is an act the government wants to undertake

according to the proposer; therefore it is the concept of interest.

The member is negative about the closing.

Sometimes the actual wording of the motion suggests that the

government does well, but given the request part of the motion

it becomes clear that actually something is to be added, namely:

“… but the government does not tell how to proceed or how she

will proceed.” In the request part of the motion follows a sug-

gestion on how to do it. On the other hand there are also mo-

tions suggesting there is no policy, the government does not do

anything. In her reaction later on the government will explain

whether this view is correct or not (see the section where the

government is related to the consideration part of the motion).

A concrete example: Motion 32123-x, nr. 63 observes that the

budget for a project has been exceeded, therefore the govern-

ment is encouraged to start new negotiations with the producer

in order to cut back the final price. In fact two observations are

made: 1) the budget has exceeded, and 2) the government does

not start new negotiations, as well as a consideration: the gov-

ernment has to start the negotiations. It looked as if the consid-

eration part is neutral, but given the interpretation of the con-

sideration, it is viewed as negative.

Relating the Member of the House of Representatives

to the Request Part Concept Node

In the consideration part of the motion the message generally

is that something is wrong. The request that follows asks to

take care of improvements, totally or only with respect to the

next step to come in the process that is under discussion. This is

considered as positive. The member does not always question

whether that what is asked for is possible by law or is realistic.

Neither does the member always know what is going on at the

side of the government. This is illustrated by an answer given

by a secretary. In motion 22894, nr. 243 the government is

asked to come with an index in which the relations between

pharmaceutical companies and physicians are documented. The

secretary: “I already informed the House that a sunshine act

will take place that goes further than what you ask for. I do not

only want to make visible the contacts, I want that the financial

relations are completely visible, we need complete transpar-

ency.”

The member in the first example motion asks to accommo-

date the center at a certain institution. Implicitly it is stated that

this guarantees independence. This part of the motion can al-

ways be understood from the text in the motion itself. Context

information might be necessary to find an explanation for some

terminology. Also the position taken by the member of the

House with regard to the request part concept in the second

example motion can be read from the motion.

Relating the Government to the Request Part

Concept Node

Finding the government’s position regarding the motion is

difficult as it is not part of the motion itself. The position must

be looked for in the transcription of the debate. The secretary

advices the House about the motion during the debate. This

advice is not always formulated in a distinct way. In the advice

generally the government’s position with respect to the request

part concept is explained. Finding the view on the consideration

part usually is more difficult. Therefore here the request part is

considered first. The advice is not always in the debate, some-

times it is presented in a letter send to the House after the de-

bate. Such a letter is most clear. It really informs whether a mo-

tion is supported or dissuaded.

Looking at the first motion it turns out that the government

has no authority with respect to where to accommodate the

center. Therefore the motion is dissuaded. This indicates a ne-

gative relation. After this advice was given, the proposer has

withdrawn the motion. The secretary is negative with respect to

the second example motion because the proposer did not start

from the correct information. The measure’s goal is not reduc-

tion of capacity, but re-socialization. The motion was rejected

after the voting.

The government can be either positive or negative with re-

spect to a motion. A motion might support the policy making.

According to the secretary motion 32123-xiv, nr. 82, asks for

something that already takes place. But it might also be that the

policy just started or even has to start. The secretary tells that

she will try to use a reverse burden of proof in case of poaching

what was asked for in motion 32500-xiii, nr. 50. In both cases

the government’s view on the request part is positive.

A negative view is found in very different situations. The

judgment is based on very different arguments however. The

secretary does not agree with the accusation in motion 32500-vi,

nr. 48 that policy with regard to vulnerable groups is not bal-

anced but biased and merciless. With respect to motion 21501-

32, nr. 366 about the government’s attitude and effort at the

amendment of the European directives for experiments on ani-

mals the secretary even insists that the claim has no ground and

that the country plays an active role when this kind of questions

are discussed.

Motion 31322, nr. 110 asks for an increase in the tax burden

for employers, which is not in line with the coalition agreement

saying that a reduction is to be realized. The secretary also

might be unwilling to change a practice that works fine. There-

fore she does not support motion 32123-viii, nr. 25 which re-

quests new methods for testing on conflicts of interest at finan-

cial funds, the present system of peer review works well. In

case something new is to be realized one has to make a decision

and cannot always continue with new research. For this reason

the secretary opposes motion 31209, nr. 106, a lot of research

has been done and that resulted in the bill as it is now.

Sometimes a motion is premature. In motion 32123-xiv, nr.

119 the government is asked to keep the promise that what is in

R. POPPING

a certain law has been realized within three years. The advice

was to wait for three years and otherwise not to follow the mo-

tion. More generally, in principle the government agrees with

that what is asked for (is positive with respect to the considera-

tion part), but nevertheless at the moment she is negative as

some lacking knowledge must be filled in. This might mean

that something related to the content of the motion is studied

and one wants to wait for the results or that something (compa-

rable to or part of what is asked for) is tried out, the results

however are not known yet. It is also possible that a motion is

dissuaded, because what was asked for has already been real-

ized, or in reverse is unrealistic. According to the secretary mo-

tion 32500-vi, nr. 60 is unnecessary, as she already promised

what is asked for in the debate. Motion 32474, nr. 16 is not rea-

listic. The secretary is unable to give a guarantee that is asked

for.

Finally motions can have unwanted or unexpected conse-

quences. Motion 32123-xiv, nr. 103 asks for more air cleaners,

but these contribute to the pollution of the environment and the

secretary does not want this.

Coders have to be careful, it occurs that a negative advice is

given, but that the secretary adds that if some parts in the mo-

tion are changed he or she might become positive.

Arguments for not supporting a motion are the most inter-

esting ones. One is not only confronted with different political

positions. An argument often heard is that the motion is not ne-

cessary, because the secretary already promised to solve the

problem. Looking at the relation between government and re-

quest part concept node one runs into a difficult choice now:

the “not necessary” suggests a negative relation (and is coded

as such), but the promise tells that the government is positive.

In summary reasons for suggesting not accepting a motion are:

 the issue is not in the field covered by the secretary;

 the house is not permitted to decide on the issue;

 the government is not permitted to decide on the issue;

 the government cannot force others to do what is asked for;

 the issue can be solved in another way than proposed;

 the information in the motion is not clear or not correct;

 unwanted consequences are possible;

 not enough time available (as there is a final date).

Relating the Government to the Consideration Part

Concept Node

In his or her response the secretary most of all reflects on the

request part of the motion. Sometimes also a position is taken

with respect to the consideration part, but in most cases the

coder has to read this position from the context (how are things

said) or from what the secretary did say earlier in the debate,

even before it was clear that a motion will be proposed. This

implies that coding the government’s view on the consideration

part of the motion requires an interpretation. The coder some-

times only has the feeling the secretary is positive or negative.

The secretary’s view can be based on several aspects: a fact

(state of affairs) as presented might be incorrect or an opinion

might not be agreed upon. The coder has to be extra careful; the

secretary sometimes gives not only the position as taken by the

government, but also a personal opinion. These two might be

different. In connection with motion 32356, nr. 15, which is

about duties of a person who is over 18, the secretary brings up

to date that in her role as a mother the motion has her sympathy.

As a mother she would consider the person still as a child. She

even mentions that she probably would have supported the

motion if she were a member of the House. As a secretary how-

ever, responsible for the country’s policymaking, she has to

take another position. She has to look at the child as being a

grown person. It might also be that the secretary is limited by

law. Motion 32500-viii, nr. 110 asks for a decrease in VAT rate,

but given European rules, the secretary has no possibility to

make changes. The government agreed with these rules, there-

fore the government has to be positive.

The center that is referred to in the example motion has to be

independent. According to the proposer this can only be real-

ized by the government. The secretary starts her answer by

explaining that it has been a decision by the WHO to do it in

the way it is achieved now and that the government cannot

overrule this decision. This is coded as that the government is

positive with regard to the way the present situation is.

With respect to the second example motion on the prison

system the secretary answers that she already indicated in the

debate that she is focusing on re-socialization of prisoners and

that it is not her intention to reduce capacity. She emphasized

the importance of the fact that prisoners are in a detention cen-

ter close to their own place of living. Therefore the government

is positive with regard to the present situation.

Regarding many issues the government will indicate that she

has some concrete action or policy in mind, but that it is neces-

sary to wait for the right moment to show this. This often hap-

pens with regard to positions to be taken in negotiations within

the European government. This is read as that the government

is positive about what is stated in the consideration part. Often

other situations close to this one are found. In motion 27428, nr.

155 the government is invited to use as point of departure in a

proposal about allowing the growth of gene plants the position

that pollution across borders with genetically modified organ-

isms should be prevented. The secretary interprets this as sup-

port for the present policy; the government is positive with

regard to the consideration part.

As regards a number of motions the secretary states some-

thing like “… it is well known we are working on it. We have

had this debate before.” It might mean the debate was earlier in

the same meeting, but it can also be in a meeting before. It is

relevant to have the information on the debate. The “we are

working on it” can mean that the government is already taking

action, which implies she has already demonstrated to be posi-

tive. It might also mean that the government has the intention to

work on it, but also that the government does not want what is

asked for and that in her view a repetition has started. The

member did not get what was wanted in the debate, therefore he

or she tries it once more by means of the motion. In that case

the government is negative with respect to the consideration

part. Another interpretation however is also possible. The pro-

blem, the consideration part, is acknowledged, but the proposed

solution is not supported.

Sometimes the government is negative with respect to the

consideration part because of non-political arguments, what is

discussed is not part of the responsibility of the secretary or is

not part of the debate.

When all positions with regard to the example motions pre-

sented are combined, it becomes clear that the games going on

here are pure competition games. The proposer is negative with

regard to the coordination part and positive with regard to the

request part. For the government the reverse is found.

The first motion has two parts, a consideration and a request

R. POPPING

part. Therefore at most only one game is possible. The second

motion has three consideration parts, therefore three games

might be found.The first consideration part of the motion in-

forms that there is a plan to close prisons. The concept cannot

be the cause for the concept asking for delay. So, here is no

game. The second part was treated above. The third part con-

tains the information that open and half-open prisons are chea-

per than closed prisons. Therefore, the open and half-open pri-

sons should not be closed. Here the reasoning is the same as

with respect to the second part, we have a second pure competi-

tion game. It is up to the investigator to decide whether all

separate parts should be coded or not. We finally used one

score for a motion and did not use sub-units. The resulting fre-

quency distribution of the games is in Table 1.

A Note on Interrater Reliability

The coding task performed is complex and has to be carried

out by human coders. For each motion the coders have to make

five decisions:

 four decisions concern the position of House and govern-

ment with respect to the two concepts;

 the fifth decision concerns the argument used by the gov-

ernment that is at the basis of the position taken with re-

spect to the request part concept.

When the first four decisions are known, it is quite easy to

determine which game is represented in the corresponding map.

In order to get an indication of how well the coders have per-

formed their tasks a random sample of all motions that had

been voted about was independently coded by both coders. This

occurred after instruction had taken place.

The coders received instruction on how to perform the task.

In this instruction thoughts as have been discussed above have

been used. After that the coders were asked to code a sample of

30 motions which had been withdrawn by the proposer, a fact

the coders did not know. The coders coded these motions inde-

pendently; the investigator also performed the task. A discus-

sion on the differences in coding followed. Next the task was

performed. The coders each time received a small set with mo-

tions that were a representative sample of the complete set that

has been voted for. Having coded this set, they received the

next one. A motion could only be part of one set. Almost half

of the sets were coded by both coders, the other sets only by

one coder. For the analysis the index pi (Scott 1955; Popping

2010) is used as a measure of intercoder agreement. The meas-

ure uses the marginal distribution over all coders as the basis

for computing deviations from expected agreement; therefore it

does not assume a priori differences between the coders. The

index is appropriate when comparing a large number of assign-

ments among trained coders. The amount of agreement found is

presented in Table 2.

After the try-out investigator and coders spoke together about

these codings and especially about the why of the differences.

From the moment the coders indicated that all was understood

they started performing the real task. The outcomes in the table

show that the amount of agreement is always over .80, which is

usually considered as a very good result. It also indicates that

the coders having talked together must have been very impor-

tant for getting the correct view on the task. The most difficult

part in the coding task was the finding of the position taken by

the government with respect to the consideration part in the

motion.

Table 1.

Frequency distribution of the games.

House Government

Label

consid. req. consid. req.

Freq.

Social dilemma gameneg pos neg neg 121

Pure competition gameneg pos pos neg 374

Coordination game neg pos neg pos 156

Battle of the sexes gameneg pos pos pos 78

Total 729

Table 2.

Pairwise agreement-results for the coders.

π z p N

Try out 30

Gov. consid. part concept .71 14.96 0

Gov. request part concept .85 25.17 0 Coders

Argument .87 46.46 0

Gov. consid. part concept .80 29.39 0

Gov. request part concept .83 41.84 0

Coders +

Investigator

Argument .87 83.63 0

Final 303

Gov. consid. part concept .85 189.33 0

Gov. request part concept .96 272.43 0 Coders

Argument .84 439.71 0

The final score for each position or argument is the score

given by the coder. In those situations where both coders per-

formed the coding and where a different opinion was found, a

final decision was made by the main investigator. The longer

the coders had been performing the coding task the less differ-

ences in assigning were found.

Conclusion

This paper discussed the coding process to be followed by

researchers interested in applying cognitive mapping within

studies in which the investigator looks for the presence of game

theoretic models in motions. It was demonstrated that the struc-

ture of motions is complex and that coders need a clear under-

standing of what is going on in order to be able to come to the

best coding. The high amounts of agreement show that the final

decisions by the coders resemble each other very much. This is

one step in the process of ascertaining that the correct type of

game is found. The next step is relating the games to different

other variables related to the motions.

These game theoretic models are used to investigate for ex-

ample differences in performance of political parties or changes

in approaches to the government over time. It might concern all

kinds of events from the past. Apart of these also more techni-

cal questions can be posed. In case a motion contains several

game models, should these be of the same type? One might

argue that if this is not true, it indicates that several techniques

R. POPPING

are followed by the proposer. This indicates a weak motion, it

is not enforcing towards the government. By critically review-

ing discrepancies in applications of our coding scheme, we

hope to have both improved our methodology and conveyed it

more clearly for others’ use.

REFERENCES

Anthony, D. L., Heckathorn, D. D., & Maser, S. M. (1994). Rational

rhetoric in politics—The debate over ratifying the US constitution.

Rationality and Society, 6, 489-518.

doi:10.1177/1043463194006004004

Axelrod, R. (1976). The analysis of cognitive maps. In R. Axelrod (Ed.),

Structure of decision. The cognitive maps of political elites (pp. 55-

73). Princeton, NJ: Princeton University Press.

Bara, J., Weale, A., & Bicquelet, A. (2007). Analysing parliamentary

debate with computer assistance. Swiss Political Science Review, 13,

577-605. doi:10.1002/j.1662-6370.2007.tb00090.x

Carley, K. M. (1993). Coding choices for textual analysis: A compari-

son of content analysis and map analysis. In P. V. Marsden (Ed.),

Sociological methodology 1993 (pp. 75-126). Cambridge, MA: Basil

Blackwell.

Harris, K. L. (1996). Content analysis in negotiation research: A review

and guide. Behavior Research Methods, Instrumentation, and Com-

puters, 28, 458-467. doi:10.3758/BF03200525

Holsti, O. (1976). Foreign policy formation viewed cognitively. In R.

Axelrod (Ed.), Structure of decision. The cognitive maps of political

elites (pp. 18-54). Princeton, NJ: Princeton University Press.

Krippendorff, K. (2004). Content analysis: An introduction to its meth-

odology. Thousand Oaks: Sage.

Laver, M., & Garry, J. (2000). Estimating policy positions from politi-

cal texts. American Journal of Political Sciences, 44, 619-634.

doi:10.2307/2669268

McLean, C., & Patterson, A. (2006). A precautionary approach to for-

eign policy? A preliminary analysis of Tony Blair’s speeches on Iraq.

British Journal of Politics & International Relations, 8, 351-367.

doi:10.1111/j.1467-856X.2006.00242.x

Popping, R. (2010). Ag09. A computer program for interrater agree-

ment for judgments. Social Science Computer Review, 28, 391-396.

doi:10.1177/0894439309351394

Popping, R., & Roberts, C. W. (2009). Coding issues in semantic text

analysis. Field Methods, 21, 244-264.

doi:10.1177/1525822X09333433

Roberts, C. W. (1989). Other than counting words: A linguistic ap-

proach to content analysis. Social Forces, 68, 147-177.

Roberts, C. W., Popping, R., & Pan, Y. (2009). Modalities of democ-

ratic transformation: Forms of public discourse within Hungary’s

largest newspaper, 1990-1997. International Sociology, 24, 498-525.

doi:10.1177/0268580909334500

Scott, W. A. (1955). Reliability of content analysis: The case of nomi-

nal scale coding. Public Opinion Quarterly, 19, 321-325.

doi:10.1086/266577

Shapiro, G., & Markoff, J. (1998). Revolutionary demands: A content

analysis of the Cahiers de Doléances of 1789. Stanford: Stanford

University Press.

Slapin, J. B., & Proksch, S. O. (2008). A scaling model for estimating

time-series party positions from texts. American Journal of Political

Science, 52, 705-722. doi:10.1111/j.1540-5907.2008.00338.x

Wijnja, M. (2010). Beleidsmakers of hypeparlement? Groningen: De-

partment of Sociology.

R. POPPING

Motions and Debates

Motions and debates are in the Acts of the House of Represen-

tatives [Handelingen van de TweedeKamer] or in reports on

committee meetings. They can be found at the places listed

below.

Motion Act

Page

(motion)

Page

(advice) Date

21501-32, nr. 366 36 3433 3435 December 9, 2009

22894, nr. 243 18 1459 1462 October 29, 2009

22894, nr. 244 18 1459 1462 October 29, 2009

24587, nr. 361 5 309 311 September 23, 2009

27428, nr. 155 52 4798 4799 February 9, 2010

31209, nr. 106 36 3540 3541 December 9, 2009

31322, nr. 110 113 56 61 November 22, 2009

32123-viii, nr. 25 33 36 38 November 2, 2009

32123-x, nr. 63 33 3185 3196 December 3, 2009

32123-xiv, nr. 82 33 3136 3151 December 3, 2009

32123-xiv, nr. 103 33 3143 3152 December 3, 2009

32123-xiv, nr. 119 33 3147 3153 December 3, 2009

32356, nr. 15 47 52 55 February 2, 2011

32474, nr. 16 24 69 71 November 18 2010

32500-vi, nr. 48 29 28 43 December 1, 2010

32500-vi, nr. 60 29 37 45 December 1, 2010

32500-viii, nr. 110 132 58 65 December 15, 2010

32500-xiii, nr. 50 67 39 50 November 11, 2010

Note: In bold are not acts but reports on committee meetings.