S1: How to Think About Research Design

Introduction

Research design is about how to pose questions and fashion scholarly inquiry to make valid descriptive and causal inferences about the social world.

In political science, researchers adopt diverse methodological tools; quantitative and qualitative, but they have a shared standard standard of evaluation.

No statistical technique can substitute for good research design and subject matter knowledge.

Qualitative research strategies usually combine a small number of cases with complex arguments with the implication that there are more variables than observations: the collapse of eastern Germany in 1989.

Case study research is empirical work carefully tailored to the subject.

Neither quantitative nor qualitative research is superior to the other, and deciding which strategy to pursue ought to be conditioned by the research question.

Both approaches to social science, however, must pay attention to the rules of scientific inference and adopt shared standards and procedures of social inquiry.

Intelligent commentary is not research.

Research is a scientific process of inquiry that occurs within a stable structure of rules and procedures. In academia, it’s not your opinion that matters, it’s what you can demonstrate!

rethinking-social-inquiry

Causal inference 

For King, Keohane and Verba (1994) all political scientific research, regardless of method, shares the following characteristics:

  • The goal is inference (to infer beyond the data to make a meaningful claim about the world that cannot be directly observed)
  • The procedures are public (the data can be reliably assessed by others to determine its validity)
  • The conclusions are uncertain (to construct arguments that can be falsified)
  • The content is the method (the material of inquiry is endless, the unity consists in the methods of inquiry)

For analytical purposes political scientists usually break the process of research design into four interactive components: the research question, the theory, the generation of data and the use of data.  I will go through these in more detail in the coming weeks.

In the quantitative-statistical template, many of the problems of research design are defined away with more and better statistical controls. This is why qualitative researchers are often accused of not abiding by the same rigorous rules of scientific inquiry.

But all researchers who rely on observational data need case studies. Knowledge of cases and context contributes to achieving valid inferences about the political/social world.

Critical thinking is more than technical wizardry. Analytic rigour in all research is difficult. The appearance of methodological rigour can be highly deceptive.

This module argues that research is an iterative process. The steps of research design are constantly being constructed by the researcher. But it also adopts the political science assumption that causal inference is the objective of social science.

The process of turning interesting ideas into arguments usually involves:

  • Identifying a puzzle (explaining the success/failure of economic adjustment strategies)
  • Formulating a research question to address the puzzle (does austerity explain Ireland’s export-led recovery recovery?)
  • Presentation of a debate in the literature (troika policy choices versus long-term institutional effects)
  • Proposing a different hypothesis/argument/theory (the presence/absence of a US service sector firms)
  • Gathering empirical material to address the question (sectoral composition of economic growth and exports)
  • Drawing conclusions and inferences (export led recovery more associated with US business cycle than Europe).

Social science as a debate 

All interesting ideas have to be constructed as a specific empirical research question linked to a theoretical debate with real world significance.

Intelligent research design is about constructing better arguments, and using the procedures of research design to make better and more valid descriptive and causal inferences about the political and social world.

A central step in designing a masters or doctoral thesis is identifying a gap or problem in a clearly defined body of academic literature, and then formulating a specific research question to address this.

A thesis is about depth not breadth.

Early researchers often start out with a ‘grand theory’ that they want to ‘prove’ or ‘disprove’ such as ‘the European Union is a neoliberal project’. They then amass as much data as possible to prove this is true. This is not a good strategy to adopt.

Master or doctoral thesis makes a contribution to social science when they are mid-level theories, specific and test for causal relationships that challenge theoretical assumptions.

The political world that social science tries to understand is highly unpredictable and very uncertain. Just think about Trump and Brexit!

There are competing social scientific visions of how the social and political world is constituted (ontology); what we can know about that world (epistemology); and how we can develop empirical knowledge of that world (methodology).

Unsurprisingly, therefore, the basic architecture of political science often takes the form of an academic debate.

But it is not a sophist debate. It is a scientific debate aimed at solving puzzles: explaining the origins of democracy, economic development, causes of war, policy responses to crisis and success of international trade agreements.

New facts rarely settle theoretical debates 

Puzzles almost always relate back to a theory or an argument that is under fire.

For example, what causes unemployment?

A lot of economic theory suggests that institutions which keep labour markets from self-clearing are a cause of unemployment? If this is true what would be the observable implications of this theory? High unemployment in countries with strong welfare states?

A theory never entirely disappears, no matter how often it has been falsified (i.e. self-clearing labour markets) but they usually find reincarnations in a new set of falsifiable predictions.

New facts rarely, if ever, settle long standing theoretical debates. This is the lesson we have learnt from Imre Lakatos. It requires a new theory.

We discuss this further next week.

Research question 

All research starts with a question. Half the battle of writing a masters or doctoral thesis is constructing that question. It’s iterative. The question will evolve.

Research design gives you the tools to adequately answer the question. The process of linking empirical observations to theories (arguments) = research design.

When someone asks you about your thesis is about, always try to answer it by stating: “I am asking the question…” or “I am trying to understand….”

A social scientific question should be stated in such a way that it could be wrong. Don’t ask a question that implicitly contains the answer. Frame it empirically.

The literature review is your construction of the competing academic explanations that attempt to answer that question.

When you propose and construct your explanation (hypothesis/argument/claim) the difficult part begins: finding the observable implications of your theory.

A large part of research, therefore, is taken up with systematically collecting data on as many observable implications of a suggested theory for a social phenomenon.

For example, in the study of inequality one of the major contributions of Thomas Piketty’s book ‘Capital in the 21st Century‘ is producing a longitudinal time-series dataset to assess his theory of R>G. The theory is contested but not the data.

Ideas can emerge from anywhere. But what makes for a good research question?  Bob Hanckè (2009) outlines the following criteria:

  1. Relevance to real world problems
  2. Pre-research and engagement with empirical material
  3. Engaging an existing theoretical debate
  4. Balance between concreteness and abstractness
  5. Falsifiability (i.e. not a statement that contains the answer)
  6. Researchability (i.e. don’t questions about the future)

Next week we will discuss causal inference in the social and political sciences.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s