Week 9: Ethics of Internet Research - contesting the human subjects research model

Bassett, E. and O'Riordan, K. (2002)

Wide range of internet research, e.g.

• virtual ethnography
• linguistic and discourse-bases analyses of computer-mediated communication
• research into the internet as a culture

The perception of the internet as a social domain as led to the widespread application for the human subjects research (HSR) model in internet research. HSR ethics regards the rights of the human subject as primary and the aims of the researcher as secondary. Strong argument for use of HSR model when conducting research into activity such as email between individuals and groups, particularly when groups and individuals use CMC to convey 'private' material through channels that have some mechanism to limit access.

But when applied indiscriminately HSR model risks subsuming the human subject into the technology and merging the textuality of the internet. In a climate of fast developing technologies it is often difficult to define the actual object of internet research precisely. The cost of this blanket approach may be a resulting failure to analyse the significant textuality of internet media.

The internet is not simply a virtual space in which human actors can be observed: it is a medium through which a wide variety of statements are produced.

The internet is not simply a text-based medium made up of communities, newsgroups and email lists. It is also a medium of publication, and significantly one where users can take control of the means of production, create their own cultural artifacts and intervene in the production of existing ones. The internet can thus be perceived as a form of cultural production, in a similar framework to that of broadcast, TV & radio.

Three models to consider:

1. Text as reflection of the author.

• 'text as a precursor to the virtual self'. Literature as a textual opening into the mind of the revered author (text as reflection), or the 'conflation of text and author'. "Great literature is the product of Great Men, and its value lies chiefly in allowing us intimate access to their souls" (Terry Eagleton, 1983). A 'mind made word'.

2. Text as object

• The School of New Criticism challenged the above viewpoint in the post-war period, and was concerned with the interpretation of texts. With New Criticism, the text-as-virtual-writer is replaced by the text-as-object, severed from context, authorial intent, and the response of the reader. Akin to semiotics, it proposed that the meaning of the text lay solely in the language used, regardless  of the intentions of the author at the point of writing. The text was considered to be an organic whole and not a window onto the soul of the author. The text, now considered a elf-sufficient object, was also liberated from both social and historical context of production. 
• The assumption that a piece of text is an object underlies the logic of copyright law. IN copyright law, the text is not akin to a window onto the author's soul but more akin to a box. The contents belong to the author and s/he is free to distribute them and adapt them as s/he wishes. The author sells the distribution of the text, either in units, receiving payment each time the text is reproduced, or outright, by signing the authorial rights over to a publisher for a certain length of time. The reader, by buying a copy of the book, purchases the right to read the work but does not acquire the rights to redistribute it. The author has a right to be recognised as the author of the text, but this means recognition of the fact that s/he owns the text and produced it, not that the text relfects her point of view.
• The registration of the copyright of texts distributed via the Internet counters the notion that the internet is a purely spatial realm.

3. Text as 'reader-response'

• The text separates the author from the reader "...the book divides the act of writing and the act of reading into two sides, between which there is no communication. The text thus produces a double-eclipse of the reader and the writer" (Ricoeur, 1981). The resulting separation between author, text and reader prompts the question of to what degree a text can be said to have any intentional or reflective meaning. "[as] the meanings of the text are the 'production' or 'creation' of the individual reader... there is no one correct meaning, either of the linguistic parts or of the artistic whole of the text" (Abrams, 1941).
• The notion that the meaning of a text is brought about in the process of reading - i.e. that the reader constitutes the text - contests the principles of objective and positivist research. In these paradigms the role of the researcher is to uncover the 'truth' of the object of study. This principle does not only apply to the study of inanimate objects: it is a traditional tenet of anthropological research that the researcher attempt to leave the object of enquiry unchanged. 
• Contemporary research perspectives acknowledge the impossibility of conducting research without altering the meaning of what is observed. This view holds that the meaning of any situation or text does not exist prior to observation, but is inscribed in the process of research. This understanding has led to the use of reflective and 'participant-observation' methodologies that clearly convey the participatory role of the researcher.
• 'Post-paradigm' anthropology and feminist sociological research both require the researcher to account for the dynamics of the relationship between researched, researcher and results. Feminist documentary and film-making has also stressed the relationship between the observed or filmed and the producer of the text. In these paradigms the text is a contested and multiple output with a multiplicity of stakeholders. In other words: the implication for the social construction of knowledge through research is that it is a contested an relational construct.

Question: is it okay to quote a dialogue you copied from a chatroom?

1. The chatroom is a 'space' and the dialogue is between people in it, all of whom are due the protection accorded them under the human subjects research model.
2. The dialogue you have is a 'text', it's in the public domain, and therefore, aside from considerations of copyright, is available for reproduction.

Online communities simultaneously commodify identity and utopian desire, working through a liberal agenda of identity without exclusion, to maximise market share whilst capitalising on fragmentation.

The decision to disguise online activity, justified through the use of a rhetoric of 'protection' may result in furthering the unequal power relations of media production by blocking full representation of alternative media.

Conclusion: advice for internet researchers

Spatial descriptions of the internet are consistent with the Human Subjects Research model, BUT only some types of Internet use can usefully be described in spatial terms. Academic research that positions the internet as a social space containing cultural activity ripe for observation ignores the range of textual applications that the internet supports.

As internet researchers, we need to develop models that acknowledge both spatial and textual understandings of the internet. Such models are required if we are to produce research that examines the complex intersection of technologies, form, genre, and content that the internet supports.

Users may be less informed about the issues involved in textual production via the internet than in print media or traditional broadcast media such as the radio or television. However, researchers cannot afford to assume that individuals engaging in cultural practices through the internet are always at more risk than those using other media.

If an individual or group has chosen to use internet media to publish their opinions then the researcher needs to consider their decision to the same degree that they would with a similar publication in traditional print media.

To maintain a research model akin to the HSR model would be to risk impeding and potentially eliminating promising research. It is not always possible, for example, to gain the consent of a large number of participants who may have changed their email address or ceased posting to a web site on which the material under research is located. This should not prevent research of textual material that they have chosen to output via the internet anymore than it would for textual products in other print or broadcast media.

We should attempt to find a way of acknowledging the hybridity of the internet, acknowledging that the texts it supports are neither virtual selves nor objects completely distinct from those who write them. A reading of the text that includes the content, form, history and researchers' response all need to be considered as part of these relations of production, and power is a central issue. These relations include, for example, statements of intention in the text about whether it is intended for publication or for consumption by a circumscribed group of people... These relations will effect how users perceive their acts of participation and production: for example, is the site like a public forum / message board, or is it more like a newsletter intended only for members of a group? These structures clearly contribute to the user's perceptions of their activity on it.

Commonly deployed spatial models of the internet appear to be seeing a resurgence through the dominant construction of the internet as a place in which human subjects act. We argue that a hybrid model of relational ethics that incorporates text, space and bodies needs to be applied to Internet research. The HSR model only has limited application, and where it is applied indiscriminately or as the 'safest' option it in fact can be unethical and not 'safe' at all.

___________________________________________________________________________________

Key points from online discussion: \

https://modules.lancs.ac.uk/mod/forum/discuss.php?d=22328

From David Pike:

Hi Sue,

Using your "obligation to protect people" and a bit of googling about the idea of duty of care (which seemed like a good phrase) I found there's something called the Caparo Test........so with some intellectual alchemy I would say........

1) If you send/bring/encourage students, assuming no prior experience, to an place/community online/part of an environment - you have an obligation to protect

2) If you gather the students who are part of an existing network e.g. Facebook - you have an obligation to protect only within the bubble you create

3) If you cause students to create new artefacts which are displayed out on the internet you have an obligation to protect by ensuring that students know the rights they hold in relation to said 'things' (i'm thinking of art students, musicians, maketeers, programmers etc).

4) If in any of the cases above you will be working with a group who could / are considered vunerable [e.g 1 perhaps something like the definition and who is sections?] [e.g. 2 - defined as a child] you automatically have an obligation to protect

What does everyone think? Question is, how? This probably varies by technology.....

___________________________________________________________________________________

From Nafeth:

Hello David and Ayman,

I believe that users don't have to learn the technical stuff of information security terminologies to be safe. A big part of the responsibility is on the owners of the websites. One thing they can do is to exercise the 'Due Care' standard that is a the minimum and customary practice of responsible protection of information published through the website. Due care is exercised to ensure that appropriate controls are implemented . Applying this standard implies reasonable care and competence.

___________________________________________________________________________________

Ontology, Epistemology, Methodology, methods and sources

Ontology:

What we believe constitutes social reality. This helps us to understand how different scholarly traditions embedded in fundamentally different cultural contexts can have different views of the world and different assumptions supporting their approaches to social enquiry. Grix (2004) in his book “The Foundations of Research” argue that Ontology is prior to Epistemology.

Ontological problems mainly concern with the nature of social entities: Whether or not social entities can be and should be considered objective (and therefore external) to social actors; or whether social entities could be constructed based on the actions and perceptions of social actors (Bryman & Bell, 2007).

Epistemology:

Epistemology is concerned with the theory of knowledge, especially in regard to its methods, validation and ‘the possible ways of gaining knowledge of social reality’. It is about ‘How what we assumed to exist can be known’

Epistemology focuses on the knowledge-gathering process and is concerned with developing new models or theories that are better than competing models and theories.

Epistomological issues concern the questions of what is (or should be) regarded as acceptable knowledge in a discipline: Whether or not the social world can be and should be studied according to the same principles, procedures, and ethos as the natural science (Bryman & Bell, 2007)


Therefore, each step of these consider different viewpoints:

• Epistomological considerations: Positivism vs Interpretivism. Since Nafeth did mentioned about this, I have no way of improving his explanations

• Ontological considerations: Objectivism vs Constructionism

Objectivism implies an ontological stance that social phenomena confront us as external facts that are beyond our reach or influences.

Constructionism advocates that social phenomena and their meanings are continually being not only built up based on the social interactions but also revised constantly.


Following are two examples of ‘contrasting’ epistemological positions:

• Positivism: this is an epistemological position that advocates the application of the methods of the natural sciences to the study of social reality and beyond

• Interpretivism: an epistemological position that believes that a strategy is required that respects the differences between people and the objects of natural sciences and therefore requires the social scientist to grasp the subjective meaning of social action

Choosing one of these epistemological views will lead you to employ a different methodology than you would if you choose the other.

If Ontology is about what we may know, then Epistemology is about how we come to know what we know.

Our research questions are normally shaped based on our ontological and epistemological positions. This can be thought of as what is explained in the following diagram:

Based on the steps listed above, we can conclude that choosing our methods is based on the research questions posed and the sources of data collected.

On the other hand, some researchers believe that research can start at any of the stages that are listed in the diagram above. For example, a researcher might choose a favourite method and then work reversely through his methodology, epistemology and ontology. This may work, but with extreme care to make sure that all of these components are logically compatible with each other.

Some researchers prefer ‘question-based’ research (using the steps listed in the above figure) while others might opt for ‘method-led research’. You could argue that choosing a research method before having a research question is not logical...

Week 8: isms and ologies - summary of discussion

Questions 1: do I believe truth is discovered or made?

Possible viewpoints:

• 'relative' truth (i.e. as it happens to be at a given moment based on a set of evidence/information) is decided or discovered by a process or argument and reasoning e.g. an academic paper / court of law. BUT this doesn't make it universal/absolute truth because it is based on the wisdom of people with a range of interesting and subjective outlooks. 
• is that 'discovering' truth or 'making' truth based on the perspectives of the process and participants?
• does 'truth as a discovery' only happen after we filter the cultural, social, historical etc. bias out?
• Any perception that involves opinions/feelings/ideas or simply convincing people one way over another probably involves viewing truth through some form of 'filter', i.e. cultural, social, historical. Ergo a perception of truth might look highly misguided to someone who does not see life through the same  filters.
• Truth-making is related to knowing, experience, context, association, power, and choice. For example, the connections we choose among findings, our choice of relations with actors (researchers, subjects, policies), contribute to (or construct) a truth, which can be socially accepted (or not) within a community(s).
• The 'truth' can change based on new knowledge, therefore there are no absolute truths - truth is discovered. OR is 'truth' a static value and instead it is that our knowledge, understanding or perception of it that changes?
• e.g. everyone believes the 'truth' that a plant is green. However, new discoveries reveal that in fact the green is a result of the plant reflecting particular colours and that the structure of the molecule causes the pigmentation. These physical factors have always been there (i.e. constant), it's just that we didn't know. Our discoveries have improved our understanding.
• not being able to discover the absolute truth doesn't mean that it doesn't exist.
• most of the time truth is made, however a quasi-absolute truth ('real' reality) may be discovered once we filter out the how and why of the “the meaning-making mechanisms of the social, mental and linguistic worlds that individuals inhabit”.
• ‘truth, and any agreement regarding what is valid knowledge – arises from the relationship between members of some stake-holding community’.  This reinforces the impact of communities, cultures and political systems in their subjective construction and understanding of what is ‘real’ and what is ‘valid truth’ and the critical role of language and discourse in shaping and negotiating these views (and beliefs?). Is this temporal view of a subjective and dynamic truth therefore created, reinforced and legitimised by those who construct it?
• At what point does a ‘notion/idea/concept/ belief’ become truth’?  At what point does the ‘communal test of validity through the argumentation of the participants in a discourse’ decide that something is ‘true’ or is ‘truth’ only ‘true’ to those who believe it to be so?

• Accepting that validity does not constitute a single truth, but can only provide a “thoroughly partial understanding” of a topic is likely to be an uncomfortable choice for many.

Question 2: as a researcher am I part of the action or resolutely outside looking in?

Possible viewpoints:

• the very choice to research a subject and the choice of instruments -- not to mention the very design of the instruments themselves -- are embedded with culture, history, and power. This would suggest to me that the researcher is indirectly (involuntarily?) part of the action.
• because history, politics, power, and social references are embedded in a paradigm, researchers are contributing to a wider context (our meaning and knowledge of something) through the action of their research -- which is often a reaffirmation, causation and/or consequence of  what is embedded in the paradigm. 

My thoughts:

• All truth is subjective. Even physical laws are subject to doubt arising from what we don't yet know. 

Week 7: e-Research

Anderson & Kanuka (2003): Designing e-Research

 

Anderson & Kanuka: Designing e-Research

"It is the theory that decides what can be observed" - Albert Einstein

When the purpose of the research is to generate theory, qualitative research is an appropriate strategy.

When the purpose is to test theory, quantitative research is more appropriate.

Task of the e-researcher: "to design research that asks meaningful and answerable questions and that coherently answers those questions in ways that match the personal worldview of the researcher, the sponsor of the research, and the subjects of investigation"

 

Advantages of using the Internet as a research tool

• ability to access research 'subjects' that would otherwise be inaccessible due to distance, nationality, language barriers etc.

  Provides anonymity to research subjects who may be unwilling to provide data using other methods 

Disadvantages of using the internet as a research tool

• because the internet is a relatively new research tool we 'don't know what we don't yet know' about its limitations

 

Research designs with usually include the following:

• A research methodology paradigm
• The use of related and relevant literature
• The purpose and/or objectives of the study
• A problem statement, research questions, or hypotheses
• An acknowledgement of limitations and setting of delimitations
• A statement of the significance of the study
• A plan for data collection and analysis
• A statement about how the study advances methods and procedures for data collection and analysis

e-researchers must try to share the findings of their study and explore how the net influence the search process and the procedure for data cllection and analysis.

How do you do e-research?

• decide on a problem in need of investigation - typically research focuses on understanding a phenomenon, testing a theory in a real-world context, explaining an observed event, or developing a solution to an existing problem
• ask the question in a 'researchable' manner:
• Avoid value judgements involving 'should ' or 'should not'
• Avoid questions that invite an absolute response e.g. 'yes or no'
• make sure the research question avoids the so what factor
• when structuring your question use language that is compatible with the study, method or paradigm  

Are technologies 'neutral' in their effect on educational outcomes?

"[M]edia are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in nutrition...only the content of the vehicle can influence achievement" (Richard Clark, 1983: 445)

The influence of media is a complex phenomenon (Chandler, 1996) and shape our experience through their selectivity. Chandler identified the key features of technological impact that reshape many of our social structures:

• Selectivity: Selectivity is the way a technology is formalised within its own system. The outcome of selectivity occurre when some aspects of our experiences are amplified through the use of communication technologies causing others, in turn, to be reduced.
• Transparency: Transparency occurs when we become so familiar with a technology that we do not recognise its influence and hence, the consequences of its use. This results in an inability to exercise our choices.
• Transformation: Transformation occurs when the use of technology becomes an end in itself. Specifically, when using a technology, we function within its structure.
• Resonance: Resonance refers to the significance or enduring effect attached to the use of one technology over another. How significant the resonance is depends on the media transformations, which, in turn, derive from the nature and use of a particular technology.

e-researchers need to be aware of all of the above concerns when designing a research study.

Quantitative vs qualitative research paradigms

The method e-researchers choose often reveals what they value as knowledge and, perhaps more importantly, what their perspective of the nature of reality is. For example, quantitative methods are based on an assumption that the world is comprised of observable and measurable facts. In the context of e-research, this involves using the internet as a data-gathering tool to find cause and effect through an objective, formal, an deductive design, which aims to be context and value free. Quantitative researchers strive for transparency and objectivity, thus allowing their results to generalise to other places and people.

Common quantitative methods: experimental studies, questionnaires

In contrast, qualitative methods assume multiple realities, which are subjective, value-laden, complex, and context-bound.  Accuracy is achieved through credibility, transferability, confirmability, and dependability (Guba, 1981)... The qualitative e-researcher interacts with the research using an in-depth inductive process and an emerging design that is identified during the research process. With respect to the Net, the qualitative e-researcher generally views it as more than simply a tool for data gathering... Thus the Net and the research results are indivisible, in that the e-researcher's biases and beliefs about the Net will influence the research process, and hence, should be reflected on and addressed through the study.

Common qualitative methods: ethnographies, grounded theory investigations, case studies, interviews and phenomenological studies

The different assumptions of each research approach will not only influence the methodological approach selected, but also the purpose of the research and the role of the e-researcher... Thus is it important for e-researchers to be self-aware to the extent that they can articulate their own personal worldview.

Tips for writing good research questions

• When developing your research question, make sure it does not require a value judgement
• Avoid asking questions that can be answered in any absolute way (i.e. "yes" or "no") or seem to imply universality over large domains or fields of study
• Match the language to the method of the study (quantitative vs qualitative)
• Pay attention to the wording of published research studies to identify the perspective, methodology, or reserach paradigm from the problem statement
• Consider how the Net may influence responses to your research question
• Be sure that the research question guides your use of the Net rather than letting the Net guide the research question

Week 6: Design-based research

What is design-based research?

 

"Design-based research integrates the development of solutions to practical problems in learning environments with the identification of reusable design principles" (Herrington et al, 2007)

Issues

Kolodner (2001) states that education researchers have been "grappling with issues in methodology". She argues that the problems with Ethnography lie in the breadth of data collection without regard for the goals, and that Interaction Analysis is also problematic as it too fails to address analysis goals.

Herrington et al (2007) argue that educational technology research "has sought to demonstrate the achievement gains of technology-facilitated learning over conventional methods of teaching with little regard for an understanding of how or why the gains might have been realized". But they also believe that "future progress in improving teaching and learning through technology can be realized through design-based research as an alternative model for inquiry in the field of educational technology.

"Many research proposals begin with an introduction to an area of study, and a description of what the study will do. However, for  design-based research in education, the identification and exploration of a significant educational problem is a crucial first step. It is this problem that creates a purpose for the research, and it is the creation and evaluation of a potential solution to this problem that will form the focus of the entire study... The statement of the problem in design -based research should identify an issue or an opportunity, explore its history or  background, and provide a convincing and persuasive argument that this problem is significant and worth researching. This includes articulating both the practical and scientific relevance of the study." (Herrington et al, 2007)

Carrying out design-based research

 

"Edelson (2006) commented on the assumptions of design-based research, pointing out that: "It begins with the basic assumption that existing practices are inadequate or can, at least, be improved upon, so that new practices are necessary. The underlying questions behind design research are the same as those that drive innovative design:

• What alternatives are there to current educational practices? 
• How can these alternatives be established and sustained? (p.103) (Herrington et al (2007)

 

 Both diagrams taken from Herrington et al (ND) http://personal.crocodoc.com/tQ573CK

Examples of design based research  

http://www.personal.psu.edu/mdm392/blogs/design/2010/09/carrying-out-design-based-research.html

More info on design-based research

http://www.personal.psu.edu/axg251/blogs/instructional_design_and_higher_education/2011/09/

References:
Kolodner, J. L. (2001). A Note From the Editor. The Journal of the Learning Sciences, 10 (12), p. 1-4.

Week 5: Phenomenology & Phemonenography

Wilson (2002)

Phenomenology

Phenomenology is a philosophical school concerned with how people make sense of the world. It examines how a person's consciousness organises and classifies his or her knowledge and experience.
It originated through the work of Edmund Husserl and was expanded further by Alfred Schutz. According to some interpretations it can also be a methodological approach.

Aim

• to comprehend the meaning of things as seen through the eyes and mind of particular groups or individuals (Trowler - PhD tutor)
• to study human phenomena without considering questions of their causes, their objective reality, or even their appearances (Wilson, 2002)
• to study how human phenomena are experienced in consciousness, in cognitive and perceptual acts, as well as how they may be values or appreciated aesthetically (Wilson, 2002)
• to understand how persons construct mearning

Phenomenology is not a hypothesis testing mode of research (although it may result in hypotheses that we may wish to test by other means), nor is it one that must be guided by theoretical models. Rather, one is urged to get as close as possible to what the participants in the behaviour of interest are experiencing. Rather, It is about the nature of human experience and the meaning that people attach to their experiences. In trying to arrive at this kind of understanding, the researcher is asked to 'bracket', or suspend belief in the phenomena of the external world, to put them aside and focus on the consciousness of that world (Wilson 2002).

Definitions

• Typifications: shared understandings of categories

• Commonsense knowledge: groups of typifications comprise 'commonsense knowledge' for communities and this allows them to impose meaning on the world (Trowler). 

• Phenomenography: 'drawing understandings of the world' (Trowler)

• Intersubjectivity: experiencing the world with and through others. Human experience of the world is 'intersubjective'. Whatever meaning we create has its roots in human actions, and the totality of social artefacts and cultural objects is grounded in human activity (Wilson, 2002)

People construct their social life through interaction with others. It does not matter if their interpretations are 'true' - this is not a concern of phenomenology. 

Phenomenography

Phenomenography is a method of research, as opposed to phenomenology which is a philosophical school of thought. That said, it is still largely based on phenomenological tenets. 

Aim: to examine the variation in 'ways of seeing something'. Schutz's methodology starts from the idea that the difference between social research and physical sciences research is that the former involves the participation of 'research objects' (i.e. people) who are 'themselves interpreting the social world that we, as scientists, are also aiming to analyse. (Wilson 2002)

Outcome:  various categories of description called an 'outcome space'. These are usually related to each other, often in a hierarchical format, where those above include but go beyond the descriptions of those below. 

How does it work? Usually by interviewing, followed by detailed analysis of interview transcriptions to identify the 'outcome space'. In analysing the world it is inevitable that we employ the same methods of interpretation as the person whom we are studying. Therefore the social scientist must adopt the position of the 'disinterested observer': he or she has no involvement in the life of the observee. The goals of the observee 'are not of any practical interest, but only of cognitive interest' (Wilson 2002).

By operating in such a manner the researcher builds models of human action, of which the general 'postulates' are:

1. the postulate of logical consistency, whereby the objective validity of the scientist's constructs are guaranteed and are distinguished from the constructs of everyday life;
2. the postulate of subjective interpretation, whereby the scientist can refer '...all kinds of human action or their result to the subjective meaning such action or result of an action had for the actor.", and
3. the postulate of adequacy: that is, the constructs created by the researcher should be understandable by the individual social actor and his/her fellows. Compliance with this postulate ensures that the scientific constructs are consistent with the constructs of common-sense experience of the social world (Wilson 2002).

Schutz also examines how socially-derived knowledge originates and, again, uses ideal types to explore this phenomenon. Apart from ourselves, as the central (to our minds) figures in social interaction, he suggests that the sources of socially-acquired knowledge can be seen as four ideal types:

1. the eyewitness, that is, someone who reports to me something that he or she has observed in the world within that person's reach;
2. the insider: someone who, because of his relationship to a group which is more direct than my own, is able to report some event, or the opinions of others, with the authority of sharing the same system of relevance as the other members of the group. I accept the insider's information as 'true' or valid, at least in part, because the insider's knowledge of the context of the situation is deeper than my own;
3. the analyst: someone who shares my system of relevances, who has collected information and organized that information in conformity with that system of relevance; and
4. the commentator: someone who does not share my system of relevances, but who has collected information in the same way as the analyst and has presented that information in such a way that I can form 'a sufficiently clear and precise knowledge of the underlying deviating system of relevances' (Wilson 2002).

Again, each of us may, at different times and in different circumstances, take roles that fit these ideal types.

Methodology vs Method

Wilson points out that people often misuse the term 'methodology' by using it simply to describe the 'method' of conducting a research study. He states that methodology "is prior to method and more fundamental, it provides the philosophical groundwork for methods. To state one's methodological position is to describe one's view of the nature of reality: for the positivist, the methodological position is that the facts of the world represent real objects, while for the phenomenologist, the world (or at least the one he/she chooses to explore) is one of intersubjectively constructed meanings" (Wilson 2002).

Week 5: Mixed Methods research

Notes from Week 4 discussion on a Mixed Methods approach to research.

From Mixed Methods group precis

Creswell asserts that a researcher should formulate a framework when preparing a research proposal so that research methods can be developed that are justifiable to the author and audience. Three frameworks to support the development of an appropriate proposal are described (‘approaches to research’): quantitative, qualitative and mixed methods, as well as three framework elements (‘elements of inquiry’), alternative knowledge claims, strategies of inquiry and methods. 

Three Elements of Inquiry

1. Alternative Knowledge Claims

Knowledge claims are the theoretical assumptions made by researchers about how and what they will learn during their research. These claims can be described as philosophical assumptions, epistemologies, ontologies or research methodologies.  Knowledge claims can relate to what constitutes knowledge (ontology), how we know if something is knowledge (epistemology), what values go into knowledge (axiology), how we write about knowledge (rhetoric) and the process of studying knowledge (methodology).
Creswell proceeds to discuss four ‘schools of thought’ about knowledge claims and aims to reduce these to a practical context. He outlines four primary alternative knowledge claim positions, postpositivism, constructivism, advocacy/participatory and pragmatism.

a. Postpositive Knowledge Claims
 These assumptions are primarily associated with scientific research and the concept of ‘cause and effect’.They are also referred to as quantitative research, positivist/post positivist research, and empirical science. Postpositivism means challenging the traditional notion of the absolute truth and understands that we cannot be “positive” about our claims of knowledge when studying the behaviour and actions of humans. The approach adopted by postpositivist researchers is to develop a theory, collect data to prove/disprove the theory and then make any required alterations before conducting any additional tests.

b. Social Constructivism
 The assumption that governs social constructivism is that the environment where the research is conducted gives new meaning and interpretation to the research based on the historical and social perspective. Human beings are stakeholders in this research position as they contribute to constructing the knowledge by their own social experiences and backgrounds. It recognises that the subjectivity of individuals’ experiences and involves exploration the complexity that results from this. Researchers use open-ended questioning to capture the participants’ views and interpret what they say in the relevant social and historical context to generate meaning. Unlike the postpositive approach where a researcher starts with a theory which is then tested, the process of qualitative research involves the inquirer developing a theory or meaning based on the data collected in the field.
 
c. Advocacy/participatory
 As a result of the limitations of postpositivist and social constructivist assumptions, advocacy/participatory approach evolved during 1980s and 1990s. This was based on the opinion that the other theories did not fully address issues of ‘social justice’ and the marginalised in society. This approach suggests that research should be linked with politics, and address contemporary social issues in a collaborative way to give participants a ‘voice’ and improve their lives. The researcher advocates for reform, considering social issues like ‘empowerment, inequality, oppression, domination, suppression and alienation’.
 
d. Pragmatism
 In this approach the focus is on “problem” and “what works” to solve the given problem instead of considering the methods used. Researchers using this approach do not subscribe to a specific philosophy or approach to research. Therefore researchers have a freedom of choice and employ elements of qualitative and quantitative approaches to collect data. They accept that research takes place in a social, historical and political context and believe that ‘truth is what works at the time’.

Strategies of Inquiry

The Strategies of Inquiry introduced in this paper are 1. Quantitative 2. Qualitative and 3. Mixed Methods.

1. Quantitative approach has long been established using data gathered through experimental practices. Creswell notes two quantitative strategies of inquiry;

• Experiments: True – with random subject assignment. Quasi-experiments – with nonrandomized design.
• Surveys: Cross-sectional and longitudinal studies. Surveys aim to generalise from a sample to a population.

2. Qualitative approach methods have gathered momentum over the last 20 years, by incorporating several approaches such as ethnographies; grounded theory; case studies, phenomenological and narrative research.
Cresswell outlines the qualitative approaches as follows:-

• Ethnographies; Researchers aim to explore cultural phenomena by gathering observational data over time from a specific cultural group. This involve a flexible process that adapts to the ongoing researched situation.
• Grounded Theory; Researchers aim to generate or discover a theory from data systematically obtained from social research. It draws abstract theory from participants’ views in a study. The theory involves several stages and collection of data from different groups with evolving comparisons between data categories.
• Case Studies; Researchers analyse an actual event to gain detailed information, through various data collection techniques. There is a detailed study of a program, process, activity or individuals.
• Phenomenological Research; Researchers relate their own experiences with how those are perceived by the participants in a given study. It aims to understand the “lived experience” of a small number of participants regarding a phenomenon. This has philosophical and methodological features.
• Narrative Research; Researchers and collects real-life stories to create a narrative involving both researcher’s and participants’ views.

3. Mixed Methods approach. These approaches include elements of qualitative and quantitative methods e.g combining ‘field’ methods such as interviews (qualitative) with traditional surveys (quantitative). The premise is that biases in one method may offset biases in another to produce more reliable data. This approach led to triangulation methods, which aim to seek convergence across research methods.
Creswell notes three mixed methods strategies of inquiry:
(i) Sequential procedures: The researcher begins with either a qualitative or quantitative method, and then develops findings with the other.
(ii) Concurrent procedures: Quantitative and qualitative data are collected at the same time to develop a comprehensive picture. The researcher may embed one data form in a larger data collection.
(iii) Transformative procedures: The design involves quantitative and qualitative data analysed through a theoretical lens. The lens gives a focus for factors such as outcomes or data collection. The data collection procedures may involve a sequential or concurrent approach.

 

Research Methods

The researcher must specify the methods of data collection and analysis. The choice of methods used by a researcher depends on whether the intent is to specify the type of information to be collected in advance of the study or to allow it to emerge from participants in the project. The knowledge claims, strategies and the method all contribute to a research approach.

A quantitive approach is one in which the investigator primarily uses post positivist claims for developing knowledge and employs strategies of inquiry such as experiments and surveys.

A qualitative approach is one in which the inquirer often makes knowledge claims based primarily on constructivist perspectives or advocacy/participatory perspectives . It uses strategies of inquiry such as narratives.

A mixed methods approach is one in which the researcher tends to based knowledge claims on pragmatic grounds.

Three criteria to consider when selecting a research method are;
1. The research problem
2. Personal experiences of the researcher
3. The audience to whom the report will be written

1. The research problem:

A quantitative approach is best to test a theory or explanation. A qualitative approach is best if there is concept or phenomenon that needs to be understood because little research has been done on it. When the researcher does not know the important variables to examine, a qualitative approach is most appropriate. In general, this approach is ideal for exploring new topics.

Mixed methods can be used to capture the best of both quantitative and qualitative approaches. The researcher may initially adopt a qualitative approach to learn about what variables to study and then uses a quantitative approach to study the variables within a large sample. Alternatively, the researcher may initially use a quantitative approach on a large sample, then use a quantitative approach to go in depth with a few individuals from that sample.

2. Personal experiences of the researcher

Personal training and experiences should be considered when deciding on a method. A mixed methods researcher needs to be familiar with both quantitative and qualitative research, and also needs to understand the rationales for combining both forms of data.

Quantitative research is most common. There is therefore more support available for those choosing this approach. Mixed methods research takes the longest because of the need to collect and analyse both qualitative and quantitative data.

3. Audience

When deciding on an approach, the experience and preferences of those who will support and read your research should be considered. For students, preferences of their supervisors need to be considered.

Summary

Creswell outlines how when research philosophy, strategies and methods are combined, it is possible to generate different frameworks under which to conduct research. Researchers must choose the most appropriate approach based on the research question, their own experiences of research and the potential audience. The research approaches that are available to researchers are qualitative, quantitative and mixed methods. Each is underpinned by assumptions about knowledge claims, strategies of inquiry and specific research methods.

 

From Don Passey

The important element for me when I'm setting up a study is to match what I want to study with appropriate methods that will allow me to do that. Planning then becomes important - identifying the how, the when, and the extent of data gathering that can be done within time periods and still enable adequate data analysis and outcomes to be drawn out.

From Seema Pillai

Mayer, R. E. (2003), the research should be “issue-driven rather than method-driven”.  If the chosen research problem demands a mix of quantitative and qualitative approach to better address the issue then definitely researcher should carry out a detailed study on this innovative research method to minimize the chances of a pitfall.

The typology suggested by Creswell and Plano Clark (2007) provides a clear role for each component in terms of timing, weighting, and mixing and is thus very useful for novice researchers. "Creswell and Plano Clark (2007) proposed four main design types, with multiple variants:

(1) the triangulated design, to obtain different but complementary data;

(2) the embedded design, in which one data set provides a supportive secondary role;

(3) the two-phase explanatory design, which builds or explains quantitative results; and
(4) the exploratory design, which is also two-phased but led by the qualitative."
The results of Jerome (2011) study using triangulation mixed method and embedded design in two different educational contexts will be useful to learn more about the benefits and challenges of mixed method research.

From Mary Hamilton

The reality is that people use different starting points. The "ideal" (expressed carefully by Don in his post above)  is to identify the research problem or question(s) you want to investigate and design your  research strategy around this.
Commonly, however, what is practically possible in terms of access to different kinds of data, researchers predilections, training and beliefs act as filters to the kinds of methods you may consider in a given study.

Bob and I like to talk in terms of a "chain of evidence" that a researcher has to build in order to present a convincing story of their project from start to finish. The idea is that the questions, framework, design and data have to link in a logical way in order to produce "knowledge claims" at the end of the day that other people will (hopefully!) accept as valid.

Week 4: The issue of quality in qualitative reserach

Hammersley (2007) The issue of quality in qualitative research. 

Available at http://www.tandfonline.com/doi/abs/10.1080/17437270701614782

Focus: Examines whether criteria are necessary and considers whether a single set of criteria is achievable.


Problems with qualitative research:

• Criticised for not servin evidence-based practice well
• Argument that there is no clearly defined set of quality criteria available for judging it, ' so that it is of uncertain quality'

This assumes that:

• There are already clearly defined criteria of quality for quantitative research
• That explicit assessment criteria are needed

Problem faced by lay users e.g. policymakers and practitioners):

• how to assess research findings in the terms in which they present themselves
• how to relate these findings to what they already know and how they normally judge validity

Importance of trust - researchers largely have to trust that other researchers have carried out their work efficiently, presented their findings honestly, that normal processes of criticism with in the research community have reduced the danger of error etc. There is no mechanism that can check independently that the knowledge they produce is sound, or even that they have pursued this knowledge in the most effective way possible. Dilemma: the whole point of research is that it may produce un-commonsensical conclusions, therefore findings must not be dismissed because they are counter-intuitive. It is not possible for researchers to make their judgements transparent, in the sense of fully intelligible to anyone, irrespective of background knowledge and experience. Indeed, there are limits to the extent to which these judgements can be made intelligible even to fellow researchers, because of the situated nature of judgement.

Paradigms

Different 'paradigms' of qualitative research (Thomas Kuhn) - i.e. qualitative research is divided by divergent theoretical, methodological and value assumptions about the nature of the phenomena being investigated and about how they can and should be researched. The differences between qualitative researchers stem from the reality that each comes from their own unique situated practice and each employs a different approach to examining the process of research. Hammersley therefore believes that 'paradigm conflict' arises from a difference in practices rather than just a 'clash of ideas'.

Max Weber (& Neo-Kantianism) believed that social scientists strive to understand 'social phenomena in their socio-historical contexts' rather than attempting to establish 'universal laws' as is the case with natural science. This is an important difference as it means that social scientists will always draw on different sets of values and assumptions about the factual nature of the social phenomena, which will inevitably bring about divisions within educational research.

Scientific method and rigour

Historical clash of opinion between researchers who believed that physics / Botany etc. provided the exemplar for scientific method and those who believed that the study of social phenomena required a very different approach. Some researchers believed that understanding social phenomena could only be achieved 'from within' in order for the researcher to fully understand the meanings and cultural references of those people being studied. This led not to a wholesale rejection of scientific method but rather to a conception of science that was better suited to studying social phenomena.

Discourse analysts vs ethnographers

It is often the belief of Discourse Analysts that readers must be presented with the data to enable them to form their own opinions as to the validity of the data. Some also state that what is inferred must not go beyond what can be 'observed' in the collected data. This differs from the argument made by many ethnographers that it is not possible to make all the data available to readers, and that the accuracy and relevance of their observations is wholly dependent upon their successful observation and interpretation of the culture of the people being studied.

Realism vs constructionism

Constructionists believe that social phenomena

• 'do not exist independently of people's understandings of them, that those understandings play a crucial generative role'
• 'are culturally constituted and can only be understood by describing the processes by which they are culturally constituted as the things they are'

This means that the social phenomena, their causes and effects are no longer the focus of investigation. Rather it is the processes by which they are identified and in effect created through identification by culture members' that becomes the goal of enquiry.

Conclusions

• There is a need to try and reduce the significant differences in perspective of approaches to social science research in order to try and establish guidelines for what is and is not good quality work.
• The plurality of methodological approaches and respective criteria for what constitutes 'quality' should not simply be accepted unquestioningly.