Writing Chapter 4 is usually one of the most grueling and tiresome points of the dissertation process, but for once timing was on my side as I am enjoying the opportunity to write my first draft of Chapter 4 in warm and sunny Florida. I have been looking forward to this opportunity for over a month when I began to feel prepared to write the chapter.
Since Thanksgiving weekend, I have not had the opportunity to write Chapter 4 because I was either busy preparing or enjoying holiday events, visits to or from family and friends, or writing member checks for my participants. During my downtime in December, however, I was plotting and outlining Chapter 4 in my mind. I never got the chance to sit down and write any of it until last week.
My family and I have only been in Florida for 6 days and yet I have already written 55 pages of Chapter 4. Two of these 6 days don't count because one was the day of arrival and the other was Christmas day, so I really wrote 55 pages in 4 days. Today marks the point in which I completed writing the major components of my grounded theory, so I am quite satisfied now.
Looking over those rough 55 pages, there is plenty of room to flesh out my weaker arguments and transitions. I also have a lot of ideas of how to reorganize the content of those pages. I believe in my second edit of these pages, I would probably remove as much as or a little less than I would add.
My goal is to hit 100 pages before we leave Florida, and I'm making good progress on that. With the bulk of my argument already covered, I am worried less about quantity and worried more about the quality. With the 55 pages covering the biggest cross-case patterns, I anticipate the next 10-15 pages will cover the significant single-case findings that directly answer my research questions. So I plan on spending Monday and maybe Tuesday (the remainder of 2013) writing that section.
After writing the significant single-case findings, I plan to tie my grounded theory with at least two established models, one in the field of acculturation psychology and the other in the field of English language education. I'm trying to imagine a graphic organizer to represent how I plan to combine my grounded theory with these two models. For now, this imagined graphic organizer is too complicated and messy, although the main ideas seem clear. I need to create a graphic organizer that is clear, if not clearer than, the main ideas. I believe that writing out these combined theories and models (and representing the data graphically) will take me (close) to the target of 100 pages.
Once I have written the end of Chapter 4, then I will need to rewrite the introduction of Chapter 4 to better mirror the ending. I believe that this will definitely take me to 100 pages if I haven't already reached that.
Why is 100 pages my target? The model dissertation that I am following has a 95-page dissertation, and that dissertation is considered one of the longer dissertations in the recent history of my department. I believe in the philosophy of writing more pages for the first draft than than the anticipated final draft because it is easier to remove than to add. I had this same philosophy for writing Chapter 2, where I had 50 pages for the target for my first draft, then my co-chairs requested another section to be added, so I added 72 more pages for the second draft. And in the final draft I ended cutting out whole chunks of sections to reduce the page limit to 47. I hope that this pattern is not repeated for Chapter 4.
I am trying very hard to limit my writing so far in Chapter 4. It is difficult to constrain myself when making a point because I want to provide evidence, and it's difficult for me to judge whether I'm providing my readers with too much or too little evidence. I have purposely played with doing both at times to see where my co-chairs' preferences are. At this point, I have no idea if my impression of too much or too little evidence is the same as theirs. Reflecting upon the writing process so far, I feel as though I have practiced restraint for the most part. In some instances the restraint feels effective in others it feels like I might be giving off the impression that I don't have enough evidence when I do. That's one practical reason for chairs and dissertation committees, although I don't know how similar my dissertation committee is to a peer review panel of a research journal.
Anyway, the feeling today is satisfaction. I am glad I'm producing a substantial amount of pages and getting my ideas out on "paper." My organization is comprehensible but rough and I believe a second edit will provide a more coherent and cohesive presentation of my grounded theory.
Sunday, December 29, 2013
Sunday, December 15, 2013
Participant Summaries
This weekend I spent several hours writing individual summaries for each of my participants for two purposes: 1) to send to the respective participants for the purpose of member checking, and 2) to use in Chapter 4. This writing process amazed me by how little and how much I could write. Because these were summaries of the participants and not any theory, I tried my best to describe them without hypothesizing or forming a ground theory. In this sense, I wrote little in the way of my findings based on any previous conceptual models or theories.
However, I produced a total of about fifteen pages that could be inserted into Chapter 4. This is a bit alarming. These fifteen pages mostly serve as single-case analyses. There is no cross-case analysis, which is where most of the grounded theory is emerging. I fear that I leave this fifteen pages as they are, which I most likely will not, the scholarly reader may want me to get to the point.
Writing these fifteen pages or so of participant summaries show how I could have written small case studies on each. There were a few participants that seemed to have provided me enough data for a single case study, and there were a couple that did not. In this sense, the writing process has helped me reflect on how I succeeded and failed in the data collection process. For example, there was one participant who answered interview questions very succinctly. Even during the time of the interview, I was not satisfied with the quality of his answers, but I felt like I would be pestering him by asking him to elaborate on nearly every question. Even though I was able to get him to elaborate on a few key questions, this writing processes demonstrated that I should have been more aggressive. However, this may have jeopardized any rapport I had with him. Perhaps this is evidence of my newness to conducting qualitative interview. One of my principles as a researcher is to respect my participants so they feel like active contributors to the research process. I don't want them to feel like they are just regurgitating data for my benefit.
Another surprising element in this writing process is that I was able to identify at least three themes for each participant and describe them with bits of evidence from their blogs and interviews within a few pages. I was quite happy that I was able to do this for the purposes of publishing in a journal with very strict limitations on word count. However, I was a bit upset because I felt like that I could have written at least triple the amount of pages for most of the participants. If I had done this, the single-case study reports would be around fifty pages. That's fifty pages of description with little or no grounded theory and no cross-case analysis.
That reflection helped me to develop a strategy for tackling the monstrosity that can be Chapter 4. The most important point of Chapter 4 is to describe my grounded theory in the cross-case analysis clearly with enough support and evidence from the single-case analyses. I do not want to misdirect my report towards the unique themes that arose from each case. There are enough cross-case patterns that answer my research questions that I need to focus on report on those to make my own case.
Finally, writing a summary for each of my participants is another example of how often I revisit my participants data. Looking back at the minimum number of past visits after the first blog reading or interview, here is a rough count of the number of times I have read through the data for each participant:
As a side note, this posting may provide an example of how research can encourage obsessive-compulsive behavior. If I weren't doing research, this behavior would be truly upsetting to my family, my participants, and myself.
However, I produced a total of about fifteen pages that could be inserted into Chapter 4. This is a bit alarming. These fifteen pages mostly serve as single-case analyses. There is no cross-case analysis, which is where most of the grounded theory is emerging. I fear that I leave this fifteen pages as they are, which I most likely will not, the scholarly reader may want me to get to the point.
Writing these fifteen pages or so of participant summaries show how I could have written small case studies on each. There were a few participants that seemed to have provided me enough data for a single case study, and there were a couple that did not. In this sense, the writing process has helped me reflect on how I succeeded and failed in the data collection process. For example, there was one participant who answered interview questions very succinctly. Even during the time of the interview, I was not satisfied with the quality of his answers, but I felt like I would be pestering him by asking him to elaborate on nearly every question. Even though I was able to get him to elaborate on a few key questions, this writing processes demonstrated that I should have been more aggressive. However, this may have jeopardized any rapport I had with him. Perhaps this is evidence of my newness to conducting qualitative interview. One of my principles as a researcher is to respect my participants so they feel like active contributors to the research process. I don't want them to feel like they are just regurgitating data for my benefit.
Another surprising element in this writing process is that I was able to identify at least three themes for each participant and describe them with bits of evidence from their blogs and interviews within a few pages. I was quite happy that I was able to do this for the purposes of publishing in a journal with very strict limitations on word count. However, I was a bit upset because I felt like that I could have written at least triple the amount of pages for most of the participants. If I had done this, the single-case study reports would be around fifty pages. That's fifty pages of description with little or no grounded theory and no cross-case analysis.
That reflection helped me to develop a strategy for tackling the monstrosity that can be Chapter 4. The most important point of Chapter 4 is to describe my grounded theory in the cross-case analysis clearly with enough support and evidence from the single-case analyses. I do not want to misdirect my report towards the unique themes that arose from each case. There are enough cross-case patterns that answer my research questions that I need to focus on report on those to make my own case.
Finally, writing a summary for each of my participants is another example of how often I revisit my participants data. Looking back at the minimum number of past visits after the first blog reading or interview, here is a rough count of the number of times I have read through the data for each participant:
- Skimming the blog data to make sure the participant met the criteria
- Reading through the blog data after the participant consented for the interview
- Selecting relevant parts of the blog to bring in to parts of the interview
- Copying, pasting, and organizing the blog data onto MS Word documents
- Copying, pasting, and organizing the interview survey data onto MS Word documents
- Transcribing and organizing the online video conferencing interview data onto MS Word documents
- Reading through the first part of interview answers to identify areas to follow up on in the second stage
- Reading through the second stage of interview answers to identify areas to follow up on in the third stage
- Reading through the third stage of interview answers to identify areas to follow up in the closing
- Reading though all of the blog and interview data for the initial stage of coding
- Reorganizing all of the blog and interview data to prepare for the second stage of coding
- Reading through the reorganized and shortened blog and interview data to prepare for analysis
- Organizing the blog and interview data into major themes
- Reading through the blog and interview data now divided into themes instead of participants for cross-case analysis
- Reorganizing the data back to participants with the themes identified in order of strength
- Reading through the reorganized data to form the participant summaries
- Reading through the participant summaries to check if they made more sense to my participants than my dissertation committee for the purpose of member checking
- Preparing the same summaries for Chapter 4 with purpose of using them for reporting on a grounded theory
As a side note, this posting may provide an example of how research can encourage obsessive-compulsive behavior. If I weren't doing research, this behavior would be truly upsetting to my family, my participants, and myself.
Sunday, December 8, 2013
Grounded in Grounded Theory
Today I have begun my cognitive transition from data analysis to reflecting upon my research methods, particularly while reading through Constructing Grounded Theory by Kathy Charmaz. The chapters that require my attention the most are Chapters 3-5. Chapter 3 is about coding in grounded theory practice, something which I believe I have completed. Chapter 4 is about memo-writing, which I believe I have been doing and continue to do. Chapter 5 is about theoretical sampling, saturation, and sorting, which I believe is my next step.
The purpose of this posting is to share my thought processes, in which I am making sure I did the best that I could in terms of coding, that I am writing memos as appropriate to myself and my project, and that I am ready for theoretical sampling, saturation, and sorting and maybe even bits of Chapter 6, reconstructing theory into grounded theory studies.
Coding: What have I done?
In September, I began initial coding through my 3000+ pages of data. The most obvious part of this coding was to identify the chunks of data that directly addressed my research questions, that indirectly addressed my research questions, and that did not address my research questions. I have previous blogged about this step before. An important point in Charmaz' book about this stage was to "move quickly through the data" (p.49). I did not want to get bogged down in this tome of data.
On the next page, Charmaz describes word-by-word and line-by-line coding, both of which I could do for my interview data but would last months if not years if I did so for the blog data. After I selected blog posts that directly and indirectly addressed my research questions, I was then able to do line-by-line and sometimes paragraph-by-paragraph coding. This coding process was different than my coding practice for two qualitative methods courses that I took because the data in those courses were limited to either a certain number of pages or a certain amount of time (for interviews). In those courses, 20 pages of data seemed daunting to most of us.
After my initial coding process, I shared the themes that emerged from the data with my dissertation committee co-chairs. They helped me to discern a common heuristic that unified most of the themes I identified. This heuristic, in term, helped initiate the second stage, which Charmaz called focused coding. I spent the last few weeks engaged in this focused coding, which was faster than I anticipated for a number of reasons:
The second type is theoretical coding, which I thought of because the guiding heuristic is very close to a theoretical model. I decided against invoking the theoretical model for theoretical coding as I wanted the data to emerge without a theory at first as I believe the codes already have a coherent relationships.
Memo-writing: What am I doing?
I have been writing memos ever since my pilot study data collection last year, before my research proposal was approved. The most significant writing times were at various stages of data collection and all throughout the data analysis phase. I discovered that my memo writing at the beginning of data collection revealed that I was too eager to find patterns. I believe I see evidence of my maturity to let the patterns emerge from the data instead of forcing it. I also found that I began to embrace the tenet of KISS (keep it simple, stupid) as I began to gather over a thousand pages of data.
Charmaz writes that there is no formal way or writing memos, and that it depends on the researcher's writing style, personality, and data. While reading through chapter 4, I found many of the techniques to be similar to those from my creative writing courses in how to best organize your thoughts. Although I prefer to be modest, I have discovered that this is one of my strengths. I know how to take notes and communicate effectively to myself.
My favorite bit of advice is to write memos as soon as possible. In one class, we practiced memo writing immediately after collecting data. I found that memo writing after analyzing a selected chunk of data to be more helpful. For data collection, my best memos were when I saw a pattern emerging after the midpoint of the data collection process. Like I wrote earlier, the first part of my data collection memos is most garbage for the study, although it reveals a lot about me.
Theoretical Sampling, Saturation, and Sorting: Am I doing this?
The first question in Chapter 5 that stands out is, "Can you clearly define your themes?" I can easily answer yes. The next concern is that I may not have enough data for these themes, which I do not. When I begin a cross-case analysis, each theme should have at least a full page of evidence; some may even have more than five pages.
Because I believe I have enough data for each category, I do not feel that I need to interview my participants more. However, I believe that a member check with each of my participants may prompt one or more of them to clarify my interpretation of the data.
Reading further in Chapter 6, I realize that, for some of my participants, I have clearly saturated the data because they have conclusions to the experiences I am investigating. Those participants who are still going through these experiences may have a change of perspective by the time of the member check, but I had to decide upon an arbitrary cut point in time or I would have a difficult time ending the project. Besides, I already have 3000+ pages of data, which seems like a good indicator of saturation.
It is clear to me that most theoretical sampling is done and that I have most likely saturated the data for each major category, but I have not physically sorted the categories yet. I have done this to some extent in my head, but that's not reliable enough. I need to write down and visualize theoretical sorting. Once this is done, then I seem to be ready for the next chapter in the textbook, "Reconstructing Theory in Grounded Theory Studies."
The first step is here is to make sure I have a consistent worldview when I use the word "theory" and apply it in my report. Reading through this section of Chapter 7, I am surprised to find myself more on the positivist end than the interpretive end of defining theory. Perhaps this is so because, in the larger scheme of things, I would prefer a mixed-methods approach to the study. Although I'm not measuring anything in this project, I hope to open the door for measuring variables if this study is compelling enough for further investigation.
In the next section of Chapter 7, Charmaz compares constructivist grounded theory to objectivist grounded theory, and I believe I share more the of constructivist's perspective, especially in that one participant's narrative differs slightly between interview data and blog data, mostly in tone. I am a bit surprised to discover that some withhold information from the interview that was revealed (earlier) in the blog.
Since I haven't developed my grounded theory yet, I will stop with a reflection on "Theorizing in Grounded Theory" on page 133. Charmaz shares how researchers view or define the theory of grounded theory:
The purpose of this posting is to share my thought processes, in which I am making sure I did the best that I could in terms of coding, that I am writing memos as appropriate to myself and my project, and that I am ready for theoretical sampling, saturation, and sorting and maybe even bits of Chapter 6, reconstructing theory into grounded theory studies.
Coding: What have I done?
In September, I began initial coding through my 3000+ pages of data. The most obvious part of this coding was to identify the chunks of data that directly addressed my research questions, that indirectly addressed my research questions, and that did not address my research questions. I have previous blogged about this step before. An important point in Charmaz' book about this stage was to "move quickly through the data" (p.49). I did not want to get bogged down in this tome of data.
On the next page, Charmaz describes word-by-word and line-by-line coding, both of which I could do for my interview data but would last months if not years if I did so for the blog data. After I selected blog posts that directly and indirectly addressed my research questions, I was then able to do line-by-line and sometimes paragraph-by-paragraph coding. This coding process was different than my coding practice for two qualitative methods courses that I took because the data in those courses were limited to either a certain number of pages or a certain amount of time (for interviews). In those courses, 20 pages of data seemed daunting to most of us.
After my initial coding process, I shared the themes that emerged from the data with my dissertation committee co-chairs. They helped me to discern a common heuristic that unified most of the themes I identified. This heuristic, in term, helped initiate the second stage, which Charmaz called focused coding. I spent the last few weeks engaged in this focused coding, which was faster than I anticipated for a number of reasons:
- I had already gone through the data twice, so I was much more familiar with the participants and their data. I was getting a better feel for the flow of the blog and interview data.
- The initial codes were still in place, so I could easily look for them and determine how to focus on them.
- I did most of the focused coding over Thanksgiving break, in which I had the most free time since I started my full-time job. I was able to completely immerse myself in the data.
The second type is theoretical coding, which I thought of because the guiding heuristic is very close to a theoretical model. I decided against invoking the theoretical model for theoretical coding as I wanted the data to emerge without a theory at first as I believe the codes already have a coherent relationships.
Memo-writing: What am I doing?
I have been writing memos ever since my pilot study data collection last year, before my research proposal was approved. The most significant writing times were at various stages of data collection and all throughout the data analysis phase. I discovered that my memo writing at the beginning of data collection revealed that I was too eager to find patterns. I believe I see evidence of my maturity to let the patterns emerge from the data instead of forcing it. I also found that I began to embrace the tenet of KISS (keep it simple, stupid) as I began to gather over a thousand pages of data.
Charmaz writes that there is no formal way or writing memos, and that it depends on the researcher's writing style, personality, and data. While reading through chapter 4, I found many of the techniques to be similar to those from my creative writing courses in how to best organize your thoughts. Although I prefer to be modest, I have discovered that this is one of my strengths. I know how to take notes and communicate effectively to myself.
My favorite bit of advice is to write memos as soon as possible. In one class, we practiced memo writing immediately after collecting data. I found that memo writing after analyzing a selected chunk of data to be more helpful. For data collection, my best memos were when I saw a pattern emerging after the midpoint of the data collection process. Like I wrote earlier, the first part of my data collection memos is most garbage for the study, although it reveals a lot about me.
Theoretical Sampling, Saturation, and Sorting: Am I doing this?
The first question in Chapter 5 that stands out is, "Can you clearly define your themes?" I can easily answer yes. The next concern is that I may not have enough data for these themes, which I do not. When I begin a cross-case analysis, each theme should have at least a full page of evidence; some may even have more than five pages.
Because I believe I have enough data for each category, I do not feel that I need to interview my participants more. However, I believe that a member check with each of my participants may prompt one or more of them to clarify my interpretation of the data.
Reading further in Chapter 6, I realize that, for some of my participants, I have clearly saturated the data because they have conclusions to the experiences I am investigating. Those participants who are still going through these experiences may have a change of perspective by the time of the member check, but I had to decide upon an arbitrary cut point in time or I would have a difficult time ending the project. Besides, I already have 3000+ pages of data, which seems like a good indicator of saturation.
It is clear to me that most theoretical sampling is done and that I have most likely saturated the data for each major category, but I have not physically sorted the categories yet. I have done this to some extent in my head, but that's not reliable enough. I need to write down and visualize theoretical sorting. Once this is done, then I seem to be ready for the next chapter in the textbook, "Reconstructing Theory in Grounded Theory Studies."
The first step is here is to make sure I have a consistent worldview when I use the word "theory" and apply it in my report. Reading through this section of Chapter 7, I am surprised to find myself more on the positivist end than the interpretive end of defining theory. Perhaps this is so because, in the larger scheme of things, I would prefer a mixed-methods approach to the study. Although I'm not measuring anything in this project, I hope to open the door for measuring variables if this study is compelling enough for further investigation.
In the next section of Chapter 7, Charmaz compares constructivist grounded theory to objectivist grounded theory, and I believe I share more the of constructivist's perspective, especially in that one participant's narrative differs slightly between interview data and blog data, mostly in tone. I am a bit surprised to discover that some withhold information from the interview that was revealed (earlier) in the blog.
Since I haven't developed my grounded theory yet, I will stop with a reflection on "Theorizing in Grounded Theory" on page 133. Charmaz shares how researchers view or define the theory of grounded theory:
- An empirical generalization - I think I may go down this path. To what extent, I don't know.
- A category - I believe I created a category (a group of English language teachers or sojourners) before grounded theory, but I'll need to investigate this category with the my theory.
- A predisposition - This is similar to "an empirical generalization," and I'm unsure at this point which of the two is stronger.
- An explication of a process - This was my intention for the project, but I believe my data may only reveal a partial explication
- A relationship between variables - There is a good possibility for my grounded theory to do this.
- An explanation - Of course my grounded theory will explain something. It will most likely be a partial explanation. To declare a whole explanation is impossible for an exploratory study like mine.
- An abstract understanding - I hope not. With so much data, I believe I can analyze certain categories more deeply for a grounded theory that is an abstract understanding.
- A description - I hope this is the by-product of my grounded theory. I want to shed more light on the experiences of my target population.
Saturday, December 7, 2013
Single-Case Data Analysis Completed
Although my data analysis is not completed, I believe I have gone deep enough analyzing the interview and blog data for each of my participants. Consistent patterns have emerged enough that I am ready to begin forming a grounded theory. This is when I take all the pieces I have found and arranging them into patterns.
Because the most complex pattern in my data is about social relationships, I will use the data to create graphic organizers (undecided on which type at this time) to visualize the social relationships and how they help answer and address my research questions. I believe I have some firm grounding based in the literature I reviewed, most notably one model that address the host culture complex. I believe I can connect this model with the graphic organizers I will be creating for each of my participants. After I can visualize the social relationships in connection to my research questions and one or more models and theories for each of my participants, then I believe I can begin a cross-case analysis.
This morning I completed this last stage of data analysis for my "last" participant. I consider her the last one only because she was the last participant I interviewed. I wrote in my memos that this analysis affected me more than the other in terms of questioning my research methods. Since this just happened, I'm unsure if I should include this part of self-reflection and critical analysis elsewhere than in my memos. I believe this critical analysis was raised in at least one of my works cited in the literature review, however it is not related to my research questions directly. It raises more philosophical concerns for analyzing qualitative data, making me more aware of my own biases than previously, or at least in the recent past. I am thankful that this last exercise in data analysis has raised this awareness before I write about my findings.
I am now at the point where I need to review the key chapters and sections of my grounded theory texts to make sure I'm ready for the next phase, to better identify the next phase, and to better equip me with the mindset and vocabulary to discuss the next analytical process.
Because the most complex pattern in my data is about social relationships, I will use the data to create graphic organizers (undecided on which type at this time) to visualize the social relationships and how they help answer and address my research questions. I believe I have some firm grounding based in the literature I reviewed, most notably one model that address the host culture complex. I believe I can connect this model with the graphic organizers I will be creating for each of my participants. After I can visualize the social relationships in connection to my research questions and one or more models and theories for each of my participants, then I believe I can begin a cross-case analysis.
This morning I completed this last stage of data analysis for my "last" participant. I consider her the last one only because she was the last participant I interviewed. I wrote in my memos that this analysis affected me more than the other in terms of questioning my research methods. Since this just happened, I'm unsure if I should include this part of self-reflection and critical analysis elsewhere than in my memos. I believe this critical analysis was raised in at least one of my works cited in the literature review, however it is not related to my research questions directly. It raises more philosophical concerns for analyzing qualitative data, making me more aware of my own biases than previously, or at least in the recent past. I am thankful that this last exercise in data analysis has raised this awareness before I write about my findings.
I am now at the point where I need to review the key chapters and sections of my grounded theory texts to make sure I'm ready for the next phase, to better identify the next phase, and to better equip me with the mindset and vocabulary to discuss the next analytical process.
Saturday, November 30, 2013
Deep Analysis
I've spent most of my Thanksgiving break (since Wednesday) conducting deeper single case analyses of my participants. Today I have completed the fourth of five, and I'm pleased with the patterns I'm finding. After I complete the fifth single case analysis either tomorrow or next weekend, I will be able to start organizing the data into categories to form a grounded theory.
My strategy at this point is to continue my immersion into the data without much interference from the literature. From 2011 to early 2013, I have been immersed in the literature, and I believe I have a few theories and models that will fit in well with my grounded theory. I don't know for sure, and I don't want to check as I the data needs to guide the analysis. If I crack open the literature now, I may be tempted to make the data fit a model or theory, which will weaken my grounded theory.
I'm interested to see how my grounded theory supports or aligns with pre-existing theories. Even more so, I'd like to see if my grounded theory shows that another theory or two may be standing on shaky ground because I may have contradictory evidence. However, I would be elated if my grounded theory help link two or more pre-existing theories as my research is at the intersection of multiple disciplines: English language teacher education, sociocultural anthropology, intercultural communication, and adjustment psychology. At this point, I only have feelings and inklings, so I need to be patient and finish the single case analysis.
Because this is a multiple case study, I need to find patterns that are found across all the studies before forming a unified grounded theory. I may be able to create several smaller theories based on a fraction of the multiple cases, but this is less important now. However, my data may reveal that the unique data for each case may help create a more compelling theory.
If you are a well-disciplined and experienced researcher, you may wince at the previous few paragraphs because I seem so naive, but this is the purpose of this post. This is what the mindset is like for someone working on his own research project for the first time from start to finish. To put your mind at ease, once I complete the single case analysis for my fifth and final participant, I plan to revisit the textbooks concerning grounded theory and qualitative data analysis. These will guide me more than my inklings and feelings, which are logged in my data analysis memos.
Another reason I am posting this is that I am a bit giddy with getting this far in my analysis and I love the process of discovery. Every stage of data analysis has been rewarding, and I continue to learn more and more from my data in addition to learning more and more about the research process. Perhaps I'm good at fooling myself, but I believe I'm lucky that I am enjoying the initial stages of data analysis just as much as data collection.
The frustration is that not many others I know care about the whole study. It's like being the only one the theater that laughed at a particular line or situation in the film or play. I wish I could share this joy but I cannot. I'm beginning to understand the mad scientist.
My strategy at this point is to continue my immersion into the data without much interference from the literature. From 2011 to early 2013, I have been immersed in the literature, and I believe I have a few theories and models that will fit in well with my grounded theory. I don't know for sure, and I don't want to check as I the data needs to guide the analysis. If I crack open the literature now, I may be tempted to make the data fit a model or theory, which will weaken my grounded theory.
I'm interested to see how my grounded theory supports or aligns with pre-existing theories. Even more so, I'd like to see if my grounded theory shows that another theory or two may be standing on shaky ground because I may have contradictory evidence. However, I would be elated if my grounded theory help link two or more pre-existing theories as my research is at the intersection of multiple disciplines: English language teacher education, sociocultural anthropology, intercultural communication, and adjustment psychology. At this point, I only have feelings and inklings, so I need to be patient and finish the single case analysis.
Because this is a multiple case study, I need to find patterns that are found across all the studies before forming a unified grounded theory. I may be able to create several smaller theories based on a fraction of the multiple cases, but this is less important now. However, my data may reveal that the unique data for each case may help create a more compelling theory.
If you are a well-disciplined and experienced researcher, you may wince at the previous few paragraphs because I seem so naive, but this is the purpose of this post. This is what the mindset is like for someone working on his own research project for the first time from start to finish. To put your mind at ease, once I complete the single case analysis for my fifth and final participant, I plan to revisit the textbooks concerning grounded theory and qualitative data analysis. These will guide me more than my inklings and feelings, which are logged in my data analysis memos.
Another reason I am posting this is that I am a bit giddy with getting this far in my analysis and I love the process of discovery. Every stage of data analysis has been rewarding, and I continue to learn more and more from my data in addition to learning more and more about the research process. Perhaps I'm good at fooling myself, but I believe I'm lucky that I am enjoying the initial stages of data analysis just as much as data collection.
The frustration is that not many others I know care about the whole study. It's like being the only one the theater that laughed at a particular line or situation in the film or play. I wish I could share this joy but I cannot. I'm beginning to understand the mad scientist.
Saturday, November 23, 2013
The Long Road to Analysis
In late September, I completed my data collection, which actually ended in early August when including the removal of the 6th participant. In early October, I met with one of my dissertation co-chairs to discuss the data and what to do with it. It was agreed that I should code for themes with a date set in late October, which I didn't believe I could meet because my full-time job was becoming more hectic and the added stress would take away from productive time for my dissertation. While that was true for a week or so, I was able to meet the deadline because there was a long enough lull in October for me to complete coding for themes.
Although only 10 themes were requested, I submitted 11 themes that stood out among all the participants. I also included smaller themes that only emerged across one or some of the participants. Once my co-chairs received my themes, I knew that I wouldn't hear from them soon because I sent them a lot more than probably both parties anticipated. Additionally, I submitted the themes right before a busy conference season and shortly after the midterm week of the fall semester.
To bide my time, I took the suggestion from the same co-chair to revise chapter 3, in which I proposed my research methods. Now that the data collection was finished and the data analysis had begun, I could change the whole chapter from future tense to past tense. I also deleted the report on my pilot study from the chapter because the pilot study was not approved by IRB, thus prohibiting me from using and publishing the data.
I got a little anxious a few weeks later when I felt like I wasn't being productive with my dissertation. I couldn't move forward with the data analysis because I hadn't received feedback from my themes yet. To feel productive, I started looking for scholarly journals to inspire me to write about my seemingly unique data collection process as it regards to social media. I have received feedback from a few peers that believe it is worth writing about for publication. In addition, I wanted to start engaging social media more directly too.
Just as I was gaining steam with these projects, I got an automated message from IRB asking for an annual update on the project. I hadn't realized that it was almost a year since I started filling out the IRB application. Because data collection was over, I didn't expect IRB to find any issues with my progress. I was wrong. On my application, I didn't describe my member checking process that I planned to initiate after I completed data analysis. Although I implied it in the email scripts and interview questions, it was not made clear in the application. Understandably, IRB loves clarity. Fortunately for me, it only took about a week to resolve this issue. A few days later, I received feedback about my themes, and it was a joyous occasion.
Getting the green light to continue my analysis was one reason for my elation, but another was that the feedback was more helpful than I expected as a guide. I could instantly plan a strategy for analysis that would take me to forming a grounded theory in line with some established models and theories from the literature. It took me a good week to calm down from this euphoria before I could resume analysis. Another reason for the week delay was yet another hectic week at my job.
Now that Thanksgiving week has arrived, I will have plenty of time to get a large chunk of the deeper analysis completed. I am now projecting to complete these stage of analysis by the second week in December, leaving me the time up to Christmas to determine if I have enough evidence and a strong enough argument for a grounded theory. I am certain that I have enough evidence, but I'm not as confident with my argument yet, and that's what this next stage of analysis is all about.
You don't need to guess what I will be thankful for this year's Thanksgiving! All in all, I am still enjoying the majority of this dissertation process. The key to this enjoyment is my passion for the project and already realizing practical implications through social media and my current full-time job helping faculty with professional and curriculum development.
Although only 10 themes were requested, I submitted 11 themes that stood out among all the participants. I also included smaller themes that only emerged across one or some of the participants. Once my co-chairs received my themes, I knew that I wouldn't hear from them soon because I sent them a lot more than probably both parties anticipated. Additionally, I submitted the themes right before a busy conference season and shortly after the midterm week of the fall semester.
To bide my time, I took the suggestion from the same co-chair to revise chapter 3, in which I proposed my research methods. Now that the data collection was finished and the data analysis had begun, I could change the whole chapter from future tense to past tense. I also deleted the report on my pilot study from the chapter because the pilot study was not approved by IRB, thus prohibiting me from using and publishing the data.
I got a little anxious a few weeks later when I felt like I wasn't being productive with my dissertation. I couldn't move forward with the data analysis because I hadn't received feedback from my themes yet. To feel productive, I started looking for scholarly journals to inspire me to write about my seemingly unique data collection process as it regards to social media. I have received feedback from a few peers that believe it is worth writing about for publication. In addition, I wanted to start engaging social media more directly too.
Just as I was gaining steam with these projects, I got an automated message from IRB asking for an annual update on the project. I hadn't realized that it was almost a year since I started filling out the IRB application. Because data collection was over, I didn't expect IRB to find any issues with my progress. I was wrong. On my application, I didn't describe my member checking process that I planned to initiate after I completed data analysis. Although I implied it in the email scripts and interview questions, it was not made clear in the application. Understandably, IRB loves clarity. Fortunately for me, it only took about a week to resolve this issue. A few days later, I received feedback about my themes, and it was a joyous occasion.
Getting the green light to continue my analysis was one reason for my elation, but another was that the feedback was more helpful than I expected as a guide. I could instantly plan a strategy for analysis that would take me to forming a grounded theory in line with some established models and theories from the literature. It took me a good week to calm down from this euphoria before I could resume analysis. Another reason for the week delay was yet another hectic week at my job.
Now that Thanksgiving week has arrived, I will have plenty of time to get a large chunk of the deeper analysis completed. I am now projecting to complete these stage of analysis by the second week in December, leaving me the time up to Christmas to determine if I have enough evidence and a strong enough argument for a grounded theory. I am certain that I have enough evidence, but I'm not as confident with my argument yet, and that's what this next stage of analysis is all about.
You don't need to guess what I will be thankful for this year's Thanksgiving! All in all, I am still enjoying the majority of this dissertation process. The key to this enjoyment is my passion for the project and already realizing practical implications through social media and my current full-time job helping faculty with professional and curriculum development.
Thursday, September 19, 2013
IRB and Social Media Research
This week I encountered an interesting situation regarding IRB and my data collection procedures. My last participant, who I blogged about in the previous posts, published his online video response to my interview questions on YouTube. Although I knew he would create online videos, I didn't expect him to post it online so quickly. This route of collecting interview data was not approved by IRB, so I understand the ethical implications here. I informed IRB that I would collect data via Qualtrics or Adobe Connect.
Back in January, I spent a few weeks through email discussions finding the best way to collect interview data from my participants who live abroad. My first suggestions were email interviews and Skype interviews, which have been done many times in the literature I reviewed. These were not acceptable methods for IRB because there is a confidentiality risk because the University of Iowa does not host my participant's email accounts (so they could approve my emails to them, but not their emails to me) and the university does not host Skype. Because I had experience with Adobe Connect, the web conferencing alternative was easy to choose. However, I had difficulty accepting Qualtrics as an alternative to email interviews because it's a survey software program, so the interview would flow more like a survey with 99% open-ended questions. Nowhere in the literature did I find Qualtrics being used for interviews. Even after collecting the data, it was more like a survey, but IRB approved that route. And the university's IT advisor on data collection also agreed that Qualtrics was the best way to go.
So when my participant suggested submitting a video, I knew that IRB would not support it given the arguments for not permitting their email accounts or Skype. It would have to be something that the university could provide. In retrospect, perhaps IRB had a waiver for the participant to sign saying that he acknowledges that his data would not be confidential. He didn't care as he willingly and knowingly made it public. He didn't have the time for spending time on Qualtrics. Perhaps though stronger persuasion, he may have done an Adobe Connect interview. But I got the impression he wanted to be in control of the data, so perhaps where IRB drew the line.
IRB's conclusion was to order me to destroy the data, which is an order that informed me that IRB does not clearly understand social media research. First of all, I do not own the data, so how can I destroy it? They did not request me to ask/tell/order my participant to take the video off YouTube. I'm sure that's another ethical complication there as he has the right to refuse the order. Nonetheless, thousands of people have already viewed the video so the damage, if any, is done.
The damage is completely internal or procedural, but the main purpose of IRB is protect human subjects from physical and psychological harm. It seems to me that I've been asked to "destroy" the data out of a technicality that had no threat of physical or psychological harm to my participant, especially given the fact that he has published over a thousand of YouTube videos with similar or even more personal/revealing narratives.
Finally, I'd like to analyze the term "destroy" a bit further. Once something is published online, it is never destroyed. It always exists in a server somewhere, so I cannot completely physically destroy the data. One of my dissertation co-chairs used the term "remove," but she didn't specify from what. I cannot remove it from the Internet, but I can remove it from my dissertation. The data still exists for anyone to collect, analyze, and write about. I am happy to have played a role in providing this data to those who find it useful. Looking at the comments under the YouTube video, I found that some English language teachers have already found it helpful, so one of my goals for publishing my dissertation has been accomplished. So I'm learning that this type of publication may affect more change than publishing through a peer-reviewed article, which very few practitioners read anyway. I'm all for accessibility.
Back in January, I spent a few weeks through email discussions finding the best way to collect interview data from my participants who live abroad. My first suggestions were email interviews and Skype interviews, which have been done many times in the literature I reviewed. These were not acceptable methods for IRB because there is a confidentiality risk because the University of Iowa does not host my participant's email accounts (so they could approve my emails to them, but not their emails to me) and the university does not host Skype. Because I had experience with Adobe Connect, the web conferencing alternative was easy to choose. However, I had difficulty accepting Qualtrics as an alternative to email interviews because it's a survey software program, so the interview would flow more like a survey with 99% open-ended questions. Nowhere in the literature did I find Qualtrics being used for interviews. Even after collecting the data, it was more like a survey, but IRB approved that route. And the university's IT advisor on data collection also agreed that Qualtrics was the best way to go.
So when my participant suggested submitting a video, I knew that IRB would not support it given the arguments for not permitting their email accounts or Skype. It would have to be something that the university could provide. In retrospect, perhaps IRB had a waiver for the participant to sign saying that he acknowledges that his data would not be confidential. He didn't care as he willingly and knowingly made it public. He didn't have the time for spending time on Qualtrics. Perhaps though stronger persuasion, he may have done an Adobe Connect interview. But I got the impression he wanted to be in control of the data, so perhaps where IRB drew the line.
IRB's conclusion was to order me to destroy the data, which is an order that informed me that IRB does not clearly understand social media research. First of all, I do not own the data, so how can I destroy it? They did not request me to ask/tell/order my participant to take the video off YouTube. I'm sure that's another ethical complication there as he has the right to refuse the order. Nonetheless, thousands of people have already viewed the video so the damage, if any, is done.
The damage is completely internal or procedural, but the main purpose of IRB is protect human subjects from physical and psychological harm. It seems to me that I've been asked to "destroy" the data out of a technicality that had no threat of physical or psychological harm to my participant, especially given the fact that he has published over a thousand of YouTube videos with similar or even more personal/revealing narratives.
Finally, I'd like to analyze the term "destroy" a bit further. Once something is published online, it is never destroyed. It always exists in a server somewhere, so I cannot completely physically destroy the data. One of my dissertation co-chairs used the term "remove," but she didn't specify from what. I cannot remove it from the Internet, but I can remove it from my dissertation. The data still exists for anyone to collect, analyze, and write about. I am happy to have played a role in providing this data to those who find it useful. Looking at the comments under the YouTube video, I found that some English language teachers have already found it helpful, so one of my goals for publishing my dissertation has been accomplished. So I'm learning that this type of publication may affect more change than publishing through a peer-reviewed article, which very few practitioners read anyway. I'm all for accessibility.
Subscribe to:
Comments (Atom)