Writing Chapter 4 is usually one of the most grueling and tiresome points of the dissertation process, but for once timing was on my side as I am enjoying the opportunity to write my first draft of Chapter 4 in warm and sunny Florida. I have been looking forward to this opportunity for over a month when I began to feel prepared to write the chapter.
Since Thanksgiving weekend, I have not had the opportunity to write Chapter 4 because I was either busy preparing or enjoying holiday events, visits to or from family and friends, or writing member checks for my participants. During my downtime in December, however, I was plotting and outlining Chapter 4 in my mind. I never got the chance to sit down and write any of it until last week.
My family and I have only been in Florida for 6 days and yet I have already written 55 pages of Chapter 4. Two of these 6 days don't count because one was the day of arrival and the other was Christmas day, so I really wrote 55 pages in 4 days. Today marks the point in which I completed writing the major components of my grounded theory, so I am quite satisfied now.
Looking over those rough 55 pages, there is plenty of room to flesh out my weaker arguments and transitions. I also have a lot of ideas of how to reorganize the content of those pages. I believe in my second edit of these pages, I would probably remove as much as or a little less than I would add.
My goal is to hit 100 pages before we leave Florida, and I'm making good progress on that. With the bulk of my argument already covered, I am worried less about quantity and worried more about the quality. With the 55 pages covering the biggest cross-case patterns, I anticipate the next 10-15 pages will cover the significant single-case findings that directly answer my research questions. So I plan on spending Monday and maybe Tuesday (the remainder of 2013) writing that section.
After writing the significant single-case findings, I plan to tie my grounded theory with at least two established models, one in the field of acculturation psychology and the other in the field of English language education. I'm trying to imagine a graphic organizer to represent how I plan to combine my grounded theory with these two models. For now, this imagined graphic organizer is too complicated and messy, although the main ideas seem clear. I need to create a graphic organizer that is clear, if not clearer than, the main ideas. I believe that writing out these combined theories and models (and representing the data graphically) will take me (close) to the target of 100 pages.
Once I have written the end of Chapter 4, then I will need to rewrite the introduction of Chapter 4 to better mirror the ending. I believe that this will definitely take me to 100 pages if I haven't already reached that.
Why is 100 pages my target? The model dissertation that I am following has a 95-page dissertation, and that dissertation is considered one of the longer dissertations in the recent history of my department. I believe in the philosophy of writing more pages for the first draft than than the anticipated final draft because it is easier to remove than to add. I had this same philosophy for writing Chapter 2, where I had 50 pages for the target for my first draft, then my co-chairs requested another section to be added, so I added 72 more pages for the second draft. And in the final draft I ended cutting out whole chunks of sections to reduce the page limit to 47. I hope that this pattern is not repeated for Chapter 4.
I am trying very hard to limit my writing so far in Chapter 4. It is difficult to constrain myself when making a point because I want to provide evidence, and it's difficult for me to judge whether I'm providing my readers with too much or too little evidence. I have purposely played with doing both at times to see where my co-chairs' preferences are. At this point, I have no idea if my impression of too much or too little evidence is the same as theirs. Reflecting upon the writing process so far, I feel as though I have practiced restraint for the most part. In some instances the restraint feels effective in others it feels like I might be giving off the impression that I don't have enough evidence when I do. That's one practical reason for chairs and dissertation committees, although I don't know how similar my dissertation committee is to a peer review panel of a research journal.
Anyway, the feeling today is satisfaction. I am glad I'm producing a substantial amount of pages and getting my ideas out on "paper." My organization is comprehensible but rough and I believe a second edit will provide a more coherent and cohesive presentation of my grounded theory.
Sunday, December 29, 2013
Sunday, December 15, 2013
Participant Summaries
This weekend I spent several hours writing individual summaries for each of my participants for two purposes: 1) to send to the respective participants for the purpose of member checking, and 2) to use in Chapter 4. This writing process amazed me by how little and how much I could write. Because these were summaries of the participants and not any theory, I tried my best to describe them without hypothesizing or forming a ground theory. In this sense, I wrote little in the way of my findings based on any previous conceptual models or theories.
However, I produced a total of about fifteen pages that could be inserted into Chapter 4. This is a bit alarming. These fifteen pages mostly serve as single-case analyses. There is no cross-case analysis, which is where most of the grounded theory is emerging. I fear that I leave this fifteen pages as they are, which I most likely will not, the scholarly reader may want me to get to the point.
Writing these fifteen pages or so of participant summaries show how I could have written small case studies on each. There were a few participants that seemed to have provided me enough data for a single case study, and there were a couple that did not. In this sense, the writing process has helped me reflect on how I succeeded and failed in the data collection process. For example, there was one participant who answered interview questions very succinctly. Even during the time of the interview, I was not satisfied with the quality of his answers, but I felt like I would be pestering him by asking him to elaborate on nearly every question. Even though I was able to get him to elaborate on a few key questions, this writing processes demonstrated that I should have been more aggressive. However, this may have jeopardized any rapport I had with him. Perhaps this is evidence of my newness to conducting qualitative interview. One of my principles as a researcher is to respect my participants so they feel like active contributors to the research process. I don't want them to feel like they are just regurgitating data for my benefit.
Another surprising element in this writing process is that I was able to identify at least three themes for each participant and describe them with bits of evidence from their blogs and interviews within a few pages. I was quite happy that I was able to do this for the purposes of publishing in a journal with very strict limitations on word count. However, I was a bit upset because I felt like that I could have written at least triple the amount of pages for most of the participants. If I had done this, the single-case study reports would be around fifty pages. That's fifty pages of description with little or no grounded theory and no cross-case analysis.
That reflection helped me to develop a strategy for tackling the monstrosity that can be Chapter 4. The most important point of Chapter 4 is to describe my grounded theory in the cross-case analysis clearly with enough support and evidence from the single-case analyses. I do not want to misdirect my report towards the unique themes that arose from each case. There are enough cross-case patterns that answer my research questions that I need to focus on report on those to make my own case.
Finally, writing a summary for each of my participants is another example of how often I revisit my participants data. Looking back at the minimum number of past visits after the first blog reading or interview, here is a rough count of the number of times I have read through the data for each participant:
As a side note, this posting may provide an example of how research can encourage obsessive-compulsive behavior. If I weren't doing research, this behavior would be truly upsetting to my family, my participants, and myself.
However, I produced a total of about fifteen pages that could be inserted into Chapter 4. This is a bit alarming. These fifteen pages mostly serve as single-case analyses. There is no cross-case analysis, which is where most of the grounded theory is emerging. I fear that I leave this fifteen pages as they are, which I most likely will not, the scholarly reader may want me to get to the point.
Writing these fifteen pages or so of participant summaries show how I could have written small case studies on each. There were a few participants that seemed to have provided me enough data for a single case study, and there were a couple that did not. In this sense, the writing process has helped me reflect on how I succeeded and failed in the data collection process. For example, there was one participant who answered interview questions very succinctly. Even during the time of the interview, I was not satisfied with the quality of his answers, but I felt like I would be pestering him by asking him to elaborate on nearly every question. Even though I was able to get him to elaborate on a few key questions, this writing processes demonstrated that I should have been more aggressive. However, this may have jeopardized any rapport I had with him. Perhaps this is evidence of my newness to conducting qualitative interview. One of my principles as a researcher is to respect my participants so they feel like active contributors to the research process. I don't want them to feel like they are just regurgitating data for my benefit.
Another surprising element in this writing process is that I was able to identify at least three themes for each participant and describe them with bits of evidence from their blogs and interviews within a few pages. I was quite happy that I was able to do this for the purposes of publishing in a journal with very strict limitations on word count. However, I was a bit upset because I felt like that I could have written at least triple the amount of pages for most of the participants. If I had done this, the single-case study reports would be around fifty pages. That's fifty pages of description with little or no grounded theory and no cross-case analysis.
That reflection helped me to develop a strategy for tackling the monstrosity that can be Chapter 4. The most important point of Chapter 4 is to describe my grounded theory in the cross-case analysis clearly with enough support and evidence from the single-case analyses. I do not want to misdirect my report towards the unique themes that arose from each case. There are enough cross-case patterns that answer my research questions that I need to focus on report on those to make my own case.
Finally, writing a summary for each of my participants is another example of how often I revisit my participants data. Looking back at the minimum number of past visits after the first blog reading or interview, here is a rough count of the number of times I have read through the data for each participant:
- Skimming the blog data to make sure the participant met the criteria
- Reading through the blog data after the participant consented for the interview
- Selecting relevant parts of the blog to bring in to parts of the interview
- Copying, pasting, and organizing the blog data onto MS Word documents
- Copying, pasting, and organizing the interview survey data onto MS Word documents
- Transcribing and organizing the online video conferencing interview data onto MS Word documents
- Reading through the first part of interview answers to identify areas to follow up on in the second stage
- Reading through the second stage of interview answers to identify areas to follow up on in the third stage
- Reading through the third stage of interview answers to identify areas to follow up in the closing
- Reading though all of the blog and interview data for the initial stage of coding
- Reorganizing all of the blog and interview data to prepare for the second stage of coding
- Reading through the reorganized and shortened blog and interview data to prepare for analysis
- Organizing the blog and interview data into major themes
- Reading through the blog and interview data now divided into themes instead of participants for cross-case analysis
- Reorganizing the data back to participants with the themes identified in order of strength
- Reading through the reorganized data to form the participant summaries
- Reading through the participant summaries to check if they made more sense to my participants than my dissertation committee for the purpose of member checking
- Preparing the same summaries for Chapter 4 with purpose of using them for reporting on a grounded theory
As a side note, this posting may provide an example of how research can encourage obsessive-compulsive behavior. If I weren't doing research, this behavior would be truly upsetting to my family, my participants, and myself.
Sunday, December 8, 2013
Grounded in Grounded Theory
Today I have begun my cognitive transition from data analysis to reflecting upon my research methods, particularly while reading through Constructing Grounded Theory by Kathy Charmaz. The chapters that require my attention the most are Chapters 3-5. Chapter 3 is about coding in grounded theory practice, something which I believe I have completed. Chapter 4 is about memo-writing, which I believe I have been doing and continue to do. Chapter 5 is about theoretical sampling, saturation, and sorting, which I believe is my next step.
The purpose of this posting is to share my thought processes, in which I am making sure I did the best that I could in terms of coding, that I am writing memos as appropriate to myself and my project, and that I am ready for theoretical sampling, saturation, and sorting and maybe even bits of Chapter 6, reconstructing theory into grounded theory studies.
Coding: What have I done?
In September, I began initial coding through my 3000+ pages of data. The most obvious part of this coding was to identify the chunks of data that directly addressed my research questions, that indirectly addressed my research questions, and that did not address my research questions. I have previous blogged about this step before. An important point in Charmaz' book about this stage was to "move quickly through the data" (p.49). I did not want to get bogged down in this tome of data.
On the next page, Charmaz describes word-by-word and line-by-line coding, both of which I could do for my interview data but would last months if not years if I did so for the blog data. After I selected blog posts that directly and indirectly addressed my research questions, I was then able to do line-by-line and sometimes paragraph-by-paragraph coding. This coding process was different than my coding practice for two qualitative methods courses that I took because the data in those courses were limited to either a certain number of pages or a certain amount of time (for interviews). In those courses, 20 pages of data seemed daunting to most of us.
After my initial coding process, I shared the themes that emerged from the data with my dissertation committee co-chairs. They helped me to discern a common heuristic that unified most of the themes I identified. This heuristic, in term, helped initiate the second stage, which Charmaz called focused coding. I spent the last few weeks engaged in this focused coding, which was faster than I anticipated for a number of reasons:
The second type is theoretical coding, which I thought of because the guiding heuristic is very close to a theoretical model. I decided against invoking the theoretical model for theoretical coding as I wanted the data to emerge without a theory at first as I believe the codes already have a coherent relationships.
Memo-writing: What am I doing?
I have been writing memos ever since my pilot study data collection last year, before my research proposal was approved. The most significant writing times were at various stages of data collection and all throughout the data analysis phase. I discovered that my memo writing at the beginning of data collection revealed that I was too eager to find patterns. I believe I see evidence of my maturity to let the patterns emerge from the data instead of forcing it. I also found that I began to embrace the tenet of KISS (keep it simple, stupid) as I began to gather over a thousand pages of data.
Charmaz writes that there is no formal way or writing memos, and that it depends on the researcher's writing style, personality, and data. While reading through chapter 4, I found many of the techniques to be similar to those from my creative writing courses in how to best organize your thoughts. Although I prefer to be modest, I have discovered that this is one of my strengths. I know how to take notes and communicate effectively to myself.
My favorite bit of advice is to write memos as soon as possible. In one class, we practiced memo writing immediately after collecting data. I found that memo writing after analyzing a selected chunk of data to be more helpful. For data collection, my best memos were when I saw a pattern emerging after the midpoint of the data collection process. Like I wrote earlier, the first part of my data collection memos is most garbage for the study, although it reveals a lot about me.
Theoretical Sampling, Saturation, and Sorting: Am I doing this?
The first question in Chapter 5 that stands out is, "Can you clearly define your themes?" I can easily answer yes. The next concern is that I may not have enough data for these themes, which I do not. When I begin a cross-case analysis, each theme should have at least a full page of evidence; some may even have more than five pages.
Because I believe I have enough data for each category, I do not feel that I need to interview my participants more. However, I believe that a member check with each of my participants may prompt one or more of them to clarify my interpretation of the data.
Reading further in Chapter 6, I realize that, for some of my participants, I have clearly saturated the data because they have conclusions to the experiences I am investigating. Those participants who are still going through these experiences may have a change of perspective by the time of the member check, but I had to decide upon an arbitrary cut point in time or I would have a difficult time ending the project. Besides, I already have 3000+ pages of data, which seems like a good indicator of saturation.
It is clear to me that most theoretical sampling is done and that I have most likely saturated the data for each major category, but I have not physically sorted the categories yet. I have done this to some extent in my head, but that's not reliable enough. I need to write down and visualize theoretical sorting. Once this is done, then I seem to be ready for the next chapter in the textbook, "Reconstructing Theory in Grounded Theory Studies."
The first step is here is to make sure I have a consistent worldview when I use the word "theory" and apply it in my report. Reading through this section of Chapter 7, I am surprised to find myself more on the positivist end than the interpretive end of defining theory. Perhaps this is so because, in the larger scheme of things, I would prefer a mixed-methods approach to the study. Although I'm not measuring anything in this project, I hope to open the door for measuring variables if this study is compelling enough for further investigation.
In the next section of Chapter 7, Charmaz compares constructivist grounded theory to objectivist grounded theory, and I believe I share more the of constructivist's perspective, especially in that one participant's narrative differs slightly between interview data and blog data, mostly in tone. I am a bit surprised to discover that some withhold information from the interview that was revealed (earlier) in the blog.
Since I haven't developed my grounded theory yet, I will stop with a reflection on "Theorizing in Grounded Theory" on page 133. Charmaz shares how researchers view or define the theory of grounded theory:
The purpose of this posting is to share my thought processes, in which I am making sure I did the best that I could in terms of coding, that I am writing memos as appropriate to myself and my project, and that I am ready for theoretical sampling, saturation, and sorting and maybe even bits of Chapter 6, reconstructing theory into grounded theory studies.
Coding: What have I done?
In September, I began initial coding through my 3000+ pages of data. The most obvious part of this coding was to identify the chunks of data that directly addressed my research questions, that indirectly addressed my research questions, and that did not address my research questions. I have previous blogged about this step before. An important point in Charmaz' book about this stage was to "move quickly through the data" (p.49). I did not want to get bogged down in this tome of data.
On the next page, Charmaz describes word-by-word and line-by-line coding, both of which I could do for my interview data but would last months if not years if I did so for the blog data. After I selected blog posts that directly and indirectly addressed my research questions, I was then able to do line-by-line and sometimes paragraph-by-paragraph coding. This coding process was different than my coding practice for two qualitative methods courses that I took because the data in those courses were limited to either a certain number of pages or a certain amount of time (for interviews). In those courses, 20 pages of data seemed daunting to most of us.
After my initial coding process, I shared the themes that emerged from the data with my dissertation committee co-chairs. They helped me to discern a common heuristic that unified most of the themes I identified. This heuristic, in term, helped initiate the second stage, which Charmaz called focused coding. I spent the last few weeks engaged in this focused coding, which was faster than I anticipated for a number of reasons:
- I had already gone through the data twice, so I was much more familiar with the participants and their data. I was getting a better feel for the flow of the blog and interview data.
- The initial codes were still in place, so I could easily look for them and determine how to focus on them.
- I did most of the focused coding over Thanksgiving break, in which I had the most free time since I started my full-time job. I was able to completely immerse myself in the data.
The second type is theoretical coding, which I thought of because the guiding heuristic is very close to a theoretical model. I decided against invoking the theoretical model for theoretical coding as I wanted the data to emerge without a theory at first as I believe the codes already have a coherent relationships.
Memo-writing: What am I doing?
I have been writing memos ever since my pilot study data collection last year, before my research proposal was approved. The most significant writing times were at various stages of data collection and all throughout the data analysis phase. I discovered that my memo writing at the beginning of data collection revealed that I was too eager to find patterns. I believe I see evidence of my maturity to let the patterns emerge from the data instead of forcing it. I also found that I began to embrace the tenet of KISS (keep it simple, stupid) as I began to gather over a thousand pages of data.
Charmaz writes that there is no formal way or writing memos, and that it depends on the researcher's writing style, personality, and data. While reading through chapter 4, I found many of the techniques to be similar to those from my creative writing courses in how to best organize your thoughts. Although I prefer to be modest, I have discovered that this is one of my strengths. I know how to take notes and communicate effectively to myself.
My favorite bit of advice is to write memos as soon as possible. In one class, we practiced memo writing immediately after collecting data. I found that memo writing after analyzing a selected chunk of data to be more helpful. For data collection, my best memos were when I saw a pattern emerging after the midpoint of the data collection process. Like I wrote earlier, the first part of my data collection memos is most garbage for the study, although it reveals a lot about me.
Theoretical Sampling, Saturation, and Sorting: Am I doing this?
The first question in Chapter 5 that stands out is, "Can you clearly define your themes?" I can easily answer yes. The next concern is that I may not have enough data for these themes, which I do not. When I begin a cross-case analysis, each theme should have at least a full page of evidence; some may even have more than five pages.
Because I believe I have enough data for each category, I do not feel that I need to interview my participants more. However, I believe that a member check with each of my participants may prompt one or more of them to clarify my interpretation of the data.
Reading further in Chapter 6, I realize that, for some of my participants, I have clearly saturated the data because they have conclusions to the experiences I am investigating. Those participants who are still going through these experiences may have a change of perspective by the time of the member check, but I had to decide upon an arbitrary cut point in time or I would have a difficult time ending the project. Besides, I already have 3000+ pages of data, which seems like a good indicator of saturation.
It is clear to me that most theoretical sampling is done and that I have most likely saturated the data for each major category, but I have not physically sorted the categories yet. I have done this to some extent in my head, but that's not reliable enough. I need to write down and visualize theoretical sorting. Once this is done, then I seem to be ready for the next chapter in the textbook, "Reconstructing Theory in Grounded Theory Studies."
The first step is here is to make sure I have a consistent worldview when I use the word "theory" and apply it in my report. Reading through this section of Chapter 7, I am surprised to find myself more on the positivist end than the interpretive end of defining theory. Perhaps this is so because, in the larger scheme of things, I would prefer a mixed-methods approach to the study. Although I'm not measuring anything in this project, I hope to open the door for measuring variables if this study is compelling enough for further investigation.
In the next section of Chapter 7, Charmaz compares constructivist grounded theory to objectivist grounded theory, and I believe I share more the of constructivist's perspective, especially in that one participant's narrative differs slightly between interview data and blog data, mostly in tone. I am a bit surprised to discover that some withhold information from the interview that was revealed (earlier) in the blog.
Since I haven't developed my grounded theory yet, I will stop with a reflection on "Theorizing in Grounded Theory" on page 133. Charmaz shares how researchers view or define the theory of grounded theory:
- An empirical generalization - I think I may go down this path. To what extent, I don't know.
- A category - I believe I created a category (a group of English language teachers or sojourners) before grounded theory, but I'll need to investigate this category with the my theory.
- A predisposition - This is similar to "an empirical generalization," and I'm unsure at this point which of the two is stronger.
- An explication of a process - This was my intention for the project, but I believe my data may only reveal a partial explication
- A relationship between variables - There is a good possibility for my grounded theory to do this.
- An explanation - Of course my grounded theory will explain something. It will most likely be a partial explanation. To declare a whole explanation is impossible for an exploratory study like mine.
- An abstract understanding - I hope not. With so much data, I believe I can analyze certain categories more deeply for a grounded theory that is an abstract understanding.
- A description - I hope this is the by-product of my grounded theory. I want to shed more light on the experiences of my target population.
Saturday, December 7, 2013
Single-Case Data Analysis Completed
Although my data analysis is not completed, I believe I have gone deep enough analyzing the interview and blog data for each of my participants. Consistent patterns have emerged enough that I am ready to begin forming a grounded theory. This is when I take all the pieces I have found and arranging them into patterns.
Because the most complex pattern in my data is about social relationships, I will use the data to create graphic organizers (undecided on which type at this time) to visualize the social relationships and how they help answer and address my research questions. I believe I have some firm grounding based in the literature I reviewed, most notably one model that address the host culture complex. I believe I can connect this model with the graphic organizers I will be creating for each of my participants. After I can visualize the social relationships in connection to my research questions and one or more models and theories for each of my participants, then I believe I can begin a cross-case analysis.
This morning I completed this last stage of data analysis for my "last" participant. I consider her the last one only because she was the last participant I interviewed. I wrote in my memos that this analysis affected me more than the other in terms of questioning my research methods. Since this just happened, I'm unsure if I should include this part of self-reflection and critical analysis elsewhere than in my memos. I believe this critical analysis was raised in at least one of my works cited in the literature review, however it is not related to my research questions directly. It raises more philosophical concerns for analyzing qualitative data, making me more aware of my own biases than previously, or at least in the recent past. I am thankful that this last exercise in data analysis has raised this awareness before I write about my findings.
I am now at the point where I need to review the key chapters and sections of my grounded theory texts to make sure I'm ready for the next phase, to better identify the next phase, and to better equip me with the mindset and vocabulary to discuss the next analytical process.
Because the most complex pattern in my data is about social relationships, I will use the data to create graphic organizers (undecided on which type at this time) to visualize the social relationships and how they help answer and address my research questions. I believe I have some firm grounding based in the literature I reviewed, most notably one model that address the host culture complex. I believe I can connect this model with the graphic organizers I will be creating for each of my participants. After I can visualize the social relationships in connection to my research questions and one or more models and theories for each of my participants, then I believe I can begin a cross-case analysis.
This morning I completed this last stage of data analysis for my "last" participant. I consider her the last one only because she was the last participant I interviewed. I wrote in my memos that this analysis affected me more than the other in terms of questioning my research methods. Since this just happened, I'm unsure if I should include this part of self-reflection and critical analysis elsewhere than in my memos. I believe this critical analysis was raised in at least one of my works cited in the literature review, however it is not related to my research questions directly. It raises more philosophical concerns for analyzing qualitative data, making me more aware of my own biases than previously, or at least in the recent past. I am thankful that this last exercise in data analysis has raised this awareness before I write about my findings.
I am now at the point where I need to review the key chapters and sections of my grounded theory texts to make sure I'm ready for the next phase, to better identify the next phase, and to better equip me with the mindset and vocabulary to discuss the next analytical process.
Saturday, November 30, 2013
Deep Analysis
I've spent most of my Thanksgiving break (since Wednesday) conducting deeper single case analyses of my participants. Today I have completed the fourth of five, and I'm pleased with the patterns I'm finding. After I complete the fifth single case analysis either tomorrow or next weekend, I will be able to start organizing the data into categories to form a grounded theory.
My strategy at this point is to continue my immersion into the data without much interference from the literature. From 2011 to early 2013, I have been immersed in the literature, and I believe I have a few theories and models that will fit in well with my grounded theory. I don't know for sure, and I don't want to check as I the data needs to guide the analysis. If I crack open the literature now, I may be tempted to make the data fit a model or theory, which will weaken my grounded theory.
I'm interested to see how my grounded theory supports or aligns with pre-existing theories. Even more so, I'd like to see if my grounded theory shows that another theory or two may be standing on shaky ground because I may have contradictory evidence. However, I would be elated if my grounded theory help link two or more pre-existing theories as my research is at the intersection of multiple disciplines: English language teacher education, sociocultural anthropology, intercultural communication, and adjustment psychology. At this point, I only have feelings and inklings, so I need to be patient and finish the single case analysis.
Because this is a multiple case study, I need to find patterns that are found across all the studies before forming a unified grounded theory. I may be able to create several smaller theories based on a fraction of the multiple cases, but this is less important now. However, my data may reveal that the unique data for each case may help create a more compelling theory.
If you are a well-disciplined and experienced researcher, you may wince at the previous few paragraphs because I seem so naive, but this is the purpose of this post. This is what the mindset is like for someone working on his own research project for the first time from start to finish. To put your mind at ease, once I complete the single case analysis for my fifth and final participant, I plan to revisit the textbooks concerning grounded theory and qualitative data analysis. These will guide me more than my inklings and feelings, which are logged in my data analysis memos.
Another reason I am posting this is that I am a bit giddy with getting this far in my analysis and I love the process of discovery. Every stage of data analysis has been rewarding, and I continue to learn more and more from my data in addition to learning more and more about the research process. Perhaps I'm good at fooling myself, but I believe I'm lucky that I am enjoying the initial stages of data analysis just as much as data collection.
The frustration is that not many others I know care about the whole study. It's like being the only one the theater that laughed at a particular line or situation in the film or play. I wish I could share this joy but I cannot. I'm beginning to understand the mad scientist.
My strategy at this point is to continue my immersion into the data without much interference from the literature. From 2011 to early 2013, I have been immersed in the literature, and I believe I have a few theories and models that will fit in well with my grounded theory. I don't know for sure, and I don't want to check as I the data needs to guide the analysis. If I crack open the literature now, I may be tempted to make the data fit a model or theory, which will weaken my grounded theory.
I'm interested to see how my grounded theory supports or aligns with pre-existing theories. Even more so, I'd like to see if my grounded theory shows that another theory or two may be standing on shaky ground because I may have contradictory evidence. However, I would be elated if my grounded theory help link two or more pre-existing theories as my research is at the intersection of multiple disciplines: English language teacher education, sociocultural anthropology, intercultural communication, and adjustment psychology. At this point, I only have feelings and inklings, so I need to be patient and finish the single case analysis.
Because this is a multiple case study, I need to find patterns that are found across all the studies before forming a unified grounded theory. I may be able to create several smaller theories based on a fraction of the multiple cases, but this is less important now. However, my data may reveal that the unique data for each case may help create a more compelling theory.
If you are a well-disciplined and experienced researcher, you may wince at the previous few paragraphs because I seem so naive, but this is the purpose of this post. This is what the mindset is like for someone working on his own research project for the first time from start to finish. To put your mind at ease, once I complete the single case analysis for my fifth and final participant, I plan to revisit the textbooks concerning grounded theory and qualitative data analysis. These will guide me more than my inklings and feelings, which are logged in my data analysis memos.
Another reason I am posting this is that I am a bit giddy with getting this far in my analysis and I love the process of discovery. Every stage of data analysis has been rewarding, and I continue to learn more and more from my data in addition to learning more and more about the research process. Perhaps I'm good at fooling myself, but I believe I'm lucky that I am enjoying the initial stages of data analysis just as much as data collection.
The frustration is that not many others I know care about the whole study. It's like being the only one the theater that laughed at a particular line or situation in the film or play. I wish I could share this joy but I cannot. I'm beginning to understand the mad scientist.
Saturday, November 23, 2013
The Long Road to Analysis
In late September, I completed my data collection, which actually ended in early August when including the removal of the 6th participant. In early October, I met with one of my dissertation co-chairs to discuss the data and what to do with it. It was agreed that I should code for themes with a date set in late October, which I didn't believe I could meet because my full-time job was becoming more hectic and the added stress would take away from productive time for my dissertation. While that was true for a week or so, I was able to meet the deadline because there was a long enough lull in October for me to complete coding for themes.
Although only 10 themes were requested, I submitted 11 themes that stood out among all the participants. I also included smaller themes that only emerged across one or some of the participants. Once my co-chairs received my themes, I knew that I wouldn't hear from them soon because I sent them a lot more than probably both parties anticipated. Additionally, I submitted the themes right before a busy conference season and shortly after the midterm week of the fall semester.
To bide my time, I took the suggestion from the same co-chair to revise chapter 3, in which I proposed my research methods. Now that the data collection was finished and the data analysis had begun, I could change the whole chapter from future tense to past tense. I also deleted the report on my pilot study from the chapter because the pilot study was not approved by IRB, thus prohibiting me from using and publishing the data.
I got a little anxious a few weeks later when I felt like I wasn't being productive with my dissertation. I couldn't move forward with the data analysis because I hadn't received feedback from my themes yet. To feel productive, I started looking for scholarly journals to inspire me to write about my seemingly unique data collection process as it regards to social media. I have received feedback from a few peers that believe it is worth writing about for publication. In addition, I wanted to start engaging social media more directly too.
Just as I was gaining steam with these projects, I got an automated message from IRB asking for an annual update on the project. I hadn't realized that it was almost a year since I started filling out the IRB application. Because data collection was over, I didn't expect IRB to find any issues with my progress. I was wrong. On my application, I didn't describe my member checking process that I planned to initiate after I completed data analysis. Although I implied it in the email scripts and interview questions, it was not made clear in the application. Understandably, IRB loves clarity. Fortunately for me, it only took about a week to resolve this issue. A few days later, I received feedback about my themes, and it was a joyous occasion.
Getting the green light to continue my analysis was one reason for my elation, but another was that the feedback was more helpful than I expected as a guide. I could instantly plan a strategy for analysis that would take me to forming a grounded theory in line with some established models and theories from the literature. It took me a good week to calm down from this euphoria before I could resume analysis. Another reason for the week delay was yet another hectic week at my job.
Now that Thanksgiving week has arrived, I will have plenty of time to get a large chunk of the deeper analysis completed. I am now projecting to complete these stage of analysis by the second week in December, leaving me the time up to Christmas to determine if I have enough evidence and a strong enough argument for a grounded theory. I am certain that I have enough evidence, but I'm not as confident with my argument yet, and that's what this next stage of analysis is all about.
You don't need to guess what I will be thankful for this year's Thanksgiving! All in all, I am still enjoying the majority of this dissertation process. The key to this enjoyment is my passion for the project and already realizing practical implications through social media and my current full-time job helping faculty with professional and curriculum development.
Although only 10 themes were requested, I submitted 11 themes that stood out among all the participants. I also included smaller themes that only emerged across one or some of the participants. Once my co-chairs received my themes, I knew that I wouldn't hear from them soon because I sent them a lot more than probably both parties anticipated. Additionally, I submitted the themes right before a busy conference season and shortly after the midterm week of the fall semester.
To bide my time, I took the suggestion from the same co-chair to revise chapter 3, in which I proposed my research methods. Now that the data collection was finished and the data analysis had begun, I could change the whole chapter from future tense to past tense. I also deleted the report on my pilot study from the chapter because the pilot study was not approved by IRB, thus prohibiting me from using and publishing the data.
I got a little anxious a few weeks later when I felt like I wasn't being productive with my dissertation. I couldn't move forward with the data analysis because I hadn't received feedback from my themes yet. To feel productive, I started looking for scholarly journals to inspire me to write about my seemingly unique data collection process as it regards to social media. I have received feedback from a few peers that believe it is worth writing about for publication. In addition, I wanted to start engaging social media more directly too.
Just as I was gaining steam with these projects, I got an automated message from IRB asking for an annual update on the project. I hadn't realized that it was almost a year since I started filling out the IRB application. Because data collection was over, I didn't expect IRB to find any issues with my progress. I was wrong. On my application, I didn't describe my member checking process that I planned to initiate after I completed data analysis. Although I implied it in the email scripts and interview questions, it was not made clear in the application. Understandably, IRB loves clarity. Fortunately for me, it only took about a week to resolve this issue. A few days later, I received feedback about my themes, and it was a joyous occasion.
Getting the green light to continue my analysis was one reason for my elation, but another was that the feedback was more helpful than I expected as a guide. I could instantly plan a strategy for analysis that would take me to forming a grounded theory in line with some established models and theories from the literature. It took me a good week to calm down from this euphoria before I could resume analysis. Another reason for the week delay was yet another hectic week at my job.
Now that Thanksgiving week has arrived, I will have plenty of time to get a large chunk of the deeper analysis completed. I am now projecting to complete these stage of analysis by the second week in December, leaving me the time up to Christmas to determine if I have enough evidence and a strong enough argument for a grounded theory. I am certain that I have enough evidence, but I'm not as confident with my argument yet, and that's what this next stage of analysis is all about.
You don't need to guess what I will be thankful for this year's Thanksgiving! All in all, I am still enjoying the majority of this dissertation process. The key to this enjoyment is my passion for the project and already realizing practical implications through social media and my current full-time job helping faculty with professional and curriculum development.
Thursday, September 19, 2013
IRB and Social Media Research
This week I encountered an interesting situation regarding IRB and my data collection procedures. My last participant, who I blogged about in the previous posts, published his online video response to my interview questions on YouTube. Although I knew he would create online videos, I didn't expect him to post it online so quickly. This route of collecting interview data was not approved by IRB, so I understand the ethical implications here. I informed IRB that I would collect data via Qualtrics or Adobe Connect.
Back in January, I spent a few weeks through email discussions finding the best way to collect interview data from my participants who live abroad. My first suggestions were email interviews and Skype interviews, which have been done many times in the literature I reviewed. These were not acceptable methods for IRB because there is a confidentiality risk because the University of Iowa does not host my participant's email accounts (so they could approve my emails to them, but not their emails to me) and the university does not host Skype. Because I had experience with Adobe Connect, the web conferencing alternative was easy to choose. However, I had difficulty accepting Qualtrics as an alternative to email interviews because it's a survey software program, so the interview would flow more like a survey with 99% open-ended questions. Nowhere in the literature did I find Qualtrics being used for interviews. Even after collecting the data, it was more like a survey, but IRB approved that route. And the university's IT advisor on data collection also agreed that Qualtrics was the best way to go.
So when my participant suggested submitting a video, I knew that IRB would not support it given the arguments for not permitting their email accounts or Skype. It would have to be something that the university could provide. In retrospect, perhaps IRB had a waiver for the participant to sign saying that he acknowledges that his data would not be confidential. He didn't care as he willingly and knowingly made it public. He didn't have the time for spending time on Qualtrics. Perhaps though stronger persuasion, he may have done an Adobe Connect interview. But I got the impression he wanted to be in control of the data, so perhaps where IRB drew the line.
IRB's conclusion was to order me to destroy the data, which is an order that informed me that IRB does not clearly understand social media research. First of all, I do not own the data, so how can I destroy it? They did not request me to ask/tell/order my participant to take the video off YouTube. I'm sure that's another ethical complication there as he has the right to refuse the order. Nonetheless, thousands of people have already viewed the video so the damage, if any, is done.
The damage is completely internal or procedural, but the main purpose of IRB is protect human subjects from physical and psychological harm. It seems to me that I've been asked to "destroy" the data out of a technicality that had no threat of physical or psychological harm to my participant, especially given the fact that he has published over a thousand of YouTube videos with similar or even more personal/revealing narratives.
Finally, I'd like to analyze the term "destroy" a bit further. Once something is published online, it is never destroyed. It always exists in a server somewhere, so I cannot completely physically destroy the data. One of my dissertation co-chairs used the term "remove," but she didn't specify from what. I cannot remove it from the Internet, but I can remove it from my dissertation. The data still exists for anyone to collect, analyze, and write about. I am happy to have played a role in providing this data to those who find it useful. Looking at the comments under the YouTube video, I found that some English language teachers have already found it helpful, so one of my goals for publishing my dissertation has been accomplished. So I'm learning that this type of publication may affect more change than publishing through a peer-reviewed article, which very few practitioners read anyway. I'm all for accessibility.
Back in January, I spent a few weeks through email discussions finding the best way to collect interview data from my participants who live abroad. My first suggestions were email interviews and Skype interviews, which have been done many times in the literature I reviewed. These were not acceptable methods for IRB because there is a confidentiality risk because the University of Iowa does not host my participant's email accounts (so they could approve my emails to them, but not their emails to me) and the university does not host Skype. Because I had experience with Adobe Connect, the web conferencing alternative was easy to choose. However, I had difficulty accepting Qualtrics as an alternative to email interviews because it's a survey software program, so the interview would flow more like a survey with 99% open-ended questions. Nowhere in the literature did I find Qualtrics being used for interviews. Even after collecting the data, it was more like a survey, but IRB approved that route. And the university's IT advisor on data collection also agreed that Qualtrics was the best way to go.
So when my participant suggested submitting a video, I knew that IRB would not support it given the arguments for not permitting their email accounts or Skype. It would have to be something that the university could provide. In retrospect, perhaps IRB had a waiver for the participant to sign saying that he acknowledges that his data would not be confidential. He didn't care as he willingly and knowingly made it public. He didn't have the time for spending time on Qualtrics. Perhaps though stronger persuasion, he may have done an Adobe Connect interview. But I got the impression he wanted to be in control of the data, so perhaps where IRB drew the line.
IRB's conclusion was to order me to destroy the data, which is an order that informed me that IRB does not clearly understand social media research. First of all, I do not own the data, so how can I destroy it? They did not request me to ask/tell/order my participant to take the video off YouTube. I'm sure that's another ethical complication there as he has the right to refuse the order. Nonetheless, thousands of people have already viewed the video so the damage, if any, is done.
The damage is completely internal or procedural, but the main purpose of IRB is protect human subjects from physical and psychological harm. It seems to me that I've been asked to "destroy" the data out of a technicality that had no threat of physical or psychological harm to my participant, especially given the fact that he has published over a thousand of YouTube videos with similar or even more personal/revealing narratives.
Finally, I'd like to analyze the term "destroy" a bit further. Once something is published online, it is never destroyed. It always exists in a server somewhere, so I cannot completely physically destroy the data. One of my dissertation co-chairs used the term "remove," but she didn't specify from what. I cannot remove it from the Internet, but I can remove it from my dissertation. The data still exists for anyone to collect, analyze, and write about. I am happy to have played a role in providing this data to those who find it useful. Looking at the comments under the YouTube video, I found that some English language teachers have already found it helpful, so one of my goals for publishing my dissertation has been accomplished. So I'm learning that this type of publication may affect more change than publishing through a peer-reviewed article, which very few practitioners read anyway. I'm all for accessibility.
Monday, September 9, 2013
Diving Into Analysis
My sixth and apparently final participant has finally found time for the interview part of my data collection. Last week, I finally completed viewing about 40 of his online videos regarding teaching English in Japan and/or adjusting to living and working in Japan. The remaining 1000 videos were more about current events in Japan or general information about Japanese culture.
As the interviews take place over the course of the week, I hope, I will start the more in-depth phase of analysis of all of my participants' data. At the same time, I will begin writing my first draft of Chapter 4. Actually, I already wrote the first paragraph giving a general introduction to the participants. Besides the final interview, I perceive much time of this week spent on organizing my approach to analysis as I write Chapter 4.
Coincidentally, as I am diving into analysis of my data, I am also beginning my new weekly routine of swimming. Since I have started my full-time job, there are two things that I want to continue that abruptly stopped back in April: 1) using the university's facilities that I'm paying for, 2) getting some good exercise. Although this seems like aside to the rest of this blog, I believe that physical exercise may enhance my mental exercise for data analysis.
As I mentioned earlier, my preliminary analysis took part during data collection of the blog data. I created two very broad themes: 1) teaching English and 2) cultural adjustment. The golden ticket is when these themes overlap, which seems to happen often for some participants and not much for others. This is where I will look to develop a grounded theory. Although this seems easy, I'm dealing with the murky nature of my participants' beliefs, attitudes, and perceptions which are made more complex with own beliefs, attitudes, and perceptions. My dissertation chairs and committee will demonstrate how this gets even more complicated when my interpretation is interpreted by them. This is not science.
Another matter I need to consider is condensing my analysis into a limited number of pages for Chapter 4. My first 3 chapters are already longer than most other dissertations I have seen. I've been considering a page limit when describing the context and adjustment process for each participant. I have arbitrarily chosen 4 pages, thus 24 pages for all 6 participants. This does not include the general introduction or the cross-case analysis or the establishment of any grounded theory. I am the type of person who can write a lot and does not mind writing a lot, so I need to exert some type of self-control. I'd like to thank all of my writing teachers who brought me to this point. However, I didn't mention the quality of my writing. Perhaps you can take a guess by the mostly informal prose you see here.
As the interviews take place over the course of the week, I hope, I will start the more in-depth phase of analysis of all of my participants' data. At the same time, I will begin writing my first draft of Chapter 4. Actually, I already wrote the first paragraph giving a general introduction to the participants. Besides the final interview, I perceive much time of this week spent on organizing my approach to analysis as I write Chapter 4.
Coincidentally, as I am diving into analysis of my data, I am also beginning my new weekly routine of swimming. Since I have started my full-time job, there are two things that I want to continue that abruptly stopped back in April: 1) using the university's facilities that I'm paying for, 2) getting some good exercise. Although this seems like aside to the rest of this blog, I believe that physical exercise may enhance my mental exercise for data analysis.
As I mentioned earlier, my preliminary analysis took part during data collection of the blog data. I created two very broad themes: 1) teaching English and 2) cultural adjustment. The golden ticket is when these themes overlap, which seems to happen often for some participants and not much for others. This is where I will look to develop a grounded theory. Although this seems easy, I'm dealing with the murky nature of my participants' beliefs, attitudes, and perceptions which are made more complex with own beliefs, attitudes, and perceptions. My dissertation chairs and committee will demonstrate how this gets even more complicated when my interpretation is interpreted by them. This is not science.
Another matter I need to consider is condensing my analysis into a limited number of pages for Chapter 4. My first 3 chapters are already longer than most other dissertations I have seen. I've been considering a page limit when describing the context and adjustment process for each participant. I have arbitrarily chosen 4 pages, thus 24 pages for all 6 participants. This does not include the general introduction or the cross-case analysis or the establishment of any grounded theory. I am the type of person who can write a lot and does not mind writing a lot, so I need to exert some type of self-control. I'd like to thank all of my writing teachers who brought me to this point. However, I didn't mention the quality of my writing. Perhaps you can take a guess by the mostly informal prose you see here.
Saturday, August 10, 2013
The Last But Not Least Participant
After my web conferencing interview, I have completed interviewing three of the six participants who gave me their informed consent. Two of the remaining three participants have completed two of their three interview sessions, and I believe one will be done by the end of the weekend. I have not yet begun interviewing the sixth and perhaps last participant for my study because he has been on vacation for the past few weeks, but he should be back soon if not already.
The last participant provides an interesting approach to data collection as he is more of a vlogger than a blogger. (A vlogger is a video blogger.) Actually three of my participants have or had vlogs on their own YouTube channels, but this participant is the most dedicated towards his videos. Of the other two participants, one started vlogging but switched to blogging, and that is where most of my document data for him is found. For the most part, the other participant's videos support her blog, and she doesn't produce as many videos as she used to.
So how dedicated is this last participant to his online videos? I say very much so as I spent the last few weeks cataloging his videos, which basically means listing all of his videos in chronological order and initially coding them when their titles are directly relevant to my research questions. Unlike for most other bloggers, I did not date each video because to find the date, I have to play the video, and when I play a video, I lose my place on YouTube where I was cataloging videos. This isn't usually a problem, but, as of yesterday, I have cataloged 1036 videos!
He has more than 1036 videos because halfway through my cataloging I stopped listing repeated videos, which don't happen often, and a series of videos on sumo wrestling, which do not directly answer my research questions. If sumo wrestling videos dominated his channel, then I would have considered that they may, but they don't.
With 1036 cataloged videos, I have come to a dilemma. How can I possibly thoroughly code all these videos? Coding all these videos requires thoroughly transcribing all of them. The average length of these videos seems to be around 10 minutes. There are many that go beyond 15 and 20 minutes, but there are also some that are less than 5. If the average is truly 10 minutes, then that's 10,360 minutes of video. That 172 hours and 40 minutes! Now, when I took courses on qualitative research I had to practice transcribing recorded interviews. When I wasn't tired, I could transcribe about 20 minutes of raw data in 1 hour. So to transcribe 10,360 minutes would at least take me over 31,000 hours which is 518 hours. If I dedicated 8 hours a day to transcribing these videos, it would take me over 2 months straight, and that's if I didn't slow down! After that, then I could start coding. I wonder how many pages of text that would be.
Now, if I were rich or well-funded, I would hire an army of people to transcribe and code his videos. No!
So...I decided to not be as rigorous with analyzing his videos as I am with my other participants, who average about 10 videos each. This is not a case study, it's a multiple case study, and I want to spend roughly the same amount of time analyzing the data for each participant. Otherwise, this last participant would definitely dominate my time and perhaps my study.
Fortunately, fortunately, my initial coding turned up only 42 videos of the 1036 that directly answer my research questions. One of my other participants didn't even blog 42 times. Anyway, once I transcribe one of these videos, I will check how many pages long it is because nearly all of my bloggers didn't blog more than a page per post. I think I can handle a rough estimate of 420 minutes (7 hours) of videos to transcribe and code.
And I didn't even get to write about how I will collect and analyze the last participant's interview data. That's another interesting post for later.
The last participant provides an interesting approach to data collection as he is more of a vlogger than a blogger. (A vlogger is a video blogger.) Actually three of my participants have or had vlogs on their own YouTube channels, but this participant is the most dedicated towards his videos. Of the other two participants, one started vlogging but switched to blogging, and that is where most of my document data for him is found. For the most part, the other participant's videos support her blog, and she doesn't produce as many videos as she used to.
So how dedicated is this last participant to his online videos? I say very much so as I spent the last few weeks cataloging his videos, which basically means listing all of his videos in chronological order and initially coding them when their titles are directly relevant to my research questions. Unlike for most other bloggers, I did not date each video because to find the date, I have to play the video, and when I play a video, I lose my place on YouTube where I was cataloging videos. This isn't usually a problem, but, as of yesterday, I have cataloged 1036 videos!
He has more than 1036 videos because halfway through my cataloging I stopped listing repeated videos, which don't happen often, and a series of videos on sumo wrestling, which do not directly answer my research questions. If sumo wrestling videos dominated his channel, then I would have considered that they may, but they don't.
With 1036 cataloged videos, I have come to a dilemma. How can I possibly thoroughly code all these videos? Coding all these videos requires thoroughly transcribing all of them. The average length of these videos seems to be around 10 minutes. There are many that go beyond 15 and 20 minutes, but there are also some that are less than 5. If the average is truly 10 minutes, then that's 10,360 minutes of video. That 172 hours and 40 minutes! Now, when I took courses on qualitative research I had to practice transcribing recorded interviews. When I wasn't tired, I could transcribe about 20 minutes of raw data in 1 hour. So to transcribe 10,360 minutes would at least take me over 31,000 hours which is 518 hours. If I dedicated 8 hours a day to transcribing these videos, it would take me over 2 months straight, and that's if I didn't slow down! After that, then I could start coding. I wonder how many pages of text that would be.
Now, if I were rich or well-funded, I would hire an army of people to transcribe and code his videos. No!
So...I decided to not be as rigorous with analyzing his videos as I am with my other participants, who average about 10 videos each. This is not a case study, it's a multiple case study, and I want to spend roughly the same amount of time analyzing the data for each participant. Otherwise, this last participant would definitely dominate my time and perhaps my study.
Fortunately, fortunately, my initial coding turned up only 42 videos of the 1036 that directly answer my research questions. One of my other participants didn't even blog 42 times. Anyway, once I transcribe one of these videos, I will check how many pages long it is because nearly all of my bloggers didn't blog more than a page per post. I think I can handle a rough estimate of 420 minutes (7 hours) of videos to transcribe and code.
And I didn't even get to write about how I will collect and analyze the last participant's interview data. That's another interesting post for later.
Friday, August 9, 2013
First Web Conferencing Interview
On Sunday, I conducted my first real web conferencing interview for research purposes. I mention real because I've conducted two online interviews as pilots, however they were through Skype, which my IRB does not approve because Skype calls can be easily intercepted, and therefore I cannot guarantee privacy or confidentiality. However, with the latest news concerning the US government PRISM surveillance program and corporate compliance, I don't think I can guarantee privacy or confidentiality with any online communication. The IRB believed that Adobe Connect offered more security than Skype, so I was able to conduct my first web conferencing interview using Adobe Connect. One reason I chose Adobe Connect was that I had 3 years of experience using the program to assist the Education Policy & Leadership Studies department in the College of Education with their hybrid online courses.
If you're interested in how I did this, I created a meeting on Adobe Connect for this purpose. When you create a meeting, a link is provided to be shared with guest participants. I shared the link with my research participant who logged in as a guest, and then I helped her set up her webcam and mic. Before that, however, I let her know the specifications for joining an Adobe Connect meeting. The only preferred specification that she couldn't meet was avoiding a wireless connection. Fortunately for us, the wireless connection was strong enough that it was only dropped once during the 2.5-hour interview.
Adobe Connect offers a recording option, but I opted out because I did not want the conference stored in the institution's Adobe Connect account and I did need to capture video. Instead I captured the audio portion of the interview using Audacity. Because I was using my Macbook Pro, I could have easily used Garageband as well but I piloted with Audacity, and it demonstrated reliability so there was no reason for me to change. If I wanted to publish the audio interview, then Garageband would have been easier, but I cannot and will not publish the raw interview.
What did I learn from this type of interview process?
If you're interested in how I did this, I created a meeting on Adobe Connect for this purpose. When you create a meeting, a link is provided to be shared with guest participants. I shared the link with my research participant who logged in as a guest, and then I helped her set up her webcam and mic. Before that, however, I let her know the specifications for joining an Adobe Connect meeting. The only preferred specification that she couldn't meet was avoiding a wireless connection. Fortunately for us, the wireless connection was strong enough that it was only dropped once during the 2.5-hour interview.
Adobe Connect offers a recording option, but I opted out because I did not want the conference stored in the institution's Adobe Connect account and I did need to capture video. Instead I captured the audio portion of the interview using Audacity. Because I was using my Macbook Pro, I could have easily used Garageband as well but I piloted with Audacity, and it demonstrated reliability so there was no reason for me to change. If I wanted to publish the audio interview, then Garageband would have been easier, but I cannot and will not publish the raw interview.
What did I learn from this type of interview process?
- Adobe Connect has a greater broadcasting delay problem than Skype. I have used Skype to communicate with pilot participants and friends in Japan and Germany, and I experienced little to no delay in the communication. However, Adobe Connect had a noticeable delay for my communication to South Korea. I do not believe South Korea is at fault for this delay because no country is more wired than South Korea.
- The technical issues doubled my anxiety for the first 30 minutes. Not only was I nervous because I wanted my first interview to go well, but my concern for having little to no technical issues made it more difficult for me to focus on my participant. It was during the first 30 minutes that the connection dropped, so I was a little freaked out at that point.
- The delay took a while to get used to. Not only did I have to learn a new way to pace the interview, but I also had to cope with hearing myself delayed on her end. Every time I asked a question, my echo would repeat that question about 5-10 seconds later. That echo sometimes helped me figure out when she would start speaking, so I didn't feel uncomfortable with the silent pauses on her end.
- Even if you want to plow through a lengthy interview, take a break when you get one. I purposefully divided my interview into 3 sections. Although I skipped the first break, I took the second one, and I needed it. After that break, I was much calmer and more confident about the interview process. That break also gave Adobe Connect a break in that I noticed the echo issue was toned down in the second half or third third.
- An interview script is your friend. When I piloted my interview face-to-face, it was easier to let it flow into a conversation. But because of the irregularities of a web conferencing interview, this flow could not be transferred. Perhaps if I had more practice interviewing with delayed online communication, perhaps I could discover a new conversational flow. However, I would prefer to avoid this type of delay if I could. The interview script always kept me focused when a technical issue distracted me. The interview script worked well on my computer because I could set up the script next to the video of my participant, so I could maintain eye contact much better than a face-to-face interview. So that's one positive aspect of conducting an interview online.
Tuesday, July 16, 2013
Recruitment and Interview Update
Within the last week, I have successfully recruited 5 participants, and I've already completed interviewing the third participant. I still need to interview the first 2 one more time. I haven't started interviewing the last 2 yet, one of which will be on vacation for the next 3 weeks. I believe one reason that I'm having a hard time recruiting people is that many of my participants are on vacation or have left the country for good. July is the month of transition for my target participant population. Once they are settled back home or back for the fall semester, then I should be able to recruit and interview more. However, I may not need to do this as I seem to have a sufficient number of participants now. That being said, some of them may not complete their interviews.
I plan to contact 5 more participants for recruitment within the next week. I also plan to send reminders to about 5 others that I attempted to recruit a month ago. The last time I sent out reminders, I successfully recruited one. If my current 5 participants all complete the interviews within the month, then I am done collecting data. However, if they do not, then I will plan another phase of recruitment.
One of the biggest changes since my last post was that I decided not to recruit through Twitter as my pilot test failed. Apparently, a recruitment message can get lost in the Twitter feed very quickly and/or go unnoticed in the contact section for months. Twitter is also too risky to recruit privately. Instead I've decided to use YouTube which notifies its users well. Although the pilot test worked, none of my contacts through YouTube have replied yet.
With regular email recruitment, I've found that most of my respondents reply within 24 hours of my message being sent. Only one has replied a few days later. The correspondence with one who declined my invitation spanned nearly 3 weeks! I was trying to gauge the frequency of blog posting to be roughly equivalent to the frequency they check and/or reply to their email, but I found this to be difficult and usually inaccurate.
As for interviews, nobody has chosen to be interviewed through Adobe Connect, the web conferencing program used at the University of Iowa and Kirkwood Community College. I'm a bit relieved because that would involve an extra step of transcribing all the interviews. However, I'm finding that some of my participants answer interview questions on a survey platform quite tersely. One participant insisted that he send his interview answers via online video. This will be interesting to see one I receive them.
I plan to contact 5 more participants for recruitment within the next week. I also plan to send reminders to about 5 others that I attempted to recruit a month ago. The last time I sent out reminders, I successfully recruited one. If my current 5 participants all complete the interviews within the month, then I am done collecting data. However, if they do not, then I will plan another phase of recruitment.
One of the biggest changes since my last post was that I decided not to recruit through Twitter as my pilot test failed. Apparently, a recruitment message can get lost in the Twitter feed very quickly and/or go unnoticed in the contact section for months. Twitter is also too risky to recruit privately. Instead I've decided to use YouTube which notifies its users well. Although the pilot test worked, none of my contacts through YouTube have replied yet.
With regular email recruitment, I've found that most of my respondents reply within 24 hours of my message being sent. Only one has replied a few days later. The correspondence with one who declined my invitation spanned nearly 3 weeks! I was trying to gauge the frequency of blog posting to be roughly equivalent to the frequency they check and/or reply to their email, but I found this to be difficult and usually inaccurate.
As for interviews, nobody has chosen to be interviewed through Adobe Connect, the web conferencing program used at the University of Iowa and Kirkwood Community College. I'm a bit relieved because that would involve an extra step of transcribing all the interviews. However, I'm finding that some of my participants answer interview questions on a survey platform quite tersely. One participant insisted that he send his interview answers via online video. This will be interesting to see one I receive them.
Tuesday, June 18, 2013
Recruitment Issues
It's been about a month since my last posting, and I have successfully recruited one participant. As of today, I have contacted 13 people for interviews with 1 acceptance and 1 rejection. I just contacted 3 people today, but aside from them I haven't heard from the remaining 8. What makes my recruitment difficult is that all of my participants are overseas, and I get their contact information from their blogs, microblogs, or vlogs. I assume that some of these bloggers have posted outdated or barely used email addresses. Another assumption for the low response rate is that many people in my target population are getting ready to leave their host country. They are busy enough saying goodbyes and preparing for life at home that they are not ready to commit to a series interviews that may take months to complete.
One of the stickiest issues is the method of contacting my participants. My IRB has approved of contacting them through email, but I have two other methods of contacting them online which IRB may or may not consider "email." One is through Twitter which allows me to send private messages to my those who follow me and public messages to those who don't. Both types of messages are limited to Twitter's well known 140-character limit. This is probably the most difficult to work around as I cannot capture my entire IRB-approved email message within 140 characters. Currently we are working on solutions. If I we come to one, then I will be able to contact 8 more people, one of which I probably have a good chance of recruiting.
There are 12 other people that are best contacted through their YouTube channels. YouTube allows messaging through their service with no character limits. I wrote to IRB today to see if they qualify this as "email." In addition, I would like to send a recruitment video containing the same message as the email. I believe this will be more effective based on the literature I have read and my own feelings since my contacts will be able to see who I am, and it may be easier to build rapport if I don't appear creepy. Furthermore, I believe a link to my recruitment video would be more effective to send on Twitter as well. I'd really like to work on this as soon as I know where IRB stands, then I would nearly double the amount of people to contact.
I am fortunate to have the technological capability with high-tech facilities to record, edit, and link this potential recruitment video. I might as well take the chance since I have this opportunity.
One of the stickiest issues is the method of contacting my participants. My IRB has approved of contacting them through email, but I have two other methods of contacting them online which IRB may or may not consider "email." One is through Twitter which allows me to send private messages to my those who follow me and public messages to those who don't. Both types of messages are limited to Twitter's well known 140-character limit. This is probably the most difficult to work around as I cannot capture my entire IRB-approved email message within 140 characters. Currently we are working on solutions. If I we come to one, then I will be able to contact 8 more people, one of which I probably have a good chance of recruiting.
There are 12 other people that are best contacted through their YouTube channels. YouTube allows messaging through their service with no character limits. I wrote to IRB today to see if they qualify this as "email." In addition, I would like to send a recruitment video containing the same message as the email. I believe this will be more effective based on the literature I have read and my own feelings since my contacts will be able to see who I am, and it may be easier to build rapport if I don't appear creepy. Furthermore, I believe a link to my recruitment video would be more effective to send on Twitter as well. I'd really like to work on this as soon as I know where IRB stands, then I would nearly double the amount of people to contact.
I am fortunate to have the technological capability with high-tech facilities to record, edit, and link this potential recruitment video. I might as well take the chance since I have this opportunity.
Wednesday, May 22, 2013
Now Collecting Data
Since my last posting, I started a full-time job, which is conveniently within a comfortable and reasonable driving distance of my home and my university. The job is thankfully not too similar to my research interest, so I can view my research as a welcoming break from my job and vice-versa. At this early point in my research and job, I can say that I enjoy them equally.
My new job has/had a large learning curve, so I decided to take a one-month hiatus from my research and focus solely on the new job. However, one week after my first month I went to Canada for a 5-day business trip and so I couldn't dive back into my dissertation until returning and recovering from the Canadian trip.
Although I officially wasn't collecting data, I was reading a book written by a potential participant in my study. I decided to read it during the 5-week hiatus to keep my brain interested in my research topic. I did not want to lose complete interest in my dissertation because that would have terrible consequences. For the most part, the pressure to continue my strong work ethic from my co-chairs and my committee is off. Actually, the book really got me excited about participant selection and data collection, so I was ready to start when I got the chance.
When I selected my first participant, I realized that I forgot to contact IRB to make a change to my invitation email. One sentence had to be removed because it no longer applied to my study and it would confuse my participants. Fortunately for me, it took an hour to make the change and two days for IRB to approve the modification. By then, I was six weeks into my job without collecting any data.
This past week is when I started collecting document data from my first participant. I still need to collect interview data, but I already have about 800 pages of document data from the first of up to fifteen participants. That appears daunting, but I would say about 1/3 of those pages do not directly answer my primary research question. I cannot describe anymore than that without violating confidentiality. It's also somewhat less daunting because I have a framework set up to start initial coding. The deeper digging will begin once a pattern emerges from the initial coding process.
Although I have just started collecting data, reading the documents from my first participant and the book by the potential participant (who I may not select at all for reasons I cannot disclose) has been personally rewarding. Doing this research is rewarding professionally and emotionally because my participants have had similar life experiences as me. If I do not complete this research project, I will have gained enough to be personally satisfied. However, I do not intend to give up because I would also like to be professionally satisfied with a PhD, evidence that I can conduct research at a research-intensive university.
My new job has/had a large learning curve, so I decided to take a one-month hiatus from my research and focus solely on the new job. However, one week after my first month I went to Canada for a 5-day business trip and so I couldn't dive back into my dissertation until returning and recovering from the Canadian trip.
Although I officially wasn't collecting data, I was reading a book written by a potential participant in my study. I decided to read it during the 5-week hiatus to keep my brain interested in my research topic. I did not want to lose complete interest in my dissertation because that would have terrible consequences. For the most part, the pressure to continue my strong work ethic from my co-chairs and my committee is off. Actually, the book really got me excited about participant selection and data collection, so I was ready to start when I got the chance.
When I selected my first participant, I realized that I forgot to contact IRB to make a change to my invitation email. One sentence had to be removed because it no longer applied to my study and it would confuse my participants. Fortunately for me, it took an hour to make the change and two days for IRB to approve the modification. By then, I was six weeks into my job without collecting any data.
This past week is when I started collecting document data from my first participant. I still need to collect interview data, but I already have about 800 pages of document data from the first of up to fifteen participants. That appears daunting, but I would say about 1/3 of those pages do not directly answer my primary research question. I cannot describe anymore than that without violating confidentiality. It's also somewhat less daunting because I have a framework set up to start initial coding. The deeper digging will begin once a pattern emerges from the initial coding process.
Although I have just started collecting data, reading the documents from my first participant and the book by the potential participant (who I may not select at all for reasons I cannot disclose) has been personally rewarding. Doing this research is rewarding professionally and emotionally because my participants have had similar life experiences as me. If I do not complete this research project, I will have gained enough to be personally satisfied. However, I do not intend to give up because I would also like to be professionally satisfied with a PhD, evidence that I can conduct research at a research-intensive university.
Friday, March 15, 2013
Research Proposal Meeting
I haven't written much in the past two months because the only thing I had to do concerning my dissertation was schedule and hold my research proposal meeting. If I weren't job hunting in that period of time, I would have considered those past couple of months as an extended break. During this hiatus, I received some advice of how to constructively occupy the time and I took some of that advice.
Hiatus Advice
I also spent the last month brushing up on my Spanish skills using Duolingo, a free online language learning software program. I did this for a number of reasons both personal and professional, and I found it more enjoyable than I expected.
One of the more useful ways of spending my hiatus time was working on my professional blog. I took the opportunity to explore my ideas about incorporating multiliteracies into second language education, a second research interest of mine. I wanted to practice my skills of curriculum development with this new idea as well as learn more about the development of education technology and its effects on learning and pedagogy. This exercise actually prepared me for the job I just accepted.
Research Proposal Meeting
On my birthday, February 14, my co-chairs gave me the green light to schedule my research proposal meeting with my dissertation committee: 3 faculty from my department, 1 from another department within the College of Education, and 1 from my cognate area of Sociocultural Anthropology. I used Doodle to schedule it, and found that March 7 was the best earliest time for everyone.
By request, I sent 4 committee members a digital copy of my proposal (the first 3 chapters of my dissertation) and 1 member a hard copy in a 3-ring binder. They all received their copies within a week of February 14.
A series of winter storms swept passed Iowa, about one a week, leading up to my proposal meeting. One of these storms prevented one of my co-chairs from attending the meeting, but the meeting proceeded well without her.
I spent the first 10 minutes presenting a synopsis of my research proposal, ending it with some of my concerns. And over the next 80 minutes, I discovered that my concerns overlapped quite well with my committees. Nobody's questions or concerns took me by surprise, and I was very happy to receive such constructive feedback. I'm not sure how typical this is for PhD candidates as I haven't met many who like to share details about their meeting.
By the end of the meeting, I had a good idea of how to make changes to my first 3 chapters before collecting the data. At that time, it seemed like these changes would take a few months and I could start collecting data by mid to late summer 2013. I was surprised after reconnecting with my co-chairs that I did not have to make as many drastic changes as I was ready to make. The best outcome for me is that it seems that I do not need to go through IRB again.
Today, I just submitted my action plan to my co-chairs. And if all goes well, I should be able to start collecting data as soon as late spring/early summer 2013. In the meantime, I will be starting my new job, which will most likely slow things down. I don't mind as much now that my family and I are more financially secure.
Hiatus Advice
- Submit some manuscripts to scholarly journals. A colleague and I had been planning to submit our co-written paper to a journal for a few months, and this hiatus provided the best opportunity to finally submit it.
- Begin another research project. I played around with this idea, coming up with different research questions involving the same population under investigation in my dissertation. I limited myself to brainstorming research questions and doing some preliminary literature searching.
- Submit proposals to conferences. In my case, the deadlines for proposals for many upcoming conferences had already passed. I spent a little time brainstorming ideas for proposals for conferences during the 2013/14 academic year.
- Work more on the pilot study. I already had two opportunities to work on my pilot study since August, and I didn't want to adjust or add anything more being so close to the research proposal meeting.
I also spent the last month brushing up on my Spanish skills using Duolingo, a free online language learning software program. I did this for a number of reasons both personal and professional, and I found it more enjoyable than I expected.
One of the more useful ways of spending my hiatus time was working on my professional blog. I took the opportunity to explore my ideas about incorporating multiliteracies into second language education, a second research interest of mine. I wanted to practice my skills of curriculum development with this new idea as well as learn more about the development of education technology and its effects on learning and pedagogy. This exercise actually prepared me for the job I just accepted.
Research Proposal Meeting
On my birthday, February 14, my co-chairs gave me the green light to schedule my research proposal meeting with my dissertation committee: 3 faculty from my department, 1 from another department within the College of Education, and 1 from my cognate area of Sociocultural Anthropology. I used Doodle to schedule it, and found that March 7 was the best earliest time for everyone.
By request, I sent 4 committee members a digital copy of my proposal (the first 3 chapters of my dissertation) and 1 member a hard copy in a 3-ring binder. They all received their copies within a week of February 14.
A series of winter storms swept passed Iowa, about one a week, leading up to my proposal meeting. One of these storms prevented one of my co-chairs from attending the meeting, but the meeting proceeded well without her.
I spent the first 10 minutes presenting a synopsis of my research proposal, ending it with some of my concerns. And over the next 80 minutes, I discovered that my concerns overlapped quite well with my committees. Nobody's questions or concerns took me by surprise, and I was very happy to receive such constructive feedback. I'm not sure how typical this is for PhD candidates as I haven't met many who like to share details about their meeting.
By the end of the meeting, I had a good idea of how to make changes to my first 3 chapters before collecting the data. At that time, it seemed like these changes would take a few months and I could start collecting data by mid to late summer 2013. I was surprised after reconnecting with my co-chairs that I did not have to make as many drastic changes as I was ready to make. The best outcome for me is that it seems that I do not need to go through IRB again.
Today, I just submitted my action plan to my co-chairs. And if all goes well, I should be able to start collecting data as soon as late spring/early summer 2013. In the meantime, I will be starting my new job, which will most likely slow things down. I don't mind as much now that my family and I are more financially secure.
Wednesday, January 16, 2013
An Interesting Tweet
This past weekend, I received a tweet (on Twitter) to participate in a study that is similar to mine. The researcher is also a PhD candidate, but she is from a university that I just learned about from a job position announcement last week. Although the job announcement is mostly irrelevant, I am very interested in this candidate's study. I am not too worried about our study being too similar because our participant populations have a very low chance of overlapping.
I agreed to take participate in her survey and found that we asked roughly the same demographic questions, but the other research questions differed a little. Although I am using survey software, my research is more like an interview. However, at the end of taking her survey, I was asked if I would like to participate in an online interview via Skype. I agreed and was a bit jealous that her IRB allows her to use an easier method of data collection, although there is a greater risk of losing confidentiality through Skype.
Our studies are also similar in that I assume that we would both qualify as participants in our own studies, meaning I could in mine and she in hers. However, I am not sure if she would qualify for mine. We are also similar, I believe, in that I assume that she has worked in the same English Language Fellowship I have.
I wonder about the ethical and other implications of interviewing a participant like me who may know the research theories and frameworks quite well. How would this benefit or contaminate her data collection? It is her choice to make as I felt it was my ethical duty to let her know I was conducting research on a similar inquiry. Perhaps this is someone I can collaborate with once our dissertations are defended and our PhDs earned. It would be interesting to compare and contrast our studies once completed.
I agreed to take participate in her survey and found that we asked roughly the same demographic questions, but the other research questions differed a little. Although I am using survey software, my research is more like an interview. However, at the end of taking her survey, I was asked if I would like to participate in an online interview via Skype. I agreed and was a bit jealous that her IRB allows her to use an easier method of data collection, although there is a greater risk of losing confidentiality through Skype.
Our studies are also similar in that I assume that we would both qualify as participants in our own studies, meaning I could in mine and she in hers. However, I am not sure if she would qualify for mine. We are also similar, I believe, in that I assume that she has worked in the same English Language Fellowship I have.
I wonder about the ethical and other implications of interviewing a participant like me who may know the research theories and frameworks quite well. How would this benefit or contaminate her data collection? It is her choice to make as I felt it was my ethical duty to let her know I was conducting research on a similar inquiry. Perhaps this is someone I can collaborate with once our dissertations are defended and our PhDs earned. It would be interesting to compare and contrast our studies once completed.
Tuesday, January 8, 2013
Email Interviews
I have learned an interesting detail about conducting asynchronous online interviews. Last year, I read through the literature on online interviews to learn that the most common method of asynchronous online interviews was via email. Many studies have used this approach to collect all or some of their interview data. However, the Internal Review Board at the University of Iowa acknowledges that email is not a secure method of data collection. Therefore I run the risk of losing confidentiality through this process. The literature also mentions this risk; nonetheless, the researchers were able to conduct email interviews. However, the IRB does not advise this method, and I am not too surprised. Although I agree that email is not the most secure method of collecting interview data, the risk is not much lower with other methods. And the University of Iowa's IRB is known to be more cautious than most others. The main issue here is not contacting participants or sending them questions via email; it is the receipt of written responses. It is my responsibility to provide my participants with a secure method of answering my interview questions.
My alternative plan is to use a web survey software program supported by the University of Iowa. This program is widely used by researchers, so I assume that IRB will find this a safer alternative to email. Although this plan will consume a little bit more of my time, I will be building research skills associated with the utility of web survey software programs. Perhaps it will end up saving me time at the end of the data collection process. The lesson I learned is that even though most of the literature reports mostly positive results of a certain data collection method, it may no longer or may not be the safest or best possible way.
My alternative plan is to use a web survey software program supported by the University of Iowa. This program is widely used by researchers, so I assume that IRB will find this a safer alternative to email. Although this plan will consume a little bit more of my time, I will be building research skills associated with the utility of web survey software programs. Perhaps it will end up saving me time at the end of the data collection process. The lesson I learned is that even though most of the literature reports mostly positive results of a certain data collection method, it may no longer or may not be the safest or best possible way.
Subscribe to:
Posts (Atom)