In a previous blog, I posted that I do not understand statistics in the least. After studying to the point of insanity, I now see statistics in an entirely new way. I can now read percentages and question where these numbers came from. Are they a true representative sample of the population? How big was their sample size? What kinds of bias are involved with the results? How much error do we need to take into consideration?
I have always been skeptical of studies I have read based on the organization sponsoring the research. I figure most organizations are going to find a way to manipulate their data to prove their point. Why publish data that doesn’t support their claim?
Now I am looking at research in an entirely different light. I now do not believe it is possible to get any accurate numbers in any study in any subject. Too many factors interfere with results. Too much bias exists whether the bias stems from the researcher, from those funding the experiments, or even from the subjects themselves.
People have got to be the most difficult subjects to study. For example, if I were to go out and question complete strangers on their drug use, even if I swore that all data collected would be confidential and that they would remain anonymous, what percentage of those questions would be 100% honest? There is also a this thing called the Hawthorne Effect where participants in studies are more affected by being involved in an experiment than any variable in the experiment. Some people will answer or act in a way they think the researcher wants them to. Some may not even do this intentionally; just the thought of being watched and studies has an affect on a person’s reactions. With drug studies, how many people are REALLY experiencing side effects and how many are imagining the side effects? Could a simple headache be confused with an adverse reaction?
Anybody can throw out percentages and numbers to attempt to prove validity, but where are these numbers REALLY coming from? How can we ensure that results from experiments represent reality? I see so many errors when analyzing a study’s reliability and validity I’m not sure anything can be proven definitely. I can see how easy it is for conflicting studies to exist. It now makes sense to me how one experiment can show butter to be healthy and another illustrates that it is unhealthy.
So with this newfound awareness of the infinite number of flaws with experimentation, how can we believe anything we are told with absolute certainty if numbers and percentages can never be accurate?
All we CAN do is take into consideration every aspect of a given data set. When you see a news article stating that 75% of those who consume sugar end up with mental illness, ask yourself how this information was obtained. How many people participated in the study – was it 10 or thousands? How did they choose these participants – randomly or did they offer cash for participation? Was the sample taken from one city or from many different cities? Were the members of the control group equivalent to those of the experimental group? How were the measurements obtained? Did the researcher report possible causes of bias? If not, why didn’t they? Are they hiding something? This question is particularly important since in any experiment there is bound to be a flaw somewhere and it is imperative that it is reported. Studies that are reported to have gone off without a hitch should send up a red flag right away. Don’t just read an alarming statistic and take it as fact. We need to be fully aware of how these statistics were produced before we can assess the strength of the argument.
Homework assignment (just for fun): Can you find any conflicting statistics on the web? Look for percentages that are completely different on two separate sources. You’ll be amazed how often this occurs!
I stopped trusting most stats after taking Statistics class. It's so hard to tell how well the original information was taken that you have to wonder what may, and probably already did, go wrong.
~ *~
This is a signature, an automated thingy that pops up when I comment, not a demand to see my blog!
Mind Control is Easier Than You Think
Per your question about drug abuse, I'm going to change it slightly to prove a point, and because I just heard this lecture last night.
If you ask a bunch of men if they were abused as a child, the vast majority will say no, even if every single one of them was abused. Now, if you go about it from a different angle, asking more specific questions describing abuse, you'll get more to say yes to those questions. It's called the victimization effect, I believe, and it started being studied in the 80's.
Additionally, some crimes people have no problem explaining in detail. The guy that did the lecture last night interviews rapists for his research. He says out front that if he is ever subpoenaed, he will have to give over the taped interviews for evidence against these people. They still go into vast detail about what they've done. It's a crime to commit rape, but it's not punished nearly as much as it should be (something from like 1-5% of rapists are actually prosecuted), so it's not seen as much threat.
~C
Check out the latest entry in the Between The Lines column!
Want the highest rated list to change? RATE those blogs, then!
That's really interesting about the vicimization effect, but I wonder how many of the questions can implant false memories in the participants, also affecting the results of an experiment. For example, if someone is told that they exhibit all of the symptoms of being sexually abused and they are consistently pushed to remember their sexual abuse, the power of suggestion may give them false memories to report. Now this person would not be lying, they would just come to believe that the false memory is true. I would be willing to bet that some questions can be worded in such a way to make the participant believe they have experienced something that in reality never happened or maybe it DID happen but the circumastances were completely innocent. Determining truth in personal experiences is a touchy area.
One example of this is the photographer who took pictures of her kids jumping on the bed in their underwear. There was nothing sexual about her motives, she was just taking pictures of her kids playing early in the morning before they had dressed for the day. When the pictures were developed, the developer believed that these pictures were sexual in nature and reported the woman for sexual abuse. She had to fight for years to get her kids back simply because her photos were misinterpreted. Her children had to attend therapy and be told that they were molested when in reality they were not. Who were her children to believe? The police, the therapists, the media or their mother?
I find it highly unlikely that a study would be so easily able to implant false memories. Weeks and weeks of therapy sessions, maybe, but not simple questions by a study.
Common sense is as rare as genius. ~Emerson
My Latest Blog Posts
http://www.rense.com/general45/falsemen.htm
http://www.spring.org.uk/2008/02/therapists-can-implant-false-beliefs.ph...
http://faculty.washington.edu/eloftus/Articles/sciam.htm
Even having participant read a short story and them rephrasing some questions could cause a participant to confuse fact and fiction. It doesn't take months and months of therapy to confuse reality. It can be done in one sitting.
What you gave me above are three articles about studies whose sole purpose was to prove that false memories can be implanted. I never said I didn't believe that false memories can be implanted.
You mentioned that you wondered whether posing questions to a subject could get them to believe they had experienced abuse that had never happened. I don't believe that a study about sexually abuse where participants are merely questioned could produce false memories.
Suppose you participated in a study where they asked you if you liked chicken and you said "Yes, I do." But then they repeatedly asked you, "Are you sure you like chicken?" You wouldn't miraclously go, "No, I really don't like chicken, but I had totally forgotten until you repeatedly questioned me about it" when the truth is that you really do like chicken. They wouldn't be able to convince you otherwise by just asking you.
An experiment trying to prove a person's ability to implant false memories doesn't prove that all data-collecting studies will implant false memories and therefore screw up data results.
Common sense is as rare as genius. ~Emerson
My Latest Blog Posts
Can you give me an example of a question you are talking about?
The way I see it, if some kept asking me if I was sure about something, I would probably lie just to get them to stop asking the same question. I guess I don't see how rewording a question can get people to report something different either on a questionnaire or in an interview setting.
Just so I am clear, you're arguing that results from interviewing are accurate because a different study showed that rewording will bring 100% truthful results?
I am NOT being sarcastic or anything, I know it's hard to tell in writing whether something is sincere or not, I just got a little confused on what I was proving so I am not sure what to look up.
Thanks!
We have to be on totally different pages here because I have no idea what the heck you are talking about.
Common sense is as rare as genius. ~Emerson
My Latest Blog Posts
LOL! I get that a lot. ;) I don't think I describe my thoughts as well in words as I can in person. Or maybe no one gets me and people in person just smile and nod to get me to shut up. rofl rofl
The problem lies in the perception. If you don't perceive that you were raped, or abused, or whatever, then you will honestly answer 'no' when someone asks if you were raped, abused, or whatever. But if they start going into detail asking, say "did your father ever yell at you unnecessarily?", you could say 'well, yeah, that happened, but it's not abuse...'. That was what I was getting at.
It's rather difficult to plant false memories into someone by asking them something once. And usually, those most susceptible to that type of false memory implantation are very stressed (such as those with PTSD), and who are subjected to stressful questioning. A researcher typically doesn't want the situation to be stressful... they are merely trying to gather information. Of course, there is always a bias, so a researcher could lead a questioning without intending to, but I can't imagine that happening all that often.
~C
Check out the latest entry in the Between The Lines column!
Want the highest rated list to change? RATE those blogs, then!
You're right.
I am thinking more along the lines of a researcher possibly asking leading questions trying to gear participants to answer in a way that will prove the researcher's hypothesis. An example I can think of would be a cigarette company trying to conduct studies that will show that cigarettes aren't dangerous (I know that's a bad example). They would want to ask vague questions, like, "Have you ever smoked" trying to classify more healthy people as smokers even though they may have only had one cigarette in their entire life. Participants could answer the question honestly and still portray bad results.
As far as your example of the parent yelling, it could be a study that only allows a yes or no answer, not giving a participant an opportunity to say they did not feel as if they were abused. Or even ask "has a parent ever hit you" - this can be perceived by someone asking the question as a spanking or even being punched in the face. Since the severity is not addressed, a researcher can determine from their results that more people were abused than those who really WERE abused and those who were maybe just tapped on the rear.
I guess what I am trying to show is that, especially in studies where written questionnaires are distributed, the questions can be formulated in a way that will receive biased answers by not being detailed enough. Most of the time it is dependent on what the researcher is trying to prove. It is especially true when companies fund research since they have money invested in the findings. Anything can be skewed, it's all in the wording.
As I said, there is always a bias. But the research I was talking about, it's completely open ended questions. They want you to explain. I'm going to be doing an interview with the researcher I'm talking about, and I'll be sure to ask him how his research is conducted.
~C
Check out the latest entry in the Between The Lines column!
Want the highest rated list to change? RATE those blogs, then!
Let me know what he says! I have never really spoken with researchers and I'd really be interested to know the things they do to prevent bias and get accurate results;)
One of the issues I keep very close tabs on is illegal immigration. The very nature of this issue means that it is very difficult to get accurate numbers because these people are less willing then most to stand up and be counted.
Here are some statistics that are commonly discussed in the illegal immigration debate:
- 12 to 20 million total illegals with some saying 30 million
- 65% working with a fake or stolen social security number and 35% working in the black market economy.
- 60% of illegals crossed the border and 40% are visa overstays that never went home
- $10 billion is approximate net cost to California taxpayers of illegal immigrants.
All of these "facts" are suspect. If one accepts that they are not exact numbers but rather estimates or loose measures they are still quite useful because they allow an understanding of the magnitude of a problem.
Does it really matter if the number of illegal aliens is 9 or 12 or 15 or 18 or 21 million? We know it is a huge number likely in the low double digit millions. The 12 million number gives us a useful starting point in discussion even though we know it is not exact.
Does it really matter if 45% or 65% or 75% of illegal aliens are paying employment taxes. We know that a lot are and we also know that there is a lot of tax evasion by both them and their employers. The 65% number is loose estimate but it gives us a starting point. We can now multiply 12 million by 65% and arrive at a new number which is useful in that we can state the number of illegals who pay employment taxes. Of course anybody who understands numbers knows that when you multiply one erroneous number time another erroneous number the result is likely to amplify the errors.
The facts need to be taken with several grains of salt but they are still extremely useful.
Excellent example! I have a friend of mine who had her social security number stolen and 12 people were using it for employment. Since many employers don't validate the name with the social security number, just that the number is valid, a lot of people can use the same social and work in the United States. All they need to do is find one that works. It's almost impossible to know the true statistics since people don't go into any type of agency and report that they are an illegal alien and working under a false SSN.
I have to take a statistics class. I'm afraid of it. I suck at math. Worse than that, numbers make my brain turn off. How bad is it? It's not required for my degree, but I think I want some understanding of statistics before I tackle the data for my capstone.
http://www.progressiveu.org/blog/ediblewoman
I was completely lost until I got a tutor. I didn't have any trouble with the math part since the software (SPSS) does all the math for you - and I mean ALL of it. You just plug in the numbers and the software give the results. The thing I hated about the class is that since I don't really understand WHY the answers are the answers, I really have no way of checking my answers, I am just counting on the software to be correct. I am never good at anything if I don't understand the whole "why" behind it. It took a PhD student in research to sit with me for a few hours drawing pictures before I could get a little grasp of what I was actually figuring out.
Good luck with the class when you take it! It will help to provide you will a better understanding on how things are studied and what scientists consider "significantly" improtant.
My high school math teacher always said that statistics wasn't math, because it's so vastly different from, say, Algebra. It's not too bad, though. If I were you and had the choice, I'd see what kinds of stats classes are offered and see if there's one along the lines of "practical" stats, which would put things in context and usually makes the concepts easier to grasp.
I am treated as evil by people who claim that they are being oppressed because they are not allowed to force me to practice what they do. ~D. Dale Gulledge
I will look into that. Thanks!
http://www.progressiveu.org/blog/ediblewoman