Dr. Lorandos Interviewed on Larry King Live
 
 
About The Author

Demosthenes Lorandos is an attorney, forensic psychologist, lecturer and author.  He is in constant demand across the country for his authoritative and tireless advocacy on behalf of the falsely accused.   In defense of his clients Dr. Lorandos brings to bear his extensive expertise in the science of coerced confessions, shaken baby syndrome, battered women’s syndrome, parental alienation syndrome, recovered memories, false accusations, repeated question effects, interviewer bias, behavioral indicators of abuse and other related subjects.  Dr. Lorandos is a member of the California, Michigan, New York, Tennessee and Washington, D.C. bars, and a member of the bar of the United States Supreme Court.   He is also the co-author of such works as Cross Examining Experts in the Behavioral Sciences, Benchbook in the Behavioral Sciences and The International Handbook of Parental Alienation Syndrome. Dr. Lorandos may be recognizeable from his appearances on The Today Show, The View, Larry King Live and Court TV, and from his extensive online catalog of free legal advice videos.

 
Falsely accused of physical or sexual abuse?
Have you been falsely accused?
 
 
Exner’s Egocentricity Index and Issues of Peer Review
Posted On December 22nd, 2014  By admin  

Related data do not support other facets of the Exner system. Despite its theoretical assumptions to the contrary, Exner’s “Egocentricity Index” appears unrelated to personality characteristics such as self focus, self esteem, and narcissism. Nezworski and Wood’s 1995 review reporting these findings concluded by indicating:

“At this time there is insufficient evidence to support the validity of the EGOI [Egocentricity Index] pairs, and reflections, as measures of self esteem, self focus, narcissism, ego functioning, or depression. We recommend that clinicians regard these measures as experimental. The available scientific evidence does not support their use for clinical decision making.”[1]

This review of the TRACS also pointed out that many of the studies cited as supporting it have never undergone peer review. In a response to this review, Exner indicated:

“I continue to regard many of the studies that I have listed as ‘unpublished’ in the various volumes related to the Comprehensive System as milestones in our early efforts … I have also cautioned repeatedly that the work concerning the Comprehensive System remains unfinished …”[2]

At Falsely-Accused.net we emphasize that if the development of the Comprehensive System remains unfinished as Exner indicates, then it amounts to an experimental procedure that has not achieved general acceptance. Similarly, the previously cited 1997 study of Carlson and her colleagues concluded that Exner’s Rorschach system is “not fulfilling the publication promise.” Carlson and her colleagues also concluded:

“… something happened on the way from the research lab to the clinical field. Clearly more research needs to be done on these as well as other TRACS indices if clinicians are to have confidence in their validity for psychological diagnoses and treatment as well as for clinical research.”[3]

A 1997 study reported data it interpreted as supporting the EGOI.[4] This study used the Rorschach to identify clients in treatment who met DSM IV criteria for the diagnosis of Narcissistic Personality Disorder (NPD). The study identified the number of “reflection responses” and “idealization” responses as the best variables for identifying NPD. The scoring of “idealization” responses, however, is not addressed by the TRACS. Instead, the scoring of “idealization” responses relies on the work of Lerner (1991).[5] This consideration raises the question of how many psychologists using the EGOI are familiar with Lerner’s procedures?

At Falsely-Accused.net we note that this same 1997 study also advised caution when interpreting reflection responses on the Rorschach:

“Therefore, it is not suggested that clinicians use the presence of one or more reflection responses on the Rorschach as an indication of NPD. To do so would be a very concrete interpretation of the data presented here and would be clinically unsophisticated. What these data do suggest is that the presence of one or more reflection responses typically will be developed in outpatient populations by individuals who meet some of the DSM IV criteria for NPD, but possibly not five of the nine criteria.”[6]

Cross examining Exner’s Egocentricity Index and Issues of Peer Review
[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS
IF YOU HAVE NOT YET DONE SO]

1.  The “Egocentricity Index” of the Rorschach claims to assess personality characteristics such as self focus, self esteem, and narcissism Correct?

2.  The journal Clinical Psychology: Science and Practice is a generally recognized and accepted, peer reviewed journal in your field Correct?

3.  And a 1995 article by Nezworski and Wood published in Clinical Psychology: Science and Practice titled “Narcissism in the Comprehensive System for the Rorschach” might be relevant to your opinions in this case Correct?

4.  Please consider how Nezworski and Wood concluded their review:

– [ read ] –

“At this time there is insufficient evidence to support the validity of the EGOI [Egocentricity Index] pairs, and reflections, as measures of self esteem, self focus, narcissism, ego functioning, or depression.”

Now my question: If there is insufficient evidence to support the Egocentricity Index as a measure of self esteem, self focus, narcissism, ego functioning, or depression, then your relying on it is ill advised Correct?

5.  Please consider how Nezworski and Wood further concluded their review:

– [ read ] –

“We recommend that clinicians regard these measures as experimental.”

Now my question: If the Egocentricity Index is an experimental measure, then by definition it is not generally accepted by your scientific and professional community Correct?

6.  And if you have relied on an experimental measure not generally accepted by your scientific and professional community you could misinform and mislead this proceeding Correct?

7.  Please also consider how Nezworski and Wood additionally concluded their review:

– [ read ] –

“The available scientific evidence does not support their use for clinical decision making.”

Now my question: If the available clinical evidence does not support the use of the Egocentricity Index for clinical decision making, then your relying on it could also misinfom and mislead this proceeding Correct?

8.  Please consider these additional comments by Nezworski and Wood in their 1995 article:

– [ read ] –

“… the Workshop Studies of the EGOI cited in the Rorschach (Exner, 1993) are unavailable for scholarly examination.”

Now my question: If Exner’s Workshop Studies are unavailable for scholarly examination, that means those studies have not undergone peer review Correct?

9.  And we have already established how critically important peer review is in the development and advancement of your field Correct?

[This was done via the 22 foundational questions from Previous posts]

10.  Please consider how Exner responded to the criticisms of Nezworski and Wood:

– [ read ] –

“I continue to regard many of the studies that I have listed as ‘unpublished’ in the various volumes related to the Comprehensive System as milestones in our early efforts …”

Now my question: If Exner acknowledged that his own studies are “unpublished,” then he also acknowledges that they have not undergone peer review Correct?

11.  Please consider Exner’s continued response to the criticisms of Nezworski and Wood:

– [ read ] –

“I have also cautioned repeatedly that the work concerning the Comprehensive System remains unfinished …”

Now my question: If Exner acknowledges that work concerning the Comprehensive System remains unfinished, his acknowledgment further confirms that the TRACS is an experimental procedure Correct?

12. Please consider Standard 2.02 (a) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(a) Psychologists who develop, administer, score, interpret, or use psychological assessment techniques, interviews, tests, or instruments do so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

Now my question: Your work in this case obligates you to comply with this standard Correct?

13. If work concerning the Comprehensive System remains “unfinished,” your relying on it neglects Standard 2.02 (a) of your ethical code Correct?

14. Because if work concerning the Comprehensive System remains “unfinished,” you cannot use it in a manner that is “appropriate in light of the research on or evidence of the usefulness and proper application of the technique” Correct?

15. Also please consider Standard 2.02 (b) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

Now my question: Your work in this case obligates you to also comply with this standard Correct?

16. If work concerning the Comprehensive System remains “unfinished,” your relying on it also neglects Standard 2.02 (b) of your ethical code Correct?

17. Because if work concerning the Comprehensive System remains “unfinished,” you are at risk for misusing this Aassessment technique” and its “interpretations” Correct?

18. The journal Psychological Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

19. And a 1997 article published in Psychological Assessment by Hilsenroth and his colleagues titled “Narcissism in the Rorschach revisited: Some reflections on empirical data” might be relevant to your opinions in this case Correct?

20. Hilsenroth and his colleagues reported that for their study:

– [ read ] –

“The administration and original scoring followed the procedures articulated by Exner, except in the case of the idealization variable, which was scored according to the procedures of Lerner (1991).”

Now my question: Did you use the scoring procedures of Lerner, for the idealization responses, found in his 1991 book Psychoanalytic Theory and the Rorschach published by the Analytic Press?

21. And without using Lerner’s scoring procedure for “idealization responses” on the Rorschach, it may be inappropriate for you to cite the 1997 study of Hilsenroth and his colleagues as supporting your use of the Rorschach Correct?

22. Please consider these additional comments from Hilsenroth and his colleagues regarding reflection responses on the Rorschach: ”

– [ read ] –

“Therefore, it is not suggested that clinicians use the presence of one or more reflection responses on the Rorschach as an indication of NPD.[Narcissistic Personality Disorder] To do so would be a very concrete interpretation of the data presented here and would be clinically unsophisticated.”

Now my question: If relying on one or more reflection responses to identify NPD [Narcissistic Personality Disorder] is “clinically unsophisticated,” then your use of the Rorschach for this purpose may also be “clinically unsophisticated” Correct?


 

[1] Nezworksi, M.T. & Wood, J.M. (1995). Narcissism in the Comprehensive System for the Rorschach. Clinical Psychology: Science and Practice, 2, 179 199, at page 196.

[2]Exner, J.E. (1995). Comment on “Narcissism in the Comprehensive System for the Rorschach.” Clinical Psychology: Science and Practice, 2, 200 206, at page 205.

[3]Carlson, C.F., Kula, M.L. & St. Laurent, C. (1997). Rorschach revised DEPI and CDI with inpatient depressives and borderline personality disorder with major depression: Validity issues. Journal of Clinical Psychology, 53, 51 58 at page 56.

[4]Hilsenroth, M.J., Fowler, J.C., Padawer, J.R. & Handler, L. (1997). Narcissism in the Rorschach revisited: Some reflections on empirical data. Psychological Assessment, 9, 113 121.

[5]Lerner, P. (1991). Psychoanalytic theory and the Rorschach. Hilldale, NJ: Analytic Press.

[6]Id. at page 119.

 
 
Exner’s Depression Index
Posted On December 15th, 2014  By admin  

At Falsely-Accused.net we note that Exner’s “Depression Index” (DEPI) supposedly identifies people who are clinically depressed. A review of the relevant literature, however, does not support the DEPI. A 1991 study investigated the DEPI R in children and adolescents. It found no significant relationships between (1) the original (DEPI) or revised Depression Index (DEPI R) and clinical elevations on the Depression scale of the Personality Inventory for Children in an outpatient sample, or (2) the diagnostic judgments of treatment teams in an outpatient sample.[1]

We at Falsely-Accused.net emphasize that a 1997 study found little agreement between the positive values of the DEPI R and DSM III diagnoses of Major Depressive Disorder.[2] In this 1997 study, two other independent measures of depression the MMPI 2 Depression Scale and the Beck Depression Inventory validated the presence of depression and correlated highly with each other, but not with the DEPI R. Similarly, a 1993 study found no significant relationship between the DEPI R and patients’ self reports of depression.[3]

Cross examining Exner’s Depression Index

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS
IF YOU HAVE NOT YET DONE SO]

1. The DEPI R of the (TRACS) Rorschach refers to the “Depression Index Revised” Correct?

2. And the DEPI R of the (TRACS) Rorschach supposedly identifies people who are depressed Correct?

3. And the DEPI R can supposedly be used with children Correct?

4. The Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

5. And a 1991 study by Ball and his colleagues published in the Journal of Personality Assessment titled “Rorschach Depression Indices with Children and Adolescents: Concurrent Validity Findings” might be relevant to your opinions in this case Correct?

6. Did you know that this 1991 study found no relationship between children’s DEPI R scores, and their scores on the Depression scale of the Personality Inventory for Children?

7. This outcome suggests that either the Depression scale of the Personality Inventory for Children, or the DEPI R, or both, are inadequate for identifying depressed children Correct?

8. Did you know that this same 1991 study found no relationship between the DEPI R and the diagnostic judgments of inpatient treatment teams?

9. If the DEPI R is unrelated to both the Depression scale of the Personality Inventory for Children, and the diagnostic judgments of inpatient treatment teams, this suggests problems with the DEPI R Correct?

10. You have not published any data in a peer reviewed journal necessitating that we reconsider the 1991 findings of Ball and his colleagues Correct?

11. You cannot cite any data published in a peer reviewed journal necessitating that we reconsider the 1991 findings of Ball and his colleagues Correct?

12. Scale 2 the Depression Scale of the MMPI 2 is a generally recognized and accepted instrument for assessing depression in an adult population Correct?

13. And the Beck Depression Inventory is also a generally recognized and accepted instrument for assessing depression in an adult population Correct?

14. In other words, if patients score low on Scale 2 fo the MMPI 2, we would expect them to score low on the Beck Depression Inventory Correct?

15. And if patients score high on Scale 2 fo the MMPI 2, we would expect them to also score high on the Beck Depression Inventory Correct?

16. The Journal of Clinical Psychology is a generally recognized and accepted, peer reviewed journal in your field Correct?

17. And a 1997 study by Carlson and her colleagues published in the Journal of Clinical Psychology titled “Rorschach Revised DEPI and CDI with Inpatient Depressives and Borderline Personality Disorder with Major Depression: Validity Issues” might be relevant to your opinions in this case Correct?

18. Carlson and her colleagues reported:

– [ read ] –

“… two other measures of depression, the MMPID [scale 2] and the BDI [Beck Depression Inventory], further validate the presence of depression and correlate highly with each other …”

Now my question: When scale 2 of the MMPI and the Beck Depression Inventory correlate highly with each other, that is to be expected Correct?

19. Carlson and her colleagues further reported:

– [ read ] –

“… two other measures of depression, the MMPID and the BDI further validate the presence of depression and correlate highly with each other, but not with the DEPI R.”

Now my question: If either the Beck Depression Inventory, or Scale 2 of the MMPI 2, do not correlate with the Rorschach DEPI R, it appears that the DEPI R is an uncorrelated measure of depression Correct?

20. In other words, we do not know what the DEPI R assesses other than knowing it does not assess depression as it is assessed by the Beck Depression Inventory and Scale 2 of the MMPI 2 Correct?

21. And the Beck Depression Inventory and Scale 2 of the MMPI 2 are both objective measures of depression Correct?

22. And compared to the Beck Depression Inventory and Scale 2 of the MMPI 2, the Rorschach DEPI R is a more subjective index Correct?

23. And the objectivity of the Beck Depression Inventory and Scale 2 of the MMPI 2, compared to the relative subjectivity of the Rorschach DEPI R, indicates we can place more confidence in the Beck Inventory and the MMPI 2 Correct?

24. You have not published anything in a peer reviewed journal necessitating that we reconsider the 1997 findings of Carlson and her colleagues Correct?

25. You cannot cite any data published in a peer reviewed journal necessitating that we reconsider the 1997 findings of Carlson and her colleagues Correct?

26. We previously established that the Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

27. And a 1993 study by Meyer published in the Journal of Personality Assessment titled “The Impact of Response Frequency on the Rorschach Constellation Indices and on Their Validity with Diagnostic and MMPI 2 criteria” might be relevant to your opinions in this case Correct?

28. And would it surprise you to know that Meyer also found no relationship between the DEPI R and the MMPI 2?

29. And if Meyer’s 1993 study found no significant relationship between the Rorschach DEPI R and the MMPI 2, that outcome further challenges the validity of the Rorschach DEPI R Correct?

30. Please consider Standard 2.02 (a) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(a) Psychologists who develop, administer, score, interpret, or use of psychological assessment techniques, interviews, tests, or instruments do  so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

Now my question: Your work in this case obligates you to comply with this standard Correct?

31. Without the availability of validity data supporting the DEPI R, your relying on that index neglects Standard 2.02 (a) of your ethical code Correct?

32. Because without the availability of validity data supporting the DEPI R, you cannot use that index in a manner that is “appropriate in light of the research on or evidence of the usefulness and proper application of the technique” Correct?

33. Also please consider Standard 2.02 (b) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

Now my question: Your work in this case also obligates you to comply with this ethical standard Correct?

34. Without the availability of validity data supporting the DEPI R, your relying on that index neglects Standard 2.02 (b) of your ethical code Correct?

35. Because without the availability of validity data supporting the DEPI R, you are at risk for misusing this “assessment technique” and its “interpretations” Correct?

36. And without the availability of validity data supporting the DEPI R, your relying on that index could misinform and mislead this proceeding Correct?


 

[1]. Ball, J.D., Archer, R.P., Gordon, R.A. & French, J. (1991). Rorschach depression indices with children and adolescents: Concurrent validity findings. Journal of Personality Assessment, 57, 465 476.

[2]. Carlson, C.F., Kula, M.L. & St. Laurent, C. (1997). Rorschach revised DEPI and CDI with inpatient depressives and borderline personality disorder with major depression: Validity issues. Journal of Clinical Psychology, 53, 51 58.

[3]. Meyer, G.J. (1993). The impact of response frequency on the Rorschach constellation indices and their validity with diagnostic and MMPI 2 criteria. Journal of Personality Assessment, 60, 153 180.

 

 

 

 

 
 
Rorschach Validity General Considerations
Posted On December 8th, 2014  By admin  

As previously explained in prior posts, the value of any psychological test inevitably corresponds to its validity. The validity of a test corresponds to whether it actually assesses what it claims to assess. Moreover, it is necessary to clearly specify what a test is valid for. The Minnesota Multiphasic Personality Inventory (MMPI), for example, is valid for assessing various dimensions of personality. The MMPI, however, is not valid for assessing intelligence. We at Falsely-Accused.net teach that consequently, it is always mistaken to speak of a “valid test” without specifying for what it is valid. For more than five decades, challenge after challenge has been directed at the validity of the Rorschach. Concerned by the brevity of validity data to support the Rorschach, an otherwise favorable 1946 review confessed:

“Frankly, the reviewer would prefer to keep this material from the eyes of ‘academic’ psychologists until more nearly conclusive proofs of Rorschach validity are available … one must agree that the Rorschach is largely an art …” [1]

In his 1953 review of Beck’s Rorschach’s Test III: Advances in Interpretations, Lee Cronbach concluded:

“We have now had thousands of research studies, some well conducted, which have failed to establish validity of many interpretations commonly made [using the Rorschach] … We find Beck turning his back on research evidence in a way that is too common in the literature and in the conversations of clinicians, by saying, in effect, ‘Evidence or no evidence, these propositions [popular Rorschach interpretations] have clinical utility.”[2]

In 1959, Eysenck summarized his review of Rorschach validity data published during the five previous years. Eysenck concluded:

“There is no evidence that conscious or unconscious conflicts, attitudes, fears or fantasies in patients can be diagnosed by means of projective techniques in such a way as to give congruent results with assessments made by psychiatrists independently … There is ample evidence to show that the great majority of studies in the field of projective techniques are inadequately designed, have serious statistical errors in the analysis of the data, and/or are subject to damaging criticisms on the grounds of contamination between the test and criterion.”[3]

In 1965, Professor Dana suggested the time had come for Rorschach loyalists to acknowledge the many problems related to this technique:

“… there is continual pressure to demonstrate that the test meets the usual psychometric criteria of objectivity, reliability, validity, additivity, scaling procedures etc., which in fact it does not … In spite of our persistent attempts to convert the Rorschach into a psychometric instrument, we have failed.”[4]

Despite his admitting the Rorschach’s many shortcomings, Dana nonetheless advocated that there remained a place for it in clinical practice. Curiously enough, it appears that numerous psychologists continue to embrace this thinking almost 40 years later. “Conclusive proofs” of the Rorschach’s validity are as elusive as they were in 1946; and as a result, the Rorschach remains more “art” than science.

We at Falsely-Accused.net emphasize that since the 1940s, the Rorschach has more often than not failed to deliver on its promises. Now in the 21st century, the verdict is clear and evident Relying on the Rorschach to assess conscious and/or unconscious experience is ill advised at best, and illogical and ill informed at worst. Nevertheless, legions of mental health professionals continue to react as Cronbach described them in 1953 – “Evidence or no evidence,” my clinical experience persuades me that the Rorschach is a valuable assessment procedure.

We at Falsely-Accused.net consider, for example, the feeble position assumed by a psychologist as she responded to cross examination regarding the Rorschach’s validity.

Attny: What does Standard 6.1 say? [referring to the 1985 Standards for Educational and Psychological Testing in effect at that time].

Psych: “Test users should evaluate the available written documentation on the validity and reliability of tests for the specific use intended.”

Attny: Have you complied with that standard in the use of the Rorschach?

Psych: Yes, I have.

Attny: How?

Psych: By reading articles, going to conferences, consulting with colleagues. What I have said to you, several times, was that I do not catalog and remember what read by author, date, title, etcetera. I have satisfied myself through other means, including reading. I will bring with me, if we have to go trial, recent articles that I have read that say use of the Rorschach is perfectly legitimate if one chooses to do so. Do I have them on the top of my head to recite to you? No.

Despite this psychologist’s vehement protests, the fact remains that she neglected to comply with 1985 Testing standard 6.1 at the time she used the Rorschach in this case. Whatever material she cites in the future will not change this fact.

Evaluating the validity of Exner’s Rorschach (TRACS) necessitates assessing the validity of its many subscales. The DEPI (Depression) Index, for example, claims to identify clinically significant levels of depression. The SCZI (Schizophrenia) Index claims to discriminate between psychotic and non psychotic disorders. The validity of the (TRACS) Rorschach ultimately corresponds to the validity of these and other specialized indices. Without sufficient data demonstrating the validity of these indices, relying on them disregards Standards 2.02 (a b) of the 1992 Ethical Code for psychologists. Standards 2.02 (a b) addressing “Competence and Appropriate Use of Assessments and Interventions” state:

“(a) Psychologists who develop, administer, score, interpret, or use psychological assessment techniques, interviews, tests, or instruments do so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

_____________________________________________________________________________

[1] Harriman, P.L. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 146-147.

[2]  Cronbach, L.J. (1959). Review of Rorschach texts in O.K. Buros (Ed.), The Fifth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. Page 979

[3]  Eysenck, H.J. (1959). Review of Rorschach in O.K. Buros (Ed.), The Fifth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 276 – 277

[4]  Dana, R.H. (1965). Review of Rorschach in O.K. Buros (Ed.), The Sixth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. Page 495

 

 
 
Exner’s Rorschach System and Inter Rater Reliability
Posted On December 1st, 2014  By admin  

We at Falsely-Accused.net emphasize that the use of the Rorschach has long involved problems of inter rater reliability. Less than adequate levels of inter rater reliability indicate that two or more psychologists, independently scoring the same Rorschach responses, could score them quite differently. Such variations in scoring would then lead to very different interpretive conclusions based on the same Rorschach data. Without adequate levels of inter rater reliability, therefore, the evidentiary value of any psychological test is thoroughly compromised. The British psychologist, H.J. Eysenck commented on the Rorschach and its problems of inter rater reliability in 1959:

“In addition to the damning evidence regarding the validity of the Rorschach, it should perhaps also be pointed out that studies of the reliabilities of different Rorschach scores have shown these to be very low indeed.” (*61).

Psychologists using the Rorschach typically claim that Exner’s TRACS (The Rorschach: A Comprehensive System) has resolved the problems of inter rater reliability long associated with this technique. Rather than report levels of inter rater reliability, however, Exner reported the percentage of agreement between different psychologists using the TRACS. We at Falsely-Accused.net note that as a result, Wood and his colleagues emphasized: “It appears that the interrater reliability of the Comprehensive System [TRACS] is presently unknown” . Previous claims regarding the inter rater reliability of TRACS relied on unpublished studies not subjected to the scrutiny of peer review .

For research purposes, there are standards ensuring adequate inter rater reliability for the Rorschach. Since 1991, the Journal of Personality Assessment has required a minimum inter rater agreement of 80% for any Rorschach study submitted for publication . This level of research reliability, however, is not necessarily related to the Rorschach’s field reliability. The psychologists participating in these studies underwent specialized training to reach an acceptable level of inter rater agreement.

The Rorschach’s field reliability refers to how consistently it is scored by practitioners using it clinically rather than for research purposes. In fact, there are no peer reviewed data demonstrating how the TRACS has influenced the community practitioners who use it . In other words, the extent to which practicing psychologists carefully comply with the TRACS scoring procedures remains unknown. Because the TRACS involves 54 different ratios and percentages in its scoring procedure, we at Falsely-Accused.net feel that the potential for error is enormous.

A 1999 broad scale analysis of the Rorschach and the MMPI 2 concluded that both tests work equally well . This analysis, however, relied on the inter rater reliabilities for the Rorschach realized by research teams. Extrapolating from these data to assume that community practitioners score the Rorschach reliably is ill advised. Addressing the issue of “research reliability” versus “field reliability,” another 1999 article indicated:

“… the Comprehensive System is a complex administration and scoring system, and evidence of scoring reliability in research contexts would not automatically generalize to clinical practice. To develop and maintain proficiency in the Comprehensive System requires enormous time and effort. At present we have no evidence for the field reliability of system.”

This same 1999 article continued:

“Thus, without compelling evidence that a Rorschach protocol is properly administered and scored according to the rules of the Comprehensive System, the validity of the interpretation arising from the scoring cannot be assumed.”

Cross examining Exner’s Rorschach System and Inter Rater Reliability

1.  Your assessment work obligates you to comply with the 1999 edition of the Standards for Educational and Psychological Testing Correct?

2.  The 1999 Testing Standards specifically state:

- [ read ] -
“The applicability of the Standards to an evaluation device or method is not altered by the label applied to it (e.g., test, assessment, scale, inventory)”

And now my question: Given the statement I just cited from the 1999 Testing Standards, those Standards apply directly to your assessment work in this case Correct?

3.  Please consider Standard 2.10 of the 1999 Standards for Educational and Psychological Testing which states:

- [ read ] -
“When subjective judgment enters into test scoring, evidence should be provided on both inter rater consistency in scoring and within examinee consistency over repeated measurements.” (p. 33).

Now my question: Your work in this case obligates you to comply with this standard Correct?

4.  Evidence of inter rater consistency in scoring also refers to what is generally recognized and accepted in your field as inter rater reliability Correct?

5.  Psychologists using the Rorschach often claim that Exner’s system has resolved the problems of inter rater reliability Correct?

6.  In using the Rorschach in this case, did you rely on Exner’s TRACS or what is otherwise known as the Comprehensive System?
[If the psychologist answers no, establish what scoring system was used
and continue to ask the questions below].

7.  For research purposes, we know that the Rorschach can potentially be scored in a consistent manner Correct?

8.  Since 1991, the Journal of Personality Assessment has required a minimum inter rater agreement of 80% before publishing an article submitted to it Correct?

9.  But that requirement corresponds to what we might call the research consistency of the Rorschach Correct?

10.  The psychologists participating in those research studies underwent specialized training to reach acceptable levels of inter rater agreement Correct?

11.  What specialized training have you undergone to assist you in scoring the Rorschach in an acceptable manner?

[The vast majority of psychologists have not undertaken this kind of training.
Prior to cross examination, check the psychologist’s c.v. for post doctoral
training in the Rorschach]

12.  And the Afield reliability” of the Rorschach refers to how practicing psychologists such as yourself use it Correct?

13.  Can you cite any data published in a peer reviewed journal reporting the field reliability for the TRACS? [or whatever other system the psychologist used].

14.  In other words, the extent to which practicing psychologists carefully comply with the TRACS scoring procedures [or any other scoring procedure] remains unknown Correct?
[USE THE 22 FOUNDATIONAL QUESTIONS FROM CHAPTER ONE
IF YOU HAVE NOT YET DONE SO]

15.  The journal Psychological Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

16.  And a 1999 article by Hunsley and Bailey published in Psychological Assessment titled AThe Clinical Utility of the Rorschach: Unfulfilled Promises and an Uncertain Future” might be relevant to your opinions in this case Correct?

17.  Please consider these comments of Hunsley and Bailey from their 1999 article:

- [ read ] -
“… the Comprehensive System is a complex administration and scoring system, and evidence of scoring reliability in research contexts would not automatically generalize to clinical practice.”

Now my question: If evidence of Rorschach scoring reliability in research contexts does not automatically generalize to clinical practice, then it is inappropriate for you to cite data from research contexts suggesting that you score the Rorschach in a reliable manner Correct?

18.  Please consider these further comments expressed by Hunsley and Bailey in their 1999 article:

- [read] -
“At present we have no evidence for the field reliability of [the Exner] system.”

Now my question: You have not published any data in a peer reviewed journal necessitating that we reconsider the position of Hunsley and Bailey in this regard Correct?

19.  You cannot cite any data published in a peer reviewed journal necessitating that we reconsider the position of Hunsely and Bailey in this regard Correct?

20.  Without the availability of peer reviewed data indicating otherwise, your profession generally recognizes and accepts that there are no data supporting the field reliability of the Exner scoring system Correct?

21.  Please consider these additional comments from Hunsley and Bailey’s 1999 article.

- [ read ] -
“Thus, without compelling evidence that a Rorschach protocol is properly administered and scored according to the rules of the Comprehensive System, the validity of the interpretation arising from the scoring cannot be assumed.” (6).

Now my question: You cannot provide this proceeding with “compelling evidence” demonstrating that you properly administered and scored the Rorschach in this case Correct?

22.  And if you cannot demonstrate via “compelling evidence” that you administered and scored the Rorschach properly in this case, then it is inappropriate for us to assume the validity of your Rorschach interpretations Correct?
[ASK THE FOLLOWING QUESTIONS ONLY IF THE PSYCHOLOGIST HAS NOT USED A COMPUTERIZED INTERPRETATION OF THE RORSCHACH]

23.  The TRACS includes 54 different ratios and percentages in its scoring procedure Correct?

24.  And because the TRACS includes includes 54 different ratios and percentages in its scoring procedure, the potential for error is enormous Correct?

25.  Given the burden of scoring 54 different ratios and percentages, do you really think you did it with no errors whatsoever?

26.  But you cannot cite any data published in a peer reviewed journal demonstrating that there is a substantial likelihood of your scoring these 54 different ratios and percentages in an accurate manner Correct?

 ________________________________________________________________________

1.  Eysenck, H.J. (1959). Review of Rorschach in O.K. Buros (Ed.), The Fifth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press:
“most of the alleged verifications of Rorschach hypotheses are achieved only through the admission of uncontrolled sources of error.” page 277

2.  Percentage of agreement may be a misleading concept: Jensen provided an important look at the basis for some of the confounding research results promulgated by the Rorschachers:

“A word of caution concerning improper estimates of Rorschach reliability: these often consist of reporting the percentage of agreement between two or more judges. It should be clear that percentage agreement is not a legitimate measure of reliability and tells us none of the things we want to know when we ask about the reliability of a test. What we want to know is the proportion of variance in the scores that is not error variance. The reliability coefficient tells us this; the percentage agreement does not.”

Jensen, A.R. (1965). Review of Rorschach in O.K. Buros (Ed.), The Sixth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. Page 501 page 504

3.  Wood, J.M., Nezworski, M.T. & Stejskal, W.J. (1996). Thinking critically about the Comprehensive System for the Rorschach: A reply to Exner. Psychological Science, 7, 14 17, at page 14.

4.  Wood, J.M., Nezworski, M.T. & Stejskal, W.J. (1996). The Comprehensive System for the Rorschach: A Critical Review. Psychological Science, 7, 3 -10.

5.  Hunsley, J. & Bailey, J.M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future. Psychological Assessment, 11, 266 277.

6.  Id.

7.  Hiller, J.B., Rosenthal, R., Bornstein, R.F., Berry, D.T. & Brunell Neuleib, S. (1999). A comparative meta analysis of Rorschach and MMPI validity. Psychological Assessment, 11, 278 296.

8.  Hunsely, J. & Bailey, J.M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future. Psychological Assessment, 11, 266 277 (p. 272).

9.  Id.

10.  American Psychological Association (1999). Standards for Educational and Psychological Testing. Washington, D.C.: Author.

 

 
 
The Rorschach and Supplementary Information
Posted On November 24th, 2014  By admin  

At Falsely-Accused.net we feel that any psychologist who advocates using the Rorschach for assessment purposes must identify its legitimate uses. It is obviously inappropriate to use the Rorschach for any and all assessment tasks. It becomes necessary to ask, then, for what assessment endeavors might the Rorschach be appropriate? Meyer characterized Stricker and Gold’s 1999 article as advocating:

“… the Rorschach is a valuable instrument for clinicians (and researchers) when the goal is to understand a person in his or her full complexity, particularly when clinicians rely on a sophisticated, theoretically driven synthesis of Rorschach data in conjunction with other sources of information.”

 

Meyer’s comments raise the question of exactly what other sources of information warrant consideration. There are potentially an infinite number of information sources about a client that one might review. The issue therefore is what sources of information are most relevant to the assessment questions at hand? Selecting the most appropriate sources of information for supplementing Rorshach data inevitably relies on clinical judgment. Relying on clinical judgment to identify appropriate sources of information characteristically leads to substantial variations of opinion between two or more psychologists (see previous posts for a review of clinical judgment). In other words, there is no standardized procedure available for identifying what sources of information should be used to supplement Rorschach data.

In their 1999 article, Stricker and Gold nonetheless commented favorably on a 1990 review of Kleinmuntz advocating:

“… an approach toward assessment that integrates one’s cognitive awareness of the statistical meanings of data with an intuitive, clinically based approach. This integration enriches the assessment and allows the limitations of both perspectives to be corrected.”

Attempts at integrating statistical data and intuitive impressions drive psychologists into the chaotic abyss of clinical judgment. Relying on such “intuitive” impressions is tantamount to overriding actuarial data with case specific information. Rather than reduce error, resorting to intuitive impressions in this manner typically increases it. As pointed out in previous posts, the accuracy of statistical data (or actuarial methods) exceeds intuition (clinical judgment) by a margin of eight to one.

At Falsely-Accused.net we note that the Rorschach can be interpreted via a “nomothetic” or “ideographic” approach. The nomothetic approach relies on presumably objective data detailing supposed relationships between Rorschach responses and personality characteristics. Exner’s TRACS (The Rorschach A Comprehensive System) is an attempt at using the Rorschach in a nomothetic manner. The ideographic approach to Rorschach interpretation is theoretically driven, and specific to the particular person taking it. In a 1998 article, Smith discussed the differences between the nomothetic and ideographic approaches.

“We score a Rorschach protocol, compute various ratios and indexes from these scores, and draw inferences about the person based on the empirical correlates of those scores [nomothetic approach]. Or we look at the sequence ogical constructs based on our theoretical propositions, and derive inferences about the person from those constructs [ideographic approach].”

 

At Falsely-Accused.net we teach that ideographic interpretations of the same Rorschach can vary substantially depending on who is doing the interpretation. To the extent a psychologist relies on ideographic interpretations of the Rorschach, it is no longer a nomothetic instrument. Instead, it is an instrument relying on speculation and guesswork.

In a 1998 article, Handler advised against what he termed “reductionistic” (or nomothetic) interpretations of Rorschach responses involving color. Psychologists using the Rorschach assume how people respond to the colors of the cards reveals information about their emotional experiences. Handler specifically wrote:

“… not every C [color] response is the same. The response of fire or blood to a red area is quite different from , for example, the following response: ‘This looks like a bit of sky, the pale blue you sometimes see early in spring.’ Both of these responses are C [color], but the second response has an aesthetic emphasis in its subtlety. It lacks the primitive impulsiveness and possible helplessness of the former response. Aesthetic appreciation, the focus on subtlety, and the focused elaboration all speak to a higher level of development, compared with the first response.”

Handler therefore recommends ideographic interpretations of Rorschach responses. In doing so, he invites psychologists to indulge in the kind of conjecture and speculation that ultimately amounts to guesswork.

Cross examining The Rorschach and Supplementary Information
[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS IF YOU HAVE NOT YET DONE SO]

1. The journal Psychological Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

2. And a 1999 article by Meyer published in Psychological Assessment titled “Introduction to the Special Series on The Utility of The Rorschach for Clinical Assessment” might be relevant to your opinions in this case Correct?

3. Please consider Meyer’s comments regarding the value of the Rorschach:

“… the Rorschach is a valuable instrument for clinicians (and researchers) when the goal is to understand a person in his or her full complexity, particularly when clinicians rely on a sophisticated, theoretically driven synthesis of Rorschach data in conjunction with other sources of information.”

Now my question: If it is necessary to synthesize Rorschach data in conjunction with other sources of information, we are considering supplementary information Correct?

4. And in this case, you considered supplementary information when interpreting the Rorschach data you obtained Correct?

5. There are potentially an infinite number of information sources that a psychologist might review for supplementary purposes Correct?

6. The issue therefore becomes what sources of supplementary information are most relevant for a particular assessment Correct?

7. Selecting the most appropriate sources of information for supplementing Rorschach data relies on clinical judgment Correct?

8. And relying on clinical judgment to identify appropriate sources of supplementary information can lead into substantial variations of opinion between two or more psychologists Correct?

9. Psychologist A might want to supplement the Rorschach with the MMPI 2, but psychologist B wants to supplement the Rorschach with the MCMI III Correct?

10. In other words, there is no standardized procedure available for identifying what sources of information should be used to supplement Rorschach data Correct?

11. Because there is no standardized procedure available for identifying what sources of information should be used to supplement Rorschach data, your use of supplementary information threatens to misinform and mislead this proceeding Correct?

12. Your testimony threatens to misinform and mislead this proceeding because other psychologists could have relied on supplementary information quite different than yours, and reached very different conclusions as a result Correct?

13. The Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

14. And a 1998 article by Smith published in the Journal of Personality Assessment titled: AThe Rorschach Experience: Comments on articles by Handler, Leichtman, Lerner, and Peterson and Sayer” might be relevant to your opinions in this case Correct?

15. Please consider Smith’s comments from his 1998 article:

“We score a Rorschach protocol, compute various ratios and indexes from these scores, and draw inferences about the person based on the empirical correlates of those scores. Or we look at the sequence of responses and the precise verbalizations, translate those verbalizations into psychological constructs based on our theoretical propositions, and derive inferences about the person from those constructs.”

Now my question: When Smith refers to computing “various ratios and indexes from these scores, and draw inferences about the person based on the empirical correlates of those scores” he is referring to a nomothetic interpretation of the Rorschach Correct?

16. And when Smith refers to looking:

“at the sequence of responses and the precise verbalizations, translate those verbalizations into psychological constructs based on our theoretical propositions, and derive inferences about the person from those constructs” he is referring to an ideographic interpretation of the Rorschach Correct?

17. Psychologists can also interpret the Rorschach using both nomothetic and ideographic approaches Correct?

18. In this case, did you interpret the Rorschach using both nomothetic and ideographic approaches?

[If the psychologist says he or she relied only on a nomothetic approach, then ask the following question:  Another psychologist assessing the parties in this case could have used both nomothetic and ideographic approaches Correct?]

19. Ideographic interpretations of the same Rorschach can vary substantially depending on who is doing the interpretations Correct?

20. In other words, if Psychologist A, and Psychologist B, and Psychologist C all undertake ideographic interpretations of the same Rorschach, there can be substantial differences in their interpretations Correct?

21. We previously established that the Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

22. And a 1998 article by Handler published in the Journal of Personality Assessment titled “Schactel and teaching: What color was the couch?” might be relevant to your opinions in this case Correct?

23. Please consider Handler’s opinions regarding color responses on the Rorschach:

“… not every C [color] response is the same. The response of fire or blood to a red area is quite different from , for example, the following response: ‘This looks like a bit of sky, the pale blue you sometimes see early in spring.’ Both of these responses are C [color], but the second response has an aesthetic emphasis in its subtlety.”

Now my question: When Handler says “not every color response is the same,” he is advocating ideographic interpretations of the Rorschach Correct?

24. And to the extent any psychologist relies on ideographic interpretations of the Rorschach, it is no longer a nomothetic instrument Correct?

25. And using the Rorschach in a ideographic manner involves the kind of conjecture and speculation that amounts to guesswork Correct?

 
 
Reliance on Psychoanalytic Theory
Posted On May 15th, 2014  By admin  

The Rorschach has been associated with psychoanalytic theory beginning with Hermann Rorschach himself. Hermann Rorschach identified closely with the classical psychoanalytic theory of Sigmund Freud which emphasized assumptions of “infantile sexuality”, “penis envy”, and “anal eroticism”[1]. In 1946 Super’s comments regarding the Rorschach indicated: “Psychoanalytic interpretations are frequent and glibly incorporated into other statements with no citation of evidence to justify them.”[2] Also in 1946, Rabin addressed the close relationship between the Rorschach and psychoanalytic theory:

“… the striking psychoanalytic and otherwise symbolic explanations of various answers and bits of independent content are sometimes over‑dogmatic and not readily digestible …”[3]

We at Falsely-Accused.net note that psychologists who currently use the Rorschach frequently align themselves with one or more of the many variations of psychoanalytic theory. Stricker and Gold, advocates of the Rorschach, suggest that it is compatible with theoretical orientations other than psychoanalytic. Nevertheless, they also indicated:

“The affiliation of between the Rorschach and psychoanalytic theory is both an accident of history (the technique was developed and nurtured in a psychoanalytic environment) and a result of the powerful fit between the theory and the type of data that the technique provides” [4]

Stricker and Gold further indicated:

“On the broad spectrum of sensitivity, the Rorschach is uniquely suited for understanding the underlying organization of the personality and for generating hypotheses about unconscious contents of the mind.”[5]

Unbridled enthusiasm for exploring unconscious experiences too often leads to orgies of conjecture and speculation. Such enthusiasm also passionately embraces psychoanalytic theory. We at Falsely-Accused.net feel that as a result, it is necessary to ask: How does one verify the accuracy with which the Rorschach, or any other assessment instrument, assesses unconscious levels of experience? By definition, we do not have direct access to unconscious experiences. Therefore, we can never accurately identify them because they exist outside our awareness while rapidly shifting and changing. At Falsely-Accused.net we feel that consequently, attempting to identify unconscious experiences is forever limited to conjecture and speculation.

Cross‑Examining the Rorschach and Psychoanalytic Theory

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS

IF YOU HAVE NOT YET DONE SO]

1.       The journal Psychological Assessment is a generally recognized and accepted, peer‑reviewed journal in your field ‑‑ Correct?

2.       And an 1999 article by Stricker and Gold published in Psychological Assessment ‑ titled: “The Rorschach: Toward a Nomothetically Based, Idiographically Applicable Configurational Model” ‑ might be relevant to your opinions in this case ‑‑ Correct?

3.       Please consider Stricker and Gold’s comments about the Rorschach:

- [ read ] -

“On the broad spectrum of sensitivity, the Rorschach is uniquely suited for understanding the underlying organization of the personality and for generating hypotheses about unconscious contents of the mind.”

         Now my question: If the Rorschach is “uniquely suited” for “enerating hypotheses about the unconscious contents of the mind,” then using the Rorschach embraces one or more variations of psychoanalytic or Freudian theory ‑‑ Correct?

4.       Our unconscious experiences exist outside of our awareness ‑‑ Correct?

5.       And our unconscious experiences can rapidly shift and change ‑‑ Correct?

6.       Therefore, we can never accurately identify experiences that are both outside our awareness, and which can rapidly shift and change ‑‑ Correct?

7.       Consequently, it becomes impossible to verify the accuracy with which the Rorschach assesses unconscious levels of experience ‑‑ Correct?

8.       Attempting to identify experiences outside our awareness, that rapidly shift and change, invites conjecture and speculation ‑‑ Correct?

9.       Psychologist A and psychologist B can therefore arrive at very different interpretations of the same person’s unconscious experiences ‑‑ Correct?

10.     And when psychologist A and psychologist B arrive at their different interpretations of someone’s unconscious experiences, we do not know who is right ‑‑ Correct?

11.     And when psychologist A and psychologist B arrive at their different interpretations of someone’s unconscious experiences, they can both be mistaken ‑‑ Correct?

12.     And given mistaken interpretations of someone’s unconscious experiences, those errors threaten to misinform and mislead this proceeding ‑‑ Correct?

[The cross‑examining attorney may also want to follow‑up these questions with the questions addressing psychoanalytic theory found in Chapter Four]

 

[1].       Meissner, W. (1985). History of Psychiatry in Kaplan, H. and Sadock, B. (Eds.) Comprehensive Textbook of Psychiatry IV, New York: Williams & Wilkins. page 358-359. See also: Freud, S. (1905). Three essays on the theory of sexuality. The Standard Edition of the Complete Psychological Works of Sigmund Freud. Vol. 7  London: Hogarth.; Freud, S. (1910) Five Lectures on psycho-analysis. The Standard Edition of the Complete Psychological Works of Sigmund Freud. Vol. 11 London: Hogarth.; Strachey, James (Ed.) In collaboration with Anna Freud (1953 – 1964). The Standard Edition of the Complete Psychological Works of Sigmund Freud. 24 vols. London: Hogarth.

Id. and see also: Freud, S. (1916-17). Introductory lectures on psycho-analysis. The Standard Edition of the Complete Psychological Works of Sigmund Freud. Vols. 15-16 London: Hogarth; and Campbell, R.J. (1981) Psychiatric Dictionary, 5th Ed. New York: Oxford University Press :

“Freud says that when the little girl realizes she has no penis, she reacts either by hoping that some day she will have one or by denying that she does not have one……Penis-envy is responsible for ‘a loosening of the girl’s relation with her mother as a love-object,’ because the girl blames her mother for the alleged loss.” page 220.

  1. and see also: Campbell, R.J. (1981) Psychiatric Dictionary, 5th Ed. New York: Oxford University Press :

“In psychoanalytic theory, the earliest concentration of libido is in the oral zone; with further development, most of it shifts to the anal region……” page 32.

[2].       Super, D.E. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 158

[3].       Rabin, A. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 159

[4].       Stricker, G. & Gold, J.R. (1999). The Rorschach: Toward a nomothetically based, idiographically applicable configurational model. Psychological Assessment, 11, 240-250, page 243..

[5].       Id. at page 244.

 
 
The Rorschach Technique
Posted On November 5th, 2013  By admin  

The Rorschach technique, or what is sometimes called the “Inkblot test,” is the most frequently used projective instrument.  Subjects taking the Rorschach are asked to look at its inkblots and describe what they see.  Psychologists who use the Rorschach assume that the ambiguous features of its inkblots result in subjects projecting, or revealing, various characteristics of their personalities via their responses. At Falsely-Accused.net we believe however, this is another example of theory prevailing despite the unavailability of supportive data.

As long ago as 1949, reviews of the Rorschach were quite critical.  In his review appearing in the Third Mental Measurements Yearbook, Professor J.R. Wittenborn observed:

“What passed for research in this [Rorschach] field is usually naively conceived, inadequately controlled, and only rarely subjected to the usual standards of experimental rigor with respect to statistical tests and freedom from ambiguity.  Despite these limitations, the test flourishes, its admirers multiply, and its claims proliferate.”

 

The originator of the Rorschach technique, Herman Rorschach, developed his own scoring and interpretive procedures in the 1920’s.  Subsequently, additional scoring and interpretive systems were developed by Beck in 1937, Hertz in 1938, Klopfer and Kelly in 1942, Piotrowski in 1947, Rappaport in 1945 and 1956, and Exner in 1974.  Given these different administrative, scoring, and interpretive systems, we at Falsely-Accused.net feel it is mistaken to speak of the Rorschach as a homogenous instrument.  These administrative, scoring, and interpretive variations create different Rorschachs that sometines only faintly resemble each other.  Sargent commented on these variations in 1953.

“There are several different procedures for the administration of the Rorschach method, all of which have enthusiastic adherents and severe critics … The variety of scoring systems are at first bewildering …”

 

Psychologists who use the Rorschach frequently claim that Exner’s system has standardized its administration and scoring.  Authoritative opinion, however, indicates otherwise.  Since 1940 the many volumes of Oscar Buros and the Buros Institute, originally published as the Mental Measurements Yearbooks and then Tests in Print, have served as the authoritative texts regarding psychological testing.  In its overview of the Rorschach, the 1999 edition of tests in print states: “Many variations in scoring and interpretation are in use with no single method generally accepted.”  For example, a 1997 article examining the Rorschach and dissociative responses stated: “Administration involved the traditional free association and inquiry phases following standard Beck directions (Beck et al., 1961).”  We at Falsely-Accused.net feel that quite obviously, claims regarding the general acceptance of the Exner method are not supported by authoritative opinion.  As was true in 1953, there continues to be no generally accepted scoring and interpretation method for the Rorschach.

Despite the current popularity of Exner’s system, practicing psychologists often combine it with other systems.  Compared to the Exner method of administration, the Rappaport method obtains different kinds of responses In other words, the type of administrative procedures influence Rorschach responses.  Consequently, two different administrative methods could obtain different Rorschach responses ‑ with varying interpretations ‑ from the same person.  Eron commented on these administrative problems in 1965:

AThe quantity and quality of responses elicited by the inkblots are affected to a measurable degree by the individual characteristics and overt behavior of the administrator.”

 

In a 1998 article, Lerner emphasized the significance of the Apatient‑examiner relationship” and its influence on Rorschach responses [1].  In clarifying his position, Lerner cited a portion of Schactel’s 1966 book, Experiential Foundations of Rorschach’s Test:

ATo understand a Rorschach record it is essential to be aware of its interpersonal aspects and implications.  This means primarily that we must never forget that the Rorschach performance and the experience of the testee in the Rorschach situation are an interpersonal performance and interpersonal experiences.”[2]

 

In 1998, therefore, Lerner recognizes and emphasizes what Schactel addressed in 1966 ‑ The Rorschach procedure involves examiners and examinees both influencing ‑ and being influenced by ‑ each other. We at Falsely-Accused.net feel that quite clearly, then, the Rorschach responses of any examinee could be different if reacting to another examiner.  Consider, for example, the report of a psychologist who described the Rorschach in the following manner:

Psych:         At one point considered highly subjective and thus not considered valid or reliable, this is no longer the case as this test is now and has been for some time highly structured with strong psychometric properties and good predictive validity.

Authoritative opinion and the relevant data, however, demonstrate that this psychologist was engaged in wishful thinking.

 

 Cross Examining Multiple Scoring and Interpretive Systems

1.       Dr. X, you used the Rorschach technique in this case ‑‑ Correct?

2.       And Hermann Rorschach developed the first Rorschach technique in the 1920s ‑‑ Correct?

3.       And the “inkblot” cards Hermann Rorschach developed are the very same cards you used in this case ‑‑ Correct?

4.       And Hermann Rorschach developed his own system of scoring and interpreting the Rorschach technique ‑‑ Correct?

5.       But you did not rely on Hermann Rorschach’s original scoring and interpretive system in this case ‑‑ Correct?

6.       And Beck developed his system for scoring and interpreting the Rorschach in 1937 ‑‑ Correct?

7.       And some psychologists still use portions of Beck’s scoring or interpretive system today ‑‑ Correct?

8.       And Hertz developed his system for scoring and interpreting the Rorschach technique in 1938 ‑‑ Correct?

9.       And some psychologists still use portions of Hertz’s scoring or interpretive system today ‑‑ Correct?

10.     And Klopfer and Kelly developed their system for scoring and interpreting the Rorschach technique in 1942 ‑‑ Correct?

11.     And some psychologists still use portions of Klopfer and Kelly’s scoring or interpretive system today ‑‑ Correct?

12.     And Piotrowski developed his system for scoring and interpreting the Rorschach in 1947 ‑‑ Correct?

13.     And some psychologists still use portions of Piotrowski’s scoring or interpretive system today ‑‑ Correct?

14.     And Rappaport developed his system for scoring and interpreting the Rorschach in 1956 ‑‑ Correct?

15.     And some psychologists still use portions of Rapport’s scoring or interpretive system today ‑‑ Correct?

16.     Between 1920 and 1956, there were at least six different scoring and interpretive systems developed for the Rorschach technique ‑‑ Correct?

17.     So, between 1920 and 1956, there was no single scoring and interpretive system for the Rorschach that was generally accepted ‑‑ Correct?

18.     Exner’s 1974 system claims general recognition for scoring and interpreting the Rorschach ‑‑ Correct?

19.     But many psychologists will combine portions of Exner’s system with those of Beck, or Hertz, or Kloper and Kelly, or Piotrowski, or Rappaport ‑‑ Correct?

20.     And these idiosyncratic ways of combining Exner’s system with other systems continue to indicate that there is no generally accepted method for administering, scoring, and interpreting the Rorschach ‑‑ Correct?

21.     The reviews of Oscar Buros and the Buros Institute, originally published as the Mental Measurement Yearbooks and Tests in Print, are generally recognized and accepted as authoritative texts regarding psychological testing ‑‑ Correct?

22.     Please consider the following comments from the 1999 issue of Tests in Print V regarding the Rorschach:

- [ read ] -

          “Many variations in scoring and interpretation are in use with no single method generally accepted.”

          Now my question: If there are many variations in scoring and interpretation available for the Rorschach, then a psychologist other than you could have administered and scored the Rorschach differently than you did ‑‑ Correct?

23.     And if another psychologist scored and administered the Rorschach differently than you, that psychologist could also have interpreted the Rorschach responses differently than you ‑‑ Correct?

24.     And Rorschach responses can be influenced by who administers this procedure ‑‑ Correct?

25.     The Journal of Personality Assessment is a generally recognized and accepted, peer‑reviewed journal in your field ‑‑Correct?

26.     And a 1998 article by Lerner published in the Journal of Personality Assessment ‑ titled “Schactel and Experiential Rorschach Assessment” ‑ might be relevant to your opinions in this case ‑‑ Correct?

27.     In his 1998 article, Lerner cites Schactel’s 1966 book ‑ Experiential foundations of Rorschach’s test, and writes:

- [ read ] -

          “To understand a Rorschach record it is essential to be aware of its interpersonal aspects and implications.  This means primarily that we must never forget that the Rorschach performance and the experience of the testee in the Rorschach situation are an interpersonal performance and interpersonal experiences.”

          Now my question: In understanding a Rorschach record, if it is essential to be aware of its interpersonal aspects and implications, that means anyone’s Rorschach responses can be influenced by examiner‑examinee interactions ‑‑ Correct?

28.     In other words, a Rorschach assessment amounts to a situation where examinee and examiner both influence ‑ and are influenced by ‑ each other ‑‑ Correct?

29.     Therefore, the Rorschach responses of any examinee can be different if reacting to another examiner ‑‑ Correct?

30.     Consequently, another psychologist could have interpreted the Rorschach in this case differently than you as a result of obtaining different responses ‑‑ Correct?

31.     You have not published anything in a peer‑reviewed journal necessitating that we reconsider Lerner’s position in this regard ‑‑ Correct?

32.     You cannot cite any article published in a peer‑reviewed journal necessitating that we reconsider Lerner’s position in this regard ‑‑ Correct?

 




[1].       Lerner, P.M. (1998). Schactel and experiential Rorschach assessment. Journal of Personality Assessment, 71, 182‑188.

[2].       Schactel, E. (1966). Experiential foundations of Rorschach’s test. New York: Basic Books (p. 269).

 
 
Psychological Testing in I.Q. Cases
Posted On October 31st, 2013  By admin  

          We at Falsely-Accused.net realize that there are a number of high quality analyses available from courts around the country on questions of psychological tests for I.Q. and competency.  Recently, Federal District Judge David Ezra in Honolulu dealt with an Assistant U.S. Attorney trying to block Dr. Myla Young, a neuropsychologist, from testifying during the guilt phase of felony murder case. Dr. Young had administered a variety of neuropsychological and psychological tests and assessment tools to assess the defendant’s neural functioning. These procedures tested, defendant’s intellectual functioning, his motor, attention, memory and learning skills, and his “executive functioning”[1]. Included in Dr. Young’s battery were the Wechsler Adult Intelligence Scale (“WAIS-III”) and the Test of Non-Verbal Intelligence (“TONI-3”). In a detailed Daubert hearing, Judge Ezra took proofs about the methodology and validity and reliability and found, “Dr. Young used generally reliable methodology. While it may be true that additional testing may have further uncovered the existence or extent of Defendant’s brain damage, there was nothing inherently unreliable about the tests Dr. Young did conduct. Dr. Young’s methods and testing procedures appear to be generally accepted and found to be useful by other physicians in the field.”[2] 

In People v. Superior Court [3] the California Supreme Court stood one hundred years of intelligence testing on its head, just to avoid a death penalty. While their attitude toward the death penalty may be laudatory, the California Court pretended that a defendant, who had killed in cold blood, was retarded. In order to sustain this fantasy about intelligence testing and what actually constitutes retardation, the Court overlooked the meager record built by State prosecutors on the differential diagnosis of retardation with full scale, verbal and performance I.Q.[4]   

Another recent I.Q./competency analysis can be found in U.S. v. Lopez-Hodgson.[5] In this case, the defendant was convicted for reentry of a deported alien after a conviction for an aggravated felony, and he appealed.  The Federal District appointed a forensic psychologist to evaluate the defendant and numerous additional psychologists evaluated the defendant as well. The analysis of the Tenth Federal Circuit’s panel concerning the admissibility of the various tests and evaluations, is instructive with respect to legal issues involved.[6]

          Atkins v. Virginia,[7] has achieved a great deal of notoriety as the case that abolished the death penalty for the mentally retarded. We indulge in the gallows humor so important to litigators and call this the “they really wanted to kill this guy” case. In Atkins, after the United States Supreme Court reversed the defendant’s death sentence[8] and stated that the executions of mentally retarded individuals constituted cruel and unusual punishment, a jury determined that Mr. Atkins failed to prove that he was mentally retarded and reinstated his death sentence. When the defendant appealed, the Virginia Supreme Court held inter alia that the testifying clinical psychologist was not skilled in the administration of measures of adaptive behavior, and thus he was not qualified to testify and express an expert opinion as to whether the defendant was mentally retarded. Further, the trial court’s disclosure to the jury that another jury had already decided that defendant should receive the death penalty prejudiced the defendant’s right to a fair trial on the issue of his mental retardation.[9]

          In his appeal to the Virginia Supreme Court, Atkins complained of 38 errors in the judgment of the circuit court. One of these errors involved the circuit court appointment of clinical psychologist Stanton Samenow, Ph.D. to evaluate Atkins “concerning the existence or absence of [Atkins’] mental retardation.”[10] During voir dire, Dr. Samenow testified that he administered a Weschler Adult Intelligence Scale (WAIS-III) to Atkins two days after a clinical psychologist, who testified for Atkins, had given Atkins the test.   After Atkins’ clinical psychologist administered the WAIS-III, it seems he advised Atkins not to take the test again. When Dr. Samenow arrived at the prison where Atkins was incarcerated, Atkins informed Dr. Samenow that he had already taken the WAIS-III and did not feel he should take it again. Dr. Samenow was not previously aware of the recent administration of the WAIS-III and elected to repeat it because he did not have another test for intellectual functioning with him.[11] It seems that Dr. Samenow also testified about the Adaptive Behavior Assessment System-II (ABAS-II), and during voir dire, incorrectly named a measure of adaptive behavior the “ABS”.[12]  After this exchange, Atkins moved to disallow Dr. Samenow’s testimony on the grounds that Dr. Samenow had not conducted the evaluation of Atkins in accordance with Virginia law and (2) because he had not assessed Atkins’ intellectual functioning in accordance with accepted professional practice and had not administered any standardized measure for assessing adaptive behavior.[13] The Virginia Supreme Court agreed and reversed again.[14]

          In Christopher R. v. Commissioner of Mental Retardation,[15] the Supreme Court of Connecticut took up the challenge of a minor applicant, through his father, concerning the decision of the Commissioner of Mental Retardation that this young man was not eligible for services from the Department. The Supreme Court held inter alia that the Commissioner could consider the results of more than one general intelligence test when determining whether an applicant was mentally retarded. When faced with conflicting test results, the Commissioner could consider other evidence, including other intelligence test scores, partial components of tests taken by an applicant, and the effect that an applicant’s mental disorders may have had on his test scores.[16] We at Falsely-Accused.net note that the case involved numerous tests including the Weschler Intelligence Scale for Children-III Edition, the Kaufman Assessment Battery for Children, and the Peabody Picture Vocabulary Test.

          The record also reflected several psychological, social and medical evaluations over the years that had varyingly diagnosed the minor applicant as learning disabled, having pervasive developmental disorder and having obsessive compulsive disorder.[17]

          In 1984, researcher James Flynn discovered an intriguing pattern in IQ testing: scores increased steadily over time, with testers of a later generation significantly outperforming their generational predecessors on the same test.[18] [19] This phenomenon, now known as the Flynn effect, occurs all over the world.[20] While there are many theories about why the Flynn effect occurs, including improved educational systems, increased childhood nutrition, a trend towards smaller families, and greater environmental complexity, none has gained widestream acceptance.[21] It is important to note that the Flynn effect is becoming an important consideration in death penalty cases dealing with mental retardation under Atkins v. Virginia.

          One such case is Wiley v. Epps,[22] in which the government appealed the trial court’s finding that defendant Wiley was ineligible for the death penalty based on his mental retardation. Wiley had been given several IQ tests over the years, with steadily increasing scores. A psychologist testified that despite the changes in score, these tests were actually consistent due to the Flynn effect.[23] The Fifth Circuit Court of Appeals upheld the lower court’s finding of Wiley’s mental retardation.[24]

Another case involving the Flynn effect was U.S. v. Hardy.[25] This case, involving the death sentence issued to a man with borderline mental retardation. Hardy gives a thorough explication of the Flynn effect and its ramifications for I.Q. scoring. As it explains,

When a new IQ test is developed, it is standardized through a current population sample, creating a “new” average of 100. Since this new test is normed with a population that has steadily done better (“is smarter”) than the prior generation, the “new” average of 100 is actually “higher” than the old 100. For example, if the old test-normed as it was before the population “got smarter”-were given today, the average score would not be 100, but something closer to perhaps 103. But when the new test is standardized, that “old 103” must be redefined as the “new 100,” because the average score of the test is defined to be 100. What this means in practical terms is that someone who receives a score of 80 on a test that was normed a decade ago could be expected, on average, to score a 77 on a newly normed test-without any actual change in his intelligence.[26]

 

The court underwent an in-depth survey of both psychological opinion regarding the Flynn effect, including peer-reviewed studies, as well as legal opinion, in which it specifically cited Wiley v. Epps, among other cases, in observing a national trend towards acceptance of the Flynn effect.[27] The United States District Court, considering both this information and the defendant’s own IQ test history. In doing so the Court held that the Flynn effect was well-established scientifically and applied it to find that the defendant was in fact mentally retarded and thus ineligible for the death penalty.[28]

In any case in which an I.Q. test relied on to prove intelligence above an cutoff score for mental retardation is more than a few years old, we at Falsely-Accused.net emphasize that a cross-examiner must be prepared to educate the court about the Flynn effect.

          A very interesting I.Q./adaptive behavior assessment process was undertaken in the Federal Court in Texas recently over the insanity plea of a firefighter/fire-starter.  In United States v. Eff[29] Federal District Court Judge Ron Clark dealt with a slew of psychological tests[30] and Klinefelter’s Syndrome.[31]

          It seems that Ryan Eff was employed as a firefighter for the United States Forest Services (USFS). A USFS Special Agent was investigating a series of forest fires which occurred in the Davy Crockett National Forest, and received information that made Eff a suspect in the case. Agents discovered tire tracks at the point of origin of several fires which were similar to the tread on the tires of Eff’s government vehicle. The USFS Special Agent obtained a Global Positioning Satellite (GPS) device and had it installed on Eff’s government vehicle. Utilizing the data from the GPS, the Special Agent was able to place Eff’s vehicle at the point of origin in three subsequent forest fires.[32]

When Eff was charged with maliciously setting forest fires, he gave notice of an insanity defense. He identified two expert witnesses to testify about Klinefelter’s Syndrome, and how that affected his ability to appreciate the nature and quality or wrongfulness of his acts. Analyzing the proposed testimony, the court concluded that the experts’ testimony should be excluded because, as Judge Clark saw it, the witnesses failed to apply the principles and methods of science reliably to the facts of the case.  The court also ruled that any probative value of the testimony was substantially outweighed by its potential to mislead and confuse the jury.[33]  In the Daubert hearing which lead up to the judge’s decision about the expert testimony, the government did not dispute that Eff was diagnosed with Klinefelter’s Syndrome. 

During the Daubert hearing, Eff’s neuropsychologist witness testified to his test results and offered that Eff’s learning and memory skills were within normal limits and his immediate recall reproduction of geometric figures was nearly within the high average range. Eff’s verbal subtests included scores ranging from borderline to average levels. Eff’s math calculation ability was “borderline” and low average scores were present on subtests for verbal abstractions and basic attention.[34] Based upon her examination, one of Eff’s experts concluded that Eff’s current IQ scores were within expectation for educational and occupational level. She stated that the results “indicate that the patient lacks the basic brain ‘hardware’ to exert consistent, reasoned control over his behavior”.[35] She went on to testify that Eff approached situations at the developmental level of a child.   She testified that Eff’s ability to appreciate the wrongfulness of his actions was at the level of a 10- to 12-year-old.[36]

          Eff also brought a neurodevelopmentalist. To conduct her evaluation, this expert reviewed the findings of the neuropsychologist and determined that Eff suffered from “poor executive function and diminished frontal lobe capacity, which is highly associated with his neurogenetic disorder of XXY or Klinefelter Syndrome…His late diagnosis and lack of treatment resulted in diminished cognitive capacity and limited control over his own behavior, based on Dr. Boone-Bauer.” At the Daubert hearing, the neurodevelopmentalist opined that Eff was operating at an 8-year-old level.   She stated that one might expect a child of that age to do something wrong and correct it in order to impress a parent.[37]

          Judge Clark explained that in his analysis, he had to consider many different “factors”. Referring to the Daubert trilogy, Judge Clark informed that the first factor was whether the theory or technique in question has been, or can be, tested.[38] Judge Clark determined that the neuropsychologist administered a battery of standardized psychological tests, including the 13 separate subtests that comprise the Weschler Adult Intelligence Scale-III (WAIS-III) and the 18 separate subtests that comprise the Weschler Memory Scale. The court noted that in the course of their evaluation, the neuropsychologist and neurodevelopmentalist interviewed Eff and reviewed Eff’s statements. The tests used, and the interview, were part of a standard psychological evaluation, and can be repeated. On this factor, the court determined that, “[w]hile it is not so clear that the conclusions of insanity or any link between Klinefelter Syndrome and insanity can be, or has been, tested, this factor weighs in favor of admitting the experts’ testimony.”[39]  Referring again to the Daubert trilogy, the court explained that another factor it must look to is the potential rate of error, or the confidence level. In so doing, the court noted that a study used by the neuropsychologist stated that fewer than ten cases of Klinefelter Syndrome associated with fire-setting behavior had been reported.[40] This, reasoned Clark, indicated strong possibilities for error. 

          We at Falsely-Accused.net note that Judge Clark went on with his “factor” analysis noting that the court also looks at whether the theory or technique has been subjected to peer review.[41] While the standard tests used by the neuropsychologist had been widely administered, Judge Clark  explained that Eff presented no evidence that there has been peer review of a link between Klinefelter Syndrome with “insanity.”[42] Continuing, the court noted that another factor in this analytical scheme is whether the testimony offered by the two experts is based on research the experts have conducted independent of the litigation.  There didn’t seem to be much of that in this case.[43] Clark’s analysis continued, with the court asking whether the expert had adequately accounted for obvious alternative explanations.[44] Clark also looked to the application of the principles and methodology of science by the experts asking whether the expert has employed the same care in reaching the litigation-related opinions as the expert employs in performing his or her regular professional work.[45] Noting the dearth of research actually connecting Klinefelter Syndrome with “insanity” as defined by Congress,[46] Clark wondered if there was “too great an analytical gap” between the data and the opinion.[47] Finally, Judge Clark turned to a Rule 403 balancing test, which he determined was inextricably bound to the balancing of Rule 702 factors. With respect to Federal Rule of Evidence 403, Judge Clark explained that in this case, an argument could be made that the testimony was, “at least somewhat reliable, so that it should be admitted and a jury could determine its weight. But a court must still consider, whether the probative value of testimony is substantially outweighed by the danger of confusion of the issues or its tendency to mislead the jury.”[48]

          After this veritable template analysis, Judge Clark concluded that, “[t]he court finds that even if the experts’ testimony were admitted, it would not allow a reasonable jury to conclude that Eff was unable to appreciate the nature and quality or wrongfulness of his actions.”[49]

          We at Falsely-Accused.net poke so much fun at Texas State trial judges in this treatise that we must also respectfully suggest that all Texas litigators, and the rest of us for that matter, would be wise to print out this brief list of Judge Clark’s decisional “factors” and brief it for our next State court judge.

Judge Clark’s decisional “factors”

 

¯ whether the theory or technique in question has been, or can be,     tested;

¯ the  potential rate of error, or the confidence level;

¯ whether the theory or technique has been subjected to peer review;

¯ whether the testimony offered by the two experts is based on research the experts have conducted independent of the litigation;

¯ whether the expert had adequately accounted for obvious alternative explanations;

¯ whether the expert has employed the same care in reaching the litigation-related opinions as the expert employs in performing his or her regular professional work;

¯ whether there is “too great an analytical gap” between the data and the opinion.

 

In State v. Griffin,[50] the Connecticut courts spent valuable resources on a teenager who was arguably not worth the value of his composite parts. 

The defendant was 14 year old Cyrus Griffin who, along with some teenage friends, robbed 18 year old Tyshan Allbrooks at gunpoint, taking a necklace made of gold. Tyshan reported this to the police and a short time later, Griffin saw her walking nearby. Cyrus was aware that Tyshan had reported the robbery to the police and as he got out of his friend’s car, he was heard to say “snitches get stitches.” Cyrus then chased Tyshan down and shot her twice in the chest and four times in the back with his pistol. [51]

          He was convicted following a jury trial of manslaughter in the first degree with a firearm and carrying a pistol without a permit. He appealed, arguing inter alia that he was too dumb to understand the Miranda warnings given to him by the police. In support of his mental deficiency defense, counsel called experts and tried to side-step the state’s Daubert test.[52]

          In this matter, prior to a hearing on defendant’s wish to suppress his confession, the State filed a motion in limine to exclude “any and all opinion testimony of any expert witness regarding the waiver of Miranda rights predicated upon an evaluative protocol created by Thomas Grisso or related to such protocol.” The state argued that such evidence was based on “scientific, technical and/or specialized knowledge which is unreliable.” [53]

          At the evidentiary hearing on the defendant’s motion to suppress and the State’s motion in limine, the defendant’s attorneys brought the testimony of Madelon Baranoski, a clinical psychologist employed by the Connecticut Mental Health Center at Yale University. Baranoski was an associate clinical professor at Yale and the associate director of the Yale court clinic, which is affiliated with the law and psychiatry division of Yale’s department of psychiatry. Baranoski testified that as part of her professional duties, she evaluates approximately 200 separate defendants in an average year to evaluate whether they are competent to stand trial.

Baranoski testified that evaluating an individual’s competency in regard to a particular stage of trial proceedings involves identifying what tasks are involved at such stage of the proceedings, and determining whether the individual possesses the competency to understand those issues and tasks. She testified that she evaluated the defendant to determine whether he possessed the competency to understand his Miranda rights. Baranoski explained that her evaluation involved several methods: A clinical interview, IQ testing, personality testing, testing for reading and spelling proficiency, testing for arithmetic ability and general achievement testing. In addition to testing the defendant to determine his “overall competency”, Baranoski also tested the defendant with a set of questions that had to do with the specific tasks involved in understanding the Miranda warning, and making a choice to waive the rights.

          Baranoski explained that those questions were part of a protocol developed by Thomas Grisso, a forensic psychologist who has devoted his professional efforts to issues regarding “juvenile competency” and who works with a research group that studies issues of competency. She also testified that the Grisso testing “instrument”, which is part of the study protocol, consists of four parts that are scored by the test administrator.[54]

Baranoski explained the defendant’s results on each aspect of the Grisso test. She reported that he scored in the bottom 20 percent of juvenile test takers. She opined, on the basis of the defendant’s results on the Grisso test, as well as on the basis of the defendant’s results on the other evaluative measures, that the defendant “did not understand the right to remain silent as it applied to incriminating information, and he also did not understand the role of an attorney during the interrogation process.”[55]

The appellate court carefully reviewed the trial court’s findings and reiterated the trial court’s conclusions that the defendant had “failed to prove that the Grisso test has sufficient scientific validity in order for the court to accept it as reliable evidence.”[56] The court found that, “since the Grisso test was formulated in 1981…it has not been the subject of an adequate amount of testing.”[57] The court also found that the test had not been subject to adequate peer review and publication, noting that the defendant attempted to demonstrate stringent peer review and publication by citing publications written by Grisso himself. The court labeled Grisso’s efforts in this regard as “self-promotion”, and also found that the defendant had not demonstrated that the Grisso test “has been generally accepted in the relevant scientific community.”[58]

          Reviewing the testimony of Baranoski in its entirety, the appellate panel was not impressed with the limited data concerning peer review of the Grisso test. Further, the panel was less than impressed with her citation to the scientific community, “…[e]ssentially, Baranoski did not cite any evidence, apart from her beliefs, that supported a finding that the test had gained widespread acceptance in the relevant scientific community.”[59]

          We at Falsely-Accused.net note that the Connecticut Supreme Court took up Griffin,[60] and echoed their appellate court that the protocol utilized by Baranoski to determine the validity of the waiver of Miranda rights had been subject to preliminary determination of scientific reliability. The Court found that the Grisso test and protocol used by Baranoski to determine the defendant’s competency to waive his Miranda rights was not scientifically reliable.[61]

In State v. Kelley,[62] the defendant was convicted of attempted rape and appealed. The Utah Supreme Court undertook an analysis of the testimony of a person qualified as a specialist in mental retardation. The opinion is noteworthy for the analysis of the experience base of the proposed expert. The opinion is also noteworthy for its cogent analysis of “mental age”, and a disabled woman’s ability to knowingly consent to sexual interaction.

          Three days after it released its Kelly opinion, the Utah Supreme Court released State v. Adams.[63] In Adams, the Court took up a challenge to the admissibility of expert testimony on a mentally disabled minor’s ability to be coached and to fabricate a story of sexual abuse.[64] Adams is noteworthy for the analysis of the manner in which the State Supreme Court sidestepped its own test for novel scientific testimony.[65]

 




[1] U.S. v. Williams, 2009 WL 424583 (D. Hawaii) 78 Fed. Evid. Serv. 1024 – page 4.

 

[2]  Id. 2009 WL 424583 (D. Hawaii) 78 Fed. Evid. Serv. 1024 – page 5.

 

[3] 40 Cal.4th 999, 155 P.3d 259 (Sup. Ct. of California 2007).

 

[4]  40 Cal.4th 999, 155 P.3d 259, 261.  The murderer, one Jorge Junior Vidal had measured Wechsler performance I.Q.’s of  96; 99; 109; 119 and 126 – thoroughly normal.  The Supreme Court glossed over a skimpy record about the defendant’s verbal problems and his command of English in its quest toward “political correctness”.

 

[5] U.S. v. Lopez-Hodgson, 333 Fed.Appx.347; 2009 WL 1483131 (10th Cir. New Mexico).

[6] Unfortunately, U.S. v. Lopez-Hodgson, 333 Fed.Appx.347; 2009 WL 1483131 (10th Cir. New Mexico) has been published with a Not selected for publication in the Federal Reporter memorandum.  Its value therefore, will be only for the construction of argument.

 

[7] Most recently: 272 Va. 144, 631 S.E.2d 93 (Supreme Court of Virginia, 2006).

 

[8] Atkins v Virginia, 536 U.S. 304, 122 S.Ct. 2242 (2002) “Atkins III”.

 

[9] The trial judge remarked: “…If he is mentally retarded, and that is the fact issue that you will determine, another jury has already made the determination as to what would happen to him.   If he is mentally retarded, by law, his sentence would now be commuted to life in prison.   If you find that he is not mentally retarded then another jury has determined what would happen to him; that is, that he would be executed.” 631 S.E.2d at 99.

[10] 631 S.E.2d at 96.

[11] Id. 631 S.E.2d at 96.

 

[12] Id.

 

[13] Id. 631 S.E.2d at 96-97.

 

[14] Id. 631 S.E.2d at 102.

 

[15] 277 Conn. 594, 893 A.2d 431 (Supreme Court of Connecticut, 2006).

 

[16] Id.

 

[17] Id. 893 A.2d at 444.

 

[18] Flynn, James R. “What is Intelligence?: Beyond the Flynn Effect”. 2007 Cambridge University Press, NY, NY.

 

[19] Jelte M. Wicherts, Conor V. Dolan, David J. Hessen, Paul Oosterveld,

G. Caroline M. van Baal, Dorret I. Boomsma, Mark M. Span, “Are intelligence tests measurement invariant over time? Investigating the nature of the Flynn effect”, Intelligence 32 (2004) 509–537, 510

[20] Id.

[21] Id.

[22] 625 F.3d 199 (5th Cir. 2010).

 

[23] Id. at 202-203.

 

[24] Id. at 214.

 

[25] — F.Supp.2d —-, 2010 WL 4909550 (E.D.La. 2010).

[26] Id. at *7.

 

[27] Id. at *7-*10.

 

[28] Id. at *13.

[29] 461 F.Supp.2d 529 (E.D.Texas, 2006).

 

[30] The Wechsler Adult Intelligence Scale-III (WAIS-III), Rey Auditory Verbal Learning Test, Rey-Osterereith Complex Figure, Verbal Fluency, Wechsler Memory Scale-Revised, Trailmaking, Boston Naming Test, Comalli Stroop Test, Ruff Figural Fluency Test, Wisconsin Card Sorting Test, Wide Range Achievement Test-4, and Finger Tapping, just to name a few. 461 F.Supp.2d at 531.

 

[31] Klinefelter’s syndrome is a term used to describe males who have an extra X chromosome in most of their cells.  Instead of having the usual XY chromosome pattern that most males have, these men have an XXY pattern. Klinefelter syndrome is named after Dr. Henry Klinefelter, who first described a group of symptoms found in some men with the extra X chromosome.  Even though all men with Klinefelter syndrome have the extra X chromosome, not every XXY male has all of those symptoms. Because not every male with an XXY pattern has all the symptoms of Klinefelter syndrome, it is common to use the term XXY male to describe these men, or XXY condition to describe the symptoms.  Scientists believe the XXY condition is one of the most common chromosome abnormalities in humans.  About one of every 500 males has an extra X chromosome, but many don’t have any symptoms.

http://www.nichd.nih.gov/health/topics/klinefelter_syndrome.cfm

See, also: http://www.nichd.nih.gov/publications/pubs/klinefelter.cfm

 

[32] 461 F.Supp.2d at 531.

 

[33] Id.  at 530.

 

[34] Id. at 531-532.

 

[35] Id.

 

[36] Id.

 

[37] Id. at 532.

 

[38] Id. at 533 referring to Daubert, 509 U.S. at 593, 113 S.Ct. at 2796.

 

[39] 461 F.Supp.2d at 533.

 

[40] Id. 461 F.Supp.2d at 533.Eytan, A.  et al., (2002) Fire-Setting Behavior Associated with Klinefelter Syndrome. 32(4) International Journal of Psychiatry in Medicine 395.

 

[41] Id. 461 F.Supp.2d at 533.

 

[42] Id.

 

[43] Id. at 533 – 534.

 

[44] Id. at 534.

 

[45] Id.

 

[46] Insanity Defense Reform Act of 1984, 18 U.S.C. §  17 (IDRA).

 

[47] Id. 461 F.Supp.2d at 535.

 

[48] Id.

 

[49] Id. at 536.

[50]  77 Conn. App. 424; 823 A.2d 419 (2003).

 

[51]  823 A.2d 419, 423 – 424.

 

[52]  823 A.2d 419,426.  “…The defendant in his principal brief argued that simply because [his expert] testified regarding a specific and less recognized area of mental competency (competency of a juvenile to waive Miranda rights) does not mean the relevant methodology [she used] and [her] testimony were subject to the ‘gatekeeper’ analysis…’ The defendant further argued that the court ‘only had to use its ‘powers of observation and comparison’ to understand the scoring protocol of the [Grisso test] since it was based on a simple comparison of [the defendant’s] answers to the answers provided by the target group of juveniles from the initial…study.’”  Connecticut is a Daubert State: State v. Porter, 241 Conn. 57, 698 A.2d 739 (1997).

 

[53]  823 A.2d 419,429.

[54]  The first part tests a person’s ability to explain accurately, in his or her own words, what aspects of the Miranda warnings mean.  The second part tests “recognition” of Miranda rights, and the third part tests comprehension of the vocabulary related to the warnings.  Finally, the fourth part, which involves pictures and stories about fictional persons being interrogated, tests a person’s ability to recognize, during an interrogation, the function of the Miranda warnings. 823 A.2d 419, 425.

 

[55]  823 A.2d 419, 425.

 

[56]  823 A.2d 419, 430.

 

[57]  Id.

 

[58]  823 A.2d 419,430.

 

[59]  823 A.2d 419,431.

 

[60] 273 Conn. 266, 869 A.2d 640 (Supreme Court of Connecticut 2005).

 

[61] Id. at 273 Conn. 266, 285. With respect to the “Grisso test”, see also: People v Cole, 807 N.Y.S.2d 166, 170 (2005): “The record supports the court’s conclusions that the tests had not gained sufficient acceptance for reliability and relevance in the scientific community, and the vocabulary used to gauge defendant’s understanding of the Miranda warnings differed substantially from the warnings defendant received.  Further, the expert testified at length at trial, during which defense counsel had substantial opportunity to challenge defendant’s ability to understand and waive the Miranda warnings as given.”

 

[62] State v Kelley, 1 P.3d 546 (Utah, 2000). Another recent explication of I.Q. tests and case decision making can be found in In re Donald W., 17 A.D.3d 728, 793 N.Y.S.2d 217 (New York App 2005).

 

[63] State v Adams, 5 P.3d 642 (Utah, 2000).

 

[64] Id. 5 P.3d 642.

 

[65] State v. Rimmasch, 775 P.2d 388 (Utah 1989).

 

 
 
PSYCHOLOGICAL TESTING IN SEX CASES
Posted On September 27th, 2013  By admin  

          Just as I.Q. testing forms the basis of many cross examiner attacks, so do the various “tests” used in sex cases.  Indeed, we at Falsely-Accused.net maintain that when provided with appropriate direct and cross examination, trial courts are capable of good quality analyses of psychological test data. Even when it gets a bit goofy. As an example, in the “potential” child abuse case of Lasater v. Lasater[1], an examiner elicited this statement from a proposed expert:

…Now, the CAPI, the Child Abuse Potential Inventory, is a 160-question inventory that is often used, I often use it in cases where there is a question concerning abuse. I’ll use it in CPS cases that I evaluate, or child custody cases. It doesn’t confirm or disconfirm whether abuse has occurred or whether a person, you know, truly is an abuser or not an abuser, but it helps me to compare their scores with those of bona fide, adjudicated abusers.[2]

 

Obviously, this was a set up for the cross examiner.  Clearly, the proposed expert was suggesting that something called the Child Abuse Potential Inventory could aid him in determining whether someone might or might not act out against children.[3]  Clearly, the cross examiner should have followed up with questions concerning test development; sample size; normative group size; test – retest reliability; inter-rater reliability; split-half reliability and so on.  Remember, if the proposed expert cannot define these terms, he or she fails the “datum of the discipline”[4] test. If the proposed expert cannot define these terms and describe their applicability and their reliable connection to the data, they fail the “fit” test.[5]

          Two recent federal cases illustrate this point. U.S. v. Birdsbill,[6] and U.S. v. White Horse[7] both involved the trial of Native American men accused of serious sex offenses against Native American children and both involved the proposed use of a psychological assessment program called “Abel Testing.”[8]

Senior Judge Lovell of the Federal District Court in Montana did an exemplary analysis of this psychological test, which was offered by the defense in U.S. v. Birdsbill. Abel testing is the brainchild of Dr. Gene Abel of Abel Screening Inc., of Atlanta Georgia. The assessment procedure is composed of two parts. Part I measures the amount of time that a subject views slides of clothed and partially clothed people, premised on the belief that a correlation exists between sexual response and the length of time spent viewing sexual stimuli. This is called the visual reaction time (“VRT”). It also includes a questionnaire that is designed to detect the subject’s sexual behavior, as well as to ascertain the subject’s willingness to admit to experiencing common social problems (as a kind of “lie scale”).  In Part II of the Abel Assessment, the results from this questionnaire are combined with the viewing time results to create a score. The protocol subjects this score to three predictive equations in order to estimate the likelihood that the subject is sexually interested in children under fourteen and would lie about his interest.[9]

We at Falsely-Accused.net remind the reader that after completion with the test subject, both the questionnaire results and the VRT results are sent to Abel Screening, Inc. After analyzing the data according to his proprietary formula, Dr. Abel then faxes back to the test administrator a summary (bar graphs, etc.) of the test results.[10] It should be noted that in the development of the protocol, Dr. Abel explained that incest-only cases were excluded from his validation studies because, as he and colleagues opined, offenders in incest cases often are motivated by reasons other than sexual interest.[11]

          When faced with the proffer of a defense witness armed with the Abel Assessment, Judge Lovell examined the district court’s analysis in U.S. v. White Horse,[12] and a Massachusetts superior court’s analysis in Ready v. Massachusetts,[13] as well as the underlying scientific material offered by the defense and prosecution. Judge Lovell began his analysis with relevance considerations, correctly noting that according to Federal Rule of Evidence 401, evidence is relevant if it has any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence.

          The court addressed several relevancy problems immediately.  First, the AASI is a psychological instrument to be used for treatment, not diagnostic purposes, and it is not designed to detect whether a person has sexually abused children. ‘There has never been a claim that The Abel Assessment could be used to screen pedophiles from normals’”.[14] The Court went on to explain:

…Second, the AASI tests sexual interest with slides of Caucasian and African-American children. This is a case involving allegations of sexual abuse of a Native American child by a Native American adult. All three victims testifying in this case were Native American children. Additional alleged victims presented to the Court by the government are also Native American children. Defendant’s AASI test results may report accurately the Defendant’s sexual interest in Caucasian and African-American children, but those results are not relevant to the ultimate fact to be decided by the jury– whether a particular Native American adult sexually abused a particular Native American child.[15]

 

Furthermore, the court reasoned that as the value tested by the AASI is sexual interest and the creator of the test had explained that he excluded incest-only cases from his study of the AASI; this defendant’s AASI scores were not relevant. The underlying rationale for this point was one of validity, just as in the “fit” analysis in U.S. v. Whitehorse: the protocol was standardized on Caucasians and Blacks and non-incest only offending.  We at Falsely-Accused.net find that here, as in Whitehorse, the defendants were both Native American and the allegations were exclusively of inter-familial incest.

In fairness to his Daubert and gatekeeping responsibilities, Judge Lovell did not stop there. He went on to provide an analysis of reliability concerns. First, he examined, “[w]hether [the] theory or technique can be tested…”[16] and determined that there was a fundamental problem in any attempt to replicate or verify the validity of the AASI because the formula used is proprietary information.[17] The court noted that Dr. Abel testified that he does not share his formula with anyone outside his company.    Because the test originator has not released his “formula” for verification or validation by other scientists, the court concluded that “…it remains merely an untested and unproven theory.  This alone has prevented true verification and replication of AASI test results.”[18]

          Turning to an analysis as to whether the protocol has been subjected to peer review and publication, the court again looked to Ready and the Massachusetts courts’ examination of four studies of the AASI and found that each of these studies failed to support the measure’s validity.[19] Next, the Court examined the known or potential error rate of the theory or technique and opined that, “…its error rate makes it a highly unreliable instrument, particularly in the hands of a moderately intelligent subject bent upon manipulating the test results.”[20]

Finally, the court examined whether the theory or technique enjoys general acceptance within the relevant scientific community, and remembered that, “a known technique which has been able to attract only minimal support within the community may properly be viewed with skepticism.”[21]

The court proposed that its analysis of this factor made it clear that the relevant scientific community (scientists and clinical professionals dealing with sex offenders) do not generally accept the AASI test as a diagnostic test for pedophilia, although it may have other commonly accepted uses in treatment and corrections.[22] For these reasons, the Court concluded that, “…[t]herefore, a psychological instrument that calculates Defendant’s sexual interest in an irrelevant group of children (Caucasian and African-American children), that ignores completely issues of family dynamics and the Defendant’s own psychosocial history, is not relevant to the ultimate fact to be decided by the jury.[23]

We at Falsely-Accused.net feel This is precisely the kind of analysis the cross examiner should set up in a well thought out examination.

When an appellate panel for the Ninth Federal Circuit got Birdbill,[24] they made a careful point that the district court did not abuse its discretion by excluding the Abel testing. The appellate panel determined that District Judge Lovell properly applied the factors set forth in Daubert, and gave sound reasons for why the test was neither relevant nor reliable under the circumstances. The most telling analytical factors for the appellate panel were that:

(1) [T]he AASI test is a psychological instrument to be used for treatment, not for diagnostic purposes, and it is not designed to assess the tendency of a person to abuse children sexually; 

(2) Dr. Abel did not use a control group and it is unclear how or whether the test ferrets out ‘fakers’; 

(3) the test has not been subject to adequate peer review because Dr. Abel has proprietary rights over the test and has not shared his formula with other scientists (and those who have tried to assess the test’s validity have come up with dubious and inconsistent results); 

(4) the potential error rate ‘varies from poor…to appalling,’ which makes it an unreliable instrument;  and

(5) the relevant scientific community does not generally accept the AASI test as a diagnostic test for pedophilia.[25]

          Another, much more recent send up of the Abel test can be found in Figueroa v. State[26].  In this Texas case from 2009, Alejandro Figueroa underwent a psychological evaluation prior to his trial for sexual misconduct. Unfortunately for Alejandro, clinical social worker Mark Steege administered the Abel Assessment for Sexual Interest. One wonders what a clinical social worker is doing administering a psychological test, but that’s a question for the Texas State Board of Social Worker Examiners. The problem for Mr. Figueroa and his attorney was that Texas is a Daubert state, and Mr. Steege could not demonstrate validity or reliability with the measure.

We at Falsely-Accused.net often see in sex cases, less than well qualified “experts” trying to slip in, as justification for their opinions, goofy ideas like the Child Sexual Abuse Accommodation Syndrome as a “diagnostic” device. In State v. Marrington,[27] the Oregon Supreme Court found a prosecution witness trying to slip into an Oregon trial what state witnesses routinely get away with in California. 

          In Marrington, Anne Shouse, a junior level psychologist at “Family and Friends” in Oregon, testified to the Child Sexual Abuse Accommodation Syndrome as diagnostic of child sexual abuse. The slick manner in which this is usually done is that in the opening statement, a prosecutor will offer something like:

“‘…[I]t was indicative of sex abuse that [the victim] waited a month and a half before disclosing what had happened.’ In closing argument, the prosecutor stated that, ‘[i]n addition, [the victim] delayed making these allegations until a month and a half after they happened. That, in and of itself, is a characteristic of someone who has been sexually abused as our expert testified to.’…”[28]

 

The prosecutor will also elicit testimony from the expert by asking a question like whether the victim had “…display[ed] any characteristics of a sexually abused child.” Whereupon the expert responds that the victim had “…display[ed] a very prevalent characteristic of abused children in her delayed reporting of the sexual abuse.[29]

Not to be dissuaded from its gatekeeping responsibilities, Oregon’s Supreme Court reversed the defendant’s conviction and explained that, “…it is that assertion–that delayed reporting is a predominant feature of disclosure in otherwise verified cases of child sexual abuse–that frames the evidentiary question…”[30]

 

In a detailed analysis, Justice DeMuniz explained that junior psychologist Shouse prattled on about Roland Summit and his work as a psychiatrist at “Harbor Medical School at UCLA.” [31] Not allowing the party who proposed such preposterous testimony to pull the wool over the Court’s eyes, the Court explained that it appeared the State might have intended the trier of fact to consider Shouse’s testimony about the victim’s delay in reporting the incident as a “characteristic” of a child who had been sexually abused or, in other words, as substantive proof that the child had been abused.[32]

          The Court went on to note that the defendant argued that Shouse’s testimony regarding the victim’s delayed reporting of the alleged abuse was “scientific” evidence and therefore must comply with the standards for admission of “scientific” evidence. The state responded that “evidence about typical reactions of victims of abuse, when offered to explain otherwise puzzling behavior does not require such a foundation.” The Court simply held “… we agree with defendant.”[33]

Justice DeMuniz went on to remind that the Oregon Supreme Court has made it clear that expert testimony concerning matters within the sphere of the behavioral sciences possesses the increased potential to influence triers of fact as scientific assertions, just as expert testimony dealing with the “hard” sciences does.[34] The Justice pointed out that in State v. Milbradt,[35] the court observed that evidence of “how normal children usually react to sexual abuse” or sexual abuse “syndrome” testimony was scientific evidence subject to the Brown foundational requirements.”[36]

          The Court reasoned that in her substantive testimony, Shouse referred to “the characteristics of sexually abused children,” implying that there is a well-defined, empirically verified, set of characteristics that a significant percentage of sexually abused children display. The Court noted that Shouse asserted delayed reporting is “a very prevalent characteristic of abused children.” In discussing delayed reporting, she referred to the “body of literature” initiated by Dr. Roland Summit from Harbor Medical School at UCLA, “his work as a psychiatrist,” and “his study of sexual abuse.”  Shouse’s use of the terms such as “study,” “body of literature,” and “psychiatrist” involve the vocabulary of scientific research.”[37]

          The court went on to point out that an expert like Shouse, who has a background in behavioral sciences and who claims that her knowledge is based on studies, research, and the literature in the field, announces to the factfinder that the basis of her testimony is “scientific,” i.e., is grounded in conclusions that have been reached through application of a scientific method to collected data. Because that is how the factfinder would understand it, said the Court, the trial court had a duty to ensure that such information possessed the necessary indices of scientific validity.[38]

          At Falsely-Accused.net we emphasize that the Court concluded by making it clear that the State, as the proponent of Shouse’s testimony regarding delayed reporting, had the obligation to show that the asserted rule of behavior was scientifically valid under the standards established in their decisions about validity, reliability and admissibility. “The trial court erred in not requiring the state to make that showing.” [39]

 




[1] 809 N.E.2d. 380, 390-391 (Indiana App. 2004) emphasis added.

 

[2] Lasater v. Lasater, 809 N.E.2d. 380, 390-391 (Indiana App. 2004) emphasis added.

 

[3] The cross examiner should have pointed out that a California court disallowed the use of psychological testing to establish a ‘profile’ characteristic of fathers who commit sexual abuse.  According to the expert witness, 85 percent of fathers who commit sexual abuse have personality traits of high pas­sive-dependency and highly guarded and defensive tendencies. The court pointed out that the statistical evidence was far less than persuasive. In re Cheryl H., 200 Cal. Rptr. 789 (Ct. App 1984).

 

[4] Experts must know and base their opinions reliably on the datum of their discipline. Daubert v Merrell Dow Pharmaceuticals Inc., 509 U S 579, 113 S Ct 2786, at 2796,  125 L Ed 2d 469, at 482 (1993)

 

[5] This is the second level of “fit”, and a  properly prepared voir dire will aid the gatekeeper in analyzing it.  The Supreme Court endorsed this approach in Daubert when it located within Rule 702 the obligation of the trial court to determine whether the proffered scientific evidence “properly can be applied to the facts in issue.”  It is important to remember that the Supreme Court adopted terminology used by Judge Becker in United States v. Downing, 753 F.2d 1224, 1242 (3d Cir. 1985), and characterized this consideration as one of “fit”: Daubert  113 S. Ct. 2786, at 2796.  The expert’s testimony must provide “a valid scientific connection to the pertinent inquiry.”

 

[6]  243 F.Supp.2d 1128; 2003 WL 256914 (D. Montana) January 24th, 2003.

 

[7]  316 F. 3d 769 (8th Cir. January 15th, 2003).

 

[8] Also called Abel Assessment for Sexual Interest (AASI).

[9]  United States v Whitehorse, 316 F. 3d 769, 774 – 775.

 

[10]  Id. note 3.

 

[11]  See, e.g.: Abel, Jordan, Hand, Holland, and Phipps, (2001) Classification Models of Child Molesters Utilizing the Abel Assessment for Sexual Interest. 25 Child Abuse and Neglect:  The International Journal  705.

 

[12]  177 F.Supp.2d 973, 975 (D.S.D.2001).

 

[13]  2002 WL 1255800 (Mass.Super.Ct. May 17, 2002).

 

[14]  243 F.Supp.2d 1128, 1131 internal citations omitted. Citing to Ready v. Massachusetts, 2002 WL 1255800 (Mass.Super.Ct. May 17, 2002) Quoting Dr. Gene G. Abel’s Letter to the Editor, Sexual Abuse: A Journal of Research and Treatment .

 

[15]  Id. 243 F.Supp.2d 1128, 1131-1132.

 

[16]  243 F.Supp.2d 1128, 1133 Citing to : Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 589, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993).

 

[17]  243 F.Supp.2d 1128, 1133 Citing to Ready v. Massachusetts, 2002 WL 255800 (Mass.Super.Ct. May 17, 2002) page 14.   See, also: In the Interest of CDK, JLK, and BJK, 64 S.W.3d 679, 683-84 (Texas Court of Appeals 2002) Comparing Dr. Abel’s assessment procedure to the “…magic of young Harry Potter’s mixing potions at the Hogwarts School of Witchcraft and Wizardry.”

 

[18] 243 F.Supp.2d 1128, 1133.

 

[19] “None of these studies has independently verified the validity of the AASI… Essentially, there have been no independent studies conducted for the purpose of verifying the theory underlying the AASI. The AASI test cannot satisfy the Daubert factor of peer review and publication.” 243 F.Supp.2d 1128, 1134 & 1135 citing to  Ready v. Massachusetts, 2002 WL 1255800 (Mass.Super.Ct. May 17, 2002).

 

[20]  243 F.Supp.2d 1128, 1135. “Because of the significant error rate and the fact that there is no way to know whether a subject is falsifying his test results, the AASI is basically useless when used by itself (i.e., without a PPG test and a polygraph test) as a measurement of pedophilia in the criminal justice context.” 243 F.Supp.2d 1128, 1136.

 

[21]  243 F.Supp.2d 1128, 1136. Citing to  Daubert, 509 U.S. at 594, 113 S.Ct. 2786.

 

[22]  243 F.Supp.2d 1128, 1136.

 

[23]  243 F.Supp.2d 1128,1132. See, also: U.S. v Robinson, 94 F.Supp. 2d 751 W.D. La., 2000).

 

8597 Fed.Appx. 721, 2004 WL 958061 (9th Cir.(Mont.))) (2004) not selected for publication.

 

[25] Id. And see: State v. Brannon, 971 So.2d 511, 2007-431 (Louisiana App. 2007) “We further note that Defendant never objected at trial to testimony concerning the results imparted by the Abel Sexual Interest Assessment or to the scientific basis for the test, despite conducting a lengthy cross-examination of Dr. Brennan on that subject.” 971 So.2d 511, 518.

 

[26] Figueroa v. State, 2009 WL 2183460 (Tex.App. – San Antonio) with Do Not Publish Memorandum.

 

[27] 355 Oregon 555; 73 P. 3d 911 (Supreme Court of Oregon 2003).

 

[28]  73 P. 3d 911, 914 Note 1.

 

[29]  73 P. 3d 911, 914 Note 2.

 

[30]  Id.

 

[31]  Id. 73 P. 3d at 913.

 

[32]  Id. 73 P. 3d at 914.

 

[33]  Id. And see: Kurtz v Commonwealth, 172 S.W.3d 409, 2005 WL 2319110 (Supreme Court of Kentucky, 2005); State v Francis, 75 Connecticut App. 1; 815 A.2d 191 (2003). There are numerous cases from around the country describing this goofy construct as Junk Science. See, e.g.: Gersten v. Senkowski, 426 F.3d 588, 611 (2005); Bell v Commonwealth, 245 S.W. 3d 738 (Ky.  2008).

 

[34]  73 P. 3d 913, 914. Citing Jennings v. Baxter Healthcare Corp., 331 Or. 285, 304, 14 P.3d 596 (2000): Scientific evidence subject to requirements of Brown and O’Key even though science involved is not “hard” science.

 

[35]  305 Or. 621, 756 P.2d 620 (1988).

 

[36]  73 P. 3d 911, 914.

 

[37]  Id. 73 P. 3d at 915.

 

[38]  Id. 73 P. 3d at 916.

 

[39]  Id.

 

 
 
Good Gatekeeping With Psychological Tests
Posted On July 8th, 2013  By admin  

We at Falsely-Accused.net find that unfortunately, it seems there is no shortage of behavioral science professionals who will offer junk science testimony when asked. Perhaps this is one of the reasons so many are now subject to sanction by their State Board(s) of Psychology.[1] There are a variety of additional cases wherein the cross examiner has taken on psychological testing and done an exemplary job. For example, in Irving v. State,[2] a defendant was convicted of sexual battery on a child under 12 years of age and lewd assault on a child. In the trial, the defendant vigorously argued a motion in limine to prevent one Dr. James Hord from testifying for the prosecution. Psychologist Hord proposed to testify to several of the hallmarks of junk science in the circumstance of child sexual abuse accusations: (1) that the child “exhibited symptoms that were consistent with a child who has been sexually abused”[3] and (2) that based upon his “personal experience in these kinds of cases,” the child exhibited symptoms consistent with sexual abuse.[4]  The reason for inclusion in this Chapter is that Hord testified that he also used “two projective tests that he testified were ‘generally viewed as being valid…”[5]   

The Florida Appellate Court made it clear that as this was: “(1) expert testimony, and (2) offered to prove the alleged victim of    sexual abuse exhibits symptoms consistent with those of one who has been sexually abused.”[6] It had to pass reliability and validity tests under the State’s law.  They reversed the conviction.

A great example of what a dedicated cross-examiner can accomplish can be found in last year’s U.S. v. Shenghur.[7] This case concerned the competency of a defendant to stand trial, which turned into a “battle of experts”.[8] While the state’s expert was able to produce test-based evidence showing that the defendant was malingering, the defense’s expert, Dr. Janoson, acquitted himself somewhat less admirably. The tests used by Dr. Janoson (such as Human Figure Drawings) were, “of doubtful validity reliability…[and] analysis of the…tests Dr. Janoson administered is a highly subjective enterprise”.[9] As to Dr. Janoson’s use of the Bender-Gestalt test, the Court was also less than impressed, writing that:

Third, the bases of Dr. Janoson’s conclusions were obscured by his methodology. For example, during the hearing, Dr. Janoson attempted to “rescore” Mr. Shenghur’s results on the Bender-Gestalt test using Patricia Lacks’s scoring system. Tr. at 232, 235-36. However, neither the Government nor Dr. Paradis could review his reassessment because he permitted Mr. Shenghur to draw each of his drawings one on top of the other, id. at 265, 340, thus leaving no independent data for review. Even if this were the standard protocol for administering the test, as Dr. Janoson insisted, id. at 341, Dr. Janoson’s conclusions about Mr. Shenghur’s competency based on that test were much less convincing as a result.[10]

The coup de grace came when Dr. Janoson attempted to testify to Shenghur’s “diagnosis” of Williams Syndrome. The cross examiner was able to draw out the basis for this diagnosis: Dr. Janoson had noticed a one-word notation in Shenghur’s medical record (“icyphosis”) and based on one reference to that word in an article describing Williams Syndrome, reported that Shenghur had been diagnosed with that disorder.[11] The Court completely wrote off Dr. Janoson’s “expertise” at this point, writing that, “his willingness in his report to transform a passing reference to a term he knew nothing about into an assertion that Shenghur had been diagnosed with a serious mental illness speaks volumes about the credibility of his conclusions generally”.[12]

In Argen v. New York State Bd. of Law Examiners,[13] the District Court rejected the testimony of psycholo­gists who purported to establish that a plaintiff had a learning disabil­ity that entitled him to accommodation in taking the bar examination. The Court determined that the proposed expert relied on tests not generally accepted in the scientific community to establish a learning disability.[14] Another witness who reported on his interpretation of the tests was unable to explain how he used the test to arrive at his con­clusions.[15]

In Chapple v. Granger,[16] the District Court in Washington State applied a Daubert analysis to a neuro-psychological test battery. The District Judge opined:

The court accepts the test results as they indicate normal scores in most areas.  As to those areas which show below normal scores, there is not sufficient scientific evidence to support the conclusion those scores are indicative of permanent organic brain damage… General acceptance is a factor to be considered; however, it is not dispositive.  The focus is on the ‘methodology’ of the experts, and not the conclusions which they generate.  This does not mean, however, that a conclusion will be admissible merely because some part must be valid.  A credible link must be established between the reasoning and the conclusion.  Once that is accomplished, the inquiry crosses the line from one of admissibility to one of the weight the trier of fact should accord the conclusion.[17]

We at Falsely-Accused.net remind that the well prepared cross examiner’s Daubert challenge to psychological testing may well be successful. This is especially the case when, with a little work, the cross examiner can expose “tests” of severely limited utility.[18] In Foreman v. American Road Lines, Inc.,[19] Federal District Court Judge William Steele struggled with competing claims over Post Traumatic Stress Disorder and psychological tests administered in an evaluation of one of the plaintiffs. It seems that this particular plaintiff was trying to block the defendants’ psychologist from testifying about the Minnesota Multiphasic Personality Inventory (MMPI) and the Millon Clinical Multiaxial Inventory. The plaintiff put forth the opinion that clinical psychologists must give controlling, decisive weight to objective test instruments in forming clinical diagnoses and recommendations, and that the failure to do so strips a psychologist’s opinions of reliability to the point that his testimony flunks a Daubert analysis. While we argue a position similar to this (psychologists diverge from actuarial assessments at their peril), the plaintiff in Foreman argued that adherence to actuarial assessments was the only methodology that could pass a Daubert analysis. As the District Court pointed out, the plaintiff’s argument would reduce the discipline of clinical psychology to nothing more than rote recitation of standardized computer-generated profiles based on the subject’s responses to stylized true-false questions. In this view of clinical psychology said the District Judge, patient interviews would be irrelevant and unnecessary. Subjective testing would likewise be superfluous. Exercise of clinical judgment would be impermissible. Judge Steele went on to point out that the reductio ad absurdum of the plaintiff’s position is that there is no need for clinical psychologists at all, inasmuch as the MCMI and MMPI require little training to administer. The plaintiff, said the District Judge, maintained that any exercise of professional judgment or clinical observations in interpreting those test results is scientifically unreliable to the point of violating baseline admissibility standards under the Federal Rules of Evidence. While a step too far from baseline actuarial assessment measures used in the proper context could  be scientifically untenable, the District Judge concluded that, “[t]his Court is unwilling to sound the death knell for an entire field of expertise in the manner proposed. . . ”[20]

Another recent example can be found in Baxter v. Temple,[21] where the Supreme Court of New Hampshire struggled with an attack on the use of the Boston Process Approach to the assessment of neurological dysfunction.[22] 

In Baxter a mother, as next friend of a child who allegedly suffered from lead paint poisoning, brought a negligence action against the landlord of the apartment in which she and the child lived. When the trial court ruled that two of the plaintiff’s expert witnesses who relied on the Boston Process Approach (BPA) for the assessment of neurological dysfunction were inadmissible, the mother appealed.

Citing to treatises in the field of neurological assessment, the Supreme Court explained that the BPA is a variation of the flexible battery approach that adds a qualitative element to evaluating brain function. The BPA, reasoned the Court, uses a collection of core and satellite tests to assess various domains of cognitive functioning, such as verbal memory, visual memory, planning, attention span, language, visual perception, academic performance, and self-control.[23]

“Because this flexible approach was within the State’s Daubert evidentiary parameters, the prohibition against the flexible psychological testing regime was reversed and remanded.”

We at Falsely-Accused.net find that another example can be found in Palmer v Asarco Inc.[24] In Palmer the Plaintiffs brought a well qualified expert, and Judge Claire Eagan of the Northern District of Oklahoma did an excellent job of wending her way through the arguments of both plaintiffs and defendants on the issue of psychological testing. In Bado-Santana v. Ford Motor Co.,[25] Federal District Court Judge Gustavo Gelpí in Puerto Rico was faced with competing claims about psychologist Maria Margarida and psychological testing concerning Mild Traumatic Brain Injury. In this case, Judge Gelpí undertook a detailed evaluation of the proposed expert’s qualifications, the manner in which her credentials and methodology “fit” the facts at issue in the case and the benefits of her psychological testing.

Since the original publication of Cross Examining Experts in the Behavioral Sciences we at Falsely-Accused.net have stressed that the usefulness of psychological testing depends on validity and reliability data. In fact, many statistical procedures utilized by psychological test builders have been developed for this purpose.[26] This is why we attempt to make the point that in cross examining experts, data is everything. 

Several recent statements from our State appellate and Federal courts have made this data analysis point for us. In Glenn v. Overhead Door Corp.,[27] a Mississippi appellate panel explained that “Talking ‘off the cuff’-deploying neither data nor analysis – is not an acceptable methodology.”[28]  In this case the appellate panel analyzed the work of an expert and found that he merely offered an opinion, with no explanation of any methodology employed in arriving at that opinion.  The panel noted that the expert did not form his opinion based on his own testing or on statistical data gathered by others. They concluded with a memorable quote for the cross examiner: “[a]n expert who supplies nothing but a bottom line supplies nothing of value to the judicial process.”[29]

In Valentine v. Conrad,[30] the Supreme Court of Ohio offered a good statement of the reasons for this type of analysis. Experts, the Court reasoned, often base their opinions on data and research from within their field of study. The admissibility rule requires not only that those underlying resources are scientifically valid, but also that they support the opinion.   Although scientists certainly may draw inferences from a body of work, trial courts must ensure that any such extrapolation accords with scientific principles and methods. A court may conclude that there is simply too great an analytical gap between the data and the opinion proffered. They, too,  concluded with a good quote for the cross examiner: “[b]ecause expert opinion based on nebulous methodology is unhelpful to the trier of fact, it has no place in courts of law.”[31]

Data analysis, “fit” analysis, and “datum of discipline” analysis, all find a home in cases dependent on I.Q. and “adaptive behavior” testing.  In the next three cases explicated here, we see this being played out. We at Falsely-Accused.net have found that in each case, it is the record made by the cross examiner that the appellate courts looked to in making their final determination.

It should be noted that merely because there is some controversy over a particular psychological assessment procedure, this does not in and of itself render the procedure unusable. In Harris v. State,[32] the defendant brutally murdered his estranged wife’s employer and tried to kill her as well.  At trial, he claimed that on the morning of the shootings he had ingested approximately five beers and four or five tablets of Valium. He also presented expert testimony about his general psychological makeup, including relatively low I.Q. scores and possible bipolar disorder. Working on a theory of malingering, the prosecution sought to call its own expert in rebuttal to the diminished capacity defense. The State’s expert used a variety of assessment techniques with the defendant in an attempt to ascertain whether the defendant was “malingering,” i.e., feigning cognitive and memory deficits. This expert also compiled information to assess defendant’s score on the Hare Psychopathy Checklist.[33]

The Court noted that the Hare Psychopathy Checklist is routinely used to determine whether a person poses a threat to others. Further, the Checklist is often employed in capital-sentencing proceedings (e.g. to show the defendant is a continuing threat to society) and civil commitment proceedings (e.g. to justify involuntary commitment of a sexual predator).   The court also noted that any ability of the Checklist to predict future behavior must necessarily be based on its ability to indicate tendencies presently existing in the subject’s personality, which is based, in part, on an examination of the subject’s past behavior.[34]


[1]  Anecdotally, it has been our experience that State Medical Boards and State Boards for Social Work and also for Marriage and Family Counseling, have been reluctant to take their licensees to task for junk science testimony.  This is not the case with psychologists whose rigorous American Psychological Association ~  Ethical Principles of Psychologists and Code of Conduct ; Guidelines for Child Custody Evaluations and Specialty Guidelines for Forensic Psychologists, are taken quite seriously by licensing bodies.

[2]  Irving v State, 705 So.2d 1021 (Fla. App., 1998).

[3] Id. 705 So.2d 1021, 1022.  emphasis added.

[4] Id. emphasis added.

[5] Id. emphasis added.

[6] Id. 705 So.2d 1021, 1023.

[7] 734 F.Supp.2d 552 (S.D.N.Y. 2010).

[8] Id. at 553.

[9] Id. at 555.

[10] Id.

[11] Id. at 556.

[12] 734 F.Supp.2d at 556.

[13] Argen v. New York State Bd. Of Law Examiners, 860 F. Supp. 84 (W.D.N.Y. 1994).

[14] Id. 860 F. Supp. 84 , 85.

[15] Id.

[16] Chapple v. Granger, 851 F.Supp 1481 (E.D. Wash. 1994).

[17] Chappel,  851 F.Supp 1481, at 1496.

[18] One of the goofiest of these so called psychological “tests” is the Rorschach ink blot test.  And we should not forget that the Rorschach has a long been blocked from use and discounted in forensic settings.See for example: People v Jenko, 410 Ill. 478, 481, 102 NE 2d 783 (1952) Rorschach testimony barred;  State of Utah v Phillip Rimmasch, 775 P 2d 388 (Utah Sup., 1989) Rorschach testimony is violative of Rule 702; McCarty v McCarty, (WL 259363 Neb App 1993) Trial court properly disregarded Rorschach testimony.

For a recent attack on the Static 99, please see: State v Ewers,  2005 WL 2334958, (Ohio App. 2005).

For a recent attack on the MMPI-2, please see: Waldrip v Head, 279 Ga. 826, 620 S.E.2d 829 (Supreme Court of Georgia 2005).

[19] Foreman v. American Road Lines, Inc., 623 F.Supp.2d 1327 (S.D. Alabama December, 2008).

[20] Foreman v. American Road Lines, Inc., 623 F.Supp.2d 1327 at 1334.

[21] 157 N.H. 280 (New Hampshire 2008).

[22] 157 N.H. 280, 287, 288.

[23] 157 N.H. 280, 287, 288; citing to W.P. Milberg et al., The Boston Process Approach to Neuropsychological Assessment, in Neuropsychological Assessment of Neuropsychiatric Disorders 65, 67 (Igor Grant & Kenneth M. Adams eds., 1986).

[24] 510 F.Supp.2d 519  (N.D. Oklahoma 2007).

[25] 482 F.Supp.2d 192 (D. Puerto Rico 2007).  Judge Gustavo Gelpí is the youngest United States District Judge for the District of Puerto Rico. He is a former Assistant Federal Public Defender and law clerk to Juan M. Perez Gimenez, his predecessor and now colleague on the federal bench.

[26] For example:

Split‑half method The scores on a randomly chosen half of the test questions are correlated with the scores on the other half.

Cronbach’s a This is the mean value of all the possible split‑half reliability coefficients.

Test‑retest reliability A correlation is obtained between scores on the test administered to the same subject,  but on separate occasions.

Alternate‑form reliability The second form of the test is as nearly equivalent to the first as possible.

See: “A Behavioral Science Glossary” in  Lorandos, D. & Campbell, T.W. (2005)  Benchbook in the Behavioral Sciences – Psychiatry – Psychology – Social Work –   Carolina Academic Press, Durham, North Carolina.

[27] 935 So.2d 1074 (Mississippi App., 2006).

[28] Id. 935 So.2d at 1089, citing to Lang v. Kohl’s Food Stores, Inc., 217 F.3d 919, 924 (7th Cir.2000).

[29] Id. 935 So.2d 1089 -1090 citing to Mid-State Fertilizer Co. v. Exchange Nat. Bank of Chicago, 877 F.2d 1333, 1339 (7th Cir.1989).

[30] 110 Ohio St.3d 42, 850 N.E.2d 683 (Supreme Court of Ohio 2006).

[31] Id. 850 N.E.2d  at 687 citing to General Electric Co. v Joiner, 522 US 136, 146; 118 S Ct 512 (1997).

[32]  84 P. 3d 731 (2004) Court of Criminal Appeals of Oklahoma.

[33]  Please see our discussion of the various Hare assessment modalities in CHAPTER NINE and also see: Hare, R.D. (1991). Manual for the Hare Psychopathy Checklist- Revised. Toronto, Ont:Multi-Health Systems; Quinsey V.L., Harris, G.T., Rice, M.E. & Cormier, C.A. (1998). Violent offenders: Appraising and managing risk. Washington, DC: American Psychological Association (p. 241); Douglas, K.S., Ogloff, J.R., Nicholls, T.L. & Grant, I. (1999). Assessing risk for violence among psychiatric patients: The HCR-20 violence risk assessment scheme and the Psychopathy Checklist: Screening Version. Journal of Consulting and Clinical Psychology, 67, 917-930.

[34]  84 P. 3d 731, 746 citing to  Lockett v. State, 53 P.3d 418, 425 (OK 2002): Psychopathy evidence admitted in capital sentencing phase and  In re Commitment of Burgess, 262 Wis.2d 354, 665 N.W.2d 124, 134-35 (2003): Psychopathy evidence used to support civil commitment of convicted sexual predator.

 
 
 
 
 
 
By Gardner, Sauber, and Lorandos, has become the standard reference work for PAS. The International Handbook features clinical, legal, and research perspectives from 32 contributors from eight countries.
 
The International Handbook of
Parental Alienation Syndrome
 
By Terrence W. Campbell and Demosthenes Lorandos, is a must for every family law practitioner. This two-volume practice set provides step-by-step guidance how to refute behavioral scientists.
 
Cross Examining Experts in the
Behavioral Sciences
 
By Lorandos and Campbell, provides immediate access to authoritative information and immediate decision-making tools for judges and attorneys.
 
Benchbook in the Behavioral Sciences