Dr. Lorandos Interviewed on Larry King Live
 
 
About The Author

Demosthenes Lorandos is an attorney, forensic psychologist, lecturer and author.  He is in constant demand across the country for his authoritative and tireless advocacy on behalf of the falsely accused.   In defense of his clients Dr. Lorandos brings to bear his extensive expertise in the science of coerced confessions, shaken baby syndrome, battered women’s syndrome, parental alienation syndrome, recovered memories, false accusations, repeated question effects, interviewer bias, behavioral indicators of abuse and other related subjects.  Dr. Lorandos is a member of the California, Michigan, New York, Tennessee and Washington, D.C. bars, and a member of the bar of the United States Supreme Court.   He is also the co-author of such works as Cross Examining Experts in the Behavioral Sciences, Benchbook in the Behavioral Sciences and The International Handbook of Parental Alienation Syndrome. Dr. Lorandos may be recognizeable from his appearances on The Today Show, The View, Larry King Live and Court TV, and from his extensive online catalog of free legal advice videos.

 
Falsely accused of physical or sexual abuse?
Have you been falsely accused?
 
 
Overview of the Rorschach
Posted On January 19th, 2015  By admin  

Irving Weiner, who has written extensively regarding the Rorschach, contends that it endures unwarranted criticism. Weiner insists:

“Current literature reflects a persistent inclination in some quarters to denigrate the Rorschach Inkblot Method as an invalid and useless instrument for assessing personality functioning. Although perhaps warranted to some extent in years past, such harsh criticism of the Rorschach runs counter to abundant contemporary data demonstrating its psychometric soundness and practical utility.”[1]

There are some data demonstrating the Rorschach’s psychometric soundness. A 1999 study, for example, found that the validity coefficients of the Rorschach and the MMPI 2 are approximately equal.[2] These data, however, do not necessarily support Weiner’s claims of “practical utility” for the Rorschach. The 1999 study reviewed Rorschach data obtained from studies published in peer reviewed journals. As we at Falsely-Accused.net pointed out in prior sections, without satisfactory levels of inter rater scoring consistency, peer reviewed journals will not publish Rorschach research.

Research teams working diligently to achieve satisfactory levels of Rorschach scoring consistency can succeed in doing so. Extrapolating from these data to how practicing psychologists score and interpret the Rorschach is ill advised. As we at Falsely-Accused.net also pointed out in prior sections of this Blog, it is necessary to discriminate between the “research” utility of the Rorschach, and its “field” utility. Compared to research psychologists, practicing psychologists typically do not have sufficient time available to hone their Rorschach scoring skills. We at Falsely-Accused.net feel that consequently, the “field utility” of the Rorschach most certainly does not even approximate its “research utility.” Claiming otherwise is tantamount to the first author contending that because we use the same make of clubs, his golfing prowess equals that of Tiger Woods!

Psychologists using the Rorschach may claim that their specialized training with this technique allows them to administer, score, and interpret it accurately. Supporting these claims would necessitate training that uses a criterion level for certifying Rorschach competence. A criterion level would define how sample Rorschach responses should be scored and interpreted. After reaching a predetermined level of scoring and interpretive accuracy, a psychologist could claim Rorschach competence. To the best of our knowledge, there are no Rorschach training programs designed to bring trainees to a defined level of accuracy. Claims of accuracy for scoring and interpreting therefore amount to totally unsubstantiated claims.

Cross Examining the Clinical Use of the Rorschach

1.  You claim that there are numerous studies published in peer reviewed journals supporting your use of the Rorschach Correct?

2.  Without satisfactory levels of inter rater scoring consistency, peer reviewed journals will not publish Rorschach research Correct?

3.  And we know that research teams working diligently to obtain satisfactory levels of Rorschach scoring consistency can succeed in doing so Correct?

4.  Compared to the research psychologists who publish Rorschach data, practicing psychologists do not have the time available to hone their Rorschach scoring skills Correct?

[THE FOLLOWING QUESTIONS ARE TAKEN FROM PRIOR SECTIONS]

5.  The psychologists participating in Rorschach research studies underwent specialized training to reach acceptable levels of inter rater scoring consistency Correct?

6.  What specialized training have you undergone to assist you in scoring the Rorschach in an acceptable manner?

[The vast majority of psychologists have not undertaken this kind of training.
Prior to cross examination, check the psychologist’s c.v. for post doctoral
training in the Rorschach]

7.  And the “field reliability” of the Rorschach refers to how practicing psychologists such as yourself use it Correct?

8.  You cannot cite any data published in a peer reviewed journal reporting the field reliability for the TRACS Correct? [or whatever other system the professional used].

9.  In other words, the extent to which practicing psychologists carefully comply with the TRACS scoring procedures [or any other scoring procedure] remains unknown Correct?

10.  If the extent to which practicing psychologists carefully comply with the TRACS scoring procedure remains unknown, we do not know how carefully you complied with that scoring procedure Correct?

11.  And if you resorted to your own idiosyncratic scoring method, your interpretations of the Rorschach in this case could differ from another professional Correct?

12.  Because assumptions about the interpretive accuracy of the Rorschach assume that it was scored properly and interpreted in a uniform manner Correct?

13.  Other than accepting your claims, we have no way of knowing if you scored the Rorschach accurately in this case Correct?

14.  And if you scored the Rorschach inaccurately, you could misinform and mislead this proceeding Correct?


[1]Weiner, I.B. (1996). Some observations on the validity of the Rorschach inkblot method. Psychological Assessment, 8, 206 213 (p. 206).
[2]Hiller, J.B., Rosenthal, R., Bornstein, R.F., Berry, D.T.R. & Brunell Neuleib, S. (1999). A comparative meta analysis of Rorschach and MMPI validity. Psychological Assessment, 11, 278 296.

 
 
Computerized Interpretation of the Rorschach
Posted On January 14th, 2015  By admin  

We at Falsely-Accused.net take note that there is a commercially available program for the computerized interpretation of the Rorschach. This computer based test interpretation (CBTI) is for sale from a service identifying itself as Ror Scan (www.ror scan.com). All Ror Scan CBTIs contain the following disclaimer:

“The following interpretive hypotheses are generated from statistical comparisons of the structural features of the person’s Rorschach test performance with that of groups of people with known personality characteristics. In addition, interactive clinical reasoning is used to deduce some narrative statements. Because all statements are tentative hypotheses based on generalizations, decisions and conclusions about this person cannot be made solely from this limited resource. The clinician must cross validate, refine, and modify these hypotheses by using additional test data and other clinical information. Accurate application of Comprehensive System [TRACS] administration and scoring procedures is essential to the use of this interpretive system.”

In previous posts, we discussed the many problems undermining the evidentiary value of CBTIs for the MMPI 2. Those same problems also undermine the evidentiary value of the Ror Scan. Additionally, it is necessary to carefully review the disclaimer cited immediately above. First of all, the disclaimer indicates:

“The following interpretive hypotheses are generated from statistical comparisons of the structural features of the person’s Rorschach test performance with that of groups of people with known personality characteristics.”

This statement raises the question of what “groups of people with known personality characteristics”? These groups of people are not clearly identified by the Ror Scan computerized print out. Additionally, the statement regarding “known personality characteristics” is not sufficiently clarified. There is no information available from the Ror Scan addressing what these “known personality characteristics” are, nor how they were identified, nor by whom.

We at Falsely-Accused.net feel that cross examining the Ror Scan therefore necessitates two questions directed at challenging the validity of its interpretive statements. Assume, for example, a Ror Scan CBTI states:

“The person prefers to very strongly overcontrol his feelings, trying not to express them in everyday functioning.”

Interpretive statements such as these necessitate asking the psychologist relying on the Ror Scan:

(1) Have you published any data in a peer reviewed journal that validates this interpretation.

(2) Can you cite any data in a peer reviewed journal that validates this interpretation?

Relying on such interpretations, without the availability of peer reviewed data to support them, also neglects Standards 2.02 (a b) of the 1992 Ethical Code for psychologists (See Section II I in this chapter).

The Ror Scan disclaimer also indicates:

“In addition, interactive clinical reasoning is used to deduce some narrative statements.”

This statement necessitates asking any psychologist who relies on the Ror Scan:

“Do you know which Ror Scan interpretations rely on ‘statistical comparisons,’ and which interpretations rely on ‘interactive clinical reasoning?”

The information necessary to answer this question is not available from the Ror Scan print out.

The Ror Scan disclaimer further indicates:

“Because all statements are tentative hypotheses based on generalizations, decisions and conclusions about this person cannot be made solely from this limited resource.”

If the interpretive statements of the Ror Scan are ‘tentative hypotheses,’ then substantially relying on them is ill advised. Moreover, if the Ror Scan is a “limited resource,” then its evidentiary reliability is also severely limited.

The Ror Scan disclaimer additionally indicates:

“The clinician must cross validate, refine, and modify these hypotheses by using additional test data and other clinical information.”

The problems associated with considering “additional test data and clinical information” were outlined in previous sections. Those sections are also recommended for cross examining the Ror Scan.

We at Falsely-Accused.net note that the Ror Scan disclaimer moreover indicates:

“Accurate application of Comprehensive System [TRACS] administration and scoring procedures is essential to the use of this interpretive system.”

This portion of the disclaimer raises all the problems of inter rater scoring consistency related to the TRACS. These problems were discussed in Prior Sections of this blog. Those sections are also recommended for cross examining the Ror Scan. Additional strategies for cross examaining the Ror Scan can be found in Prior Sections (Assumptions Related to the Rorschach) and Prior Sections (Speculative Excesses) of this blog.

 

 
 
Speculative Excesses of the Rorschach
Posted On January 5th, 2015  By admin  

The Emphasis at Falsely-Accused.net is that ultimately, the Rorschach invites psychologists to indulge in imaginative speculations unsupported by empirical data. This problem was recognized as long ago as 1942 in a review published in the British Journal of Psychology:

“… the possible interpretations are so various and the boundaries between one category and another so soft that it may become very difficult to strike a balance between individual judgment and conventional, perhaps esoteric, categorization.”[1]

A 1947 review of was even more emphatic in its criticisms of the speculative excesses associated with the Rorschach:

“… the use of the Rorschach test is as far removed from scientific psychology as palmistry or graphology … it does not offer convincing evidence on the test’s reliability or validity”.[2]

In his generally accepted psychological testing text, Groth Marnat expressed similar comments in 1992:

“… accuracy has often been dependent more on intuition and clinical lore rather than on clearly validated interpretive rules … Significant weaknesses are its moderate and widely varying reliabilities and validates …”[3]

Exner’s system (TRACS) also invites imaginative speculation from those who use it. For example, a reflection response (e.g., seeing trees reflected in a body of water) leads to unsubstantiated conjecture about the subject’s personality. In particular, Exner interprets such responses as indicating that the subject’s personality involves:

“… a narcissistic like feature that includes a marked tendency to overvalue personal worth”.[4]

A human experience response (e.g., two people deeply in love, gazing longingly at each other) supposedly indicates that the subject’s:

“… attempts to deal with issues of self image and/or self value in an overly intellectualized manner that tends to ignore reality”.[5] A food response (e.g., a drumstick from a Thanksgiving turkey) leads to the unwarranted conclusion that the subject “… can be expected to manifest many more dependency behaviors than usually expected”.[6]

We at Falsely-Accused.net note that psychologists who use the Rorschach also frequently assume there are symbolic characteristics associated with some of its cards. Responses to card IV, for example, are often regarded as related to how examinees feel about their fathers. Responses to card VII are often regarded as revealing how examinees feel about their mothers. In a 1998 article, however, Handler advises against relying on these symbolic assumptions:

“… I caution students not to engage in stereotypes concerning a particular set of expectations for each card content.”[7]

In discussing this issue, Handler cites the 1966 work of Schactel:

“Some authors discuss card IV as the ‘father card’ and card VII as the ‘mother card’ … Such an assumption will lead to faulty interpretations … I am convinced that it is mistaken to assume that any specific context can be assigned to any particular card and serve as the basis for the assumption that, whatever the testee’s reaction, it will have some relation to this assumed ‘meaning’ of the inkblot.”[8]

Obviously, then, symbolic interpretations of the Rorschach cards are ill advised.

In this same 1998 article, Handler further discusses symbolic interpretations of Rorschach responses. In particular, Handler cited the case of:

“… a 21 year old girl who almost died from aplastic anemia because of a male physician’s error and who had been treated badly by the various men in her life, including her father and step father, she saw elephants on four cards. When I presented her Rorschach in class the students all had different ideas about what the elephants might represent for her. On person suggested power and strength, one suggested playfulness and gentleness, and one, masculinity.”[9]

The “elephant” example demonstrates how symbolic interpretations of the Rorschach lead to substantial variations in opinion. One student sees symbolic themes of power and strength. Another sees symbolic themes of playfulness and gentleness; and still another sees symbols of masculinity. We at Falsely-Accused.net feel that quite obviously, we have no way of ascertaining which symbolic interpretation is accurate; and it is also possible that they are all mistaken.

Handler continued to clarify his position indicating:

“They [the students] seemed to be responding with interpretations based on their own feelings about elephants. This is exactly the situation Schactel warned us to avoid, because we may be using our own idiosyncratic interpretations rather than those of the patient.[10]

When symbolic interpretations of the Rorschach respond to the idiosyncratic feelings of the examiner, those interpretations reveal more about the examiner than the examinee.

Cross Examining the Speculative Excesses of the Rorschach

1.  You agree that the Rorschach encourages psychologists to rely on conjecture and imagination Correct?

2.  You indicated that you used Exner’s “Comprehensive System” to administer and score this Rorschach Correct?

3.  Therefore, you have used on of Exner’s manuals for your administration and scoring Correct?

4.  And as a result, you consider Exner an authority regarding the Rorschach Correct?

5.  Exner, for example, says that reflection responses seeing trees reflected in a body of water corresponds to narcissism including a “marked tendency to overvalue personal worth” Correct?

6.  You have not published any data in a peer reviewed journal supporting the accuracy of this interpretation Correct?

7.  You cannot cite any data published in a peer reviewed journal supporting the accuracy of this interpretation Correct?
8.  Exner also claims that a human experience response two people deeply in love gazing longingly at each other indicates attempts at dealing “with issues of self image and/or self value in an overly intellectualized manner that tends to ignore reality” Correct?

9.  You have not published any data in a peer reviewed journal supporting the accuracy of this interpretation Correct?

10.  You cannot cite any data published in a peer reviewed journal supporting the accuracy of this interpretation Correct?

11.  Exner also claims that a food response a drumstick from a Thanksgiving turkey indicates that the subject “can be expected to manifest many more dependency behaviors than usually expected” Correct?

12.  You have not published any data in a peer reviewed journal supporting the accuracy of this interpretation Correct?

13.  You cannot cite any data published in a peer reviewed journal supporting the accuracy of this interpretation Correct?

14.  Please consider Standard 2.02 (a) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(a) Psychologists who develop, administer, score, interpret, or use psychological assessment techniques, interviews, tests, or instruments do so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

Now my question: Your work in this case obligates you to comply with this standard Correct?

15.  Without peer reviewed data to support these speculative interpretations, relying on them neglects Standard 2.02 (a) of your ethical code Correct?

16.  Because without peer reviewed data to support these speculative interpretations, you cannot use them in a manner that is “appropriate in light of the research on or evidence of the usefulness and proper application of the technique” Correct?

17.  Also please consider Standard 2.02 (b) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

 

Now my question: Your work in this case obligates you to comply with this standard Correct?

18.  Without peer reviewed data to support these speculative interpretations, your relying on them also neglects Standard 2.02 (b) of your ethical code Correct?

19.  Because without peer reviewed data to support these speculative interpretations, you are at risk for misusing this “assessment technique” and its “interpretations” Correct?

20.  Psychologists using the Rorschach often attribute symbolic characteristics to various cards Correct?

21.  Responses to card IV, for example, are often regarded as related to how examinees feel about their fathers Correct?

22.  And responses to card VII are often regarded as revealing how examinees feel about their mothers Correct?

23.  And do you subscribe to these assumptions regarding the symbolic characteristics of various Rorschach cards?

[If the psychologist answers “No,” proceed to Question #28]

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS IF YOU HAVE NOT YET DONE SO]

24.  The Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

25.  And a 1998 article by Handler published in the Journal of Personality Assessment titled “Schactel and teaching: What color was the couch?” might be relevant to your opinions in this case Correct?

26.  Please consider Handler’s comments advising against symbolic interpretations of the Rorschach:

– [ read ] –

“… I caution students not to engage in stereotypes concerning a particular set of expectations for each card content.”

Now my question: Handler appears to be advising against symbolic interpretations of various Rorschach cards Correct?

27.  Please consider how Handler further clarified his position, in this same article, regarding symbolic interpretations of Rorschach cards by citing the 1966 work of Schactel:

– [ read ] –

“Some authors discuss card IV as the ‘father card’ and card VII as the ‘mother card’ … Such an assumption will lead to faulty interpretations … I am convinced that it is mistaken to assume that any specific context can be assigned to any particular card and serve as the basis for the assumption that, whatever the testee’s reaction, it will have some relation to this assumed ‘meaning’ of the inkblot.”

Now my question: If it is mistaken to assume that any specific context can be assigned to any particular card, your symbolic interpretations of various Rorschach cards is ill advised Correct?

28.  Consistent with Exner’s approach, do you rely on other symbolic interpretations of the Rorschach?
[If the psychologist answers “No,” cease this line of questioning]

29.  Please consider the following example of symbolic interpretations taken from Handler’s 1998 article:

– [ read ] –

“… a 21 year old girl who almost died from aplastic anemia because of a male physician’s error and who had been treated badly by the various men in her life, including her father and step father, she saw elephants on four cards. When I presented her Rorschach in class, the students all had different ideas about what the elephants might represent for her. On person suggested power and strength, one suggested playfulness and gentleness, and one, masculinity.”

Now my question: If the symbolic interpretations of the elephant responses emphasize power and strength, or playfulness and gentleness, or masculinity, we do not know which interpretation is accurate Correct?

30.  And all of those interpretations could be mistaken Correct?

31.  Please consider Handler’s additional remarks regarding the symbolic interpretations of the elephant responses:

– [ read ] –

“They [the students] seemed to be responding with interpretations based on their own feelings about elephants. This is exactly the situation Schactel warned us to avoid, because we may be using our own idiosyncratic interpretations rather than those of the patient.”

Now my question: When symbolic interpretations of the Rorschach respond to the idiosyncratic feelings of the examiner, those interpretations can reveal more about the examiner than the examinee Correct?


[1]Author. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 153

[2]Author. (1953). Review of Rorschach texts in O.K. Buros (Ed.), The Fourth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page

[3]Groth-Marnat, G. (1992). Handbook of Psychological Assessment . New York: Wiley-Interscience. Pages 275-284.

[4]Exner, J.E. (1991). The Rorschach: A comprehensive system: Vol 2. Interpretation (2nd ed.). New York: Wiley, at page 173.

[5]Id. at page 176.

[6]Id. at page 184.

[7]Handler, L. (1998). Schactel and teaching: What color was the couch? Journal of Personality Assessment, 71, 172 181 (p. 175).

[8]Schactel, E. (1966). Experiential foundations of Rorschach’s test. New York: Basic Books (p. 31-33).

[9]Handler, L. (1998). Schactel and teaching: What color was the couch? Journal of Personality Assessment, 71, 172 181 (p. 180).

[10]Id.

 

 

 
 
Assumptions Related to the Rorschach
Posted On December 29th, 2014  By admin  

Wood and his colleagues explain that current use of the Rorschach relies on four basic assumptions:

“1.  In contrast to earlier Rorschach systems, the Comprehensive System [TRACS] has demonstrated a high level of inter-rater reliability.

2.  The clinical interpretations generated by the Comprehensive System are consistent with research findings and have been well validated.

3.  The various indices of the Comprehensive System have performed well in cross validation samples.

4.  The research base of the Comprehensive System is well documented and has been scrutinized and confirmed by independent scholars.”[1]

We at Falsely-Accused.net concur entirely with Wood and his colleagues who emphasized there are no available data to support these assumptions.
Psychologists who use the Rorschach also sometimes assume that it provides reliable information because it presumably resists “faking.” These assumptions further claim that the ambiguity of the Rorschach precludes faking because subjects have no idea of how to do it. We at Falsely-Accused.net consider, for example, the claims a psychologist expressed on behalf of the Rorschach:

“…. it is particularly useful as a part of a test battery for forensic evaluations because it is not amenable to approach biases. In other words, clients do not have any idea how to ‘fake’ their responses.”

Nevertheless, a 1980 study by Albert[2] and his colleagues demonstrated that university students can successfully fake schizophrenia on the Rorschach. After watching a 25 minute film about schizophrenia, 72% of the students successfully faked schizophrenia compared to 46% of the students who had not seen the film. The psychologists asked to review the Rorschach protocols were all quite experienced (Fellows of the Society for Personality Assessment). Ewven when alerted that some of the Rorschach records were faked, these experienced psychologists could not consistently identify the faked records.

Cross examining Assumptions Related to the Rorschach

1.  Does your use of the Rorschach assume that the Comprehensive System [TRACS] has demonstrated a high level of inter-rater reliability?

2.  You have not published any data in a peer reviewed journal to support your position in this regard Correct?

3.  You cannot cite any data published in a peer reviewed journal to support your opinion in this regard Correct?

4.  Does your use of the Rorschach assume that the clinical interpretations generated by the Comprehensive System are consistent with research findings?

5.  You have not published any data in a peer reviewed journal to support your position in this regard Correct?

6.  You cannot cite any data published in a peer reviewed journal to support your opinion in this regard Correct?

7.  Does your use of the Rorschach assume that the various indices of the Comprehensive System have performed well in cross validation samples?

8.  You have not published any data in a peer reviewed journal to support your position in this regard Correct?

9.  You cannot cite any data published in a peer reviewed journal to support your opinion in this regard Correct?

10.  Does your use of the Rorschach assume that the research base of the Comprehensive System is well documented and confirmed by independent scholars?

11.  You have not published any data in a peer reviewed journal to support your position in this regard Correct?

12.  You cannot cite any data published in a peer reviewed journal to support your opinion in this regard Correct?

13.  Does your use of the Rorschach assume that it provides reliable information because it presumably resists faking?

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS
IF YOU HAVE NOT YET DONE SO]

14.  The Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

15.  And a 1980 study by Albert and his colleagues published in the Journal of Personality Assessment titled “Faking Psychosis on a Rorschach: Can Expert Judges Detect Malingering” might be relevant to your opinions in this case Correct?

16.  Please consider the following 5 steps of this study:
[Consider printing these five steps on a piece of paper you can hand the witness]

(1) Half the college students participating in the study watch a 25 minute film about schizophrenia. The remaining students do not see the film.

 

(2) Half the students in both groups are then asked to fake schizophrenia on the Rorschach.

 

(3) Fellows of the Society for Personality Assessment review the Rorschach responses of all students. These experienced psychologists are asked to identify any Rorschach records they consider “faked.”

 

(4) 72% of the students asked to fake schizophrenia, and who watched the film, did so successfully without detection.

 

(5) 46% of the students asked to fake schizophrenia, and who did not see the film, also faked successfully without detection.

Now my question: If 72% of the students who watched the film could fake schizophrenia without detection, we know that the Rorschach can be faked Correct?

17. And if 46% of the students who did not see the film can fake schizophrenia without detection, we further know that the Rorschach can be faked with no coaching Correct?

18. Therefore, your assumptions about the Rorschach resisting faking may mislead and misinform this proceeding Correct?


[1]Wood, J.M., Nezworski, M.T. & Stejskal, W.J. (1996). The Comprehensive System for the Rorschach: A Critical Review. Psychological Science, 7, 3 10, at page 9.

[2]Albert, S., Fox, H.M. & Kahn, M.W. (1980). Faking psychosis on the Rorschach: Can expert judges detect malingering? Journal of Personality Assessment, 44, 115 119.

 

 

 
 
Exner’s Egocentricity Index and Issues of Peer Review
Posted On December 22nd, 2014  By admin  

Related data do not support other facets of the Exner system. Despite its theoretical assumptions to the contrary, Exner’s “Egocentricity Index” appears unrelated to personality characteristics such as self focus, self esteem, and narcissism. Nezworski and Wood’s 1995 review reporting these findings concluded by indicating:

“At this time there is insufficient evidence to support the validity of the EGOI [Egocentricity Index] pairs, and reflections, as measures of self esteem, self focus, narcissism, ego functioning, or depression. We recommend that clinicians regard these measures as experimental. The available scientific evidence does not support their use for clinical decision making.”[1]

This review of the TRACS also pointed out that many of the studies cited as supporting it have never undergone peer review. In a response to this review, Exner indicated:

“I continue to regard many of the studies that I have listed as ‘unpublished’ in the various volumes related to the Comprehensive System as milestones in our early efforts … I have also cautioned repeatedly that the work concerning the Comprehensive System remains unfinished …”[2]

At Falsely-Accused.net we emphasize that if the development of the Comprehensive System remains unfinished as Exner indicates, then it amounts to an experimental procedure that has not achieved general acceptance. Similarly, the previously cited 1997 study of Carlson and her colleagues concluded that Exner’s Rorschach system is “not fulfilling the publication promise.” Carlson and her colleagues also concluded:

“… something happened on the way from the research lab to the clinical field. Clearly more research needs to be done on these as well as other TRACS indices if clinicians are to have confidence in their validity for psychological diagnoses and treatment as well as for clinical research.”[3]

A 1997 study reported data it interpreted as supporting the EGOI.[4] This study used the Rorschach to identify clients in treatment who met DSM IV criteria for the diagnosis of Narcissistic Personality Disorder (NPD). The study identified the number of “reflection responses” and “idealization” responses as the best variables for identifying NPD. The scoring of “idealization” responses, however, is not addressed by the TRACS. Instead, the scoring of “idealization” responses relies on the work of Lerner (1991).[5] This consideration raises the question of how many psychologists using the EGOI are familiar with Lerner’s procedures?

At Falsely-Accused.net we note that this same 1997 study also advised caution when interpreting reflection responses on the Rorschach:

“Therefore, it is not suggested that clinicians use the presence of one or more reflection responses on the Rorschach as an indication of NPD. To do so would be a very concrete interpretation of the data presented here and would be clinically unsophisticated. What these data do suggest is that the presence of one or more reflection responses typically will be developed in outpatient populations by individuals who meet some of the DSM IV criteria for NPD, but possibly not five of the nine criteria.”[6]

Cross examining Exner’s Egocentricity Index and Issues of Peer Review
[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS
IF YOU HAVE NOT YET DONE SO]

1.  The “Egocentricity Index” of the Rorschach claims to assess personality characteristics such as self focus, self esteem, and narcissism Correct?

2.  The journal Clinical Psychology: Science and Practice is a generally recognized and accepted, peer reviewed journal in your field Correct?

3.  And a 1995 article by Nezworski and Wood published in Clinical Psychology: Science and Practice titled “Narcissism in the Comprehensive System for the Rorschach” might be relevant to your opinions in this case Correct?

4.  Please consider how Nezworski and Wood concluded their review:

– [ read ] –

“At this time there is insufficient evidence to support the validity of the EGOI [Egocentricity Index] pairs, and reflections, as measures of self esteem, self focus, narcissism, ego functioning, or depression.”

Now my question: If there is insufficient evidence to support the Egocentricity Index as a measure of self esteem, self focus, narcissism, ego functioning, or depression, then your relying on it is ill advised Correct?

5.  Please consider how Nezworski and Wood further concluded their review:

– [ read ] –

“We recommend that clinicians regard these measures as experimental.”

Now my question: If the Egocentricity Index is an experimental measure, then by definition it is not generally accepted by your scientific and professional community Correct?

6.  And if you have relied on an experimental measure not generally accepted by your scientific and professional community you could misinform and mislead this proceeding Correct?

7.  Please also consider how Nezworski and Wood additionally concluded their review:

– [ read ] –

“The available scientific evidence does not support their use for clinical decision making.”

Now my question: If the available clinical evidence does not support the use of the Egocentricity Index for clinical decision making, then your relying on it could also misinfom and mislead this proceeding Correct?

8.  Please consider these additional comments by Nezworski and Wood in their 1995 article:

– [ read ] –

“… the Workshop Studies of the EGOI cited in the Rorschach (Exner, 1993) are unavailable for scholarly examination.”

Now my question: If Exner’s Workshop Studies are unavailable for scholarly examination, that means those studies have not undergone peer review Correct?

9.  And we have already established how critically important peer review is in the development and advancement of your field Correct?

[This was done via the 22 foundational questions from Previous posts]

10.  Please consider how Exner responded to the criticisms of Nezworski and Wood:

– [ read ] –

“I continue to regard many of the studies that I have listed as ‘unpublished’ in the various volumes related to the Comprehensive System as milestones in our early efforts …”

Now my question: If Exner acknowledged that his own studies are “unpublished,” then he also acknowledges that they have not undergone peer review Correct?

11.  Please consider Exner’s continued response to the criticisms of Nezworski and Wood:

– [ read ] –

“I have also cautioned repeatedly that the work concerning the Comprehensive System remains unfinished …”

Now my question: If Exner acknowledges that work concerning the Comprehensive System remains unfinished, his acknowledgment further confirms that the TRACS is an experimental procedure Correct?

12. Please consider Standard 2.02 (a) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(a) Psychologists who develop, administer, score, interpret, or use psychological assessment techniques, interviews, tests, or instruments do so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

Now my question: Your work in this case obligates you to comply with this standard Correct?

13. If work concerning the Comprehensive System remains “unfinished,” your relying on it neglects Standard 2.02 (a) of your ethical code Correct?

14. Because if work concerning the Comprehensive System remains “unfinished,” you cannot use it in a manner that is “appropriate in light of the research on or evidence of the usefulness and proper application of the technique” Correct?

15. Also please consider Standard 2.02 (b) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

Now my question: Your work in this case obligates you to also comply with this standard Correct?

16. If work concerning the Comprehensive System remains “unfinished,” your relying on it also neglects Standard 2.02 (b) of your ethical code Correct?

17. Because if work concerning the Comprehensive System remains “unfinished,” you are at risk for misusing this Aassessment technique” and its “interpretations” Correct?

18. The journal Psychological Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

19. And a 1997 article published in Psychological Assessment by Hilsenroth and his colleagues titled “Narcissism in the Rorschach revisited: Some reflections on empirical data” might be relevant to your opinions in this case Correct?

20. Hilsenroth and his colleagues reported that for their study:

– [ read ] –

“The administration and original scoring followed the procedures articulated by Exner, except in the case of the idealization variable, which was scored according to the procedures of Lerner (1991).”

Now my question: Did you use the scoring procedures of Lerner, for the idealization responses, found in his 1991 book Psychoanalytic Theory and the Rorschach published by the Analytic Press?

21. And without using Lerner’s scoring procedure for “idealization responses” on the Rorschach, it may be inappropriate for you to cite the 1997 study of Hilsenroth and his colleagues as supporting your use of the Rorschach Correct?

22. Please consider these additional comments from Hilsenroth and his colleagues regarding reflection responses on the Rorschach: ”

– [ read ] –

“Therefore, it is not suggested that clinicians use the presence of one or more reflection responses on the Rorschach as an indication of NPD.[Narcissistic Personality Disorder] To do so would be a very concrete interpretation of the data presented here and would be clinically unsophisticated.”

Now my question: If relying on one or more reflection responses to identify NPD [Narcissistic Personality Disorder] is “clinically unsophisticated,” then your use of the Rorschach for this purpose may also be “clinically unsophisticated” Correct?


 

[1] Nezworksi, M.T. & Wood, J.M. (1995). Narcissism in the Comprehensive System for the Rorschach. Clinical Psychology: Science and Practice, 2, 179 199, at page 196.

[2]Exner, J.E. (1995). Comment on “Narcissism in the Comprehensive System for the Rorschach.” Clinical Psychology: Science and Practice, 2, 200 206, at page 205.

[3]Carlson, C.F., Kula, M.L. & St. Laurent, C. (1997). Rorschach revised DEPI and CDI with inpatient depressives and borderline personality disorder with major depression: Validity issues. Journal of Clinical Psychology, 53, 51 58 at page 56.

[4]Hilsenroth, M.J., Fowler, J.C., Padawer, J.R. & Handler, L. (1997). Narcissism in the Rorschach revisited: Some reflections on empirical data. Psychological Assessment, 9, 113 121.

[5]Lerner, P. (1991). Psychoanalytic theory and the Rorschach. Hilldale, NJ: Analytic Press.

[6]Id. at page 119.

 
 
Exner’s Depression Index
Posted On December 15th, 2014  By admin  

At Falsely-Accused.net we note that Exner’s “Depression Index” (DEPI) supposedly identifies people who are clinically depressed. A review of the relevant literature, however, does not support the DEPI. A 1991 study investigated the DEPI R in children and adolescents. It found no significant relationships between (1) the original (DEPI) or revised Depression Index (DEPI R) and clinical elevations on the Depression scale of the Personality Inventory for Children in an outpatient sample, or (2) the diagnostic judgments of treatment teams in an outpatient sample.[1]

We at Falsely-Accused.net emphasize that a 1997 study found little agreement between the positive values of the DEPI R and DSM III diagnoses of Major Depressive Disorder.[2] In this 1997 study, two other independent measures of depression the MMPI 2 Depression Scale and the Beck Depression Inventory validated the presence of depression and correlated highly with each other, but not with the DEPI R. Similarly, a 1993 study found no significant relationship between the DEPI R and patients’ self reports of depression.[3]

Cross examining Exner’s Depression Index

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS
IF YOU HAVE NOT YET DONE SO]

1. The DEPI R of the (TRACS) Rorschach refers to the “Depression Index Revised” Correct?

2. And the DEPI R of the (TRACS) Rorschach supposedly identifies people who are depressed Correct?

3. And the DEPI R can supposedly be used with children Correct?

4. The Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

5. And a 1991 study by Ball and his colleagues published in the Journal of Personality Assessment titled “Rorschach Depression Indices with Children and Adolescents: Concurrent Validity Findings” might be relevant to your opinions in this case Correct?

6. Did you know that this 1991 study found no relationship between children’s DEPI R scores, and their scores on the Depression scale of the Personality Inventory for Children?

7. This outcome suggests that either the Depression scale of the Personality Inventory for Children, or the DEPI R, or both, are inadequate for identifying depressed children Correct?

8. Did you know that this same 1991 study found no relationship between the DEPI R and the diagnostic judgments of inpatient treatment teams?

9. If the DEPI R is unrelated to both the Depression scale of the Personality Inventory for Children, and the diagnostic judgments of inpatient treatment teams, this suggests problems with the DEPI R Correct?

10. You have not published any data in a peer reviewed journal necessitating that we reconsider the 1991 findings of Ball and his colleagues Correct?

11. You cannot cite any data published in a peer reviewed journal necessitating that we reconsider the 1991 findings of Ball and his colleagues Correct?

12. Scale 2 the Depression Scale of the MMPI 2 is a generally recognized and accepted instrument for assessing depression in an adult population Correct?

13. And the Beck Depression Inventory is also a generally recognized and accepted instrument for assessing depression in an adult population Correct?

14. In other words, if patients score low on Scale 2 fo the MMPI 2, we would expect them to score low on the Beck Depression Inventory Correct?

15. And if patients score high on Scale 2 fo the MMPI 2, we would expect them to also score high on the Beck Depression Inventory Correct?

16. The Journal of Clinical Psychology is a generally recognized and accepted, peer reviewed journal in your field Correct?

17. And a 1997 study by Carlson and her colleagues published in the Journal of Clinical Psychology titled “Rorschach Revised DEPI and CDI with Inpatient Depressives and Borderline Personality Disorder with Major Depression: Validity Issues” might be relevant to your opinions in this case Correct?

18. Carlson and her colleagues reported:

– [ read ] –

“… two other measures of depression, the MMPID [scale 2] and the BDI [Beck Depression Inventory], further validate the presence of depression and correlate highly with each other …”

Now my question: When scale 2 of the MMPI and the Beck Depression Inventory correlate highly with each other, that is to be expected Correct?

19. Carlson and her colleagues further reported:

– [ read ] –

“… two other measures of depression, the MMPID and the BDI further validate the presence of depression and correlate highly with each other, but not with the DEPI R.”

Now my question: If either the Beck Depression Inventory, or Scale 2 of the MMPI 2, do not correlate with the Rorschach DEPI R, it appears that the DEPI R is an uncorrelated measure of depression Correct?

20. In other words, we do not know what the DEPI R assesses other than knowing it does not assess depression as it is assessed by the Beck Depression Inventory and Scale 2 of the MMPI 2 Correct?

21. And the Beck Depression Inventory and Scale 2 of the MMPI 2 are both objective measures of depression Correct?

22. And compared to the Beck Depression Inventory and Scale 2 of the MMPI 2, the Rorschach DEPI R is a more subjective index Correct?

23. And the objectivity of the Beck Depression Inventory and Scale 2 of the MMPI 2, compared to the relative subjectivity of the Rorschach DEPI R, indicates we can place more confidence in the Beck Inventory and the MMPI 2 Correct?

24. You have not published anything in a peer reviewed journal necessitating that we reconsider the 1997 findings of Carlson and her colleagues Correct?

25. You cannot cite any data published in a peer reviewed journal necessitating that we reconsider the 1997 findings of Carlson and her colleagues Correct?

26. We previously established that the Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

27. And a 1993 study by Meyer published in the Journal of Personality Assessment titled “The Impact of Response Frequency on the Rorschach Constellation Indices and on Their Validity with Diagnostic and MMPI 2 criteria” might be relevant to your opinions in this case Correct?

28. And would it surprise you to know that Meyer also found no relationship between the DEPI R and the MMPI 2?

29. And if Meyer’s 1993 study found no significant relationship between the Rorschach DEPI R and the MMPI 2, that outcome further challenges the validity of the Rorschach DEPI R Correct?

30. Please consider Standard 2.02 (a) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(a) Psychologists who develop, administer, score, interpret, or use of psychological assessment techniques, interviews, tests, or instruments do  so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

Now my question: Your work in this case obligates you to comply with this standard Correct?

31. Without the availability of validity data supporting the DEPI R, your relying on that index neglects Standard 2.02 (a) of your ethical code Correct?

32. Because without the availability of validity data supporting the DEPI R, you cannot use that index in a manner that is “appropriate in light of the research on or evidence of the usefulness and proper application of the technique” Correct?

33. Also please consider Standard 2.02 (b) of the 1992 Ethical Code for psychologists addressing “Competence and Appropriate Use of Assessments and Interventions” which states:

– [ read ] –

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

Now my question: Your work in this case also obligates you to comply with this ethical standard Correct?

34. Without the availability of validity data supporting the DEPI R, your relying on that index neglects Standard 2.02 (b) of your ethical code Correct?

35. Because without the availability of validity data supporting the DEPI R, you are at risk for misusing this “assessment technique” and its “interpretations” Correct?

36. And without the availability of validity data supporting the DEPI R, your relying on that index could misinform and mislead this proceeding Correct?


 

[1]. Ball, J.D., Archer, R.P., Gordon, R.A. & French, J. (1991). Rorschach depression indices with children and adolescents: Concurrent validity findings. Journal of Personality Assessment, 57, 465 476.

[2]. Carlson, C.F., Kula, M.L. & St. Laurent, C. (1997). Rorschach revised DEPI and CDI with inpatient depressives and borderline personality disorder with major depression: Validity issues. Journal of Clinical Psychology, 53, 51 58.

[3]. Meyer, G.J. (1993). The impact of response frequency on the Rorschach constellation indices and their validity with diagnostic and MMPI 2 criteria. Journal of Personality Assessment, 60, 153 180.

 

 

 

 

 
 
Rorschach Validity General Considerations
Posted On December 8th, 2014  By admin  

As previously explained in prior posts, the value of any psychological test inevitably corresponds to its validity. The validity of a test corresponds to whether it actually assesses what it claims to assess. Moreover, it is necessary to clearly specify what a test is valid for. The Minnesota Multiphasic Personality Inventory (MMPI), for example, is valid for assessing various dimensions of personality. The MMPI, however, is not valid for assessing intelligence. We at Falsely-Accused.net teach that consequently, it is always mistaken to speak of a “valid test” without specifying for what it is valid. For more than five decades, challenge after challenge has been directed at the validity of the Rorschach. Concerned by the brevity of validity data to support the Rorschach, an otherwise favorable 1946 review confessed:

“Frankly, the reviewer would prefer to keep this material from the eyes of ‘academic’ psychologists until more nearly conclusive proofs of Rorschach validity are available … one must agree that the Rorschach is largely an art …” [1]

In his 1953 review of Beck’s Rorschach’s Test III: Advances in Interpretations, Lee Cronbach concluded:

“We have now had thousands of research studies, some well conducted, which have failed to establish validity of many interpretations commonly made [using the Rorschach] … We find Beck turning his back on research evidence in a way that is too common in the literature and in the conversations of clinicians, by saying, in effect, ‘Evidence or no evidence, these propositions [popular Rorschach interpretations] have clinical utility.”[2]

In 1959, Eysenck summarized his review of Rorschach validity data published during the five previous years. Eysenck concluded:

“There is no evidence that conscious or unconscious conflicts, attitudes, fears or fantasies in patients can be diagnosed by means of projective techniques in such a way as to give congruent results with assessments made by psychiatrists independently … There is ample evidence to show that the great majority of studies in the field of projective techniques are inadequately designed, have serious statistical errors in the analysis of the data, and/or are subject to damaging criticisms on the grounds of contamination between the test and criterion.”[3]

In 1965, Professor Dana suggested the time had come for Rorschach loyalists to acknowledge the many problems related to this technique:

“… there is continual pressure to demonstrate that the test meets the usual psychometric criteria of objectivity, reliability, validity, additivity, scaling procedures etc., which in fact it does not … In spite of our persistent attempts to convert the Rorschach into a psychometric instrument, we have failed.”[4]

Despite his admitting the Rorschach’s many shortcomings, Dana nonetheless advocated that there remained a place for it in clinical practice. Curiously enough, it appears that numerous psychologists continue to embrace this thinking almost 40 years later. “Conclusive proofs” of the Rorschach’s validity are as elusive as they were in 1946; and as a result, the Rorschach remains more “art” than science.

We at Falsely-Accused.net emphasize that since the 1940s, the Rorschach has more often than not failed to deliver on its promises. Now in the 21st century, the verdict is clear and evident Relying on the Rorschach to assess conscious and/or unconscious experience is ill advised at best, and illogical and ill informed at worst. Nevertheless, legions of mental health professionals continue to react as Cronbach described them in 1953 – “Evidence or no evidence,” my clinical experience persuades me that the Rorschach is a valuable assessment procedure.

We at Falsely-Accused.net consider, for example, the feeble position assumed by a psychologist as she responded to cross examination regarding the Rorschach’s validity.

Attny: What does Standard 6.1 say? [referring to the 1985 Standards for Educational and Psychological Testing in effect at that time].

Psych: “Test users should evaluate the available written documentation on the validity and reliability of tests for the specific use intended.”

Attny: Have you complied with that standard in the use of the Rorschach?

Psych: Yes, I have.

Attny: How?

Psych: By reading articles, going to conferences, consulting with colleagues. What I have said to you, several times, was that I do not catalog and remember what read by author, date, title, etcetera. I have satisfied myself through other means, including reading. I will bring with me, if we have to go trial, recent articles that I have read that say use of the Rorschach is perfectly legitimate if one chooses to do so. Do I have them on the top of my head to recite to you? No.

Despite this psychologist’s vehement protests, the fact remains that she neglected to comply with 1985 Testing standard 6.1 at the time she used the Rorschach in this case. Whatever material she cites in the future will not change this fact.

Evaluating the validity of Exner’s Rorschach (TRACS) necessitates assessing the validity of its many subscales. The DEPI (Depression) Index, for example, claims to identify clinically significant levels of depression. The SCZI (Schizophrenia) Index claims to discriminate between psychotic and non psychotic disorders. The validity of the (TRACS) Rorschach ultimately corresponds to the validity of these and other specialized indices. Without sufficient data demonstrating the validity of these indices, relying on them disregards Standards 2.02 (a b) of the 1992 Ethical Code for psychologists. Standards 2.02 (a b) addressing “Competence and Appropriate Use of Assessments and Interventions” state:

“(a) Psychologists who develop, administer, score, interpret, or use psychological assessment techniques, interviews, tests, or instruments do so in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.”

“(b) Psychologists refrain from misuse of assessment techniques, interventions, results, and interpretations and take reasonable steps to prevent others from misusing the information these techniques provide. This includes refraining from releasing raw test results or raw test data to persons, other than to patients or clients as appropriate, who are not qualified to use such information.”

_____________________________________________________________________________

[1] Harriman, P.L. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 146-147.

[2]  Cronbach, L.J. (1959). Review of Rorschach texts in O.K. Buros (Ed.), The Fifth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. Page 979

[3]  Eysenck, H.J. (1959). Review of Rorschach in O.K. Buros (Ed.), The Fifth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 276 – 277

[4]  Dana, R.H. (1965). Review of Rorschach in O.K. Buros (Ed.), The Sixth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. Page 495

 

 
 
Exner’s Rorschach System and Inter Rater Reliability
Posted On December 1st, 2014  By admin  

We at Falsely-Accused.net emphasize that the use of the Rorschach has long involved problems of inter rater reliability. Less than adequate levels of inter rater reliability indicate that two or more psychologists, independently scoring the same Rorschach responses, could score them quite differently. Such variations in scoring would then lead to very different interpretive conclusions based on the same Rorschach data. Without adequate levels of inter rater reliability, therefore, the evidentiary value of any psychological test is thoroughly compromised. The British psychologist, H.J. Eysenck commented on the Rorschach and its problems of inter rater reliability in 1959:

“In addition to the damning evidence regarding the validity of the Rorschach, it should perhaps also be pointed out that studies of the reliabilities of different Rorschach scores have shown these to be very low indeed.” (*61).

Psychologists using the Rorschach typically claim that Exner’s TRACS (The Rorschach: A Comprehensive System) has resolved the problems of inter rater reliability long associated with this technique. Rather than report levels of inter rater reliability, however, Exner reported the percentage of agreement between different psychologists using the TRACS. We at Falsely-Accused.net note that as a result, Wood and his colleagues emphasized: “It appears that the interrater reliability of the Comprehensive System [TRACS] is presently unknown” . Previous claims regarding the inter rater reliability of TRACS relied on unpublished studies not subjected to the scrutiny of peer review .

For research purposes, there are standards ensuring adequate inter rater reliability for the Rorschach. Since 1991, the Journal of Personality Assessment has required a minimum inter rater agreement of 80% for any Rorschach study submitted for publication . This level of research reliability, however, is not necessarily related to the Rorschach’s field reliability. The psychologists participating in these studies underwent specialized training to reach an acceptable level of inter rater agreement.

The Rorschach’s field reliability refers to how consistently it is scored by practitioners using it clinically rather than for research purposes. In fact, there are no peer reviewed data demonstrating how the TRACS has influenced the community practitioners who use it . In other words, the extent to which practicing psychologists carefully comply with the TRACS scoring procedures remains unknown. Because the TRACS involves 54 different ratios and percentages in its scoring procedure, we at Falsely-Accused.net feel that the potential for error is enormous.

A 1999 broad scale analysis of the Rorschach and the MMPI 2 concluded that both tests work equally well . This analysis, however, relied on the inter rater reliabilities for the Rorschach realized by research teams. Extrapolating from these data to assume that community practitioners score the Rorschach reliably is ill advised. Addressing the issue of “research reliability” versus “field reliability,” another 1999 article indicated:

“… the Comprehensive System is a complex administration and scoring system, and evidence of scoring reliability in research contexts would not automatically generalize to clinical practice. To develop and maintain proficiency in the Comprehensive System requires enormous time and effort. At present we have no evidence for the field reliability of system.”

This same 1999 article continued:

“Thus, without compelling evidence that a Rorschach protocol is properly administered and scored according to the rules of the Comprehensive System, the validity of the interpretation arising from the scoring cannot be assumed.”

Cross examining Exner’s Rorschach System and Inter Rater Reliability

1.  Your assessment work obligates you to comply with the 1999 edition of the Standards for Educational and Psychological Testing Correct?

2.  The 1999 Testing Standards specifically state:

- [ read ] -
“The applicability of the Standards to an evaluation device or method is not altered by the label applied to it (e.g., test, assessment, scale, inventory)”

And now my question: Given the statement I just cited from the 1999 Testing Standards, those Standards apply directly to your assessment work in this case Correct?

3.  Please consider Standard 2.10 of the 1999 Standards for Educational and Psychological Testing which states:

- [ read ] -
“When subjective judgment enters into test scoring, evidence should be provided on both inter rater consistency in scoring and within examinee consistency over repeated measurements.” (p. 33).

Now my question: Your work in this case obligates you to comply with this standard Correct?

4.  Evidence of inter rater consistency in scoring also refers to what is generally recognized and accepted in your field as inter rater reliability Correct?

5.  Psychologists using the Rorschach often claim that Exner’s system has resolved the problems of inter rater reliability Correct?

6.  In using the Rorschach in this case, did you rely on Exner’s TRACS or what is otherwise known as the Comprehensive System?
[If the psychologist answers no, establish what scoring system was used
and continue to ask the questions below].

7.  For research purposes, we know that the Rorschach can potentially be scored in a consistent manner Correct?

8.  Since 1991, the Journal of Personality Assessment has required a minimum inter rater agreement of 80% before publishing an article submitted to it Correct?

9.  But that requirement corresponds to what we might call the research consistency of the Rorschach Correct?

10.  The psychologists participating in those research studies underwent specialized training to reach acceptable levels of inter rater agreement Correct?

11.  What specialized training have you undergone to assist you in scoring the Rorschach in an acceptable manner?

[The vast majority of psychologists have not undertaken this kind of training.
Prior to cross examination, check the psychologist’s c.v. for post doctoral
training in the Rorschach]

12.  And the Afield reliability” of the Rorschach refers to how practicing psychologists such as yourself use it Correct?

13.  Can you cite any data published in a peer reviewed journal reporting the field reliability for the TRACS? [or whatever other system the psychologist used].

14.  In other words, the extent to which practicing psychologists carefully comply with the TRACS scoring procedures [or any other scoring procedure] remains unknown Correct?
[USE THE 22 FOUNDATIONAL QUESTIONS FROM CHAPTER ONE
IF YOU HAVE NOT YET DONE SO]

15.  The journal Psychological Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

16.  And a 1999 article by Hunsley and Bailey published in Psychological Assessment titled AThe Clinical Utility of the Rorschach: Unfulfilled Promises and an Uncertain Future” might be relevant to your opinions in this case Correct?

17.  Please consider these comments of Hunsley and Bailey from their 1999 article:

- [ read ] -
“… the Comprehensive System is a complex administration and scoring system, and evidence of scoring reliability in research contexts would not automatically generalize to clinical practice.”

Now my question: If evidence of Rorschach scoring reliability in research contexts does not automatically generalize to clinical practice, then it is inappropriate for you to cite data from research contexts suggesting that you score the Rorschach in a reliable manner Correct?

18.  Please consider these further comments expressed by Hunsley and Bailey in their 1999 article:

- [read] -
“At present we have no evidence for the field reliability of [the Exner] system.”

Now my question: You have not published any data in a peer reviewed journal necessitating that we reconsider the position of Hunsley and Bailey in this regard Correct?

19.  You cannot cite any data published in a peer reviewed journal necessitating that we reconsider the position of Hunsely and Bailey in this regard Correct?

20.  Without the availability of peer reviewed data indicating otherwise, your profession generally recognizes and accepts that there are no data supporting the field reliability of the Exner scoring system Correct?

21.  Please consider these additional comments from Hunsley and Bailey’s 1999 article.

- [ read ] -
“Thus, without compelling evidence that a Rorschach protocol is properly administered and scored according to the rules of the Comprehensive System, the validity of the interpretation arising from the scoring cannot be assumed.” (6).

Now my question: You cannot provide this proceeding with “compelling evidence” demonstrating that you properly administered and scored the Rorschach in this case Correct?

22.  And if you cannot demonstrate via “compelling evidence” that you administered and scored the Rorschach properly in this case, then it is inappropriate for us to assume the validity of your Rorschach interpretations Correct?
[ASK THE FOLLOWING QUESTIONS ONLY IF THE PSYCHOLOGIST HAS NOT USED A COMPUTERIZED INTERPRETATION OF THE RORSCHACH]

23.  The TRACS includes 54 different ratios and percentages in its scoring procedure Correct?

24.  And because the TRACS includes includes 54 different ratios and percentages in its scoring procedure, the potential for error is enormous Correct?

25.  Given the burden of scoring 54 different ratios and percentages, do you really think you did it with no errors whatsoever?

26.  But you cannot cite any data published in a peer reviewed journal demonstrating that there is a substantial likelihood of your scoring these 54 different ratios and percentages in an accurate manner Correct?

 ________________________________________________________________________

1.  Eysenck, H.J. (1959). Review of Rorschach in O.K. Buros (Ed.), The Fifth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press:
“most of the alleged verifications of Rorschach hypotheses are achieved only through the admission of uncontrolled sources of error.” page 277

2.  Percentage of agreement may be a misleading concept: Jensen provided an important look at the basis for some of the confounding research results promulgated by the Rorschachers:

“A word of caution concerning improper estimates of Rorschach reliability: these often consist of reporting the percentage of agreement between two or more judges. It should be clear that percentage agreement is not a legitimate measure of reliability and tells us none of the things we want to know when we ask about the reliability of a test. What we want to know is the proportion of variance in the scores that is not error variance. The reliability coefficient tells us this; the percentage agreement does not.”

Jensen, A.R. (1965). Review of Rorschach in O.K. Buros (Ed.), The Sixth Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. Page 501 page 504

3.  Wood, J.M., Nezworski, M.T. & Stejskal, W.J. (1996). Thinking critically about the Comprehensive System for the Rorschach: A reply to Exner. Psychological Science, 7, 14 17, at page 14.

4.  Wood, J.M., Nezworski, M.T. & Stejskal, W.J. (1996). The Comprehensive System for the Rorschach: A Critical Review. Psychological Science, 7, 3 -10.

5.  Hunsley, J. & Bailey, J.M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future. Psychological Assessment, 11, 266 277.

6.  Id.

7.  Hiller, J.B., Rosenthal, R., Bornstein, R.F., Berry, D.T. & Brunell Neuleib, S. (1999). A comparative meta analysis of Rorschach and MMPI validity. Psychological Assessment, 11, 278 296.

8.  Hunsely, J. & Bailey, J.M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future. Psychological Assessment, 11, 266 277 (p. 272).

9.  Id.

10.  American Psychological Association (1999). Standards for Educational and Psychological Testing. Washington, D.C.: Author.

 

 
 
The Rorschach and Supplementary Information
Posted On November 24th, 2014  By admin  

At Falsely-Accused.net we feel that any psychologist who advocates using the Rorschach for assessment purposes must identify its legitimate uses. It is obviously inappropriate to use the Rorschach for any and all assessment tasks. It becomes necessary to ask, then, for what assessment endeavors might the Rorschach be appropriate? Meyer characterized Stricker and Gold’s 1999 article as advocating:

“… the Rorschach is a valuable instrument for clinicians (and researchers) when the goal is to understand a person in his or her full complexity, particularly when clinicians rely on a sophisticated, theoretically driven synthesis of Rorschach data in conjunction with other sources of information.”

 

Meyer’s comments raise the question of exactly what other sources of information warrant consideration. There are potentially an infinite number of information sources about a client that one might review. The issue therefore is what sources of information are most relevant to the assessment questions at hand? Selecting the most appropriate sources of information for supplementing Rorshach data inevitably relies on clinical judgment. Relying on clinical judgment to identify appropriate sources of information characteristically leads to substantial variations of opinion between two or more psychologists (see previous posts for a review of clinical judgment). In other words, there is no standardized procedure available for identifying what sources of information should be used to supplement Rorschach data.

In their 1999 article, Stricker and Gold nonetheless commented favorably on a 1990 review of Kleinmuntz advocating:

“… an approach toward assessment that integrates one’s cognitive awareness of the statistical meanings of data with an intuitive, clinically based approach. This integration enriches the assessment and allows the limitations of both perspectives to be corrected.”

Attempts at integrating statistical data and intuitive impressions drive psychologists into the chaotic abyss of clinical judgment. Relying on such “intuitive” impressions is tantamount to overriding actuarial data with case specific information. Rather than reduce error, resorting to intuitive impressions in this manner typically increases it. As pointed out in previous posts, the accuracy of statistical data (or actuarial methods) exceeds intuition (clinical judgment) by a margin of eight to one.

At Falsely-Accused.net we note that the Rorschach can be interpreted via a “nomothetic” or “ideographic” approach. The nomothetic approach relies on presumably objective data detailing supposed relationships between Rorschach responses and personality characteristics. Exner’s TRACS (The Rorschach A Comprehensive System) is an attempt at using the Rorschach in a nomothetic manner. The ideographic approach to Rorschach interpretation is theoretically driven, and specific to the particular person taking it. In a 1998 article, Smith discussed the differences between the nomothetic and ideographic approaches.

“We score a Rorschach protocol, compute various ratios and indexes from these scores, and draw inferences about the person based on the empirical correlates of those scores [nomothetic approach]. Or we look at the sequence ogical constructs based on our theoretical propositions, and derive inferences about the person from those constructs [ideographic approach].”

 

At Falsely-Accused.net we teach that ideographic interpretations of the same Rorschach can vary substantially depending on who is doing the interpretation. To the extent a psychologist relies on ideographic interpretations of the Rorschach, it is no longer a nomothetic instrument. Instead, it is an instrument relying on speculation and guesswork.

In a 1998 article, Handler advised against what he termed “reductionistic” (or nomothetic) interpretations of Rorschach responses involving color. Psychologists using the Rorschach assume how people respond to the colors of the cards reveals information about their emotional experiences. Handler specifically wrote:

“… not every C [color] response is the same. The response of fire or blood to a red area is quite different from , for example, the following response: ‘This looks like a bit of sky, the pale blue you sometimes see early in spring.’ Both of these responses are C [color], but the second response has an aesthetic emphasis in its subtlety. It lacks the primitive impulsiveness and possible helplessness of the former response. Aesthetic appreciation, the focus on subtlety, and the focused elaboration all speak to a higher level of development, compared with the first response.”

Handler therefore recommends ideographic interpretations of Rorschach responses. In doing so, he invites psychologists to indulge in the kind of conjecture and speculation that ultimately amounts to guesswork.

Cross examining The Rorschach and Supplementary Information
[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS IF YOU HAVE NOT YET DONE SO]

1. The journal Psychological Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

2. And a 1999 article by Meyer published in Psychological Assessment titled “Introduction to the Special Series on The Utility of The Rorschach for Clinical Assessment” might be relevant to your opinions in this case Correct?

3. Please consider Meyer’s comments regarding the value of the Rorschach:

“… the Rorschach is a valuable instrument for clinicians (and researchers) when the goal is to understand a person in his or her full complexity, particularly when clinicians rely on a sophisticated, theoretically driven synthesis of Rorschach data in conjunction with other sources of information.”

Now my question: If it is necessary to synthesize Rorschach data in conjunction with other sources of information, we are considering supplementary information Correct?

4. And in this case, you considered supplementary information when interpreting the Rorschach data you obtained Correct?

5. There are potentially an infinite number of information sources that a psychologist might review for supplementary purposes Correct?

6. The issue therefore becomes what sources of supplementary information are most relevant for a particular assessment Correct?

7. Selecting the most appropriate sources of information for supplementing Rorschach data relies on clinical judgment Correct?

8. And relying on clinical judgment to identify appropriate sources of supplementary information can lead into substantial variations of opinion between two or more psychologists Correct?

9. Psychologist A might want to supplement the Rorschach with the MMPI 2, but psychologist B wants to supplement the Rorschach with the MCMI III Correct?

10. In other words, there is no standardized procedure available for identifying what sources of information should be used to supplement Rorschach data Correct?

11. Because there is no standardized procedure available for identifying what sources of information should be used to supplement Rorschach data, your use of supplementary information threatens to misinform and mislead this proceeding Correct?

12. Your testimony threatens to misinform and mislead this proceeding because other psychologists could have relied on supplementary information quite different than yours, and reached very different conclusions as a result Correct?

13. The Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

14. And a 1998 article by Smith published in the Journal of Personality Assessment titled: AThe Rorschach Experience: Comments on articles by Handler, Leichtman, Lerner, and Peterson and Sayer” might be relevant to your opinions in this case Correct?

15. Please consider Smith’s comments from his 1998 article:

“We score a Rorschach protocol, compute various ratios and indexes from these scores, and draw inferences about the person based on the empirical correlates of those scores. Or we look at the sequence of responses and the precise verbalizations, translate those verbalizations into psychological constructs based on our theoretical propositions, and derive inferences about the person from those constructs.”

Now my question: When Smith refers to computing “various ratios and indexes from these scores, and draw inferences about the person based on the empirical correlates of those scores” he is referring to a nomothetic interpretation of the Rorschach Correct?

16. And when Smith refers to looking:

“at the sequence of responses and the precise verbalizations, translate those verbalizations into psychological constructs based on our theoretical propositions, and derive inferences about the person from those constructs” he is referring to an ideographic interpretation of the Rorschach Correct?

17. Psychologists can also interpret the Rorschach using both nomothetic and ideographic approaches Correct?

18. In this case, did you interpret the Rorschach using both nomothetic and ideographic approaches?

[If the psychologist says he or she relied only on a nomothetic approach, then ask the following question:  Another psychologist assessing the parties in this case could have used both nomothetic and ideographic approaches Correct?]

19. Ideographic interpretations of the same Rorschach can vary substantially depending on who is doing the interpretations Correct?

20. In other words, if Psychologist A, and Psychologist B, and Psychologist C all undertake ideographic interpretations of the same Rorschach, there can be substantial differences in their interpretations Correct?

21. We previously established that the Journal of Personality Assessment is a generally recognized and accepted, peer reviewed journal in your field Correct?

22. And a 1998 article by Handler published in the Journal of Personality Assessment titled “Schactel and teaching: What color was the couch?” might be relevant to your opinions in this case Correct?

23. Please consider Handler’s opinions regarding color responses on the Rorschach:

“… not every C [color] response is the same. The response of fire or blood to a red area is quite different from , for example, the following response: ‘This looks like a bit of sky, the pale blue you sometimes see early in spring.’ Both of these responses are C [color], but the second response has an aesthetic emphasis in its subtlety.”

Now my question: When Handler says “not every color response is the same,” he is advocating ideographic interpretations of the Rorschach Correct?

24. And to the extent any psychologist relies on ideographic interpretations of the Rorschach, it is no longer a nomothetic instrument Correct?

25. And using the Rorschach in a ideographic manner involves the kind of conjecture and speculation that amounts to guesswork Correct?

 
 
Reliance on Psychoanalytic Theory
Posted On May 15th, 2014  By admin  

The Rorschach has been associated with psychoanalytic theory beginning with Hermann Rorschach himself. Hermann Rorschach identified closely with the classical psychoanalytic theory of Sigmund Freud which emphasized assumptions of “infantile sexuality”, “penis envy”, and “anal eroticism”[1]. In 1946 Super’s comments regarding the Rorschach indicated: “Psychoanalytic interpretations are frequent and glibly incorporated into other statements with no citation of evidence to justify them.”[2] Also in 1946, Rabin addressed the close relationship between the Rorschach and psychoanalytic theory:

“… the striking psychoanalytic and otherwise symbolic explanations of various answers and bits of independent content are sometimes over‑dogmatic and not readily digestible …”[3]

We at Falsely-Accused.net note that psychologists who currently use the Rorschach frequently align themselves with one or more of the many variations of psychoanalytic theory. Stricker and Gold, advocates of the Rorschach, suggest that it is compatible with theoretical orientations other than psychoanalytic. Nevertheless, they also indicated:

“The affiliation of between the Rorschach and psychoanalytic theory is both an accident of history (the technique was developed and nurtured in a psychoanalytic environment) and a result of the powerful fit between the theory and the type of data that the technique provides” [4]

Stricker and Gold further indicated:

“On the broad spectrum of sensitivity, the Rorschach is uniquely suited for understanding the underlying organization of the personality and for generating hypotheses about unconscious contents of the mind.”[5]

Unbridled enthusiasm for exploring unconscious experiences too often leads to orgies of conjecture and speculation. Such enthusiasm also passionately embraces psychoanalytic theory. We at Falsely-Accused.net feel that as a result, it is necessary to ask: How does one verify the accuracy with which the Rorschach, or any other assessment instrument, assesses unconscious levels of experience? By definition, we do not have direct access to unconscious experiences. Therefore, we can never accurately identify them because they exist outside our awareness while rapidly shifting and changing. At Falsely-Accused.net we feel that consequently, attempting to identify unconscious experiences is forever limited to conjecture and speculation.

Cross‑Examining the Rorschach and Psychoanalytic Theory

[USE THE 22 FOUNDATIONAL QUESTIONS FROM PREVIOUS POSTS

IF YOU HAVE NOT YET DONE SO]

1.       The journal Psychological Assessment is a generally recognized and accepted, peer‑reviewed journal in your field ‑‑ Correct?

2.       And an 1999 article by Stricker and Gold published in Psychological Assessment ‑ titled: “The Rorschach: Toward a Nomothetically Based, Idiographically Applicable Configurational Model” ‑ might be relevant to your opinions in this case ‑‑ Correct?

3.       Please consider Stricker and Gold’s comments about the Rorschach:

- [ read ] -

“On the broad spectrum of sensitivity, the Rorschach is uniquely suited for understanding the underlying organization of the personality and for generating hypotheses about unconscious contents of the mind.”

         Now my question: If the Rorschach is “uniquely suited” for “enerating hypotheses about the unconscious contents of the mind,” then using the Rorschach embraces one or more variations of psychoanalytic or Freudian theory ‑‑ Correct?

4.       Our unconscious experiences exist outside of our awareness ‑‑ Correct?

5.       And our unconscious experiences can rapidly shift and change ‑‑ Correct?

6.       Therefore, we can never accurately identify experiences that are both outside our awareness, and which can rapidly shift and change ‑‑ Correct?

7.       Consequently, it becomes impossible to verify the accuracy with which the Rorschach assesses unconscious levels of experience ‑‑ Correct?

8.       Attempting to identify experiences outside our awareness, that rapidly shift and change, invites conjecture and speculation ‑‑ Correct?

9.       Psychologist A and psychologist B can therefore arrive at very different interpretations of the same person’s unconscious experiences ‑‑ Correct?

10.     And when psychologist A and psychologist B arrive at their different interpretations of someone’s unconscious experiences, we do not know who is right ‑‑ Correct?

11.     And when psychologist A and psychologist B arrive at their different interpretations of someone’s unconscious experiences, they can both be mistaken ‑‑ Correct?

12.     And given mistaken interpretations of someone’s unconscious experiences, those errors threaten to misinform and mislead this proceeding ‑‑ Correct?

[The cross‑examining attorney may also want to follow‑up these questions with the questions addressing psychoanalytic theory found in Chapter Four]

 

[1].       Meissner, W. (1985). History of Psychiatry in Kaplan, H. and Sadock, B. (Eds.) Comprehensive Textbook of Psychiatry IV, New York: Williams & Wilkins. page 358-359. See also: Freud, S. (1905). Three essays on the theory of sexuality. The Standard Edition of the Complete Psychological Works of Sigmund Freud. Vol. 7  London: Hogarth.; Freud, S. (1910) Five Lectures on psycho-analysis. The Standard Edition of the Complete Psychological Works of Sigmund Freud. Vol. 11 London: Hogarth.; Strachey, James (Ed.) In collaboration with Anna Freud (1953 – 1964). The Standard Edition of the Complete Psychological Works of Sigmund Freud. 24 vols. London: Hogarth.

Id. and see also: Freud, S. (1916-17). Introductory lectures on psycho-analysis. The Standard Edition of the Complete Psychological Works of Sigmund Freud. Vols. 15-16 London: Hogarth; and Campbell, R.J. (1981) Psychiatric Dictionary, 5th Ed. New York: Oxford University Press :

“Freud says that when the little girl realizes she has no penis, she reacts either by hoping that some day she will have one or by denying that she does not have one……Penis-envy is responsible for ‘a loosening of the girl’s relation with her mother as a love-object,’ because the girl blames her mother for the alleged loss.” page 220.

  1. and see also: Campbell, R.J. (1981) Psychiatric Dictionary, 5th Ed. New York: Oxford University Press :

“In psychoanalytic theory, the earliest concentration of libido is in the oral zone; with further development, most of it shifts to the anal region……” page 32.

[2].       Super, D.E. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 158

[3].       Rabin, A. (1949). Review of Rorschach texts in O.K. Buros (Ed.), The Third Mental Measurements Yearbook. Highland Park, NJ: Gryphon Press. page 159

[4].       Stricker, G. & Gold, J.R. (1999). The Rorschach: Toward a nomothetically based, idiographically applicable configurational model. Psychological Assessment, 11, 240-250, page 243..

[5].       Id. at page 244.

 
 
 
 
 
 
By Gardner, Sauber, and Lorandos, has become the standard reference work for PAS. The International Handbook features clinical, legal, and research perspectives from 32 contributors from eight countries.
 
The International Handbook of
Parental Alienation Syndrome
 
By Terrence W. Campbell and Demosthenes Lorandos, is a must for every family law practitioner. This two-volume practice set provides step-by-step guidance how to refute behavioral scientists.
 
Cross Examining Experts in the
Behavioral Sciences
 
By Lorandos and Campbell, provides immediate access to authoritative information and immediate decision-making tools for judges and attorneys.
 
Benchbook in the Behavioral Sciences