Manualized or Flexible Treatment? A case example and commentary

Tsvi E. Gil, B.Sc., M.A.

Address for correspondence: Tsvi Gil, POB 3626, Haifa, Israel, 31036

Keywords: Psychotherapy (dynamic, short-term), Depression, Schizoid personality, Manualized treatments (treatment guideline).

Abstract: The paper discusses the issue of manualized treatments in rather a critical view. While the manulaized treatment (or treatments via guidelines) approach received wide recognition as the `state of the art`, an `evidence based psychotherapy`, and acquired a formal credit from the American NIMH as the only treatment design that merits research budgeting, a large critic had been aroused from clinicians and researches in the field. The article briefly reviews these critics, and illustrates the debate throughout a concise case presentation. This case deems to demonstrate the benefits and advantages of a short-term, non-manualized but theoretically-driven though flexible therapeutic approach, when compared to the seemingly narrowly-perceived and goal-restricted manualized approach.

Dedicated to beloved Anna

Manualized or Flexible Treatment? A case example and commentary

Manualized treatment can be briefly defined as a treatment which is conducted according to a written manual. Such a manual dictates the therapist's behavior and the treatment according to anticipated lines. Hence the therapist is not free to use whatever interventions that come to his or her mind, but rather bring into action, in the proper occasions those kind of interventions recommended by the manual. While some cognitive-behavioral treatments for specific syndromes can be taken as examples for manualized treatments, a dynamic short-term psychotherapy can only be regarded as sloppy; conversely, psychoanalysis, humanistic psychotherapy, or client-centered counseling, are examples of non-manualized treatments, since the therapists in these approaches are encouraged to use their imagination, inner world, and creativity. The unexpectedness of the course of such treatments comprises a central feature in their nature.
The American Institute of Mental Health (NIMH) policy to fund only manualized treatments that target DSM disorders, as well as the American Psychological Association (APA) explicit relying on manualization as a major criterion in qualifying empirically supported treatments, have been largely criticized. However the justifications for those policies sounds convincing, when considering the history of psychotherapy research.
In the beginning, treatments were reported by their clinicians, without any trial for objectification by external agents or evidence. A treatment was judged to be successful when the therapist claimed it to be so. Assessment of the problem, course planning, and outcome evaluation, had been executed according to the therapist's theoretical paradigm.
Following H. Eysenck`s (1952) critics, as well as the criticism emanating from behavioral therapy and from academic psychology, the turn in psychotherapy research has demanded more `scientific` method – similar to medical, psychiatric, and psychopharmacological methodologies. The last word reached is, then, the NIMH policy that tries to make psychotherapy as scientific as possible, putting aside vague and impressionistic descriptions of treatments. When following the `manualized treatments` policy, the first step in a methodical treatment is a clear and objective diagnosis executed according to the Diagnostic Manual (DSM). The subject for treatment is chosen according to objective procedures, not personal preference, while other subjects with the same diagnosis are directed, through the same procedure, to control treatments (either waiting list or a default treatment). This is similar to control groups in pharmacological trials, which include other medications with known activity, or placebo. Unfortunately, there is still no way to make the therapist blind to the nature of the treatments he or she is going to conduct – unlike the physician who is blind to the composition of the pill he or she gives the subject. The treatment itself is being held through clear procedures and written manuals, so the therapist, as well as an objective observer, can judge at any moment what is being done, and why. Duration and course of timing in treatment are controlled, thus the end of treatment is predictable and expected. When manual controlled techniques are used, results are measured by objective means, such as the use of unbiased observers and questionnaires. The results of the treatments are measured against the original diagnosis, which facilitate a cure or at least significant relief for the patient. Achievements are expected to be long-term, as evidenced by follow-up procedures. And finally, manual controlled treatment procedures and results are expected to be reproducible when applied by other clinicians.
The manualized treatment policy is today presented as the current state-of-the-art in the field of psychotherapy (Hill, O`Grady & Elkin, 1992). This policy influences what is taught in training programmes, what is permitted for continuing education, and perhaps most serious of all, as Drozd and Goldfried (1996) remark, what is allowed by managed care and third party payers.
Support for manualized approach is widespread. Manualized treatments allow therapists to standardized techniques, discriminate between alternative approaches, and evaluate the adherence and competence with which treatments are administered in controlled clinical trials (Luborsky & Barber, 1993; Luborsky & DeRubeis, 1984; Waltz, Addis, Koerner & Jacobson, 1993). Manuals rely on research findings (rather than on clinical subjective impressions), and can also provide theoretical frameworks, concrete descriptions of therapeutic techniques, and case examples (Lambert & Ogles, 1988; Wilson, 1996). “Advocated as a way of providing the clearly specified psychological treatments required for the identification of empirically supported treatments manualization has been praised by many as a way of ensuring that therapies are applied in an accurate and replicable way, even when administered by different clinicians to different clients in different geographical settings (Malik et al 2003).
It would seem to be difficult to raise objections to such a scientific and objective approach; but such objections have been widely raised. One such objection to manualized treatment is on the issue of uniformity and homogeneity. Manualized treatment assumes uniformity and homogeneity of all patients who receive a specific DSM diagnosis, regardless of their personality, personal and socio-cultural characteristics, and different possible aetiologies of their illness. According to the manualized treatment approach, all patients receiving a particular diagnosis were viewed as having the same problem, differing only in severity (Persons, 1991). Therefore, this kind of treatment treats the disorder – not the client (Luborsky & Barber, 1993). However, the practicing clinician does not meet the ‘average’ patient (whose existence is doubtful) but a particular patient with his or her particular characteristics (Howard, Krause & Vessey 1994). In research settings many patients are omitted from treatment protocols because they do not meet study design criteria: but those patients do deserve treatment, and they do receive it from the practicing therapist, who does not have the privilege, nor the will, to reject applying patients because they do not display ‘pure’ diagnosis or particular characteristic. Properties of academic studies differ fundamentally from those characterize applied psychotherapy, which are: (a) flexible duration – treatment lasts until patient improves or terminates treatment, not by in advance imposed length; (b) psychotherapy is self-correcting and does not follow rigid and dictated course; (c) patients typically chose their therapist and participate in determining treatment setting, rather than being allocated to a dictated therapist and setting; (d) patients exhibit diffuse complaints and symptoms, rather than a coherent set of distinguished syndrome; and lastly, (e) treatment goals are mutually agreed on by therapist and patient, and flexibly modified throughout treatment progress, rather than determined in advance due to theoretical assumptions (following Seligman, 1995). Some authors (for example, Fensterheim & Raw 1996, Goldfried & Wolfe, 1996) even went further in suggesting that patients in clinical practice and subjects of academic treatment studies derive essentially from different populations.
The focus on diagnosis is not random – it originates out of the medical model. However in medicine, as Persons (1991) correctly points out, diagnoses are intended to reflect known or hypothesized underlying pathogenic mechanisms. The presumed pathophysiology is the basis and the justification for the treatment technique. On the other hand, in psychiatry, and especially in the American DSM, diagnoses are not based on such underlying mechanisms, but rather there is attempt to describe problems at the symptom level, and explicitly avoid any discussion of underlying mechanisms. Additionally, `current categories are based on committee consensus, historical precedent, and political argument and opinion` (Mulder, 2002), and current diagnostic categories in the field of psychiatry still lack reliability and validity (Mojtabai and Rieder, 1998). For those reasons it is much less reasonable to base treatment in psychiatry on diagnosis. Indeed, current theories of psychotherapy do not base treatment on diagnosis, but rather on the underlying psychological mechanisms hypothesized by the theory. The design of psychotherapy outcome research, which is a major constituent of the manualized approach, may be conceptually incompatible with the models of psychotherapy evaluated in those studies. This is so because outcome studies deal with patients who receive standardized treatments that are assigned on the basis of their psychiatric diagnoses, rather than having individualized treatments based on theory-driven psychological assessment of the individual's difficulties (Persons, 1991).
Another important argument opposing the manualized approach is the observation that following manuals does not guarantee competent and successful interventions (Crits-cristoph 1993), that the manualized treatment approach focuses on clinician adherence rather than clinician competence (Wampold, 1997), and in general, "treatment can be delivered but therapy does not necessarily take place" (Henry, Strupp, Butler, Schacht & Binder, 1993). The emphasis on hard ‘scientific’ data tends to devalue sophisticated clinical expertise, largely derived experience, and the detailed study of individual patients. The value and the role of the therapist is being undermined, the psychological and social aspects of the curative profession are neglected with the danger of creating a new utilitarian orthodoxy (Williams and Garner 2002). Therapist’s personality, qualities and shortcomings, and attitudes are not taken into account. The therapist who strives to follow a manualized treatment may neglect other aspects of his therapeutic behavior, such as being fluid, dynamic, and flexible with his or her on-going decision-making (Campbell, 1996). Such adherence to prescribed modality of treatment may have the adverse consequence of decreases in quality of generic therapist functioning (Henry et al 1993a, 1993b), and deterring the establishment of rapport, or therapeutic alliance, between patient and therapist (Henry et al, 1993). This rapport is an element of the treatment which is perceived to be essential for the treatment success (Hovarth & Greenberg, 1994). While conventional medicine afraid of loosing its humanity, psychiatry seems to strive to get rid of it. It seems that we forget Osler’s advice, that while medicine has to be rooted in science, the doctors should be healers (Bliss 1999). Psychotherapy in general, as Strupp & Anderson (1997) claim, and the therapeutic relationship in particular, represent a very broad and multifaceted social influence. These authors are troubled by the threat (which probably had already become a reality!) that the skillful, theoretically sophisticated therapist is being replaced by technicians with very limited training and expertise. They conclude that "manuals can only minimize variability at the expense of other essential therapeutic phenomena". A manualized treatment approach typically focuses on what the therapist does (i.e., therapist's interventions), but indeed treatment process does not flow in one direction – from therapist to patient – but the nature of the therapeutic process is inherently interactional; patient and therapist mutually affect each other and the process outcome (Ablon & Jones 2002). From an Intersubjective perspective, as R. Stolorow points out ( Stolorow, 1999), the center of all psychological phenomena, as well as of psychotherapy, is the human interaction. Unfortunately this interaction may be put aside and even eliminated for the sake of an `effective` ensuing of a manualized treatment. A good therapist is one who is willing and able to alter his or her therapeutic style and setting according to the patient's needs. On the other hand, therapists who adhere to manualized, `pure form` treatments may overlook the accumulated experience and wisdom of other therapies and therapists which may benefit in the particular treatment. Formulating differently, eclectic and integrative approaches, and the accumulated knowledge of a century of practicing psychotherapy (of all kinds), are put aside for the sake of the pure form, manualized treatment (Norcross & Goldfried, 1994). Strupp & Anderson (1997) warn that "further emphasis on the standardization of `techniques` and manualized control of the `therapist factor` may impede, rather than further the advancement of productive models for the therapist training". Stricker et al (Stricker, Abrhamson, Bologna, Hollon, Robinson & Reed, 1999) emphasize that determining the goal of the treatment has a significant influence on its cost/effectiveness evaluation: some goals, as personal development or enhanced quality of life, may not be covered by the benefit plan. A cost/effectiveness approach is not negative in and of itself – but the patient should be informed in advance that such an approach is in action, so the he or she can make an informed decision about the treatment. As Stricker et al (1999) note, "It is unfair and potentially fraudulent to provide limited service, either through explicit or implicit organizational policies, and then to present it to the public as comprehensive care". Or, as J. Schwartz puts it (Schwartz, 1997), "What about growth?… what about the subtle symptoms of neglect or destructiveness that warp personality and relationships but don't reach the level of axis I diagnosis?".
Objections had been raised also on an empirical basis. For example, for depression, which won the most established and documented manualized treatment (Elkin, 1994; Elkin, Parloff, Hadley & Autry, 1985), it was found that manualized treatments are not more effective than non-manualized treatments (Robinson, Berman & Neimeyer, 1990). The methodological line that establishes a manualized treatment overlooks alternative lines for collecting relevant information, such as naturalistic inquiry, case material and accumulated personal experience, (Williams and Garner 2002). While homogeneity in group composition and uniformity of treatment manuals is necessary in order to assess and compare treatment outcomes, the result may then be an exclusion of other interesting and potentially important effects and qualities of the treatment – “some of which may be important to understand the phenomenon of psychotherapy” (Wampold 1997). The manualized treatment approach understandable efforts to homogenize and standardize treatment may put aside variations which are the core of the therapeutic process. Small but meaningful reports are put aside in favor of large studies which exhibit the required results. Only factors that can be measured are recognized as important (Williams and Garner 2002). In other words, quantitative research is being favored over qualitative. Such approach “omits too many crucial elements of what is done in the field”. (Seligman 1996). The type of research that is typical and actually essential for identifying an effective treatment manual is one that emphasizes homogeneity (in patients and therapists characteristics and the interventions performed) and therefore leads to dilution of the most important ingredients of successful therapies; “In other words, the emphasis on evaluating treatment strategies has led to a neglect of the contribution of the qualities of the clinician, the nature of the therapeutic relationship, and the intricacies of clinical judgments necessary to determine hoe best to respond to client’s concerns” (Dobson et al, 1999, responding to Garfield, (1998), Henry (1998), and Wampold (1997)).
Critic was also raised concerning the methodology of treatment evaluation claiming that declaring a particular protocol was in action does not necessarily imply that the treatment in action differed from another treatment which followed another protocol. Non-specific factors, leaning on the therapist knowledge, in-situ behaviors, clinical skill, and accumulated experience, rather than specific factors consist of accurate manualized interventions, may contribute to the therapeutic sequels, yielding misleading conclusion as for the value and usefulness to the particular treatment. In Ablon and Jones (2002) words, "Relying on brand names of therapy can be misleading… the basic premise of controlled clinical trials (i.e., that the compared interventions represent separate and distinct treatments) may not have been met". This is so because "…the nature of the process fostered by therapists administering (one kind of treatment) was so similar to that fostered by therapists administering (another kind)". And Wampold (1997) ironically comments that “comparative studies are designed to detect differences due to unique ingredients, but when differences are found, inequivalence of common ingredients was invoked as an alternative explanation, elevating the importance of common ingredients”. Patients who participate and cooperate in manualized treatments do not necessarily represent the typical patient who attend psychotherapy (Williams and Garner 2002), and therefore generalized conclusions are dubious. While treating their identified-diagnosis the treatment tends to ignore other possible diagnoses (many patients have more than a single diagnosis, typically suffer from personality disorder), their personal and social circumstances. The cost and pragmatics of conducting long-term psychotherapy studies make most treatment trials unrepresentatively brief (Dobson et al, 1999; Gabbard and Freedman 2006). Therefore a reported efficacy in a manualized treatment trial does not guarantee effectiveness in the natural conditions in the clinical ‘field’ (Williams and Garner 2002).
Trials to found empirical-based manualized treatments won the critics that while most psychotherapies prove superiority to no-treatement condition (see for example Lambert & Bergin 1994), difference found in their efficacy are mostly very slight, making it irrelevant to base a preference of a particular treatment to another (Robinson et al 1990). Another argument was that therapist and even researcher’s therapeutic inclination has a strong influence on the treatment results, even in well controlled studies, and when allegiance of the researcher to the treatment was controlled differences in efficacy among classes of treatments disappeared (Robinson et al 1990, Hollon & Beck, 1994).
The issue of manualized treatments had been largely (if not exclusively) originated from academic psychology, creating a schism with clinical psychology, the latter seems the academic proposals to render psychotherapy efficiency as irrelevant to psychotherapy as being practiced in natural settings, contrasted with psychological laboratories in artificial experimental settings. As Fensterheim and Raw (1996) titled their article, “Psychotherapy research is not psychotherapy practice”. Nezu (1996) had argued that even among the most homogeneous groups of patients and therapists “a therapy cookbook does not and probably cannot exist”, and quotes Hersen (1981) that “complex problems require complex solutions”. Persons and Silberschatz (1998) claim that randomized controlled trials are powerful tool for assessing circumscribed, highly specified procedures, but not psychotherapy.

Discussion
It is understandable, natural and even inevitable, that people who apply for psychotherapy would ask for a removal or at least a significant relief of the symptoms which brought them to the therapy in the first place. From a point of view of patient's rights it was claimed that "the patient is entitled to relief from pain, anxiety and depression in the shortest time possible and with the least intrusive intervention" (Cummings & Sayama, 1995). The question in issue is whether judgment of such consequence of the treatment is sufficient measure of success. Should we put aside dimensions such as work productivity, studies performance, care for family, marital satisfaction and quality of life (see Frisch, Cornell, Villanueva & Retzlaff, 1992; Krupnick & Pincus, 1992; Silverman, 1996) even though the client did not complain about them in his initial interview?
Cummings (1999) claims that "treatments protocols work… they are both effective and efficient, bringing relief from pain, depression, and anxiety in the shortest time possible and with the least intrusive intervention… the therapist's worth is measurable…by the ability to heal, not by elegance of phraseology". But other approaches were proposed and may yield to be relevant. J. Persons (1991), for example, proposes that instead of manualized treatment approach, based on DSM formal diagnosis, an empirical, hypothesis-testing approach should be conveyed. The treatment is then being preceded by a formulated assessment. The case is being re-assessed and reformulated thorough treatment, interventions taken being shaped accordingly. The treatment outcome evaluation would then allow idiographic and theory-driven assessment, and individualized treatment plan and interventions. Gabbard and Freedman (2006) suggest to concentrate on finding which interventions are shown to be most helpful for which patients.
Another nevertheless similar solution was proposed by the Piper's team (Piper & Ogrodniczuk, 1999). From a point of view of time-limited dynamic psychotherapy they proposed to use general guidelines rather than detailed technical behaviors. The focus of the guideline is in facilitating psychodynamic process of interpretations, while the psychotherapists are encouraged to use their judgment regarding the number and timing of the interpretations they provided. The guidelines were aided to assist the therapists to follow five categories of behaviors which were assumed to be therapeutic: Alliance, Receptive approach, Interpretive approach, Transferential reactions, and Problem focus. Such approach enables an objective appraisal of therapist's competence and adherence, and their research yielded fair results.
According to Spence`s theorization of the `patient`s theory of mind` (Spence, 1993), people hold different `theories of mind`, ways of thinking, feeling, and grasping themselves and the surrounding reality, which are hypothesized to be constructed around the age of two to five. Hence, patients who are given the same diagnosis and even have similar basic personality traits and life circumstances can differ significantly on their `theory of mind` which may lead to or necessitates entirely different modes of therapeutic interventions. Striving for a uniformed treatment for all patients receiving the same diagnosis inescapably yield to detrimental results. As Wampold (1997) notes, two psychotherapeutic treatments may have equivalent effects but work through different causal paths. The effective and successful therapist is one who has a vast knowledge of theory and research in relevant fields, and who uses his knowledge in a flexible way for the benefit of each patient. Goldfried and wolfe (1996) suggest that what we need are not new therapies artificially invented, but “identifiable processes of patient change and therapist behaviors that bring them about”.
The case of Mr. A. demonstrates the potential advantages of flexible therapeutic approach over manualized treatment. Of course it is not a systematic study, and a methodical investigation was not done. The case did not follow a manualized procedure, and failed in curing the presenting problem of anxiety. But a flexible and theory-rooted approach enabled the patient to accomplish a significant progress in the improvement of his life – an outcome which most likely would not have been reached through a symptom-oriented, narrowly-formulated manualized procedure, nor is it aimed to.
The case of manualized treatment is hard to conclude, because the argument is on the questions that are worth asking. For example, what is the objective of the treatment – the client or the disorder? The manifest problem or its hypothesized underlying dynamics? Another type of questions worth asking is on the appropriate methodology for getting answers, whether controlled studies or single case study? It seems that concord can hardly be obtained, regarding the difference in professional views and styles. Manualized therapists would ask for methodical, `scientific` evidence; flexible therapists would not be wiling to give up what they know to do and what they believe to benefit their clients. While more clinical, theoretical, and empirical data, gathered in various means, seem necessary to assist therapist crystallizing their approach, a consented policy is likely to be far ahead of current state of practice.

REFERENCES

Ablon J.S., Jones E.E. (2002), Validity of controlled clinical trials of psychotherapy: findings from the NIMH treatment of depression collaborative research program. American Journal of Psychiatry, 159: 775-783.

Anscomb R. (1989). The myth of the true self. Psychiatry 52: 209-217.

Beck A.T., Rush A.J., Shaw B.F. et al (1979). Cognitive Therapy of Depression. New-York, Guilford.

Berman E. (1999). Editoial in Sihot – Israel Journal of Psychotherapy 13: 145-147.

Bliss M., (1999), Williams Osler: a Life in Medicine. Oxford, Oxford University press.

Campbell L.F. (1996), The treatment outcome pursuit: a mandate for the clinician and researcher working alliance. Psychotherapy, 33: 190-196.

Crits-cristoph P. (1993), Efficacy of brief psychotherapy: reply. American Journal of Psychiatry, 150: 174.

Cummings N.A. (1999). Was the death of clinical romanticism by murder, suicide, or natural causes? The standardization view. Journal of Humanistic Psychology 39: 38-46.

Cummings N.A., Sayama M. (1995). Focused Psychotherapy: A Casebook of Brief, Intermittent Psychotherapy Throughout The Life Cycle. New-York, Bruner/Mazel.

Dobson K.S., Johnston Ch., Mikail S., Hunsley J. (1999), Empirically supported treatments in psychology: implications for Canadian professional psychology. Canadian Psychology 40: 289-305.

Docherty (Eds.), Psychodynamic Treatment Research: A handbook for Clinical Practice. New-York, Basic Books.

Drozd J.F., Goldfried M.R. (1996), A critical evaluation of the state-of-the-art in psychotherapy outcome research. Psychotherapy, 33: 171-180.

Elkin I. (1994), The NIMH treatment of depression collaborative research program: where we began and where we are. In A.E. Bergin, S.L. Garfeld (eds.), Handbook of Psychotherapy and Behavior Change. 4th ed., pp. 114-139. New-York; John Wiley.

Elkin I., Parloff M., Hadley S., Autry J. (1985). NIMH treatment of depression collaborative research program: background and research plan. Archives of General Psychiatry, 42: 305-316.

Eysenck H.S. (1952),The effects of psychotherapy: an evaluation. Journal of Consulting Psychology, 319-324.

Fensterheim H., Raw S. D. (1996), Psychotherapy research is not psychotherapy practice. Clinical Psychology: Science and Practice, 3: 168-171.

Frisch M.B., Cornell J., Villanueva M., Retzlaff P.J. (1992). Clinical validation of the quality of life inventory: a measure of life satisfaction for use in treatment: planning and outcome assessment. Psychological Assessments 4: 92-101.

Gabbard G. O, Freedman R. (2006), Psychotherapy in the Journal: what’s missing? (editorial). American journal of Psychiatry 163: 182-184.

Garfield S. L. (1998), Some comments on empirically supported treatments. Journal of Consulting and Clinical Psychology 66: 121-125.

Goldfried M. R., Wolfe B. (1996), Psychotherapy practice and research: repairing a strained alliance. American Psychologist 51: 1007-1016.

Hellerstein D., Little S., Samstag L.W. et al. (2001). Adding group psychotherapy to medication treatment of disthymia – a randomized prospective study. Journal for Psychotherapy Practice & Research 10: 93-103.

Henry W. P. (1998), Science, politics, and the politics of science: the use and misuse of empirically validated treatment research. Psychotherapy research 8: 126-140.

Henry W.P., Strupp H,H,, Butler S,F,, Schacht T,E, Binder J.L. (1993a), Effects of training in time-limited dynamic psychotherapy: changes in therapist behavior. Journal of Consulting & Clinical Psychology, 61: 430-440.

Henry W. P., Schacht T. E., Strupp H. H., Butler S. F., Binder J. L. (1993b), Effects of training in time-limited dynamic psychotherapy: mediators of therapist’s responses to training. Journal of Consulting and Clinical Psychology, 61: 441-447.

Hersen M. (1981), Complex problems require complex solutions. Behavior Therapy 12: 15-29.

Hill C.E., O`Grady K.E., Elkin I. (1992), Applying the Collaborative Study Psychotherapy Rating Scale to adherence in cognitive-behavior therapy, interpersonal therapy, and clinical management. Journal of Consulting & Clinical Psychology, 60: 73-79.

Hollon S. D., Becj A. T. (1994), Cognitive and cognitive-behavioral therapies. In A. E. Bergin & S. L. Garfield (eds.), Handbook of Psychotherapy and Behavior Change, 4th ed., pp. 428-466, New-York: Wiley & sons.

Hovarth A.O., Greenberg L.S. (Eds.) (1994), The Working Alliance: theory, research and practice. New-York: John Wiley.

Howard K. I., Krause M. S., Vessey J. T., (1994), Analysis of clinical trial data: the problem of outcome overlap. Psychotherapy 31: 302-307/

Khan M.R. (1986). Clinical aspects of the schizoid personality: affects and technique. International Journal of Psychoanalysis, 41: 1960. Also in MR Khan, The Privacy of The Self. London; Hogarth Press.

Krupnick J.L., Pincus H.A. (1992). The cost-effectiveness of psychotherapy: a plan for research. American Journal of Psychiatry 149: 1295-1305.

Lambert M. J., Bergin A. E. (1994), The effectiveness of psychotherapy. In A. E. Bergin, S. L. Garfield (eds.), Handbook of Psychotherapy and Behavior Change, 4th ed., pp. 143-189. New-York, Wiley & sons.

Lambert M.J., Ogles B.M. (1988), Treatment manuals: problems and promise. J Integrative Eclectic Psychotherapy, 7: 187-204.

Luborsky L. (1984). Principles of Psychoanalytic Psychotherapy: A Manual for Supportive-Expressive Treatment. New-York, Basic Books.

Luborsky L., Barber J.P. (1993), Benefits of adherence to psychotherapy manuals, and where to get them. In M.E. Miller, L. Luborsky, J.P. Barber, J.P.

Luborsky L., DeRubeis R.J. (1994), The use of psychotherapy treatment manuals: a small revolution in psychotherapy research style. Clinical Psychology Review, 4: 5-14.

Luborsky L., Mark D., Hole A.V. et al. (1995). Supportive-expressive dynamic psychotherapy of depression: a time-limited version. Pp. 13-42 in J.P, Barber, P. Crits-christop (Eds.), Dynamic Therapies for Psychiatric Disorders (Axis I). new-York, Basic Books.

MacKenzie K.R., Grabovach A.D. (2001). Interpersonal psychotherapy group for depression. Journal for Psychotherapy Practice & Research 10: 46-51.

Malik M. L., Beutler L. E., Alimohamed Sh., Thompson D. G., Thompson L. (2003), Are all cognitive therapies alike? A comparison of cognitive and noncognitive therapy process and implications for the application of empirically supported treatments. Journal of Consulting and Clinical Psychology 71: 150-158.

McCullough J.P. (1999). Treatment for Chronic Depression: Cognitive-Behavioral analysis System of psychotherapy. New-York, Guilford.

Mojtabai R. and Rieder R. O. (1998), Limitations of the symptom-oriented approach to psychiatric research. British Journal of Psychiatry 173: 198-202.

Mulder RT. (2002), Personality pathology and treatment outcome in major depression: a review. American Journal of Psychiatry, 159: 359-371.

Nezu A. M. (1996), What are we doing to our patients and should we care if anyone else knows? Clinical Psychology: Science and Practice, 3: 160-163.

Norcross J.C., Goldfried M.R. (Eds.) (1994), Handbook of Psychotherapy Integration. New-York; Basic Books.

Persons J.B. (1991), Psychotherapy outcomes studies do not accurately represent models of psychotherapy. American Psychologist, 99-106.

Persons J. B., Silberschatz G. (1998), Are results of randomized controlled trials useful to psychotherapists? Journal of Consulting and Clinical Psychology 66: 126-135.

Piper W.E., Ogrodniczuk K.S. (1999). Therapy manuals and the dilemma of dynamically oriented therapists and researchers. American Journal Psychotherapy 53: 467-482.

Rice A.H. (2001). Evaluating brief structured group treatments of depression. Research in Social Work Practice 11: 53-78.

Robinson L.A., Berman J.S., Neimeyer R.A. (1990), Psychotherapy for the treatment of depression: a comprehensive review of controlled outcome resources. Psychological Bulletin , 108: 30-49.

Schwartz J. (1997), Meaning vs. medical necessity: can psychoanalytical treatments exist in a managed care world? Psychotherapy, 34: 115-123.

Seligman M.E. P. (1995), The effectiveness of psychotherapy: the consumer reports study. American Psychologist 50: 965-974.

Silverman W.H. (1996). Cookbooks, manuals, and paint-by-numbers: psychotherapy in the 90`s. Psychotherapy 33: 207-215.

Spence D.P. (1993). Beneath the analytic surface: the analysand`s theory of mind. International Journal of Psychoanalysis 74: 729-738.

Stolorow R. (1999), The intersubjective perspective. Lecture held in Haifa University.

Stricker G., Abrahmson D.J., Bologna N.C., Hollon S.D., Robinson E.A., Reed G.M. (1996), Treatment guidelines: the good, the bad, and the ugly. Psychotherapy, 36: 69-79.

Strupp H.H., Howard K,I. (1992). A brief history of psychotherapy research. In D.K. Freedheim (ed.), History of Psychotherapy: a century of change. pp 309-334. Washington D.C.; American Psychological Association.

Strupp H.H., Anderson T. (1997), On the limitations of therapy manuals. Clinical Psychology Science & practice, 4: 76-82.

Waltz J., Addis M.E., Koerner K., Jacobson N.S. (1993), Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. Journal of Consulting & Clinical Psychology, 61: 620-630.

Wampold B. E. (1997), Methodological problems in identifying efficacious psychotherapies. Psychotherapy research 7: 21-43.

Weissman M.M., Markowitz J.C., Klerman G.L.(2000). Comprehensive Guide to Interpersonal Psychotherapy. New-York, basic Books.

Williams D. D. R., Garner J. (2002), The case against ‘the evidence’: a different perspective on evidence-based medicine. British Journal of Psychiatry 180: 8-12.

Wilson G.T. (1996), Manual-based treatments: the clinical application of research findings. Behavioral Research & Therapy 34: 295-314.