20240912 - actuarial formulas

Author: isaactpetersen

11-Clinical-Actuarial-Judgment.Rmd

@@ -41,6 +41,7 @@ Hypothetically, humans reviewing and adjusting the results from the statistical
 However, human input also could lead to the possibility or exacerbation of [biased](#bias) [predictions](#prediction).\index{clinical judgment}\index{bias}
 In general, with very few exceptions, actuarial approaches are as accurate or more accurate than clinical judgment [@Dawes1989; @Aegisdottir2006; @Baird2000; @Grove1996; @Grove2000].\index{clinical judgment}\index{actuarial}
 Moreover, the superiority of actuarial approaches to clinical judgment tends to hold even when the clinician is given more information than the actuarial approach [@Dawes1989].\index{clinical judgment}\index{actuarial}
+In addition, actuarial predictions outperform human judgment even when the human is given the result of the actuarial prediction [@Kahneman2011].\index{clinical judgment}\index{actuarial}
 Allowing clinicians to override actuarial [predictions](#prediction) consistently leads to lower predictive accuracy [@Garb2019].\index{clinical judgment}\index{actuarial}
 
 In general, [validity](#validity) tends to increase with greater structure (e.g., [structured](#structuredInterview) or [semi-structured](#semiStructuredInterview) [interviews](#interview) as opposed to free-flowing, [unstructured interviews](#unstructuredInterview)), in terms of administration, responses, scoring, interpretation, etc.\index{clinical judgment}\index{actuarial}\index{validity}\index{interview!structured}\index{interview!semi-structured}\index{interview!unstructured}\index{structured!administration}\index{structured!responses}\index{structured!scoring}
@@ -105,18 +106,23 @@ Different clinicians may have different prototypes, leading to lower [inter-rate
 
 ### Advantages of Computers {#advantagesOfComputers}
 
-Here are some advantages of computers over humans:\index{actuarial}
+Here are some advantages of computers over humans, including "experts":\index{actuarial}
 
 - Computers can process lots of information simultaneously.\index{actuarial}
 So can humans.\index{clinical judgment}
 But computers can to an even greater degree.\index{actuarial}\index{clinical judgment}
 - Computers are faster at making calculations.\index{actuarial}\index{clinical judgment}
+- Given the same input, a formula will give the exact same result everytime.\index{actuarial}\index{clinical judgment}
+Humans' judgment tends to be inconsistent both across raters and within rater across time, when trying to make judgments or predictions from complex information [@Kahneman2011].\index{actuarial}\index{clinical judgment}
+As noted in Section \@ref(reliabilityVsValidity), [reliability](#reliability) sets the upper bound for [validity](#validity), so unreliable judgments cannot be valid.\index{actuarial}\index{clinical judgment}
 - Computations by computers are error-free (as long as the computations are programmed correctly).\index{actuarial}\index{clinical judgment}
 - Computers' judgments will not be [biased](#bias) by fatigue or emotional responses.\index{actuarial}\index{clinical judgment}\index{bias}
 - Computers' judgments will tend not to be [biased](#bias) in the way that humans' cognitive [biases](#bias) are, such as with anchoring [bias](#bias), representativeness [bias](#bias), confirmation [bias](#bias), or recency [bias](#bias).\index{actuarial}\index{clinical judgment}\index{bias}
 Computers are less likely to be over-confident in their judgments.\index{actuarial}\index{clinical judgment}\index{over-confidence}
 - Computers can more accurately weight the set of predictors based on large data sets.\index{actuarial}
 Humans tend to give too much weight to singular predictors.\index{clinical judgment}
+Experts may attempt to be clever and to consider complex combinations of predictors, but doing so often reduces validity [@Kahneman2011].\index{clinical judgment}
+Simple combinations of predictions often outperform more complex combinations [@Kahneman2011].\index{clinical judgment}
 
 ### Advantages of Humans {#advantagesOfHumans}
 
@@ -168,6 +174,7 @@ Findings consistently show that actuarial methods are as [accurate](#validity) o
 
 Actuarial methods are particularly valuable for criterion-referenced assessment tasks, in which the aim is to [predict](#prediction) specific events or outcomes [@Garb2019].\index{actuarial}
 For instance, actuarial methods have shown promise in [predicting](#prediction) violence, criminal recidivism, psychosis onset, course of mental disorders, treatment selection, treatment failure, suicide attempts, and suicide [@Garb2019].\index{actuarial}
+Actuarial methods are especially important to use in low-[validity](#validity) environments (like clinical psychology) in which there is considerable uncertainty and unpredictability [@Kahneman2011].\index{actuarial}
 Psychometric methods of scale construction such as [factor analysis](#factorAnalysis) may be preferred to statistical [prediction](#prediction) rules for [norm-referenced](#norm) assessment tasks such as describing personality and psychopathology [@Garb2019].\index{factor analysis}\index{norm-referenced}
 
 Moreover, actuarial methods are explicit; they can be transparent and lead to informed scientific criticism to improve them.\index{actuarial}
@@ -223,6 +230,7 @@ By contrast, simpler models showed better [calibration](#calibration) in the int
 The [probability nomogram](#bayesianUpdating) with one or multiple assessment findings showed better [calibration](#calibration) than LASSO models because they accounted for the original [base rate](#baseRate), so they did not over-diagnose bipolar disorder.\index{probability!nomogram}\index{Naïve Bayesian}\index{calibration}\index{base rate}\index{least absolute shrinkage and selection option}
 
 In summary, the best models are those that are relatively simple (parsimonious), that can account for one or several of the most important [predictors](#prediction) and their optimal weightings, and that account for the [base rate](#baseRate) of the phenomenon.\index{parsimony}\index{base rate}
+Multiple regression and/or prior literature can be used to identify the weights of various [predictors](#prediction).\index{multiple regression}
 Even unit-weighted formulas (formulas whose [predictors](#prediction) are equally weighted with a weight of one) can sometimes generalize better to other samples than complex weightings [@Garb2019].
 Differential weightings sometimes capture random variance and [over-fit](#overfitting) the model, thus leading to [predictive accuracy](#predictiveValidity) shrinkage in cross-validation samples [@Garb2019], as described below.\index{shrinkage}\index{over-fitting}\index{validity!predictive}
 The choice of [predictive](#prediction) variables often matters more than their weighting.