[null,null,["最后更新时间 (UTC)：2024-08-13。"],[[["\u003cp\u003eCounterfactual fairness is a metric that evaluates fairness by comparing predictions for similar individuals, differing only in a sensitive attribute like demographic group.\u003c/p\u003e\n"],["\u003cp\u003eThis metric is particularly useful when datasets lack complete demographic information for all examples but contain it for a subset.\u003c/p\u003e\n"],["\u003cp\u003eWhile counterfactual fairness can identify individual-level biases, it might not address broader, systemic biases across subgroups.\u003c/p\u003e\n"],["\u003cp\u003eOther fairness metrics, such as demographic parity and equality of opportunity, provide a more holistic view but may require complete demographic data.\u003c/p\u003e\n"],["\u003cp\u003eSelecting the appropriate fairness metric depends on the specific application and desired outcome, with no single "right" metric universally applicable.\u003c/p\u003e\n"]]],[],null,["# Fairness: Counterfactual fairness\n\nThus far, our discussions of fairness metrics have assumed that our training\nand test examples contain comprehensive demographic data for the demographic\nsubgroups being evaluated. But often this isn't the case.\n\nSuppose our admissions dataset doesn't contain complete demographic data.\nInstead, demographic-group membership is recorded for just a small percentage\nof examples, such as students who opted to self-identify which group they\nbelonged to. In this case, the breakdown of our candidate pool into accepted\nand rejected students now looks like this:\n**Figure 5.** Candidate pool, with demographic-group membership unknown for nearly all candidates (icons shaded in gray).\n\nIt's not feasible here to evaluate model predictions for either demographic\nparity or equality of opportunity, because we don't have demographic data\nfor 94% of our examples. However, for the 6% of examples that do contain\ndemographic features, we can still compare pairs of individual predictions\n(a majority candidate vs. a minority candidate) and see if they have been\ntreated equitably by the model.\n\nFor example, let's say that we've thoroughly reviewed the feature data\navailable for two candidates (one in the majority group and one in the minority\ngroup, annotated with a star in the image below), and have determined that they\nare identically qualified for admission in all respects. If the model makes\nthe same prediction for both of these candidates (i.e., either rejects both\ncandidates or accepts both candidates), it is said to satisfy **counterfactual\nfairness** for these examples. Counterfactual fairness stipulates that two\nexamples that are identical in all respects, except a given sensitive attribute\n(here, demographic group membership), should result in the same model\nprediction.\n**Figure 6.** Counterfactual fairness is satisfied for the two identical examples (only varying in demographic group membership) annotated with a star, as the model makes the same decision for both (Rejected).\n\nBenefits and drawbacks\n----------------------\n\nAs mentioned earlier, one key benefit of counterfactual fairness is that it\ncan be used to evaluate predictions for fairness in many cases where using\nother metrics wouldn't be feasible. If a dataset doesn't contain a full set of\nfeature values for the relevant group attributes under consideration, it won't\nbe possible to evaluate fairness using demographic parity or equality of\nopportunity. However, if these group attributes are available for a subset\nof examples, and it's possible to identify comparable pairs of equivalent\nexamples in different groups, practitioners can use counterfactual fairness\nas a metric to probe the model for potential biases in predictions.\n\nAdditionally, because metrics like demographic parity and equality of\nopportunity assess groups in aggregate, they may mask bias issues that affect\nthe model at the level of individual predictions, which can be surfaced by\nevaluation using counterfactual fairness. For example, suppose our admissions\nmodel accepts qualified candidates from the majority group and the minority\ngroup in the same proportion, but the most qualified minority candidate is\nrejected whereas the most qualified majority candidate who has the exact same\ncredentials is accepted. A counterfactual fairness analysis can help identify\nthese sorts of discrepancies so that they can be addressed.\n\nOn the flipside, the key downside of counterfactual fairness is that it doesn't\nprovide as holistic a view of bias in model predictions. Identifying and\nremediating a handful of inequities in pairs of examples may not be sufficient\nto address systemic bias issues that affect entire subgroups of examples.\n\nIn cases where it's possible, practitioners can consider doing both an aggregate\nfairness analysis (using a metric like demographic parity or equality of\nopportunity) as well as a counterfactual fairness analysis to gain the widest\nrange of insights into potential bias issues in need of remediation.\n\nExercise: Check your understanding\n----------------------------------\n\n**Figure 7.** Negative and Positive predictions for a batch of examples, with three pairs of examples labeled as A, B, and C. \nIn the set of predictions in Figure 7 above, which of the\nfollowing pairs of identical (excluding group membership)\nexamples received predictions that violate counterfactual fairness? \nPair A \nPair A's predictions satisfy counterfactual fairness, as both the example in the majority group (blue) and the example in the minority group (orange) received the same prediction (Negative). \nPair B \nPair B's predictions satisfy counterfactual fairness, as both the example in the majority group (blue) and the example in the minority group (orange) received the same prediction (Positive). \nPair C \nPair C's predictions are for two examples that both belong to the majority group (blue). The fact that the model produced different predictions for these identical examples suggests that there may be broader performance issues with the model that should be investigated. However, this result does not violate counterfactual fairness, whose conditions only apply in cases where the two identical examples are each drawn from different groups. \nNone of these pairs violate counterfactual fairness \nThe predictions for Pairs A and B satisfy counterfactual fairness because in both cases, the example in the majority group and the example in the minority group receive the same prediction. Pair C's examples both belong to the same group (the majority group), so counterfactual fairness is not applicable in this case.\n\nSummary\n-------\n\n[Demographic parity](/machine-learning/crash-course/fairness/demographic-parity),\n[equality of opportunity](/machine-learning/crash-course/fairness/equality-of-opportunity),\nand counterfactual fairness each provide different mathematical definitions of\nfairness for model predictions. And those are just three possible\nways to quantify fairness. Some definitions of fairness are even [**mutually\nincompatible**](/machine-learning/glossary#incompatibility-of-fairness-metrics),\nmeaning it may be impossible to satisfy them simultaneously for a\ngiven model's predictions.\n\nSo how do you choose the \"right\" fairness metric for your model? You need to\nconsider the context in which it's being used and the overarching goals you\nwant to accomplish. For example, is the goal to achieve equal representation\n(in this case, demographic parity may be the optimal metric) or is it to\nachieve equal opportunity (here, equality of opportunity may be the best\nmetric)?\n\nTo learn more about ML Fairness and explore these issues in more depth, see\n[*Fairness and Machine Learning: Limitations and Opportunities*](https://fairmlbook.org/) by Solon Barocas, Moritz Hardt, and Arvind Narayanan.\n| **Key terms:**\n|\n| - [Bias (ethics/fairness)](/machine-learning/glossary#bias-ethicsfairness)\n| - [Demographic parity](/machine-learning/glossary#demographic-parity)\n- [Mutually incompatible](/machine-learning/glossary#incompatibility-of-fairness-metrics) \n[Help Center](https://support.google.com/machinelearningeducation)"]]
<ph type="x-smartling-placeholder">