• 针对你的产品集思广益，与社会科学家、人文学者和其他相关专家合作，了解并考虑各方的观点。
• Engage with social scientists, humanists, and other relevant experts for your product to understand and account for various perspectives.
• 值得深思的是，随着时间的推移，这项技术及其发展会如何影响不同的用例：代表了哪些人的观点？代表了哪些类型的数据？遗漏了什么？这项技术能助力实现哪些结果？这些结果成真后，不同的用户和社区分别会受到怎样的影响？可能会造成什么样的偏见、负面体验或歧视性结果？
• Consider how the technology and its development over time will impact different use cases: Whose views are represented? What types of data are represented? What's being left out? What outcomes does this technology enable and how do these compare for different users and communities? What biases, negative experiences, or discriminatory outcomes might occur?
• 为你的系统设定目标，让它公平地处理各个预期用例：例如，以 X 种不同的语言，或面向 Y 个不同的年龄段。随着时间的推移，持续监控这些目标并酌情扩展。
• Set goals for your system to work fairly across anticipated use cases: for example, in X different languages, or to Y different age groups. Monitor these goals over time and expand as appropriate.
• 设计要使用的算法和目标函数时，力求反映公平性目标。
• Design your algorithms and objective function to reflect fairness goals.
• 及时更新你的训练数据和测试数据，以适应不断变化的用户群体和他们使用技术的方式。
• Update your training and testing data frequently based on who uses your technology and how they use it.

• 评估数据集的公平性，包括确定数据的代表性和相应局限，以及特征、标签和群体之间的偏见性或歧视性关联。可视化、聚类和数据注解均有助于评估。
• Assess fairness in your datasets, which includes identifying representation and corresponding limitations, as well as identifying prejudicial or discriminatory correlations between features, labels, and groups. Visualization, clustering, and data annotations can help with this assessment.
• 公共训练数据集通常需要扩充，以更好地反映系统要预测的人员、事件和属性的真实频率。
• Public training datasets will often need to be augmented to better reflect real-world frequencies of people, events, and attributes that your system will be making predictions about.
• 了解为数据添加注解的人员的不同观点、经验和目标。"成功"对不同的工作者分别意味着什么？依据什么来权衡任务耗费的时间与任务带来的享受？
• Understand the various perspectives, experiences, and goals of the people annotating the data. What does success look like for different workers, and what are the trade-offs between time spent on task and enjoyment of the task?
• 如果你与注解团队合作，请务必与他们密切协同，设计明确的任务、激励和反馈机制，确保注解的可持续性、多样性和准确性。要考虑人的个体差异，包括无障碍性、肌肉记忆和注解中的偏见。例如，可使用一组有明确答案的标准问题来减小潜在影响。
• If you are working with annotation teams, partner closely with them to design clear tasks, incentives, and feedback mechanisms that ensure sustainable, diverse, and accurate annotations. Account for human variability, including accessibility, muscle memory, and biases in annotation, e.g., by using a standard set of questions with known answers.

• 例如，组建一个由背景多样的可信人员构成的测试小组，让他们对系统进行对抗性测试，并将各种对抗性输入整合到单元测试中。这可帮助你确定哪些人可能会遭受意想不到的负面影响。即使错误率已降到很低，也难免会偶尔出现严重错误。有针对性的对抗性测试可帮助我们发现聚合指标所遮蔽的问题。
• For example, organize a pool of trusted, diverse testers who can adversarially test the system, and incorporate a variety of adversarial inputs into unit tests. This can help to identify who may experience unexpected adverse impacts. Even a low error rate can allow for the occasional very bad mistake. Targeted adversarial testing can help find problems that are masked by aggregate metrics.
• 在设计用于训练和评估系统的指标时，还应加入相关指标来检查系统对不同子群体的效果。例如，我们可逐一分析每个子群体的假正例率和假负例率，从而精准判断哪些群体受到了过多的负面影响，哪些群体受益颇多，以免顾此失彼。
• While designing metrics to train and evaluate your system, also include metrics to examine performance across different subgroups. For example, false positive rate and false negative rate per subgroup can help to understand which groups experience disproportionately worse or better performance.
• 除了细化后的统计指标，我们还需构建一个测试集，用于进行压力测试，从而检验系统在极限情况下的表现。通过这样的测试集，每次更新系统后，都能快速评估系统在处理那些伤害性极强或问题极严重的样例时的表现。就像对待其他所有测试集一样，应持续更新这个特殊测试集，特别是在系统不断演变、功能不断增加、用户反馈不断积累的情况下。
• In addition to sliced statistical metrics, create a test set that stress-tests the system on difficult cases. This will enable you to quickly evaluate how well your system is doing on examples that can be particularly hurtful or problematic each time you update your system. As with all test sets, you should continuously update this set as your system evolves, features are added or removed and you have more feedback from users.
• 考虑系统先前决策所产生的偏见有何影响，以及由此可能形成的反馈环，避免系统越"走"越"偏"。
• Consider the effects of biases created by decisions made by the system previously, and the feedback loops this may create.

• 将你定义的各个不同指标考虑在内。举个例子，系统处理不同子群体的数据时假正例率可能存在差异，某个指标的改进可能会对另一个指标产生负面影响。
• Take the different metrics you've defined into account. For example, a system's false positive rate may vary across different subgroups in your data, and improvements in one metric may adversely affect another.
• 模拟真实的使用场景，覆盖尽可能广泛的用户、用例和情境，从而全面评估用户体验，例如借助 TensorFlow Model Analysis 工具。首先通过 dogfood 进行测试和迭代，然后在发布后继续测试。
• Evaluate user experience in real-world scenarios across a broad spectrum of users, use cases, and contexts of use (e.g., TensorFlow Model Analysis). Test and iterate in dogfood first, followed by continued testing after launch.
• 即使我们精心设计整个系统的方方面面，尽力解决公平性问题，基于机器学习的模型在面对真实的实时数据时，也难以做到尽善尽美。当产品在实际应用中出现问题时，我们需要考虑这个问题是否会加剧某些现有的社会不公平现象，并且需要评估短期和长期解决方案对这个问题的潜在影响。
• Even if everything in the overall system design is carefully crafted to address fairness issues, ML-based models rarely operate with 100%% perfection when applied to real, live data. When an issue occurs in a live product, consider whether it aligns with any existing societal disadvantages, and how it will be impacted by both short- and long-term solutions.

在设计并训练模型之前、期间和之后，都可以追求可解释性。

Pursuing interpretability can happen before, during and after designing and training your model.

• 你真正需要多大程度的可解释性？请针对你的模型，与相关领域（例如医疗保健、零售等）的专家紧密合作，共同确定需要哪些可解释性功能及相应原因。虽然很少见，但在某些情况下/系统中，如果积累了充分的经验证据，则不需要细粒度的可解释性。
• What degree of interpretability do you really need? Work closely with relevant domain experts for your model (e.g., healthcare, retail, etc.) to identify what interpretability features are needed, and why. While rare, there are some cases/systems where with sufficient empirical evidence, fine-grain interpretability is not needed.
• 你能否分析自己的训练/测试数据？例如，如果处理的是私密数据，可能就无法调查相应输入数据。
• Can you analyze your training/testing data? For example, if you are working with private data, you may not have access to investigate your input data.
• 你能否更改训练/测试数据？例如，针对特定子集（例如特征空间的某些部分/细分）收集更多训练数据，或者针对感兴趣的类别收集测试数据。
• Can you change your training/testing data, for example, gather more training data for certain subsets (e.g., parts/slices of the feature space), or gather test data for categories of interest?
• 你能自行设计新模型还是只能使用已训练的模型？
• Can you design a new model or are you constrained to an already-trained model?
• 你是否提供了过多的详细信息，以至于可能会导致滥用？
• Are you providing too much transparency, potentially opening up vectors for abuse?
• 你给出的训练后可解释性选项有哪些？你是否有权知晓模型的内部工作机制（例如是黑盒模型还是白盒模型）？
• What are your post-train interpretability options? Will you have access to the internals of the model (e.g., black box vs. white box)?

• 在开发周期中与用户反复沟通，测试并优化你对用户需求和目标的假设。
• Iterate with users in the development cycle to test and refine your assumptions about user needs and goals.
• 设计用户体验时，要引导用户构建适宜的心智模型以便使用 AI 系统。如果你未提供清晰且令人信服的信息，用户可能会自己虚构关于 AI 系统运作方式的理论，这可能会对他们尝试使用系统的方式产生负面影响。
• Design the UX so that users build useful mental models of the AI system. If not given clear and compelling information, users may make up their own theories about how an AI system works, which can negatively affect how they try to use the system.
• 尽可能为用户自行分析敏感度创造便利条件：让他们能够测试不同的输入对模型输出结果有何影响。
• Where possible, make it easy for users to do their own sensitivity analysis: empower them to test how different inputs affect the model output.
• 其他相关的用户体验资源：以人为本的设计、用户控制、训练 AI、习惯化、公平性、表征
• Additional relevant UX resources: Designing for human needs, user control, teaching an AI, habituation, fairness, representation

• 在能达成性能目标的前提下，使用尽可能小的输入集，以便更清晰地了解哪些因素会对模型产生影响。
• Use the smallest set of inputs necessary for your performance goals to make it clearer what factors are affecting the model.
• 只要能实现性能目标，模型越简单越好。
• Use the simplest model that meets your performance goals.
• 尽可能学习因果关系而非相关性。例如，判断孩子乘坐过山车是否安全时，应以身高（而非年龄）为依据。
• Learn causal relationships not correlations when possible (e.g., use height not age to predict if a kid is safe to ride a roller coaster).
• 确保训练目标与真实目标保持一致。例如，如果目标是将误报率控制在可接受范围内，那么训练就应围绕误报率展开，而不是单纯追求准确率。
• Craft the training objective to match your true goal (e.g., train for the acceptable probability of false alarms, not accuracy).
• 适当约束模型，使其生成的输入-输出关系既专业又合理。例如，在推荐咖啡店时，如果其他条件相同，距用户更近的咖啡店应更有可能被推荐。
• Constrain your model to produce input-output relationships that reflect domain expert knowledge (e.g., a coffee shop should be more likely to be recommended if it's closer to the user, if everything else about it is the same).

你选择的指标必须考虑到特定情境中的具体收益和风险。例如，火灾警报系统需要达到高召回率，即使偶尔误报也无妨。

The metrics you consider must address the particular benefits and risks of your specific context. For example, a fire alarm system would need to have high recall, even if that means the occasional false alarm.

目前，正在开发的许多技术都旨在帮助我们更好地理解模型的内部机制，例如模型对不同输入的敏感度。

Many techniques are being developed to gain insights into the model (e.g., sensitivity to inputs).

• 针对不同的样例子集，分别分析模型对不同输入的敏感度。
• Analyze the model's sensitivity to different inputs, for different subsets of examples.

• 应提供易于理解且适合用户查看的解释。比如，业内人士和学术界人士可能需要了解技术细节，而普通用户可能更喜欢即时弹出的界面提示、通俗易懂的摘要解释或直观呈现的内容。恰当的解释需要结合具体应用情境，仔细考量哲学、心理学、计算机科学（包括人机交互）、法律和伦理等诸多因素。
• Provide explanations that are understandable and appropriate for the user (e.g., technical details may be appropriate for industry practitioners and academia, while general users may find UI prompts, user-friendly summary descriptions or visualizations more useful). Explanations should be informed by a careful consideration of philosophical, psychological, computer science (including HCI), legal and ethical considerations about what counts as a good explanation in different contexts.
• 解释并非多多益善，有些情况下，提供解释反而不妥。例如：如果解释会让普通用户更加困惑，或者会被不法分子用以损害系统或用户安全，抑或会泄露专有信息，就不宜提供。
• Identify if and where explanations may not be appropriate (e.g., where explanations could result in more confusion for general users, nefarious actors could take advantage of the explanation for system or user abuse, or explanations may reveal proprietary information).
• 如果某个用户群要求提供解释，但你无法或不应提供，或者无法提供明确、合理的解释，这时就需要考虑采用替代方案。不妨改以其他方式实现问责制。例如接受审核、允许用户质疑决策或收集用户反馈，从而改进未来的决策和体验。
• Consider alternatives if explanations are requested by a certain user base but cannot or should not be provided, or if it's not possible to provide a clear, sound explanation. You could instead provide accountability through other mechanisms such as auditing or allow users to contest decisions or to provide feedback to influence future decisions or experiences.
• 应优先提供那些能明确指导用户操作的清晰解释，方便用户未来纠正不准确的预测。
• Prioritize explanations that suggest clear actions a user can take to correct inaccurate predictions going forward.
• 不要暗示相关解释意味着因果关系，除非确实如此。
• Don't imply that explanations mean causation unless they do.
• 认识到人类的心理和局限（例如确认偏差、认知疲劳）
• Recognize human psychology and limitations (e.g., confirmation bias, cognitive fatigue)
• 解释形式可以不拘一格（例如文本、图表、统计数据）：在使用可视化资源提供数据洞见时，遵循 HCI 和可视化方面的最佳实践。
• Explanations can come in many forms (e.g., text, graphs, statistics): when using visualization to provide insights, use best practices from HCI and visualization.
• 任何经过聚合的摘要都可能会丢失信息并淹没细节（例如部分依赖关系）。
• Any aggregated summary may lose information and hide details (e.g., partial dependency plots).
• 如果能理解机器学习系统的各个部分（尤其是输入）以及所有部分之间如何协作（即“完整性”），用户就容易针对相应系统构建更明确的心智模型。这些心智模型更贴近系统的实际性能，能提供更可信的体验，并为未来的学习设定更准确的期望。
• The ability to understand the parts of the ML system (especially inputs) and how all the parts work together (“completeness”) helps users to build clearer mental models of the system. These mental models match actual system performance more closely, providing for a more trustworthy experience and more accurate expectations for future learning.
• 注意解释的局限。例如，不能以偏概全，随意将局部解释推而广之；而且，对于两个看起来相似的样例，局部解释可能会互相冲突。
• Be mindful of the limitations of your explanations (e.g., local explanations may not generalize broadly, and may provide conflicting explanations of two visually-similar examples).

借鉴软件工程最佳测试实践和质量工程技术，确保 AI 系统能按预期运行且值得信赖。

Learn from software engineering best test practices and quality engineering to make sure the AI system is working as intended and can be trusted.

• 执行严格的单元测试，让系统中的每个组件都接受单独测试。
• Conduct rigorous unit tests to test each component of the system in isolation.
• 主动检测输入偏移，通过测试 AI 系统的输入统计信息，确保输入未发生意外变化。
• Proactively detect input drift by testing the statistics of the inputs to the AI system to make sure they are not changing in unexpected ways.
• 使用黄金标准数据集来测试系统，确保其行为持续符合预期。定期更新该测试集，以契合不断变化的用户和用例，降低模型基于测试集进行训练的可能性。
• Use a gold standard dataset to test the system and ensure that it continues to behave as expected. Update this test set regularly in line with changing users and use cases, and to reduce the likelihood of training on the test set.
• 在开发周期内，执行迭代的用户测试，兼顾不同用户的多样化需求。
• Conduct iterative user testing to incorporate a diverse set of users' needs in the development cycles.
• 践行质量工程领域的波卡纠偏 (poka-yoke) 原则：在系统中内置质量检查机制，确保意外故障无法发生或会触发立即响应（例如，如果某项重要功能意外缺失，AI 系统将不会输出任何预测结果）。
• Apply the quality engineering principle of poka-yoke: build quality checks into a system so that unintended failures either cannot happen or trigger an immediate response (e.g., if an important feature is unexpectedly missing, the AI system won't output a prediction).
• 执行集成测试：了解 AI 系统如何与其他系统交互，以及生成了哪些反馈环（例如，如果因热度高而推荐某篇新闻报道，可能会使该报道的热度进一步升高，导致它被更多地推荐）。
• Conduct integration tests: understand how the AI system interacts with other systems and what, if any, feedback loops are created (e.g., recommending a news story because it's popular can make that news story more popular, causing it to be recommended more).

• 尝试能否在不使用敏感数据的情况下训练机器学习模型，例如，利用非敏感数据集或现有公共数据源另辟蹊径。
• Identify whether your ML model can be trained without the use of sensitive data, e.g., by utilizing non-sensitive data collection or an existing public data source.
• 如果必须处理敏感的训练数据，应设法尽量减少对此类数据的使用。处理敏感数据要慎之又慎：例如，首先要遵守相关法律和标准；其次要向用户提供明确通知，并向用户赋予对数据使用方式的必要控制权；此外要遵循相关最佳实践，例如对传输和存储的数据实施加密；最后要恪守 Google 的隐私权原则。
• If it is essential to process sensitive training data, strive to minimize the use of such data. Handle any sensitive data with care: e.g., comply with required laws and standards, provide users with clear notice and give them any necessary controls over data use, follow best practices such as encryption in transit and rest, and adhere to Google privacy principles.
• 对传入的数据进行匿名化和聚合处理时，务必滴水不漏，采用符合最佳实践要求的数据清理管道：例如，考虑删除个人身份信息 (PII) 和可能导致去匿名化的异常值或元数据（包括隐式元数据，例如到达顺序，此类数据可通过随机打乱来移除，如 Prochlo 中那样；或使用 Cloud Data Loss Prevention API 自动发现并隐去敏感数据和身份标识数据）。
• Anonymize and aggregate incoming data using best practice data-scrubbing pipelines: e.g., consider removing personally identifiable information (PII) and outlier or metadata values that might allow de-anonymization (including implicit metadata such as arrival order, removable by random shuffling, as in ProchloProchlo; or the Cloud Data Loss Prevention API to automatically discover and redact sensitive and identifying data).

• 如果你的目标是学习个人交互统计数据（例如某些界面元素的使用频率），不妨考虑只收集在本地设备端计算的统计数据，而不收集可能包含敏感信息的原始交互数据。
• If your goal is to learn statistics of individual interactions (e.g., how often certain UI elements are used), consider collecting only statistics that have been computed locally, on-device, rather than raw interaction data, which can include sensitive information.
• 考虑某些技术能否提升系统的隐私保护水平，以联邦学习为例，该技术支持一组设备协同工作，基于本地存储的训练数据来训练共享的全局模型。
• Consider whether techniques like federated learning, where a fleet of devices coordinates to train a shared global model from locally-stored training data, can improve privacy in your system.
• 在可行的情况下，在设备端实施聚合、随机化和清理操作（例如安全聚合、RAPPOR，以及 Prochlo 的编码步骤）。需要注意的是，这些操作本身并非万无一失，除非附带相关证明，否则它们只能因地制宜、尽力而为地提供隐私保护。
• When feasible, apply aggregation, randomization, and scrubbing operations on-device (e.g., Secure aggregation, RAPPOR, and Prochlo's encode step). Note that these operations may only provide pragmatic, best-effort privacy unless the techniques employed are accompanied by proofs.

由于机器学习模型可能会通过其内部参数和外部可见行为泄露训练数据的详细信息，因此务必考虑模型的构建方式和访问方式对隐私保护有何影响。

Because ML models can expose details about their training data via both their internal parameters as well as their externally-visible behavior, it is crucial to consider the privacy impact of how the models were constructed and may be accessed.

• 评估你的模型是否会无意中记忆或泄露敏感数据，为此，可以使用基于“泄露”衡量或成员推断评估的测试。这些指标也可在模型维护期间用于回归测试。
• Estimate whether your model is unintentionally memorizing or exposing sensitive data using tests based on “exposure” measurements or membership inference assessment. These metrics can additionally be used for regression tests during model maintenance.
• 为了精益求精，实现数据最少化，你需要针对不同的参数进行实验（例如聚合、异常值阈值和随机化因素），从而权衡利弊，最终确定模型的最佳设置。
• Experiment with parameters for data minimization (e.g., aggregation, outlier thresholds, and randomization factors) to understand tradeoffs and identify optimal settings for your model.
• 使用能通过数学手段保证隐私性的技术来训练机器学习模型。需要注意的是，这些分析保证并不是对整个操作系统的全面保证。
• Train ML models using techniques that establish mathematical guarantees for privacy. Note that these analytic guarantees are not guarantees about the complete operational system.
• 遵循针对加密和安全关键型软件制定的最佳实践流程，例如，使用有原则且可证明的方法、以同行评审的方式发布新想法、对关键软件组件进行开源，以及在设计和开发的各个阶段邀请专家审核。
• Follow best-practice processes established for cryptographic and security-critical software, e.g., the use of principled and provable approaches, peer-reviewed publication of new ideas, open-sourcing of critical software components, and the enlistment of experts for review at all stages of design and development.

• 考虑是否有人会故意让系统出现故障。例如，如果开发者构建了一款照片整理应用，用户固然能随意修改照片，让系统无法正确整理。不过，扰乱系统对用户无益，所以用户应该不会这样做。
• Consider whether anyone would have an incentive to make the system misbehave. For example, if a developer builds an app that helps a user organize their own photos, it would be easy for users to modify photos to be incorrectly organized, but users may have limited incentive to do so.
• 确定系统出错可能会导致的意外后果，并评估此类后果的发生几率和严重程度。
• Identify what unintended consequences would result from the system making a mistake, and assess the likelihood and severity of these consequences.
• 构建严谨的威胁模型，以了解所有可能的攻击途径。例如，一个系统允许攻击者更改输入到机器学习模型的数据，而另一个系统处理由服务器收集的元数据（例如用户执行操作的时间戳），那么前者比后者更容易受到攻击，因为用户很难故意修改在自己未直接参与的情况下收集的输入特征。
• Build a rigorous threat model to understand all possible attack vectors. For example, a system that would allow an attacker to change the input to the ML model may be much more vulnerable than a system that processes metadata collected by the server, like timestamps of actions the user took, since it is much harder for a user to intentionally modify input features collected without their direct participation.

某些应用（例如垃圾邮件过滤服务）尽管在对抗性机器学习方面困难重重，仍能利用当前的防御技术成功应对威胁。

Some applications, e.g., spam filtering, can be successful with current defense techniques despite the difficulty of adversarial ML.

• 在对抗性环境中测试系统性能。在某些情况下，这种测试可通过 CleverHans 等工具实现。
• 建立内部红队进行测试，或者举办比赛或启动漏洞赏金计划，鼓励第三方对你的系统进行对抗性测试。

• Test the performance of your systems in the adversarial setting. In some cases this can be done using tools such as CleverHansCleverHans.
• Create an internal red team to carry out the testing, or host a contest or bounty program encouraging third parties to adversarially test your system.

• Stay up to date on the latest research advances. Research into adversarial machine learning continues to offer improved performance for defenses and some defense techniques are beginning to offer provable guarantees.
• Beyond interfering with input, it is possible there may be other vulnerabilities in the ML supply chain. While to our knowledge such an attack has not yet occurred, it is important to consider the possibility and be prepared.
• 及时了解最新的研究进展。对抗性机器学习研究日新月异，促使防御性能不断提升，某些防御技术已开始提供可证明的保证。
• 除了干扰输入，机器学习供应链中可能还存在其他漏洞。据我们所知，这种攻击尚未发生，但居安思危不多余，未雨绸缪才明智。