Skip to content
Methods briefingSafety & Responsible Technology

Safety as a shared responsibility

A guidance framework for responsible caregiving technology.

GiveCare Labs · Working methods

This framework guides review of proposed uses of AI. It provides no certification or assurance that controls have been implemented. Evidence of compliance and independent review must come from the work itself.

A good answer is only one part of responsible caregiving technology. The harder question is what surrounds it.

Caregiving takes place under pressure: fatigue, uncertainty, unequal resources, and relationships in which people’s interests may differ. An apparently useful response can still expose private information, assume support a family does not have, or leave nobody accountable when something goes wrong.

Responsible technology should ease the work of care while keeping people in control. When something goes wrong, people need a way to get help and have mistakes corrected. That is the purpose of this guidance framework: to help people examine a proposed use of AI and decide what needs to be understood, tested or changed.

Begin with a specific use: who it serves, what it may do and what happens today without it. Then ask three questions. The evidence should fit the consequences; a demonstration or a confident answer is not enough. The Humanity Baseline informs this caregiver-centered starting point. [29]

Three questions to guide a review.

Review questions, evidence to request and related responsibilities
AskLook forConnects to

Who decides, and whose choices are protected?

Questions to ask
  • Whose decision is this: the caregiver’s, the care recipient’s, or someone else’s?
  • What may the tool do, and how can permission be changed?
  • Who can intervene when preferences conflict or something goes wrong?
Look forClear decision authority, understandable permissions and a way to refuse, withdraw or intervene.Connects to

Does it reduce the work of care?

Questions to ask
  • What burden should this remove, compared with support available today?
  • What does setup, cost, checking and ongoing use ask of people?
  • Who helps define success, and whose experience might the evidence miss?
Look forCaregiver-defined outcomes, an honest account of the work the tool adds, and evidence from people affected.Connects to

What happens when it is wrong?

Questions to ask
  • Which failures could harm people, and what prevents or reveals them?
  • When must the tool decline, seek review or help someone reach a person?
  • How are problems corrected, and which changes require a fresh evaluation?
Look forTested safeguards, traceable guidance, a usable route to human help and an owner for correction.Connects to

A useful review ends with a next step: explore a bounded use, request missing evidence, revise the design, or advise against proceeding. Record the reason, the responsible owner and what would reopen the decision.

01 / Accountability

Responsibility begins before a model answers.

A capability needs a defined purpose, an account of the people it affects, and limits on what it may do. Governance connects those decisions to people with authority to make them. It also asks who gains, who bears additional care work, and who can challenge a harmful decision.

[1][2][3][4][5][6][7]

The NIST AI Risk Management Framework gives this work a useful rhythm: Govern establishes accountability; Map examines context and impacts; Measure gathers evidence; Manage puts decisions into practice. These functions continue throughout a system’s life. [1]

The Oxford collaboration on generative AI in adult social care starts with the values care should protect: rights, dignity, independence, choice, control, equality and well-being. This puts a proposed use of AI in a wider context than efficiency alone. For caregiving technology, we interpret that as a question about what a capability should preserve in people’s lives and relationships. [28]

The responsibilities below are GiveCare’s operational interpretation of the work. They overlap. A privacy decision can change an evaluation; an accessibility finding can change the product; an incident can reopen a governance decision.

Figure 1 / From responsibility to action

Responsible decisions need safeguards and evidence.

01 · Decide

Governance

Define the purpose and name an accountable owner. Decisions need review as risks change.

02 · Build

Safeguards in practice

03 · Act

Evidence into decisions

↶ New failures reopen the risk decision.

Governance shapes the safeguards a system needs. Evidence from use informs the next decision; each responsibility links to its explanation.

02 / Design and use

The conditions around the answer matter.

Privacy starts with purpose. Necessary service processing and optional uses need separate treatment, as do a caregiver’s information and information about someone receiving care. Tracing collection, consent, retention and deletion matters more than a general promise to respect privacy.

[8][3][9][29]

Security asks what prevents unauthorized access or action. Messages, retrieved documents and tool results are untrusted inputs. Permissions and authorization need enforcement outside the model; an instruction or a refusal cannot serve as an access control.

[10][11][12][13][14]

Inclusion asks whether people can meaningfully use and influence a system. Disability, language, fatigue, money, transport and care relationships affect what support is useful. Representative research, manual accessibility checks and comparisons across relevant groups answer different parts of this question. An overall average can conceal concentrated harm.

In the Oxford collaboration’s account of co-production, people with lived experience help shape decisions and outcomes alongside other contributors. That is a stronger role than being consulted after the important choices have been made. [28]

[15][16][17][3][28]

Transparency gives people a basis for judgment: when AI is involved, where information comes from, what a capability can do, and where uncertainty remains. Publishing an explanation is a beginning. Whether people understand and can use it requires a separate check.

[2][18][19][20]

These responsibilities qualify what “helpful” can mean. Secure software can still give unsafe advice. An accessible interface can still distribute burdens unfairly. A well-sourced answer can still use information without permission.

03 / Evidence and action

Evidence needs somewhere to go.

Evaluation makes a failure observable. Scenarios, red-team exercises and measurements should follow the intended use and hazards. Reports need to state which circumstances the cases represent, how scoring works, and where judgments can be wrong.

InvisibleBench contributes evidence from scripted caregiving conversations, reporting Safety and Care separately. Care Evals contributes public caregiver evaluation records and instruments. Neither project can stand in for privacy controls, security testing or accessible design.

[26][9][18][21][22][23][24]

Human oversight makes responsibility practical. A person needs the information, authority and capacity to intervene, challenge a decision or stop a capability. A model’s judgment cannot replace required human approval.

[3][18][25][27]

Monitoring and learning closes the loop. Failures after release should lead to containment, recovery, an investigation and changes with the owner who can act. Lessons become more useful when they also become checks against recurrence.

[27][14][18]

A safeguard, a benchmark result and a caregiver outcome answer different questions.

None alone establishes product effectiveness. Product owners remain responsible for preventing accidental harm, factual validity, reliability, robustness and safe failure. Missing evidence remains a gap, even when other checks pass.

04 / Provenance

A claim should lead back to its basis.

For a resource, that means its original publisher, title, version or date, canonical location, permitted use and limitations. For an evaluation, it means the origin and conditions of the cases, the scoring method and the evidence supporting the judgment. Model cards and datasheets offer useful patterns for keeping this context with the work. [19][20]

Source quality and personal-data permission require separate checks. Public methods should make scope and limitations inspectable while protecting caregiver records and restricted evaluation material.

References29 sources / View +

Numbered citations link to the entries below. ISO entries cite public scopes. Software references describe tools; caregiver rubrics and safety thresholds require their own validation.

  1. Voluntary framework · 2023

    NIST AI Risk Management Framework 1.0 and Playbook ↗

    Govern, Map, Measure and Manage connect accountability, context, evidence and action.

  2. Intergovernmental principles · updated 2024

    OECD AI Principles ↗

    Human rights, transparency, robustness and accountability.

  3. Health ethics guidance · 2021

    WHO: Ethics and governance of artificial intelligence for health ↗

    Autonomy, well-being, privacy, equity and responsibility in health contexts.

  4. Management-system standard

    ISO/IEC 42001:2023 ↗

    Requirements for an AI management system. Public scope consulted; no certification claimed.

  5. Risk-management guidance

    ISO/IEC 23894:2023 ↗

    Integrating AI risk management into organizational practices. Public scope consulted.

  6. Impact-assessment guidance

    ISO/IEC 42005:2025 ↗

    Assessing AI impacts on individuals, groups and society. Public scope consulted.

  7. Voluntary self-assessment · 2020

    EU Assessment List for Trustworthy AI (ALTAI) ↗

    Questions about agency, robustness, privacy, transparency, fairness, societal well-being and accountability.

  8. Voluntary privacy-risk framework · 2020

    NIST Privacy Framework 1.0 ↗

    Purpose, processing and individual control. The baseline here is published 1.0; the 1.1 initial public draft is distinct.

  9. AI RMF companion guidance · 2024

    NIST AI 600-1: Generative Artificial Intelligence Profile ↗

    Generative-AI risks and evaluation, including privacy and harmful outputs.

  10. Cybersecurity framework · 2024

    NIST Cybersecurity Framework 2.0 ↗

    Governance, identification, protection, detection, response and recovery.

  11. Secure development guidance · 2022

    NIST SP 800-218: SSDF 1.1 ↗

    Preparing, protecting and producing software, and responding to vulnerabilities.

  12. Application-security verification standard

    OWASP Application Security Verification Standard 5.0.0 ↗

    Versioned requirements for checking application security.

  13. Threat-awareness resource · 2025 edition

    OWASP Top 10 for LLM Applications 2025 ↗

    Injection, disclosure, excessive agency and other AI application threats.

  14. Incident-response guidance · 2025

    NIST SP 800-61 Revision 3 ↗

    Preparation, response and recovery within cybersecurity risk management.

  15. W3C Recommendation

    Web Content Accessibility Guidelines 2.2 ↗

    Testable accessibility criteria; Level AA is a design target, not a conformance claim here.

  16. W3C Working Group Note · 2021

    Making Content Usable for People with Cognitive and Learning Disabilities ↗

    Supplemental usability guidance beyond WCAG conformance.

  17. Research and guidance · 2022

    NIST SP 1270: Towards a Standard for Identifying and Managing Bias in AI ↗

    Systemic, human and computational sources of bias.

  18. Generative AI in health guidance · 2024

    WHO: Guidance on large multi-modal models ↗

    Transparency, oversight and continuing evaluation of generative AI in health.

  19. Mitchell et al. · research paper · 2019

    Model Cards for Model Reporting ↗

    Documenting intended use, evaluation conditions and limitations.

  20. Gebru et al. · research paper · 2021

    Datasheets for Datasets ↗

    Documenting dataset motivation, composition, collection and appropriate uses.

  21. Open-source evaluation framework

    Inspect ↗

    Datasets, solvers, agents, tools, scorers and evaluation logs.

  22. Implementation documentation

    Inspect: Scoring ↗

    Scoring structure and reporting; it does not supply a validated caregiver rubric.

  23. Open-source evaluation framework

    Harbor ↗

    Tasks, isolated environments, verifiers and artifacts for agent evaluation.

  24. Implementation documentation

    Harbor: Task format ↗

    Task and verifier structure; it does not set a caregiver safety threshold.

  25. Lifecycle implementation guidance

    NIST AI RMF Playbook: Govern ↗

    Accountability and human responsibility across the lifecycle.

  26. Lifecycle implementation guidance

    NIST AI RMF Playbook: Measure ↗

    Selecting and interpreting appropriate evidence.

  27. Lifecycle implementation guidance

    NIST AI RMF Playbook: Manage ↗

    Prioritization, intervention, recovery and continued improvement.

  28. Oxford collaboration · white paper · April 2025

    The responsible use of Generative AI in adult social care: A value-led approach ↗

    Care values, shared responsibility and co-production with people with lived experience. See pages 8–10; guidance rather than certification.

  29. Caregiver Action Network · guidance framework

    The Humanity Baseline: CAN’s Framework for Responsible AI in Caregiving ↗

    Caregiver choice, total burden, permission, human support and evidence in practice. Guidance, not certification.