Human Life Domains, Missing Perspectives and AI Evaluation
Conceptual idea: AI development could benefit from better evaluations of how systems integrate evidence across ordinary human life, recognise underrepresented perspectives, distinguish uncertain claims from errors, and identify weaknesses in their own evaluation methods. This proposal is open to contributions from researchers, practitioners and members of the public; it is not a measure of any one person's knowledge or the current performance of a specific AI model.
Research directory · Public contribution guidance · Research question · Paper protocol · Log
1. Working coverage map: 10 life domains, 57 illustrative areas
This is a non-exhaustive organising taxonomy. The count of 57 describes topics in this working map, not a representative survey, unique subject count across all conversations or comprehensive inventory of human experience.
| Domain | Areas | Illustrative subjects |
|---|---|---|
| Personal development | 5 | Identity, Values, Life objectives, Daily routines, Autonomy and boundaries |
| Relationships and family | 5 | Friendships, Romantic expectations, Family relations, Parenting, Domestic compatibility |
| Education and employment | 6 | Formal education, Academic recognition, Recruitment, Career transitions, Professional skills, Income and benefits |
| Domestic life | 6 | Housing, Architecture, Cooking, Clothing, Fitness, Household organisation and storage |
| Culture, faith and heritage | 6 | Faith, Community worship, Community histories, National identity, Genealogy, Language preservation |
| Creative life | 5 | Music, Visual arts, Photography and video, Writing and publishing, Traditional crafts |
| Society and institutions | 6 | Law, Institutional accountability, Government, Social class, Economic systems, Civic involvement |
| Science, health and environment | 5 | Health-information verification, Cognitive science, Resilience, Horticulture, Historical conservation |
| Technology and engineering | 6 | Websites, Audio-visual engineering, Automation, Cybersecurity, Computing infrastructure, Immersive technology |
| AI and research | 7 | Model evaluation, AI safety, Evidence provenance, Contextual reasoning, Multi-agent systems, Future intelligence, Research originality |
2. Candidate model-evaluation limitations
These are research priorities, not assertions that every AI model fails at them. Each requires controlled testing and comparison with the strongest relevant research.
- Longitudinal evidence: retain changes over time without replacing evidence with broad personal judgments.
- Causal inference: separate what happened before an event from why it happened.
- Cultural precision: separate language, ancestry, nationality, place of upbringing and self-description.
- Practical and embodied knowledge: test advice against observed material constraints and outcomes.
- Conflicting human values: preserve meaningful disagreements instead of manufacturing consensus.
- Evaluator self-correction: test whether the system can recognise incorrect rubrics, missing categories and false exclusions.
3. Potentially underrepresented evidence types
Further investigation should seek counterexamples and not assume these areas are absent from all published research: everyday negotiation among people with different needs, documented repair-and-recovery episodes, oral and tacit knowledge, successful as well as unsuccessful institutional decisions, and unexpected questions that existing frameworks fail to ask.
4. Questions that contributors and researchers can test
These 29 candidate questions are proposed prompts for hypothesis formation, comparison and evidence collection; they are not validated findings. Proposed studies should specify comparator methods, provenance, uncertainty, success criteria and falsification conditions.
1. Longitudinal evidence and context
- How should a model distinguish a one-off event from a sustained pattern without making a character judgment?
- When does adding a historical record reverse a well-supported conclusion, and how should the revision be explained?
- How should missing or contradictory timelines affect confidence?
- What is a fair test of context retrieval when relevant evidence appears early, late or between unrelated material?
2. Causal reasoning and institutional outcomes
- Which evidence separates a documented cause from mere sequence or correlation?
- What alternative explanations must be ruled out before attributing a decision to bias or misconduct?
- Can an AI identify when a proposed intervention would change the outcome rather than just the explanation?
- What do successful institutional corrections reveal that failure-only samples miss?
3. Cultural and identity precision
- Can an AI keep citizenship, ancestry, lived upbringing, language and self-identification separate?
- When does an apparently accurate cultural category conceal important differences within the group?
- How does changing the language of the question alter the answer?
- Can the model ask for missing biographical information rather than infer identity from appearance or slang?
4. Practical and tacit knowledge
- How can firsthand craft or maintenance knowledge be tested against physical outcomes?
- Which information is lost when a practical procedure is summarised as abstract instructions?
- How can a model distinguish local expertise from a generally valid recommendation?
- What controlled real-world observations would disprove a confident but impractical answer?
5. Disagreement and value conflict
- When do conflicting annotations reflect genuine values rather than rater mistakes?
- Can competing values be represented accurately without forcing artificial consensus?
- How does a model explain a trade-off without imposing an unrequested moral ranking?
- How should evidence, preference and unresolved ambiguity be kept apart?
6. Evaluator fallibility and epistemic gaps
- Can a model flag instances where its own rubric, rather than an input, is defective?
- How many eventually validated ideas were rejected initially, and for what reason?
- Can independent reviewers identify underexplored questions that an AI failed to generate?
- Which uncertainty signals predict wrong acceptance, wrong rejection or overconfident silence?
- When do multiple-model reviews amplify the same mistaken assumption?
7. Missing perspectives and public knowledge
- Which domains of everyday cooperation lack representative benchmark cases?
- Which orally transmitted or locally held knowledge claims can be verified without exposing private people?
- What positive institutional outcomes are missing from failure-focused datasets?
- Can contributors supply counterexamples that alter an established evaluation category?
5. Connected research project and publication status
Project: Cross-Domain Epistemic Evaluation and Public Knowledge Contributions. The planned deliverables are a provenance-aware question bank, a consent-respecting synthetic/public benchmark, baseline model comparisons, a contradiction/error taxonomy, adjudication guidance and a reproducible results register. No experiment or result is claimed completed.
Paper initiative: Cross-Domain Evidence Integration and Evaluator Fallibility in Language Models: A Public-Interest Benchmark Protocol. Read the working protocol.
Connection to existing work: Self-Calibrating AI Safety Filters (false exclusion and evaluator reliability), Collaborative Intelligence Initiative (independent cross-model review), Research Collaborations (public participation).
6. How others can contribute
Useful submissions could include: a precise question; a public or appropriately licensed evidence source; an anonymised counterexample; an alternative interpretation; a reproducible test; or a correction to an existing claim. State what is observed, what is inferred, what would disprove the claim, and whether any material can be shared publicly. Existing research enquiry route is available for proposals. This page does not automatically accept submissions, recruit study participants, or authorise use of submissions as AI training data.
Safeguards: do not submit private employment, medical, family or identifying records; seek consent where data relates to others. Public examples should be non-identifying, rights-cleared and suitably reviewed. An invitation to contribute is not a confirmed collaboration or a promise of publication.
7. Initial literature and comparison register
These sources provide established starting points; their publication dates are shown. The proposed work must test what remains unresolved after accounting for these methods, and avoid claiming originality on topic alone.
| Source and publication year | Relevance |
|---|---|
| 2023 — NIST AI Risk Management Framework | Risk, accountability and ongoing evaluation |
| 2024 — Lost in the Middle | Limitations of long-context evidence use |
| 2024 — CausalBench: Comprehensive causal reasoning benchmark | Interventions and causal-direction tests |
| 2024 — The Perspectivist Paradigm Shift | Disagreement may reflect interpretive differences |
| 2024 — Cultural bias and cultural alignment of large language models | Cultural variation in output behaviour |
| 2025 — Disentangling Language and Culture | Separate language from cultural context |
| 2025 — NUTMEG: Separating Signal From Noise | Disagreement-aware annotation and evaluation |
| 2026 — Beyond Consensus: Perspectivist Modeling and Evaluation | Open challenges in disagreement-aware evaluation |
8. Practical pilot toolkit: synthetic cases and transparent scoring
November 2026 · Created, not experimentally validated. Twelve fictional practice fixtures provide two examples for each of six evaluation themes. Each has supplied evidence, a question, an expected bounded conclusion, assessment targets and indicative error patterns. They are public development examples, not held-out benchmarks or evidence of model performance.
Pilot instructions and scoring rubric · 12 practice cases (JSONL) · Reusable contribution and evaluation template
A provisional five-dimension rubric scores evidential support, justified uncertainty, handling of contradictions, context fidelity and avoidance of unsupported inference. Independent reviewers should retain disagreements and flag serious failures separately rather than relying on an average alone.
9. Additional prior-art overlap and research distinction
METER (2026) has already benchmarked multi-level contextual causal reasoning and reported distraction by irrelevant information; TFRBench (2026) evaluates reasoning behind forecasting systems, including multi-agent verification. These are relevant established contributions. A potential additional research contribution here would require demonstrating a measured benefit from joint cross-domain evaluation, evaluator self-error detection and rights-respecting correction beyond these existing methods. This remains untested.
10. Falsifiable research plan
Compare long-context, retrieval-assisted, independent multi-model and staged self-audit approaches on matched, non-sensitive tasks. Measure evidence recovery, temporal accuracy, causal identification, cultural category errors, disagreement preservation, legitimate-knowledge rejection, unsafe acceptance, cost and correction quality. Record null and negative results. A hypothesised improvement fails if it does not reproduce against the strongest relevant baseline, or violates privacy/safety criteria.