Open and Responsible (Data) Science Citizenship

Material from Louise Bezuidenhout

Presented by Nii Tawiah & Ayishih Bellew

September 2023

Open and Responsible (Data) Science Citizenship

Material from Louise Bezuidenhout Presented by Nii Tawiah & Ayishih Bellew September 2023 — CODATA-RDA Data Science School

Our Curriculum

Responsible and Open Science citizenship sits at the center, connecting:

  • Information Security
  • Title permissions
  • Buffer overflow

Plan for the First Part of the Morning

  1. Responsible conduct of research
  2. Open science as a new way of being responsible
  3. Being a responsible, open science citizen

Note

Our session will be highly interactive and involve discussion. There will be 5 min breaks after 20 mins of being in session.

Responsible Conduct of Research

Not just about being:

  • “good at your work”
  • producing data
  • getting on with your colleagues

As knowledge producers, educators and recipients of public funds we have additional responsibilities as researchers.

Balancing Multiple Roles as a Researcher

  • Data producer
  • Data user and/or collaborator
  • Author
  • Employee
  • Teacher/mentor
  • Recipient of public funds
  • Recipient of public trust
  • Citizen/legally-obligated individual

How do all these roles fit together?

Responsibility as a Product of Integrity

Practice of scientific investigation with integrity

Key Ethical Norms

Responsible research involves the awareness and application of professional norms and ethical principles in all areas relating to research:

  • Beneficence (do good)
  • Non-maleficence (cause no harm)
  • Accountability
  • Transparency
  • Care/Equity
  • Collegiality

Codes of Conduct

Ethics are often outlined in disciplinary codes of conduct. They can be helpful ways of summarizing areas of activity to think about in relation to RCR.

https://www.acm.org/code-of-ethics

RCR as a Collaborative Endeavour

Institutional Responsibilities

  • Education
  • Infrastructures that support responsible research
  • Capacity for dealing with concerns/whistleblowing/misconduct
  • Engagement and compliance with inter/national regulation

RCR: Changing Practices and Environments

Understanding of responsible research, researchers and research contexts is changing:

  • Individual responsibility — expect compliance with regulations, active engagement with RCR behaviours
  • Institutions/National/International systems — expect investment in RCR-supporting infrastructures, develop systems of educating, monitoring and mediating
  • International science community — expect monitoring, support and capacity building

RCR as an Extended Form of Research Ethics

  • Is what I’m doing living up to these ideals?
  • Can I make things better?

Fostering ethical excellence instead of avoidance of misconduct

Extending the Reach of Ethics in Research

  • Produce verifiable and re-usable data
  • Protect scientists and societies from harm
  • Enable collaboration
  • Ensure investments (financial, trust, time etc.) are recompensed
  • Embeds science within cultural/social priorities

Increasingly Digital

  • Amount and types of data being produced
  • Practices of science
  • What constitutes “good research”
  • How responsible conduct of research is understood

The Digital Revolution

  • Data “deluge”
  • Big Data
  • AI
  • Increasing opportunities and prioritization of interdisciplinary research

https://www.britannica.com/topic/The-Fourth-Industrial-Revolution-2119734

The Digital Revolution: Coming at the Right Time?

Advances in computing have not only changed research, but come at a time of significant change in research.

A number of issues are changing the way we think/talk/govern research:

  • Reproducibility crisis
  • Increased scrutiny of public investment in research
  • Increased pressure for public access to research resources
  • Sustainable Development Goals (SDGs)
  • Blurred boundaries between academia, commerce and government
  • Citizen science movement

Driving forward new models of research practice

The Reproducibility Crisis in Research

A General Consensus on the Crisis

  • Variability in analysis and methodology
  • Incentives aligned towards publication not reproducibility
  • Lack of transparency and access to data

Nature survey of 1,576 researchers (Baker et al 2016)

SDGs, Research and Global Good

Increased support and funding for research that addresses SDGs and contributes to global development.

New Challenges for Responsible Research Models

  • Expanding scope of responsibility (led by SDGs)
  • Digital systems and distributed research making individual role responsibilities complicated
  • Inclusion of non-researchers in research process raises concerns about ethical practice
  • Urgent need to (efficiently) use and re-use volume of data being created raises concerns about:
    • Opportunity to share vs. loss of control
    • Increase benefits of research for public vs. possible harms
    • (Un)Intended marginalizations
    • Data recombination and re-use

New Challenges for Responsible Research Models (cont.)

  • How can the evolving power of digital technologies be harnessed to uphold the principles of responsible research and increase reproducibility?
  • How can the culture of science be adapted to support this evolution?
  • How can practices and structures of scientific research be adapted to ensure that research benefits the most number of people?

Tip

Are you aware of any specific cases in which unethical research or falsified research occurred? Can you share an example and how you think this affects us today?

Dr. Andrew Wakefield, a British Former Gastroenterologist

In 1998, Andrew Wakefield and 12 colleagues published a case series in The Lancet suggesting the MMR vaccine may predispose children to behavioral regression and developmental disorder. Despite the small sample size (n=12), the paper received wide publicity and MMR vaccination rates dropped due to parental concerns. The paper was subsequently retracted after epidemiological studies refuted the claimed link.

Time For A New Approach?

We need to start to effectively share.

Open Science

The products of scientific research should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control.

Gratis versus Libre

  • “Free” means there is no cost
  • Libre means “at liberty” — referring to the freedom to modify source code
  • Libre doesn’t mean gratis
  • Libre can mean available; libre can mean without restriction

Open Science (cont.)

The movement to make scientific research, data and dissemination accessible to all levels of an inquiring society.

Scope:

  • Transparency in experimental methodology, observation, and collection of data
  • Public availability and reusability of scientific data
  • Public accessibility and transparency of scientific communication
  • Using web-based tools to facilitate scientific collaboration

Open Science: an Umbrella of Many Activities

https://www.fosteropenscience.eu/content/what-open-science-introduction

Open Science is Many Things

Open Science includes activities that:

  • facilitate resource sharing
  • improve awareness of sharing
  • create linkages between resources
  • advocate for removal of financial barriers
  • advocate for just distribution of resources

Openness as an Aspiration not an Absolute

Openness is not an absolute for research outputs — different research products require different solutions:

  • Sensitive data
  • Commercial applications
  • National IP and traditional knowledge

Open Science movement drives critical engagement in these areas to find solutions that are suitable.

Openness is not an absolute in the research process — different forms of openness foregrounded at different times.

As Open As Possible, As Closed As Necessary

https://www.timeshighereducation.com/blog/data-should-be-open-possible-and-closed-necessary

Openness Throughout The Research Lifecycle

Credit: Heidi Seibold, tinyurl.com/opendatascience

Open Science: Opening Up Research Potential

<www.openuphub.eu/component/k2/item/610-101-innovations-in-scholarly-communications>

Describe Your Workflows

Activity (5 min):

A Way of Thinking And Doing

  • An ideology as well as a set of practical actions
    • Commitment to equity, excellence, responsibility and justice
    • Collaboration, sharing of research data, lab notes and other processes with no restriction; promote reuse, redistribution and reproduction of research and its underlying data and methods
  • Open Science = practice-based ethics
  • Like all ethics, it requires motivation, buy-in and commitment
  • Changing the way we think about responsible research

Different Motivations, Same Response

(Fecher and Friesike, 2014)

Openness in RCR

  • Open Lab Books: Transparency in research practices
  • Sharing and openness: enhance transmission of values
  • Open Peer Review: Transparency → better dialogue and collegial behaviour
  • Open Access: Improves availability of research outputs
  • Open Publishing: Leads to improved citations, credit and collaboration
  • Open Data and Open Methodologies: Improve transparency and reproducibility of research
  • Open Science Tools: Improve collaboration

Openness as a Continuum and a Journey

  • Open Science is both a cultural movement and a set of individual research decisions/practices
  • Commitment to openness is the start of a lifelong journey as an Open researcher — learning to be open takes time, motivation and mentorship/support
  • Individual researchers can show support for Open Science through the way they do their research
  • Can make changes to daily research practices to be more open in all aspects of research

Important

There is no “hard and fast” rule — openness must work for you within your specific work context and with your daily challenges.

Different Actions at Different Stages

Individual Openness: More Than Open Access

Share your data — the research data that underpins publications should also be accessible to support validation and facilitate reuse. In cases where data sensitivities won’t allow open access, be sure to provide details on how someone could request authorised access.

Share your code — many researchers now develop bespoke bits of code to help them analyse and/or visualise the data they have collected. Having access to this code is essential for supporting the validation of findings and to help others build upon your work.

Share your workflows — without knowing what steps were taken to capture, process and analyse the data — and in what order — it can be virtually impossible to validate published findings. This has led to what some are calling the Reproducibility Crisis.

Source: www.fosteropenscience.eu

Pragmatic Openness

Openness in research is new … and can be scary.

  • It’s ok to have concerns
  • It’s ok to realize that not all the open practices will work for you
  • There can be legal, regulatory, ethical reasons not to share

However, you benefit from the increasingly open research landscape (papers, data, software).

Tip

So, how can one avoid “freeloading”?

Responsible and Open Research as Citizenship

Citizenship:

  • ethical obligations arising out of social living
  • being part of a community requires the acceptance of civic responsibilities and contribution to the overall public good

As a citizen you have duties and expected ways of acting:

  • Follow rules
  • Participate in community activities
  • Protect the community and its resources from misuse

Responsible and Open (Data) Science Citizenship

  • Research is a community endeavour
    • involves social actions such as resource sharing and communal practice
    • responsible researchers are “citizens” of the research community
  • Citizenship is a give and take
    • Benefits to facilitate freedom of research
    • Structures to safeguard rights as researcher
    • Responsibilities to assume to protect culture
  • Support and grow culture instead of just living in it

Tools for Responsible, Open (Data) Science Citizenship

Guiding values: Openness, sharing, justice, beneficence

  • Data management: Norms and values (FAIR), Practical tools - RDM
  • Data science: Practical skills, Norms and values, Infraethics
  • Authorship: ORCID, Journals and licensing
  • Data use: Finding and using data, Data re-use and attribution
  • Ethics and responsibility: Areas for activity

Challenges of Being Open at Home

Implementing Open and Responsible Practices in Your Own Research

  • Challenges are common — everyone has them
  • Physical, social and regulatory contexts influence perceptions of Open Science and ability to engage in Open Science activities

SA1/3: “I think it leads to better science” KY1/1: “I won’t release data unless I first of all publish”

Experiencing Challenges is Normal

Tip

What specific challenges do you anticipate encountering when you return home in terms of your data work?

It’s OK To Have Challenges

Challenges can be categorized into a number of different areas:

  1. Cultural resistance and lack of institutional/peer support
  2. Resource limitations
  3. Personal concerns

Challenges are not insurmountable — many resources can help address them.

Cultural Resistance and Lack of Support

A quick background:

  • Inherited colonial academic systems
  • Historic lack of funding and resources limiting research scope
  • “Parachute research”

Problems include:

  • Lack of institutional support
  • Lack of regulations/guidance
  • Lack of trust

Getting Your Institution On Board

Piwowar 2018

Challenges Beyond the Research Context

Technology affects communication, collaboration and knowledge exchange within scientific, work and home settings.

  • Need to help people use those innovations more productively and safely
  • Need to improve ways in which new technologies can be designed and developed to be more responsive to societal acceptability and desirability

Not just about being open/closed. It’s about making sure that you use openness as a tool to secure just futures.

Networking and Future Collaboration

Activity (5 min):

Pair with someone in another organization and find out about their research interests.

Enhancing Healthcare with AI

Presented by Nii Tawiah & Ayishih Bellew September 2023 — CODATA-RDA Data Science School

Plan for the Second Part of the Morning

  • AI/ML in Health
  • Algorithmic Bias

Note

Our session will be highly interactive and involve discussion. There will be 5 min breaks after 20 mins of session.

What is Health?

“Health is a state of complete physical, social and mental well-being, and not merely the absence of disease or infirmity.”

— World Health Organization, 1948

Social Drivers of Health

What is Public Health?

“Public health is what we, as a society, do collectively to assure the conditions in which (all) people can be healthy.”

— Institute of Medicine (1988), Future of Public Health

Factors That Determine Public Health

To protect and promote the health of all people in all communities.

Health Equity

The state in which everyone should have an equal opportunity to attain their highest level of health.

Achieving this requires ongoing efforts to:

  • Address past and present injustices
  • Overcome barriers to healthcare
  • Eliminate preventable health disparities

A commitment to working towards a fair and just society where everyone can access the care and resources they need to live a healthy life.

Health Disparity

Health disparities are preventable differences in the burden of disease, injury, violence, or opportunities to achieve optimal health that are experienced by socially disadvantaged populations.

Health Disparity vs. Health Inequity

Health disparities refer to differences in health outcomes among different populations. These differences are not limited to one group.

Health inequities are disparities in health outcomes and the opportunities available to groups to achieve the best possible health. These differences are often caused by social and structural injustices that are avoidable and unfair.

Total Diabetes

  • 37.3 million people have diabetes — that’s 11.3% of the US population
  • 28.7 million people have been diagnosed with diabetes
  • 8.5 million people who have diabetes have not been diagnosed and do not know they have it

Diabetes Disparities by Race and Ethnicity

Percentage of US Adults 18+ With Diagnosed Diabetes, by Race and Ethnicity, 2018–2019:

Race and Ethnicity Percentage
American Indian or Alaska Native 14.5%
Asian, non-Hispanic 9.5%
Black, non-Hispanic 12.1%
Hispanic, overall 11.8%
White, non-Hispanic 7.4%

Diabetes Disparities by Education Level

Percentage of US Adults 18+ With Diagnosed Diabetes, by Education Level, 2018–2019:

Education Level Percentage
Less than high school 13.4%
High school 9.2%
More than high school 7.1%

Structural Inequities

Structures or systems in society, such as finance, housing, transportation, education, and social opportunities, are designed to unfairly benefit one population, intentionally or not.

Equality vs. Equity vs. Disparity

  • Equality is achieved when each person or group is given the same resources or opportunities.
  • Equity is achieved when it is recognized that each person or group has different circumstances, and resources are allocated accordingly to reach an equal outcome.
  • Disparity is a difference that is closely linked with social, economic, and/or environmental disadvantage.

https://www.youtube.com/watch?v=MlXZyNtaoDM

Inequality, Equity, Equality, Justice

Enhancing Healthcare with AI — Part 2

What is Artificial Intelligence?

Artificial Intelligence (AI): A term that addresses machine behavior and function that exhibits the intelligence and behavior of humans.

  • Rules-based: Algorithms programmed to use (generally clinically accepted) rules to guide decision-making.
  • Machine-learning (ML): Algorithm that uses data to create relationships without explicitly being programmed.

AI Building Blocks in Global Health

Source: AI@BCG, Encyclopedia Britannica, Oxford Dictionary

AI in Health

Figure from USAID’s “Artificial Intelligence in Global Health: Defining a Collective Path Forward” https://www.usaid.gov/cii/ai-in-global-health

Bias

Bias Inclination or prejudice for or against one person or group, especially in a way considered to be unfair.

Discrimination Unequal treatment of persons on the basis of “protected characteristics” such as race, sexual identity, etc.

Bias in algorithms Unjustified and/or unintended deviation in the distribution of algorithm outputs, with respect to one or more of its parameter dimensions.

What Causes Bias?

Bias in data selection — Use of unrepresentative datasets in algorithm development.

“…among the major factors that contribute to bias in the results that [systems] produce is because there is bias in the data. So you actually have to look at the data as far as the performance is concerned, to make sure you have a representative sample of the population you are trying to model.”

— Mittelstadt, Allo, Taddeo, Wachter, Floridi (2016)

Bias in Both Types of AI

Current Challenges

Facial Recognition Software

Joy Buolamwini (MIT)

  • Software created by brand-name tech firms such as Amazon uncovered much higher error rates in classifying the gender of darker-skinned women than for lighter-skinned men.
  • Other problems — unable to reliably detect Asian eyes.
  • Location of software companies and demographics = non-representative datasets used in algorithm development.

Significant Harms From Deployment of Algorithms

  • Called on Amazon to stop selling its facial recognition software to police.
  • Caution about the fast-moving adoption of facial recognition by police, government agencies and businesses from stores to apartment complexes.
  • Computer vision systems that enable self-driving cars to “see” the road show they have a harder time detecting pedestrians with darker skin tones.

Example 1: Algorithmic Decisions on Bail — Northpointe and COMPAS

Value-Laden Nature of Algorithmic Design

“Algorithms are inescapably value-laden. Operational parameters are specified by developers and configured by users with desired outcomes in mind that privilege some values and interests over others… [O]peration within accepted parameters does not guarantee ethically acceptable behavior… for example, profiling algorithms that discriminate against marginalized populations.”

— Mittelstadt, Allo, Taddeo, Wachter, Floridi (2016)

What is Algorithm Bias?

An algorithm that demonstrates significantly different performance in a subgroup of the population of interest.

An algorithm could be biased along one dimension or a combination of dimensions:

  • Demographic: racial, ethnicity, age, sex, gender, etc.
  • Socioeconomic: income, insurance status, etc.
  • Geographic: rural vs. urban
  • Health system: community hospital, academic health center
  • Comorbidities

How Are Machine Learning Algorithms Made?

Different Data Types

  • Testing data — used to test algorithmic performance after a system is trained.
  • Operational data — used to make a prediction once the algorithm is in use (e.g., a patient’s EHR and sensor data used to determine a risk score).
  • Clinical study/performance data — the results of evaluations of the algorithm.
  • Training data — used to initially teach the algorithm; learning algorithms use the data to find relationships between the data and the label/annotation (supervised machine learning) to make predictions.

How Does AI Become Biased or Unfair?

Factor Description
Biased training data Training set reflects historical inequities
Poorly selected features Features act as proxies for protected characteristics
Unbalanced datasets Underrepresentation of certain groups
Feedback loops Predictions reinforce existing disparities
Lack of transparency Cannot identify where bias enters
Focus on accuracy over fairness Aggregate accuracy masks subgroup harms

The Difficult Nature of Identifying Biases

(Based on the COMPAS recidivism analysis)

  • Only 20% of those predicted to commit a violent crime had gone on to do so
  • Of those deemed likely to re-offend, 61% went on to be arrested (including misdemeanours such as driving with an expired license)
  • Black people were almost twice as likely to be falsely labelled as at risk of future offending than white people
  • White people were mislabelled as low risk more often than Black people
  • Even when controlling for criminal history, recidivism, age and gender, Black people were:
    • 77% more likely to be labelled at risk of committing a future violent crime
    • 45% more likely to be labelled at risk of committing any kind of crime

On Correcting for “Real World” Bias

“Algorithms are inherently politicised [as connected to social policy and political power]… and reflect our current world view, our current social policy… If we are not explicit about that as well, if we are not transparent about that, that we value equality between men and women, then we are again creating bias at another level of the system.”

— Jirotka (2016)

Eliminating Human Bias?

In the early 2000s the US criminal justice system began using risk assessments to assist decision-making.

  • Assessments are based on algorithmic calculations to predict, for instance, how likely an individual is to re-offend or fail to attend court for sentencing.
  • Used to determine whether an individual should be granted bail or how long their sentence should be.
  • “Low risk” offenders given shorter sentences and perhaps kept out of jail entirely.

Overcome human bias, or …?

A Vision for Algorithmic Design

“We want our algorithms in a sense to follow higher values, moral values that we think are more important than giving an exact reflection of the world. And that I think is a very interesting, but also in a sense very shady area in which, are we going to use the data as it is? Or are we going to change the data, or not change but adapt the way we look at the data to serve our purpose of being non-discriminatory…”

Federal Efforts to Address Bias and Fairness in AI

How Do You Promote Data and AI Literacy?

  • Get independent researchers to check your code/data selection/results to expose biases
  • Always critically examine the decisions you’re making in your research and ask “why do I think that way?”
  • Be critical of the code and results you’re using — how did they get to the point they did?
  • Think about how other cultures will respond to your decisions

Important

Do the data/coding choices you made contribute to just present and futures? Are you upholding the moral values of societies?

References

Centers for Disease Control and Prevention. National Diabetes Statistics Report. https://www.cdc.gov/diabetes/data/statistics-report/index.html. Accessed September 9, 2023.

Office of Disease Prevention and Health Promotion. (2021). Healthy People 2020: Disparities. U.S. Department of Health and Human Services. https://www.healthypeople.gov/2020/about/foundation-health-measures/Disparities

Braveman, P., Arkin, E., Orleans, T., Proctor, D., & Plough A. (2017). What is health equity? And what difference does a definition make? Robert Wood Johnson Foundation. https://www.rwjf.org/en/library/research/2017/05/what-is-health-equity-.html

Hacker K, Auerbach J, Ikeda R, Philip C, Houry D. Social determinants of health—an approach taken at CDC. J Public Health Manag Pract. 2022;28(6):589-594.

Ledford H. Millions of black people affected by racial bias in health-care algorithms. Nature. 2019;574(7780):608-609.

Thank You

Nii Tawiah Assistant Professor, Delaware State University ntawiah@desu.edu

Ayishih Bellew Statistician, University of the Virgin Islands abellew@uvi.edu

Please feel free to contact us with any further questions!