Hello! Would you like to take part in our survey before we show you the data from our first wave of recruitment? The survey is about controversial topics in psychology, and it takes about 20 minutes to complete [Note: the survey is currently a demo, and will not save any responses. It's just for you to explore, if you wish]. If you click "Do survey" you will be taken to more detailed study information; otherwise you will be taken to the project overview.
[If popups are blocked, please click here to do survey.]
The purpose of this project is to understand the relationship between the cognitive biases and personality traits of researchers — here, around 8000 academic psychologists and people from some affiliated disciplines — and their views on a set of 16 controversial topics in psychology.
These topics include the importance of mathematical models in psychological theories, the relative contributions of nature and nurture to our cognitive capacities, the underlying similarity of all human minds, and the limits of our ability to understand patterns in nature. Our goal is to use these data to better understand whether fractures in scientific disciplines are caused by more general biases that researchers bring to their work.
Our data for the project include individual differences in cognitive style, as well as citation patterns among researchers, clustered into subfields. However, the present visualization just gives an overview of one part of our data: where respondents stand on the 16 controversial topics that we included in our survey.
This visualization is provided for exploratory purposes, and does not include statistical analyses. We recommend using a desktop/laptop computer as the visualizations are not optimized for mobile devices.
If you'd like to cite or keep track of our project and resulting publications, please bookmark the following OSF repository: https://osf.io/zyec9/.
This research was partially funded by the Templeton Foundation's project on Metaknowledge.
Choose a topic to display:
Mouse over for more detail. Click a cell to display regression plots.
Originally, we distributed the survey to people who published articles in journals classed as "Psychology" in the Web of Science database. Now, we are sharing the link to the visualization (and its accompanying survey) quite freely.
If you're interested in behavior, cognition or brains and think you might want to do the survey, we suggest that you look at the first few questions, and see if they're something you'd like to share an opinion on. If not, simply navigate away from the survey.
We welcome participation from both inside and outside academia. If you're interested in behavior, cognition or brains, we suggest you look at the first few questions and see if they're something you'd like to share an opinion on. If not, simply navigate away from the survey.
The survey is built with jsPsych, an open-source JavaScript library for running experiments in a web browser (plus a few custom plugins).
We recognize that these are complex issues and we want to learn more about people's attitudes to them. If each question took full account of all the relevant nuances, it would turn into an essay-length affair and we would get very few responses. Further, we aim to reach as broad an audience as possible, and there is probably no level of nuance that would make everyone happy.
We piloted all the questions several times, and used questions that had a reasonable distribution across the full range of the scale, so we are confident that most people are able to respond to our scales.
If you sometimes agree with one side and sometimes agree with the other, perhaps think if one of them is more central to your research than the other. If not, just choose one of the opt-out options, or just skip the question.
For the latest version of the survey, we included scales (or subscales) from following articles:
If you took part in the last couple of weeks, please just be patient and check back soon. We will manually review responses periodically, and add them to the visualization data.
If you took part a while ago and have a personal code (e.g., in an email from us, or in the current URL), but your responses are still not being highlighted, please email me with your code (justin.sulik@gmail.com).
This is merely an overview of one part of our data, for exploratory purposes. Detailed methods, participant demographics, and formal analyses are reported in our papers linked at https://osf.io/zyec9/.
The data is visualized using D3.js, an open-source JavaScript library.
We aim to maintain the dashboard here as long as possible, but please bookmark the project page at https://osf.io/zyec9/, where we will post any changes.
We are not web developers, so rather optimizing for different devices, we have chosen to focus on desktop/laptop computers and their browsers.
The data are available at https://osf.io/zyec9/ if you wish to conduct any analyses.
If you responded to an email invitation in the first wave of data collection, we retrieved your email address from Web of Science, an online scientific citation indexing service. It stores information from published research, including author information. If authors publish their email addresses along with their work, these are included in the database.
Otherwise, we would only have your email if you chose to provide it while answering the survey.
Our project falls into two main parts.
One part is a cluster analysis based on articles published in scientific journals. This involves widely available information such as author's names, email addresses, and article abstracts. We have retrieved this information from the Web of Science database of publications. Their privacy statement is available here. We process this information based on a legitimate interest in finding patterns in citation networks.
The second part is the present survey, exploring attitudes to core foundations in psychology. We process this information based on consent to take part in the survey. The analysis only occurs after we strip out identifying information.
Additionally, we hope to make an anonymized link between the two parts, as described in the following answer. Your consent to make this connection is asked for at the end of the survey.
After downloading the online data file for this survey, our first act is to strip out any personal information and replace it with anonymous key codes (e.g. 15493). At that point, we will have one file that contains just key codes and emails (where volunteered), and another file that has the anonymized survey responses, along with the key codes. Thus, the survey responses will have been anonymized before we analyse them.
In our cluster analysis of publications, individual articles are identified by another anonymous key code (e.g. MEJGALQ), and a separate file stores the author information published with those articles, along with those second key codes.
If you consent to us anonymously linking your survey responses to your publication area by providing us with your email address, we would generate a third key code (e.g. A4X3G6), and then run a script that uses your volunteered information to find and replace the first two codes, so that both the survey responses and publication models now have the same new code (e.g., A4X3G6). This new code is not stored together with personal information such as email addresses. Thus, both the publication model and survey model are anonymous at the time of analysis, and the link between them is also made anonymously. We stress that the aforementioned publication model represents authors as numeric vectors in abstract high-dimensional spaces, and thus does not contain identifying information.
We will only use personal data for the purposes described in the survey.
In the survey, you can consent to our making a link to your research area, described above.
Additionally and separately, after completing the survey we provide you with a personalised link, so you can see your own responses in relation to others. This personalised link is not connected to any information like your email address (where volunteered). However, if you share this link, anyone using it will be able to see your responses. We thus recommend that you do not share it.
Finally, if you leave a comment on the survey that invites a response from us, and if you have provided an email address, we will use that email address to respond to you.
We will only use anonymized data for the preparation of scientific journal articles.
Personal identifying data (e.g., where participants consent to our using their email address to link their survey data and their publication record, which we scrub after making the link) is only accessible to the project team named at https://osf.io/zyec9/.
Anonymous survey responses are shared publicly at the same link (https://osf.io/zyec9/).
We will gladly oblige if you request that your survey responses and associated information are deleted, request that we do not anonymously connect your survey responses to your publication area, or request to see what data we hold for you.
Simply email justin.sulik@gmail.com with your request. For this reason, we provided an anonymous code at the end of the survey, and recommended that you record it.
There are some inaccuracies in the Web of Science database. Let us know by emailing justin.sulik@gmail.com, and we'll update our records.
If you would like Web of Science to correct the data they hold for you, please contact them at data.privacy@clarivate.com.
We will retain a copy of the email/key-code list until the scientific articles resulting from this research are published. Since participants can withdraw consent or ask us to delete their data at any time before publication, we need to retain this information until that time. We will retain the anonymized data indefinitely.
In published papers, we report data at the aggregate level. For example, based on our analysis of publication clusters, we might conclude that research subfield X is more likely than research subfield Y to endorse a particular belief. Those research clusters will not be described in any way that could identify individuals. Thus, if a research cluster only contains a few authors, we would not report on it.
For similar reasons, while processing participants' reported research topics, we excluded any topics studied by fewer than 10 people, so that unique topics cannot be matched to an individual.
The survey responses are initially collected and stored in the USA on the MongoDB platform. Their data protection statement is at https://www.mongodb.com/cloud/trust/compliance/gdpr.
When one of the research team named below downloads the raw data from MongoDB, their first act is to strip out the personal information (such as provided email addresses), replace it with anonymous key codes. We do not retain a copy of the processed survey responses together with personal information.
The email/key-code list is stored on the research team's password-protected computers, and is stored in encrypted and password protected form, only being accessed at specific moments for the purposes described above. The anonymized survey responses are also stored on the research team's computers, but are not encrypted since they contain no personal information.
Finally, anonymous data is stored on at the aforementioned project repository at https://osf.io/zyec9/. The data for this visualization (a subset of the OSF data) uses a new randomly generated anonymous code to identify each response.
For questions about the data dashboard/visualization or the survey, please contact Justin Sulik:
More details about the project (including project team), see https://osf.io/zyec9/.