Methodology

Our paper conducts a survey of U.S. workers (similar to the Pew and Bick, Blandin, and Deming (2024) Generative AI surveys) with IncQuery specifically asking about the use of LLMs in specific tasks and occupations. One goal is to determine whether AI automatable tasks identified by papers such as Eloundou et al (2023), Brynjolfsson, Mitchell, Rock (2018), Webb (2020), and Felten et al. (2023), are actually being used in the context of LLMs.

The survey used a simple random sampling approach with a probabilistic draw of the sample from Dynata. The unit of analysis are workers in the U.S. The survey was self-administrated using a CAWI script.

The questions used in our survey are similar to the survey conducted by Bick, Blandin, and Deming (2024) (which instead uses Qualtrics), asking several questions about Generative AI use as well as other democratic questions about education level, industry, income and many other metrics. Many of their questions are repeated verbatim in our survey. Many of their questions are focused on the extensive margin in terms of AI use. With respect to time and frequency of Generative AI use, their central question (which we repeat in our survey) is fairly generic and begins with defining Generative AI and Large Language Models. A complete overview of our survey structure can be downloaded below.*

For the purpose of maximizing our sample size of respondents using Generative AI, we ask demographic questions prior to the Generative AI questions and then screen respondents out who do not use Generative AI at work while maintaining the demographic data of those removed in order to produce estimates of Generative AI use.

We also screen out respondents under the age of 18 like Bick, Blandin, and Deming (2024), however we include those above age 65 as one difference between surveys. We also use several attention checks.

When using our survey please cite:

Hartley, Jonathan and Jolevski, Filip and Melo, Vitor and Moore, Brendan, The Labor Market Effects of Generative Artificial Intelligence (December 18, 2024). Available at SSRN: https://ssrn.com/abstract=5136877 or http://dx.doi.org/10.2139/ssrn.5136877