Defining JUstness: 2015 Methodology
JUST Capital’s 2015 research proceeded in five phases, the first two of which ran concurrently in February and March of 2015. Across every phase of research, JUST Capital ensured that surveys matched the U.S. Census across major demographic markers: age, household income, employment status, census sub-region, ethnicity, education level, and gender.
Phase 1 : Determining how to Measure the Elements of Justness
In Phase 1 (January-February), we conducted an online survey to test different methods of assigning weights to factors in a model for corporate justness. Its results confirmed the usefulness of a discrete choice model (DCM), which was implemented in Phase 4 of the research and is described in depth below.
Phase 2: Listening to the American Public
In Phase 2 (February), we conducted qualitative research that involved open-ended conversations in focus groups, online, and by telephone, asking 212 people from all regions and walks of life what they thought corporations should and should not do, where corporations’ obligations lay, and how they thought corporate leaders should handle hard situations where obligations were in conflict. This research produced a comprehensive list of unique topics defining JUST corporate behavior.
Focus groups took place in 10 US cities and sought to balance representation to census categories and political ideology. We also took steps to include the views of people who are unlikely to be represented in the populations from which focus groups are drawn. Since focus groups largely draw on suburban and urban residents, we conducted an online discussion group to reach rural people across the country. Finally, we conducted one-on-one telephone interviews to ensure representation of offline Americans (who are disproportionately low-income or visually impaired) and those from underrepresented regions.
Our focus group moderator, online moderator, and phone representatives followed a discussion guide that facilitated a wide-ranging conversation about what constitutes JUST corporate behavior
22 Focus Groups
Albuquerque, Allentown, Birmingham,
Chicago, Dallas, Los Angeles, New Orleans
New York, Seattle, Wichita
Grouping by ideology (conservative, moderate, liberal, apolitical, and mixed)
32–Person online Ethosphere discussion group
Used to target online rural populations
26 In–Depth Interviews (IDIs)
Phone interviews with offline Americans (including visually impaired and low-income) and residents of underrepresented regions
Phase 3: Grouping the Elements of Justness
In Phase 3 (May), we conducted an online survey of 15,001 people conducted in order to consolidate and clarify the key elements in the public’s definition of justness.
First, we organized the themes raised by respondents in previous surveys into 38 logical categories, such as cost of living raises, CEO-to-average-worker pay ratios, equal pay by gender, and more. We then prepared a survey designed to confirm or amend what the qualitative respondents had said, based on a demographically representative online sample of 15,001 people. We chose this large sample size in order to allow us to make statistically valid assessments of subgroup opinions.
This survey asked people whether they agreed that specific items belonged in the definition of each category of justness. It also asked them to rank topics by importance within each category, and to rank categories relative to each other.
Phase 4: Grouping Justness into Clear Categories
In Phase 4 (July), we conduced an online survey of 20,001 people, using a 'discrete choice model' to determine the ’relative importance (or weight) of different topics in the minds of respondents.
Information we learned about the public’s definitions and priorities in each category was used to consolidate the 38 categories of Phase 3 into 15 attributes for our Phase 4 weighting survey. The 15 items that resulted represented a comprehensive, non-overlapping definition of corporate justness according to the American public
While most corporate responsibility evaluation models set weights based on expert opinion, we derived weights from public opinion using the discrete choice model, a technique that has been widely used in product research and demonstrated to predict behavior.
How Did the Discrete Choice Model Work?
Respondents were shown sets of hypothetical corporations, described as having different levels of performance on attributes of justness. Respondents chose which corporation they considered the most just, and whether that corporation was just. Statistical analysis consisted of a Hierarchical Bayesian analysis using a part-worth model to derive implicit weights of factors in respondents’ decision-making.
Our July 2015 Phase 4 weighting survey, like the May Phase 3 definition survey, was mapped to census categories. It drew on a sample of 20,001 people. Again, we chose a large sample size in order to allow us to make statistically valid assessments of subgroup opinions. The survey produced a set of relative weights which the 15 attributes, on average, have in people’s evaluations of corporate behavior.
Phase 5: Streamlining Categories for Clarity
In Phase 5 (August), we conducted a survey of 5,002 online respondents to confirm the rankings, definitions and best terminology to describe the factors of corporate justness.
First, we performed factor analysis, a statistical technique that identifies correlations among attributes, and determined that some, including the quantitative and qualitative aspects of employee relations, were closely correlated. Informed by this analysis, we combined several pairs of attributes, producing a final model with ten factors. While this combination has no effect on the weights of attributes or the eventual corporate scores the model will produce, it clarifies the content and makes it more accessible and memorable.
The 2015 model had 10 factors, derived from the 15 attributes of the Phase 4 survey, and ultimately from the most significant of the original topics raised by respondents in the qualitative phase.
- Employee pay and benefits
- Employee satisfaction and fair hiring
- Product benefit or harm
- Human rights record
- Governance and profitability
- Environmental impact
- Customer satisfaction and value
- Community relations and charitable giving
- Corporate ethics and honesty
- US job creation
JUST Capital solicited third-party assessments of the survey work done in 2015 from two independent academics: Tom Guterbock of University of Virginia and Joel Huber of Duke University. These assessments were reviewed by the team at Penn Schoen Berland, JUST Capital’s partner in survey research, and were addressed in a response. All three methodological review documents can be found below.