Methodology
A. Needs Assessment
NAHMS develops study objectives by exploring existing literature and contacting industry members about their informational needs and priorities during a needs-assessment phase. A driving force of the needs assessment was the desire of NAHMS to receive as much input as possible from a variety of operators, as well as from industry experts and representatives, veterinarians, extension specialists, universities, cattle organizations, allied industry groups, and other stakeholders. Information was collected via a needs assessment survey.
The objective of the needs assessment survey for the NAHMS Health Management on U.S. Feedlots 2021 study was to identify critical information needs concerning cattle management and health on feedlots. The online survey gathered opinions from a variety of stakeholders regarding beef feedlot management priorities, health priorities, antimicrobial stewardship, industry burdens, and participation incentives for the study. The survey was available online from March 19 through April 30, 2019. The online questionnaire was distributed via email lists and stakeholder announcements. All individuals involved in the cattle industry were encouraged to participate, regardless of cattle ownership. In total, 206 individuals from 36 States completed the study’s needs assessment survey.
Respondents to the needs assessment survey represented the following affiliations:
- Veterinary practitioners or consultants—42 percent of respondents.
- Beef producers (feedlot owners/managers, cow-calf producers)—27 percent of respondents.
- Government and university employees—21 percent of respondents.
- Other affiliation—10 percent of respondents.
Based on input from the needs assessment, reviews from the scientific literature, and input from government and industry researchers, primary study objectives were identified:
- Describe health management practices on U.S. feedlots with 50 or more head
- Estimate the prevalence of important feedlot cattle diseases
- Describe antibiotic use and stewardship practices on U.S. feedlots
- Describe trends in feedlot cattle health management practices and important feedlot cattle diseases
B. Sampling and Estimation
1. State selection
The goal for NAHMS national studies is to include states that account for at least 70 percent of the animals and operations being studied. This method helps to ensure that the representation of the sample collected, and the statistical inferences made based on the sample data, can be generalized to the target population.
States were selected for inclusion in the study based on the number of feedlots and the number of cattle in two capacity categories: small feedlots (50 to 999 head capacity) and large feedlots (1,000 or more head capacity). Small feedlots were selected from 18 States and large feedlots from 17 States.
Of the 18 States chosen for the small component, 5 (Indiana, Michigan, Ohio, Pennsylvania, and Wisconsin) were chosen only for the small component, and 13 (California, Colorado, Idaho, Illinois, Iowa, Kansas, Minnesota, Missouri, North Dakota, Nebraska, South Dakota, Texas, and Wyoming) were chosen for both the small and large components. Of the 17 States chosen for the large component, 4 were chosen only for the large component (Montana, Oklahoma, Utah, and Washington), and the 13 previously mentioned States were chosen for both the large and small components.
2. Feedlot selection
The list frame from which feedlots were sampled is managed by NASS and was updated with information from the 2017 Census of Agriculture prior to sample selection. Within each State, a stratified random sample was selected in which strata were defined by cattle on feed inventory. For small feedlots, a total sample of 3,165 feedlots was selected. All large feedlots (2,177) with 1,000 or more head inventories on the NASS list frame in the 17 participating States were selected.
For the small component of the study, the total sample size was computed to achieve prespecified precision criteria at the 95-percent confidence level, while accounting for the estimated population size, design effect, and expected response rate. The sample size was allocated to strata approximately proportional to feedlot capacity, based on a weighted average number of feedlots and the total inventory within the strata. For the large component of the study, there was no sample selection because all 2,177 feedlots with 1,000 or more head capacity in the 17 participating States were selected. This sampling design allows for logistical efficiencies in administering the survey and prespecified precision for estimates.
3. Population inferences
The 18 States for the small component accounted for at least 95.7 percent of the inventory on feedlots with 50 to 999 head and 94.0 percent of feedlots with 50 to 999 head. The 17 States for the large component accounted for at least 95.5 percent of the cattle on feed on feedlots with 1,000 or more head inventories, and 94.4 percent of feedlots with 1,000 or more head inventory.
SUDAAN software (RTI, version 11.0.4) was used to produce population estimates and their standard errors. The SUDAAN software allows estimation of standard errors for complex sampling designs using Taylor series linearization.
a. Phase I: Health Management on U.S. Feedlots 2021 Phase I questionnaire
To construct the Phase I survey weights, the inverse of the probability of selection (with probabilities being approximately proportional to stratum size) was used as the initial weight. Nonresponse was accounted for using an additional adjustment according to the proportion of nonrespondents within each stratum, using a propensity score model. Calibration to population totals was performed using information available for respondents and nonrespondents.
Estimates for Phase I represent 38.5 percent of feedlots in the 22 participating States (18 States for the 50 to 999 head small component and 17 States for the 1,000 or more head large component, with most states participating in both the small and large components), after taking into account the survey design and weighting (see Section II.E.1. for more information on the calculation of the weighted response rate).
C. Data Collection
1. Phase I: Health Management on U.S. Feedlots 2021 Phase I questionnaire
Due to restrictions in place to prevent the spread of COVID-19, interviews took place over the telephone or through a web survey, rather than the in-person interviews around which the study was originally designed.
From March 1 through April 30, 2021, producers completed the Health Management on U.S. Feedlots 2021 Phase I questionnaire via a self-administered paper survey sent through the mail, a self-administered web survey, or a telephone interview with a NASS enumerator. Producers were provided with a phone number to a NASS enumerator and with a supplemental sheet to help them answer questions on the survey. The interview took an average of 58 minutes to complete.
Upon completion of the interview, producers were asked to provide consent to allow NASS to turn contact information over to APHIS for the opportunity to participate in Phase II of the study. This completed Phase I of the study. NASS provided the list of producers willing to participate in the second phase of the study to NAHMS so that NAHMS coordinators in each state could begin contacting consenting producers for Phase II of the study. Results from the Phase II questionnaire will be reported in future publications.
D. Data Analysis
1. Validation
Data were entered by NASS staff into a SAS data file and checked for validity. NAHMS staff independently performed data validation checks on the data set to identify consistency and statistical issues. Consistency issues include logical inconsistencies within a survey and were identified using summaries of responses to check for invalid responses (e.g., a response of ‘3’ for a 0/1 response variable); threshold checks (e.g., identifying invalid total sums of cattle on feed inventory); and, if-then checks (e.g., if all cattle were born and raised on the feedlot, then there should not be a source for added cattle).
Statistical issues were identified by investigating summary measures of responses for variables; extreme outliers were investigated by data analysts and subject-matter experts. Inconsistencies were identified using SAS software, and electronic questionnaire data was reviewed by data analysts and subject-matter experts. Identified inconsistencies were addressed using item-level imputation measures if appropriate values could be logically deduced.
2. Estimation and confidence interval calculations
Summarization and estimation were performed using SUDAAN software, which accounts for the stratified sampling study design. Confidence intervals were computed for estimate proportions, means, and ratios using the methods described in detail in the SUDAAN Language Manual for SUDAAN version 111 and described briefly here. For percentages, a logit transformation was used to enforce bounding of the confidence interval bounds between 0 and 1. Student’s t confidence interval bounds are computed on the logit scale and are then back-transformed to the percentage scale. For means and ratios, standard Student’s t confidence intervals are computed directly on the scale of the data.
Estimates were generated by one analyst, and numbers and estimation code were reviewed by a second analyst, to ensure accurate reporting of estimates.
E. Sample Evaluation
This section provides counts and percentages of feedlots by response category, which can be used to compute various measures of response. Historically, the term “response rate” was used as a catch-all parameter, but there are many ways to define and calculate response rates. Therefore, counts and percentages of feedlots by response code category are presented below so that response rates can be calculated according to the preferred definition of “response rate.”
Additionally, the Office of Management and Budget (OMB) has provided guidance regarding the calculation and reporting of response rates in their Standards and Guidelines for Statistical Surveys (2006), Section 3.2. The response rate advocated in the OMB guidance estimates the percentage of eligible feedlots that completed the questionnaire. The calculation of this specific response rate is presented for Phase I of the study below.
1. Phase I response rates
Of the 5,342 operations selected for participation, 1,300 were ineligible (no cattle on feed, out of business, backgrounder/stocker operation only, or otherwise out of scope). Of the 4,042 eligible operations, 1,967 were not contacted (office holds, purposefully not contacted, and inaccessible operations). Of the 2,075 eligible feedlots that were contacted, 1,025 (390 + 635) provided complete questionnaire data. Of those, 390 feedlots agreed to be contacted for the Phase II of the study.
*Weighted percentages calculated using the initial sampling weights.
According to the OMB guidance, the response rate for this study would be calculated according to the following formula:
Letters a, b, and d represent the counts (or percentages) of operations in each response-category group in the table above and ρ is the proportion of the noncontacted operations expected to be in-scope. Specifically,
Thus, the OMB guidance-based response rate for Phase I of the NAHMS Health Management on U.S. Feedlots 2021 study is calculated as follows:
Approximately 31.2 percent of eligible feedlots completed the Phase I questionnaire. The weighted OMB guidance-based response rate for Phase I of the NAHMS Health Management on U.S. Feedlots 2021 study is 38.5 percent (calculated using the initial sampling weights), which means that Phase I questionnaire information is available for approximately 38.5 percent of feedlots in the 22 participating States (18 States for the 50 to 999 head small component and 17 States for the 1,000 or more head large component, with most States participating in both the small and large components), after taking into account the survey design and weighting.
Additionally, due to the high number of operations that were not contacted, it is instructive to observe the cooperation rate (the American Association of Public Opinion Research’s defined cooperation rate number 3)2. This rate is defined according to the following formula.
Or approximately 49.4 percent of contacted eligible feedlots were willing to complete the Phase I questionnaire.
2. Communicating response rates
The unweighted response rate, 31.2 percent, for Phase I is the rate that will be used, generally, to communicate the response rate for Phase I of the NAHMS Health Management on U.S. Feedlots 2021 study, as it represents the likelihood that eligible feedlots completed the Phase I survey.
In addition, when communicating specifically about cooperation, the cooperation rate (49.4 percent for Phase I) will be used to communicate the likelihood that contacted, eligible producers were willing to complete Phase I of the NAHMS Health Management on U.S. Feedlots 2021 study.
1 Research Triangle Institute (2012). SUDAAN Language Manual, Volumes 1 and 2, Release 11. Research Triangle Park, NC: Research Triangle Institute.
2 American Association of Public Opinion Research (2023) Standard Definitions, Final Dispositions of Case Codes and Outcome Rates for Surveys. https://aapor.org/wp-content/uploads/2023/05/Standards-Definitions-10th-edition.pdf.