Weighting a snowball sample through a pseudo-calibration: the case study of same-sex civil unions in Italy

Abstract

This paper presents an approach to addressing the challenges of selection bias and non-probabilistic characteristics of snowball sampling design, particularly in the context of social research involving hidden or hard-to-reach populations. The primary aim is to refine the weighting methodology of snowball sampling by introducing the pseudo-calibration technique to adjust the direct weights, equal to one, according to some available auxiliary variables, with the ultimate goal of producing more reliable and unbiased estimates.

The data for this case study is derived from a snowball sample survey, called the “Over the Rainbow” project, carried out on Instagram users who tag their photos with popular LGBTQ+ community hashtags. The username list is collected using web-scraping tools that identify relevant users. To address the sampling design's limitations, the study employs calibration and post-stratification methods. Calibration involves adjusting the weights of the sample data to match known population totals, while post-stratification involves dividing the sample into subgroups that align with known demographic distributions.

The proposed sampling weights are benchmarked on the Istat survey on all civil unions between same-sex couples, celebrated by Italian municipalities since 2016; this known totals source represents a reliable external reference to adopt sampling weights. The expected results of this study are twofold. First, the application of calibration should yield sample weights that are more representative of the target population, by aligning the sample distribution with known population totals. Second, post-stratification is anticipated to define pseudo-weights, adjusting the direct weights equal to one, refining the sample by ensuring that the subgroups within the sample correspond proportionally to those in the broader population. The combination of these methods is expected to significantly reduce the biases associated with traditional snowball sampling and attribute sampling weights not just equal to 1, as usually happens in snowball sampling.

The paper contributes to the statistical and social research field by offering a methodologically sound approach to improving the accuracy of snowball sampling designs, with practical implications for studying hard-to-reach populations on social media platforms.