This is a linkpost for https://laurenpolicy.substack.com/p/reading-glasses-might-be-one-of-the
Most development interventions are considered “good buys” (for donors or governments) if they return $15 in social benefits for every $1 spent.
Every $1 spent on reading glasses could result in $46 more in earnings.
I'm extremely skeptical of the finding that reading glasses dramatically increase income. After looking into this topic for the past year (having initially been excited by these studies), I would now guess that the significant findings are more a result of experimenter demand effects than of reading glasses.
For example, in the paper on tea pickers you mention, the research team made seven unannounced visits to assess “compliance with study glasses” over the course of the 11-week trial. But we know from past research (e.g. Zwane et al. 2011) that monitoring changes behavior (as evidenced by the uptick in glasses usage). In that case, the estimate from the trial will capture the effect of monitoring + reading glasses which is not an effect we'd ever observe in the real world. The productivity of the control group also increases by 20% between baseline and endline but I don't believe the authors provide any potential explanations for this substantial increase.
Perhaps more tellingly, I've now visited tea growing regions in rural Kenya multiple times this past year. I've observed hospitals giving reading glasses to tea pluckers and I've asked people if they would use them when plucking tea and they all say no. Or if I go to an area where the hospital has distributed glasses in the past, I have never found workers wearing them in the field. That's not to say they don't appreciate the glasses. Many people love them and report using them daily, but they're using the glasses to read the bible or look at their phone! Yes, this evidence is anecdotal, but it's hard to reconcile the study findings with my observations.
Furthermore, the idea that someone in rural Kenya would have to travel to Nairobi to get a pair of reading glasses is not accurate in my experience (unless they live really close to Nairobi). The hospitals I went to in rural Kenya offered reading glasses and there are also multiple optometrists in larger towns. Yes, people won't be able to buy reading glasses at the grocery store or at CVS like here in the US, but glasses aren't crazily inaccessible.
On this point, if reading glasses really did generate an extra $140 in earnings a year, why isn't everyone clamoring to get a pair? There would be huge incentives for businesses to expand and offer the product in rural areas at higher prices. And workers should expend substantial effort to obtain a pair. In practice, we don't see this happening. In my view, it is more likely that THRIVE overestimates the return to reading glasses than that hundreds of millions of workers worldwide substantially underestimate the benefit.
Additionally, the 33.4% statistic you cite from THRIVE isn't even statistically significant! The authors report the coefficient from a median regression, but the standard error for this estimate is not mentioned.
At the moment, I'm working on a formal comment aggregating these considerations + many others and I aim to wrap that up shortly.
Also a longer response: I do think the lack of demand is worrying and could be suggestive that these studies are not showing real world effects. I haven't spent enough time in rural Kenya to know how hard it is to get glasses, but I am updating based on what you say!
I do think it is easy to underestimate how bad your vision has gotten and not use glasses you need. Personally, I have failed to notice that my prescription has gotten out of date and continued to use old glasses, and then finally get around to getting new ones and I do notice a productivity improvement. Since your brain can somewhat compensate, I think it's easy to underestimate the returns to a correct Rx.
I am very excited to see your longer comment when it's done - make sure to sent it to both OP and Erin Crossett at GiveWell as well; it sounds like we will all benefit from it!
I share a lot of Drew's skepticism about the study, especially the experimenter demand effects. If monitoring alone is enough to increase productivity, I think it's quite plausible that there is some further response (beyond a direct effect of the glasses on vision) to monitoring plus the provision of glasses. Even as a large proponent of quantile regression in many applications, I do think OLS is more appropriate for a cost effectiveness analysis. A median shift could be consistent both with a much larger or much smaller (even negative) impact on aggregate utility.
However, I do think the point about glasses as an experience good is a good one and could quite possibly be at play here. If getting glasses for work is not a normal activity, it could be easy to underestimate the benefits of doing so.
Connor makes great points as always, and I appreciate the detailed response and the openness Lauren!
I very much agree with you both that glasses are an experience good and people systematically underestimate the value as their eyesight slowly gets worse. I think it'd be very interesting to study different ways of stimulating demand and generating awareness. I'm especially interested in the free screening model some systems like LV Prasad have adopted.
And to be clear, I would guess that providing reading glasses is cost-effective based on the disability aversion alone. People were very happy to glasses and did benefit from them. I am just skeptical there are many further income benefits beyond that. On that topic, I have finished a first draft of the comment, so hopefully can circulate soon!
Thanks so much for this comment Drew. I love on-the-ground experience like this and the insights you can bring, especially paired with data analysis like you've done here. We don't get this very much on the forum.
If Kenya is anything like UgAnda, reading glasses are available in most towns like you say but they are a bit absurdly expensive.
I'm interested that the study quoted there 33 precent increase in productivity rather than the delta between the intervention and control group (12 percent). That seems quite sloppy.
The demand things is fascinating, although I've got moved feelings on it there also isn't much demand for buying mosquito nets either.
Please post and comment more!
Very interested to see it when it's done! Always happy to update my thoughts based on new evidence.
Weird off-the-cuff question but maybe intentionally inducing something like experimenter demand effects would be a worthwhile intervention? After figuring out a way of not making recipients feel cheated or patronized, of course.
I believe that Seva and the Fred Hollows Foundation (Both in The Life You Can Save's top charities list) both do distribution of eyeglasses.
On this page Fred Hollows says they distributed 154,476 pairs of glasses in 2023: https://www.hollows.org/what-we-do/our-impact/
Seva distributed 59,005 pairs of glasses in 2023 according to their annual report. The first page of the report is a picture of a 10 year-old who got a new pair of glasses!: https://www.seva.org/site/DocServer/Seva_annual_report_2023.pdf
There's also the impact of providing glasses for children on education outcomes. From the 2023 GEAP report:
"Mass testing and distribution of eyeglasses to students with refraction errors improved test scores in two separate studies in four different locations in China (China-1, 2), although positive impacts were not found in all counties. The benefits were greater for underperforming students (China-1). Poor quality of schools might explain why glasses did not improve learning everywhere (China-2). At least one study was large-scale (2,500 schools). In these studies, eyeglasses have been shown to be highly cost-effective at increasing learning."
I think one thing about this cause area that is particularly interesting is that it has (to my eyes) strong marketing potential in the developed world. That is to say, because large numbers of people have experience using spectacles, the benefits are a very "easy sell" and there are also clear engagement touchpoints (Opticians, annual workplace eyetests etc) For example, you could easily imagine a partnership with an optician to "buy one pair, donate one pair" on new glasses sold. So i'm particularly interested in the are as potentially an area where an EA organisation could primarily engage or work with non-effective altruist donors. (Eg. even if at the margin, we felt that this was less effective than e.g malaria nets if the intervention is still highly effective and the counterfactual dollar would otherwise not have been donated at all it would be a very positive intervention).
Spectacles do look fairly promising, especially if conventional estimates of the benefits factor in income and not the pure health/sight aspect (depends on how much you think GBD disability weights of vision loss transfer over to myopia)
Yeah have thought for some time extremely promising. One of the biggest issue here I think is cheap distribution - NGOs know how to waste money on this front... Even a CE charity which had the aim of demonstrating how to do it on the cheap, and driving down costs could a good shout?
I think you're right re: cheap distribution. My guess is it would be hard for a new charity to beat VisionSpring, who have been executing on this strategy for a while and have the resources (~300 staff, $15M grant from MacKenzie Scott, etc), the results to show for it (e.g. 1.9 million eyeglasses distributed last year with 535 partners, etc), and "expansion plans" secured (e.g. their new $70M flagship project to screen 8M workers over the next 5-7 years based on the PROSPER RCT). Their 2022 financial summary (page 17) claims to have distributed 1.52 million eyeglasses for $11.9M in total expenses i.e. ~$7.80 per eyeglasses, nearly half opex (mgmt ops, fundraising, program & sales ops) and ~2/3rds of the remainder program delivery costs, although the expenses were inflated by the delivery of ~1.3 mil "covid safety materials" (PPEs etc) so I'd guess the true figure is closer to $5-6 per eyeglasses all-inclusive.
EinDollarBrille (OneDollarGlasses), a german charity works in this area, although they are pretty small.
IIRC, they also teach locals to build parts of these glasses themselves, making the production cheaper.