A few years ago, Alyssa Vance posted a bunch of reasons that academic papers are an imperfect forum for discussion of topics such as those of interest to LessWrong. Lots of the reasons might apply to effective altruism too:
- The time lag is huge; it's measured in months, or even years.
- Most academic publications are inaccessible outside universities.
- Virtually no one reads most academic publications.
- It's very unusual to make successful philosophical arguments in paper form.
- Papers don't have prestige outside a narrow subset of society.
- Getting people to read papers is difficult.
- Academia selects for conformity.
- The current community isn't academic in origin.
- Our ideas aren't academic in origin.
- Papers have a tradition of violating the bottom line rule.
- Academic moderation is both very strict and badly run.
Of these, four is debateable and eight is less true for effective altruism than for LessWrong. But the rest seem as true and pertinent for today's effective altruists as they were for the author.
Diego Caleiro recently suggested that books might be superior to papers for promoting his ideas.
[Books] are long enough to convey interesting ideas (of a more philosophical type) second, though they also follow exponential distributions, there are several marketing strategies that can aid publication and increase number of readers. They can be, like papers and unlike blogs, cited as decent academic evidence in good standing. They can be monetized as well, whereas one must pay to publish papers.
He continues to put his point in the strongest possible terms:
I am glad to hear counterarguments, because as it stands, this seems like the ultimate no-brainer, there is not a single thing papers are better than books for.
- Number of readers
- Conveying complex ideas
- Getting feedback on ideas
- Money
- Career prospects
- Author's name being remembered and sought after
- Resilience over long stretches of time
- Compatibility with Technological advances (current and expected)
- Odds of finding collaborators in virtue of having written them.
... The few properties like "peer feedback" and "short and easier to write" that papers beat books at have are completely dominated by blogging, 'specially on public science or philosophy blogs.
So are books simply superior to academic papers for conveying world-changing ideas? Now that people have been trying to advance the case for the importance of safe development of artificial intelligence (arguably one of the most important ideas, and one that has been quite controversial), we can ask which texts have most helped to advance that idea so far.
Instinctually, Bostrom's book Superintelligence seems to have been most helpeful for this. Although it has only been moderately cited, it's influence was felt in the discussions that it provoked from Musk, Hawking, Wozniak and an array of other scientists and technologists. It was also a bestseller, and seems to have been a major contributor to the change in discourse on this topic in the last couple of years. The book stood out as a landmark event. The Puerto Rico was similarly groundbreaking - both helped grow the reach of Bostrom's ideas in a way that dozens of papers would not (and had not).
Rather, Bostrom's papers have tended to be nonconformist, and their degree of readership and reputation has suffered from this to a greater degree than his new book. In two other points of concern to Alyssa Vance, his book was also more widely available, and was accessed in much wider parts of society. It was also advantaged on some counts raised by Diego: it allowed Bostrom to convey unusually complex ideas, and made his name more well-known.
The example of AI safety seems to speak to the value of books and in-person conversation.
If we look at the founding of CEA, we see a similar story, of initial connections being made between Toby Ord and William MacAskill, as well as with some in-person student groups at Oxford University. Even though the context was academic, journals were not the main mode of passing information or inspiration. However, it journalistic publication does seem to have helped people like Ord and MacAskill to earn a living and maintain their academic career prospects. Moreover, in the cases of Bostrom and CEA, it may have been valuable to gain a significant academic reputation before performing outreach activities. Such would certainly be the case for Peter Singer, who also admittedly gained most of his eventual reputation through books.
So academic papers aren't generating big breakthroughs in interest, but a lot of the leading thinkers tend to be academics.
So how helpful does being an academic seem to be? Well, it's not strictly necessary. If we look at the inception of MIRI (when it was the Singularity Institute), we see Eliezer, who was writing on online messageboards, and was allergic to academia. Despite often writing in a way that was technical and informed by science, he was not in any academic role, and had no academic prestige. Eliezer is as always a bit of an anomaly, though, as his writing talents have repeatedly proven to be extremely popular, and this may have allowed his writing to cut through in the absence of academic status. If you look somewhere like Edge, TED or Big Think a lot leading public intellectuals and writers are academics. However, it's hard to discern the causation. It could be that academia is equipping them to better promote their useful thoughts, or it could just be that academia sucks in more than its share of the best thinkers in the first place.
So what reasons do we see for writing academic papers? For a start, they can help one to identify an academic expert, which is possibly useful. MIRI, CSER, FHI and others derive a lot of their reputation (which in turn gives them some political and persuasive strength) in this way, as have Ord and Bostrom, and this might be good for anyone who wants to be a public intellectual. Another reason to write academic papers would be to be able to point to the existence of an academic field. This seems to be what MIRI wants to be able to do - to point technical and conscientious AI researchers to such a body of work. These kinds of investments will not pay off for some time, and may prove hard to evaluate. For the rest of us, why write papers? Well, it could be strategically useful for those who wish to secure an otherwise undemanding professorial position. They might help us secure a research-based job. But otherwise, and especially from an altruistic point of view, they don't seem to do much. So Alyssa and Diego's reservations about the value of papers seem pretty compelling to me.
I find a lot of value in academic papers, especially in STEM fields, and am going to spend some time outlining my defense for writing them. That being said, I'm not necessarily looking at them as a "discussion forum for EAs". I think there are many reasons why academic papers can be useful to Effective Altruism even if they're not directly geared towards promoting EA-type ideas (though they certainly can be). Specifically, I think they're hugely important in research, and not just for helping secure a research-type job.
Academic papers are, in general, more mathematically rigorous than books. This is not to say that books can never be as rigorous as academic papers, but the overall trend is that books tend to summarize information or lay out information in layman's terms, whereas academic papers lay out detailed information in very data-heavy terms. Thus, academic papers may or may not be good for discourse depending on the audience's interests. Take Kahneman's books on cognitive psychology/behavioural economics--although his books are probably more popular than his papers (by people-at-large), his books reference his papers. It would be folly to try to publish his experimental results as a book (where would you store the results and supplemental materials)?
Academic papers also take less time/energy to write than books, especially for incremental research. For instance, if you're trying to demonstrate the efficacy of a particular anti-malarial drug, it's much less time consuming to publish this in a paper than it is to write a book about it.
Most importantly, academic papers are peer-reviewed. You can trust that papers that have come out in reputable journals have the support of experts in the same field. You really can't do the same for books. Take the large amount of books that exist in support of intelligent design. None of these would hold a dime in any reputable genetics or biology journal. In many cases, the reputability of a book is often dependent on the number of citations it has to journal papers or other material. (Again, this is not to say that books can never be reputable--you just have less confidence about whether one is or not.)
I'm more curious about specific cases of where these book-vs-journal-paper questions arise.
Some additional related points:
1) Joao Fabiano looked recently into acceptance likelihood for papers in the top 5 philosophical journals. It seems that 3-5% is a reasonable range. It is very hard to publish philosophy papers. It seems to be slightly harder to publish in the top philosophy journals than in Nature, Behavioral and Brain Sciences, or Science magazine, and this is after the filter of 6 positions available for 300 candidates that selects PHD candidates in philosophy (harder than Harvard medicine or economics).
2) Bostrom arguably became very influential before Superintelligence within academia and the Lesswrong world. I have frequently found academics in the fields of Philosophy of mind, ethics, bioethics, physics, and even philosophy of language who knew his name and had some idea of who he was. Bostrom is a very prolific academic in terms of paper publication, and notably he pays attention to having an open website with his ideas available to the public. I believe his success prior to Superintelligence is explained, besides sheer brilliance, by actually displaying his ideas online as well as creating the World Transhumanist Organization and using techniques like talking about normal seeming topics (the matrix, cars on the other lane) in papers which are also strategies that helped make David Chalmers, another brilliant academic, become more prominent.
3) Joao Fabiano's inquire also found that David Lewis published, singlehandedly, 6.3% of top five journal publications. All women together published 3.6% (including Ruth Millikan, very prolific author). To me this indicates that the power law for publications is incredibly strong, with only extremely brilliant people like Lewis making the most substantial dents. The field is also male-skewed in an awkward way.
4) For people who consider themselves intellectual potentials and intend to continue in academia, my suggestion is to create a table of contents for a book, and instead of going ahead and writing the chapters, find the closest equivalent of some chapter that could become a paper, and try to write a paper about that. If you get accepted, this develops your career, and allows you to be one of the stand-outs like Ord, MacAskill and Bostrom who will end up working in the top universities. If you continue to be systematically rejected, you can still get around by publishing books and being influential in the way say Sam Harris or Richard Dawkins became influential. Since I find this to be the optimal strategy I'm aware of at the moment, it is the one I'm taking.
5) One behavior I found dangerous in the past is to "save your secret amazing idea from the world" protecting it by not testing it against other minds or putting it out for publication. Idea theft may be common in academia, but the response to that should be to simply have another better idea and carry on the work. Most ideas are too complex for non-authors to be able to steal. I think great value can come from publishing your ideas on Lesswrong (Stuart Armstrong does this) or Effective-altruism.com before transforming them into a paper, and eventually a book chapter. This is how I've been reasoning lately. (To put my money where my mouth is: if anyone wants to take a look at the table of contents of my academic Altruism: past, present, propagation book by the way I'm open to that, message me privately) I welcome any criticism of that blog--> Paper --> Chapter strategy.
6) Most of what I said applies to philosophy and the humanities, and I'd be interested to know if people who published in STEM fields like Paul Christiano, Nate Soares, Benja Fallenstein, Scott Aaronson and Tegmark think this is a valuable alternative there as well.
"1) Joao Fabiano looked recently into acceptance likelihood for papers in the top 5 philosophical journals. It seems that 3-5% is a reasonable range. It is very hard to publish philosophy papers. It seems to be slightly harder to publish in the top philosophy journals than in Nature, Behavioral and Brain Sciences, or Science magazine, and this is after the filter of 6 positions available for 300 candidates that selects PHD candidates in philosophy (harder than Harvard medicine or economics)."
This is a very real problem. Many people outside of philosophy do not realize how difficult it is to become an actual tenured philosopher, and even if that happens, to become a regularly-published philosopher. Philosophy itself is already highly self-selecting (highest average GRE of any grad school-bound college students), and the acceptance rates are very, very low. Further, only about half of those students who are accepted to the top PhD programs complete them. Of those who do complete them, I'd say about half (of the top programs) end up with tenure-track positions. Those who do get such positions may or may not have difficulty gaining publication, but it isn't a certainty that their ideas will be widespread in any real way. So, for Joe Smart sitting at home on his couch, wanting his great ideas to be read by a lot of smart people, becoming a philosopher is probably a terrible way of accomplishing his goal.
"4) For people who consider themselves intellectual potentials and intend to continue in academia, my suggestion is to create a table of contents for a book, and instead of going ahead and writing the chapters, find the closest equivalent of some chapter that could become a paper, and try to write a paper about that. If you get accepted, this develops your career, and allows you to be one of the stand-outs like Ord, MacAskill and Bostrom who will end up working in the top universities. If you continue to be systematically rejected, you can still get around by publishing books and being influential in the way say Sam Harris or Richard Dawkins became influential. Since I find this to be the optimal strategy I'm aware of at the moment, it is the one I'm taking."
This seems like a good suggestion. I do still think that good ideas can be published in journals, and books should contain journal-worthy ideas, but the journal process can be quite slow. This may be a way to kill two birds with one stone.
As a reference, I wrote a paper in 2012 that is going to be published in the next couple of months. I do not hold a PhD, but my co-author does. Our paper was presented at a very prestigious conference (multiple household names were also presenting, including a couple of superstars). We were rejected by five journals before we were accepted, and some of those that rejected us were less prestigious than the one that eventually accepted us. It was quite a long process, and I am certain that it is not the ideal way to advance one's EA-related message.
I suspect a major divide in the usefulness of academic publication is whether we're trying to establish specific empirical claims, or develop a philosophical framework. For the former, if you want to make STEM claims, it's difficult to get people to take you seriously without having published results. This is what MIRI is doing. Many other EA problems, such as disease mitigation and economic development already have a developed literature, meaning much of the problem right now is applying that literature to donating strategy. As EA becomes more prevalent and we begin targeting problems other than the low hanging fruit, we will push the boundary of what that literature has to say. Givewell is running into this problem already. While the discussions should advance beyond what is published, it makes sense to have a paper trail of evidence that various methods are as effective as they're claimed to be. While the discussion shouldn't be limited to academia, there should be an academic branch to EA, particularly in STEM.
I come from the STEM perspective, but for me, if I see a publication that isn't peer-reviewed, I immediately dismiss the claims in it. I'm disappointed that this paper has been shared so much lately, for example: http://www.alcor.org/Library/pdfs/Persistence.of.Long.Term.Memory.in.Vitrified.and.Revived.C.elegans.pd There are so many bad non-peer-reviewed scientific papers that peer review is a really good heuristic for whether a paper can be taken seriously.
My impression is that Eliezer's style both helped him and hurt him. Some people loved it, but many people (including me and Luke Muelhauser) found it hard to read: http://lesswrong.com/lw/hzt/writing_style_and_the_typical_mind_fallacy/
For years I dismissed Eliezer's ideas as glib and crankish because of his writing style. I only began to take AI risks seriously when Bostrom wrote about them.
Of the eleven reasons given at the top, I thought the most obvious issue is raised by number three: Virtually no one reads most academic publications. This seems to be true in many cases even for an academics writing to other academics. The sheer number of papers published is staggering, and unless one is already very prominent in his or her field, it seems unlikely that a large audience will be exposed to one's paper. There are exceptions of course, such as the reputation of the journal, but if we are to assume that we are not referring to academic superstars publishing in the absolute top journals in a given field, the audience of academics is also likely to be small.
If one's goal is to write books, there may potentially be a problem in becoming perceived as legitimate enough to gain the attraction of a publisher if one doesn't hold a research position and doesn't publish in journals. I mentioned this to Diego as well. It is speculation on my part, and I'm sure there are many exceptions to this rule, but it does seem like a non-academic will be regarded by publishers differently than academics. I'd be interested to hear the thoughts of someone with direct knowledge of the publishing industry on this point.