The Selected Papers Network (Part 2)

Last time Christopher Lee and I described some problems with scholarly publishing. The big problems are expensive journals and ineffective peer review. But we argued that solving these problems require new methods of

selection—assessing papers

and

endorsement—making the quality of papers known, thus giving scholars the prestige they need to get jobs and promotions.

The Selected Papers Network is an infrastructure for doing both these jobs in an open, distributed way. It’s not yet the solution to the big visible problems—just a framework upon which we can build those solutions. It’s just getting started, and it can use your help.

But before I talk about where all this is heading, and how you can help, let me say what exists now.

This is a bit dangerous, because if you’re not sure what a framework is for, and it’s not fully built yet, it can be confusing to see what’s been built so far! But if you’ve thought about the problems of scholarly publishing, you’re probably sick of hearing about dreams and hopes. You probably want to know what we’ve done so far. So let me start there.

SelectedPapers.net as it stands today

SelectedPapers.net lets you recommend papers, comment on them, discuss them, or simply add them to your reading list.

But instead of “locking up” your comments within its own website—the “walled garden” strategy followed by many other services—it explicitly shares these data in a way that people not on SelectedPapers.net can easily see. Any other service can see and use them too. It does this by using existing social networks—so that users of those social networks can see your recommendations and discuss them, even if they’ve never heard of SelectedPapers.net!

The idea is simple. You add some hashtags to let SelectedPapers.net know you’re talking to it, and to let it know which paper you’re talking about. It notices these hashtags and copies your comments over to its publicly accessible database.

So far Christopher Lee has got it working on Google+. So right now, if you’re a Google+ user, you can post comments on SelectedPapers.net using your usual Google+ identity and posting process, just by including suitable hashtags. Your post will be seen by your usual audience—but also by people visiting the SelectedPapers.net website, who don’t use Google+.

If you want to strip the idea down to one sentence, it’s this:

Given that social networks already exist, all we need for truly open scientific communication is a convention on a consistent set of tags and IDs for discussing papers.

That makes it possible to integrate discussion from all social networks—big and small—as a single unified forum. It’s a federated approach, rather than a single isolated website. And it won’t rely on any one social network: after Google+, we can get it working for Twitter and other networks and forums.

But more about the theory later. How, exactly, do you use it?

Getting Started

To see how it works, take a look here:

https://selectedpapers.net

Under ‘Recent activity’ you’ll see comments and recommendations of different papers, so far mostly on the arXiv.

Support for other social networks such as Twitter is coming soon. But here’s how you can use it now, if you’re a member of Google+:

• We suggest that you first create (in your Google+ account) a Google+ Circle specifically for discussing research with (e.g. call it “Research”). If you already have such a circle, or circles, you can just use those.

• Click Sign in with Google on https://selectedpapers.net or on a paper discussion page.

• The usual Google sign-in window will appear (unless you are already signed in). Google will ask if you want to use the Selected Papers network, and specifically for what Circle(s) to let it see the membership list(s) (i.e. the names of people you have added to that Circle). SelectedPapers.net uses this as your initial “subscriptions”, i.e. the list of people whose recommendations you want to receive. We suggest you limit this to your “Research” circle, or whatever Circle(s) of yours fit this purpose.

Note the only information you are giving SelectedPapers.net access to is this list of names; in all other respects SelectedPapers.net is limited by Google+ to the same information that anyone on the internet can see, i.e. your public posts. For example, SelectedPapers.net cannot ever see your private posts within any of your Circles.

• Now you can initiate and join discussions of papers directly on any SelectedPapers.net page.

• Alternatively, without even signing in to SelectedPapers.net, you can just write posts on Google+ containing the hashtag #spnetwork, and they will automatically be included within the SelectedPapers.net discussions (i.e. indexed and displayed so that other people can reply to them etc.). Here’s an example of a Google+ post example:

This article by Perelman outlines a proof of the Poincare conjecture!

#spnetwork #mustread #geometry #poincareConjecture arXiv:math/0211159

You need the tag #spnetwork for SelectedPapers.net to notice your post. Tags like #mustread, #recommend, and so on indicate your attitude to a paper. Tags like #geometry, #poincareConjecture and so on indicate a subject area: they let people search for papers by subject. A tag of the form arXiv:math/0211159 is necessary for arXiv papers; note that this does not include a # symbol.

For PubMed papers, include a tag of the form PMID:22291635. Other published papers usually have a DOI (digital object identifier), so for those include a tag of the form doi:10.3389/fncom.2012.00001.

Tags are the backbone of SelectedPapers.net; you can read more about them here.

• You can also post and see comments at https://selectedpapers.net. This page also lets you search for papers in the arXiv and search for published papers via their DOI or Pubmed ID. If you are signed in, the homepage will also show the latest recommendations (from people you’re subscribed to), papers on your reading list, and papers you tagged as interesting for your work.

Papers

Papers are the center of just about everything on the selected papers network. Here’s what you can currently do with a paper:

• click to see the full text of the paper via the arXiv or the publisher’s website.

• read other people’s recommendations and discussion of the paper.

• add it to your Reading List. This is simply a private list of papers—a convenient way of marking a paper for further attention later. When you are logged in, your Reading list is shown on the homepage. No one else can see your reading list.

• share the paper with others (such as your Google+ Circles or Google+ communities that you are part of).

• tag it as interesting for a specific topic. You do this either by clicking the checkbox of a topic (it shows topics that other readers have tagged the paper), by selecting from a list of topics that you have previously tagged as interesting to you, or by simply typing a tag name. These tags are public; that is, everyone can see what topics the paper has been tagged with, and who tagged them.

• post a question or comment about the paper, or reply to what other people have said about it. This traffic is public. Specifically, clicking the Discuss this Paper button gives you a Google+ window (with appropriate tags already filled in) for writing a post. Note that in order for the spnet to see your post, you must include Public in the list of recipients for your post (this is an inherent limitation of Google+, which limits apps to see only the same posts that any internet user would see – even when you are signed-in to the app as yourself on Google+).

• recommend it to others. Once again, you must include Public in the list of recipients for your post, or the spnet cannot see it.

We strongly suggest that you include a topic hashtag for your research interest area. For example, if there is a hashtag that people in your field commonly use for posting on Twitter, use it. If you have to make up a new hashtag, keep it intuitive and follow “camelCase” capitalization e.g. #openPeerReview.

Open design

Note that thanks to our open design, you do not even need to create a SelectedPapers.net login. Instead, SelectedPapers.net authenticates with Google (for example) that you are signed in to Google+; you never give SelectedPapers.net your Google password or access to any confidential information.

Moreover, even when you are signed in to SelectedPapers.net using your Google sign-in, it cannot see any of your private posts, only those you posted publicly—in other words, exactly the same as what anybody on the Internet can see.

What to do next?

We really need some people to start using SelectedPapers.net and start giving us bug reports. The place to do that is here:

https://github.com/cjlee112/spnet/issues

or if that’s too difficult for some reason, you can just leave a comment on this blog entry.

We could also use people who can write software to improve and expand the system. I can think of fifty ways the setup could be improved: but as usual with open-source software, what matters most is not what you suggest, but what you’re willing to do.

Next, let mention three things we could do in the longer term. But I want to emphasize that these are just a few of many things that can be done in the ecosystem created by a selected papers network. We don’t need to all do the same thing, since it’s an open, federated system.

Overlay journals. A journal doesn’t need to do distribution and archiving of papers anymore: the arXiv or PubMed can do that. A journal can focus on the crucial work of selection and endorsement—it can just point to a paper on the arXiv or PubMed, and say “this paper is published”. Such journals, called overlay journals, are already being contemplated—see for example Tim Gowers’ post. But they should work better in the ecosystem created by a selected papers network.

Review boards. Publication doesn’t need to be a monogamous relation between a journal and an author. We could also have prestigious ‘review boards’ like the Harvard Genomics Board or the Institute of Network Science who pick, every so often, what they consider to be best papers in their chosen area. In their CVs, scholars could then say things like “this paper was chosen as one of the Top Ten Papers in Topology in 2015 by the International Topology Review Board”. Of course, boards would become prestigious in the usual recursive way: by having prestigious members, being associated with prestigious institutions, and correctly choosing good papers to bestow prestige upon. But all this could be done quite cheaply.

Open peer review. Last time, we listed lots of problems with how journals referee papers. Open peer review is a way to solve these problems. I’ll say more about it next time. For now, go here:

• Christopher Lee, Open peer review by a selected-papers network, Frontiers of Computational Neuroscience 6 (2012).

A federated system

After reading this, you may be tempted to ask: “Doesn’t website X already do most of this? Why bother starting another?”

Here’s the answer: our approach is different because it is federated. What does that mean? Here’s the test: if somebody else were to write their own implementation of the SelectedPapers.net protocol and run it on their own website, would data entered by users of that site show up automatically on selectedpapers.net, and vice versa? The answer is yes, because the protocol transports its data on open, public networks, so the same mechanism that allows selectedpapers.net to read its users’ messages would work for anyone else. Note that no special communications between the new site and SelectedPapers.net would be required; it is just federated by design!

One more little website is not going to solve the problems with journals. The last thing anybody wants is another password to remember! There are already various sites trying to solve different pieces of the problem, but none of them are really getting traction. One reason is that the different sites can’t or won’t talk to each other—that is, federate. They are walled gardens, closed ecosystems. As a result, progress has been stalled for years.

And frankly, even if some walled garden did eventually eventually win out, that wouldn’t solve the problem of expensive journals. If one party became able to control the flow of scholarly information, they’d eventually exploit this just as the journals do now.

So, we need a federated system, to make scholarly communication openly accessible not just for scholars but for everyone—and to keep it that way.

77 Responses to The Selected Papers Network (Part 2)

  1. Todd Trimble says:

    This sounds very exciting! Sorry if I haven’t been reading attentively enough, but I haven’t seen much mention yet of authors’ agreements to have their work reviewed publicly like this. It must be there somewhere.

    Also: this might be a trivial remark, but who designed the Open Access logo? It reminds me an awful lot of communist propaganda posters, and I almost get the sense that’s intentional (!). It’s not something that bothers me personally — not at all — but I can imagine its being off-putting to a lot of people.

    (And of course discussion of federated systems is something that comes up frequently in anarchist literature, on seriously proposed ways of organizing societies along non-hierarchical lines.)

    • John Baez says:

      Sorry if I haven’t been reading attentively enough, but I haven’t seen much mention yet of authors’ agreements to have their work reviewed publicly like this.

      Right now, SelectedPapers.net lets people comment on papers that are freely available on the arXiv or PubMed Central, as well as papers that have already been published. There’s no need to obtain the author’s agreements to do this. People are already discussing these papers in a host of forums.

      If a journal decides to use “open peer review” to decide which papers to publish, they’ll need to make it clear to people submitting articles that this is how it works.

      Also: this might be a trivial remark, but who designed the Open Access logo?

      I don’t know.

      It reminds me an awful lot of communist propaganda posters, and I almost get the sense that’s intentional (!).

      Sure, the style in which the figure is drawn is based on those old posters. I like having pictures in my blog entries, so that the G+ link has something eye-catching. It was hard to find an attractive picture for open access. I thought this one was fun: a wry allusion to ‘revolution’. But I’m going to remove it from this particular article, because (for annoying technical reasons) I failed to get it to appear in my G+ post linking to this article, and now that you mention it, I don’t want people to be distracted from the actual point of this post.

    • Todd says:

      John–deep thanks for your work here. I think the site needs more “sex appeal” (such as making it resemble Quora) to attract the shallow tastes of cultural Academia. It needs legitimacy to coddle their egos. We need a stronger movement of people that believe in the free circulation of ideas–and that’s it. The most honest, free-thinking scientist knows this is important to advance the clock of human progress: some legitimate platform/resource for the open exchange of scientific knowledge/discovery has to exist eventually. But it needs at first a Facebook-like allure and importance to make it popular among the upper epsilons.

      • Todd says:

        It should use the Twitter protocol to gain mainstream appeal.. hash tags as references to papers. papers are requested through the system in a similar way. it becomes popular, and papers cannot be moderated once released (unless the author requests so). junk is filtered out in a Wikipedian manner. eventually, a mass exodus occurs, and the absurdity of current scientific publishing standards elapses.

      • John Baez says:

        I encourage people with programming skills to start making the spnetwork portal more appealing! The great thing about it is that, as a federated system, there does not need to be just one interface. So, we can have a competition to set up the most attractive portal. But what’s important is that it’s not just ‘a site’, which someone can own and exploit: it’s a protocol.

  2. MonkeyBoy says:

    I my brief perusal of spn I saw no way to find which papers have comments. Then again if such a list existed there would probably arise the problem of people spamming their own papers.

  3. Charles Rezk says:

    I’m not having much success with the site. If I use the search function to look for an “arXiv id”, I get responses like

    Error
    arXiv api core

    incorrect id format for arXiv:1306.2072

    Where is the right place to report bugs?

    • John Baez says:

      The official place to report bugs is here:

      https://github.com/cjlee112/spnet/issues

      I’ve never tried what you’re trying; I’ve been using G+ to post comments on arXiv articles—often articles I want to keep reading later—and then referring to those comments later on SelectedPapers.net. That works fine. But you should expect to find lots of bugs.

      • John Baez says:

        But this isn’t actually a bug. According to Chris, when you’re explicitly typing in a search for “arxiv ID”, you shouldn’t prepend “arXiv:” in front of the arxiv ID. I could have thought of that if I’d thought about it.

        As time goes on, I imagine we’ll document things better and also try to make the system better able to intelligently respond to a wide variety of mistakes user behaviors. There’s obviously a whole art to doing this, which some class of software experts have mastered. I’m not one, and if anyone out there is one, we could use your help.

  4. John Baez says:

    Over on the G+ post about this, Moshe Kamensky writes:

    Thanks, that’s great! Is there some way to combine the rss feeds from the arxiv with the reading list? I would like to have a button when reading the feed (or at least when visiting the arxiv page) to add the paper to the reading list.

    I replied:

    That should be possible. Only a few features have been included so far, since it’s mainly just Christopher Lee doing the programming. I’ll copy your request to a comment on the blog article. We need to grow a community of programmers (note: not me!) to keep improving this software.

  5. This looks exciting! Can I suggest adding some explanatory text to the selectedpapers.net landing page itself? I keep going back to that tab and forgetting what I should do next. :-)

  6. John Iskra says:

    I’d think another item on the to-do list is to get g+ to lobby google to enable mathjax (or some such) on + so we can express ourselves using mathematical notation.

    • John Baez says:

      A bunch of us have pushed for that, so far to no effect. We should push harder. Math underlies the internet; the internet should return the favor.

      • John Iskra says:

        I think spn gives us a little more leverage. I think that google wants + to be used for things like this.

  7. Noon says:

    My comment will be much similar to that I made on G+, but now seeing SelectedPapers.net I do find it remarkably similar to scirate – https://scirate3.herokuapp.com/ – the obvious difference being that Scirate doesn’t yet support other Journals (and the open comments idea, which personally I’m not sure is necessary…)

    Perhaps it’s good to have more than one system like this, or perhaps it would be better for the two to be merged in some way (or at least build upon each other…)

    I’ve started a discussion about this on the Scirate mailing list – https://groups.google.com/forum/?fromgroups=#!forum/scirate-dev.

  8. Bryan says:

    What a wonderful resource. How can other repositories request to participate? For example, I’m sure that philsci-archive.pitt.edu (where I’m a board member) would be very happy to be involved.

    • John Baez says:

      Great! For now the best way to request to participate is to make sure everyone in your team thinks that’s a good idea, and then post a request here:

      where Chris will see it. Please describe the type of ID your system uses that makes it easy to automatically find papers online: e.g., given an arXiv ID

      arXiv:1305.4988

      it is easy to automatically infer the URL

      http://arxiv.org/abs/1305.4988

      Given this information, Christopher Lee (or other, not-yet-existent programmers who join the project) should be able to add the necessary code to the software. Of course I can’t make any estimate on how long this will take.

  9. Great work, Christopher Lee!

    Don’t know what I’m doing wrong. I tried to add two of my papers by adding tags to my old Google+ posts. The posts do not seem to have appeared on selectedpapers yet.

    The links to the posts are:

    https://plus.google.com/u/0/118441471563356538361/posts/96Zjm5iE9Hc

    and

    https://plus.google.com/u/0/118441471563356538361/posts/K75QR1GHYui

    Manoj

    • John Baez says:

      We’ll see if that works. There are various things to consider:

      1) At best it takes about 15 minutes for SelectedPapers.net to notice your posts on G+.

      2) I don’t know if changes in old posts work the same way; they do work, since I’ve done them, but they could take longer, for all I know.

      3) You put the arXiv reference before #spnetwork and mixed with other text; I’ve never tried that.

      • Thanks, John! I put the arXiv reference inline in only one of the two papers, the other was tagged more conventionally. Neither of the papers are up yet.

        • John Baez says:

          Did you follow the steps I described? I don’t think you joined the Selected Papers Network. I see other people’s names there, but not yours. Try doing this, if you haven’t yet:

          • Click Sign in with Google on https://selectedpapers.net or on a paper discussion page.

          • The usual Google sign-in window will appear (unless you are already signed in). Google will ask if you want to use the Selected Papers network, and specifically for what Circle(s) to let it see the membership list(s) (i.e. the names of people you have added to that Circle). SelectedPapers.net uses this as your initial “subscriptions”, i.e. the list of people whose recommendations you want to receive. We suggest you limit this to your “Research” circle, or whatever Circle(s) of yours fit this purpose.

        • Yes I have joined the Selected Papers Network. I do not know why you can not see my name on it.

    • John Baez says:

      Your problem is being discussed here:

      https://github.com/cjlee112/spnet/issues/13

      The problem is that your old posts are over 10 days old. See the discussion for how this might be dealt with, and feel free to join this discussion! You need to register with github, but it’s easy.

  10. John Baez says:

    As Alexander Kruel points out, Eliezer Yudkowsky posted a short note on Less Wrong about the selected papers network here. The most interesting part so far is this comment of his, which imagines some of the many things this system permits. It may help to remember that Less Wrong has a reputation system called ‘karma’:

    I should post separately about this at some point.

    Suppose we have a Collective Judgment of Science system in which scientific karma enters the system at highly agreed-upon points, e.g. very well-replicated, significant findings. Is there a system with the following properties:

    • The karma entry points need not necessarily be the most trusted people. Let’s say you made a significant discovery, but 70% of the field disagrees with most of your opinions, and someone who hasn’t made a significant discovery is trusted by 95% of the people who make significant discoveries. We should perhaps believe the latter person over you; making one discovery is not proof of perfect epistemic reliability.

    • If someone goes rogue and endorses a thousand trolls, who in turn endorse a million trolls, the million trolls can do no more karmic damage / produce no more karmic distortion, than the original person.

    • If I make three significant discoveries or write three good papers, there is no incentive to spread those papers out over 3 pseudonyms, or coauthor them with 3 others, in terms of how much influence I will have afterward. There may potentially be some incentive to centralize, although this would also not be good.

    • Downvoting or strongly downvoting an idea that many reliable epistemic voters think is correct may potentially be taken as evidence by the system that you sometimes downvote good ideas.

    • Rather than give people an incentive to waste labor by systematically downvoting everything that person X vote, there is a centralized “I think this person is a complete idiot” button. After pressing this button, further systematic downvoting has no effect. Obviously the order of operations should not be significant here, i.e., this button must have as much effect as downvoting everything. Perhaps you might be asked to look at the person’s 3 most upvoted beliefs and asked if you really want to downvote those too (vs. an “I hate most but not all things you say” rating) given that indicating “I uniformly hate everything you say” may then potentially reflect poorly on your reliability.

    • Within these constraints, it should be generally true that one person who’s gotten a large karma prize cannot outvote 100 people who were all endorsed by trusted epistemics with karma originating from sources outweighing that single prize.

    • We’re okay with this system using terabytes or even petabytes of memory to scale, so long as it’s not exabytes and it can compute updates in real time, or at least less than an hour.

    • Being able to run on upvotes and downvotes is great, failing that having people click on a 5-star level or a linear spectrum is about as much info as we should ask, since most users will not provide more info than this on most occasions. We could potentially have a standard 5-star scale which by leaving the mouse present for 5 seconds can go to 6 stars, or a 7-star rating which can be given once per month, or something. We can’t ask users to rate along 3 separate dimensions.

    • We should take into account that some people have pickier standards and downvote more easily or upvote more rarely than others, or conversely someone who endorses almost everything is only providing discriminatory Bayesian evidence about a threshold on the low end of the quality scale.

    • We can suppose that papers are clustered in a 3-level hierarchy by broadest area, subject, and subspecialization but probably shouldn’t suppose any more clustering in the data than this. It’s possible we shouldn’t try to assess it at all.

    • A consequence of this system is that as a philosopher, you can potentially achieve great endorsement of your perspicacity, but only by convincing people who were upvoted by people who delivered well-replicated significant experimental results. This strikes me as a bug, not a feature. I don’t know of any particularly better way to decide which philosophers are reliable.

    • It can potentially be possible to bet karma on predictions subject to definite settlement a la a prediction market, since this can only operate to increase reliability of the system. If an open question that people opinionated about is definitely settled, anyone who was bold in predicting a minority correct answer should have their karma in some way benefit. Again we do not want an incentive to create pseudonyms to get independent karma awards here, though.

    For people who haven’t thought about this stuff, it’s probably worth pointing out that various different kind of systems, like karma points, can be set up on different ‘portals’ that all use the common infrastructure of the selected papers network. This allows people to try out different systems and see what they like, without imposing a uniform system on all users.

    • John Iskra says:

      Although I am frequently visited by it, there is plenty of evidence to suggest that I do not understand karma. At all. And it might be that I don’t understand the karma point system either. That said, the system above seems to amplify one of the dangers (in my opinion) of points systems. No, not that they might be ‘gamed’, but that they discourage the Hortons of the world and their ability to hear the whos and, more importantly, to go to bat for them. We need to ensure that voicing dissent does not carry risks beyond that of voicing popular opinions.
      And that’s my dissenting opinion on karma :) (the points not the universal law…)

  11. Charles Rezk says:

    It seems I have to post to +Public on Google+ in order to get spnetwork to see the post. Which is a little unfortunate; I’d rather be able to restrict the posting to circles containing only people who might possibly care.

    • Charles, I don’t think that’s a good way to use Google+ as it is currently designed. There are people who aren’t even signed up for Google+, let alone who are in your circles, who might want to read something you wrote, so unless you really want to restrict access, you are probably better off making posts public.

      Ideally, Google+ would allow people to filter the posts that come from people they circle, e.g. by hashtag. Then you’d be able to indicate your audience that way. But right now, the audience and the privacy and lumped together into which circle you share with, and I think it makes sense to use circles for privacy, and to let Google+’s automated methods make a guess as to which posts to show in someone’s stream.

    • John Baez says:

      Charles wrote:

      It seems I have to post to +Public on Google+ in order to get spnetwork to see the post. Which is a little unfortunate; I’d rather be able to restrict the posting to circles containing only people who might possibly care.

      Chris told me that it’s impossible for spnetwork or any other software outside Google+ to access posts that you have not posted to Public.

      And that’s basically a good thing: you don’t want anyone able to spy on the posts you send to your best friends and relatives.

      (Except, of course, the NSA.)

      But: people can easily set up ‘pages’ on Google+, which are like alter egos devoted to specific purposes. For example, I have an Azimuth page on Google+. And for a while now I’ve been testing out the selected papers network using posts made from another page, called John Baez (spnetwork). Since few people knew about that page, it was effectively secret. At this point I’m happy for everyone to find out about it. But I still need to decide whether I’ll post mosst of my spnetwork stuff from that page or my normal self, John Baez. The issue, with me as with you, is whether I want to pester my usual fans with lots of spnetwork post.

  12. I have edited some of my older posts with an #spnetwork hashtag and arxiv reference, namely the ones shown here, but none of them are showing up on The Selected Papers Network. These posts are at least several months old. Are only recent posts searched for hashtags? Or do I just have to wait longer? I clicked on the “Check Google+ for Updates” button, but it didn’t produce anything.

  13. If you want to know what it is and what you can do to help it become a success, then you may wish to stop reading this post and turn straight away to a post by John Baez, who has been closely involved with the venture and understands it better than I do. But let me […]

  14. As a technical matter, the whole idea of polling (G+, and every other service you might care to add in the future), to find out when spnetwork posts have been added, strikes me as BAD (broken as designed). A push-based solution (which were invented for this purpose) is the only thing that makes sense.

    As a sociological matter, I don’t see even a half-hearted analysis of how one might attempt to game the system (and provisions to thwart/minimize such attempts). Obviously, at present, no one cares enough about spnetwork to bother. But if it ever caught on …

    • John Baez says:

      We thought it was best to get more than one person working to develop the system before describing ways to try to game the system and ways to try to prevent this. At present ‘the system’ is just a bare-bones infrastructure. But of course you’re right, as soon as it catches on, people will try to spam it and game it, so analyzing this will be a key part of developing the system.

    • In the context of “push”, I should have linked to this (and, e.g. this).

      Obviously you have grander ambitions, but the part you’ve implemented so far looks like a reinvention of the square wheel.

      • John Baez says:

        If you want to discuss software issues like this push versus pull business, and you can do so civilly, I suggest raising it as an issue at https://github.com/cjlee112/spnet/issues/4.

        • People have been talking about building a replacement for the refereeing system for twenty years. The reason why it hasn’t happened yet (despite numerous attempts) is because it is hard. Not technologically hard, sociologically hard.

          If you want to discuss software issues like this push versus pull business, and you can do so civilly, …

          That is the easy (as in “already been solved”) part of the problem.

          The reason I pointed to existing implementations was not to get into an elaborate discussion of Trackbacks and XML-RPC and whatnot. Rather, I was trying to make clear that the real difficulties (the ones that stymied previous efforts along these lines) lie elsewhere.

          Sorry that, in the process, I offended you.

        • John Baez says:

          Jacques wrote:

          People have been talking about building a replacement for the refereeing system for twenty years. The reason why it hasn’t happened yet (despite numerous attempts) is because it is hard. Not technologically hard, sociologically hard.

          Yes. People who don’t think hard about the ‘prestige economy’ in academia tend to think you can just start up a new supposedly better system and have everyone jump on board. They don’t realize that academics are mostly prestige-maximizing agents with limited time horizons, so most of them will prefer to publish a paper in a prestigious traditionally run journal even if it’s very expensive and poorly refereed, rather than try a new system. Similarly most academics will prefer to work as an editor for such a journal, and so on… because it gives them more prestige.

          And prestige is not something you can summon up by snapping your fingers, or even with money. A journal is prestigious if it has has prestigious editors and publishes papers by prestigious authors. An author is prestigious if they work at a prestigious institution and can get their papers accepted by prestigious journals. And so on: it’s recursively defined, so what counts as prestigious can only change rather slowly.

          The arXiv caught on quickly because it did not require contributors to give up one iota of prestige… except for people in fields where it’s common to publish in journals like Science and Nature, which forbid their authors from freely distributing their work, at least during some time period… and in such fields, the arXiv has not caught on.

          Nonetheless the arXiv got a bunch of people to notice that we don’t need journals for distribution and archiving of information anymore: we mainly need them for selection and endorsement. (Endorsement means: handing out prestige.)

          Now we are facing the tough part of the battle.

          So, the idea of the Selected Papers Network is not to work a miracle over night, but continue ‘softening the ground’ by getting people into the habit of reading and writing evaluations of papers without using journals. It will be ‘just for fun’ at first… but luckily, some prestigious mathematicians like Terence Tao and Tim Gowers say they want to use it, and so do the people at Mathoverflow. So at least in math, it may catch on.

          It’s fine if it catches on slowly, because it will take a while for people to figure out good ways to use it. One easy way is to use it as a substitute for the traditional arXiv interface: it has the extra feature that you can mark papers for future reading, and comment on them. But you can also look at other people’s comments. You can also just add #spnetwork to your Google+ and Twitter posts—not much work to get your remarks added to the database.

          But these are the relatively trivial uses. The more interesting uses won’t catch on as fast. So, I expect a slow growth process for a few years at least, with lots of developments I can’t predict now.

        • …but luckily, some prestigious mathematicians like Terence Tao and Tim Gowers say they want to use it, and so do the people at Mathoverflow. So at least in math, it may catch on.

          Mathematics has a long tradition of informal and semi-formal and informal reviews, as exemplified by the blogs of the aforementioned Fields Medalists and by Math Reviews (and MathOverflow and the nlab and …) . So, on the one hand, mathematicians are particularly well-acclimated to adopt spnetwork to aggregate/disseminate such reviews.

          On the other hand, of all the scientific disciplines, they also have the most rigourous and valourized traditional refereeing process — which (as far as I can tell) most mathematicians would be loath to give up.

          (These two points, as I have remarked elsewhere, are actually intimately-related — two sides of the same coin.)

          So I’m not sure whether you are softening the ground or merely paving the cowpaths.

  15. porton says:

    I would like to recommend this my paper through your system:
    http://ijpam.eu/contents/2012-74-1/6/index.html
    but it seems that it has no DOI, and how shamefully it sounds I am not endorsed fo arXiv.

    • John Baez says:

      A journal whose online papers don’t have DOIs is a very bad idea. This is the main system that people use to reliably locate papers online. I suggest you talk to the editors of International Journal of Pure and Applied Mathematics and check to see if they have DOIs—and if not, urge them to get them.

      Your paper says it’s indexed by SCOPUS. I’m not familiar with that system, but if it’s important, someday the Selected Papers Network will know about it.

  16. Mike Shulman says:

    For example, if there is a hashtag that people in your field commonly use for posting on Twitter, use it.

    What if I don’t use Twitter? How do I find out whether there is such a hashtag and if so what it is?

    Also, why do the comments on individual article pages appear in reverse chronological order? It’s very hard to read a discussion backwards.

    • John Baez says:

      Mike wrote:

      Chris advised:

      For example, if there is a hashtag that people in your field commonly use for posting on Twitter, use it.

      What if I don’t use Twitter? How do I find out whether there is such a hashtag and if so what it is?

      Do you think lots of people working on homotopy theory, n-categories and the like use Twitter for communicating about these subjects?

      I don’t use Twitter either, so I can’t be sure, but I bet the answer is no. So, just make something up.

      Also, why do the comments on individual article pages appear in reverse chronological order? It’s very hard to read a discussion backwards.

      There are a lot of blogs, for example, that show comments in reverse chronological order, based on the theory that old crap ain’t as interesting as new crap.

      I chose forwards chronological order for this one, based on the theory that you can’t understand the new crap if you haven’t seen the old crap.

      Since this is the sort of binary choice that people can never agree on, like which end of the boiled egg to crack, I think the best solution is a toggle that lets you pick which kind of view you want.

      This is the kind of user interface issue that should be refined more and more as time goes on. If this issue really matters to you, please add a remark to the ‘issues page’ here:

      https://github.com/cjlee112/spnet/issues

      (I’m testing to see if you care about this enough for Chris to add it to his rapidly growing to-do list.)

  17. John Baez says:

    Please visit the Selected Papers Network Issues Site and learn about 7 open issues which may be close to resolved, but need testing and/or discussion:

    • MathJax support (using TeX)
    • A forum for discussing the spnetwork
    • Searching for old-style arXiv papers
    • Drop down list of tags not working?
    • Newly tagged old posts not appearing?
    • “Post invitation” button generates error?
    • Margins on homepage

    To comment you need to register on GitHub, which is a famous site for developing open-source code. It’s so easy even I could do it in a minute, and it’s benign.

  18. […] Gowers and John Baez discuss in their blogs the new system for distributing and reviewing research […]

  19. darij grinberg says:

    It seems like this one hasn’t been answered on math2.0, so I’ll take the liberty to post it here again. How much does SPN depend on the social networks used for posting comments (Google+ and whatever else is to come)? Do the comments themselves get saved on the SPN, or is it fetching them from the respective social network every time they are being requested?

  20. Serge says:

    I think selected papers front end should be structured the same way as arxiv – not just block of newest entries, but entries grouped by arxiv sections and subsections – Physics:subsections:papers, Mathematics:subsections:paper etc.
    Just my opinion

  21. […] Azimuth introduces the selected papers network. It is an attempt to update how we communicate science, dealing with a) selection (there is a need to filter) and b) endorsement (so we can show our scientific status). Didn’t see much psychology at the network. But, an interesting attempt. […]

  22. darij grinberg says:

    My posts so far haven’t been coming through ( https://plus.google.com/105276556179500976438/posts ). Yes, that’s a test post, but I need to know how the markup works before I submit any actual content…

  23. John Baez says:

    First question: have you registered with the spnetwork? You need to do that, here:

    https://selectedpapers.net/

    If you did that, here’s a second question: did you wait 15 minutes after posting your article, to give the spnetwork time to pick it up?

    By the way, the right place for bug reports is not this blog, but rather, the spnetwork issues page. I mentioned your issue there as part of a question about how I could find out who is registered on spnetwork.

  24. Bogdan says:

    Hm. As I understand, one of the purpose of this is to make public attention to the selected (=best) mathematical papers. Ok, I would like to know what are the recent mathematical results which attract most attention. So, I would like to see the list of papers ordered by, say, number of people recommended them, or number of comments, or any other raiting like this. I do not see this on https://selectedpapers.net/ I just see several random “most recent” comments and that is all… Am I misunderstanding something?

    • John Baez says:

      The idea is that when the Selected Papers Network really catches on, lots of people who want to recommend or discuss scientific papers will do so there. Eventually we (and that includes you!) will develop methods to list papers in various ways—like according to how many people recommended them, but also many more useful ways.

      But right now the system is just starting, so you only see a small amount of stuff. You can see what papers Terence Tao and Timothy Gowers and I recommend, and that’s already not nothing. But it’s just a start.

      So, the right approach is not to passively wait for some nice recommendations to be delivered to you, but to write your own!

      By the way, the Selected Papers Network is not just for mathematics. It just seems that way because mathematicians are cooler than everyone else, so we’re early adopters.

  25. Jason Polak says:

    I think this is great! However, I feel it is important to have a method of interacting with this network, such as posting comments and entering discussions, without the the need to have an account with a proprietary social network such as Google+ or Facebook. Now I did read in your post that other social networks could and will be used, but there should be at least one method that either (a) does not use any social network, or (b) interfaces with an open-source distributed one such as Diaspora.

    For instance, one could have some protocol that would allow individuals to register their personal domain or website with the Selected Papers Network, and then some simple code on their domain could allow the individuals to post comments on their own site using hashtags or the like, and then the Selected Papers Network would periodically read the data and display it on its own website.

    This system, being a push for open methods in publishing should in principle be completely open, and not be reliant on the whims of corporations (imagine a world where one entity owns every proprietary social network) for the astoundingly simple service of providing a discussion mechanism.

      • darij grinberg says:

        I agree, too. This is not just a matter of trusting social networks (which I don’t neecessarily do even when I use them for chatter and minutiae), but also a matter of unnecessary complexity. And as my experience with spn illustrates, unnecessary complexity leads to bugs.

    • cjlee112 says:

      That’s exactly what we’re going to do next — there is every reason to expand this beyond just Google+. For example, as Darij pointed out, Google+’s indexing bugs cause a lot of problems and clunky interface. But we’ve only put about two weeks of development work into this, so I would say it was not bad as a starting point. Expanding this to Twitter and people’s personal blogs seems like a good next step. We can also greatly improve how SPN works with Google+, now that we understand its limitations better. What do people think would be most useful?

  26. imriss says:

    Thank you for this initiative. I have a question. Is there any way to see my own activities on Selectedpapers.net? I logged in using my google account, but I could not find a way to see such an option. Thanks.

  27. I was wondering: how hard is it to convince arXiv.org to add a link to selectedpapers.net?

    What I have in mind is a use case like this: whenever a researcher goes to arXiv.org to read a paper, that page already shows several link-outs. For example, there is NASA ADS, then there are various bookmarks like Mendeley, Sciencewise, etc. So now if I click on Sciencewise, it takes me to the Sciencewise page for the paper. Can we have a similar bookmark to selectedpapers.net that takes us directly to the page where this paper is being discussed? Then one can directly post to selectedpapers by going to the arXiv page of the paper one is interested in, and clicking on the accompanying selectedpapers link.

    If this can be worked out, its a great way of getting the word about selectedpapers out to a large number of researchers. When they go to arXiv to read the paper, they can also access/ add to reviews of that paper.

  28. […] 2013-12-15 Thomas Lumley pointed me to a couple of papers on the Selected Network, which would be one way of dealing with prestige/quality/recognition […]

  29. […] Actually, this is perhaps sort of the idea behind the selected papers network (see homepage and Baez’s and Gowers’ blog […]

You can use Markdown or HTML in your comments. You can also use LaTeX, like this: $latex E = m c^2 $. The word 'latex' comes right after the first dollar sign, with a space after it.

This site uses Akismet to reduce spam. Learn how your comment data is processed.