About those missing blog posts... part 2: link rot

I covered some of my challenges with blogging in the first of this "about those missing blog posts..." series. Today is about link rot, which is what happens when you link to a live url (uniform resource locator) in a blog post, but at some point, the link to this content breaks. Why does this happen? And what are some options for dealing with link rot?

Screen cap of an April 2014 post with link rot

Screen cap of an April 2014 post with link rot

Apparently, the average webpage link has a half-life of two years. The Internet Archive has created the Wayback Machine, which takes periodic snapshots of webpages to create an archive.

Since references and bibliographies are essential to scholarly research, I carry this practice over into my blog posts, inserting lots of links. When I teach students to write science blog posts, the grading scheme assesses how well their post is supported with url links. However, when I review my earlier posts, I often encounter links with lines through them (top right): this is the dreaded link rot!

Link rot happens for many reasons:

  1. Webpage content gets updated, and the information on the version of the page to which you linked, disappears. This happened to me with a feature article that I wrote for University Affairs in 2005. With permission, I uploaded my 2005 feature article, Coaching for my Life, that I wrote for University Affairs into Yorkspace, York University's Institutional Repository. This means that it is still available online.
  2. Webpage content gets reorganized and moved to another webpage with a different url. With a bit of searching, the digital content can be re-discovered.
  3. Entire websites die. Perhaps the owners of a website only rented the domain for a short time and once the period expires, and so do your links to its content. This happened with our International Polar Year project. We archived the reports that we had made available in Yorkspace.
  4. Canadian gardening magazine dead webpage for the Advent Botany 2014 parsnip pear recipe

    Canadian gardening magazine dead webpage for the Advent Botany 2014 parsnip pear recipe

    Websites and pages are deliberately shut down for many reasons, including political ones. One of the most egregious examples of the latter, was Donald Trump's removal of climate change data and information both from the Whitehouse website and later on, many US government websites.

In 2016, scientists were rightfully concerned that large chunks of publicly available, taxpayer-funded climate change data that are used by researchers around the world, would be removed from open access websites. In response, Michelle Murphy, Patrick Keilty, Matt Price and Sam-chin Li of the University of Toronto, including the Technoscience Salon and Research Unit, spearheaded the backing up of this data before it disappeared from US government servers and webpages. This is known as Guerilla Archiving.

Academics and researchers need to be aware of the rates at which knowledge and information may be dying before our eyes, and to develop strategies for addressing this. In my view, pretty much everyone, including academic institutions are doing a poor job of maintaining their digitally-created records, knowledge ecosystems, and institutional memories. In doing so, they are often needlessly contributing to link rot. The one thing that academic institutions ARE doing right, is setting up open access, self-archiving Institutional Repositories.

How should we deal with link rot and prevent it happening as much as possible? Wikipedia has a handy page of recommendations. If I'm feeling energetic, I will update broken links in my posts. Also, when I write my posts, I aim to make offline digital copies, as pdfs or screen caps, of the key digital resources that I link to in my posts. And I date these. ICYMI, this is also the reason for inserting the"date accessed" to URL and websites listed in your essay reference lists. It is part of the metadata that records your research time and effort.

Learning how to document your research process is also a research skill. Since I have already mentioned the notorious anti-science and research activities of the present USA government, let me add that the controversial Steele Dossier also most likely contains records of this nature, documenting the actual research process!