User:Dreamyshade/Archive links
These tables are inspired by and in support of Wikipedia:Requests for comment/Archive.is RFC 5#Analysis of links.
Scroll down or click here for Factfinder citation experiment.
Experiment 1: General citations
[edit]Summary:
- 13 - citation included a live link, so I removed the archive link
- 12 - citation original URL was dead, or citation only included the archive URL (no original URL within the citation)
- 8 - replaced with a live link to the original article at a different URL, found by googling the article title
- 2 - replaced with better citations because the citations were weak
- 2 - removed claim and citation because both were weak
Of the 12 dead URLs:
- 5 had archives available at archive.org
- 6 did not
- 1 wasn't properly archived at either archive.today or archive.org
General external links
[edit]| Article | Archive URL | Original dead? | On Archive.org? | Replacement strategy | Diff |
|---|---|---|---|---|---|
| Privacy Act (Canada) | https://archive.today/20070209015409/http://reports.fja.gc.ca/en/subject/privacy.html | Yes | Yes | Replaced with archive.org link | https://en.wikipedia.org/w/index.php?title=Privacy_Act_(Canada)&diff=prev&oldid=1337718652 |
| California Consumer Privacy Act | https://archive.today/20131121214538/http://oag.ca.gov/privacy/privacy-laws | Yes | Yes | Found a more useful live page | https://en.wikipedia.org/w/index.php?title=California_Consumer_Privacy_Act&diff=prev&oldid=1337715284 |
Experiment 2: Factfinder citations
[edit]Summary: I checked three random articles that cite archive.today versions of factfinder.census.gov, and I believe we could automatically or semi-automatically replace most of those archive.today URLs with live and much more useful links to the same information on data.census.gov.
Related: Wikipedia:U.S. Census Migration (2020), the effort that created most of these archive.today links, and Wikipedia:WikiProject U.S. Census.
Sample
[edit]Sample method: search for insource:factfinder.census.gov insource:archive.today with &sort=random and select the first article that uses archive.today links (or mirrors).
- Johnsburg, Illinois
- Claim: "According to the 2010 census, Johnsburg has a total area of 7.674 square miles (19.88 km2), of which 7.08 square miles (18.34 km2) (or 92.26%) is land and 0.594 square miles (1.54 km2) (or 7.74%) is water."
- Archived URL: https://archive.today/20200213045403/http://factfinder.census.gov/bkmk/table/1.0/en/DEC/10_SF1/G001/1600000US1738479
- Verifies the claim? Yes, the cited page says "18333613" for land and "1538636" for water in 2010. The numbers aren't labeled with units, but with the context from the article, the unit seems to be square meters.
- Useful? Mostly. It's helpful to know the size of the village, although the 2010 data is a bit old, and I don't need that many decimals of precision.
- Manually replaceable? Yes, using the Census Bureau instructions for finding land area, I found the latest table from 2024: https://data.census.gov/table?g=160XX00US1738479&d=GEO+Geography+Information. It says the village has 7.254 square miles of land and 0.596 square miles of water. It'd be great to replace the 2010 data with 2024 data.
- Automatically replaceable? Yes, we should be able to script this. Both URLs include the FIPS identifier, 17-38479. The Census Bureau provides an API for geography information.
- Claim: "According to the 2010 census, Johnsburg has a total area of 7.674 square miles (19.88 km2), of which 7.08 square miles (18.34 km2) (or 92.26%) is land and 0.594 square miles (1.54 km2) (or 7.74%) is water."
- Ocean Isle Beach, North Carolina
- Claim: "Historical population" table (1960 to 2020)
- Archived URL: https://archive.today/20200212193537/http://factfinder.census.gov/bkmk/table/1.0/en/DEC/10_DP/G001/1600000US3748700
- Verifies the claim? Partly. The cited page only includes the 2010 population, which is 550. The article's number for 1960-1990 are unverified.
- Useful? Yes, historical population over time is helpful context.
- Manually replaceable? Yes, same data is available here: https://data.census.gov/table/DECENNIALSF12010.P1?g=160XX00US3748700
- Automatically replaceable? Yes, we should be able to script this. We could use the Census API, but it also looks like the URLs are patterned in predictable ways that could be automatically updated with a regex script.
- Waterflow, New Mexico
- Claim: "In 2010, Waterflow reported a total population of 1,670 people. 823 were male, and 807 were female. 809 identified themselves as White, 12 identified themselves as Black, 3 identified themselves as Asian, 171 were Hispanic, and 743 were Navajo. 620 speak a language other than English at home."
- Archived URL: https://archive.today/20200210215324/http://factfinder.census.gov/servlet/SAFFFacts?_event=Search&geo_id=04000US35&_geoContext=01000US%7C04000US35&_street=&_county=&_cityTown=&_state=&_zip=87421&_lang=en&_sse=on&ActiveGeoDiv=geoSelect&_useEV=&pctxt=fph&pgsl=040&_submenuId=factsheet_1&ds_name=ACS_2008_3YR_SAFF&_ci_nbr=null&qr_name=null®=null:null&_keyword=&_industry=&show_2003_tab=&redirect=Y
- Verifies the claim? No. The archive URL shows data about New Mexico, not data specifically for this unincorporated community.
- Manually replaceable? Yes, 2010 demographic data is available here: https://data.census.gov/table/DECENNIALPL2010.P1?q=Waterflow+CDP,+New+Mexico. 2020 demographic data is available here: https://data.census.gov/table/DECENNIALDP2020.DP1?q=Waterflow+CDP,+New+Mexico&d=DEC+Demographic+Profile
- Automatically replaceable? Maybe. Unincorporated communities don't seem to have FIPS identifiers, so this might be a little trickier.
Notes toward potential automation
[edit]Resources:
- Wikipedia:Link rot/URL change requests
- Wikipedia:Bot requests
- Wikipedia:AutoWikiBrowser/Tasks
- User:Joeytje50/JWB
- Census deep linking guide
Search result count in article space only:
A challenge is that many of the original Fact Finder URLs use the Geographic Header (G001) table, which is not available on data.census.gov (see bottom of this page).
Land area
[edit]Number of articles (insource:factfinder.census.gov insource:"archive.today" insource:"10_SF1/G001"): 3,547
- Sample archive URL: https://archive.today/20200213045403/http://factfinder.census.gov/bkmk/table/1.0/en/DEC/10_SF1/G001/1600000US1738479
- Sample new URL: https://data.census.gov/table/GEOINFO2024.GEOINFO?g=160XX00US1738479
- However, the new URL is 2024 data rather than 2010 data, which may be slightly different, so you can't just do a 1:1 swap in citations without checking to see if you need to update the article text as well.
URL pattern to find:
https://archive\.today/\d+/http://factfinder\.census\.gov/bkmk/table/1\.0/en/DEC/10_SF1/G001/1600000(US\d{7})
URL pattern to replace:
https://data.census.gov/table/GEOINFO2024.GEOINFO?g=160XX00\1
2010 population
[edit]Number of articles (insource:factfinder.census.gov insource:"archive.today" insource:"10_DP/G001"): 3,407
- Sample archive URL: https://archive.today/20200212193537/http://factfinder.census.gov/bkmk/table/1.0/en/DEC/10_DP/G001/1600000US3748700
- Sample new URL: https://data.census.gov/table/DECENNIALSF12010.P1?g=160XX00US3748700
- However, the new URL is limited to 2010 population data, and the old URL had other data, so you can't just do a 1:1 swap in citations. This is only a good new URL if the claim supported is just population data.
URL pattern to find:
https://archive\.today/\d+/http://factfinder\.census\.gov/bkmk/table/1\.0/en/DEC/10_DP/G001/1600000(US\d{7})
URL pattern to replace:
https://data.census.gov/table/DECENNIALSF12010.P1?g=160XX00\1