Wikidata:Requests for permissions/Bot/CensusBot

The following discussion is closed. Please do not modify it. Subsequent comments should be made in a new section. A summary of the conclusions reached follows.

 Approved--Ymblanter (talk) 15:51, 2 January 2017 (UTC)

---

CensusBot

CensusBot (talk • contribs • new items • new lexemes • SUL • Block log • User rights log • User rights • xtools)
Operator: SasanBotMan (talk • contribs • logs)

Task/s: This bot will perform a few different tasks, including updating/creating new Wikidata pages that include Census information, updating Wikipedia Infoboxes with links to this information, and updating textual references to this information as well.

Code: https://github.com/CommerceDataService/census-wikidata-bot

Function details:
The initial bot performs the following actions:

• Ingests 2015 state population values from a static file (will be replaced with a call to the API this file was created from which has same response format)
• For each value/state, the bot searches for a corresponding State entry
• If one page is found, the bot accesses this page and checks to see if there is a population statement present
• Next, the bot checks each population claim. If there are any values with no point in time qualifier, those claims will be removed. If this an existing entry with a 2015 point in time, the value/qualifiers/references will be checked to see if they are set to the proper values.
• If any part of the claim does not match the proper format of an entry, that entry will be replaced with a proper entry including the following:

State Population Value

Qualifiers:

point in time: 2015

determination method: https://www.wikidata.org/wiki/Q637413

References:

stated in: https://www.wikidata.org/wiki/Q463769

reference URL: http://www.census.gov/data/developers/data-sets/popest-popproj/popest.html

Next Phase:
The functionality of the bot will be scaled horizontally to add more metrics concerning the State entity, followed by scaling vertically to add metrics concerning additional entities (e.g., regions, divisions, states, counties) --SasanBotMan (talk) 17:32, 17 November 2016 (UTC)

Please run a small number of test edits so we can see this in action, thanks. ArthurPSmith (talk) 15:01, 23 November 2016 (UTC)

Thanks ArthurPSmith. My complete code for the bot is now up on the GitHub repo. I have tested the bot on some pages on the test Wikidata site. Should I perform these edits on the production Wikidata site?  – The preceding unsigned comment was added by SasanBotMan (talk • contribs).

Yes. Run a small number of test edits (<=50) an check them your self. --Succu (talk) 22:50, 12 December 2016 (UTC)

I tried running my bot on production, but after performing only 3 edits I got the following message:

"pywikibot.data.api.APIError: failed-save: As an anti-abuse measure, you are limited from performing this action too many times in a short space of time, and you have exceeded this limit. Please try again in a few minutes."

Any ideas on why I would be receiving this message? I didn't get this message when performing edits on the test site.

No idea, but I count eleven edits at Arizona (Q816). The result of your operation looks bad to me. --Succu (talk) 22:39, 13 December 2016 (UTC)

Thanks for pointing out my errors Succu. I have now corrected the errors in my bot logic and I just ran a number of test edits again. Everything appears to have run correctly. --SasanBotMan

Just following up to see if anyone has had a chance to look over my most recent edits. I'm hoping to get my bot flag soon so I can run this bot for all states. Thanks! --SasanBotMan

@Succu:, @ArthurPSmith:, do you have any objections at this point?--Ymblanter (talk) 19:30, 20 December 2016 (UTC)

@Ymblanter, SasanBotMan: this looks good with one caveat - the numbers are all showing up as "+-0", I think because of a recent change in the handling of quantities by wikidata. You probably want it to be entered without the "+-" right? Or is it intended for these to be exact numbers? ArthurPSmith (talk) 20:22, 20 December 2016 (UTC)

@ArthurPSmith: Entering it without the "+-" was causing issues. I don't have any measure of uncertainty for this particular metric in the source. Also, I don't actually specify the upper or lower bound in my bot when I am setting the values, but it adds the "+-0" by default if you don't specify. I thought this was okay for a couple of reasons: First, I noticed that if I manually entered a value in the test site without the upper and lower bound, my bot would fail when trying to get the contents of the page. Secondly, I noticed all other entries in the prod site are entered in this manner. Let me know your thoughts. Thanks.--SasanBotMan

 Comment I would like to suggest changing the determination method (P459) to either demographic balance (Q15911027) or estimation (Q791801) for estimates and census (Q39825) for censuses. A complete list of determination method values can be found at Property talk:P1082#Determination method values. --Bamyers99 (talk) 23:58, 22 December 2016 (UTC)

We need to move on, and I will approve the bot in a couple of days provided there have been no objections.--Ymblanter (talk) 08:24, 31 December 2016 (UTC)

---

The above discussion is preserved as an archive. Please do not modify it. Subsequent comments should be made in a new section.