Wikidata:Requests for permissions/Bot/BthBasketbot

BthBasketbot

BthBasketbot (talk • contribs • new items • new lexemes • SUL • Block log • User rights log • User rights • xtools)
Operator: Bthfan (talk • contribs • logs)

Task/s: Import basketball players team history from Template:Infobox basketball biography from English Wikipedia

Code: User:Bthfan/bot_code.py

Function details: This bot is using pywikibot and is based on the harvest_template.py script (see https://git.wikimedia.org/blob/pywikibot%2Fcore.git/HEAD/scripts%2Fharvest_template.py). I modified the original code a lot to import the team history of a basketball player from Template:Infobox basketball biography on the English wikipedia. That template has years1, team1, years2, team2, ... to specify the single teams a player has played for in which years. This bot will combine a years* property with one team* property to create the following Wikidata claim:
member of sports team (P54) : team name
with qualifiers: start time (P580) : (start year gets extracted from years*, bot looks for four digit number); end time (P582) : (end year gets extracted from years*, bot looks for four digit number; if there is only one number it assumes start year=end year)

The bot re-uses existing member of sports team (P54) entries if there are no qualifiers attached to that claim yet. This reuse is needed as some basketball players already have a few member of sports team (P54) claims, as other bots imported for example categories like https://en.wikipedia.org/wiki/Category:Olimpia_Milano_players (every player in such a category got a member of sports team (P54) entry). But those entries have no start and no end date and such lack some information that's included in the infobox.

The bot code is not completely finished yet, it needs the patch from https://gerrit.wikimedia.org/r/#/c/125575/ to use the editEntity and JSON feature as far as I see this. The problem is that some players have played for two different teams in one year. Then the template entry in Wikipedia looks like this:
years1=2012
team1=Team A
years2=2012
team2=Team B

So I need to possibly reorder existing member of sports team (P54) claims so that the order of items is correct and corresponds to the order in the template/infobox. This is currently not possible yet with the existing pywikibot code (I would need to access the API function wbsetclaim as this one allows one to set the index of a claim). --Bthfan (talk) 08:17, 10 June 2014 (UTC)

BTW: Basically I'm waiting for the patch at https://gerrit.wikimedia.org/r/#/c/125575/ to be finished, then I could edit the whole entity and with that way reorder existing claims. --Bthfan (talk) 22:08, 28 June 2014 (UTC)

Still blocked by https://gerrit.wikimedia.org/r/#/c/125575/, that patch is buggy in some way. I guess it will take a while until the bot code is finished so that it can do some test run :/ --Bthfan (talk) 07:01, 9 July 2014 (UTC)

@Bthfan: gerrit:125575 can manage qualifiers and sorting as well. Even if it's not near to being merged, you can test it locally (SamoaBot is running on Tool Labs with that change, just now). However, your code does not appear to take full advantage of the new system. See an example of the correct usage:

claim = pywikibot.Claim(site, pid)
claim.setTarget(value)
qual = pywikibot.Claim(site, pid, isQualifier=True)
qual.setTarget(value)
if qual.getID() not in claim.qualifiers:
    claim.qualifiers[qual.getID()] = []
claim.qualifiers[qual.getID()].append(qual)

--Ricordisamoa 19:37, 23 July 2014 (UTC)

Ok, thanks for the example. I did test that patch, the patch broke the addQualifier function in pywikibot (I left a comment on gerrit). So your example code is the new way to add a qualifier, one should no longer use addQualifier? --Bthfan (talk) 20:17, 23 July 2014 (UTC)

That is the only fully working way. I plan to support addQualifier (in the same or in another patch), but editEntity() would have to be called to apply the changes. --Ricordisamoa 18:09, 24 July 2014 (UTC)

Ah, I see how it should work then :). Ok, I'll try that. --Bthfan (talk) 11:56, 25 July 2014 (UTC)

Just an update on this: This bot is currently blocked by a bug in the Wikidata API (at least it looks like a bug to me), see https://bugzilla.wikimedia.org/show_bug.cgi?id=68729 wbeditentity ignores changed claim order when using that API function. --Bthfan (talk) 08:19, 6 August 2014 (UTC)

@Bene*, Vogone, Legoktm, Ymblanter: Any 'crat to comment?--GZWDer (talk) 10:52, 11 August 2014 (UTC)

My understanding is that we are not yet ready for approval and waiting for a bug to be resolved.--Ymblanter (talk) 13:56, 11 August 2014 (UTC)

That's correct. I could try to use another API function for reordering the claims (there is one: wbsetclaim with the index parameter), but this means to modify pywikibot quite a bit as this API function is not used/implemented yet in pywikibot. I currently don't have enough time for that :) --Bthfan (talk) 14:25, 11 August 2014 (UTC)