Wikidata talk:Bots

From Wikidata
Jump to: navigation, search

Wd:Bots vs Wd:Bot[edit]

I've just created small stub for bots guildline using partially w:en:Wikipedia:Bots and w:en:Wikipedia:Bot policy. I'm going to move on this discussion page all related treads soon. --Zanka (talk) 14:23, 23 November 2012 (UTC)

There are Wikidata:Bot and Wikidata:Bots. Should we merge these two? Or rename this page into "Bot policy" or "Bot rule"? -- ChongDae (talk) 01:07, 26 November 2012 (UTC)
I created a redirect from wd:Bot to this site. Two sites only leed to confusion. --Sk!d (talk) 11:01, 26 November 2012 (UTC)

Import bot feature list[edit]

Moved from Wikidata talk:Requests for permissions#Import bot feature list --Zanka (talk) 19:32, 23 November 2012 (UTC)

I think a import bot feature list could be useful. We could divide this list in required features for bot flag and nice to have features. It could be a guideline for granting bot flags. But it would also share ideas bot programmers have had. Other bot programmers could then implement useful features at their bot framework, too. I would prefer a very detailed list. I added some main features of my bot as example: Merlissimo (talk) 02:18, 23 November 2012 (UTC)

The huge table that used to be here was moved to Wikidata:Bots/Import bot features. Legoktm (talk) 17:52, 31 March 2013 (UTC)

I've just created a page Wikidata:Bots. Could we move this discussion here and remain the link on this page? --Zanka (talk) 14:25, 23 November 2012 (UTC)

Sure. Merlissimo (talk) 14:38, 23 November 2012 (UTC)

At the Moment i would use my botrights mainly to create bigger and faster api request for Wikidata:Labels and descriptions task force and i would slowly an carefully start automatic adding of items. --Sk!d (talk) 23:33, 24 November 2012 (UTC)

Choboty adds labels based on the naming convention. For example, label for "en:George IV of the United Kingdom" would be "George IV". "of the United Kingdom" part is a kind of disam part. -- ChongDae (talk) 01:07, 26 November 2012 (UTC)

Your Bot causes results like this: . But i do not have any idea how this can be solved automatically at the moment. Merlissimo (talk) 01:42, 26 November 2012 (UTC)
Is there any problem? I removed disamb part from the label. -- ChongDae (talk) 02:46, 26 November 2012 (UTC)
No. Only sth. that could be improved if somebody has a good idea how to include the title disambig part in the description. Merlissimo (talk) 09:58, 26 November 2012 (UTC)
We can start by adding the part in brackets to the description. (Without brakets of course) --Sk!d (talk) 11:01, 26 November 2012 (UTC)

I was thinking: "I think is not difficult to set the codes of a bot so that if reads x then adds y." More precisely, if it reads in the description field in English x, then puts that x and the associated element somewhere, it keep them in mind in one page and continues reading. At the end it will check what are the most frequent descriptions and draw up a list of the most common descriptions (it's fine!) that a human can read and translate in more tongues. Once that the translations arrived it hands them in the pages in which he had kept in mind there is that English description. The productivity of Wikibase explodes in this manner. That is a kind of "match, split and replace". Raoli (talk) 17:34, 26 November 2012 (UTC)

Approval process[edit]

I don't understand this strict workflow for approval. In most wikis it is normal to run a limited amount of test edits before applying for botflag and the community does not need to agree twice. A bot owner may want to know if (s)he is able to do the task before telling the community (s)he wants to do it. A few dozens of test edits with a limited speed won't harm anything. I propose a more flexible approach. Bináris (talk) 21:52, 30 November 2012 (UTC)

Symbol support vote.svg Support. It's ok to test first. --Stryn (talk) 21:56, 30 November 2012 (UTC)
Symbol support vote.svg Support. But if you want to do e.g. automatic "Hello messages" to every new user the task should be approved by the community. But this should not be a limit for botflag. If yout bot does a new task and somebody says i don't approve with this, the botowner should stop his bot (if there were no community decision) and ask the community if the majority wants the bot to do this task. --Sk!d (talk) 22:57, 30 November 2012 (UTC)
Symbol support vote.svg Support Approval before doing test edits is not needed. Limitations to that like the one Sk!d mentioned is fine to me too.--Snaevar (talk) 00:17, 2 December 2012 (UTC)
Symbol support vote.svg Support but only as many as the user can revert manually --Bene* (talk) 07:58, 2 December 2012 (UTC)
✓ Done Changed the page to reflect the conseus on this. I am going to let someone else close this though, as I voted on this proposal.--Snaevar (talk) 01:09, 30 January 2013 (UTC)

Bot-flag policy[edit]

Here are some questions, which would be used to form an bot-flag policy: 1. How many test edits should bots make?

  • About 50 --Sk!d (talk) 00:55, 2 December 2012 (UTC)
  • 50 --Bene* (talk) 08:06, 2 December 2012 (UTC)
  • Around 50 for item-related tasks and around 5 on others (archiving, create list of ...). --Zanka (talk) 13:41, 2 December 2012 (UTC)
  • 50 pages and for import bots 200 pages Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  • 50 pages sounds good, gives lots of material to look for errors in. Ajraddatz (Talk) 15:41, 2 December 2012 (UTC)

2. Should the bots get blocked if they make too many test edits?

  • Not permanent, only if the bot is spamming or clearly not working --Sk!d (talk) 00:55, 2 December 2012 (UTC)
  • The operator should be told to stop the bot but the bot should only be blocked, if it isn't working. --Bene* (talk) 08:06, 2 December 2012 (UTC)
  • If bot continues edits without bot flag/request for bot flag after request to stop then yes, of cause bot should work correctly. --Zanka (talk) 13:41, 2 December 2012 (UTC)
  • If bot does not stops after notice on bot operator talk page. Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  • If we inform the owner at the bot doesn't stop. The biggest concern should be what the bot is doing, though; if it is making unproductive edits, it should be just blocked without warning. Ajraddatz (Talk) 15:41, 2 December 2012 (UTC)

3. Should bot-flags be removed if bots don´t make any edits for awhile ?

  • Symbol oppose vote.svg Oppose --Sk!d (talk) 00:55, 2 December 2012 (UTC)
  • Symbol oppose vote.svg Oppose --Bene* (talk) 08:06, 2 December 2012 (UTC)
  • Symbol oppose vote.svg Oppose --Zanka (talk) 13:41, 2 December 2012 (UTC)
  • Symbol oppose vote.svg Oppose --ChongDae (talk) 08:30, 7 December 2012 (UTC)
  • I'd say that the flag should be removed if both the bot and owner are inactive. Ajraddatz (Talk) 15:41, 2 December 2012 (UTC)
3.1 If so, after how many months would a bot become inactive ?
  • 2 years. Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  • Six months I'd say - pretty standard. Ajraddatz (Talk) 15:41, 2 December 2012 (UTC)
  • 1 year if the bot owner are inactive in wikidata and his/her home-wiki. In phase 2, bots will update demographic data based on the 'census'. In general, censuses would be taken every 5 or 10 years. -- ChongDae (talk) 08:30, 7 December 2012 (UTC)

4. For how many days should the bot-flag request remain open ?

  • at least a week --Sk!d (talk) 00:55, 2 December 2012 (UTC)
  • a weak (same as administrator requests) --Bene* (talk) 08:06, 2 December 2012 (UTC)
  • a week --Zanka (talk) 13:41, 2 December 2012 (UTC)
  • one week + all questions answered Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  • One week + questions answered makes sense to me. Ajraddatz (Talk) 15:41, 2 December 2012 (UTC)
  • at least a week -- ChongDae (talk) 08:30, 7 December 2012 (UTC)

Apart from these there are a few things I would add to the bot-flag policy that I think are non-contriversial. Those are:

  1. The bot needs to be compatible with Wikidata (interwiki bots would have to have support for wbgetitems/wbsetitem (like mentioned in the table above) to be compatible with wikidata).
    Symbol support vote.svg Support --Bene* (talk) 08:06, 2 December 2012 (UTC)
  2. The operator will have to create an userpage for their bot (with the bot template).
    Symbol support vote.svg Support --Bene* (talk) 08:06, 2 December 2012 (UTC)
    Symbol support vote.svg Support Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  3. Interwiki bots would skip individiual interwiki links that link to an section, check for interwiki conflicts and not add entities with interwiki conflicts.
    Symbol support vote.svg Support --Bene* (talk) 08:06, 2 December 2012 (UTC)
    Additionally do not add redirects Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  4. Bots would be run on an account seperate from the operator.
    Symbol support vote.svg Support --Bene* (talk) 08:06, 2 December 2012 (UTC)
    Symbol support vote.svg Support Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  5. Bot operators must be available to answer any comments about their bot. --Snaevar (talk) 00:17, 2 December 2012 (UTC)
    Symbol support vote.svg Support --Bene* (talk) 08:06, 2 December 2012 (UTC)
    Symbol support vote.svg Support
    • Additionally for entity bots while extension is developed: bot operator must have enough programmings skills to modify code hisself. Merlissimo (talk) 15:28, 2 December 2012 (UTC)
  6. Bots should be blocked, if they aren't working, and unblocked, if the operator has fixed it. --Bene* (talk) 08:06, 2 December 2012 (UTC)
    Symbol support vote.svg Support --Bene* (talk) 08:06, 2 December 2012 (UTC)
    weak Symbol support vote.svg Support maybe not blocked, if operator stop the bot after request. --Zanka (talk) 13:41, 2 December 2012 (UTC)
    short block of few hours + operator notice Merlissimo (talk) 15:28, 2 December 2012 (UTC)
    Symbol oppose vote.svg Oppose as some bots are semi-automatic and require the bot flag to be run, bots will not run all time. For instance, imagine a bot that need a list to be generated to update labels before it is run. Once the bot as finished his work another list has to be supplied. At the beginning it will be easy to find labels that are lacking, but the more labels will be filled, the more it will be difficult to find correct rules. Perhaps a good idea would be to allow bots edition for a a certain amount of time : if the bot has nothing to do anymore, it has to request for a deletion of his permission or at the end of his permission, if it thinks it can find other rules, request for an extra time. One more fact for my opposition is that a bot can be unworking because no items anymore fit its work but that does not mean that it has nothing to do anymore. As not all items are created, a non running bot does not mean a bot has no more work to do --Thieol (Thieol) 22:57 30 December 2012 (UTC)

Symbol support vote.svg Support all things above. --Sk!d (talk) 00:55, 2 December 2012 (UTC)

Likewise support the non-controversial stuff, but perhaps only block the bot for a couple of hours if it is not working properly. Ajraddatz (Talk) 15:42, 2 December 2012 (UTC)
✓ Done Added the suggestions with a majority of votes to the page. I think this can be closed now, but since I started it, I am going to let someone else close it.--Snaevar (talk) 01:09, 30 January 2013 (UTC)

Duplication check?[edit]

Pictogram voting question.svg Question. Is there a duplication check for label/description in the new API? There are TWO king Jeongjong in Goryeo dynasty, the 3rd king (Q493659) and the 10th king (Q494135). When uploading the later by bot, it failed. After changing the description of the later entry, I can create the entity. -- ChongDae (talk) 08:19, 7 December 2012 (UTC)

Yes there should be one you also cant change items with same descriptions + labels, so you first have to alter the items and you can then change other things. --Sk!d (talk) 08:26, 7 December 2012 (UTC)
But Help:Description recommends a short description. However is there a document for new API? mw:Extension:Wikibase/API seems to be outdated. -- ChongDae (talk) 08:39, 7 December 2012 (UTC)
Is not enough? --Sk!d (talk) 08:43, 7 December 2012 (UTC)
No error code, no conflict check rule. Should I read the source code? :( -- ChongDae (talk) 09:40, 7 December 2012 (UTC)

Rethink Bot Approval[edit]

Please see User:Addshore/Wikidata:Bots. I propose this as the new way to approve bots on Wikidata. Please click around the links (they should all stay on the pages created in my userspace). All comments welcome but in order to keep a central discussion please add them to User talk:Addshore/Wikidata:Bots instead of here. ·Add§hore· Talk To Me! 22:14, 14 February 2013 (UTC)

I slightly changed the text and removed the passage: "The bot operator is responsible for the repair of any damage caused by a bot which operates incorrectly." We should not force anybody to do anything if something goes wrong I think the bot operator should try to repair the damage. Sometimes it is not possible to repair everything. --Sk!d (talk) 01:17, 15 February 2013 (UTC)
I spotted your rewording on the page and have no objection to it ·Add§hore· Talk To Me! 04:17, 15 February 2013 (UTC)
Sure, it isn´t always possible to repair everything, but I do not think that is a reason to remove the sentance. I have reworded it like so: "The bot operator should repair damage caused by his bot, whenever possible."--Snaevar (talk) 10:32, 22 February 2013 (UTC)

"bot operators should not use a bot account to respond to messages related to the bot" -- is that a real problem or a solution to a maybe-once-problem? I have never seen this happening. Bináris (talk) 14:31, 17 February 2013 (UTC)

I don't think it is really needed either. ·Add§hore· Talk To Me! 20:18, 17 February 2013 (UTC)
I have noticed bot edits that where used to respond to messages three times. But even so, I think it is pretty straight forward that bot accounts should not be used in that manner. I do not think that rule is needed.--Snaevar (talk) 10:32, 22 February 2013 (UTC)

The sentance "Further trials may be needed but should first be requested and approved." added by Addshore in this edit needs to be removed. This was allready discussed in the section #Approval process where an majority voted against needing approval before making test edits. This isn't en.wikipedia, you know.--Snaevar (talk) 10:46, 22 February 2013 (UTC)

As the sentence suggests this is only for further trials, i.e. only after the first 50 edits? Otherwise you may as well not put the initial 50 limit on the trial and just say the bot op my trial forever until approved... ·Add§hore· Talk To Me! 12:00, 22 February 2013 (UTC)
I think we need a vote on this. I also disagree with your statement that it would make the 50 edit limit useless, as there are a number of wikipedias where that works fine.--Snaevar (talk) 00:45, 27 February 2013 (UTC)
I agree a vote is needed. And in regards to my statement I dont see why "Further trials may be needed but should first be requested and approved" should be removed, as it must be true...? Otherwise you are either saying there is a set 50 edit limit for a trial and you CANNOT go over it, or that there is really no limit to the trial and a bot could carry on 'editing in a trial' for ever... ? ·Add§hore· Talk To Me! 05:19, 27 February 2013 (UTC)

Importing description (on important fields)[edit]

If there are more than one item with the same name descriptions are very important especially for using of of these items as statements. So it would bee great to have bots to add these description wherever easily possible. Often the source of information can bee the categories of the page. For example, figures of Greek mythology are very often used as names for astronomical objects. Therefore it would be good to add a description like "Figure from Greek mythology" (and translations) to all items in en:Category:Greek mythology except for a few handpicked exceptions. Can any of the existing bots do such things and would is there – or should there be – a place where suggestions for rules like this can be collected for bot programmers? -- MichaelSchoenitzer (talk) 17:44, 26 February 2013 (UTC)

Statements with references[edit]

I am thinking about writing a bot for moving some of the data stored in Commons creator templates and institution templates, like alternative names, images places of birth, Authority control, etc. into statements. I usually write my bots in python based on pywikipediabot framework, and I started to look into available functions (see mw:Manual:Pywikipediabot/Wikidata). One think I do not see how to do is how do you add a reference, like "copied from Commons:Creator:Jean Alaux" to a statement using pywikipediabot. Also are there other related bots with open source-codes which can be studied? Should we encourage bot owners to publish their source code. --Jarekt (talk) 02:12, 12 March 2013 (UTC)

If you take a look at my code, it adds references. My code is licensed as CC-Zero, just like Wikidata is.
I'm also working on adding support for writing claims/references to pywikipediabot, should be done shortly. Legoktm (talk) 04:55, 12 March 2013 (UTC)
Thanks --Jarekt (talk) 12:19, 12 March 2013 (UTC)

Bot speed[edit]

Hello, because I found nothing about it, I ask here. How fast should Bots work on Wikidata? With multithreading my FischBot makes arround 15 edits per minute. If I would work with more threads, my bot would even faster than that. --Pyfisch (talk) 17:32, 7 April 2013 (UTC)

Be very careful with threading, you should only have one thread that communicates with Wikidata (but you can obviously have more threads that work internally with parsing, etc). mw:API:Etiquette is also some good reading. There is no actual speed limit that we can enforce/require, I normally just let my bot go as fast as it can (in one thread). Legoktm (talk) 18:30, 7 April 2013 (UTC)

Excuse my incompetence, but what do you mean with "multithreading"? --Ricordisamoa 21:12, 7 April 2013 (UTC)

See en:Multithreading_(software). Legoktm (talk) 12:33, 8 April 2013 (UTC)
How does this apply to JavaScript bots? (P.S.: SamoaBot is at 24-36 epm) --Ricordisamoa 20:25, 8 April 2013 (UTC)

There is a dispatch lag which means the time between there is a wikidata edit and the edit is shown on the wikipedias. You can find it here: Special:DispatchStats. This lag is currently getting bigger and bigger. The hardware is currently just to slow and there are too many edits by bots. I think we should add a temporary edit per minute limit (see the developer comment here). I think one edit per bot per minute second should be enough (so 60 edits/min). We could contact every bot owner (which are currently hitting the limit). And wait if this would help to get the lag down. On the long term I hope we do not need this limit. --Sk!d (talk) 22:41, 7 April 2013 (UTC)

You mean one edit per minute?! That wouldn't be 60 edits/min, it would be edits/h. From my point of view this is much to slow. While there is no possibility to add multiple Claims at once my bot have to make often five edits at one item. With an editspeed of one edit per minute I could do about 15 items per hour - editing manually would be faster... --Pyfisch (talk) 13:38, 8 April 2013 (UTC)
Yeah, its not practical at all. I had a comment here, and jeblad outlined a proposal here. For now, just go with the status quo I guess. Hopefully this whole mess is sorted out soon. Legoktm (talk) 14:25, 8 April 2013 (UTC)
No sorry I meant 1 edit per second, my fault. --Sk!d (talk) 16:48, 8 April 2013 (UTC)
Oh, good that sounds like a reasonable expectation :) Legoktm (talk) 16:54, 8 April 2013 (UTC)
I have changed my bot so it should not do more then 60 edits per minute. I also contacted User:KLBot2 and User:‎Dexbot as these bots are besides Legobot and my bot are the bots with the most edits. I hope if we wait some days (the current lag is over one day) the dispatch lag should go down. --Sk!d (talk) 18:47, 8 April 2013 (UTC)

Thank you! Slowing down the bots at the moment is a very good idea. We'll do our best to improve the dispatch system asap so it is able to deal with more again. --Lydia Pintscher (WMDE) (talk) 19:21, 8 April 2013 (UTC)

My bot speed is arround 75 edits/sec. I can't set the rate very well because I am running two threads (on different machines). I think the solution is not to reduce the rate of edits, it would be better to change the way of the dispatcher works. For example, changes in labels or descriptions don't affect Wikipedia articles. The change of properties currently only affects some languages ​​and not used much. Also would be possible to delay phase 2 a few days until the changes of Phase 1 are completed and changes in the properties don't produce a change in Wikipedia until that. --Kizar (talk) 19:23, 8 April 2013 (UTC)
Ummm, there's no way you're going that fast. Maybe you meant 75 edits/minute? Legoktm (talk) 19:39, 8 April 2013 (UTC)
There are already 11 wikipedias in phase 2 afaik. The English wikipedia should been deployed today. I think wikipeda users might get frustrated on phase 2 if they are changing something on wikidata and the edit does not get shown. This could be a bigger problem then some missing data on wikidata. New users even might start to add data by themselves. There should not be a fixed limit. Maybe you can just tell your bot to wait 1 second before making an edit (That's what I did). --Sk!d (talk) 19:47, 8 April 2013 (UTC)
You are right. Now I'm making 50 edits/min. But the queue is still getting longer... --Kizar (talk) 19:49, 8 April 2013 (UTC)
The lag is still growing about ~85 edits/min. Currently there are about 450~ edit/min see [1]. So to get the dispatch log down we should try to do not more then 350 edits/min. Dextbot is currently doing about 100 edits/min and Legobot about 200 edits/min. If they slow down there bots to 60 edits/min the lag should get smaller. (small estimation: currently there are 100 +200 + 50 + 60 ~ 360 edit per minute by these 4 bots, so 90 edits by useres and other bots. If we have 4*60 edits per minute by those 4 bots we have ~250 edits per minute this means there are 100 edits per minute for other bots and users) --Sk!d (talk) 20:11, 8 April 2013 (UTC)
(edit conflict) Yes, so there is little that can be done to reduce it apart from fixing it. I see about 1.120.000 edits between 20 CEST yesterday and today, how many are bots and how many by each bot etc. We can't control bots like watchdogs.
On Wikimedia wikis the edit rate restriction is meant to prevent the bots from slowing down human editing: that's what maxlag is for, while job queue is never a big problem (although it can mean that a template edit actually shows up only after many hours). The other purpose is not cluttering watchlists and so on, but nobody watches pages like that on a database, and new pages have no watchers. As long as bots respect maxlag, on Wikidata I think no limit should be set. Is there some worst offender? --Nemo 20:13, 8 April 2013 (UTC)
(edit conflict × 4)My bot speed is around 180 edit/min. for now and it'll be less than now gradually. It's hard for me to slow down the bot because it's like ten separate codes and rerunning each one of the codes means wasting lots of resources (Algorithm of the codes is based "what links here" and the bot is crawling in articles and if i want to rerun the code, bot will start crawling from the start) Is it really serious? if it is I'll stop one or two codes. another thing, we must use put_throttle in the editclaim and creatitem functions and it must be based on the current server lag (similar to the other wikipedia codes) if you think it's good tell me to add and commit that Amir (talk) 20:14, 8 April 2013 (UTC)
I stopped two codes and my bot speed is around 60 per min. if it is too high even now tell me to disable bot temporarily Amir (talk) 20:29, 8 April 2013 (UTC)

Since yesterday the lag already decreased about 100k edits so there is currently a decreasing rate of about 10k edits / hour. :) --Sk!d (talk) 07:25, 9 April 2013 (UTC)

Yes this is the important number. The lag is still increasing because it is catching up on a lot of edits. This is a bit confusing and I'm going to see if we can make it less confusing. Please keep edit rates as low as now for a bit longer. We're on the right track. Thanks! --Lydia Pintscher (WMDE) (talk) 08:35, 9 April 2013 (UTC)
The number of edits is going down, but the time is going up? [2] Legoktm (talk) 14:04, 9 April 2013 (UTC) Just saw my IRC scrollback, the number is really the most important here. Legoktm (talk) 14:09, 9 April 2013 (UTC)
It's increasing again. Someone should decrease his edit rate. --Kizar (talk) 16:55, 10 April 2013 (UTC)
What about stopping all "big" Bots until the lag is under 12h? I think everything different will not work. This is not a good solution, but slowing down the bots has not worked. --Pyfisch (talk) 17:17, 10 April 2013 (UTC)
I contacted User:BeneBot* he was editing with about 100 edits/min (there were about 400 edits/min at all). Since 4h he didn't do any edit and the current total editrate is about 280 edit/min. I think the lag should decrease again. Over the last two days the lag count decreased about 350k edits. --Sk!d (talk) 20:56, 10 April 2013 (UTC)

Mark as policy[edit]

Hi, for a long time now this page has simply been a proposed policy. I've re-worked it so that it is more in line with our current standards and explicitly spells out the requirements that were discussed on the talk page, as well as getting it ready for translation. At this point I believe it is ready to be officially marked as a policy. Thanks, Legoktm (talk) 00:27, 10 April 2013 (UTC)

  • Symbol support vote.svg Support Legoktm (talk) 00:27, 10 April 2013 (UTC)
  • Symbol support vote.svg Support Absolutely acceptable as an approved policy, in my opinion. Missing important points where also implemented recently. Thus, I have no objections. Regards, Vogone talk 00:31, 10 April 2013 (UTC)
  • What's not clear is whether bot operators have to get approval for every task, or whether it's blanket approval. --Rschen7754 00:56, 10 April 2013 (UTC)
    Granting someone a bot flag is a sign of trust, and we expect bot operators not to abuse it, so it would be "blanket approval". I don't see this as any different from the status quo, since we already approve bots to (quotes from approved requests) "[import] labels, descriptions, statements and in future other data too" and "Adding claims to existing items", which is probably as blanket as you can get. Legoktm (talk) 01:13, 10 April 2013 (UTC)
    Well, there's a distinction between say, importing labels and running an archiver bot for user talk pages. --Rschen7754 01:17, 10 April 2013 (UTC)
    Maybe, but if you trust someone to add labels to 1000 items, wouldn't you also trust them to archive people's talk pages? There would be no advantage to having them file an additional request just to get "approved" when we already trust our bot operators not to screw up. Legoktm (talk) 01:32, 10 April 2013 (UTC)
    The reason that additional tasks should be approved is so that the community gets a chance to have a say in the matter. For an example, some wikis use bots to welcome users, and some expressly forbid them to do so. It would be a particularly bad idea on such a multilingual wiki to allow such blanket approval. --Rschen7754 01:35, 10 April 2013 (UTC)
    I think that falls under use common sense. If there's a bot owner who thinks its okay to start welcoming users in an automated way without discussion, they shouldn't be receiving the bot flag in the first place. Bot owners are expected to know what's okay to do, and what's not. As a more practical example, I technically don't have an explicit approval for adding/modifying labels, yet there would be no advantage if I went ahead and filed a request to get "approval" to do so. If I started running a script to start adding labels right now, I doubt anyone would object. Legoktm (talk) 01:45, 10 April 2013 (UTC)
    It may be that that is what the community is for, but that is not how the status quo is (we have plenty of additional task requests at RFP right now), nor how the bot policy proposal is currently written. --Rschen7754 01:48, 10 April 2013 (UTC)
    Right, some people are requesting approval for each task, and some just got the flag and went with it, since the previous proposed policy wasn't clear either.
    If it's not obvious in the current proposal is written, any suggestions to make it more clear? Or you can just change it yourself? :) Legoktm (talk) 02:01, 10 April 2013 (UTC)
    I think it would be best to send this to RFC (yet another one :P) to get further community input. --Rschen7754 02:05, 10 April 2013 (UTC)
  • Symbol oppose vote.svg Oppose The last time blanket approval was requested by an operator for their bot, it was shot down. I believe that community consensus against blanket approval, and the above conversation indicates that this needs to be reflected in WD:Bots before it becomes policy. Sven Manguard Wha? 05:30, 10 April 2013 (UTC)
    • Hmm, I really don't think Wikidata_talk:RFP#Open_ended_bot_approval went either way. I'd like to leave this discussion open for a few more days so we can iron out the minor kinks, and if consensus isn't blindingly clear, it should go to a formal RfC like Rschen suggested. Legoktm (talk) 09:00, 10 April 2013 (UTC)
  • Pictogram voting comment.svg Note Unrelated to my oppose, I made some changes to tidy up the policy. Nothing that hasn't already been agreed upon by the people active in RfBot, I don't think. Sven Manguard Wha? 05:31, 10 April 2013 (UTC)
    • Thanks for the changes! I made one minor fix: that maxlag is something you should do, but not required.
    • About usernames, you changed the wording so that "bot" became a required part of the username. I'm fine with that, but I just approved User:Docu with script as a human-editing-at-high-speeds-account aka a bot. I know that on enwiki we've approved accounts with usernames like "User (AWB)". Maybe we can adjust the wording so these types of usernames would be ok?
    • Also, regarding revocation of approval, that was supposed to be intended if another person did not agree with the task (or whatever). I'm not sure why the bot owner themselves would request it. Legoktm (talk) 08:50, 10 April 2013 (UTC)
      • Like Snaevar mentioned below, I think 250 test edits before approval seem to be for me a bit much though. 50 edits seem sufficient for judging whether a script works or not to me. Regards, Vogone talk 13:06, 10 April 2013 (UTC)
  • Symbol oppose vote.svg Oppose for three reasons.
First of all, bot flags are not given as an blanket approval and they are not given solely as an sign of trust. Users that review bot-flag requests are reviewing edits to find whether the bot has made any errors or not. Preferrably, those users should think about whether the bot is likely to break something or not. That is the whole point of bot-flags, getting an approval that confirms that other users belive that the bot is not going to break something. Surely though, an user that trusts an bot-operator does not go trough as many edits of the bot in question as an user who has either no opinion or a pretty low opionion of the bot-operator.
Secondly, I don´t see the issue why bots should not be allowed to do test edits before requesting approval. As Bináris mentioned in the #Approval process section above, an bot-operator may wan´t to test whether his bot is capable of doing the task before applying for an bot-flag. I don´t see an reason why we should deny bot-operators that option.
Thirdly, there is neither any precedent for 250 test edits nor any need for it. I have gone through the bot-policies of WMF projects and the highest number of test edits (excluding the Global bots policy) is 50 edits. Now, the Global bots policy is not really relevant in this case, as the test edits required for an global flag are mostly obtained under local bot-flags. There is no need for 250 test edits either, as there is no sign that either a number of editors are sharing the workload of going trough all of those edits or that an single user is going through all of them themselves.--Snaevar (talk) 12:47, 10 April 2013 (UTC)
  • Pictogram voting comment.svg Comment, how about inactive bots? How long time they should be without edits that their bot flag can be removed? --Stryn (talk) 13:44, 10 April 2013 (UTC)
    I don't see why bot flags should be removed due to inactivity. Flagged bots don't harm anyone if they are inactive and removal due to inactivity would probably also cause multiple requests for approval for the same tasks, just because a bot hasn't edited for a longer period. This would be unnecessary, in my opinion. Regards, Vogone talk 13:53, 10 April 2013 (UTC)
    As Wikidata is a special project I think we should not remove a botflag because of inactivity. I think about a bot importing data from only on external site: E.g. the population of a city afaik they only get updated every year. Maybe there a pages which an even greater update interval. And as Vogone said if they are inactive they don't harm. --Sk!d (talk) 14:53, 10 April 2013 (UTC)
  • Symbol support vote.svg Support --Pyfisch (talk) 15:13, 10 April 2013 (UTC)
  • Pictogram voting question.svg Question - It states under the Statement adding bots section that:
1. Bots should be able to add P107 if possible. What does this mean exactly? In which cases would it not be possible?
2. Bots should add sources. Would simply stating "imported from: English Wikipedia" be considered a source?
Thanks, FrigidNinja 02:49, 11 April 2013 (UTC)
1. Lets say I was tagging a bunch of people from "Category:Italian footballers". I can set P107-->person. Now lets say I was tagging from the category "Born in 1847". This may include pets, famous animals, etc, so it would not be possible to set P107, since the values would be different for each item.
2. Yes, that would be. But if the bot can add a more specific source, it should. Legoktm (talk) 02:57, 11 April 2013 (UTC)
  • I've finally went through the proposed policy in detail (and made a few changes), and I believe that it's almost okay. My main problem is with the trial/test run section. I find it useful to ensure that my bot actually works before requesting approval, unless I'm confident enough that it should be quite simple. Otherwise, I'm guessing that I'd have to run the tests from my main account or an unflagged account? Also, maybe we should change the quantity to no more than 100. If we need more than that, we could always ask the operator to extend the trial.  Hazard-SJ  ✈  01:10, 14 April 2013 (UTC)
    I agree with you regarding edit quantity in the trial. 50 to 100 edits should be sufficient. In my opinion, test runs should be made from the unflagged, preferably autopatrolled (by admin), bot account. One could also discuss about giving a temporary bot flag for trial runs to avoid RC flood. Regards, Vogone talk 01:26, 14 April 2013 (UTC)
    Generally the edit rate in a trial should not be too high, so the bot flag might not be necessary during trials.  Hazard-SJ  ✈  02:12, 14 April 2013 (UTC)
    Then at least the edit rate has to be defined and/or mentioned in the policy. Regards, Vogone talk 02:14, 14 April 2013 (UTC)
  • I actually oppose setting a maximum edit rate on a bot by bot basis as was just added to the proposal by Sk!d. Firstly, the dispatcher stats aren't affected by all edits, and in addition, some bots might make multiple edits within a specific time period, then none over another time period.  Hazard-SJ  ✈  03:32, 14 April 2013 (UTC)
    I understand Sk!d's addition in a different way: a bot owner should be able to set a limit meaning in times it is required to stop excessive bot edits (as discussed in the next section) the bot owner should know how to slown down his bot. Sk!d didn't mention a specific maximum edit rate in his addition at all. --Knopfkind (talk) 03:48, 14 April 2013 (UTC)

bot moratorium[edit]

Heya folks :) is looking better and better after you decreased the edits per minute. Thank you! But we really need to get this down even more. Denny made a calculation and said that 24h of no bot editing should get the lag down to 0. Can you do us a favor and stop your bots for 24h soon? Then we have a chance to get to English Wikipedia soon.

The alternative would be to throw away all those changes once which would mean no purging of those pages and no showing up on watchlists and so on. I'd _really_ like to avoid this to not make the Wikipedias unhappy.

In the meantime Katie and Daniel are working on improving our ability to deal with more changes. It's currently the top priority. --Lydia Pintscher (WMDE) (talk) 09:25, 11 April 2013 (UTC)

  1. Symbol support vote.svg Support already stopped my bot. --Sk!d (talk) 12:02, 11 April 2013 (UTC)
  2. ✓ Done for SamoaBot (I succeeded to run it continuously) --Ricordisamoa 12:11, 11 April 2013 (UTC)
  3. ✓ Done for Choboty. -- 12:33, 11 April 2013 (UTC)
  4. Symbol support vote.svg Support My Bot, will not do regular edits for now, only some testedits maybe. --Pyfisch (talk) 15:38, 11 April 2013 (UTC)
  5. ✓ Done for Legobot, though it may still edit non-item pages like WD:RfD and my userspace. Legoktm (talk) 16:06, 11 April 2013 (UTC)
  6. already ✓ Done -- Bene* talk 16:28, 11 April 2013 (UTC)
  7. ✓ Done for ValterVBot. Sorry for the delay . --ValterVB (talk) 17:04, 11 April 2013 (UTC)
  8. ✓ Done for KLBot2. My bot is making arround 18 edits per minute. If I completely stop it will be difficult to resume the task in all languages... --Kizar (talk) 19:07, 11 April 2013 (UTC)
    Ummm, it's not done. It's still editing. Legoktm (talk) 19:09, 11 April 2013 (UTC)
    It's now.--Kizar (talk) 19:45, 11 April 2013 (UTC)
    Thanks :) Legoktm (talk) 19:48, 11 April 2013 (UTC)
  9. ✓ Done Dexbot, sorry for delay I thought I watched this page Amir (talk) 19:49, 11 April 2013 (UTC)

Should we block still-operating bots? --Ricordisamoa 14:56, 11 April 2013 (UTC)

I don't think so. This should be voluntary, unless they're going at excessive speeds like 300EPM. Legoktm (talk) 16:32, 11 April 2013 (UTC)
Pictogram voting comment.svg Comment @Lydia Pintscher: Are you planning a function to add multiple claims at once? If yes, when will this API function released? --Pyfisch (talk) 15:38, 11 April 2013 (UTC)
Yes it is planned but I unfortunately don't know when it'll be available. --Lydia Pintscher (WMDE) (talk) 15:41, 11 April 2013 (UTC)
Tell me when it's done, I'll made related change in PWB frameworkAmir (talk) 19:49, 11 April 2013 (UTC)

I think all bots which are updating interwikilinks (if a page gets moved or deleted etc.) should not stop. --Sk!d (talk) 20:29, 11 April 2013 (UTC)

Hmm that's a good point. I hope those aren't too many edits though? --Lydia Pintscher (WMDE) (talk) 21:03, 11 April 2013 (UTC)
My bot is (I believe) the main bot doing that (see its contributions). It's still configured to continue doing the task, and it's still archiving, since that doesn't involve making that many edits. However, if most of the other bots were stopped, this shouldn't be too much of a problem.  Hazard-SJ  ✈  03:21, 12 April 2013 (UTC)
Yeah things like archiving are no issue since they don't lead to a dispatched job to the Wikipedias. --Lydia Pintscher (WMDE) (talk) 08:01, 12 April 2013 (UTC)

As the median lag is down to zero I think we can start the bots again. But every Bot should keep under the limit of 60 edits / min. You should also watch this pages and Special:DispatchStats, the setup of the dispatcher is still the same. AFAIK the dispatcher might gets changed next week. If the lag starts rising again we maybe have to decrease the limit of edits even further. --Sk!d (talk) 15:08, 13 April 2013 (UTC)

Ok, I'm keeping SamoaBot at 12 epm. --Ricordisamoa 15:42, 13 April 2013 (UTC)
ValterVBot started, is under 30 edits/minute --ValterVB (talk) 16:34, 13 April 2013 (UTC)
A nice way to keep track of this here ·Add§hore· Talk To Me! 16:43, 13 April 2013 (UTC)
✓ Done for KLBot2. --Kizar (talk) 17:11, 13 April 2013 (UTC)

I think we can increase the limit to 80 edits/min. I would say as long as the lag does not starts to go up edit rates at about 100 edits/min should not be a problem, too. AFAIK the dispatcher is now on a new server location and should be a little bit faster. --Sk!d (talk) 19:35, 18 April 2013 (UTC)

So, what is the actual limit? It seems that there's always 0 minutes delay lag (Special:DispatchStats). Can we make 100 edits/min and see what happen to the lag? --Viscontino (talk) 19:24, 21 April 2013 (UTC)
There is no actual limit, Speed things up until you see a delay appear, then slow things down, try to find the balance. ·Add§hore· Talk To Me! 12:24, 22 April 2013 (UTC)

Add sources?[edit]

Hello, when shoud the FischBot add sources? Because I have to do for every added Source one edit, there are more than two times more edits for one item with sources necessary, than for an item without sources. Q64632 is a good example, claims have up to three sources. Are there propertys, where you would say, that there I don't need to add references? --Pyfisch (talk) 12:12, 14 April 2013 (UTC)

I think more references are better then less references. So I hope you could add all refererences. If you are worried about the edit-rate I hope in the future a edit rate of 100 edits/min should not be a problem per bot. --Sk!d (talk) 23:31, 15 April 2013 (UTC)
I can't really edit with 100 edits/min because of bandwith..., I hope for a new API function for adding multiple claims, in the best case with references. An "updataItem" function to do all changes at once would be ideal. --Pyfisch (talk) 19:17, 16 April 2013 (UTC)

Hello Pyfisch,
what are the sources you use, to fill up in Wikidata Items? In my opinion it is not senseful to add for example the country of citizenship with a source of "imported from german Wikipedia"! If german Wikipedia will use this Wikidata entry as a source, there is a cirlce of problems. Only extern sources have sence. What do you and other bot operators think? Conny (talk) 08:40, 28 April 2013 (UTC).

I think that "imported from german Wikipedia" is not the best soure, but it is better than no source for multiple reasons: Users can see from where the information comes and you can fix it in the Wikipedia, if the information is wrong. When my bot is running stable, I will try to add sources from the GND (Gemeinsame Normdatei) or other authority control systems. Right now informations like citizenship have no source in Wikipedia because it not necessary. --Pyfisch (talk) 08:51, 28 April 2013 (UTC)

Compromise proposal for blanket approval vs. approval for each task[edit]

I may be wrong, but to me it seems that the issue about blanket approval is the main unsolved item before achieving consensus for a bot policy. I have this compromise proposal which you may consider:

  • When a bot is approved for a task, this is also considered as an approval of the task. An approved bot may perform any approved task if it can be done using standard, well tested code. The bot owner has to request permission again to perform new tasks which are not approved before, or if the bot will be using new developed code. The bot should always have an updated description of all performed tasks on the bot user page.

If it is not acceptable, what should be changed? I think it is about time a policy is decided, as bots make most of edits at Wikidata. Byrial (talk) 10:30, 25 April 2013 (UTC)

In general, I would be fine with such a compromise but some of the points will probably need further discussion. In some cases it really isn't senseful to have two bots doing the same task at the same time, for example archiving talk pages, etc. Furthermore, I do not understand the first sentence. Could you explain it, please? Regards, Vogone talk 13:10, 25 April 2013 (UTC)
I don't think anybody would want to archive talk pages if another bot already does a fine job at it. But if the bot doing it now for some reason had to stop, it would be convenient if another bot could take over without the bureaucracy of another RFP.
In the first sentence I tried to say that when you approve that a bot may do some task, then other approved bots are also allowed to do that task. But bots may not do something which was never approved for any bot. Then you don't risk that they do something which is not wanted by the community (that could for instance be welcoming new users). Byrial (talk) 13:29, 25 April 2013 (UTC)
Okay then. But I'd prefer a new RFP for controversial tasks (should be gauged by common sense), even if they already were approved for another bot. Then I'd give a Symbol strong support vote.svg full support. Regards, Vogone talk 15:37, 25 April 2013 (UTC)
GA candidate.svg Weak support In general good, but you can't use always stadard code for a task, maybe the bot makes the task a bit different than another, or it is misconfigured. Because that I propose a check page for new tasks, that other people can check if the edits are ok. --Pyfisch (talk) 16:13, 25 April 2013 (UTC)
Actually I'd be opposed to this. When I approve a task, I check that a) it works, b) I trust the operator to not screw up, and c) if it does screw up, the operator has the necessary expertise to fix it. So if someone wants to add VIAF numbers, I would want to make sure that they understand how VIAF works and have experience using it, not that they just wrote a template parsing script and want to import something. If a task was approved for importing taxonomy data, I would not want someone who has 0 experience working with taxonomy but is proficient in writing bots running it (like me!). Legoktm (talk) 16:55, 25 April 2013 (UTC)
PS: Thanks for trying to propose a solution to this debate. Any progress is better than none :)

Requirements for statement adding bots[edit]

Now that a tool for this is available, I added "Monitor constraint violation reports for possible errors generated or propagated by your bot" at "Requirements for statement adding bots". Some of these reports regularly show bot generated statements. Most operator are fairly efficient in helping fix them. --  Docu  at 06:10, 11 May 2013 (UTC)

Bot flags for Mass API queries > No Edits[edit]

Please see Wikidata:Project_chat#Bot_flags_for_Mass_API_queries_.3E_No_Edits ·Add§hore· Talk To Me! 11:38, 26 May 2013 (UTC)

More precise requirements for statement-adding bots[edit]

The current requirements for statement-adding bots is to "Add sources to any statement that is added". That's rather imprecise and in practice, we've tolerated "imported from Wikipedia" as sufficient. It is indeed sufficient for many statements which are so basic that sourcing them more precisely borders on the absurd. But recently, there's been opposition to bots adding non-trivial statements for which "imported from Wikipedia" is not considered acceptable, is hard to reconcile with Wikidata:Introduction and will be definitely against the soon-to-be adopted guideline (see Wikidata:Requests for comment/References and sources). I propose that we simply change the sentence in the requirements from "Add sources to any statement that is added" to "Add reliable sources to any statement that is added". I would also add either there or in the "Approval process" section a sentence explaining that owners of statement-adding bots must show that the non-trivial statements are imported from a sufficiently reliable source. Pichpich (talk) 21:21, 3 June 2013 (UTC)

  • Symbol support vote.svg Support --Micru (talk) 21:39, 3 June 2013 (UTC)
  • Sure, I think that makes sense. We should probably also make sure that all bot-owners currently approved for adding statements are aware of the changes with a mass-message or something. Legoktm (talk) 22:23, 3 June 2013 (UTC)
  • Symbol oppose vote.svg Oppose we should continue to import the data from Wikipedia. We did tolerate some opposition by some generally inactive user on bot approval discussions, but we shouldn't let this spread further. It's really odd to set users comment on bot approval discussions that don't practice it themself. Looks a bit like .. what's it called? --  Docu  at 23:19, 3 June 2013 (UTC)
Did you just say that people who don't write bots shouldn't have a say on what bots do? And as a side question, why do you think bots should be allowed to import unsourced data even if it goes against Wikidata's sourcing guidelines? Pichpich (talk) 01:11, 4 June 2013 (UTC)
The question is rather: "Why do you misrepresent the draft as a guideline?". Please refrain from doing so in the future. Further: "Why do you add unsourced statements to Wikidata yourself?" --  Docu  at 06:05, 4 June 2013 (UTC)
Trivial statements don't require sourcing. As for the guideline draft, the discussions are focused on little tweaks but the principle of citing reliable sources has never been questioned and in fact, verifiability has been a core principle since day 1. Pichpich (talk) 09:29, 4 June 2013 (UTC)
  • Symbol oppose vote.svg Oppose we need to achieve critical mass in active users and in number of statements to get things going. Thse kind of limitations will kill this project in it's infancy. Multichill (talk) 08:31, 4 June 2013 (UTC)
Sourcing is critical if we want Wikipedias to use Wikidata for things other than interwiki links. Pichpich (talk) 09:29, 4 June 2013 (UTC)
Later on we can do more sourcing handwork. Look on en:Nupedia, the project was closed because nobody contributed, the review process was to long and complex. Wikipedia hasn't review and always sources from the beginning but today we have reached an acceptable of quality because Wikipedia started open and people liked to contribute. I think we should learn from it, don't kill a young project with to strict rules, first create usable content and then add excellent sources. --Pyfisch (talk) 17:09, 4 June 2013 (UTC)
Bot imports can focus on data that is sourced, not necessarily from Wikipdia, why not to import that instead?--Micru (talk) 17:33, 4 June 2013 (UTC)
Indeed. There are plenty of reliable sources from which good data can be imported and there are already proposed bots doing that. Yes, that's more complicated for bot operators but it's doable. In any case, if we let bot automatically add a few million statements, it's crazy to think that you'll attract humans willing to fix the sourcing issues. The best way to attract editors from Wikipedia is to protect Wikidata's credibility for solid information. Pichpich (talk) 18:32, 4 June 2013 (UTC)
  • Symbol oppose vote.svg Oppose Per Multichill. Pyb (talk) 16:45, 4 June 2013 (UTC)
  • Symbol oppose vote.svg Oppose Per the above comments. --Pyfisch (talk) 17:09, 4 June 2013 (UTC)
  • Symbol support vote.svg Support The only force of wikidata will be the reliable data from its database. Snipre (talk) 18:52, 4 June 2013 (UTC)
  • Symbol oppose vote.svg Oppose per multichill Amir (talk) 21:09, 4 June 2013 (UTC)
  • Symbol support vote.svg Support We need those sources, even if we have to slow down until the web page datatype is available. We promised we would include them when we started this project. Filceolaire (talk) 00:08, 5 June 2013 (UTC)
    • {{Citation needed}}, where did we promise that? Multichill (talk) 19:59, 9 June 2013 (UTC)
      • Wikidata:Introduction is one of the first pages created on this project and it has always included the sentence Wikidata will record not just statements, but their sources, thus reflecting the diversity of knowledge available and supporting the notion of verifiability. If that's not a promise to include sources, I don't know what is. Pichpich (talk) 17:35, 10 June 2013 (UTC)
        • It just says we include sources. We already do that now. It's doesn't say we include sources for every claim. Multichill (talk) 20:32, 10 June 2013 (UTC)
          • If you really think that this means "we include sources when we find that convenient" then you're just making up a story that doesn't exist. So why don't we source all statements? Because every single local variant of en:Wikipedia:Verifiability explains that content which is common knowledge need not be sourced explicitly. If you want to discuss whether birth dates are common knowledge, let's have that conversation but don't pretend that sourcing is not a mission of Wikidata just because you find it inconvenient in this case. Pichpich (talk) 21:58, 10 June 2013 (UTC)
  • Absolutly Symbol support vote.svg Support It's nearly impossible to reconstruct the real sources of imported from (P143). --Succu (talk) 20:44, 11 September 2013 (UTC)

Upcoming changes to wbeditentity[edit]

The development team is currently refactoring the wbeditentity API module. We are planning to remove the "exclude" parameter. But before we do that, we would like to get your opinion on that. Is there any bot which is using this parameter at all? Can anyone give us a good reason to keep that parameter? Tobias Gritschacher (WMDE) (talk) 09:16, 5 June 2013 (UTC)

The autoEdit and slurpInterwiki gadgets use this, but I don't think there would be significant issues with removal. --Ricordisamoa 09:50, 5 June 2013 (UTC)
PWB has also support for "exclude", but it doesn't appear to use it by default. --Ricordisamoa 09:56, 5 June 2013 (UTC)
Is there any other significant "change" to the API You would deploy? --Ricordisamoa 09:58, 5 June 2013 (UTC)
Not as I can say right now. But I let you know as soon as another significant change comes up. Tobias Gritschacher (WMDE) (talk) 13:40, 5 June 2013 (UTC)

There will be another important change to wbeditentity: we're going to add a parameter called "new" to specify what type of entity should be created. Either parameter "new" or "id" has to be set but they are not allowed to be both set in the same request. See bugzilla:49526. Tobias Gritschacher (WMDE) (talk) 12:55, 14 June 2013 (UTC)

Pywikipediabot has been patched, pyrev:11680 for rewrite and pyrev:11681 for trunk. Legoktm (talk) 18:43, 21 June 2013 (UTC)
Is the "new" parameter also required if I only want to edit an existing item? At present I can't edit entities by specifying "site" and "title". API says it requires "new" or "id". IW 14:11, 9 July 2013 (UTC)
Ok, I tried to add "new=item" to the query and it works. But then the parameter is named a bit inconveniently because it is not only for creating new entities. IW 14:29, 9 July 2013 (UTC)
Hmm, that shouldn't happen... ·addshore· talk to me! 18:04, 9 July 2013 (UTC)
Can anybody update this script, please? My knowledge of python is insufficient. JAn Dudík (talk) 20:21, 9 July 2013 (UTC)

Recommendations for bot framework?[edit]

Hi, I am planning to have a bot, so I would like to hear your recommendations for a good bot framework which works nicely with Wikidata's API. It must be free code which can run under Linux. The bot should be able to set and update statements including qualifiers and sources as necessary with data collected mostly from online websites. That could for instance be population and area numbers for Danish municipalities collected from Statistics Denmark's StatBank, or data about chess players collected from World Chess Federation's rating lists, and many other things. I have programming experience, but don't want to do all the work myself. What is best and easiest to use, and where can a get it? Thank you in advance for your advice! Byrial (talk) 17:40, 9 June 2013 (UTC)

PWB. --Ricordisamoa 17:44, 9 June 2013 (UTC)
I recommend downloading the "rewrite" branch via SVN. --Ricordisamoa 17:46, 9 June 2013 (UTC)
A quick guide for Wikidata. --Ricordisamoa 17:47, 9 June 2013 (UTC)
Thank you, I am looking at it. You will probably see RFBOTS for Byrialbot before long. Byrial (talk) 21:00, 10 June 2013 (UTC)
I'm sorry, but the last time I checked, there wasn't much support for editing Wikidata with the rewrite branch. I know the trunk branch has such support, but I'm not sure if rewrite has been updated for this as yet.  Hazard-SJ  ✈  03:15, 11 June 2013 (UTC)
Rewrite can do everything except qualifiers and TimeValue (will be implemented soon). If you mean actual editing of normal page text, that works just like normal. Legoktm (talk) 10:07, 11 June 2013 (UTC)
Yes, it's much better. --Ricordisamoa 12:31, 11 June 2013 (UTC)
It may be better, but I find it hard to get started. Most of the examples I find (here and in the ducumentation) is very brief and mostly about creating items and adding sitelinks, labels and descriptions. I found two discussions about having open code repositories for Wikidata bots (here and here) with positive comments, but it is nevertheless hard to find any actual used code. It would be great if someone could show a working example of PWB code to add statements with sources where there is no previous statement, and to add my source if the statement already exists with no source or others sources than mine. Byrial (talk) 14:15, 11 June 2013 (UTC)
I've written some code that does exactly what you want, and I'll publish it tomorrow. --Ricordisamoa 14:50, 11 June 2013 (UTC)
PS: in the meanwhile, you can take a look at Legoktm's GitHub repo. --Ricordisamoa 14:55, 11 June 2013 (UTC)
Great, thank you, I will look forward to it. Byrial (talk) 15:05, 11 June 2013 (UTC)
I'm in trunk team and you can do anything in trunk (except qualifiers) I work on qualifiers soon Amir (talk) 18:56, 11 June 2013 (UTC)
P.S. see in rewrite or trunk branch Amir (talk) 19:02, 11 June 2013 (UTC)

Bots that assign page moves to Wikidata should wait a bit[edit]

Hi, since the software is able to assign page moves from wikis automatically to Wikidata, bots that do the same work should wait about three or five minutes before editing. The automatic update has better edit summary and it would be better to avoid conflicts between the software and bots. (Hoo man on dewiki). I don't know exactly what Bots are active in assigning page moves, I only know Krdbot for dewiki pages. Regards, IW 11:26, 5 August 2013 (UTC)

BetaBot wait an hour approximately. --β16 - (talk) 17:45, 7 August 2013 (UTC)

invalid ID (P107)[edit]

Can bot operators who are operating bots which add Property 107 (Main type) please stop or disable this function to avoid triggering the abuse filter? Per a closed RfC now, this property is to be deleted one all uses are removed. All new additions of this property are being disallowed by the abuser filter in order to prevent re addition of removed ones and the addition of new ones. John F. Lewis (talk) 20:54, 17 August 2013 (UTC)

BREAKING API CHANGE: Ids changing to uppercase[edit]

See [3] for more details. Legoktm (talk) 15:03, 11 September 2013 (UTC)

Thank you for reporting. Regards, IW 17:44, 11 September 2013 (UTC)
I've just updated most of my scripts, thanks. --Ricordisamoa 20:28, 11 September 2013 (UTC)

New api functionality for next weeks deployment![edit]

Hello! Just a little message to let you know that with the deployment next week we will see two lovely new features for the api!

  1. There will be a merge api module which will simply allow you to merge two items
  2. wbeditentities will be able to edit claims! Yes! This means that you can update anything in an entity in a single edit!

Have fun :) Adam Shorland (WMDE) / ·addshore· talk to me! 09:26, 9 October 2013 (UTC)

I didn't found wbmergeitems at ApiSandbox on --Succu (talk) 12:13, 9 October 2013 (UTC)
Can you explain a little more about the second feature? thank you :) Amir (talk) 13:01, 9 October 2013 (UTC)
You should see these on tommorow!
Currently in wbeditentity you can provide a serialized entity using the 'data' parameter in the module! Until now this has only been able to contain labels, descriptions, aliases and sitelinks, any claims included in the serialization will just have been ignored and not saved. Now you can also provide claims within the serialization that you pass to wbeditentity.
This means that you can edit everything in an entity in a single edit, you can add 2 references to two different claims while changing a label and adding 2 sitelinks! :)
Adam Shorland (WMDE) / ·addshore· talk to me! 14:28, 9 October 2013 (UTC)
wbeditentity : I hope the generated autosummary is meaningful and the diff human readable. --Succu (talk) 21:13, 9 October 2013 (UTC)

Merging items[edit]

gerrit:116297 enables a new and easier way of merging items via the Wikibase API. You can write e.g.

pywikibot.ItemPage(site, id).mergeInto(pywikibot.ItemPage(site, otherId), summary=u'duplicate item')

Be careful since this API module doesn't accept the 'bot' parameter (yet), so you should first login with your bot account. --Ricordisamoa 12:57, 8 March 2014 (UTC)

Subject matter restrictions on bots[edit]

What would be the appropriate policy or guideline to add subject matter restrictions on bots? One restriction that I believe should exist is that since dates in Wikidata are by default in the Gregorian calendar, bots should not extract any date from another source (such as Wikipedia) and add it to Wikidata unless the bot has a mechanism to confirm the date is stated in the Gregorian calendar. One such mechanism would be to check that the date is in the year 1924 or later, since Greece was the last country to switch from Julian to Gregorian, and that was in 1923. Jc3s5h (talk) 15:54, 25 August 2014 (UTC)

Mass-undo / mass-revert?[edit]

As we all know, mistakes can and will happen. Instead of merely delegating this issue to clean-up jobs after the fact, should it not be feasible for a Wikidata bot to itself undo what it has done? The structured data approach should entail that Wikidata is the ideal place for this kind of approach. (Although I admittedly have a limited knowledge about bot implementation, I haven't yet seen this kind of bot behavior when mistakes do happen; more often than not, semi-manual cleanups seem to be needed after mistakes occur.) -- Therefore, I would like to suggest making it a requirement for any bot that it is able to undo its last edit to any given item. (There could, for example, be a requirement that it should be able accept a list of items IDs plus a filter argument, e.g. how many of its own last edits to undo, or a date cutoff point for indicating 'all my edits since a particular date'.) Such a requirement could be adopted both in the bot approval process, making it a requirement for bot adoption, as well as in the underlying software. So what do you think, is this possible currently, or would it require some changes to the API or underlying software before it is possible to realize such a level of control? If it is already possible, or in the works, that would of course be great! Fred Johansen (talk) 17:22, 7 December 2014 (UTC)

This seems like a good idea. It's also needed for Widar; see e.g. User_talk:Yamaha5#instance_of_.28P31.29. Emw (talk) 18:54, 7 December 2014 (UTC)

GZWDer (flood) flooding to much ![edit]

GZWDer (flood) is a bot opperated by GZWDer. He was approved here (by only one contributor ?).

Firstly, this bot do not have a "proper" bot user page (with a "stop me" link). Secondly what exactly do this bot is not realy clear appart from "semi-automatic tasks" (witch is obvious for a bot).

More importantly, the user operating this bot have been warned many time about his beaviour. See : here, here, here and lastly here.

This bot have many problems in the past :

  • Creating duplicates, and so giving lot of manual chek. The reply of the user were "You can manually merge them" and "Duplicates can not be prevented".
  • Made items with no labels. The user have just replyed than another bot (Dexbot) will correct his own bot.
  • Create a bunch of wikidata stub entries without even speaking first to the related community. No reply yet.

After so many warnings, i think is about time to stop this flooding bot ! Miniwark (talk) 12:18, 30 March 2015 (UTC)

There are many such bots without "stop me" in Wikidata. I own one too. It is obivious that bots create duplicate items, those should have been connected before they were imported. AFAIK noone has to ask any project if they may create items for their pages. (I haven't seen such case since Wikidata was launched.)
So do you want to stop them all? Matěj Suchánek (talk) 12:52, 30 March 2015 (UTC)
That's not only the missing button but the way to use the bot. If you run your bot in a correct way the button is not needed but when you do thing without any agreement with the persons involved in the topic and if your bot is doing mass bad edits, the button is becoming a problem. Snipre (talk) 13:06, 30 March 2015 (UTC)

--GZWDer (talk) 05:11, 31 March 2015 (UTC)

there is a very big difference between wikipedia projects and wikisource projects : wikisource has very few contributors, and there is a reflection in progress about how to import texts, and exactly which texts are to be imported to wikidata : books, obviously... but for chapters, it's less obvious, and for pages it is clearly "N/A". Besides, the technical structure of wikisource is very different from that of wikipedia…
tonight I had to ask deletion of more than 120 pages from a book, that you imported, because these were not entire text, but text of single pages, that could not be put in Page namespace for technical reason…
besides, there is a need for data to add properties, that are much more difficult to get than from wp, because of transclusion that does not allow visualisation in wd, and categories are much less developped… all the work afterwards has to be made by hand and cannot be automated, which makes large blind imports very uneasy to manage…
there is absolutely no urge to import all wikisource very quickly, especially if items stay without claims for months (or years), because there is noone to document them… a systematic import, by author for example (or by source for articles of periodicals), is more efficient and easier to document afterwards…
The consensus of wikisource projects is to import validated texts first, then corrected texts, and then see for the others… you imported blindly, when we were working on tools to do it properly, which is why I am so irated about your bot… it works randomly, in languages it does not understand… that's crap… :(
if GZWDer just does not understand what he does, at least, just stop importing from wikisource… let ws-contributors work on it, as they are now, and let them do it properly - thanks --Hsarrazin (talk) 22:56, 13 May 2015 (UTC)


  • Properties can be added by autolist, and in the future by bot; but autolist doesn't create any new items
  • Already stopped importing from Wikisource but Dexbot will still create items for new Wikisource pages (while many items for old pages were created myself)

--GZWDer (talk) 05:00, 14 May 2015 (UTC)