Wikidata:WikiProject Authority control

From Wikidata
Jump to navigation Jump to search

Wikidata pays a lot of tribute to authority control, linking to all kinds of datasets and databases with various IDs. The holy grail of every GLAM worker Sum of All People, with links to their Works is coming about!

But we’re just at the start of a lot of work in that direction. The purpose of this project is to try and coordinate such work.

I know a few things (ViafBot, Mix-n-Match) and I'd like to help with some things, but I don't know what others are doing. --Vladimir Alexiev (talk) 08:08, 27 January 2015 (UTC)

VIAF information has been moved to Wikidata:VIAF.

Data Sources[edit]

The report Name Data Sources for Semantic Enrichment shows that when it comes to name data sources, maybe the two that matter are VIAF and Wikidata.

  • Their name coverage is fairly orthogonal: VIAF has more name variations and permutations, Wikidata has more translations (Venn diagram of names for Cranach).
  • VIAF is much bigger: 35M persons/orgs. Wikidata has 2.7M persons and maybe 1M orgs.
  • Only 0.5M of Wikidata persons/orgs are coreferenced to VIAF, with maybe another 0.5M coreferenced to other datasets, either VIAF-constituent (eg GND) or non-constituent (eg RKDartists). So coreferenced part between the two is still quite small (30%) and a lot of work remains!
  • A lot can be gained by leveraging coreferencing across VIAF and Wikidata: finding errors in Authority files, finding merge candidates in Wikidata, promulgating identifiers...
  • Wikidata has great tools for crowd-sourced coreferencing.

Please comment!

RKDArtists Coreferencing[edit]

RKDartists is an important Authority that does not yet participate in VIAF. There are already 21760 RKDartist id's on Wikidata. These could be imported to VIAF for free!

British Museum Coreferencing[edit]

The BM has several thesauri that are not co-referenced to anything in the world. I think they'd see it as a major win if the community helps them to co-reference.

This could be followed by importing the 2.5M cultural objects of the BM.

ULAN Coref Relations[edit]

ULAN does record possible matches and mismatches in their editorial system: ULAN Artists Whose Identity May be Associated or Confused With Another (608 pairs).

Looks like this:

x x_name x_bio rel y y_name y_bio
ulan:500071106 Master of 1515 Portuguese painter, active 1515 gvp:ulan1005_possibly_identified_with ulan:500025279 Afonso, Jorge Portuguese painter and court artist, born ca. 1470-1475, died before 1540
ulan:500042027 Master of the Madre de Deus Retablo Portuguese painter, active 16th century gvp:ulan1005_possibly_identified_with ulan:500025279 Afonso, Jorge Portuguese painter and court artist, born ca. 1470-1475, died before 1540
ulan:500032055 Monogrammist A. M. Spanish artist, active 19th century gvp:ulan1005_possibly_identified_with ulan:500038287 Aguirre, Marcial Spanish sculptor, 1841-1900

Here's to proper coreferencing! --Vladimir Alexiev (talk) 18:07, 12 March 2015 (UTC)

Match Persons not Disambiguation Pages[edit]

We should match persons to persons, not disambiguation pages to persons or other disambiguation pages.

Wikipedias, GND and RKD all have disambiguation pages (in GND they are called "undifferentiated names"). 13 Feb 2015:

Do you agree with my reasoning:

  • Jane said "any match is better than none"
  • I countered "A correct match is better than none"
  • the only way to make sure it's correct is to examine more data about the person, which will necessarily lead you to a real person page.
  • Look at the ULAN data above: that's good data that gives you some basis for decision. A name alone does not.

--Vladimir Alexiev (talk) 19:34, 12 March 2015 (UTC)

Symbol support vote.svg Support @Vladimir Alexiev, Randykitty, Ghuron: A GND Tn (Thesaurus name = undifferentiated) is not a stable disambiguation page. A Tn is a placeholder. It can be deleted, it can be upgraded into a Tp (Thesaurus person), or changed into a redirect. Works connected with a Tn will be checked by the library or archive who owns them and afterwards might be delinked. The database Online GND (OGND) includes only Tp numbers. --Kolja21 (talk) 00:04, 28 March 2015 (UTC)

Coreference AAT[edit]

AAT is a crucial thesaurus in cultural heritage.

    select (count(*) as ?c) {
      ?x a skos:Concept; skos:inScheme aat: }

I think that's BAD. I'm sure going to need that coref for the Europeana Food and Drink Classification Scheme that will be based on Wikidata and AAT:

Update: the AAT-Wordnet coreference described below is brought into Wikidata. AAT is actively coreferenced on Mix-n-Match: 12985 (32%) matched, 3293 (8%) awaiting confirmation, 1553 (3.8%) confirmed no-matches, and 22543 (55.7%) awaiting matching. So it's way better than 2 years ago. Help coreference this pivot thesaurus that is of immense importance for Cultural Heritage! --Vladimir Alexiev (talk) 15:04, 19 September 2017 (UTC)

Coreference AAT with Mix-n-Match[edit]

The Wikidata coref tool Mix-n-Match has mostly been used for people until now. But I hope it can be used for concepts as well.

I made an export that includes AAT URL, preferred English label (without qualifier), parents (ascendants to root) and scope note (description). Could also add alternative labels, and labels in other languages (Dutch, Spanish, Chinese).

select ?id (str(?lab) as ?label) ?parents (str(?scopeNote) as ?note) {
  ?x a gvp:Concept; dc:identifier ?id; gvp:prefLabelGVP/gvp:term ?lab;
     gvp:parentString ?parents.
  optional {?x skos:scopeNote [dct:language gvp_lang:en; rdf:value ?scopeNote]}
}

I saved as XML then converted to TDV: aat.rar.

    rset --results tsv aat.xml > aat.tdv

Also see https://meta.wikimedia.org/wiki/Talk:Mix%27n%27match#Coreference_AAT !!!

Coreference AAT through BabelNet[edit]

Mix-n-Match has good automatic matching, but that works for people.

So let's check what other vocabs that are coref to AAT may be coref to Wikidata: According to Michiel Hildebrand's famous CH LOD diagram:

CH LOD, cultural heritage linked open data (thesauri only)
  • Wordnet. No such prop in Wikidata
  • I'd guess Wiktionary is coref to Wordnet, but Wikidata got no site links to Wiktionary
  • RKD Concepts. There's prop "RKDartists" and "RKDimages" but none for concepts
  • Rijksmuseum Concepts. There's "Rijksmonument" but none for concepts
  • Joconde: aha! There's Joconde work ID (P347), and it has 2275 instances, so that's better. Joconde is 18% coref to Wikidata but I don't know how much to AAT, maybe I can gain 1k here.
    • Looked at the results: nope, Joconde are all paintings, not concepts
  • Bibliopolis: never heard of it, and nope
  • SVCN: never heard of it, and nope

Then it dawns on me.

AAT-Wordnet coref[edit]

Ok, so off to look for that AAT-Wordnet coref. - Why yes, it's part of http://semanticweb.cs.vu.nl/europeana/skos/browse/ - I got a file from somewhere that says

<aat_wordnet20_mappings>
  a void:Linkset;
    dcterms:title "AAT-Wordnet 2.0 mappings by Anna Tordai (baseline)" ;
    lib:source <http://semanticweb.cs.vu.nl/lod/getty/aat/> ;
    void:dataDump <bl_aat_wn.rdf> , <bl_norm_aat_wn.rdf> , <bl_sing_aat_wn.rdf> .

(Note: you can get those files from URLs like: http://semanticweb.cs.vu.nl/europeana/api/export_graph?graph=http://semanticweb.cs.vu.nl/lod/getty/aat/bl_sing_aat_wn.rdf&mimetype=text/plain&format=turtle)

These are called "baseline" (i.e. mostly literal matches). A quick conversion to Turtle and a line count:

$ wc -l bl*
   2300 bl_aat_wn.ttl
   4369 bl_norm_aat_wn.ttl
   4303 bl_sing_aat_wn.ttl
  10972 total

Run a query at http://semanticweb.cs.vu.nl/europeana/user/query (specify entailment=None or else!):

prefix getty:  <http://purl.org/vocabularies/getty/> 
prefix aat:  <http://purl.org/vocabularies/getty/aat/> 
select * {?x skos:inScheme getty:aat; skos:closeMatch ?y}

It returns 4592 (see below why).

AAT-Wordnet Overlaps[edit]

There is significant overlap between the files:

$ cat bl* | sort| uniq | wc -l
4596
$ cat bl* |cut -d " " -f 1 | sort| uniq | wc -l
4581

The following AAT concepts have 2 matches:

aat:bleachers
aat:boxcars
aat:cleavers
aat:feudalism
aat:groats
aat:jackstraws
aat:lats
aat:leotards
aat:morocco
aat:ninepins
aat:quoits
aat:shekels
aat:stairs

We need to reconcile them manually, eg

aat:bleachers  skos:closeMatch  <http://www.w3.org/2006/03/wn/wn20/instances/synset-bleacher-noun-1> .
aat:bleachers  skos:closeMatch  <http://www.w3.org/2006/03/wn/wn20/instances/synset-bleachers-noun-1> .

The AAT definition is:

  • aat:bleachers vp:descriptiveNote "Use for benchlike tiered seating for spectators at, for example, outdoor sporting events, usually without weather or sun protection, affording less advantageous views than grandstands; may also be used for similarly constructed, often telescoping, indoor seating."@en .
  • Inspection at Wordnet 3.1 shows that the second one is right.

That's 4.6k matches, or 11% of AAT.

AAT-Wordnet2 Representation[edit]

The coref looks like this:

    aat:wrought_iron skos:closeMatch <http://www.w3.org/2006/03/wn/wn20/instances/synset-wrought_iron-noun-1> .

And there's another file aat.ttl with rep like:

aat:wrought_iron aat:parentPreferred aat:iron_alloy .
aat:wrought_iron vp:id "300011012" .
aat:wrought_iron vp:labelPreferred "wrought iron"@en .
aat:wrought_iron vp:labelNonPreferred "iron, wrought"@en .
aat:wrought_iron vp:labelNonPreferred "wrought-iron"@en .

This is quite old rep. The new rep uses numeric URL: http://vocab.getty.edu/aat/300011012 (and a bunch more data). So we need to construct a numeric URL.

AATNED-Cornetto Mapping[edit]

Cornetto is NL Wordnet and AATNED is NL AAT. I got another file saying:

<aatned_cornetto_mappings>
  a void:Linkset ;
    dcterms:title "AATNED-Cornetto mappings by Anna Tordai (baseline)";
    lib:source <http://semanticweb.cs.vu.nl/lod/rkd/aatned/> ;
    void:dataDump <bl_aatned_cn.rdf.gz> , <bl_norm_aatned_cn.rdf.gz> , <bl_sing_aatned_cn.rdf.gz> .

Eg we have this for AAT 300191645 "salinity":

bl_aatned_cn.ttl: aatned:zoutheid  skos:closeMatch  cornetto:synset-zoutheid-1-noun .
cornetto-wn20.ttl: cornetto:synset-zoutheid-1-noun cornetto:eqNearSynonym instances:synset-brininess-noun-1 .
cornetto-wn30.ttl: cornetto:synset-zoutheid-1-noun cornetto:eqNearSynonym wn30:synset-brininess-noun-1 .
aatned.ttl: aatned:zoutheid core:notation "300191645" .

The number of AATNED-Cornetto matches is as follows:

> cat bl*|sort|uniq> bl_aatned_all.ttl
> wc -l bl_aatned_all.ttl
6917 bl_aatned_all.ttl
> cat bl_aatned_all.ttl|cut -d " " -f 1 | sort| uniq | wc -l
6857

There are more matches than AAT-Wordnet. There are also overlaps: 60 AATNED concepts (0.9%) have two Cornetto matches.

We need to merge AATNED-Cornetto with AAT-Wordnet. The correlation is simply by id, eg

aatned.ttl: aatned:zwerfkeien core:notation "300011671"
aat.ttl: aat:boulder vp:id "300011671"

I guess the overlaps between them are quite big, eg for wrought_iron:

aatned.ttl: aatned:smeedijzer core:notation "300011012"
bl_aatned_all.ttl: aatned:smeedijzer  skos:closeMatch  cornetto:synset-smeedijzer-1-noun .
cornetto-wn20.ttl:cornetto:synset-smeedijzer-1-noun cornetto:eqNearSynonym instances:synset-wrought_iron-noun-1 .
cornetto-wn30.ttl:cornetto:synset-smeedijzer-1-noun cornetto:eqNearSynonym wn30:synset-wrought_iron-noun-1 .
DBpedia-Wordnet3 coref[edit]

The other problem is bigger:

    bn:s00081730n skos:exactMatch dbpedia:Wrought_iron, lemon-WordNet:wn30-14802262-n

It doesn't look like Wordnet3 and Wordnet2 share any IDs; we'll deal with that in next section.

Lets first do some queries at http://babelnet.org/sparql/ to see what we can see. Look for DBpedia-Wordnet matches:

SELECT * WHERE {
  ?x skos:exactMatch ?y, ?z 
  filter(strstarts(str(?y),"http://dbpedia.org/resource/"))
  filter(strstarts(str(?z),"http://lemon-model.net/lexica/pwn/"))
} LIMIT 30

Download: https://www.dropbox.com/s/92gq5r1qm3yytkp/WN3toDBP.csv?dl=1.

It has 47607 rows like this (there's a decent chance this will cover the 6k AAT matches):

"http://babelnet.org/rdf/s00075206n","http://dbpedia.org/resource/Sundowner_(drink)","http://lemon-model.net/lexica/pwn/wn30-07913081-n"
"http://babelnet.org/rdf/s00039711n","http://dbpedia.org/resource/Sonora_(genus)","http://lemon-model.net/lexica/pwn/wn30-01736256-n"
"http://babelnet.org/rdf/s00070026n","http://dbpedia.org/resource/Sealskin","http://lemon-model.net/lexica/pwn/wn30-04160261-n"
Wordnet3-Wordnet2 coref[edit]

Since Wordnet3 and Wordnet2 don't share any IDs, we can try to use Wordnet2-Wordnet3 coref made by Jacco van Ossenbruggen and Marc van Assem (VU University Amsterdam) in May 2010 with this VOID (manifest):

<wn30-wn20-mappings-jacco>
	a void:Linkset ;
	dcterms:title "synset-level mappings from Wordnet 3.0 to 2.0, created by jacco's code" ;
	lib:source <http://purl.org/vocabularies/princeton/wn30/> ;
	void:dataDump
		<label-child-matches.ttl.gz> ,
		<label-childparent-matches.ttl.gz> ,
		<label-instance-matches.ttl.gz> ,
		<label-meronym-matches.ttl.gz>,
		<label-neargloss-matches.ttl.gz> ,
		<label-parent-matches.ttl.gz> ,
		<label-unique-matches.ttl.gz> ,
		<nearlabel-matches.ttl.gz> ,
		<glossmatches-m.ttl.gz> .

<wn30-wn20-mappings-sense>
	a void:Linkset ;
	dcterms:title "synset-level mappings from Wordnet 3.0 to 2.0, created by Mark using the Princeton WordSense mappings" ;
	lib:source <http://purl.org/vocabularies/princeton/wn30/> ;
	void:dataDump
		<synset-matches-based-on-multiple-sense-mappings-princeton.ttl.gz> ,
		<synset-matches-based-on-single-sense-mappings-princeton.ttl.gz> .

It's a complex affair consisting of many steps, but the major step (contributing 87% of all matches) is glossmatches-m.ttl that looks like

    wn30:synset-wrought_iron-noun-1 terms:replaces instances:synset-wrought_iron-noun-1 .

And looking at wordnet-synset.ttl, we find the required wn30 ID:

    wn30:synset-wrought_iron-noun-1 wn20schema:synsetId 114802262 .
AAT-Wikidata Sheets[edit]

After much querying and manual cleaning (over a day of effort), I made some sheets in this google folder:

  • AAT-DBpedia-Babelnet.xlsx: 3324 potential matches, fairly clean, but need checking by more people
  • AAT-DBpedia-Babelnet-80-judged.xlsx: example of correct & incorrect matches
  • AAT-Wikidata-25-judged.xlsx: example of correct & incorrect matches on Mix-n-Match

T.seppelt (talk) 21:00, 18 February 2016 (UTC) Vladimir Alexiev (talk) 11:59, 13 March 2017 (UTC) GerardM (talk) 15:58, 26 March 2017 (UTC) Jonathan Groß (talk) 17:52, 26 March 2017 (UTC) Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits Jneubert (talk) 13:47, 29 April 2017 (UTC) Framawiki (please notify !) (talk) Sic19 (talk) 20:42, 12 July 2017 (UTC) Wikidelo (talk) 21:15, 8 May 2018 (UTC) salgo60 Salgo60 (talk) 07:09, 10 June 2018 (UTC) ArthurPSmith (talk) 19:52, 22 August 2018 (UTC) PKM (talk) 19:40, 23 August 2018 (UTC) Ettorerizza (talk) 06:44, 8 October 2018 (UTC) Fuzheado (talk) 03:47, 19 December 2018 (UTC) Daniel Mietchen (talk) 16:30, 7 April 2019 (UTC) Eihel (talk) 15:13, 19 June 2019 (UTC) NAH (talk) 20:29, 18 August 2019 (UTC) Iwan.Aucamp (talk) 21:48, 3 October 2019 (UTC) Epìdosis (talk) 23:49, 22 November 2019 (UTC) Sotho Tal Ker (talk) 00:52, 1 May 2020 (UTC) Bargioni (talk) 09:48, 02 May 2020 (UTC) --Carlobia (talk) 14:34, 11 May 2020 (UTC)

Pictogram voting comment.svg Notified participants of WikiProject Authority control: I need your help!

User:Zolo
Jane023 (talk) 08:50, 30 May 2013 (UTC)
User:Vincent Steenberg
User:Kippelboy
User:Shonagon
Marsupium (talk) 13:46, 18 October 2013 (UTC)
GautierPoupeau (talk) 16:55, 9 January 2014 (UTC)
Multichill (talk) 19:13, 8 July 2014 (UTC)
Susannaanas (talk) 11:32, 12 August 2014 (UTC) I want to synchronize the handling of maps with this initiative
Mushroom (talk) 00:10, 24 August 2014 (UTC)
Jheald (talk) 17:09, 9 September 2014 (UTC)
Spinster (talk) 15:16, 12 September 2014 (UTC)
PKM (talk) 21:16, 8 October 2014 (UTC)
Vladimir Alexiev (talk) 17:12, 7 January 2015‎ (UTC)
Sic19 (talk) 21:12, 19 February 2016 (UTC)
Wittylama (talk) 13:13, 22 February 2017 (UTC)
Armineaghayan (talk) 08:40, 10 March 2017 (UTC)
Musedata102 (talk) 20:27, 26 November 2019 (UTC) Hannolans (talk) 18:36, 16 April 2017 (UTC)
User:Martingggg
Zeroth (talk) 02:21, 4 June 2018 (UTC)
User:7samurais
User:mrtngrsbch
User:Buccalon
Infopetal (talk) 17:54, 9 August 2019 (UTC)
S.ann.adams (talk) 21:05, 13 December 2019 (UTC)
Karinanw (talk) 16:38, 24 March 2020‎ (UTC)
Pictogram voting comment.svg Notified participants of WikiProject Visual arts: Yours too!

  • Do some checks (add your initials in column "check")
  • Add Q numbers to the sheet
  • Merge WD items that already have Getty AAT ID (P1014) (there are 8477) to the sheet to compare the matches (or remove them from the sheet if you're quite confident)

I could post the sheet as QuickStatements, but I think there are still 10% incorrect matches, especially for Styles and Periods (see Wikidata talk:WikiProject Visual arts/Item structure/Art movements. --Vladimir Alexiev (talk) 16:02, 7 March 2017 (UTC)

AAT-LCSH coreferencing[edit]

445 AAT-LCSH coreferences made by Getty editors.

400 of them are on the Getty LOD site (see query below), 45 are newly extracted

select * {
  ?x skos:exactMatch|skos:closeMatch ?y.
  ?x skos:inScheme aat:
  filter not exists {?y skos:inScheme aat:}}

Geonames Feature Code[edit]

T.seppelt (talk) 21:00, 18 February 2016 (UTC) Vladimir Alexiev (talk) 11:59, 13 March 2017 (UTC) GerardM (talk) 15:58, 26 March 2017 (UTC) Jonathan Groß (talk) 17:52, 26 March 2017 (UTC) Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits Jneubert (talk) 13:47, 29 April 2017 (UTC) Framawiki (please notify !) (talk) Sic19 (talk) 20:42, 12 July 2017 (UTC) Wikidelo (talk) 21:15, 8 May 2018 (UTC) salgo60 Salgo60 (talk) 07:09, 10 June 2018 (UTC) ArthurPSmith (talk) 19:52, 22 August 2018 (UTC) PKM (talk) 19:40, 23 August 2018 (UTC) Ettorerizza (talk) 06:44, 8 October 2018 (UTC) Fuzheado (talk) 03:47, 19 December 2018 (UTC) Daniel Mietchen (talk) 16:30, 7 April 2019 (UTC) Eihel (talk) 15:13, 19 June 2019 (UTC) NAH (talk) 20:29, 18 August 2019 (UTC) Iwan.Aucamp (talk) 21:48, 3 October 2019 (UTC) Epìdosis (talk) 23:49, 22 November 2019 (UTC) Sotho Tal Ker (talk) 00:52, 1 May 2020 (UTC) Bargioni (talk) 09:48, 02 May 2020 (UTC) --Carlobia (talk) 14:34, 11 May 2020 (UTC)

Pictogram voting comment.svg Notified participants of WikiProject Authority control Franzsimon Kopiersperre Jklamo ArthurPSmith S.K. Givegivetake fnielsen rjlabs ChristianKl Vladimir Alexiev User:Pintoch Parikan User:Cardinha00 User:zuphilip MB-one User:Simonmarch User:Jneubert Mathieudu68 User:Kippelboy User:Datawiki30 User:PKM User:RollTide882071 Kristbaum Andber08 Sidpark SilentSpike Susanna Ånäs (Susannaanas) User:Johanricher


Pictogram voting comment.svg Notified participants of WikiProject Companies Beat Estermann Vladimir Alexiev Ilya Sadads Astinson Strakhov Zeromonk Spinster Wittylama Daniel Mietchen Susannaanas Sic19 Jason.nlw Carlojoseph14 YULdigitalpreservation MB-one Ouvrard MartinPoulter Missvain VIGNERON Ainali Birk Weiberg Pmt Mauricio V. Genta Smallison ProtoplasmaKid 2le2im-bdc Rodrigo Tetsuo Argenton Ivanhercaz VisbyStar Patafisik Beireke1 Vahur Puik Ettorerizza Sp!ros Alexmar983 Epìdosis Buccalon Mrtngrsbch Eothan Giaccai NAH User:Fralambert Ipoellet

Pictogram voting comment.svg Notified participants of WikiProject Cultural heritage User:Zolo
Jane023 (talk) 08:50, 30 May 2013 (UTC)
User:Vincent Steenberg
User:Kippelboy
User:Shonagon
Marsupium (talk) 13:46, 18 October 2013 (UTC)
GautierPoupeau (talk) 16:55, 9 January 2014 (UTC)
Multichill (talk) 19:13, 8 July 2014 (UTC)
Susannaanas (talk) 11:32, 12 August 2014 (UTC) I want to synchronize the handling of maps with this initiative
Mushroom (talk) 00:10, 24 August 2014 (UTC)
Jheald (talk) 17:09, 9 September 2014 (UTC)
Spinster (talk) 15:16, 12 September 2014 (UTC)
PKM (talk) 21:16, 8 October 2014 (UTC)
Vladimir Alexiev (talk) 17:12, 7 January 2015‎ (UTC)
Sic19 (talk) 21:12, 19 February 2016 (UTC)
Wittylama (talk) 13:13, 22 February 2017 (UTC)
Armineaghayan (talk) 08:40, 10 March 2017 (UTC)
Musedata102 (talk) 20:27, 26 November 2019 (UTC) Hannolans (talk) 18:36, 16 April 2017 (UTC)
User:Martingggg
Zeroth (talk) 02:21, 4 June 2018 (UTC)
User:7samurais
User:mrtngrsbch
User:Buccalon
Infopetal (talk) 17:54, 9 August 2019 (UTC)
S.ann.adams (talk) 21:05, 13 December 2019 (UTC)
Karinanw (talk) 16:38, 24 March 2020‎ (UTC)
Pictogram voting comment.svg Notified participants of WikiProject Visual arts


GeoNames feature code (P2452) is applied only 33 times (see disscussion and archive, while there are 669 codes on Geonames. I'll ask Magnus to add the Geonames list to Mix-n-Match.

http://www.geonames.org/ontology/mappings_v3.01.rdf has the following mappings:

     32 dbo    http://dbpedia.org/ontology/
      5 frgeo  http://rdf.insee.fr/geo/
     79 lgdo   http://linkedgeodata.org/ontology/
     31 schema http://schema.org/

Can we use them somehow to push this coreferencing further?

I'm a little wary about importing these wholescale, because geonames in not a CC0 database. It's one thing to be providing external links to GeoNames, it's another to be importing data.
I did look at these values recently for English places with geonames links that are marked as both village (Q532) and civil parish (Q1115575) (see eg Abberton (Q3137539) for an example), to identify which Geonames link corresponded to which role; but I purposely decided not to add a GeoNames feature code (P2452) statement.
It might be useful to be able to map the codes to Q-numbers here, to facilitate sanity checking of exisiting or proposed co-references. However, even then there are difficulties -- for example, I found that PPLA3 or PPLA4 at Geonames didn't necessarily match to distinctions we would want to make in a instance of (P31) here. Jheald (talk) 14:02, 13 March 2017 (UTC)

Supplement Wikidata items with properties from authorities (GND in particular)[edit]

Data from DifferentiatedPersons of GND can be used to fill missing properties of according items, e.g.,

  • date of birth/death (directly from gnd:dateOfBirth and gndo:dateOfDeath, for entries following YYYY or YYYY-MM-DD - everything else to be skipped)
  • affiliation (P1416) can be obtained by a join of gndo:affiliation to wd organizations (may be sparse currently, but can be repeated later on)
  • country (country (P17) or country of citizenship (P27)??) requires translation from gndo:geographicAreaCode, which refers to a customized code table derived from ISO 3166 (not part of GND) (table (pdf), rules)
  • aliases - require filtering of gndo:variantNameForThePerson, which carry no language tag, re. script and presumed language (would Lingua::Identify work here?)

For appropriate source statements see project chat -- Jneubert (talk) 06:35, 21 May 2017 (UTC) (with thanks to User:MisterSynergy and User:ChristianKl)

I am a big fan of standards but the ISO 3166 is used for modern countries, it does not as a consequence give the "nationality" of people who did precede a country. Thanks, GerardM (talk) 07:03, 21 May 2017 (UTC)
NB yes there are some, but at Wikidata we know about many more former countries. GerardM (talk) 07:13, 21 May 2017 (UTC)

A "sibling" of Mix-n-Match now imports birth/death dates from authority files: https://www.wikidata.org/wiki/User:Magnus_Manske/Mix%27n%27match_date_import. Also see discussion about this in relation to Getty ULAN: https://groups.google.com/forum/#!topic/gettyvocablod/TkdelW9RP1g --Vladimir Alexiev (talk) 09:38, 2 October 2017 (UTC)

Property proposal for applying SKOS mapping relations to "external identifiers"[edit]

In order be able to map a thesaurus more completely, and - more general - to make Wikidata fit as a linking hub for knowledge organiziation systems, I've proposed a new property which allows to qualify individual links by properties of type "external identifier" as in-exact (close/broad/narrow/related) match.

Please feel free to comment at https://www.wikidata.org/wiki/Wikidata:Property_proposal/mapping_relation_type.

Cheers, Jneubert (talk) 12:27, 28 August 2017 (UTC)

Grant proposal soweego[edit]

T.seppelt (talk) 21:00, 18 February 2016 (UTC) Vladimir Alexiev (talk) 11:59, 13 March 2017 (UTC) GerardM (talk) 15:58, 26 March 2017 (UTC) Jonathan Groß (talk) 17:52, 26 March 2017 (UTC) Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits Jneubert (talk) 13:47, 29 April 2017 (UTC) Framawiki (please notify !) (talk) Sic19 (talk) 20:42, 12 July 2017 (UTC) Wikidelo (talk) 21:15, 8 May 2018 (UTC) salgo60 Salgo60 (talk) 07:09, 10 June 2018 (UTC) ArthurPSmith (talk) 19:52, 22 August 2018 (UTC) PKM (talk) 19:40, 23 August 2018 (UTC) Ettorerizza (talk) 06:44, 8 October 2018 (UTC) Fuzheado (talk) 03:47, 19 December 2018 (UTC) Daniel Mietchen (talk) 16:30, 7 April 2019 (UTC) Eihel (talk) 15:13, 19 June 2019 (UTC) NAH (talk) 20:29, 18 August 2019 (UTC) Iwan.Aucamp (talk) 21:48, 3 October 2019 (UTC) Epìdosis (talk) 23:49, 22 November 2019 (UTC) Sotho Tal Ker (talk) 00:52, 1 May 2020 (UTC) Bargioni (talk) 09:48, 02 May 2020 (UTC) --Carlobia (talk) 14:34, 11 May 2020 (UTC)

Pictogram voting comment.svg Notified participants of WikiProject Authority control

There is a new grant proposal soweego for authority control. See discussion at https://meta.wikimedia.org/wiki/Grants:Project/Hjfocs/soweego#Endorsements.

I've considered it seriously but I think it doesn't address the main problem, see a list of 11 considerations (which can also be read as a sort of programme for next important steps for WD authority control). Please express your opinion there. --Vladimir Alexiev (talk) 10:19, 2 October 2017 (UTC)

Where is this discussion with 11 considerations? Thanks, GerardM (talk) 12:22, 2 October 2017 (UTC)

Remove Obsolete Getty Vocabularies IDs[edit]

Getty Vocabs have 8k obsolete subjects (identifiers). You can find them at http://vocab.getty.edu/sparql with a query like

select * {
  ?old a gvp:ObsoleteSubject; skos:inScheme aat: .
  optional {?old dct:isReplacedBy ?new}
}

Here are the numbers: I can put up the files somewhere: obsolete-AAT-2106.tsv, obsolete-TGN-1016.tsv, obsolete-ULAN-5574.tsv.

T.seppelt (talk) 21:00, 18 February 2016 (UTC) Vladimir Alexiev (talk) 11:59, 13 March 2017 (UTC) GerardM (talk) 15:58, 26 March 2017 (UTC) Jonathan Groß (talk) 17:52, 26 March 2017 (UTC) Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits Jneubert (talk) 13:47, 29 April 2017 (UTC) Framawiki (please notify !) (talk) Sic19 (talk) 20:42, 12 July 2017 (UTC) Wikidelo (talk) 21:15, 8 May 2018 (UTC) salgo60 Salgo60 (talk) 07:09, 10 June 2018 (UTC) ArthurPSmith (talk) 19:52, 22 August 2018 (UTC) PKM (talk) 19:40, 23 August 2018 (UTC) Ettorerizza (talk) 06:44, 8 October 2018 (UTC) Fuzheado (talk) 03:47, 19 December 2018 (UTC) Daniel Mietchen (talk) 16:30, 7 April 2019 (UTC) Eihel (talk) 15:13, 19 June 2019 (UTC) NAH (talk) 20:29, 18 August 2019 (UTC) Iwan.Aucamp (talk) 21:48, 3 October 2019 (UTC) Epìdosis (talk) 23:49, 22 November 2019 (UTC) Sotho Tal Ker (talk) 00:52, 1 May 2020 (UTC) Bargioni (talk) 09:48, 02 May 2020 (UTC) --Carlobia (talk) 14:34, 11 May 2020 (UTC)

Pictogram voting comment.svg Notified participants of WikiProject Authority control Any takers to replace any old values of AAT ID search, TGN ID search, ULAN ID (P245) View with SQID with the new values from the respective files?

@Magnus Manske: rdf:type gvp:ObsoleteSubject should be removed from Mix-n-Match consideration.

Done, but only changed 5 entries for the ~2100 obsolete ones, the others were either already N/A or matched with an item. --Magnus Manske (talk) 15:22, 21 February 2018 (UTC)
Data should not be removed, but - if necessary - marked as deprecated, with a qualifier giving the reason for deprecation. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 11:02, 26 February 2018 (UTC)

Swedish National Library moves direction Linked data[edit]

Monday june 11 the Swedish National Library is moving in direction Linked data and release a new system called LIBRIS XL ==>

  1. We have a new identifier LIBRIS-URI Libris-URI (P5587) in Wikidata

Open issues are if we need to "upgraded" to support the new identifier in other systems. I have found

  1. User:Magnus_Manske/authority_control.js
    1. tool to read from VIAF.org and populate WD
  2. User:Magnus_Manske/Mix'n'match_date_import
  3. wdmapper

See also:

- Salgo60 (talk) 14:07, 10 June 2018 (UTC)

Status update 16 September 2018
A project wmse-library-data-2018 done by Alicia_Fagerving_(WMSE) (talkcontribslogs) is converting and adding about 60 000 identifiers to Wikidata....
I sent in today a question to VIAF asking about plans they have to start supporting Libris-URI (P5587) - Salgo60 (talk) 06:54, 16 September 2018 (UTC)
Status update 28 September 2019
  • Still no one has specified how LIBRIS <-> VIAF should work together see Phabricator T223259
  • We lack a change process for errors in VIAF and LIBRIS see example of error Talk:Q21522286#Summary
- Salgo60 (talk) 18:32, 28 September 2019 (UTC)

Introducing mapping relation type (P4390) for full SKOS mappings[edit]

For knowledge organization systems which do not deal with clearly defined entities (such as human (Q5)), you often find in-exact relations between the external entity and matching Wikidata items. E.g., the STW concept Yugoslavia (until 1990) is not an exact match to Yugoslavia (Q36704), which is described as "1918–1992 country in Southeastern and Central Europe". For STW's Executive selection, Assessment center (Q265558) is closely related, but categorically different (process vs. instrument). While in the latter case it might be useful to create an exactly matching item, in the former it clearly would be not.

Within the domain of traditional knowledge organization system, different mapping relations (such as "close match") have been used to cover such situations. The "external id" properties of Wikidata lacked such expressiveness, till the mapping relation type (P4390) property was introduced. The property, to be used exclusively as a qualifier on external-id properties, allows to more precisely define relations as exact match (Q39893449), close match (Q39893184), broad match (Q39894595), narrow match (Q39893967) or related match (Q39894604). These relation types reflect the according SKOS mapping properties.

Since its introduction in October last year, the property has seen some uptake, particularly in the biomedial field. The ongoing mapping of STW Thesaurus for Economics (Q26903352) to Wikidata is based on qualified relations (state of the mapping, shown as SKOS). The effort aims at creating a finally complete mapping of the STW descriptors.

Workflow for the mapping process[edit]

Our - still experimental - workflow for the STW mapping is as follows:

  1. Use Mix-n-match catalogs for each sub-thesaurus (#507, #1259/#1260) to assign STW descriptor IDs one-by-one to Wikidata items.
    This sounds simple, yet often reveals qualitiy issues. Some of these, such as obvious duplicates in Wikidata, can be resolved immediately. Sometimes an ugly mess shows up, which can only be reported to be solved by the community later on. Of course, quality flaws in STW might be revealed also. During this step, it may also be advisable to take notes about other items, which are not the closest ones and not linked via external-id entry in Wikidata, but may be worth linking from the side of the external vocabulary.
    • For non-exact relationships, immediately open the newly linked item and manually qualify STW Thesaurus for Economics ID (P3911).
    • For STW descriptors which are lacking a counterpart in Wikidata and would make sense there, add an item semi-automatically, with exact match (Q39893449) to the STW ID and all avaialable information from the thesaurus (more on that later).
  2. Assign exact match (Q39893449) to the remaining unqualified STW Thesaurus for Economics ID (P3911) via Quickstatements, with the input produced by a script executing a SPARQL query. This step can be executed multiple times during the mapping process for one sub-thesaurus, in order to keep the list of unqualified entries short.

The sequence within the Mix-n-match input file turned out to be crucial for a smooth one-by-one workflow. We sorted the generated M-n-m input by the minimal notation of attached subject categories for the descriptors, and within that alphabetically.

Quality control for mappings using mapping relation types[edit]

Maintenance and qualitiy control on a mapping have to take into account that multiple external-id values for a Wikidata item, or one external-id linked to multiple Wikidata items are possible and may perfectly make sense with in-exact mapping relations (e.g., STWs Appenzell is a "broad match" to Appenzell Ausserrhoden (Q12079) and Appenzell Innerrhoden (Q12094)). This is not reflected in the "single value" and "distinct values" constraints. Therefore, we defined a number of QS reports for the STW mapping to catch anomalies specifically in qualified mappings.

Feedback welcome[edit]

We are interested to exchange experiences with others who are mapping KOS to Wikidata, possibly with different workflows or other tools used. The reports and scripts linked above are meant to be customizable, we'd be happy to receive suggestions for improvment or github pull requests. -- Jneubert (talk) 15:09, 29 August 2018 (UTC)

JakobVoss (talk) Vladimir Alexiev (talk) 09:27, 5 July 2016 (UTC) TomT0m author  TomT0m / talk page 20:15, 21 September 2016 (UTC) Jneubert (talk) 09:22, 29 January 2018 (UTC) Ettorerizza (talk) 11:00, 26 September 2018 (UTC) ElanHR (talk) 10:01, 17 March 2019 (UTC) Dcflyer (talk) 19:46, 29 March 2019 (UTC)


Pictogram voting comment.svg Notified participants of WikiProject KOS

Item creation from a thesaurus concept via Quickstatements[edit]

During the above mentioned mapping process from STW Thesaurus for Economics (Q26903352), we sometimes want to create new items in Wikidata. With the "New item" button in Mix-n-match, only rudimentary information can be transferred to the new item. Therefore we generated a list of all not-yet-mapped STW descriptors, formatted for Quickstatements input. It includes labels and aliases (skos:preLabel/skos:altLabel) in all available languages, as well an instance-of economic concept (Q29028649) statement, sourced from the STW descriptor, optionally a link to the according GND concept, derived from the STW/GND mapping, rarely a description (skos:scopeNote), but always of course a STW Thesaurus for Economics ID (P3911) link.

The workflow is - during working through a mix-n-match list - for any missing concept simply to copy & paste the complete set of QS statements into the QS input window, removing aliases which are not appropriate for Wikidata (such as "oil platform" for "offshore industry"), and running the statements. The list is sorted exactly like the mix-n-match list and recreated every hour, so the same items are on top of both lists, and every case solved by either linking or creating an item disappears automatically from the list. In our experiences, this works quite smoothly.

If others want to adapt such a workflow for other vocabularies - thanks to SKOS and LOD standards that shouldn't be too difficult -, here is the script for generating the list, and the query called with it. -- Feedback, as always, welcome. Jneubert (talk) 12:11, 5 September 2018 (UTC)

History[edit]

Please add here references, blogs etc on the topic.

https://twitter.com/hashtag/coreferencing: tweet using tag #coreferencing. Tweets on involving Getty, British Museum thesauri, some fancy shots...

Participants[edit]

T.seppelt (talk) 21:00, 18 February 2016 (UTC) Vladimir Alexiev (talk) 11:59, 13 March 2017 (UTC) GerardM (talk) 15:58, 26 March 2017 (UTC) Jonathan Groß (talk) 17:52, 26 March 2017 (UTC) Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits Jneubert (talk) 13:47, 29 April 2017 (UTC) Framawiki (please notify !) (talk) Sic19 (talk) 20:42, 12 July 2017 (UTC) Wikidelo (talk) 21:15, 8 May 2018 (UTC) salgo60 Salgo60 (talk) 07:09, 10 June 2018 (UTC) ArthurPSmith (talk) 19:52, 22 August 2018 (UTC) PKM (talk) 19:40, 23 August 2018 (UTC) Ettorerizza (talk) 06:44, 8 October 2018 (UTC) Fuzheado (talk) 03:47, 19 December 2018 (UTC) Daniel Mietchen (talk) 16:30, 7 April 2019 (UTC) Eihel (talk) 15:13, 19 June 2019 (UTC) NAH (talk) 20:29, 18 August 2019 (UTC) Iwan.Aucamp (talk) 21:48, 3 October 2019 (UTC) Epìdosis (talk) 23:49, 22 November 2019 (UTC) Sotho Tal Ker (talk) 00:52, 1 May 2020 (UTC) Bargioni (talk) 09:48, 02 May 2020 (UTC) --Carlobia (talk) 14:34, 11 May 2020 (UTC)

Pictogram voting comment.svg Notified participants of WikiProject Authority control Please become members of this project!


[+ Add yourself to the list]

The participants listed below can be notified using the following template in discussions:

{{Ping project|Authority control}}

Related projects[edit]