Wikidata:ExtractTransformLoad

See Meta:Grants:Project/MFFUK/Wikidata_&_ETL for project page on Meta

Wikidata & ETL is a project funded by a Project Grant. Simply put, it tries to apply LinkedPipes ETL - a tool used to publish and consume Linked Data on the Web - to loading data from sources on the Web to Wikidata. The outputs are intended for volunteers who understand RDF, SPARQL, the Wikibase RDF dump format and want to mass import items and statements to a Wikibase instance such as Wikidata. This page contains outputs of that project.

Analysis and design

As the output of our Work Package 1 we created an analysis and requirements document.

Implementation

LinkedPipes ETL was dockerized and a loader component, l-wikibase, was developed, allowing pipeline designers to load data to Wikibase instances such as Wikidata.

Documentation

The developed component was documented and a tutorial demonstrating how the data loading pipelines using this component can be created was written.

Transformations

Proof of concept data loading pipelines were created and run.

• Data about Czech Remarkable Trees from the authoritative source, task has been approved and the pipeline can be imported in a LP-ETL instance and run.
• Data about Czech streets, task has been approved and the pipeline can be imported in a LP-ETL instance and run.
• Linking languages in Wikidata to languages in Language EU Vocabulary, task has been approved and the pipeline can be imported in a LP-ETL instance and run.

Communication

• Our approach has been discussed at the Wikimedia Hackathon 2019 and we received positive feedback
• It was later presented as a Poster representing the process of loading RDF data into Wikibases such as Wikidata at Wikimania 2019 and again we received positive feedback
• There was also a demo session at Wikimania 2019
• The tutorial contains also a tips & tricks section
• The Wikidata GLAM Facebook group was notified about the tutorial