Wikidata:Property proposal/earliest known OpenTimestamps

earliest known OpenTimestamps

Originally proposed at Wikidata:Property proposal/Commons

   Not done

Description	Earliest known OpenTimestamps proof of an image, video or audio file
Data type	String
Domain	OpenTimestamps proof (Q105855601)
Allowed values	base64. Regex: [^-A-Za-z0-9+/=]
Example 1	File:Monumento_en_recuerdo_de_las_víctimas_de_la_pandemia_del_Covid-19_(Madrid),_18_May_2020.jpg → AE9wZW5UaW1lc3RhbXBzAABQcm9vZgC/ieLohOiSlAEIqmhCuD/Hf4ZtJzsAHTuQh9y0nPkClhDz 1fdeopuI7h7wENd17BixRGTBvCRgkX/EAkcI8CBa+w41aQZeT67MxV8a6yzLwM5+CGD1Ulj/ab4u 4EkpCAjxIB+NHI1sm5WX8+US70Xgx4X8thEd5QPvpmRH4pwLn+qECPEgwZerXDX3dIzm2TFEDW45 Q3AtCOochAehg/J5ogyYtdgI8CA8SlvQWaw7khoCbhoQtd0+PFDMpT8t/fGo5tMn6Ris4wjwIHmO KaNIQi3LwWhGADlVG/sN7sD6GNXtPN6jI5aqbOFMCPAg9QE6QOmANEFop4w5S6YGF8pBqOcbRxit gwwxawlIm9AI8BCKpKkqvs82FFUY+pbH+c/kCPEEXsLzg/AI5DlgRjbBYDcI8SCcKPfc5sG8TJjQ Xp+kcNV/yIbcJ1TaPKuJltI1e1EdewjxIIywgDtDEexxANU6/hRjxfLVPyWpBRp9JKYBzypMvmtA CPEgh1kQ6Urwy6GpyR2Wa+53Emr8AvbXwmotNVh74fEhgYEI8CA1uwbzIusvFmTWqlf/9tg0ih3n p589opkQ/mz6Fmr0hAjxIFEwStiOuUVBzNWZXsaYM7h8rcJ56reJ9V06Ku9Wq38PCPAg5OygZlVq +00XtW3f3VyJhj7Qh7E8+sNrzshYgNw7KgMI8SDGNzqPE1+icnk1NRdBzNytsMguZAFgvogKDu5P 7CIP1AjxIN4HfpyjWeEWG4pnwXtxsEDa+RAx4Vi/mNaU8u0Hts+bCPEg6e5CofTGfhHfpM7RGXPE QjNY5Fw0CFTuCT3S5MN/WmcI8SB9I6WjLXjPYThqGqC/elvtbJu3t3gRw35gWp8jQRGNiwjxII9e CBzL40YnNfzyn4ORhBCFjpAwgtymAbK+kVsup6jSCPEgcD7U8WAEmftpJcH4Yc/5csto8TCtILdv 65lWDjwqoCAI8CAL8o45rswne7cZbmrrXg3MQP565Ryn/KVoDUhzqduBkAjxICsjI+9/MlqYf2D7 2NMhUS9GIT0H5lB9J2voTDva7ZsJCPAgYHvLDgmDgzmITM8SapnzS1oGOwpZidYbHrkdy+ElXoUI 8XEBAAAAAc+QsL9Yqo4aUWGF5T9lojorCl42C7EaS9XwbWQ9XJ0IAAAAABcWABQ/ZZ3/jRuZ0NSz EGQUN0dZQ89VRv3///8CAUYEAAAAAAAXqRQhsSNjOkrjMU/sC+TiStQnBwbTHocAAAAAAAAAACJq IPAEbKAJAAgI8CCvRyFW9QMGdd7kHReE0xQgG4/78U6FStJ51o25YB/1IwgI8SAQx2gEme6PKezd cePDHRV4oLSSgOjyG8jZDw/0S6nnZwgI8SB5Swqf8QwWAJfuX2Z3XrTQKAfnFFKTwyny51VYIxQy bQgI8CC/oEBbCu2Qe76XpB+Wg4NL/0n21GxA4HHRCQe10PhhBwgI8SCLhMxRar7DRnPmaBz/GSxP g5pppSXWVPzmbj46Jk7+QAgI8CC1MpoACmclmjs4F+NHHA1MGme/Io8mPYw+2yDlOhobkggI8SA3 edryoUc9SQw62Z7+u53TtK5ztEnGInUjWqCa/IZwLggI8SCt3iW9UHON4S11+6FdPMl5bIByFfoA OCfA/UYooNz37wgI8SDR+EV5h9mQHeZJXDGy0MwB0v9TMUtozb4N7NnCcLTUAQgI8SAG3Tj7sWdc Og8BR3TeOFdWggcP7H3wrDRtVm5wEXhAUQgI8SDit/ppsbaYVBNs1c1OlOJHVK+NrjktRaReuWDr QGvlnQgI8CB2PhvEnsOMgwvG7bzA1rR4q9+UFxzaIs8BXViUdPFeBQgIAAWIlg1z1xkBA+3AJg==
Example 2	MISSING
Example 3	MISSING

Motivation

The timestamp in the Exif metadata of an image file is only as accurate as the clock in the camera, and it may be completely wrong. Moreover, it can be easily faked with a software app. The OpenTimestamps proof can not be faked: it guarantees a file existed at a specific time or earlier. This may be critical for historically relevant images and for new original discoveries.

It can also be applied to any other file format, including video and audio files.

OpenTimestamps is open source, scalable, permissionless and free to timestamp and to verify. FrankAndProust (talk) 19:06, 9 June 2021 (UTC)

Discussion

The size of a typical pruned OpenTimestamps proof is around 1500 bytes in base64 format (String). This applies to any file, no matter how big.

The image on Example 1 was uploaded to Wikimedia Commons on 8th June 2021. The uploader might find difficult to convince the community that the image is of a much earlier date, specifically from 18th May 2020 because Exif data are easy to fake. However, the base64 representation of the OpenTimestamps timestamp, irrefutably proofs the image is from date 19th May 2020 or earlier. --FrankAndProust (talk) 19:06, 9 June 2021 (UTC)

 Comment @FrankAndProust: Can you check whether these strings fit within the limits of Wikidata string type? It looks to me like it will be too long. You can test this on test.wikidata.org - create a new property there for this and try it out. ArthurPSmith (talk) 17:34, 10 June 2021 (UTC)

 Comment @ArthurPSmith: You are right. This OpenTimestamps (OTS) proof is slightly longer than the maximum length constraint for String on Commons (1500 characters) when encoded in base64.

It would be possible to bypass the maximum length constraint if we used the "Tabular data" datatype, but that would need a cluttered hack so I consider it would not be appropriate.

Do you feel there is a suitable way of embedding the OTS proof on a property? If not, I will relinquish this property proposal and open a new one with URL as the suggested data type. --FrankAndProust (talk) 09:05, 11 June 2021 (UTC)

I suppose the timestamp is itself a file, but it's not one of the formats that Commons accepts - I guess it could be done with a "cluttered hack" approach as you suggest! Other than that, do you think the more compact en:Ascii85 format rather than base64 would work here? ArthurPSmith (talk) 12:22, 11 June 2021 (UTC)

Ascii85 provides some savings but it still falls short of the needs.

A pruned OTS proof can be consistently reflected on a tabular data datatype: a three column table is needed; the first column is the execution order, the second column is the command (mandatory), the third column is the argument (non-mandatory). Commands are executed sequentially from first to last to generate the proof. The exact replica of the the previously stated binary proof encoded as base64 as tabular data would be:

JPEG file sha256 hash (initial value): aa6842b83fc77f866d273b001d3b9087dcb49cf9029610f3d5f75ea29b88ee1e

Execution order	Command (mandatory)	Argument (non-mandatory)
1	append	d775ec18b14464c1bc2460917fc40247
2	sha256
3	prepend	1f8d1c8d6c9b9597f3e512ef45e0c785fcb6111de503efa66447e29c0b9fea84
4	sha256
5	prepend	c197ab5c35f7748ce6d931440d6e3943702d08ea1c8407a183f279a20c98b5d8
6	sha256
7	append	3c4a5bd059ac3b921a026e1a10b5dd3e3c50cca53f2dfdf1a8e6d327e918ace3
8	sha256
9	append	798e29a348422dcbc168460039551bfb0deec0fa18d5ed3cdea32396aa6ce14c
10	sha256
11	append	f5013a40e980344168a78c394ba60617ca41a8e71b4718ad830c316b09489bd0
12	sha256
13	append	8aa4a92abecf36145518fa96c7f9cfe4
14	sha256
15	prepend	5ec2f383
16	append	e439604636c16037
17	sha256
18	prepend	9c28f7dce6c1bc4c98d05e9fa470d57fc886dc2754da3cab8996d2357b511d7b
19	sha256
20	prepend	8cb0803b4311ec7100d53afe1463c5f2d53f25a9051a7d24a601cf2a4cbe6b40
21	sha256
22	prepend	875910e94af0cba1a9c91d966bee77126afc02f6d7c26a2d35587be1f1218181
23	sha256
24	append	35bb06f322eb2f1664d6aa57fff6d8348a1de7a79f3da29910fe6cfa166af484
25	sha256
26	prepend	51304ad88eb94541ccd5995ec69833b87cadc279eab789f55d3a2aef56ab7f0f
27	sha256
28	append	e4eca066556afb4d17b56ddfdd5c89863ed087b13cfac36bcec85880dc3b2a03
29	sha256
30	prepend	c6373a8f135fa2727935351741ccdcadb0c82e640160be880a0eee4fec220fd4
31	sha256
32	prepend	de077e9ca359e1161b8a67c17b71b040daf91031e158bf98d694f2ed07b6cf9b
33	sha256
34	prepend	e9ee42a1f4c67e11dfa4ced11973c4423358e45c340854ee093dd2e4c37f5a67
35	sha256
36	prepend	7d23a5a32d78cf61386a1aa0bf7a5bed6c9bb7b77811c37e605a9f2341118d8b
37	sha256
38	prepend	8f5e081ccbe3462735fcf29f83918410858e903082dca601b2be915b2ea7a8d2
39	sha256
40	prepend	703ed4f1600499fb6925c1f861cff972cb68f130ad20b76feb99560e3c2aa020
41	sha256
42	append	0bf28e39aecc277bb7196e6aeb5e0dcc40fe7ae51ca7fca5680d4873a9db8190
43	sha256
44	prepend	2b2323ef7f325a987f60fbd8d321512f46213d07e6507d276be84c3bdaed9b09
45	sha256
46	append	607bcb0e09838339884ccf126a99f34b5a063b0a5989d61b1eb91dcbe1255e85
47	sha256
48	prepend	0100000001cf90b0bf58aa8e1a516185e53f65a23a2b0a5e360bb11a4bd5f06d643d5c9d0800000000171600143f659dff8d1b99d0d4b310641437475943cf5546fdffffff02014604000000000017a91421b123633a4ae3314fec0be4e24ad4270706d31e870000000000000000226a20
49	append	6ca00900
50	sha256
51	sha256
52	append	af472156f5030675dee41d1784d314201b8ffbf14e854ad279d68db9601ff523
53	sha256
54	sha256
55	prepend	10c7680499ee8f29ecdd71e3c31d1578a0b49280e8f21bc8d90f0ff44ba9e767
56	sha256
57	sha256
58	prepend	794b0a9ff10c160097ee5f66775eb4d02807e7145293c329f2e755582314326d
59	sha256
60	sha256
61	append	bfa0405b0aed907bbe97a41f9683834bff49f6d46c40e071d10907b5d0f86107
62	sha256
63	sha256
64	prepend	8b84cc516abec34673e6681cff192c4f839a69a525d654fce66e3e3a264efe40
65	sha256
66	sha256
67	append	b5329a000a67259a3b3817e3471c0d4c1a67bf228f263d8c3edb20e53a1a1b92
68	sha256
69	sha256
70	prepend	3779daf2a1473d490c3ad99efebb9dd3b4ae73b449c62275235aa09afc86702e
71	sha256
72	sha256
73	prepend	adde25bd50738de12d75fba15d3cc9796c807215fa003827c0fd4628a0dcf7ef
74	sha256
75	sha256
76	prepend	d1f8457987d9901de6495c31b2d0cc01d2ff53314b68cdbe0decd9c270b4d401
77	sha256
78	sha256
79	prepend	06dd38fbb1675c3a0f014774de38575682070fec7df0ac346d566e7011784051
80	sha256
81	sha256
82	prepend	e2b7fa69b1b69854136cd5cd4e94e24754af8dae392d45a45eb960eb406be59d
83	sha256
84	sha256
85	append	763e1bc49ec38c830bc6edbcc0d6b478abdf94171cda22cf015d589474f15e05
86	sha256
87	sha256

The great advantage of this format is the fact that the proof is embedded on Wikimedia Commons: it is always available for everybody at any time.

The great disadvantage is the fact that the OpenTimestamps client does not understand this format, so currently it cannot be directly used as an input to the OTS validator.

This is why I feel the correct solution, at least as a first step, would be using a URL data type to a OTS timestamp file. OTS timestamps should consequently be hosted externally to the Wikimedia project. The main disadvantage is the fact the host can withdraw those files at any time and without previous notice, therefore becoming unavailable to Wikidata users. However, I don't feel there is a better solution.

So as I said, if no better suggestion is provided, I will create a new property proposal for "earliest known OpenTimestamps URL". --FrankAndProust (talk) 19:04, 11 June 2021 (UTC)

•  Comment I have no idea of the usefulness of the above data, but if Commons contributors want it stored as structured data, this is feasible.

1. In Wikibase, the length of string property is configurable. So it's technically possible that the SDC team increases it for the above.
2. Another aspect is the GUI. The value may not look that great in the GUI and aren't really useful for human readers. It may be worth to define a new sub-datatype of string values that just has some summary as displayed value and the full string value is only visible when querying or editing. There are already a few string-subtypes with special display. This would need to be requested from the SDC team as well.
3. At least in Wikidata, it's possible to convert between string datatypes, so it may be possible to start out with a string property while the new datatype is being developed. --- Jura 13:03, 13 June 2021 (UTC)

 Comment @Jura1: The solution you propose is the one I like the most: "a new sub-datatype of string values that just has some summary as displayed value and the full string value is only visible when querying or editing".

And, like you said, "it may be possible to start out with a string property while the new datatype is being developed". As we have already tested out, the standard constraint of 1500 characters is not enough if we use base64 encoding. Maybe we could start out first with a string property with a max length constraint of 3000 characters?

This is my implementation proposal. It can be "tuned" to better adapt to the current Wikimedia technical infrastructure:

1. The content of the OpenTimestamps file is copied on a text box encoded as base64 and stored on a sub-datatype of a string.
2. Wikidata calculates the SHA256 hash of the highest resolution image (or video, or audio).
3. Wikidata verifies the SHA256 hash at step 2 matches all the way up to the Merkle root on the OpenTimestamps file. This is done with the "verify" command. There are implementations of this command in Python, Java, Rust and Javascript at the OpenTimestamps Github.
4. Wikidata gets the date of the Bitcoin block at the top of the Merkle root and checks for its date and time. It is not necessary to enforce trustless programming here, it would make the process a little more cumbersome, so it would be easier to not use a Bitcoin node in this step and stick to more classical API programming. We would use APIs available publicly. For the image at this example, we would need these two API calls:

$ curl https://blockstream.info/api/block-height/630893
0000000000000000000964ee0c5d67c74783626a058a76105f77ce0469ef379c
$ curl https://blockstream.info/api/block/0000000000000000000964ee0c5d67c74783626a058a76105f77ce0469ef379c
{"id":"0000000000000000000964ee0c5d67c74783626a058a76105f77ce0469ef379c",
 "height":630893,
 "version":536870912,
 "timestamp":1589857180,
 "tx_count":2065,
 "size":1396834,
 "weight":3993379,
 "merkle_root":"71cb3d8de9d964953d5f1a0ee00deb650a16cc055bed2366881a4574617af828",
 "previousblockhash":"000000000000000000068e9a1a96347f7815cd6f0a4d8c9e1b45305a9afd8364",
 "mediantime":1589851896,
 "nonce":3231938334,
 "bits":387021369,
 "difficulty":16104807485529}

We take the "mediantime" value from the last JSON, which in this case is 1589851896. We treat it as Unix time and it gives us: "Tue May 19 2020 01:31:36 GMT+0000". This way it is proven that the example image existed that date or earlier.

5. The date obtained in the previous step is shown to the user. If the user presses a button somewhere, the content of the OTS file is made visible in base64 encoding.

n.b.: I am not affiliated with OpenTimestamps or blockstream.info .

I am a developer and I could freely help on my spare time if necessary.

What are your thoughts? --FrankAndProust (talk) 19:26, 14 June 2021 (UTC)

It is already more than two weeks with no responses. Would it be more practical to simply display the URL pointing to the external OpenTimestamps file? --FrankAndProust (talk) 09:40, 29 June 2021 (UTC)

 Oppose you should just build this as something separate from Commons. On Commons we have the upload history in which we can verify that something was uploaded at some point of time. That is enough for us. Bloating the Commons database for this is a bit too much. Multichill (talk) 16:58, 13 September 2021 (UTC)

I'm marking this as not done because the data it's intended to store is too long. If that changes, it could be revisited, but I would recommend only doing that if there is evidence that other people would use this because, right now, there are no files on Commons which even mention this, not even the example given here. - Nikki (talk) 00:24, 24 December 2021 (UTC)