Talk:Data items/Archive 1

From OpenStreetMap Wiki
Jump to navigation Jump to search
Talk:Wiki - Other languages
This is the talk page for discussing improvements to the Data items/Archive 1 article and its related topics.

English label?

What do the words “English label” in set English label to be the same as the key but without the "Key:" or "Tag:" prefixes mean? --Andrew (talk) 08:12, 18 September 2018 (UTC)

Andrew, for each item, wikibase has a label and a description for each language. For a key like highway, we have a wiki page Key:highway, so the sitelink (link in the upper right corner) will point to that. The label field will be the same but without the "Key:" -- so it will be set to "highway". Other languages should not be set at all. Maybe we should even hide the label field from the interface. --Yurik (talk) 08:24, 18 September 2018 (UTC)
Andrew, after thinking some more, the labeling restriction seems to be too silly and error prone. Feel free to change the labels in an internationally useful way, e.g. name:en -> English name. --Yurik (talk) 02:35, 20 September 2018 (UTC)

different prefix - Q complexity

> Key:bridge:movable: https://wiki.openstreetmap.org/wiki/Item:Q104

As I understand most of the osm keys/tags will be 2 Q prefixed ids. ( one for the wikidata and the other is for the osm wikibase )

example:

amenity=

aeroway=

Is it possible to change the prefix? ( now "Q"/"P" - for me better "M" or any other letter)

for me (as a human) it is very hard to work with similar id-s ( hard to debug, hard to understand, and a lot of confusion) ( my nightmare: debugging complex JSON file: inside a wikidata and an OSM wikibase id, and checking the Q and the P values )

--ImreSamu (talk) 11:36, 18 September 2018 (UTC)

ImreSamu, agree, I wrote the code, but Wikbase devs don't like it without offering any alternatives. See comments. If you can convince them, and raley support, it would be great. [1] --Yurik (talk) 13:00, 18 September 2018 (UTC)

Yurik, thanks - I see: https://phabricator.wikimedia.org/T202676 , I am not happy .. --ImreSamu (talk) 14:04, 18 September 2018 (UTC)

ImreSamu being unhappy doesn't change anything. Write to the devs, explain our position, get other people to do the same. --Yurik (talk) 18:19, 18 September 2018 (UTC)
One way to make this a non-issue would be to remove the Wikidata property from the database. Would the WMF developers really prefer us to do that?--Andrew (talk) 18:37, 18 September 2018 (UTC)
That sounds good to me! It will be quite confusing to have both wikidata and wikibase links. I'm the two terms mixed up all the time currently. I'm sure this will improve with more familiarity for myself, but it will happen again for ever single new user. --Jeisenbe (talk) 11:37, 4 August 2019 (UTC)

what is "primary tag storage for the OSM database"?

"This project's goal is NOT to replace the primary tag storage for the OSM database" - can someone tell me what is currently this primary tag storage? Is it about wiki containing tag description? Or something else Mateusz Konieczny (talk) 16:02, 18 September 2018 (UTC)

Currently tag metadata is stored in this wiki as a free text, and cannot be easily used by other tools like taginfo or iD editor. With this approach, we can store data in a structured way, usable to other tools. --Yurik (talk) 17:50, 18 September 2018 (UTC)

"that it is disallowed"

I am not sure is it intentional, but it read like attempt to be an attempt to forcefully deprecate at least some keys. AFAIK there is no consensus to do that Mateusz Konieczny (talk) 16:04, 18 September 2018 (UTC)

It may be a good idea to mention that nobody is forced or obligated to follow suggestions included in this database Mateusz Konieczny (talk) 16:06, 18 September 2018 (UTC)
it's a bit hard to forcefully deprecate anything in OSM :) This is an attempt to make sense of the data. If the data doesn't make sense to anyone, that data can be deleted as it is just noise that gets in a way. But this is not the primary focus ;) --Yurik (talk) 17:46, 18 September 2018 (UTC)
There are already deprecated tag wiki pages such as highway=incline.--Andrew (talk) 20:23, 18 September 2018 (UTC)

MW API

"use MW API to create new items" - can someone link this tool? I am unsure what is this (maybe MediaWiki API?) Mateusz Konieczny (talk) 16:05, 18 September 2018 (UTC)

Correct, MediaWiki API. The easiest is to use pywikibot. --Yurik (talk) 17:56, 18 September 2018 (UTC)

Link to create new item

Is there any easily accessible link for creating a new item? While it's possible to manually navigate to Special:NewItem, this will likely not be easy to discover for most wiki editors. Wikidata has a "Create new item" sidebar link for that purpose. --Tordanik 19:19, 18 September 2018 (UTC)

Tordanik, I don't think we should expose it the way Wikidata does. Most new items are created by a bot from TagInfo data, plus at some point I hope other tools like iD editor will create them when user enters a new key. Casual users should not be creating them. Once the project stabilizes, we should have documentation describing when how to create new items and what properties they should have. --Yurik (talk) 19:29, 18 September 2018 (UTC)
Importing data from Taginfo works for tagging-related items. But as I previously said on the forum, I feel the community should be able to use the Wikibase instance for purposes besides tagging documentation. --Tordanik 18:50, 20 September 2018 (UTC)
@Tordanik: the data is not being imported just from taginfo. Taginfo tells me how common is the key, and then i parse the KeyDescription template to extract image, description, used-on, and status (with the reference). I hope we will parse and add other things, e.g. the group and the used-with type of info. --Yurik (talk) 00:16, 21 September 2018 (UTC)

Property "Same as"

I propose this property because some tags are considered the same, and this would help with finding data. For example, if you want to find all phone numbers of an area, you will have to search for all phone=* as well as contact:phone=*. Also, artist=* and artist_name=* are the same tag. This database is perfect for finding duplicate tags like this. Janko (talk) 22:00, 19 September 2018 (UTC)

@Janko: agree in principle, but we would need a different name, because "same as" implies equality, whereas what you really want is "don't use this, use that instead". The "same-as" property could be set on both "phone=*" and "contact:phone=*", which is not what I assume you want. I created Property:P17 - "redirect-to" property. Any better name?
On top of it, we would need "different from" - copied it from wikidata -- Property:P18. --Yurik (talk) 22:55, 19 September 2018 (UTC)
@Yurik: I don't think we should use wikibase to make people tag a certain way. Wikibase should be as objective as possible. I don't know which is more right, contact:phone=* or phone=*. First one is better structured, second is easier to write. I just know they are two tags that give us the same information. Janko (talk) 00:14, 20 September 2018 (UTC)
Wikidata has a similar property, P460[2]. Janko (talk) 00:24, 20 September 2018 (UTC)
@Janko:, you are right, we shouldn't use Wikibase to tell people what to do - that would be wrong. Instead, we as a community should establish the ways certain things should be mapped. That's the reason we have a tagging mailing list, that's why we discuss things, that's why we document things on this wiki. Without it, each user could potentially come up with their own way of mapping, their own keys, thus reducing the value of OSM to near-zero. Any casual user seeing phone=* and contact:phone=* would assume these things are different in some way, causing additional mental strain. Even a simple example where local communities would start using translated keys -- while more readable for that community, the value of that data would be very low. That said, I agree that for some identical concepts we have historically ended up with multiple ways of tagging them. It would be great to document such cases, and indicate the "primary method" vs "alias-of". iD editor could use that to determine the best tag to use when a user is creating something, and we could decide some day to converge on those. At the end of the day, keys is an "internal implementation detail". New users don't care about keys, they want to draw "a house" or create "a science museum", and tools should help them by filling out all the needed keys. The "redirect-to" on the other hand could be used for when something is a no-brainer - e.g. description:DE (39) → description:de (16000). What do you think? --Yurik (talk) 01:24, 20 September 2018 (UTC)
P.S. Look at the description of Wikidata P460 -- this item is said to be the same as that item, but the statement is disputed. We might at some point need something like that, but i hope its not soon :) --Yurik (talk) 01:55, 20 September 2018 (UTC)
@Yurik: OK, there is value to having the principal tag that is encouraged, but then there should be strict rules how that is decided. For example, as you said, number of uses. And if the community decides that a tag should be overthrown in favor of a less used one, then there should be some wiki page with a vote that shows how the community decided.
Anyway, I'd like to propose a different name like "More accepted key" or "Better established key". I think when you use words like "wrong and right", edit wars will happen :)
And a new question, does it have to be "key"? What about tags? For example landuse=cemetery and amenity=grave_yard (The wiki says that amenity=grave_yard is a cemetery that is close to a church, but you get what I mean). Do we use the same Property:P17? I see no problem in using the same property, as long as a key connects with a key, and a tag connects with a tag. -- Janko (talk) 09:10, 20 September 2018 (UTC)

I think there should be both: "Equivalent key/tag" (when both tags are in use, but neither one has status (P6)=deprecated (Q5061)) and "replaced by key/tag" (for keys/tags that also have status (P6)=deprecated (Q5061), to point to the currently accepted method of tagging this). --Floscher (talk) 09:05, 26 September 2018 (UTC)

@Floscher:, Ok, so basically "has identical meaning to X", with the description explaining that community has not decided on which one to standardize on, but there is absolutely no difference between the two, and both should NOT be used at the same time? Meh, but ok.
For the "replaced by" - very often it is not "X is replaced by Y", it is "X is replaced by two items - Y & Z". Should we assume that if this property contains more than one value, they are ALL required at the same time? And in case something is too ambiguous, and the community decided to replace it with something more specific, we can have a property "X is too unspecific, and should be replaced by either one of these: Y or Z". In which case we now still face the same problem - being able to disambiguate something by multiple tags: "X should be replaced by either (Y+Z) or W". Should we start thinking about tag combinations? --Yurik (talk) 19:55, 26 September 2018 (UTC)
I found identical tags that don't have the "better" one. maxspeed=* and maxspeed:conditional=*. They tell you the same exact thing, what's the maxspeed of this road, but in different ways. Janko (talk) 13:50, 2 October 2018 (UTC)
@Janko: these are not the same tags, per description: Key:maxspeed#Time_or_other_conditions. One is unconditional speed, the other one is conditional. Different use case. --Yurik (talk) 04:42, 3 October 2018 (UTC)

Labels and IDs

A key like alt_name=* has an item Item:Q1079. That item has a sitelink (upper right corner link) to Key:alt name, plus a label alt_name. Due to MediaWiki limitations, the sitelink to Key:alt_name is the same as Key:alt name, so it is not possible to have both. Moreover, sitelink has to be using spaces instead of underscores, or else it will not be possible to link from the Key:* pages back to Wikibase items - making it very hard to find them (also affects the API). To keep the proper value, we also store the key alt_name in an ID property (P16). For now, these IDs are created only for the keys with spaces and underscores, but maybe we should add them to all items.

The label field is multilingual by design, so I think it makes sense to start using it as a short key description? E.g. "English name" or "Name in English" instead of "name:en". --Yurik (talk) 06:15, 20 September 2018 (UTC)

ID seems like a good solution to this problem, and adding it to all keys will make it easier for machine reading. Using labels as short descriptions that iD and other editors can pull also seems like the logical solution. Janko (talk) 09:30, 20 September 2018 (UTC)

Using Property:P3(Subclass) on OSM tags and keys

I see this property wasn't discussed in terms of using it on OSM tags, but I would like to use it like that. For example, amenity=grave_yard is a subclass of landuse=cemetery because (as the wiki says) grave_yards are cemeteries that are close to a church. Or, amenity=concert_hall is a subclass of amenity=music_venue. Do we use this property for this or do we invent a new one? Janko (talk) 09:53, 20 September 2018 (UTC)

@Janko: sure, but maybe we should have a different property for that? E.g. instance-of and subclass-of are for structuring meta-meta-data, e.g. this is a status, which is a subclass of osm concept, vs this key narrows down the concept of another key? Looking for a good name... :) --Yurik (talk) 23:31, 20 September 2018 (UTC)

Wikibase helper templates

I've created {{Label}}, {{OP}}, {{OQ}} templates for use with WikiBase items. They will show permanent key ID (P16) values if they exist, otherwise, they will show the 'en' label. Suggestions welcome. Teester (talk) 17:01, 20 September 2018 (UTC)

@Teester: that's awesome!!!! Thank you!!!!!! Some ideas: for {{OQ}}, I think we should show the label in the current user's language, with english fallback. Also, maybe we should show both if they are different by default? E.g. Name in English (Q1542 name:en) or name:en (Q1542)? Once again, thanks! --Yurik (talk) 23:26, 20 September 2018 (UTC) (P.S. I fixed your links above)
@Teester: do we need description as well? Or pehaps a label + description + Qid ? --Yurik (talk) 00:21, 21 September 2018 (UTC)
  • I've added a {{Desc}} template which gets the entity description. There's an optional language parameter. If it is left out, the template determines the language to use from the page name - e.g. {{desc|Q4}} will produce Un chemin est une liste ordonnée de nœuds, qui dispose normalement à minima d'un attribut ou qui est inclus dans une relation. on FR:WikiProject France and A way is an ordered list of nodes which normally also has at least one tag or is included within a Relation. on WikiProject France. You can also use {{desc|Q4|fr}} to get the french description no matter what. If there is no description for the specified language, the template falls back to English
  • Regarding {{OP}} and {{OQ}}, I suspect as the data evolves, it will become clearer how these should work. I think that any key with permanent key ID (P16) and tag with permanent tag ID (P19) should show permanent key ID (P16) and permanent tag ID (P19) (since that's the key or tag which should be used on OSM) and other entities should use localised translations e.g. way (Q4) as Chemin (Q4) on French pages
  • One interesting option is to use something like Template:Hover_title (en wikipedia) to show localised descriptions on hover for tags and keys if it is supported by this wiki
Teester (talk) 12:13, 21 September 2018 (UTC)
@Teester: looks great, thank you! Agree with the above. I will cross it out in the TODO section. Would you like to tackle the {{KeyDescription}}? Seems you have more than enough expertise to solve it :) I think it should automatically show data from wikidata with the direct link to help editing (you can add #P123 anchors to the links to jump to the right place). If the template parameter still has description, than it should override what's given in the wikidata (for now), but there should be a red/green dot indicator to show if it is the same as in wikidata or not. What do you think? Thanks! --Yurik (talk) 15:52, 21 September 2018 (UTC)
@Teester: I just spoke with @Joto: about TagInfo -- turns out they are actually parsing wiki markup (ouch!) to extract KeyDescription values out. Hopefully we can all work together on migrating it to the new system. But in the mean time, we need to figure out a way to keep TagInfo running. My idea: create a new template {{KeyInfo}} that looks similar to {{KeyDescription}}, but make it work only from Wikidata. Then, once it shows everything needed, add it to the page, and also add a new "hide" parameter to the {{KeyDescription}}. This way KeyDescription will stay on the page with all of the original params, but it will not show up. Once taginfo is migrated, we will simply delete keydescription from those pages. What do you think? BTW, this also means that TagInfo may need to support some of the same parameters that are not in Wikibase yet... --Yurik (talk) 19:54, 21 September 2018 (UTC)
I've had a brief look at {{KeyDescription}}. It looks like it's a wrapper for {{Description}}, so we should be able to develop an OSMWikiBase version of {{Description}} and transclude it into {{KeyDescription}}, while hiding the {{Description}} template at the same time. This will save us from having to change each key's page individually.
Looking at the information displayed, one thing I see is that {{KeyDescription}} shows whether or not a key can be used on each element (node (Q3), way (Q4), area (Q5), relation type (Q6)). There's also an unknown option (shown by a greyed out icon). Currently, OSMWikiBase has ERROR: Invalid ID to show that a key can be used on elements. However, we may also need a property to show that a key should not be used on elements in order to be able to show all this information. I'm not sure that there's any properties yet to cover the group, implied, see also and useful combinations sections of the template yet, either. Teester (talk) 10:12, 22 September 2018 (UTC)
On second thoughts, creating a new version of {{KeyDescription}} and transcluding the new template into {{KeyDescription}} looks a little easier! Teester (talk) 11:03, 22 September 2018 (UTC)
@Teester: I can create a don't use on property, but it seems kinda silly to have a 3 state for this. Adding status is easy - just create a new item, and set instanceof to status. Use special page. BTW, I just realized that we should have an {{O}} template that will understand both P and Q. Less confusion. Thanks!! --Yurik (talk) 21:05, 22 September 2018 (UTC)
@Teester: the more I use it, the more I think we need just one template for both, to remove any chance of mistakes. Users shouldn't use raw number, they should copy/paste the value as shown in wikibase UI. So {{O|Q2}} seems like the best choice, and we should remove the OQ and OP before they are used in many places. I have also been thinking about the migration. Some cases will not be simple to switch over to, e.g. Key:service has a field requires, which contains words or - will be hard to represent that in structured way just yet. See comment below. So I think we really ought to adjust existing template to slowly remove parameters with. And It should always highlight if the value matched WB or not. Thanks for helping with this!!! --Yurik (talk) 08:13, 23 September 2018 (UTC)
I've created {{O}} and changed {{OP}} and {{OQ}} to use it, so now, using {{O|P2}}, {{O|P2}} or {{O|P2}} should give the same result. Feel free to delete them if you wish, though. Teester (talk) 23:20, 23 September 2018 (UTC)
I've a preliminary version of an OSMWikiBase {{KeyDescription}} at User:Teester/sandbox. Issues so far:
  • No images from OSMWikiBase - since image (DEPRECATED) (P4) have to be commons files. We may have to just use the template values for now.
  • Nothing for Group, Implies, See Also or Useful Combinations since the info isn't in OSMWikiBase
  • Differences in data in the infoboxes and on OSMWikiBAse - e.g. highway (Q335) has de facto as the status in the template and approved in OSMWikiBase. The same entity also says not on relations in the template, but allowed on relations in OSMWikiBase
You can test it out by going to a page and replacing {{KeyDescription}} with User:Teester/sandbox and leaving all the variables the same and previewing the page. There's still a few bugs in the template and I haven't gone near comparing template and OSMWikiBase values yet, so please bear that in mind. Teester (talk) 00:00, 24 September 2018 (UTC)
@Teester: thanks, i deleted OP and OQ, and fixed all the usages. Also, I modified your module a bit to also handle a few other ID props. WRT images: lets also use the fallback model - if the image is specified, use it, otherwise use the property (it is set on many pros, e.g. bridge:movable (Q104). WRT groups: feel free to modify bridge:movable (Q104) to make it a perfect example of the new system. There is already group (Q12) that we can use as an instance of (P2), so just create the group(s) you need using special page. Same for implies, see also, and useful combinations (although the last one might be tricker, together with requires - it could be a simple list OR-style, or it could be in theory "X OR (Y AND Z)", which would be more difficult to represent (see discussion below). WRT data difference -- I was merging templates from all languages of the given key - that's why it might mismatch. I will review it and might reset it back to EN only for everything except descriptions. Thanks for all the hard work! --Yurik (talk) 00:27, 24 September 2018 (UTC)
P.S. Lets skin this cat one part at a time - while we can already try to show the "perfect" card, we should try to migrate existing usage at least for the few of properties initially. --Yurik (talk) 00:29, 24 September 2018 (UTC)
@Teester:, I have added a ERROR: Invalid ID, populated it with data for all of the keys we have, and also fixed the values for ERROR: Invalid ID to match English wiki. I will post the difference in languages in a separate post (i have an easy way to generate that difference now). --Yurik (talk) 01:33, 25 September 2018 (UTC)
@Teester:, I have taken a stab at implementing it using a Lua-only approach. Seems to be working, and already supports images, groups, status, and description. Please take a look, maybe you have some suggestions. {{Template:KeyDescription/Sandbox}} --Yurik (talk) 06:03, 28 September 2018 (UTC)
Unfortunately, I haven't had time to make any more progress in the past week, but your {{KeyDescription/Sandbox}} looks excellent. A couple of things I've noticed:
  • One thing it's not doing at the moment is passing a key= value into the {{Description}} template if it's not provided in {{KeyDescription/Sandbox}}, which makes it look a little odd when transcluded. It should pass permanent key ID (P16) or the en label if there's no value provided.
  • There are a couple of missing sections - perhaps fall back to provided parameters until there is a data model for them
  • I'm not sure how clear it is what the user is supposed to do when they click the various edit buttons on description section the template. Perhaps an explanatory <title> property on the edit links would help and would only be visible on hovering over the link. -Teester (talk) 13:28, 30 September 2018 (UTC)
@Teester:, thanks, fixed the automatic key/value detection. I am not sure what you mean by the missing section - any params you set on the template are automatically passed to the {{Description}}. I agree about the edit box's title - could you help out with that (have to board the plane). Feel free to modify the module. Thanks! --Yurik (talk) 19:00, 30 September 2018 (UTC)

Search suggestions to Wikibase only?

Currently, the wiki search suggests items in wikibase only. Could this please be changed (back)? This is somewhat confusing. I thought the main reason for Wikibase would be the machine-readable interface. I guess, they do not need the search suggestions...? @Yurik: --Tigerfell This user is member of the wiki team of OSM (Let's talk) 16:32, 21 September 2018 (UTC)

@Tigerfell:, thx, weird, must be a bug - it should include both the main namespace and item namespace in the suggestions. I will see how to fix that. --Yurik (talk)
P.S. the new items are still designed for humans - machines use an API :) We do want users to go to those pages to add translations and write short descriptions. Advanced users could add statements to specify validation rules, suggestions, etc. --Yurik (talk) 16:42, 21 September 2018 (UTC)

How to structure relation's metadata?

I was looking at the Relation:route, trying to figure out the best way to structure that information. First, we need to have some sort of an item that describes a relation's type. We kind of already have it in terms of a Tag, e.g. type=route (not yet in Wikibase), but this is not the same as a relation's item. Example:

property example value info
label "route relation"
instance of (P2) relation type (Q6)
<relation-type-tag> (label TBD) (item for type=route) type seems to be the only way to differentiate relations
sitelink Relation:route

The "relation's member role" is a string that could be set to a specific value like platform, or it could have free-form component, e.g. platform:<number>. I added relation member role (Q4667) (to be used with instance of (P2)) and permanent relation role ID (P21), but I don't think this would be enough. Proposed structure for storing relation member role:

property example value info
label "platform (route rel member)" indicate which relation type this role is for
instance of (P2) relation member role (Q4667)
permanent relation role ID (P21) "platform" for multi-valued ones, there will be a trailing colon
<relation> (label TBD) (item linked to Relation:route) Need to link to the item that represents route relations
ERROR: Invalid ID way (Q4), area (Q5) Same as with keys and tags
sitelink - (I don't think we have pages for them, so no sitelink)

In case the role can have extra data, use similar approach as we would for the Key elements, e.g. regex validation rule. The permanent relation role ID (P21) will have to be ending with a colon. We might want to add some other "member role type" property, similar to Keys. There will be no enum-like type similar to well-known values (Q8). Thoughts? --Yurik (talk) 07:55, 23 September 2018 (UTC)

Describing "required" constraint

Some Key:* pages have requires section in the info card, e.g. service=*. The requirement is described in plain text. How can we document it in Wikibase? Should we support arbitrary Boolean logic (any combination of must have, must not have, this OR that, this AND that, and groups? This might be overly complex to support. --Yurik (talk) 07:55, 23 September 2018 (UTC)

I'm not sure that list of required tags is complete. And even if it is, if it would be a good idea to document tag use from top to bottom. If we have taginfo that has info about service=* not being used without these other tags, than we can take that info from there automatically. Put it in the list of "frequent combinations with this tag". Janko (talk) 13:58, 23 September 2018 (UTC)

Invoking Wikidata statements

This is probably tangential to Wikibase on OSM Wiki, but for articles such as OsmAnd, it would be great to pull information from 3087512 to populate its infobox. Currently is that a possibility? Pizzaiolo (talk) 10:56, 24 September 2018 (UTC)

What would make sense here would be to build local Wikibase records with the software cards in them, especially if it means we can restart automatically building the tables TTTbot used to build.--Andrew (talk) 12:26, 24 September 2018 (UTC)

How to attribute "always used together with…"

bicycle_parking=* is defined as "always used together with" amenity=bicycle_parking. How do I attribute this on bicycle_parking (Q1099) or the wikidata page for amenity=bicycle_parking (which I don't know how to find ATM). —Preceding unsigned comment added by Tordans (talk • contribs)

Finding item is usually easy - go to the page that describes it, e.g. the bicycle_parking, and click the OpenStreetMap Wiki item link in the tools section on the left side. If the link is not there, that means there is no such item (at the moment there isn't - my bot is still creating them).
Validation rules are fun. You can do it with a property this item must be accompanied by this other key/tag (otherwise known as requires - required key or tag (P22)) on the bicycle_parking (Q1099). See describing required constraint above for the related issues, e.g. what if we need more complex rules like "this item requires either A or B", or "it requires A and (B or C)". We might actually end up with the top-down restriction model, where each OSM element must fulfill a "role" (or several). The role would describe all of the requirements, such as "must have A, B, C tags, may have D and E, must have either X or Z, ...". If the role gets too complicated, we simply split it into two separate roles. --Yurik (talk) 07:07, 26 September 2018 (UTC)
Thanks a lot @Yurik, I will wait a few days for the bot and then try my luck with validations :). PS: I love this initiative, thanks so much for this work!--Tordans (talk) 07:25, 26 September 2018 (UTC)

Should multi-valued tags be allowed?

There are several tags that hardcode value combination. Should these be allowed (makes migrating existing templates easier)? Or possibly treat them as combination of sorts (because all instances of the combination share the same key)... or? Apparently there are even wiki pages with images for them. Examples:

--Yurik (talk) 20:07, 26 September 2018 (UTC)

@Yurik: What if values change places? Is that a new tag? I think we should just group multiple tags that have the same keys into tag collections (or some name like that) and then connect that to the wiki. Janko (talk) 20:24, 29 September 2018 (UTC)
Now I've seen the buoy tags. That's complicated, the order has meaning there. I don't know.. Janko (talk) 20:25, 29 September 2018 (UTC)

Presets

I had a GitHub discussion with an author of GoMap. They have copied all of the iD's presets, so clearly there is a good grasp of what's available. It might be interesting to try to model some of that information in Wikibase, see if it would work or not. I created a feature (Q7564) item (used for instance of (P2)), where we could try to model a few of the presets, and discuss if this is a good approach, and if we can model the most complex cases. --Yurik (talk) 20:08, 27 September 2018 (UTC) P.S. Please help iron out our "demo" item - ERROR: Invalid ID. Lets model all aspects of a feature as we would want from it, e.g. requirements, nice to haves, possibly sub-requirements (e.g. if address is required, lets model address feature as well, and address should not show zip code outside of US (which probably should be a restriction on the key itself), etc. --Yurik (talk) 18:23, 28 September 2018 (UTC)

@Yurik I dont see that https://wiki.openstreetmap.org/wiki/Item:Q7565 will be a good example, since it is not preset the way ID uses presents. (ID just has a defintion of it and uses it in other presets, see https://github.com/openstreetmap/iD/search?l=JSON&q=surface). An example for an iD-Preset would be the vineyard-preset https://github.com/openstreetmap/iD/blob/master/data/presets/presets/landuse/vineyard.json. It defines two required tags and also lists a few optional tags (or sub-presets). Apart from that I just don't know enough about this new structure to model this myself; need more tutorial for that. --Tordans (talk) 13:54, 29 September 2018 (UTC)

@Tordans: awesome example, thanks! vineyard.json maps well to our data items, we may just need a few extra properties. (at least for the vineyard example - let me know if you saw example that's more complex):

  • name -- item's label + description... in every language!
  • addTags -- required key or tag (P22) - it lists individual keys and key=values
  • fields -- I am not exactly sure what this is. Is this a list of "sub-presets"? Again, modeling it would be straightforward, just not sure what to call/how to describe this property.
  • geometry -- ERROR: Invalid ID
  • removeTags -- we may need a new property for this... Something like "must not be used with", or "duplicates the meaning of" ? Not sure what to call it.
  • tags -- how is this different from addTags ?
  • terms -- not sure what these are. Is that a list of search keywords? If so, aliases would be a perfect fit.

Could @Bhousel: clear up some of it too? --Yurik (talk) 03:23, 30 September 2018 (UTC)

@Yurik: Some information about - iD presets/fields:
I have created a minimal russian version for you ( presets, fields + russian translations )
(imho) The imported iD-editor metadata : presets / fields / translations / name suggestions
  • should be "read only" in the OSMWikibase. (protected, No edit by Human editors - only just 1 import bot)
  • refreshed in every hour ? ( because of the transifex translations )
  • key name: we should use same iD-key - as in the Transifex Key - with extra prefix ( 'id:'+ 'presets.presets.advertising/billboard' )
(imho) The key problem is here - that there are multiple version of every metadata/translations
(imho) IF "This project's goal is NOT to replace the primary tag storage for the OSM database, " -
(imho) As I understand - most of keys has at least 3-4 version in the OSM-Wikibase - linked each-other
This is based on my current understandings - about the OSMWikibase project. --ImreSamu (talk) 13:55, 30 September 2018 (UTC)
@ImreSamu: thanks for the details. First, a minor misunderstanding -- I wrote that this system should not be the primary store for the tags themselves -- in other words, it should not store details of each OSM object (node/way/rel) in the wiki. E.g. "this node is a museum" shouldn't be here. I do think it should be the primary store for all of the metadata, e.g. information about the meaning of the tags, how/when to use them, groupings, validation rules, presets, etc - in other words - any documentation about OSM data. This keeps all of documentation in one place, queryable, multilingual, with the ability for automated verification, with the goal of multiple tools being able to use it consistently. I have been around for a while, but I had no idea I had to use Transifex to translate the data. I don't think its a good workflow where some of the documentation is on multiple wiki pages (one per language), some in json files (iD presets), map css (JOSM validations), some in Transifex for translation, some rules are hardcoded in various bots (no idea how to find them), etc. This leads to confusion, mistakes, inconsistencies, and makes it harder for the new data consumers to start using OSM data. I do not think we should drop everything as soon as possible - that would be too disruptive without a clear benefit. I do think we should move towards consolidation. For example, that table you created (thx!!) - it should be possible to easily create it by just running a single SPARQL query: show description-en and description-ru for all entities with instance-of=preset, that have changed in the last 3 days, and whose tags are used in OSM data more than 1000 times.
All this implies that creating a readonly copy is not applicable -- this is a master source. Your amenity=pub example will use wikibase data directly - see {{KeyDescription/Sandbox}} and {{ValueDescription/Sandbox}}, so there won't be a copy. I hope this clarifies it. Thanks! --Yurik (talk) 05:41, 3 October 2018 (UTC)
@Yurik: OK, I am still not understandings all details, but sometimes not so easy to see the big picture - at least in the decentralized / Bazaar style osm ecosystem. My vision/target is minimal now: compare all translations, and create some language reports ... so I have to link the (keys/tags)metadata. Every system has unique translations ( for example - the OSMAND translations ( https://hosted.weblate.org/translate/osmand/phrases/ru/?checksum=18b6db88189e2fbe#translations ). So different concepts, different metadata structures, different licenses - so not so easy to organize. I am still learning and researching. ( and watching this project ) --ImreSamu (talk) 22:52, 3 October 2018 (UTC)

I've been using iD a lot so I may understand the json:

  • name -- name of a feature
  • tags -- minimal amount of tags to classify something as an instance of this feature. This is akin to the tag combination I proposed.
  • addTags -- if you click a node, way or area, and tell iD to convert it into a feature, iD will add these tags. In the vineyard example, crop=grapes isn't necessary to call something a vineyard because landuse=vineyard implies crop=grapes. But iD adds it anyway. I'm not sure how to call this. Implied tags? If you added anything except "grapes", it could be considered invalid. Another example could be highway=motorway that implies oneway=yes, but it's good practice to add it anyway. Janko (talk) 13:58, 1 October 2018 (UTC)
  • fields -- These are most frequent tags that go along this feature. Something like tag attributes I proposed.
  • geometry -- ERROR: Invalid ID
  • removeTags -- When you click a feature in the iD editor, and then try to change the feature type it is (for example, change a vineyard into a orchard) these are the tags that are removed to give space for a new set of defining tags. Vineyard example removed the defining tag, the implied tag, and one of the features, grape_variety, (but not operator) because there is little chance it would be useful for other features.
  • terms -- probably keywords.

—Preceding unsigned comment added by Janko (talk • contribs)

Thanks @Janko:, so if I understood you correctly: tags = required, addTags = recommended in addition to required, but not implied (whose meaning is that you shouldn't add them because they are understood by all systems); removeTags - a subset of tags+addTags that should be removed when object is changing its type. Still unclear - fields -- is that a group-like feature? How is it different from terms? Thx! --Yurik (talk) 05:10, 3 October 2018 (UTC)

Finding missing descriptions and fields

How do you find where descriptions and other information are missing, for instance where there is no full page to import them from? --Andrew (talk) 10:34, 28 September 2018 (UTC)

@Wynndale: at the moment there is no easy way to do that, but I'm hoping to make one soon. You will be able to query all this data using SPARQL, just like you can in Wikidata (see WDQS). For now, my bot has a cached copy of the whole wikibase (which is not very big at this point), and runs some queries on it locally. --Yurik (talk) 18:24, 28 September 2018 (UTC)
What I am looking at doing is filling in gaps systematically. Ideally every tag in Wikibase, including subkeys and values that are documented as part of their parent, can get a set of values and descriptions in several languages, making a large amount of information available to software that reads the data. To do this it would be convenient to know what should be added.--Andrew (talk)
Agree, this would be the goal, and the query system should help us solve it. Lets hope I have some free time next month :) P.S. my first goal right now is to migrate to the {{KeyDescription/Sandbox}} and {{ValueDescription/Sandbox}} templates. --Yurik (talk) 02:56, 30 September 2018 (UTC)

Linking between DataItems (SOLVED: and Linking between Wikipage and DataItems)

I have a hard time understanding the system because of it's missing links.

Issue a: I created a "required"-relation from https://wiki.openstreetmap.org/wiki/Item:Q1099 to https://wiki.openstreetmap.org/wiki/Item:Q6652. So Item:Q1099 shows the link to Item:Q6652 on the relation now. But Item:Q6652 does not show the relation back. The only way to see the link from a.1 is via the "links to this page"-feature (https://wiki.openstreetmap.org/wiki/Special:WhatLinksHere/Item:Q6652). But this feature is two clicks and one page-load away, very hidden and therefore not a good solution. Ideally the Items that relate to eachother would show their links on the item page directly. --Tordans (talk) 13:34, 29 September 2018 (UTC)

SOLVED: Issue b: How to I get from a DataItem to the corresponding wiki-page? There is no "wikipage"-relation ATM, which I assume is by design, 'cause the relation will be created automatically once the Items are added to the Wiki-Page, is that right? If so, the item-page should have a prominent link to the wiki-page (see Issue a). --Tordans (talk) 13:34, 29 September 2018 (UTC)

I found the link: On the item-page, next to the headline there is the link to the wiki-page. --Tordans (talk) 14:02, 29 September 2018 (UTC)
@Tordans: not sure why you call "links to this page" as a two clicks, looks like just one to me :)). This is a fairly common Wikibase problem, and it has several solutions. First - the reason why it is done that way, is because each page shows just the values stored on that page. If you showed everything that was using this item, you may get huge amounts of useless data. For example, almost all data items right now link to tag (Q2), node (Q3), way (Q4), ... - you wouldn't want to see them all when viewing those pages (try clicking on "what links here" for them). That said, very often you will want to have an intelligent query of connected items. I have seen people use gadgets on the wikidata.org site. Those gadgets tend to use Wikidata Query Service. I don't have that set up for OSM wiki just yet, but hopefully it will be there within a month, and people will be able to build custom OSM wiki gadgets to help with this. --Yurik (talk) 02:37, 30 September 2018 (UTC)

How do I add bicycle_parking=stands?

We have the general description of bicycle_parking (Q1099) which is basically bicycle_parking=*. But we also need the (most common) values, like bicycle_parking=stands. The wiki page bicycle_parking=* has ~15 of those. How to I add them? Is there a "add sub item"-link? --Tordans (talk) 14:10, 29 September 2018 (UTC)

@Tordans: I agree we need those values, but I don't have an easy way to find them all from the wiki. I might import them from the taginfo site, but that won't have descriptions. So far, I have scanned the wiki for the {{ValueDescription}} usage, and imported thousands of those. The bicycle_parking, as you can see from the red link (i modified your post a bit), shows that we don't have a page for that. And I don't think we should have a page, but instead we do need to find a good way to parse these tables into the data items. If you can create a google spreadsheet with all these values (key, value, description_en, plus possibly more languages, image, group, status, elements_to_use_on) -- I can automatically import them. Note that you don't need to specify anything that's the same as the key data item. E.g. don't worry about the group or status if its the same as bicycle_parking key itself, or if there is no data about it. Also, there is no such thing as a "subitem" here. A tag value is simply another data item, with its instance of (P2) set to tag (Q2). See bridge=movable (Q5711) for an example. --Yurik (talk) 02:29, 30 September 2018 (UTC)

Tag combinations and Tag attributes

Tag combinations are items that have a semantic meaning only when together. But there are too many combinations. For example, shop=bicycle can have service:bicycle:retail=yes, service:bicycle:rental=yes, and lots others. So do we really make a combination for all those tags? I think we should maybe add attributes to shop=bicycle, and each attribute has a short description. That way you can also add a default value of the attribute, for example attribute oneway for highway=secondary has default value "no" if there is no oneway tag. And "yes" for motorway. Janko (talk) 20:06, 29 September 2018 (UTC)

@Janko: For the yes tags we could either have a data item for both the key and the value (that's how we have some of them already) - e.g. an item for service:bicycle:retail=yes in addition to the service:bicycle:retail=* (service:bicycle:retail (Q691)). Or we could introduce a new "yes-only" item for key type (P9), and use it instead of the well-known values (Q8). There are pros/cons for both. On one hand, there are many "yes" items, so duplicating them makes it harder to manage. Plus it is not clear how the description should be different between the key and the value. On the other, yes values very often morph into other values as well, so it will not be clear how to document those - should we change the type of the item from yes-only to known-values?
For the combinations, this is becoming even stranger. In theory, we can an infinite number of tag combinations. Which of these should we document? Apparently iD has done some preset system that we could model after. Should we just try to import their work? --Yurik (talk) 02:54, 30 September 2018 (UTC)

extreme number of some presets: count(Tags) > 10000

in the https://taginfo.openstreetmap.org/projects - some projects have an extreme number of presets (tags)

TOP5

some theoretical questions:

I don't know the answers - but we need plan ahead - and prevent some problems.

--ImreSamu (talk) 14:45, 30 September 2018 (UTC)

Listing translations

Would we be interested in storing the list of translated pages in the data items? For example, Item:Q104#P31 stores the list of Key:bridge:movable translations.

PROs
Page titles do not have to be the same as the EN page, and can be localized. Better performance - language template will work much faster, and will be created using a simple Lua script. 3rd party tools can easily find all translated pages. For EN pages, Lua can get the list of other languages by using associated data item.
CONs
A page wouldn't show up in the list of translations unless added to the data item (could be automated). Each page must have a corresponding data item (also automation). Non EN pages must specify either the data item or the corresponding EN page's title in order for Lua script to find the right data item.

--Yurik (talk) 09:57, 6 October 2018 (UTC)


imho / I don't know the technical backgrounds:
  • language neutrality - so please add "all languages" ( include EN ) to your example. - imho this is like a "Wikipedia Sites" concept in Wikidata - with differences; my suggestions "Available languages".
    • the "translated pages"/"page in other languages" is not a neutral concept,
    • sometimes the English version is a translated version. for example Key:jel - "Hungary marked hiking trails"
    • sometimes no English version. (it is possible; for example, some local keys - used by local imports)
  • should be protected by human edit. We have a fix id - the osm key, so no need human edit here.
--ImreSamu (talk) 12:55, 6 October 2018 (UTC)
ImreSamu, I would love to make it less "English-centric", but with this approach it won't make any sense. OSM wiki is designed around English being the primary language, e.g. Key:bridge:movable is always in English, and all translations add some prefix, e.g. JA:Key:bridge:movable. The primary (EN) page is already part of the data item -- see the link in the upper right corner of the Item:Q104. Duplicating it just for the sake of duplicating will lead to errors and confusion. I wish there was a different approach, but I cannot think of one.
P.S. Making it automatic is semi-possible IF each non-EN page links to the EN page in the languages template. For example, JA page could add {{Languages|ja|Key:bridge:movable}} or {{Languages|ja|Q104}}, which would tell the languages template that the language of the current page is JA, and also will show the list of other languages from the data item. A bot could also update that data item with the link to JA page. This assumes that there is only one laguage per data item. --Yurik (talk) 19:13, 6 October 2018 (UTC)
Do you mean using the Wikipedia interwiki mechanism with our own Wikibase instance?--Andrew (talk) 18:12, 6 October 2018 (UTC)
Andrew, Wikipedia interwikis (sitelinks) would not work for us because Wikibase assumes there is only one page per wiki (in other words, each wiki represents only one language). We use sitelink mechanism to connect data items with corresponding English pages, e.g. Item:Q104 links to Key:bridge:movable. Sitelink is shown in the top right corner of the data item's page. The suggestion above on the other hand is not tied to the sitelinking mechanism, but instead uses a "monolingual string" property - a string that also contains the language it is written in, or in our case - the language of the page. So we can store any number of such strings in one data item. --Yurik (talk) 19:13, 6 October 2018 (UTC)


There are tags - with wiki pages, but without English translations.
from a QA view, it is important info IF English translations exist or not. ( need some QA report to fix )
Some examples:
As I see the Taginfo don't expect - that English Wiki always exist
* https://taginfo.openstreetmap.org/tags/?key=boundary&value=religious_administration#wiki (Wiki pages about this tag : Deutsch · polski - no English!)
* https://taginfo.openstreetmap.org/tags/brand=Trilib'#wiki
* ...
And ceckin the JOSM XML preset - this is also contains the english wiki pages.
for example Highway_link info in the XML:
"link (i18n) - web link to further details (optional)"
( example from the : beautified-JOSM-preset )
So How I can detect if the "tags" has not English wiki page?
--ImreSamu (talk) 22:01, 6 October 2018 (UTC)


ImreSamu, the item shop=outpost (Q5145) uses Tag:shop=outpost as a sitelink (upper right corner), but (sadly) does not show it as a red link (technical limitation). This is how this data item can be found by everyone, despite the page not being there. You are correct that this leaves open the QA problem you mentioned. I see two options: either we always add EN links (which would be for the majority of the data items), and have a complex bot that catches things like mismatched sitelink and regular link, missing regular links when it should be there, etc, OR, use a special English string, e.g. "_" to indicate that there is no English translation for that data item. I suspect adding EN link to all data items might be more disruptive and error prone than using a special case. Underscore was chosen because no wiki page can be named that. --Yurik (talk) 22:31, 6 October 2018 (UTC)

Yurik,
  • according to the taginfo "OpenStreetMap is an international project. Tags and their descriptions can be in any language. This table lists the languages taginfo knows about and how many wiki pages there are in these languages documenting keys and tags, respectively." IMHO: languages is a highly polarized topic, so you will receive mostly emotional reactions. And this is a too high risk for a minimal local optimum.
  • Don't expect that the community will create all wiki documentation: https://taginfo.openstreetmap.org/reports/frequently_used_keys_without_wiki_page
  • You can download the taginfo osm wiki (sqlite3) database here: https://taginfo.openstreetmap.org/download
--ImreSamu (talk) 00:58, 7 October 2018 (UTC)
ImreSamu, I am afraid I did not explain properly, or you misunderstood. I am not saying we should force documentation, or that English page MUST exist, or anything like that. I am talking about a small technical element to match the current OSM wiki setup - something that 99.99% of the community would not even be aware of. If you want to introduce "EN:Key:something" pages for every key to document that Key in English, I'm all for adding that string to the property - because then there is a difference between the KEY (sitelink) and the description page in English. But without it, there is no point to storing EN string in 2 places -- sitelink and P31 property, as that would cause errors, without giving any benefit. I would even go further and suggest that P31 should almost never be set by any user. Instead, a bot should track when a page XX:Key:something is created, and automatically update that property. BTW, I am not a native English speaker, so I hope my opinion is not based on the language bias, but purely on the technical merit. --Yurik (talk) 01:13, 7 October 2018 (UTC)
Yurik, thank you! debugging my own misunderstanding:
  • the "local sitelink" = Key:bridge:movable is like a "human primary key" ?
  • And If the Users can set their preferred language (like in the wikidata) -> the sitelink (upper right corner) should be changed to the preferred language.
--ImreSamu (talk) 00:06, 8 October 2018 (UTC)
That's a good analogy, yes. I do think it would be nice to show a localized link there - but it would require some creative coding though, esp if we also want to show red link for missing page. This is not a critical requirement (most people will be reading wiki pages, not the data items), but definitely nice to have. --Yurik (talk) 00:53, 8 October 2018 (UTC)

DONE This has now been implemented as documentation wiki pages (P31) on most data items, and it includes English pages as well. --Yurik (talk) 02:58, 24 January 2019 (UTC)

Native name within the side template KeyDescription, ValueDescription

Topic moved from Wiki Translations.

Can we add the parameter |nativekey = in Template:KeyDescription, |nativekey = and |nativevalue = in Template:ValueDescription as a name internationalization project?

Many names in English are very unclear with any translation (e.g. in Google).
I have been adding it for a long time in Polish versions, but I have to use my own modified templates.
I would prefer to use standard templates, as well as other people from Poland.

An additional name in the native language inside the template, it will help them quickly understand names for people who do not know any of the languages describing a given key or tag and simplify the search.
Maybe someone knows a better way to show these native names in side templates?

E.g: 

{{ValueDescription
|key=
|value=
|nativekey=<name in native language>
|nativevalue=<name in native language>
.
.
}}

Thank you for your understanding. --Władysław Komorek (talk) 23:07, 9 December 2018 (UTC)

Władysław Komorek, thanks for raising it. I was looking at PL version of the template just yesterday -- AFAIK it is the only template that does not use {{Description}}, and it is on my todo list to fix it. If I understand correctly, nativekey and nativevalue are the only additional parameters it handles. Please take a look at the ValueDescription Intro section - I already mention the usage of those two parameters (same for Key), and that they should be added to the data items as translated item label. In the mean time, I will add handling of the nativekey/value to the Description template - it won't affect any key/tag page that doesn't use it. --Yurik (talk) 23:39, 9 December 2018 (UTC)
It would also be nice to add a missing value for |image_caption = and the line below. See taxon=*. --Władysław Komorek (talk) 23:50, 9 December 2018 (UTC)
Can't you use the already existing field "label" instead of a new "nativevalue"? (Compare above topics #English label? and #Labels and IDs) The label field is already there, it just needs to be filled and then shown by the template, perhaps somehow optionally. I think this feature would not only be interesting for Polish but also for other languages (compare this related discussion: User_talk:Geozeisig#Please, stop removing descriptions). The value for "nativekey" would have to be taken from the label field of the corresponding key-item, I assume this would be possible somehow, and it would reduce redundancy. --Hufkratzer (talk) 09:45, 12 December 2018 (UTC)

Previous discussion at Template talk:ValueDescription#native_value_and_native_key_parameters. --Andrew (talk) 00:15, 10 December 2018 (UTC)

Per above, I have switched the {{Pl:KeyDescription}} to the unified Description system. Please take a look at the new implementation, and if everything is fine, I will switch the {{Pl:ValueDescription}} as well. If it is badly broken (e.g. many pages are in a bad shape), please revert this change. --Yurik (talk) 02:48, 10 December 2018 (UTC)

So far so good. Nothing is broken. :) --Władysław Komorek (talk) 08:22, 10 December 2018 (UTC)

It would be good if the |group = parameter could create Namespace:Category. --Władysław Komorek (talk) 23:28, 9 December 2018 (UTC)

I'm not sure what you mean by Namespace:Category, could you give an existing example? If by Namespace you mean language prefix, than I already added that yesterday (ValueDescription already had it, and KeyDescription now has it automatically as well because it seems many Key:* pages did that manually). The current logic (search for "group category" in Module:DescriptionFromDataItem) -- if the group is given, check if "Category:Language:<Group>" page exists. If it does, current page is added to it. If it doesn't, than current page is added to the "Category:<Group>". Note that a page can be added to a category without the category page actually existing. --Yurik (talk) 23:47, 9 December 2018 (UTC)
Sorry. My mistake. I can see it. The only difference is that I have a group value linkable. --Władysław Komorek (talk) 00:12, 10 December 2018 (UTC)
Linking to Category:Group might not be a good idea -- Pl:Key:taxon group links to Category:Pl:Środowisko_naturalne, whereas Key:taxon group points to natural (redirects to Key:natural). If PL group linked to Pl:natural, it would have redirected to Pl:Key:natural, a much better alternative to the category. --Yurik (talk) 01:45, 10 December 2018 (UTC)
See previous topic - {{Pl:KeyDescription}} has been migrated. --Yurik (talk) 02:50, 10 December 2018 (UTC)
Maybe taxon is not the best example. Currently, I am in the process of organizing the category in the Polish version. I'm interested in linking |group = to the basic category not to the article. This will facilitate a quick search for similar tags. Also Polish name "Środowisko naturalne" is a more understandable equivalent to the English name "Natural". Category:Pl:Natural is redirected to Category:Pl:Środowisko naturalne --Władysław Komorek (talk) 10:29, 10 December 2018 (UTC)

I lost list of groups in Category:Pl:Key descriptions by group. What is there is a mess. --Władysław Komorek (talk) 00:20, 11 December 2018 (UTC)

Multi-valued regional differences

I just added regional difference storage, e.g group=... or status=... can now be different for different languages. The system is fairly straightforward for the single value ones, but it gets a bit complex for the multivalued ERROR: Invalid ID - the logic might get too complex for the casual users/data consumer. So, I see two options:

  • Keep current system, e.g. should be used on = Way (except DE,PT,RU), should not be used on = Way (only in DE,PT,RU), ....
  • Create individual properties for each type of object:
use on way = yes (no qualifiers, preferred)
use on way = no (limit to DE,PT,RU)

To set up the second system, we would need magical Q values for yes, no, unknown. Thoughts? --Yurik (talk) 02:15, 19 December 2018 (UTC)

Done - I will remove (DEPRECATED) excluding region qualifier (P27), various q items like area (Q5), and ERROR: Invalid ID. --Yurik (talk) 19:38, 24 December 2018 (UTC)

Import of redirected items

I noticed that items for the surface (Q746) tag weren't imported. I suspect the importer was programmed to skip items of which the wiki page redirects to another page, like surface=concrete. However, some of those should have an item, like surface=concrete (Q16132). How should we proceed? —M!dgard [ talk ] 22:15, 26 December 2018 (UTC)

My bot only imports infoboxes (those created with {{Description}}, such as {{ValueDescription}}). It actually ignores the redirect status of the page - e.g. if a page is a redirect but it still contains infobox, it should still be imported. At this point, the simplest would be to create an infobox inside the redirect page, and the bot will pick it up, or just set up the data item directly. Eventually all parameters should be gone, and we may even have a custom web form somewhere to automate data item creation. --Yurik (talk) 23:24, 26 December 2018 (UTC)

FYI: Wikidata Workshop in Ulm, Germany (2019)

In case anyone from Germany wants to learn more about Wikidata, there is a Workshop + Hackathon at the end of February 2019. Train tickets are sponsored. Details at https://de.wikipedia.org/wiki/Wikipedia:Wikidata-Workshop-Ulm-2019

Sigh, if only WD team would listen to the actual user requests (like allowing non "Q" items or supporting images if they are stored on the local wiki or on Commons transparently)... Still, I think Wikidata team is by far the most productive at WMF. --Yurik (talk) 17:57, 7 January 2019 (UTC)

Restructuring key validation

Currently, validation only exists as a value validation regex (P13) property. I would like to restructure it as following:

  • Introduce a new instance-of type - "validation rule"
  • key data items will have a new "validation rules" property, that links to the validation rule data items.
  • The validation rule would contain the value validation regex (P13) or possibly other properties.
  • The validation rule's label and description will describe the exact conditions, e.g. label=non-negative number and description=The value must be a non-negative number.

This structure allows us to get per-rule translations, even if the actual rule is defined outside of the data item (e.g. "geometry must not intersect lines"). This structure also allows multiple rules attached to the same key. --Yurik (talk) 21:16, 23 January 2019 (UTC)

Seems fine, I guess. —M!dgard [ talk ] 13:01, 25 January 2019 (UTC)

Regular expressions

Which regex flavour should be assumed? You have POSIX BRE and ERE, GNU BRE and ERE, PCRE, Python, Tcl…

Also: I changed to “the wrapping ^( and )$”. Just ^ and $ is not equivalent to doing a full match. Consider e.g. matching decimal or hex: [0-9]*|0x[0-9a-f]+. —M!dgard [ talk ] 09:00, 24 January 2019 (UTC)

This is a good question. I guess the rule should be "lowest common denominator" -- don't use character classes, always escape forward slashes, no named capturing and non-capturing groups, etc. Also, thanks for catching the wrapper logic bug. Please have a look at the previous section about restructuring rules as their own data items. P.S. I found the bug with the user editor checker, the bot shouldn't overwrite stuff again. I have reverted all the ones I found, and will rerun the bot on them properly. --Yurik (talk) 03:21, 25 January 2019 (UTC)
Actually "lowest common denominator" would exclude groups, because in the legacy BRE they're \(foobar\) and in ERE (and everything else) they're (foobar). If you take Tcl into account, you also lose things like word boundaries with \b (because they use \m and others). So "lowest common denominator" of which ones? POSIX ERE, PCRE and Python? —M!dgard [ talk ] 12:15, 25 January 2019 (UTC)
I guess what I meant was "most common stuff" (TM) :). Groups seem to be fairly well supported without escaping. The forward slashes are trickier (e.g. Perl/PHP/grep might have some issues with that?). We could make a list of systems we really want to support, e.g. JavaScript (client-side validation), Java (JOSM, Sophox's SPARQL), Python (various bots). Possibly others that actually do data validation (Ruby? .NET? ...?) --Yurik (talk) 19:21, 25 January 2019 (UTC)

Template to retrieve just a description

Is it possible to create a template to retrieve the description (plain text string) for a key or a value from the wikibase item? It would be very useful for use in the sections "See also" / "Related tags" that are at the bottom of most wiki pages, example. Currently all these descriptions have to be copied manually from the corresponding key/tag wiki pages, which is tedious/cumbersome. The template could look like {{TagDesc|key|value}} and should retrieve the description in the language of the wiki page where it was used --Hufkratzer (talk) 14:28, 17 February 2019 (UTC)

@Hufkratzer: that's a good idea, thanks. We already have {{O}} to link to data items, and {{Desc}} to show the description given the Q ID (Desc is not actively used so we can change its meaning). I think we can easily change it to use the "sitelink" ID instead - e.g. you will write {{desc|Tag:noexit=yes}} or {{desc|Relation:border=inner}} to add a description of the noexit=yes or the inner role for the border relation. --Yurik (talk) 22:56, 17 February 2019 (UTC)
P.S. Are there times when you want to show more than just description, e.g. show the tag (key=value) followed by the description? The wiki might be a bit more perfomant in those cases. --Yurik (talk) 23:18, 17 February 2019 (UTC)
I would like to have, always as a format, the first line where/when to use this tag/value:
Tag|key|value - {{TagDesc|key|value}}
Then the "==Description==" section for the object "key=value", where you can add more description of the given value or key , as in this example.
Often, instead of a Wikipedia link, we have a rewritten Wikipedia text. Currently, we have a different article structure. Giving some order in the creation of proposals and then a new page tag=value, simplifies reading and translation of the article. --Władysław Komorek (talk) 07:57, 18 February 2019 (UTC)
* I would not overwrite/redefine the standard template {{Desc}} because someone might want to use it in its original form and because it would be confusing; I think it's better to create or to extend other OSM specific templates.
* I find {{Desc|Tag:noexit= yes }} more complicated to write than {{TagDesc|noexit|yes}}.
* I don't think that the new template should create section headers or other additional things to the description string because this would limit its usefulness. Very few wiki pages use the same description string that is shown in the {{ValueDescription}} box also in their long text. What I had in mind was to use it in the "See also" sections like "{{Tag|amenity|waste_disposal}} - {{TagDesc|amenity|waste_disposal}}". An alternative may be to integrate the functionality into the already existing {{Tag}} template. Then, for example, instead of the above we would just need to write "{{Tag|amenity|waste_disposal|desc= yes }}" or sth. like that. But "{{TagDesc|amenity|waste_disposal}}" would have the advantage that it could also be used in tables (e.g. in Template:Map_Features:amenity). OTOH perhaps we can also extend the {{Tag}} template in a way that it can alternatively also be used this way (e.g. "{{Tag|amenity|waste_disposal|desc= only }}"). I am not sure what is better/easier.
* Without a parameter for the language the standard template {{Desc}} gets the description in the current language of the wiki user interface. This is not what we want, we always want Polish descriptions on Polish wiki pages, German descriptions on German wiki pages and so on. So we need to detect the language of the page where the template is used (probably from the filename) and then use {{Desc}} with parameter "en", "pl", "de" ... (e.g. {{Desc|Q6465|pl}}).
* For key descriptions (and relation descriptions?) we need similar things.
--Hufkratzer (talk) 21:05, 18 February 2019 (UTC)


Backup

How can I download full backup of all data items, in format that requires no full Wikidata installation to use? Mateusz Konieczny (talk) 06:34, 27 March 2019 (UTC)

The easiest is actually to query Sophox to get just want you need for you task. Apart from that, you can download it via MediaWiki API (see my bot code that does exactly that). In theory regular OSM backups should contain it (needs to be checked just in case). --Yurik (talk) 06:43, 27 March 2019 (UTC)
I thought about a full backup - one of my worries about switching from templates to data items is whatever this data is backuped in some readable format. Running queries across all items seems a poor way to do that. Mateusz Konieczny (talk) 08:28, 27 March 2019 (UTC)
Data items are no different than regular wiki pages from the back up perspective -- they are stored as json inside the same system. Getting a local mediawiki + wikibase set up is a relatively painless activity - you can simply use a docker container and have it running within minutes. The non-mw approach is what i described above. --Yurik (talk) 09:28, 27 March 2019 (UTC)
OK, then everything is fine if that data is included in a regular dump of wiki Mateusz Konieczny (talk) 11:24, 27 March 2019 (UTC)
Just to be sure, I downloaded the dump (~3GB, 35GB+ uncompressed) from /dump, and yes, all the data is there. Reading JSON in XML is a bit scary and hugely wasteful: <text xml:space="preserve" bytes="3649">{&quot;type&quot;:&quot;item&quot;,&quot;id&quot;:&quot;Q4963&quot;,&quot;labels&quot;:{&quot;en&quot;:..., but alas. --Yurik (talk) 23:21, 27 March 2019 (UTC)
Resolved: Marking as solved Mateusz Konieczny (talk) 16:40, 9 December 2020 (UTC)

Thanks!

Comming from Wikidata I really like seeing all the good work put into getting the data items to work! I like the bot-solution that automatically creates items. It works quite well I think.--PangoSE (talk) 13:30, 12 January 2020 (UTC)

Resolved: Marking as solved as there is nothing actionable here Mateusz Konieczny (talk) 16:38, 9 December 2020 (UTC)

Known Bad Tags

Would it be a good idea to have a suggested replacement for known bad tags?

For example, https://wiki.openstreetmap.org/wiki/Item:Q18785 and https://wiki.openstreetmap.org/wiki/Item:Q16174 -- the first is `surface=paved2` and the second is `surface=paved`. `paved2` is (obviously) a typo to a human, but it can be difficult to detect programmatically.

I've gone ahead and given the `surface=paved2` a rejected status for now, although it (really) isn't the best solution, and I would prefer to point to Q16174 from Q18785 as a "correct" tag.Vorpalblade77-kaart (talk) 19:56, 22 September 2020 (UTC)

Never mind. I just saw P17 (redirect to).

Resolved: Seems to solved Mateusz Konieczny (talk) 16:35, 9 December 2020 (UTC)

Watchlist complaint about bot creation

"Many data items were created by bot in large batches of several hundred in one day, including for all tags with more than 10 uses and a certain format. Many of these items only include the key=value and no additional information. This makes it difficult for wiki users to review the new items."

This complaint seems to be

  • outdated (initial run stopped and bot is inactive anyway)
  • not an actual problem - but either created empty items or just copied what was in the wiki, so there was nothing to review

I propose to remove this specific entry.

Mateusz Konieczny (talk) 10:50, 10 December 2020 (UTC)

Resolved: solved, marking as ready for archival Mateusz Konieczny (talk) 21:32, 26 March 2021 (UTC)

How can I create/find data item for element?

See https://wiki.openstreetmap.org/wiki/Tag:name=NHS - where is its data item? Mateusz Konieczny (talk) 19:51, 19 February 2021 (UTC)

I use to check existence with the following two searches:
If there is no item, the first search via the special page provides a link to create it. --Chris2map (talk) 20:27, 19 February 2021 (UTC)
@Chris2map: - it does not work. It created Name=NHS page Mateusz Konieczny (talk) 20:39, 20 February 2021 (UTC)
This is strange. I created several items in that way yesterday. What about https://wiki.openstreetmap.org/wiki/Special:NewItem ?
In user preferences I toggled on all 3 gadgets relating to data items. --Chris2map (talk) 09:47, 21 February 2021 (UTC)
Special page works, thanks: https://wiki.openstreetmap.org/wiki/Item:Q21645 Mateusz Konieczny (talk) 10:17, 21 February 2021 (UTC)
I created a (simple) JOSM plugin to help facilitate the creation of wiki data items (install `wikidata`, to use right-click on a tag from an object, and then select the create wiki data option). It looks like System-users-3.svgMateusz Konieczny (on osm, edits, contrib, heatmap, chngset com.) has resolved the problem already though. Vorpalblade77-kaart (talk) 20:50, 22 February 2021 (UTC)
@Vorpalblade77-kaart - I didn't know about the plugin. Thanks for the hint! First I was confused with the name "Wikidata". That is something different. I would appreciate, if the plugin used the term "Data Item". Yours --Chris2map (talk) 17:08, 24 February 2021 (UTC)
I'll try to remember to change it (I do have `Data Item` in the description). Just keep in mind that it is (literally) alpha quality. It may crash, it may have performance issues, it may create duplicate entries, etc. Go ahead and report bugs from it (on JOSM Trac, cc `taylor.smock` -- I don't think there is a component for it yet). Vorpalblade77-kaart (talk) 17:14, 24 February 2021 (UTC)
Resolved: fully solved in section below Mateusz Konieczny (talk) 21:31, 26 March 2021 (UTC)

What is wrong with this data item?

https://wiki.openstreetmap.org/wiki/Item:Q21679

Just created, is not detected on https://wiki.openstreetmap.org/wiki/Tag:house%3Dbungalow

I tried null edit on https://wiki.openstreetmap.org/wiki/Tag:house%3Dbungalow

Should I wait? Or is something wrong with it? Mateusz Konieczny (talk) 11:01, 24 March 2021 (UTC)

@Mateusz Konieczny: You should also add the link, the label doesn’t connect the item with the wiki page. Due to how this wiki modifies Wikibase, this is not trivial, but the JavaScript-less Special:SetSiteLink/Q21679/wiki works (of course you need to use the Qid of whichever item you want to connect). —Tacsipacsi (talk) 12:48, 24 March 2021 (UTC)
Thanks! Mateusz Konieczny (talk) 14:37, 24 March 2021 (UTC)
In addition, if you create an item with the search function https://wiki.openstreetmap.org/wiki/Special:ItemByTitle?site=wiki the wiki-link will be created automatically. In the upper field there has to be selected "wiki", in the lower field the tag has to begin with "Tag:" or "Key:" -- Chris2map (talk) 12:42, 25 March 2021 (UTC)
Thanks, this one works properly! Mateusz Konieczny (talk) 21:30, 26 March 2021 (UTC)
Resolved: Thanks for solving my problem! (though if someone is looking after data items - it may be worth documenting if it is not documented already on Data items) Mateusz Konieczny (talk) 19:24, 9 April 2021 (UTC)

creation permission error

Hello, I tried to create the item for group Emergency via special-page. But each time it ends up with this error: 

The title "Item:/" has been banned from creation. It matches the following blacklist entry: .*\/
Permission error

Can anyone clear it and help me with it? cheers,--Chris2map (talk) 17:16, 30 May 2021 (UTC)

Realized that I obviously can't create any new data item! Just tried to with Tag:geological=volcanic_lava_field. Same error as above! What is going on? Could one give me any information? Regards, --Chris2map (talk) 18:24, 6 June 2021 (UTC)

@Tigerfell: It looks as if the title blacklist/whitelist settings need to be adjusted. --Andrew (talk) 07:49, 7 June 2021 (UTC)
@Wynndale: No, items should not have such titles in the first place. They should not be named Item:/, but rather Item:Qid (id being a decimal number). I suspect it’s caused by some update or other backend change, and investigation may need direct server access (especially as Wikidata doesn’t seem to have the same issue). —Tacsipacsi (talk) 09:44, 7 June 2021 (UTC)
Thank you for your replies! I supposed a abuse filter, too. But the trace of Tacsipacsi sounds like something "deeper". Does it mean actually no one can create any item? --Chris2map (talk) 16:11, 11 June 2021 (UTC)
I think so, but, again, I can’t do deeper investigation. The only thing I can say for sure is that submitting the form gives the same error message for me. —Tacsipacsi (talk) 23:31, 11 June 2021 (UTC)
Just a heads up, I've filed a bug report here: https://github.com/openstreetmap/operations/issues/564, which is hopefully the right place for a bug report :) Vorpalblade77-kaart (talk) 14:28, 16 August 2021 (UTC)
This does seem like a permission issue. I was able to create the new item by following the link created by Tacsipacsi above -- Item:Q21769. Has Wikibase extension been updated recently? --Yurik (talk) 14:57, 16 August 2021 (UTC)
Could someone try to create a data item again (a different one from above, as that one already exists?). I can delete it later if not needed. I just adjusted rules that were changed by @Tigerfell: in May - this could be it. --Yurik (talk) 15:38, 16 August 2021 (UTC)
Confirmed to be fixed. --Yurik (talk) 15:40, 16 August 2021 (UTC)
Sorry, I missed this earlier. I changed the rules back, but added Item:/ to MediaWiki:Titlewhitelist. I created Item:Q21772 via Special:NewItem so I assume that it works again. Did I test it correctly? --Tigerfell This user is member of the wiki team of OSM (Let's talk) 08:01, 17 August 2021 (UTC)
I'll have to double-check (I've got to find something as yet undocumented for testing), but one of the issues was an admin could create the item, but a standard user could not. Presumably, the more specific whitelist rule overrules the less specific blacklist rule. Vorpalblade77-kaart (talk) 13:02, 17 August 2021 (UTC)
The MediaWiki documentation states that the Whitelist overwrites the blacklist. The system administrators enabled logging for the title blacklist and there is no entry in the list for three days. I can not see if an item was created during that time though. --Tigerfell This user is member of the wiki team of OSM (Let's talk) 13:05, 23 August 2021 (UTC)
It looks like Item:Q21774 was created today by User:Vakonof, so it should be working OK. Vorpalblade77-kaart (talk) 13:51, 23 August 2021 (UTC)
Correct, I tried adding that page a few days ago but wasn't working back then. Hopefully all fixed now. Vakonof (talk) 15:12, 23 August 2021 (UTC)
Resolved: solved, marking as ready for archiving --Chris2map (talk) 08:16, 27 November 2021 (UTC)
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Talk:Data_items/Archive_1&oldid=2227765"