WebSafe 3.7phabricator.wikimedia.org
|
|
🏠

Develop semantic / distinct representation for wikifunctions output in Parsoid DOM
Closed, ResolvedPublic

Description

Independent of whether wikifunctions usage in wikitext is going to be a parser function or an extension tag, we want to ensure that clients (specifically editing clients like VE) can easily identify wikifunction markup so as to provide dedicated editing support for their usage.

Context: Parsoid's current HTML spec doesn't have any special support for parser function markup -- they are currently represented with a "mw:Transclusion" typeof attribute with the data-mw attribute embedding the knowledge of the parser function deep inside the attribute. Since we cannot reasonably change this representation without a breaking version change, it makes more sense to use his opportunity to prototype a better representation for parser functions that we can migrate everything over to at a future date.

Current plan, as of 2025-03-17:

Before: Attribute/sub-value (jq notation)Before: ValueAfter: AttributeAfter: ValueChange
typeofmw:Transclusiontypeofmw:Transclusion mw:ParserFunction/padleftAdd mw:ParserFunction/key with the key being the message key for the parser function name
data-mw.parts[0].template.target{"wt":"padleft:xyz","function":"padleft"}data-mw.parts[0].parserfunction.target{"wt":"#padleft","key":"padleft"}template -> parserfunction ; function -> key; wt has zeroth param moved out
data-mw.parts[0].template.params{"1":{"wt":"4"},"2":{"wt":"-"}}data-mw.parts[0].template.params{"1":{"wt":"xyz"},"2":{"wt":"4"},"3":{"wt":"-"}}Zeroth param in wt is now first param, others renumbered

Note also that the data-mw.parts[0].parserfunction.target.wt value will have the # included iff the original wikitext did, but the key is the magic word ID and will never have a #.

  • Add support for generating the new format in Parsoid
  • Add mechanism for parser functions to register that they want this new format
  • Add support in Wikifunctions's VE integration to read/write the new format
  • Switch Wikifunctions's code over to the new format

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

The minor change is that the existing parser functions markup combines the first argument with the parser function name, eg the template name is tag:ref. I'll definitely want to fix that.

$ echo '{{padleft:xyz|4|-}}' | php maintenance/run.php parse.php --parsoid
parse.php: warning: reading wikitext from STDIN. Press CTRL+D to parse.

<section data-mw-section-id="0" id="mwAQ"><p id="mwAg"><span about="#mwt1" typeof="mw:Transclusion" id="mwAw" data-mw='{"parts":[{"template":{"target":{"wt":"padleft:xyz","function":"padleft"},"params":{"1":{"wt":"4"},"2":{"wt":"-"}},"i":0}}]}'>-xyz</span></p>

so xyz is going to move into the params.1 slot for sure and target.wt will be just padleft. Currently a parser function is a mw:Transclusion typeof with function as a key in the target. We can either do something subtle like add a new or different key to the target section, or do something more obvious like change mw:Transclusion to mw:ParserFunction with or without a change to the data-mw fields. The params values are technically extensible arrays, with wt meaning "the wikitext value of the argument" and if you turn on the right options Parsoid used to be able to add an html key for "the html value of the argument" as well. We could extend that universe of possible value markers further if needed.

So, we'd roughly go from:

typeof="mw:Transclusion" data-mw='{"parts":[{"template":{"target":{"wt":"padleft:xyz","function":"padleft"},"params":{"1":{"wt":"4"},"2":{"wt":"-"}},"i":0}}]}

… to:

typeof="mw:ParserFunction" data-mw='{"parts":[{"template":{"target":{"wt":"padleft","function":"padleft"},"params":{"1":{"wt":"xyz"},"2":{"wt":"4"},"3":{"wt":"-"}},"i":0}}]}

?

typeof="mw:Transclusion mw:ParserFunction" data-mw='{"parts":[{"template":{"target":{"wt":"padleft","pf":"padleft"},"params":{"1":{"wt":"xyz"},"2":{"wt":"4"},"3":{"wt":"-"}},"i":0}}]}

@Arlolra @ssastry and I discussed this and we decided for the time being at least that (a) we'd like to keep both mw:Transclusion and mw:ParserFunction in the typeof, and (b) that we'd like to change the 'function' string in the target portion to more clearly indicate the 'new' style, since the meaning of params etc has changed. WLOG we're changing that from function to pf.

Hmm, I'm not sure VE will grok typeofs with spaces meaning different values – will check with @Esanders.

Change #1115126 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] Introduce ParsoidExperimentalParserFunctionOutput config flag

https://gerrit.wikimedia.org/r/1115126

Change #1115132 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] WIP: Quick wip providing experimental parser function output

https://gerrit.wikimedia.org/r/1115132

@Jdforrester-WMF thanks for that update of the task description, that looks right to me.

Change #1115126 merged by jenkins-bot:

[mediawiki/core@master] Introduce ParsoidExperimentalParserFunctionOutput config flag

https://gerrit.wikimedia.org/r/1115126

Change #1115132 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Add experimental v3 parser function output for use by wikifunctions

https://gerrit.wikimedia.org/r/1115132

Change #1118559 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.21.0-a15

https://gerrit.wikimedia.org/r/1118559

Change #1118559 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.21.0-a15

https://gerrit.wikimedia.org/r/1118559

Change #1128404 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] ext.wikilambda.visualeditor: Add in (disabled) code for the new DOM structure

https://gerrit.wikimedia.org/r/1128404

Change #1128404 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] ext.wikilambda.visualeditor: Add in (disabled) code for the new DOM structure

https://gerrit.wikimedia.org/r/1128404

Change #1134255 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] [WIP] Move to new async Parsoid fragment provision

https://gerrit.wikimedia.org/r/1134255

Change #1134255 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Move to new async Parsoid fragment provision

https://gerrit.wikimedia.org/r/1134255

Change #1135084 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@wmf/1.44.0-wmf.24] Move to new async Parsoid fragment provision

https://gerrit.wikimedia.org/r/1135084

Change #1135084 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@wmf/1.44.0-wmf.24] Move to new async Parsoid fragment provision

https://gerrit.wikimedia.org/r/1135084

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:11:06Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:18:54Z] <jforrester@deploy1003> sync-world aborted: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]] (duration: 07m 47s)

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:19:32Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:28:20Z] <jforrester@deploy1003> sync-world aborted: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]] (duration: 08m 48s)

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:28:58Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:33:50Z] <jforrester@deploy1003> sync-world aborted: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]] (duration: 04m 52s)

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:34:35Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:41:01Z] <jforrester@deploy1003> sync-world aborted: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]] (duration: 06m 26s)

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:42:50Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]], [[gerrit:1126659|Add wikifunctionsclient dblist for production wikis that allow embedding Wikifunctions calls]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T14:49:10Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]], [[gerrit:1126659|Add wikifunctionsclient dblist for production wikis that allow embedding Wikifunctions calls]]

Mentioned in SAL (#wikimedia-operations) [2025-04-09T15:06:18Z] <jforrester@deploy1003> sync-world aborted: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]], [[gerrit:1126659|Add wikifunctionsclient dblist for production wikis that allow embedding Wikifunctions calls]] (duration: 17m 08s)

Mentioned in SAL (#wikimedia-operations) [2025-04-09T15:35:45Z] <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1135084|Move to new async Parsoid fragment provision (T373253 T388546)]], [[gerrit:1135410|Switch out various old PHP aliases to the current class names]], [[gerrit:1126659|Add wikifunctionsclient dblist for production wikis that allow embedding Wikifunctions calls]]

Closing this task as implemented. We still need to document this: T391702: Document v3 parser function DOM.

Change #1210839 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Handle deserialization of old parser functions with named arguments

https://gerrit.wikimedia.org/r/1210839

Change #1210839 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Handle deserialization of old parser functions with named arguments

https://gerrit.wikimedia.org/r/1210839

Change #1211724 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a7

https://gerrit.wikimedia.org/r/1211724

Change #1211724 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a7

https://gerrit.wikimedia.org/r/1211724

Change #1211759 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@wmf/1.46.0-wmf.4] Bump wikimedia/parsoid to 0.23.0-a7

https://gerrit.wikimedia.org/r/1211759

Change #1211759 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.46.0-wmf.4] Bump wikimedia/parsoid to 0.23.0-a7

https://gerrit.wikimedia.org/r/1211759

Mentioned in SAL (#wikimedia-operations) [2025-11-26T21:19:45Z] <cscott@deploy2002> Started scap sync-world: Backport for [[gerrit:1211759|Bump wikimedia/parsoid to 0.23.0-a7 (T204307 T373253 T410826 T410960)]], [[gerrit:1211760|Bump wikimedia/parsoid to 0.23.0-a7 (T410960)]]

Mentioned in SAL (#wikimedia-operations) [2025-11-26T21:21:56Z] <cscott@deploy2002> cscott: Backport for [[gerrit:1211759|Bump wikimedia/parsoid to 0.23.0-a7 (T204307 T373253 T410826 T410960)]], [[gerrit:1211760|Bump wikimedia/parsoid to 0.23.0-a7 (T410960)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-11-26T21:32:23Z] <cscott@deploy2002> Finished scap sync-world: Backport for [[gerrit:1211759|Bump wikimedia/parsoid to 0.23.0-a7 (T204307 T373253 T410826 T410960)]], [[gerrit:1211760|Bump wikimedia/parsoid to 0.23.0-a7 (T410960)]] (duration: 12m 38s)

Change #1213536 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a8

https://gerrit.wikimedia.org/r/1213536

Change #1213536 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a8

https://gerrit.wikimedia.org/r/1213536