Adopt a Wikipedia-trained LLM & make it learn help pages & categories to help newcomers (Community Wishlist/W442)

Stanford researchers built this neat tool trained on Wikipedia articles in so-far 25 languages: WikiChat

It's much more accurate, neutral and reliable than normal LLMs for questions where the information can be found in Wikipedia articles and, importantly, it cites Wikipedia pages that you can then go to or click on to see the specific text from the article.

However, the Wikimedia community can't decide anything regarding WikiChat's inner workings and it only works for Wikipedia article contents but not meta pages like help pages that would be of more use to many. One could ask the AI things like 'how can I create a table where one can sort a column by date?' or 'how do I make a video start at a certain time' or 'how see which files in a category have the fewest categories set' or 'I'd like to create a new category, what should I pay attention to' etc etc and get an answer/link/quote right away.

For new editors, this would often greatly decrease the barriers to (good) editing or various more difficult editing questions. Some help pages and meta help pages are hard to find and it could also learn from the talk pages and category pages etc. Note that even simply searching for help pages can be difficult for newcomers (W73 seems to be about that) – for example because the default namespace of the search is the article space. When they search for the page in Wikipedia/Commons, they can't find the page (and also one can't get questions answered via the search).

As an alternative to WikiChat, one could also consider mw:Extension:Wanda, a a MediaWiki extension that provides an AI-powered chatbot interface [for wikis]. Maybe one could also build something new.

While I think LLMs due to unreliability usually are relatively useless or overhyped, I still think this could be useful for Wikimedians. For example, this study has Building on our findings, we then propose design implications to mitigate the participation gap LLMs can lead to by fostering newcomers’ learning process and supporting editors with varying expertise in knowledge production platforms. […] In contrast, newcomers often rely on LLMs to fill in their knowledge gaps, as LLMs reduce the barriers to entry and provide a sense of guidance.

There are some further applications that could be useful:

• If it also learned about the millions of categories, it could help with finding the category the user is looking for – e.g. in W87 a user notes that searching for Albrecht Dürer engravings doesn't lead to or show c:Category:Engravings by Albrecht Dürer on Commons. There are many categories there but they are often hard to find. One needs to for example instead first see a file and then go to the categories or know the naming scheme in advance. If the LLM learned about the categories and all the redirects and linked Wikidata items, it could help – for example by the click of a button on the search results page in case didn't find what they looked for. As Juandev on Commons wrote I can imagine that today, chatbot could be trained on Commons' category names and then provide support when looking for a category using natural languages in whatever language.
• One could also ask it about proposals and ideas how to make this or that better and it would give an answer from talk pages many wishlists etc.
• One could ask things like 'has there ever been a discussion about xyz and if so summarize it'. For example, 'On Commons a user thinks Category:Street x (Location y) should be sorted into Category:Buildings in Location Y – name a policy or prior discussion about this or something similar to show they shouldn't' or 'where to find how many categories there currently are on Commons?' or 'A video shows gameplay of a card game but there is art work on the cards – is this still de minimis under copyright; please link similar deletion requests and policy pages', etc etc. There's been discussion about very many topics and these could be used to enable asking the LLM things with it giving you the link to the years-old archived discussion and a summary of it. Discussion pages include talk pages of templates, Wikidata Project Chat, the Village Pump pages, etc. It's not quick and easy for even an experienced user to find things in there – many don't even look.

• 

  Per mw:Readers/Information Retrieval Recent research shows 4–7% of all search queries on Wikipedia are phrased as natural language questions (e.g. “What”, “Who”, “How”). Question-style searches like "Who invented planes?” or queries such as “Biggest city in South America” are significantly less successful at leading users to the right article compared to simple keyword-based searches. – if e.g. the search engine detects when the query is a question or one could use something like ?question (? followed by the question text) then users seeking a particular info via querying with a question could also (and many likely more quickly) find the article section/paragraph they were looking for.
• WikiChat is currently only available from the separate site and one can't configure it to show only results from just one or a specified set of Wikipedias such as English – some Wikipedias are of much lower quality and reliability than ENWP which is one reason why one may want to do that.
• Just to reiterate, it could be a sort of mentor for newcomers that can easily and always provide simple answers/links to whatever questions they may have. Experienced and/or older users may not see the value of this but I think it can be very useful. They could still come to help desks if they know where to find them if they can't find things out using this assistant. For example, findings from Research:Progression system for newer editors include Policies, formatting, and editing rules are often confusing and not presented in one accessible place.
• Some further examples are in this issue "Enable searching meta and discussion talk pages" and I'm sure there's many more, including for admins and other very experienced users.