Content deleted Content added
WhatamIdoing (talk | contribs) |
|||
| (11 intermediate revisions by 5 users not shown) | |||
Line 60:
:I kind of agree as well. However, Google and Bing also index unreliable sources. I was initially going to put in a query "find reliable sources for $1" but then decided against it for some reason. [[User:Awesome Aasim|Awesome]] [[User_talk:Awesome Aasim|Aasim]] 14:00, 16 October 2024 (UTC)
:: That is, of course, true. However, I have never seen Google or Bing invent a very convincing-looking source out of whole <s>cloth</s> bits, creating authors who do not exist, or very complex and scholarly-looking titles that look real but do not exist (but are quite similar to pieces of other titles that do exist), and so on. If the task assigned were to add sources to an article that do not exist but would rarely get challenged, AI bots are definitely the way to go. It's quite possible vandals or lazy or clueless editors are doing this already, and it is a problem that will have to be addressed at some point. [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 15:34, 16 October 2024 (UTC)
:::When a chatbot is powered by a search engine, it is less likely to make stuff up; but that doesn't mean it doesn't pull from unreliable sources. I have tried Copilot before (not necessarily for Wikipedia tasks, but for personal tasks like clarifying math concepts) and it has not really failed me. On the other hand, ChatGPT has occasionally made stuff up, especially when it does not query from the web. [[User:Awesome Aasim|Awesome]] [[User_talk:Awesome Aasim|Aasim]] 15:45, 16 October 2024 (UTC)
:::: Imho, this page is not the place for Wikipedia editors to debate whether chatbots are more or less likely to hallucinate under this or that circumstance, and, pardon me, but anecdotal evidence about failure to fail in casual use by non-experts is close to worthless. Please use the AI discussion venues for that. Here we should debate whether a find sources module should use the results of AI, however triggered, and imho the answer to that is a slam-dunk 'no'. I will shut up now, and hopefully others will chime in. [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 16:26, 16 October 2024 (UTC)
::::: Agreed entirely with Mathglot here. [[User:Pppery|* Pppery *]] [[User talk:Pppery|<sub style="color:#800000">it has begun...</sub>]] 16:53, 16 October 2024 (UTC)
:Google's top-level "AI overview" results are rarely totally accurate, but they do give the source links, which in aggregate ends up being significantly ''more reliable'' for us than their raw top search results, which, often being shit like quora or low-quality zines, do not provide sources at all.
:I don't know if linking to another engine prompting something like ChatGPT would get better AI-enhanced results than Google or Bing in that respect, since they are putting a good deal of effort into making it give back real online-accessible sources. That seems to align with our goal here.
:Of course the other question is whether it's more enticing for the novice editor to have a shiny link saying "ChatGPT" or whatever latest AI tool is out there, instead of just clicking "Google", even if the result is the same. Also, calling exclusively the Google AI overview results does not appear possible right now as it's still considered an experimental feature, and it doesn't appear to be available in private browsing either. But that's my thought going forward. [[User:SamuelRiv|SamuelRiv]] ([[User talk:SamuelRiv|talk]]) 17:52, 16 October 2024 (UTC)
<small>'''Listed at:''' [[Wikipedia:Village pump (miscellaneous)]]. [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 16:35, 16 October 2024 (UTC)</small>
:I think that even if chatbots were 100% accurate in their output, we would still want to avoid using them for use-cases like this because of how heavily these technologies rely on Wikipedia itself as an information source. We need to stay upstream of LLMs to avoid circular referencing. <sub>signed, </sub>[[User:Rosguill|'''''Rosguill''''']] <sup>[[User talk:Rosguill|''talk'']]</sup> 16:47, 16 October 2024 (UTC)
::I wonder if with the right prompt engineering we can get these chatbots to actually spit out reliable sources. We can maybe base our entire prompt based on something like [[WP:RSPS]]. Although it probably would overfill the query parameter. [[User:Awesome Aasim|Awesome]] [[User_talk:Awesome Aasim|Aasim]] 17:25, 16 October 2024 (UTC)
::: Are you proposing to engineer a solution that operates without human intervention? Because if you are successful, you should quit your day job and launch the next AI start-up, or become CTO of one of the existing ones. [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 17:30, 16 October 2024 (UTC)
::::No, a person will still need to click on and then review what is provided, and provide follow up queries. The queries I pass into the ?q=... parameter may be a good start, but I don't think they are a good end. But us Wikipedians should know this; just as the first page of Google/Bing search results can at times be littered with stuff like [[WP:DAILYFAIL]] and [[WP:NEWSMAX]], depending on the query and depending on previous searches.
::::I have largely toyed with ChatGPT and found it is not always good. Web-based LLMs like Copilot and Gemini are a bit better, although I remember they had a bumpy start, sometimes pulling nonsense from places like Reddit and Facebook.
::::I do not agree that we should just copy and paste the exact output of an LLM. I only think AI is good to assist humans, but practically can never replace humans. [[User:Awesome Aasim|Awesome]] [[User_talk:Awesome Aasim|Aasim]] 17:38, 16 October 2024 (UTC)
:I think that including AI-oriented links is likely to cause more problems than it solves. [[User:WhatamIdoing|WhatamIdoing]] ([[User talk:WhatamIdoing|talk]]) 19:41, 17 October 2024 (UTC)
| |||