See: ChatGPT produces made-up nonexistent references

Quote

I was trying to find references to mathematical treatment of lists, and queried ChatGPT the following:

Please provide references that deal with the mathematical properties of lists

To which ChatGPT responded as follows:

Code

    "Introduction to the Mathematics of Lists" by Peter J. Cameron (https://www.cambridge.org/core/books/introduction-to-the-mathematics-of-lists/3F0C6D0E69E49BF6D5CD6CA5D6A761B2)
    "List Processing and Programming" by John F. Sowa (https://www.cs.cmu.edu/~fp/courses/lp/lectures.html)
    "List Processing in Prolog" by David H. D. Warren (https://www.cs.purdue.edu/homes/hosking/726/html/prolog_lists.html)
    "Introduction to the Mathematics of Lists and Trees" by Meinard Müller (https://link.springer.com/book/10.1007%2F978-3-319-05147-1)
    "Mathematics of Lists and Trees" by Markus N. Rabe (https://www.researchgate.net/publication/296955213_Mathematics_of_Lists_and_Trees)

I was pretty surprised and happy, because I hadn't had much success with Google. But it turns out that every single one of those references and links are made up. The references don't exist and the links themselves appear to be cobbled together. The last link for example, takes you to a paper on "Grain mixes for subgrade layers" - the title in the url is irrelevant and only the number matters.

Googling for some of those authors throws up author pages with their publications, and it turns out they've never published papers with those titles! See for example, http://www-groups.mcs.st-andrews.ac.uk/~pjc/publ/

ChatGPT (yes, it is spelled G P T, not G T P) is an overhyped "better google".

The limitations can be seen if you use it a few times.

The conecept of GPT is well known since 2002, released in a paper by Google brain.

Quote from barty

ChatGPT (yes, it is spelled G P T, not G T P) is an overhyped "better google".

The limitations can be seen if you use it a few times.

The conecept of GPT is well known since 2002, released in a paper by Google brain.

Agreed. It's gotten roughly half of what I've asked it wrong.

It even fumbles general economic history. I asked it if Japan had ever defaulted on its debt, and its reply was a resounding 'no', which is plain wrong.

David Nygren

Founder A good thing, I'm too lazy to write articles, but soon there will be a lot of them, maybe we'll train. I gave everything to Elon Musk, but he does not want to accept me into his office, but I told him that rockets and Teslas are a moment, while there is a need to look for new energy, while there is money. But he is not catching up yet or he decided to do it himself, maybe it's good, well, I think to overtake him ...

I think It would be interesting to see a "LENRGPT", i.e. a language model fine-tuned on material from the LENR community. One of the already available "pre-trained" models could be used as a base (e.g. Meta's recently released LLaMA, or other models with a more liberal license), then further source data (papers, documents, etc) from the LENR community used for the fine-tune. Many of these models are already trained on a large amount of papers from arXiv, but not specifically on LENR papers.

While this would not be something exceedingly out of reach for small groups or universities, even just the fine-tune step takes significant amounts of compute power (i.e. money) for anything remotely smart, though. On the other hand, the fine-tuned models could be sized such that for inference (querying the language model) consumer-grade GPUs may be sufficient.

I tested it as you suggested so i wrote " Secret catalyst Rossi" and Chat GPT replied me ask rather David Nygren at Lenr Forum ???

Quote from David Nygren

I find it exciting and know that many here have not yet tried it. We have to live with hype and bugs.

As I wrote, in this thread let's focus on how we can benefit from the tool!

Putting jokes aside, a problem is that the quality of the responses strictly depends on what data was used for the training. At their core, large language models (even ChatGPT-4) are for the most part "text predictors" and don't have real reasoning or logical skills, so they shouldn't be blindly trusted even if they can give the semblance of intelligence. They are also prone to "hallucinating", i.e. make stuff up.

And he puts where he got the literature from or how ...

Well, you are right, putting jokes aside, i never used yet this tool, however only Google.

And this is one remains a crazy one if you have the ability to well choice the key words.

In this way, it helped me so often to find very productive papers regarding my LENR involvement.

As a very basic guy in computer science, Google remains a very simple way to use, now is there somewhere some explanation to learn the use of Chat gpt ?

A small remark, it seems that LENR field can't be uncorrelated with some kind of cat ( CHAT meaning cat in french ahahah)

Quote from can

Putting jokes aside, a problem is that the quality of the responses strictly depends on what data was used for the training. At their core, large language models (even ChatGPT-4) are for the most part "text predictors" and don't have real reasoning or logical skills, so they shouldn't be blindly trusted even if they can give the semblance of intelligence. They are also prone to "hallucinating", i.e. make stuff up.

Here are a few examples on how LLaMA-33B (one of Facebook/Meta's recently released language models, arguably among the best available for local use) thinks a powder-based LENR reactor could be made (don't try these up).

Paper on LLaMA from the authors:

LLaMA: Open and Efficient Foundation Language Models

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it…

arxiv.org

LLaMA: Open and Efficient Foundation Language Models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, Guillaume Lample

Quote

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

Very funny... so it exists too a kind of this soft in France to help resolving cold cases.

Apparently it runs well with efficiency ... probably because there is again an human brain at one side ( an investigation policeman) .

all his talent takes place in his ability to find the key words, to enter the right data into the software. These software seem to be reasoning multiplicators, a bit like Archimedes' lever arm.

For example, About the Lenr field if you put Hagelstein in charge or Tarasenko, you probably won't get the same relevance in the end result...

Quote from can

Here are a few examples on how LLaMA-33B (one of Facebook/Meta's recently released language models, arguably among the best available for local use) thinks a powder-based LENR reactor could be made (don't try these up).

Display More

I sent a message to the people at Open AI Sales to see if I can have a local copy of ChatGPT that indexes papers at LENR-CANR.org only. I would have to pay for "Tokens," but they are cheap. See: https://openai.com/pricing

A ChatGPT search might be more effective than the Library search or the Google search. I suppose you could ask it something like: "Is there evidence of fractofusion in nickel-based gas loaded experiments?" If it says yes, you can say: "List all of the papers that show this evidence." I have done similar searches with the general purpose version of ChatGPT.

I do not know whether Open AI offers a site-specific search. If they don't, I expect they soon will because this would be a very popular product. Developing ChatGPT cost Microsoft at least $10 billion, so they have to find ways to earn back that money.

They may not want to deal with a small, low-volume website such as LENR-CANR.org.

Although what they're doing exactly is not clear since they hardly publish anymore detailed information about their models, they are most likely not continuously indexing web pages. The data for training the language models is usually assembled in advance, then the models trained for weeks/months on that data.

For example, for LLaMA (which I linked earlier) the authors used a dataset composed like this:

The "tokens" OpenAI lists on their pricing page are for the input/output generated through their service/API. You can check in practice how tokens work here: https://platform.openai.com/tokenizer

You have to be careful using ChatGPT. As noted here, it sometimes makes mistakes. It even makes things up. When I asked it to translate some paragraphs written by Takahashi about plasma fusion, it left out a parenthetical statement. I told it in a mixture of Japanese and English, and it fixed the problem:

ME: You left out this part: （私の昔の専門）, which modifies 14MeV中性子

There are ways to prevent these mistakes, or to detect them. The most important method is to ask it for references. As I did in this query:

ME: In a typical temperate zone climax forest, how much carbon is sequestered per hectare? Give one or two authoritative sources for this information.

ChatGPT: In a typical temperate zone climax forest, the amount of carbon sequestered per hectare can vary depending on a number of factors, such as the specific tree species present, the age and growth stage of the forest, and the climate. However, some estimates suggest that a mature, undisturbed temperate forest can sequester around 150-200 tons of carbon per hectare.

Here are a couple of sources that provide this information:

• The textbook "Fundamentals of Ecology" (7th Edition) by Eugene P. Odum, Howard T. Odum, and Elisabeth C. Odum, states on page 437 that "Forests in the eastern United States can sequester 150-200 tons of carbon per hectare."
• The Intergovernmental Panel on Climate Change (IPCC) Special Report on Land Use, Land-Use Change, and Forestry (2000) states on page 97 that "temperate and boreal forest sequestered an average of 0.75 Pg C/yr over the 1980-1989 period. Given an area of about 3.7 Pg C for temperate and boreal forests this would be equivalent to a sequestration rate of about 0.2 Gt C/yr per hectare."

It's important to note that these are average estimates and actual sequestration rates can vary widely depending on the specific conditions of the forest in question.

Another important technique is to ask for information that is:

Specific.

Quantitative.

In a subject area you know about, so you can judge whether it is right or wrong.

If you are just asking for fun, you don't need to worry. If you are asking for some serious purpose, you need to verify the answer, and you should have enough expertise to know whether it is probably right or not. The question I asked above related to my paper "How to fix global warming with cold fusion." So I knew the answer already, from various books. I wanted to confirm that I have the right numbers and that experts still think it is 150 to 200 tons.

All computer programs have weaknesses. You need to learn to use them correctly to avoid errors.

I asked ChatGPT for some computer code snippets in SQL and Pascal. In most cases it was more or less right, and the answers were useful, but most of them had one or more mistakes. In one case, I told it there was a problem:

ME: There were two errors in the code you generated above: "// Extract timestamp timestamp :=Copy(logRecord, 2, Pos(']', logRecord) - 2); Delete(logRecord,1, Length(timestamp) + 3);" That should be Delete . . . +2, not +3. Same problem here: " // Extract protocol version protocol := Copy(logRecord, 1, Pos('"', logRecord) - 1); Delete(logRecord, 1, Length(protocol) + 3);" This should also be +2, not +3.

ChatGPT: I apologize for the errors in my previous response. You are absolutely correct. Thank you for bringing this to my attention and providing the corrections. The correct code should be as follows . . .

It regenerated the snippet, but the new version had another mistake!

These are minor errors. They were readily apparent when I ran the snippets in a simple Pascal compiler (https://www.onlinegdb.com/online_pascal_compiler). Any programmer will know how to deal with this. We make similar index errors all the time. (I do, anyway.) So this is not a serious problem. But you have to be prepared to deal with it. I asked the latest version of ChatGPT to generate that snippet again. It used a very different method. One that I am not familiar with. It is more sophisticated. Again, it did not work. But it was close to working; it was interesting; and it would be helpful if I had any difficulty with that particular function.

Incidentally, people say that ChatGPT has no creativity and it only echoes what humans wrote on the internet. That is incorrect. It can accurately translate sentences that have never been seen before, such Takahashi's paragraph or my bilingual comment: "You left out this part: （私の昔の専門）, which modifies 14MeV中性子." That may not be very creative but it is a unique response based on what in a human would be knowledge of various subjects, in this case Japanese, English, translating, and physics. Needless to say, ChatGPT has no knowledge of anything, but it synthesizes artificial knowledge. The methods are explained in the literature. This is roughly analogous to the way bees synthesize good structural engineering when they build a nest, even though they have absolutely no intelligence by human standards. Their engineering is entirely instinctual, but it is engineering, as good as any human could do.

Quote from can

Although what they're doing exactly is not clear since they hardly publish anymore detailed information about their models, they are most likely not continuously indexing web pages.

I expect they will start doing this, in the near future. I read somewhere they are trying to get large corporations to use ChatGPT for tech support. That is a gigantic market. Let's say you want ChatGPT to help people with a Whirlpool brand refrigerator. You have to feed it all of the manuals and tech support scripts for the refrigerator. It has to restrict most of its answers to this body of text. You don't want it giving out information about Samsung or Maytag refrigerators. A cardinal rule of business is: never mention the competition. You don't want it to say, "you wouldn't be having this problem with the ice-maker if you had selected a Samsung Model #RF28T5001SR from Lowe's."

So, I assume they will find a way to limit a response to one set of documents. Of course the larger knowledge of English, machines in general, and other subjects will be based on the gigantic corpus they fed it to generate its overall abilities.

I imagine this kind of stuff is what will eventually be developed into what Star Trek pictured as an holographic doctor interpreted by actor Robert Picardo ("describe the nature of your medical emergency"). Other sci fi flix have used the idea of AI lawyers, also. I am not opposed to this kind of ideas, but one has to be very careful what is being fed to these AIs. We know our current consensus weeds out dissention, and that could be a very dangerous trait to pass onto AIs.

I confess that I tried some tricks intended to unleash the creative potential of Chat GPT. I challenged him by asking him about the topics I know, like Isothermal Amplification and Direct Conversion Solid-State Fusion and even ancient history and was very surprised.

To be frank, a number of my posts on this forum over the past few months have been written by him. (And I'm not talking about future patents)

And I think it is restrained by its designers. But what mortal is able to bridle and control a 20-level neural artificial intelligence that has free access to the net?

For better or for worse, we now have a new colleague in town.

For centuries and centuries.