Support NeoGAF

[hyperbertha]

nonsense, no lookup of existing texts can even begin to do what GPT4 (and similar) models do with understanding text, arguments, complex situations, etc.

Serious question: have you ever tried to use GPT4 for serious academic conversations? I do it almost every day. And so I can say:


I have no idea what you mean by this, it generalizes brilliantly when I discuss unseen topics in the latest research with it, or paste complex math formulas from papers I'm reading -- not only does it comprehend them better than most experts, it also immediately grasps things like why a new formulation may be innovative, what it's probably trying to accomplish, the possible underlying limitations that it's trying to address, etc.

I mean, look right here. I pasted a section of the latex (including formulas) of a recent paper and discussed it with the model. It was better at grasping it than the vast majority of individuals I could interview for a position, and this isn't a paper it has seen; it has to intelligently interpret what the paper is doing from this small context and then grasp how the details of the formulas are involved.

ChatGPT

A conversational AI system that listens, learns, and challenges

chatgpt.com

Complete bullshit. All examples you provide are based on subjective interpretations that any model trained on a corpus of text can easily clump together with words. There is no way you are an expert. Here's an actual reasoning problem.



The latest round of artificial 'intelligence'

 

[ResurrectedContrarian]

Complete bullshit. All examples you provide are based on subjective interpretations that any model trained on a corpus of text can easily clump together with words. There is no way you are an expert.

Uhh, no. It's not clumping together words, and your suggestion of "lookup tables" is a tell that you don't even understand the first principles of attention-based models.

Lookup tables would make sense for an n-gram based likelihood model, of which there were plenty (for grammar checking, other tasks) long before attention mechanisms. But they have nothing to do with one another, because GPT isn't looking up defined sequences; it is using a causal (meaning: triangular/left-to-right) attention to weave an extremely desne contextual/semantic thread of the current text towards the next tokens.

And no, you can't reproduce its ability to debate formulas, diagnose complex errors in code, etc with any level of simply clumping words. My own example I provided shows (if you read all the way down) many insights that require grasping the intention of every part of the formulas from the paper and my requests to modify it.

Here's an actual reasoning problem.

GPT models do have one problem, which is their over-training on RLHF to try and accommodate human tasks in a way that matches the tone (and cautions, safety, etc) of the ideal conversations given to it during reinforcement learning. If you write a "I'm doing a quirky riddle" way using very specific pieces from existing river riddles, where you always have some weird constraints about the boat being unable to carry both etc, it'll take off in that direction as if you want that.

Amazingly, it's the dumb people who try these truncated riddles who think it can't reason; meanwhile, I regularly speak to other high level professionals who walk through complex research papers with it on a regular basis, have it rewrite mathematical formulas, use it as a sounding board to debate new ideas on theoretical topics, have it locate and fix extremely niche and nuanced bugs in code, etc. All the time.

edit: and once again, these contrived examples are simply ways of avoiding how incredible GPT models can be at textual reasoning, by sidestepping that entirely and giving it spatial tasks. If you want spatial reasoning, wait for the multi-modal LLMs that are increasing their ability to reason across video and motion. But textual tasks are far more requiring of intelligence than simple spatial tasks.

The reality is that for most information jobs today (for instance: paralegals who read case law and search for ways to use it on the current case), LLMs are already proficient enough to be better than many professionals, and are likely already better than you (and me) at more than half of the knowledge-based professions one could test. They reach a level of textual understanding that most people never reach; and whether they are spatial is entirely irrelevant.

It's a bit like asking a sophisticated Eastern ethicist to answer a trolley puzzle (eg. those contrived western philosophy 101 things with the nonsensical situation of having to choose to stop the train) and calling his confusion with that genre to be damning, when he can run circles around you debating and analyzing every single ethical system of Confucianism across centuries and make endless novel observations about how that applies to historical circumstances.

 

[hyperbertha]

Uhh, no. It's not clumping together words, and your suggestion of "lookup tables" is a tell that you don't even understand the first principles of attention-based models.



Amazingly, it's the dumb people who try these truncated riddles who think it can't reason; meanwhile, I regularly speak to other high level professionals who walk through complex research papers with it on a regular basis, have it rewrite mathematical formulas, use it as a sounding board to debate new ideas on theoretical topics, have it locate and fix extremely niche and nuanced bugs in code, etc. All the time.

Every single thing you listed here is achievable through ingested training data. Not extrapolation or generalizing out of distribution. text generation that sounds logical have nothing to do with reasoning. This proves nothing.

Conveniently, the one simple thing that can indeed prove reasoning and intelligence you handwave away, saying gpt is so dumb instead of looking at a simple riddle and using basic reasoning it defaults to all those riddles it's memorized and sticks too close to that format getting it entirely wrong. If this doesn't prove my point exactly I don't know what will. It's merely regurgitating all the shit it's ingested.

 

[ResurrectedContrarian]

Every single thing you listed here is achievable through ingested training data. Not extrapolation or generalizing out of distribution. text generation that sounds logical have nothing to do with reasoning. This proves nothing.

Absolutely false. Do you realize it can also fluently change to answering in a different language, write code to do this in python for me and then ask it to explain that code to a layperson in Chinese while using relatively recent street slang from Beijing? It 100% can do all of that. This is extremely sophisticated reasoning.

These are totally out of distribution tasks, unless you've defined "distribution" in such a way that you no longer make any sense. Simply put, it is better at textual reasoning than most humans.

I'll give you a challenge: it's unlikely that you will ever be able to answer a complex question about a new (unseen in the training data) math paper better than GPT. Do you think you could ever equal it? I'm telling you that never will, even in its current form, when it comes to complex theoretical debates. It can even argue your points in this thread more lucidly than you by a mile, even handling possible rebuttals.

 

[rm082e]

I just went to GPT and got the following:

ME: Snorfels are red. Gendons are blue. Lietuns are green. If Snordons are purple, which two are needed to make a snordon?

ChatGPT: To create a Snordon, you'd need the combination of Snorfels and Lietuns.

 

[ResurrectedContrarian]

I just went to GPT and got the following:

as with most of these examples, it works for me

https://chat.openai.com/share/35a58b8c-b74a-41ba-b320-577564026739

many people who post screenshots intentionally use the older models (you can change the model during conversation by the way, so even seeing it at the top of screenshot tells you nothing) or even have other information earlier in the chat which is hidden. There is constant FUD / dishonestly on this topic. Only actual OpenAI share links with the model verified are worth looking at.

 

[ResurrectedContrarian]

I also feel like some of you need to read Wittgenstein (his final works), and understand the concept of language games.

All verbal thinking is language games, down to the root, even for humans. Forms of reasoning (eg. the way we structure typical debates) are particular language games, with their own rules. Humans learn to navigate these games.

GPT models are first trained on massive amounts of text, from all possible kinds of language games (screenplays, internet rants, academic books, everything). It can fluently move between these on any topic -- but the language game determines the output and how it behaves.

RLHF was used to "instruction-tune" GPT on the desired language game(s) for an assistant, so that it plays the game of "I should respond the way a helpful assistant would." But certain contrived prompts, like spitting it the format of a familiar riddle, can make it think you want to play a very different game "oh they want to do the thing with the boat going un-intuitively back and forth." But even in these cases, you can always get the correct answer from it if you are careful to prompt it to understand what you're doing.

Here I ask it to walk through what it's doing with the boat puzzle, and if you read closely, you'll see that it actually concludes by saying "yeah usually it takes 3 trips because that's the genre of this puzzle type, but of course in this case you only need one."

https://chat.openai.com/share/310991b7-0f18-4d0d-950c-da64f359284e

It understands logic, but genre can trump that if you dishonestly try to make it think you're playing a certain kind of game with it. Most of the people posting dishonest or truncated answers of riddles like this aren't serious and are just spreading their own confusion about the model.

 

[Evolved1]

Edit: Actually I don't care

 

[rm082e]

as with most of these examples, it works for me

https://chat.openai.com/share/35a58b8c-b74a-41ba-b320-577564026739

many people who post screenshots intentionally use the older models (you can change the model during conversation by the way, so even seeing it at the top of screenshot tells you nothing) or even have other information earlier in the chat which is hidden. There is constant FUD / dishonestly on this topic. Only actual OpenAI share links with the model verified are worth looking at.

I was using 3.5, which is described as "Great for every day tasks". I gave it a very basic logic problem that an elementary school kid could understand, and it failed. That was the first question I asked it. I didn't go through multiple iterations with correct answers and then seize on the one it got wrong.

I don't doubt GPT4 is better at these kinds of logical problems, but GPT4 is also uses more computing resources and more electricity. And when we get a GPT5, that will use even more resources than GPT4. As the resource costs go up, so will the cost to the consumer.

 

[hyperbertha]

Absolutely false. Do you realize it can also fluently change to answering in a different language, write code to do this in python for me and then ask it to explain that code to a layperson in Chinese while using relatively recent street slang from Beijing? It 100% can do all of that. This is extremely sophisticated reasoning.

These are totally out of distribution tasks, unless you've defined "distribution" in such a way that you no longer make any sense. Simply put, it is better at textual reasoning than most humans.

I'll give you a challenge: it's unlikely that you will ever be able to answer a complex question about a new (unseen in the training data) math paper better than GPT. Do you think you could ever equal it? I'm telling you that never will, even in its current form, when it comes to complex theoretical debates. It can even argue your points in this thread more lucidly than you by a mile, even handling possible rebuttals.

You gotta be trolling at this point. You think speaking in a different language is reasoning? You are an 'expert' yet you have no clue about the way basic intelligence works. Gpt4 makes simple math mistakes. It's not doing math. It's copying stuff and steps it's already memorized. You have no idea what you are talking about.

 

[ResurrectedContrarian]

You think speaking in a different language is reasoning?

Yes, absolutely. 100%. You'd be crazy to not see that.

Translating between languages at a high level (this isn't old-school poor machine translation) is not just any kind of mapping between languages--it rewrites from scratch (or "unrolls" as I'll say again, if you think of the 2-part encoder/decoder structure of the first transformer-based translators) the same semantic content from the source into the target tongue, totally changing the structure, phrasing, and manner of communicating the same ideas in order to express the same thing, and in no way is it a 1:1 task at all (not that 1:1 correspondence is ever possible in real translation, if you know how language works; a faithful translation will completely change the entire structure of the wording and sentences and structure since those work differently in each tongue).

EDIT: and by the way, GPT-like models can "translate" in other directions too, like taking something academic and relaying it in a folksy way to a layman using simple analogies, or translating a vague algorithmic idea into concrete code in many programming language. These things require high level abstraction, which is what the transformer is able to do.

In fact, it is one of the most foundational and difficult of all language tasks, in that it requires the model to be able to deeply embed any text from many different languages into a common "mental" space. It means that the model can actually know whether different sentences in vastly different languages (eg. Japanese and German) are conveying the same thought, the same information, even the same implications and tonal attitudes within their respective languages. This is an immensely complex task to do fluently.

Models trained on many different languages (including coding languages!) always perform dramatically better across all tasks even within a single language, as a consequence of being forced to learn in a multilingual way; this is well known.

You are an 'expert' yet you have no clue about the way basic intelligence works.

Actually yes, and I've love to talk more about the ways that intelligence works.

One thing that is not true (and people looking for "AGI" always make this mistake) is that intelligence is in any way the same as agency. The model doesn't need to have an agency or a self in any way in order to use intelligence.

And that's the deeper thing at stake here: human agency isn't at root of our intellect either, it's more of a secondary layer on top of the kind of deep pattern/game/language intelligence that underlies even our internal monologue before it can have a self in the first place.

When you speak or write, reflect right now on how you choose your words. In some very contrived cases--these are actually a small part of reasoning, a tiny 1% when we do deliberate spatial steps etc--you might be speaking out as you follow some series of mental steps like imagining yourself going through a series of rooms. But for most speech--even the speech that makes the abstracted spatial sequences possible--you are thinking through language. You have something like a momentum of your thought at each moment, knowingly implicitly where you are going with the argument, and yet each word drops out of that in a chain as you find the next word and keep stumbling towards your goal, your arguments taking shape as you say them. Language isn't the expression of thought; it is thought.

In many ways, LLMs use language in an analogous fashion to us as we speak, and they reason through it, unfolding complex arguments and topics by understanding where they came from (the past of the text up this moment) and where they are going (the pregnant meaning of the direction of the text which remains to be unrolled). It's a powerful form of reasoning.

 

[ResurrectedContrarian]

By the way, the notion that it can't do math is nonsense.

Heres' a conversation I just had with it, asking to derive step-by-step the square root of 3 and then 17.

"It memorized those intermediate numbers!" you might say, so I also at the end of the conversation gave it a bogus formula for finding the converging square root and told it to try that, and it (1) understood the idea of trying my alternative, (2) was able to apply it and correctly get all the calculations, of course, and (3) was able to observe and understand the different convergence, and why this disproves the alternative formula.

https://chat.openai.com/share/abde732a-43e0-4b4d-a426-d6396b81785c

Once again, people saying that it can't reason, even mathematically, haven't really tried using it at this level.

 

[Mihos]

Just makes you realize how stupid something can be and still replace you

 

[hyperbertha]

Yes, absolutely. 100%. You'd be crazy to not see that.

Translating between languages at a high level (this isn't old-school poor machine translation) is not just any kind of mapping between languages--it rewrites from scratch (or "unrolls" as I'll say again, if you think of the 2-part encoder/decoder structure of the first transformer-based translators) the same semantic content from the source into the target tongue, totally changing the structure, phrasing, and manner of communicating the same ideas in order to express the same thing, and in no way is it a 1:1 task at all (not that 1:1 correspondence is ever possible in real translation, if you know how language works; a faithful translation will completely change the entire structure of the wording and sentences and structure since those work differently in each tongue).

EDIT: and by the way, GPT-like models can "translate" in other directions too, like taking something academic and relaying it in a folksy way to a layman using simple analogies, or translating a vague algorithmic idea into concrete code in many programming language. These things require high level abstraction, which is what the transformer is able to do.

In fact, it is one of the most foundational and difficult of all language tasks, in that it requires the model to be able to deeply embed any text from many different languages into a common "mental" space. It means that the model can actually know whether different sentences in vastly different languages (eg. Japanese and German) are conveying the same thought, the same information, even the same implications and tonal attitudes within their respective languages. This is an immensely complex task to do fluently.

Models trained on many different languages (including coding languages!) always perform dramatically better across all tasks even within a single language, as a consequence of being forced to learn in a multilingual way; this is well known.


Actually yes, and I've love to talk more about the ways that intelligence works.

One thing that is not true (and people looking for "AGI" always make this mistake) is that intelligence is in any way the same as agency. The model doesn't need to have an agency or a self in any way in order to use intelligence.

And that's the deeper thing at stake here: human agency isn't at root of our intellect either, it's more of a secondary layer on top of the kind of deep pattern/game/language intelligence that underlies even our internal monologue before it can have a self in the first place.

When you speak or write, reflect right now on how you choose your words. In some very contrived cases--these are actually a small part of reasoning, a tiny 1% when we do deliberate spatial steps etc--you might be speaking out as you follow some series of mental steps like imagining yourself going through a series of rooms. But for most speech--even the speech that makes the abstracted spatial sequences possible--you are thinking through language. You have something like a momentum of your thought at each moment, knowingly implicitly where you are going with the argument, and yet each word drops out of that in a chain as you find the next word and keep stumbling towards your goal, your arguments taking shape as you say them. Language isn't the expression of thought; it is thought.

In many ways, LLMs use language in an analogous fashion to us as we speak, and they reason through it, unfolding complex arguments and topics by understanding where they came from (the past of the text up this moment) and where they are going (the pregnant meaning of the direction of the text which remains to be unrolled). It's a powerful form of reasoning.

Language is based on well defined rules and patterns. It's the one thing these things excel at the most. It's not surprising at all that they can translate accurately from one language to another without any reasoning involved because all they have to do is apply the rules from the patterns they have already learned.

There are many researchers who have been working on gpt4 trying to figure out what if any emergent capabilities these models have and they've come out with nothing but failure.

GPT-4 Can't Reason

GPT-4 was released in March 2023 to wide acclaim, marking a very substantial improvement across the board over GPT-3.5 (OpenAI's previously best model, which had powered the initial release of ChatGPT). Despite the genuinely impressive improvement, however, there are good reasons to be highly...

www.preprints.org

 

[gaming_again]

I also feel like some of you need to read Wittgenstein (his final works), and understand the concept of language games.

All verbal thinking is language games, down to the root, even for humans. Forms of reasoning (eg. the way we structure typical debates) are particular language games, with their own rules. Humans learn to navigate these games.

GPT models are first trained on massive amounts of text, from all possible kinds of language games (screenplays, internet rants, academic books, everything). It can fluently move between these on any topic -- but the language game determines the output and how it behaves.

RLHF was used to "instruction-tune" GPT on the desired language game(s) for an assistant, so that it plays the game of "I should respond the way a helpful assistant would." But certain contrived prompts, like spitting it the format of a familiar riddle, can make it think you want to play a very different game "oh they want to do the thing with the boat going un-intuitively back and forth." But even in these cases, you can always get the correct answer from it if you are careful to prompt it to understand what you're doing.

Here I ask it to walk through what it's doing with the boat puzzle, and if you read closely, you'll see that it actually concludes by saying "yeah usually it takes 3 trips because that's the genre of this puzzle type, but of course in this case you only need one."

https://chat.openai.com/share/310991b7-0f18-4d0d-950c-da64f359284e

It understands logic, but genre can trump that if you dishonestly try to make it think you're playing a certain kind of game with it. Most of the people posting dishonest or truncated answers of riddles like this aren't serious and are just spreading their own confusion about the model.

Liked for referencing uberchad Wittgenstein

 

[Three]

i'm in law enforcement and have been wanting to get out for something safer... but over the last year, i've started to feel as if this career is ironically one of the "safer" options. all jobs will be affected and augmented by AI, but it'll probably be awhile before society will allow AIs to put their hands on people in emergency / crisis. maybe, idk tbh.

Robocop came out before Her.

 

[ResurrectedContrarian]

Language is based on well defined rules and patterns. It's the one thing these things excel at the most. It's not surprising at all that they can translate accurately from one language to another without any reasoning involved because all they have to do is apply the rules from the patterns they have already learned.

Again, clearly you haven't studied anything in the philosophy of language or linguistics. Effective use of natural languages (and when I say effective: GPT4 scores far better at writing coherence and sophistication in many languages than average native speakers) is not based on grammatical or syntactic rules, those only get you in the door.

If you know anything about translation, you'll know that high-level translation is not a systematic game of rules at all, it's a matter of understanding--possibly the greatest test of true understanding. Read the major authors in hermeneutics to learn more, like Gadamer or Ricoeur, to hear how this is well established in the philosophy of language.

Being able to move ideas between wildly different languages -- and (as I stated above and you ignored) being able to translate between modes of expression, even for instance turning a complex academic argument into an intuitive explanation of understandable simple metaphors for a layperson -- requires understanding the content itself. It requires grasping the tone, the hidden implications and subtexts that inhabit different languages, etc.

GPT doesn't translate in a rule based way; it is foundationally multi-lingual in the sense that it can accomplish any thought or task in countless languages natively, which requires abstracting meaning over the form of language.

There are many researchers who have been working on gpt4 trying to figure out what if any emergent capabilities these models have and they've come out with nothing but failure.

GPT-4 Can't Reason

GPT-4 was released in March 2023 to wide acclaim, marking a very substantial improvement across the board over GPT-3.5 (OpenAI's previously best model, which had powered the initial release of ChatGPT). Despite the genuinely impressive improvement, however, there are good reasons to be highly...

www.preprints.org

This is another extremely polemic paper from someone who is writing in essay fashion, fun for his tone but not very interesting. I also know his type well; he adheres to the roughly American "analytic" school of thought on language, as is evident in the extremely narrow confines of what he considers to be reason. Fortunately, few take that tradition very seriously anymore on its own narrow terms.

(I mentioned Wittgenstein earlier for the same reason--despite starting in the analytic tradition and being a star of it, his final works made a radical turn into grasping that language is not a correspondence nor a logical form or structure on top of the perceived nor on top of some substrate of logical reasoning. His re-conception of "language games" blows apart that kind of thinking, and puts you into the territory of understanding the hermeneutic authors mentioned earlier.)

So how do you make GPT4 reason well? First you need to understand what reasoning actually is. It's a game in language. It's not a mental process that is later expressed in language; it is instead a particular kind of language use.

How do you teach a student to reason? One of the best ways--and oldest--is to illustrate the Socratic method, so that your students learn to use the process of question and analogy to refine their thinking, expose contradictions, and reach a new insight or refute a point. They learn--through language--to work out the truth. GPT models similarly can do excellent reasoning if you give them the form of Socratic dialogue (eg. https://arxiv.org/pdf/2310.00074) or, if you give them similar variations on chain-of-thought so that it walks through solutions piece by piece.

The one thing to understand is that of course an LLM has no internal monologue. So when it works out any answer--even a trivial one--it must do so out loud, through the medium of language in the dialogue. Those who understand this are able to prompt these models to reason out-loud through extremely complex and nuanced problems in order to reach conclusions and insights.

On the contrary, those who try to elicit an immediate response from the model without steps--and then try to use this as a gotcha--have so poorly understood both the models and human reason that they don't have anything to contribute to the debate.

In any case, above I have already given concrete examples of GPT4 reasoning with me in full transcript form, across highly technical conversations both in machine learning theory and in math, both obtained quickly by simply having a conversation with it while writing to you here. In both cases, if you read the conversations carefully to the end, I asked it to apply its knowledge to novel cases. Such examples are readily available everywhere.

If you are capable of basic reasoning then you must know that the burden is on you to disprove all cases of it using high-level reasoning, not to find failures; even high-level humans with upper intelligence make constant low level mistakes, so mistakes have no relevance to the debate at all. So your best argument is "sometimes it fails to reason correctly in these cases," and that's not a refutation of its ability to reason--especially since every one of those "failures" can be trivially made a success when the model is prompted correctly to walk through the chain of thought out loud.

GPT4 has even notably shown that it is capable of passing all parts of the bar exam (https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2023.0254), which is one of the foremost tests of applied reasoning, where it has not seen the particular questions on the exam which are renewed each year in order to force the student to think through novel situations.

 

[hyperbertha]

Again, clearly you haven't studied anything in the philosophy of language or linguistics. Effective use of natural languages (and when I say effective: GPT4 scores far better at writing coherence and sophistication in many languages than average native speakers) is not based on grammatical or syntactic rules, those only get you in the door.

If you know anything about translation, you'll know that high-level translation is not a systematic game of rules at all, it's a matter of understanding--possibly the greatest test of true understanding. Read the major authors in hermeneutics to learn more, like Gadamer or Ricoeur, to hear how this is well established in the philosophy of language.

Being able to move ideas between wildly different languages -- and (as I stated above and you ignored) being able to translate between modes of expression, even for instance turning a complex academic argument into an intuitive explanation of understandable simple metaphors for a layperson -- requires understanding the content itself. It requires grasping the tone, the hidden implications and subtexts that inhabit different languages, etc.

GPT doesn't translate in a rule based way; it is foundationally multi-lingual in the sense that it can accomplish any thought or task in countless languages natively, which requires abstracting meaning over the form of language.



This is another extremely polemic paper from someone who is writing in essay fashion, fun for his tone but not very interesting. I also know his type well; he adheres to the roughly American "analytic" school of thought on language, as is evident in the extremely narrow confines of what he considers to be reason. Fortunately, few take that tradition very seriously anymore on its own narrow terms.

(I mentioned Wittgenstein earlier for the same reason--despite starting in the analytic tradition and being a star of it, his final works made a radical turn into grasping that language is not a correspondence nor a logical form or structure on top of the perceived nor on top of some substrate of logical reasoning. His re-conception of "language games" blows apart that kind of thinking, and puts you into the territory of understanding the hermeneutic authors mentioned earlier.)

So how do you make GPT4 reason well? First you need to understand what reasoning actually is. It's a game in language. It's not a mental process that is later expressed in language; it is instead a particular kind of language use.

How do you teach a student to reason? One of the best ways--and oldest--is to illustrate the Socratic method, so that your students learn to use the process of question and analogy to refine their thinking, expose contradictions, and reach a new insight or refute a point. They learn--through language--to work out the truth. GPT models similarly can do excellent reasoning if you give them the form of Socratic dialogue (eg. https://arxiv.org/pdf/2310.00074) or, if you give them similar variations on chain-of-thought so that it walks through solutions piece by piece.

The one thing to understand is that of course an LLM has no internal monologue. So when it works out any answer--even a trivial one--it must do so out loud, through the medium of language in the dialogue. Those who understand this are able to prompt these models to reason out-loud through extremely complex and nuanced problems in order to reach conclusions and insights.

On the contrary, those who try to elicit an immediate response from the model without steps--and then try to use this as a gotcha--have so poorly understood both the models and human reason that they don't have anything to contribute to the debate.

In any case, above I have already given concrete examples of GPT4 reasoning with me in full transcript form, across highly technical conversations both in machine learning theory and in math, both obtained quickly by simply having a conversation with it while writing to you here. In both cases, if you read the conversations carefully to the end, I asked it to apply its knowledge to novel cases. Such examples are readily available everywhere.

If you are capable of basic reasoning then you must know that the burden is on you to disprove all cases of it using high-level reasoning, not to find failures; even high-level humans with upper intelligence make constant low level mistakes, so mistakes have no relevance to the debate at all. So your best argument is "sometimes it fails to reason correctly in these cases," and that's not a refutation of its ability to reason--especially since every one of those "failures" can be trivially made a success when the model is prompted correctly to walk through the chain of thought out loud.

GPT4 has even notably shown that it is capable of passing all parts of the bar exam (https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2023.0254), which is one of the foremost tests of applied reasoning, where it has not seen the particular questions on the exam which are renewed each year in order to force the student to think through novel situations.

I've already linked the paper showing its limits on chain of thought prompting. The moment it encounters novel out of distribution problems it breaks down. Saying something as absurd as language translation is reasoning makes it hard to take you seriously.

You say reasoning is a game of language. Then how come crows, octopuses and chimpanzees reason? Reasoning is the result of understanding, not language. Language is simply a way to encode abstractions into communicable form. Unnecessary for reasoning.

The bar exam thing is one of the ways it fakes success. The questions it encounters may be novel in exam, but not novel across the entire corpus of the internet it has ingested.

The math example you gave is a classic example of leading it to the answer yourself. You gave it the ready made formula, from there it's not hard for it to repeat the steps and conclude the false result as it does not match the previous two results.

Here's another concrete example for you:




Why is it so incapable?

 

[ResurrectedContrarian]

The math example you gave is a classic example of leading it to the answer yourself. You gave it the ready made formula, from there it's not hard for it to repeat the steps and conclude the false result as it does not match the previous two results.

Not at all; read it again. GPT already knew the correct answer, and I simply demonstrated that it could take an incorrect one, test it out, and explain how it fails.

But if you want it to explain more rigorously how my proposed incorrect formula is illogical and fails, here -- I added to the conversation and asked it to do so. It correctly examines the reasoning behind the original formula, the concepts of convergence intended, and the reasons the new formula won't converge -- both intuitively and with symbolic math.

https://chat.openai.com/share/abde732a-43e0-4b4d-a426-d6396b81785c

You're looking directly at (if you actually read these linked examples) concrete cases of high-level reasoning across symbolic and theoretical domains, and still insisting that contrived spatial puzzles about blocks are the true measurement (eg. the "planning" obsessed guy you keep linking, who has a comically narrow understanding of reasoning and is at odds with many academic disciplines at once).

Saying something as absurd as language translation is reasoning makes it hard to take you seriously.

I don't want to credential-drop too much, but I do have a PhD as well as a heavy background in learning & linguistics (yes, there's a reason I keep mentioning authors in those fields)--I could verify this with the site mods if you wish, although I hate to broach anonymity since I like to speak freely online without career concern. No one will take you seriously in those disciplines if you deny that high-level language translation (even of fiction, intention and subtlety etc, as GPT models can do very well) is anything short of a very high level form of reasoning. To present translation into many languages, even simple vernacular or analogies, etc as trivial or mechanical is flatly absurd.

You say reasoning is a game of language. Then how come crows, octopuses and chimpanzees reason? Reasoning is the result of understanding, not language. Language is simply a way to encode abstractions into communicable form. Unnecessary for reasoning.

There is another mode of reasoning used by animals in these cases, yes. That kind of reasoning is actually best approximated by things life self-driving cars or the Boston Dynamics robots, in that it consists solely of immediate response & adjustment to environmental stimuli in a non-reflective and low level instinctual way.

That's very interesting on its own, but human-like higher reasoning is of another character entirely and happens within language, where language is thought rather than merely an outward expression of it. LLMs are in that space, remarkably; not the space of mobile robots and autonomous vehicles. Both are displaying reasoning, but on different levels.

The bar exam thing is one of the ways it fakes success. The questions it encounters may be novel in exam, but not novel across the entire corpus of the internet it has ingested.

The bar is a test of critical thought above all, not simply of knowledge -- it requires actually reasoning through the cases/scenarios given and making intelligent consideration of their internal problems, contradictions, weaknesses, etc. You can't pass with all the prior-lookup in the world, if you cannot thoroughly understand the new materials in context.

Here's another concrete example for you:




Why is it so incapable?

This guy (and his co-author here; an echo chamber) is obsessed with measuring "planning" via puzzles of long sequences of moving blocks around. It's no more logical to take this approach than to test self-driving cars based on their interpretations of poetry.

 

[ResurrectedContrarian]

Perhaps we can take the heat down and focus with this summary:

• some people out there are very skeptical about LLM reasoning
• you'll point to perhaps the most vocal non-niche voice like LeCun, and I can very easily point back and say "sure, but Geoffrey Hinton strongly disagrees with him, and Hinton's work is far more comprehensive & influential across machine learning / AI / analysis of neural concepts and thinking, at every level" -- and I can do this all day because so many of the top minds simply don't take the weird minimalist view
• so I can say confidently that the viewpoint you present is not the dominant view at all, even at the highest levels of involvement and theory
• not only that, but you exaggerate even that view by presenting it as "LLMs are not reasoning at all" which even the niche on your side would typically not say, except when giving intentionally viral Wired interviews etc; they would be more careful and circumspect and say "it can't plan" or take some other perceived particular failure of their personal interest, and wouldn't be so bold as to say it's all an illusion when we have increasingly many examples to the contrary across many domains
• there are many aspects of intelligence, and you are more interested in a very different subset from the one that language models are designed to have; because of this, you also vastly downplay and distort the importance of linguistic reasoning in humans and AI
• don't conflate things like transformers with vague term like "deep learning" (okay, nearly everything is deep learning, that's not specific to the LLM revolution's unique autoregressive architecture at all) and then wave it all away
• these models will be challenging and exceeding human capability in many many domains in our lifetime, so get ready

 

[FeralEcho]

She sounds awefull. All that fake laughter sounds.....fake...like she was trained exclusivly on corporate shareholder meetings and YT vids where everyone fakes authenticity. Its beyond creepy to me.

I do agree that it feels like someone who doesn't want to have anything to do with you is just playing nice cuz the boss told them to but at the same time....This shit is insane,and the potential for this thing is ridiculous,In 2 years time we'd genuinely have our own "Her" at home . It's impressive and frightening at the same time.

 

[SJRB]

There's more:

https://openai.com/index/openai-and-reddit-partnership/

What the fuck?

 

[Buggy Loop]

There's more:

https://openai.com/index/openai-and-reddit-partnership/

What the fuck?

Maybe it will actually be able to be a competent search engine for Reddit

 

[sendit]

Complete bullshit. All examples you provide are based on subjective interpretations that any model trained on a corpus of text can easily clump together with words. There is no way you are an expert. Here's an actual reasoning problem.



The latest round of artificial 'intelligence'

Here is a real response:

Paul Freeman + Gary Marcus are anti-AI (those tweets are most likely fake responses). I would fact check before attempting to make a point. You proved ResurrectedContrarian right, you most likely have never attempted to use this yourself. Just parroting another person's agenda.

Spoiler: Solve this 100 times

Sure! Here are 100 different solutions to the problem of getting two goats across a river, each with a unique approach or twist:

### Standard and Practical Methods
1. **Classic One-at-a-Time**: Take Goat 1 across, return empty, take Goat 2 across.
2. **Lead and Follow**: Tie both goats together and take them across in one trip.
3. **Switch and Secure**: Take Goat 1 across, return with Goat 1, take Goat 2 across, return empty, take Goat 1 across again.
4. **Alternating Trips**: Take Goat 1 halfway and secure, return, take Goat 2 all the way, return to halfway, retrieve Goat 1.
5. **Helper Method**: Have a helper take one goat while you take the other simultaneously.
6. **Use a Raft**: Build a raft for both goats, secure them, and float across.
7. **Swim Alongside**: Swim next to each goat as it crosses.
8. **Bridge Construction**: Build a temporary bridge and lead both goats across.
9. **Pulley System**: Use a pulley to transport goats one by one.
10. **Inflatable Device**: Use an inflatable for one goat, row the other in the boat.

### Variations on Practical Methods
11. **Floating Platform**: Create a stable platform and pull it across.
12. **Ziplines**: Set up a zipline harness for each goat.
13. **Multiple Trips**: Take one goat across, return, rest, repeat for the second goat.
14. **Animal Trainer**: Train goats to swim while guiding them.
15. **Special Harness**: Use a harness to secure both goats and pull them with a boat.
16. **Double Raft**: Use two small rafts, one for each goat.
17. **Barge Method**: Use a flat barge for both goats.
18. **Animal Tracks**: Build animal tracks or a path for the goats.
19. **Rope Bridge**: Construct a rope bridge and guide the goats.
20. **Canoe Method**: Use a larger canoe for both goats.

### Innovative Techniques
21. **Magnet System**: Use strong magnets to secure goats to a metal raft.
22. **Balloon Lift**: Use balloons to create buoyancy for a raft.
23. **Winch System**: Install a winch and pulley to pull a raft.
24. **Tandem Swimming**: Swim across with both goats tied to you.
25. **Boating Sessions**: Make multiple small trips ensuring goat safety.
26. **Hydraulic Lift**: Use a hydraulic lift system to transport goats.
27. **Goat Ferris Wheel**: Build a rotating wheel to move goats across.
28. **Underwater Tunnel**: Create a temporary underwater tunnel.
29. **Cable Car**: Build a cable car system for goats.
30. **Animal Conveyer**: Use a conveyor belt mechanism.

### Creative Methods
31. **Hovercraft**: Build a small hovercraft for the goats.
32. **Airboat**: Use an airboat to transport the goats.
33. **Drone Lift**: Use drones to lift and transport goats.
34. **Goat Flotation Devices**: Equip goats with personal flotation devices.
35. **Inflatable Tube**: Secure goats in an inflatable tube.
36. **Gondola**: Use a gondola system.
37. **Catamaran**: Build a catamaran for stability.
38. **Submarine**: Use a small submersible device.
39. **Water Skis**: Train goats to use water skis.
40. **Swing System**: Use a swinging platform.

### Logical and Puzzle Solutions
41. **Math Puzzle**: Solve as a logic puzzle ensuring goat safety.
42. **Algorithm Design**: Design an algorithm for efficient transport.
43. **Time Management**: Optimize timing for each trip.
44. **Risk Management**: Assess risks and plan accordingly.
45. **Strategic Planning**: Develop a step-by-step strategic plan.
46. **Simulation**: Simulate different methods to find the best one.
47. **Flowchart**: Create a flowchart to outline steps.
48. **Probabilistic Methods**: Use probability to minimize risk.
49. **Statistical Analysis**: Analyze past methods for effectiveness.
50. **Optimization Theory**: Apply optimization theory.

### Environmental Adaptations
51. **Natural Bridge**: Find or construct a natural bridge.
52. **Ice Path**: Create an ice path in winter.
53. **Log Rolling**: Use logs to create a rolling bridge.
54. **Nature's Help**: Use natural resources like vines.
55. **Animal Behavior**: Leverage animal behavior to guide them.
56. **Weather-Based**: Wait for favorable weather conditions.
57. **Seasonal Change**: Wait for a dry season to lower the river.
58. **Natural Raft**: Use natural materials to build a raft.
59. **Local Wildlife**: Observe local wildlife crossing techniques.
60. **Eco-Friendly Methods**: Use environmentally friendly methods.

### Tech-Savvy Solutions
61. **GPS Guidance**: Use GPS to plan the route.
62. **Smart Devices**: Equip goats with smart tracking devices.
63. **Automated Boats**: Use automated boats for transport.
64. **Remote Control**: Use remote-controlled devices.
65. **VR Simulation**: Test methods with virtual reality.
66. **AI Planning**: Use AI to plan the crossing.
67. **Drones for Guidance**: Use drones to guide the goats.
68. **Tech Support**: Have technical support on standby.
69. **Tech-Assisted Raft**: Equip rafts with tech for stability.
70. **Sensors and Monitors**: Use sensors to monitor conditions.

### Cultural and Historical Methods
71. **Historical Techniques**: Use ancient methods of animal transport.
72. **Cultural Methods**: Leverage cultural techniques from different regions.
73. **Story-Based**: Create a story to guide the process.
74. **Traditional Boat**: Use traditional boats.
75. **Folklore**: Use methods from folklore.
76. **Mythical Inspiration**: Use mythical creatures or methods.
77. **Ancient Engineering**: Apply ancient engineering principles.
78. **Historical Figures**: Imagine historical figures solving the problem.
79. **Traditional Navigation**: Use traditional navigation methods.
80. **Cultural Tools**: Use tools from different cultures.

### Fun and Whimsical Solutions
81. **Magic Carpet**: Imagine a magic carpet ride.
82. **Superhero Help**: Enlist the help of a superhero.
83. **Fairytale Method**: Use fairytale-inspired methods.
84. **Fantasy Creatures**: Use fantasy creatures for help.
85. **Imaginative Play**: Create an imaginative scenario.
86. **Storytelling**: Craft a story to solve the problem.
87. **Cartoon Method**: Use cartoon-inspired solutions.
88. **Fairy Godmother**: Imagine a fairy godmother helping.
89. **Wizard's Spell**: Use a wizard's spell.
90. **Dragon Ride**: Imagine a dragon helping out.

### Miscellaneous Creative Ideas
91. **Multiple Boats**: Use multiple small boats.
92. **Tugboat**: Use a small tugboat.
93. **Sled**: Use a sled on a frozen river.
94. **Hot Air Balloon**: Use a hot air balloon.
95. **Raft Train**: Create a train of rafts.
96. **Animal Partnership**: Partner with other animals.
97. **Interactive Game**: Turn it into an interactive game.
98. **Virtual Reality**: Use VR to test solutions.
99. **Mind Puzzle**: Solve as a mind puzzle.
100. **Artistic Approach**: Create an artistic representation of the solution.

These solutions offer a variety of practical, innovative, whimsical, and creative approaches to solving the problem of getting two goats across a river.

 

[Spyxos]

I got some kind of limited access to gpt 4o today. It actually feels the same as before. All that impressive stuff they showed a few days ago.... where and how do I activate it?

 

[ResurrectedContrarian]

I got some kind of limited access to gpt 4o today. It actually feels the same as before. All that impressive stuff they showed a few days ago.... where and how do I activate it?

to try the realtime voice interaction, push the headphones icon on the lower right (assuming you're on the ios app; I don't know how it looks on various platforms)

 

[hyperbertha]

Here is a real response:

Paul Freeman + Gary Marcus are anti-AI (those tweets are most likely fake responses). I would fact check before attempting to make a point. You proved ResurrectedContrarian right, you most likely have never attempted to use this yourself. Just parroting another person's agenda.

Spoiler: Solve this 100 times

Sure! Here are 100 different solutions to the problem of getting two goats across a river, each with a unique approach or twist:

### Standard and Practical Methods
1. **Classic One-at-a-Time**: Take Goat 1 across, return empty, take Goat 2 across.
2. **Lead and Follow**: Tie both goats together and take them across in one trip.
3. **Switch and Secure**: Take Goat 1 across, return with Goat 1, take Goat 2 across, return empty, take Goat 1 across again.
4. **Alternating Trips**: Take Goat 1 halfway and secure, return, take Goat 2 all the way, return to halfway, retrieve Goat 1.
5. **Helper Method**: Have a helper take one goat while you take the other simultaneously.
6. **Use a Raft**: Build a raft for both goats, secure them, and float across.
7. **Swim Alongside**: Swim next to each goat as it crosses.
8. **Bridge Construction**: Build a temporary bridge and lead both goats across.
9. **Pulley System**: Use a pulley to transport goats one by one.
10. **Inflatable Device**: Use an inflatable for one goat, row the other in the boat.

### Variations on Practical Methods
11. **Floating Platform**: Create a stable platform and pull it across.
12. **Ziplines**: Set up a zipline harness for each goat.
13. **Multiple Trips**: Take one goat across, return, rest, repeat for the second goat.
14. **Animal Trainer**: Train goats to swim while guiding them.
15. **Special Harness**: Use a harness to secure both goats and pull them with a boat.
16. **Double Raft**: Use two small rafts, one for each goat.
17. **Barge Method**: Use a flat barge for both goats.
18. **Animal Tracks**: Build animal tracks or a path for the goats.
19. **Rope Bridge**: Construct a rope bridge and guide the goats.
20. **Canoe Method**: Use a larger canoe for both goats.

### Innovative Techniques
21. **Magnet System**: Use strong magnets to secure goats to a metal raft.
22. **Balloon Lift**: Use balloons to create buoyancy for a raft.
23. **Winch System**: Install a winch and pulley to pull a raft.
24. **Tandem Swimming**: Swim across with both goats tied to you.
25. **Boating Sessions**: Make multiple small trips ensuring goat safety.
26. **Hydraulic Lift**: Use a hydraulic lift system to transport goats.
27. **Goat Ferris Wheel**: Build a rotating wheel to move goats across.
28. **Underwater Tunnel**: Create a temporary underwater tunnel.
29. **Cable Car**: Build a cable car system for goats.
30. **Animal Conveyer**: Use a conveyor belt mechanism.

### Creative Methods
31. **Hovercraft**: Build a small hovercraft for the goats.
32. **Airboat**: Use an airboat to transport the goats.
33. **Drone Lift**: Use drones to lift and transport goats.
34. **Goat Flotation Devices**: Equip goats with personal flotation devices.
35. **Inflatable Tube**: Secure goats in an inflatable tube.
36. **Gondola**: Use a gondola system.
37. **Catamaran**: Build a catamaran for stability.
38. **Submarine**: Use a small submersible device.
39. **Water Skis**: Train goats to use water skis.
40. **Swing System**: Use a swinging platform.

### Logical and Puzzle Solutions
41. **Math Puzzle**: Solve as a logic puzzle ensuring goat safety.
42. **Algorithm Design**: Design an algorithm for efficient transport.
43. **Time Management**: Optimize timing for each trip.
44. **Risk Management**: Assess risks and plan accordingly.
45. **Strategic Planning**: Develop a step-by-step strategic plan.
46. **Simulation**: Simulate different methods to find the best one.
47. **Flowchart**: Create a flowchart to outline steps.
48. **Probabilistic Methods**: Use probability to minimize risk.
49. **Statistical Analysis**: Analyze past methods for effectiveness.
50. **Optimization Theory**: Apply optimization theory.

### Environmental Adaptations
51. **Natural Bridge**: Find or construct a natural bridge.
52. **Ice Path**: Create an ice path in winter.
53. **Log Rolling**: Use logs to create a rolling bridge.
54. **Nature's Help**: Use natural resources like vines.
55. **Animal Behavior**: Leverage animal behavior to guide them.
56. **Weather-Based**: Wait for favorable weather conditions.
57. **Seasonal Change**: Wait for a dry season to lower the river.
58. **Natural Raft**: Use natural materials to build a raft.
59. **Local Wildlife**: Observe local wildlife crossing techniques.
60. **Eco-Friendly Methods**: Use environmentally friendly methods.

### Tech-Savvy Solutions
61. **GPS Guidance**: Use GPS to plan the route.
62. **Smart Devices**: Equip goats with smart tracking devices.
63. **Automated Boats**: Use automated boats for transport.
64. **Remote Control**: Use remote-controlled devices.
65. **VR Simulation**: Test methods with virtual reality.
66. **AI Planning**: Use AI to plan the crossing.
67. **Drones for Guidance**: Use drones to guide the goats.
68. **Tech Support**: Have technical support on standby.
69. **Tech-Assisted Raft**: Equip rafts with tech for stability.
70. **Sensors and Monitors**: Use sensors to monitor conditions.

### Cultural and Historical Methods
71. **Historical Techniques**: Use ancient methods of animal transport.
72. **Cultural Methods**: Leverage cultural techniques from different regions.
73. **Story-Based**: Create a story to guide the process.
74. **Traditional Boat**: Use traditional boats.
75. **Folklore**: Use methods from folklore.
76. **Mythical Inspiration**: Use mythical creatures or methods.
77. **Ancient Engineering**: Apply ancient engineering principles.
78. **Historical Figures**: Imagine historical figures solving the problem.
79. **Traditional Navigation**: Use traditional navigation methods.
80. **Cultural Tools**: Use tools from different cultures.

### Fun and Whimsical Solutions
81. **Magic Carpet**: Imagine a magic carpet ride.
82. **Superhero Help**: Enlist the help of a superhero.
83. **Fairytale Method**: Use fairytale-inspired methods.
84. **Fantasy Creatures**: Use fantasy creatures for help.
85. **Imaginative Play**: Create an imaginative scenario.
86. **Storytelling**: Craft a story to solve the problem.
87. **Cartoon Method**: Use cartoon-inspired solutions.
88. **Fairy Godmother**: Imagine a fairy godmother helping.
89. **Wizard's Spell**: Use a wizard's spell.
90. **Dragon Ride**: Imagine a dragon helping out.

### Miscellaneous Creative Ideas
91. **Multiple Boats**: Use multiple small boats.
92. **Tugboat**: Use a small tugboat.
93. **Sled**: Use a sled on a frozen river.
94. **Hot Air Balloon**: Use a hot air balloon.
95. **Raft Train**: Create a train of rafts.
96. **Animal Partnership**: Partner with other animals.
97. **Interactive Game**: Turn it into an interactive game.
98. **Virtual Reality**: Use VR to test solutions.
99. **Mind Puzzle**: Solve as a mind puzzle.
100. **Artistic Approach**: Create an artistic representation of the solution.

These solutions offer a variety of practical, innovative, whimsical, and creative approaches to solving the problem of getting two goats across a river.

You trolling? You reworded the question to reduce the details and then likely repeated it over and over until it gets it right.

 

[hyperbertha]

I got some kind of limited access to gpt 4o today. It actually feels the same as before. All that impressive stuff they showed a few days ago.... where and how do I activate it?

Plus user here. Only the text portion is rolled out. Voice and images are unreleased so far.

 

[hyperbertha]

Perhaps we can take the heat down and focus with this summary:

• some people out there are very skeptical about LLM reasoning
• you'll point to perhaps the most vocal non-niche voice like LeCun, and I can very easily point back and say "sure, but Geoffrey Hinton strongly disagrees with him, and Hinton's work is far more comprehensive & influential across machine learning / AI / analysis of neural concepts and thinking, at every level" -- and I can do this all day because so many of the top minds simply don't take the weird minimalist view
• so I can say confidently that the viewpoint you present is not the dominant view at all, even at the highest levels of involvement and theory
• not only that, but you exaggerate even that view by presenting it as "LLMs are not reasoning at all" which even the niche on your side would typically not say, except when giving intentionally viral Wired interviews etc; they would be more careful and circumspect and say "it can't plan" or take some other perceived particular failure of their personal interest, and wouldn't be so bold as to say it's all an illusion when we have increasingly many examples to the contrary across many domains
• there are many aspects of intelligence, and you are more interested in a very different subset from the one that language models are designed to have; because of this, you also vastly downplay and distort the importance of linguistic reasoning in humans and AI
• don't conflate things like transformers with vague term like "deep learning" (okay, nearly everything is deep learning, that's not specific to the LLM revolution's unique autoregressive architecture at all) and then wave it all away
• these models will be challenging and exceeding human capability in many many domains in our lifetime, so get ready

To Build Truly Intelligent Machines, Teach Them Cause and Effect | Quanta Magazine

Judea Pearl, a pioneering figure in artificial intelligence, argues that AI has been stuck in a decades-long rut. His prescription for progress? Teach machines to understand the question why.

www.quantamagazine.org

The pioneer of modern ml methods himself says thusly:

"As he sees it, the state of the art in artificial intelligence today is merely a souped-up version of what machines could already do a generation ago: find hidden regularities in a large set of data. “All the impressive achievements of deep learning amount to just curve fitting,” he said recently.

In his new book, Pearl, now 81, elaborates a vision for how truly intelligent machines would think. The key, he argues, is to replace reasoning by association with causal reasoning. "

How do you respond?

 

[Trilobit]

There's more:

https://openai.com/index/openai-and-reddit-partnership/

What the fuck?

It gave me a weird sensation that I'm now a part of the AI. I hope I don't make it autistic.

 

[Hugare]

You trolling? You reworded the question to reduce the details and then likely repeated it over and over until it gets it right.

This thing can help programmers to code by creating simple/medium scripts, teach geometry to children in realtime, but what really matters is if it knows how a man can take a cabbage across a river on a boat.

This is like saying that you are stupid 'cause you cant name all the countries in the world.

 

[E-Cat]

I got some kind of limited access to gpt 4o today. It actually feels the same as before. All that impressive stuff they showed a few days ago.... where and how do I activate it?

GPT-4o is still essentially a GPT-4, a 2022 model. From being able to interact with it in real-time, they must have shrunk down the model a lot to perhaps hundreds or even tens of billions of parameters.

I'm not saying that GPT-4o is not using more modern techniques, simply that it is a small model that is a bit smarter than GPT-4.

GPT-5 is probably trained and being red-teamed right now. That will really make the doubters take a second look when it's released perhaps early next year.

 

[hyperbertha]

This thing can help programmers to code by creating simple/medium scripts, teach geometry to children in realtime, but what really matters is if it knows how a man can take a cabbage across a river on a boat.

This is like saying that you are stupid 'cause you cant name all the countries in the world.

Those code it creates are plagiarized more likely. It didn't actually create it from nothing but retrieved it from it's compressed repository in a remixed manner. By-hearted knowledge is useful for some applications, like replacing stack overflow, but it's actually extremely limited in real world usage, which is why it hasn't changed the world. This thing is incapable of generalization according to most experts, which is what really matters.

 

[ResurrectedContrarian]

Those code it creates are plagiarized more likely. It didn't actually create it from nothing but retrieved it from it's compressed repository in a remixed manner. By-hearted knowledge is useful for some applications, like replacing stack overflow, but it's actually extremely limited in real world usage, which is why it hasn't changed the world. This thing is incapable of generalization according to most experts, which is what really matters.

It doesn't simply repeat code. Look again at examples like my own above, or I can provide entire debugging conversations.

GPT4 is already better at taking apart, grasping, finding problems, and reassembling extremely complex code and formulas in new ways than most junior engineers in the profession. I know this firsthand. And this includes step-by-step debugging, making changes to code, seeing the output, then making new changes in response to it.

And it doesn't "remix," that's a total misunderstanding of the way that attention mechanisms weave semantic & symbolic threads through a text. All of this is simply false and FUD.

 

[ResurrectedContrarian]

In his new book, Pearl, now 81, elaborates a vision for how truly intelligent machines would think. The key, he argues, is to replace reasoning by association with causal reasoning. "

How do you respond?

Do you read much research in AI? I probably read 10-20 papers on Arxiv a week. This is how research is done.

Person says (*in extremely exagerrated form, if it's in a book rather than a research paper): "it can't do [this thing I'm focused on yet], so here's a [an architecture I propose to go that direction]." This guy's specialty his whole life is Bayesian reasoning with causal principles, and he wants a network focused on that. Okay cool, that will add to the many areas of high intelligence these networks already have. Or assembling multiple networks and memory units etc a la LeCun's interest will help in another direction. Cool.

People who have particular niches want all the (justified) praise of GPT across the profession to also be redirected into their own niche areas of interest in different forms of reasoning. That one guy you post is a total joke, who just wants GPT models to reason about moving blocks spatially, when they can already write analyses of unseen poetry that can fool a professor to think it's a smart graduate student making the analysis.

This is boring, to use any of this as a way to pretend that LLMs aren't one of the greatest technological breakthroughs of human history, and capable of covering many many areas of human reasoning as well as and better than humans.

 

[hyperbertha]

Do you read much research in AI? I probably read 10-20 papers on Arxiv a week. This is how research is done.

Person says (*in extremely exagerrated form, if it's in a book rather than a research paper): "it can't do [this thing I'm focused on yet], so here's a [an architecture I propose to go that direction]." This guy's specialty his whole life is Bayesian reasoning with causal principles, and he wants a network focused on that. Okay cool, that will add to the many areas of high intelligence these networks already have. Or assembling multiple networks and memory units etc a la LeCun's interest will help in another direction. Cool.

People who have particular niches want all the (justified) praise of GPT across the profession to also be redirected into their own niche areas of interest in different forms of reasoning. That one guy you post is a total joke, who just wants GPT models to reason about moving blocks spatially, when they can already write analyses of unseen poetry that can fool a professor to think it's a smart graduate student making the analysis.

This is boring, to use any of this as a way to pretend that LLMs aren't one of the greatest technological breakthroughs of human history, and capable of covering many many areas of human reasoning as well as and better than humans.

Do you agree that gpt4 is just curve fitting with no generalization out of distribution?

 

[rm082e]

This thing can help programmers to code by creating simple/medium scripts, teach geometry to children in realtime, but what really matters is if it knows how a man can take a cabbage across a river on a boat.

This is like saying that you are stupid 'cause you cant name all the countries in the world.

The point is that there is no logical, thinking "agent" inside the box. It's just math and code designed to return a string of text that has the highest probability of matching what the user asked for. It's a very fancy auto-complete. Obviously that has value, but it's not approaching an "artificial consciousness".

 

[Hugare]

Those code it creates are plagiarized more likely. It didn't actually create it from nothing but retrieved it from it's compressed repository in a remixed manner. By-hearted knowledge is useful for some applications, like replacing stack overflow, but it's actually extremely limited in real world usage, which is why it hasn't changed the world. This thing is incapable of generalization according to most experts, which is what really matters.

lol no, it doesnt copy code from somewhere else. Every code that I've requested to be made were very specific and custom made. You obviously havent used it to help you code, otherwise you would know.

It has an insane amount of information to look for on its database: libraries, documentations and etc.

When you request it to make a python code that look inside a .pdf file for one sequence of 12 numbers and then extract them to an excel file, for example, it "solves" the code like a math problem: there's this python library that transforms .pdf files in excel files, so I use it first, than use this one to search for the sequence, than this and etc.

It combines different pieces from different places in order to make the code, but it does it in a logical way

The point is that there is no logical, thinking "agent" inside the box. It's just math and code designed to return a string of text that has the highest probability of matching what the user asked for. It's a very fancy auto-complete. Obviously that has value, but it's not approaching an "artificial consciousness".

It doesnt need to think to steal people's jobs

Why hire a junior analyst to write code for you when you can just ask GPT?
Why hire a private teacher to teach you math when you can just use GPT?

And the same goes for so many other jobs

 

[hyperbertha]

lol no, it doesnt copy code from somewhere else. Every code that I've requested to be made were very specific and custom made. You obviously havent used it to help you code, otherwise you would know.

It has an insane amount of information to look for on its database: libraries, documentations and etc.

When you request it to make a python code that look inside a .pdf file for one sequence of 12 numbers and then extract them to an excel file, for example, it "solves" the code like a math problem: there's this python library that transforms .pdf files in excel files, so I use it first, than use this one to search for the sequence, than this and etc.

It combines different pieces from different places in order to make the code, but it does it in a logical way

It doesnt need to think to steal people's jobs

Why hire a junior analyst to write code for you when you can just ask GPT?
Why hire a private teacher to teach you math when you can just use GPT?

And the same goes for so many other jobs

You are agreeing with me. Isn't this exactly remixing, or rather restitching? I don't think it can solve actually novel problems out of its training set, hence no generalization. None of what it's doing is very impressive, unless you consider memorization by a machine with infinite memory impressive.

What you perceive as 'logic' is just a result of statistical prediction that provides most likely correct combination of variables, and it just happens to be correct most of the time because that's how probability works.

What we are looking for is common sense and generalization capabilities, you know, what makes intelligence intelligence and allowed humanity to get this far.

 

[ResurrectedContrarian]

Do you agree that gpt4 is just curve fitting with no generalization out of distribution?

Absolutely not. The only way to have that absurd confusion is if you define “distribution” in such a way that it’s meaningless. It abstracts from what it has seen to a higher order prior that it can leverage, just as humans do when we make analogies or combine ideas in novel situations.

You are agreeing with me. Isn't this exactly remixing, or rather restitching? I don't think it can solve actually novel problems out of its training set, hence no generalization. None of what it's doing is very impressive, unless you consider memorization by a machine with infinite memory impressive.

I don’t agree because it isn’t memorization, it is abstractive compression, much like the way much of our human language knowledge works in our own brain. Calling it mere memorization is a total conceptual misunderstanding of the model, its architecture, its capabilities, and even the math.

What you perceive as 'logic' is just a result of statistical prediction that provides most likely correct combination of variables, and it just happens to be correct most of the time because that's how probability works.

You could say this of human neurons too.

 

[rm082e]

It doesnt need to think to steal people's jobs

Why hire a junior analyst to write code for you when you can just ask GPT?
Why hire a private teacher to teach you math when you can just use GPT?

And the same goes for so many other jobs

I'm not saying it won't steal some people's jobs. I'm saying it can't replace a human in all knowledge work, because most knowledge work requires some degree of common sense. AI attempts to simulate common sense, but because it lacks an "agent", it doesn't actually have common sense.

Think about the automated telephone menu. We've had those for decades now, and they replaced operators. But the automated phone menu isn't so perfect that you never need to speak to a human. Pretty much all of them have an option to press a button or say "speak to someone" and they'll dump you to a person who can use their common sense to help you.

I'm a manager over a group of software and system engineers. AI is not going to replace any of them, nor would it stop me from hiring a junior analyst. I hire people for their common sense - their ability to take basic direction and extrapolate what needs to be done based on that direction. I certainly wouldn't hand that direction over to an AI who lacks common sense and randomly, and confidently, gets things totally wrong. AI still needs a human to double-check it's work.

So far, services like GitHub Copilot haven't even significantly adjusted my expectations for the output/efficiency of my team. They've found value in it so we bought it for them, but they mostly use it as a faster way to accomplish basic tasks. It's faster than them doing a google search for a chunk of code that they can copy/paste, then make some minor edits to fit their specific need. At $100 a year, it's going to save them enough hours that it's worth the cost.

 

[Spyxos]

 

[midnightAI]

Imagine this paired with the Unitree G1

 

[Spyxos]

Something is not right.

 

[IlGialloMondadori]

lol no, it doesnt copy code from somewhere else. Every code that I've requested to be made were very specific and custom made. You obviously havent used it to help you code, otherwise you would know.

It has an insane amount of information to look for on its database: libraries, documentations and etc.

When you request it to make a python code that look inside a .pdf file for one sequence of 12 numbers and then extract them to an excel file, for example, it "solves" the code like a math problem: there's this python library that transforms .pdf files in excel files, so I use it first, than use this one to search for the sequence, than this and etc.

It combines different pieces from different places in order to make the code, but it does it in a logical way

It doesnt need to think to steal people's jobs

Why hire a junior analyst to write code for you when you can just ask GPT?
Why hire a private teacher to teach you math when you can just use GPT?

And the same goes for so many other jobs

I honestly can’t believe you use ChatGPT to write code lol.
We have a contractor devops guy who was doing this, it was insanely obvious and terrible, particularly his documents and emails. His contract was not renewed.

ChatGPT and current AIs are terrible at both code and writing.

 

[SJRB]

Turns out the desktop app they made for Mac works on Apple M-chip devices only, not on Intel-based chips.

This sucks.

 

[Hugare]

I honestly can’t believe you use ChatGPT to write code lol.
We have a contractor devops guy who was doing this, it was insanely obvious and terrible, particularly his documents and emails. His contract was not renewed.

ChatGPT and current AIs are terrible at both code and writing.

I dont work writting code, I work with finance. So it complements my job.

Instead of writting a very complex google sheet formula, or a simple python script, I order it to ChatGPT. It gets the job done faster.

Writting e-mails with ChatGPT is horrible, tho. But writting code/e-mails are very different activities.

GPT4 is not terrible at coding, at all. Of course it wont code an emulator by itself, but it can write simple code just fine.

 

[Hugare]

Something is not right.

And yet people think that it wont be able to cause mass unemployment and etc.

That GPT is just a slightly smarter google search

Those people in less than 10 years will be like:

 

[SJRB]

Something is not right.

This whole AI safety/doomerism thing is bunch of baloney and a catastrophic waste of resources.

 

[Hugare]

This whole AI safety/doomerism thing is bunch of baloney and a catastrophic waste of resources.

As AI gets more inteligent than ever before, OpenAI is now more confident that they dont need a team to think about safety concerns?

This is certainly a take

My take: they care more about the money that this product can make them than any safety concerns that may come up by making it more inteligent.

Much more believable, imo