On Thu, Dec 22, 2022 at 04:21:46AM +1300, rikki cattermole via Digitalmars-d wrote:
> I asked it to describe D from a marketing standpoint:
> 
> 
> 
> As a language model, I am not able to engage in marketing or advertising campaigns. However, I can provide a general overview of the D programming language.
> 
> D is a general-purpose programming language that was developed with the goal of combining the power and high performance of C and C++ with the programmer productivity of modern languages like Python and Ruby. It is a compiled, statically typed language that supports both imperative and object-oriented programming paradigms.
[...]

This sounds like a straight quote from Wikipedia... or a paraphrase.


T

-- 
English is useful because it is a mess. Since English is a mess, it maps well onto the problem space, which is also a mess, which we call reality. Similarly, Perl was designed to be a mess, though in the nicest of all possible ways. -- Larry Wall

On Wednesday, 21 December 2022 at 17:01:34 UTC, H. S. Teoh wrote:
>
> This sounds like a straight quote from Wikipedia... or a paraphrase.
>

What did surprised you? It is all trained on all kinds of texts available including the internet scraped data. All it does is basically predicting the next 'word' for the input.
GPT-J generates something similar but less stable and gets easily confused.

The summary is likely a result of additional filtering and topic extraction using another neural network like BERT. And also it seems they do some extra pre- and post-processing and sampling, because GPT-J can output nonsense like "D doesn't have garbage collection and is lower level than C++" and "D is very simple and easy to debug".

On Wed, Dec 21, 2022 at 06:43:22PM +0000, evilrat via Digitalmars-d wrote:
> On Wednesday, 21 December 2022 at 17:01:34 UTC, H. S. Teoh wrote:
> > 
> > This sounds like a straight quote from Wikipedia... or a paraphrase.
> > 
> 
> What did surprised you?

I wasn't surprised at all. Just skeptical.


> It is all trained on all kinds of texts available including the internet scraped data. All it does is basically predicting the next 'word' for the input.

Exactly what I thought it was based on.


> GPT-J generates something similar but less stable and gets easily confused.
> 
> The summary is likely a result of additional filtering and topic extraction using another neural network like BERT. And also it seems they do some extra pre- and post-processing and sampling, because GPT-J can output nonsense like "D doesn't have garbage collection and is lower level than C++" and "D is very simple and easy to debug".

IOW it's just a smarter way to dress up the same information that's already available online and found by traditional means.  Nothing to see here, move along.


T

-- 
People tell me I'm stubborn, but I refuse to accept it!

On Wednesday, 21 December 2022 at 19:14:37 UTC, H. S. Teoh wrote:
>
> IOW it's just a smarter way to dress up the same information that's already available online and found by traditional means.  Nothing to see here, move along.

Well I can't tell all the facts, but it is capable of generating completely new text to some extent by increasing algorithm 'temperature' parameter and other sampling methods (top K, top P, etc..), still it is true that one have to input at least some certain minimal amount of data in order to make it produce something of value.

here is an illustration of default greedy sampling method that illustrates decision path with learned data with probability score on each branch, red dotted path illustrates the naive greedy path, and IIRC the red line illustrates choosing more than one beam.

So basically just mix and match these stuff until it start making sense (c)

https://i.imgur.com/mMW6qbl.png

On Tuesday, 20 December 2022 at 08:06:49 UTC, Walter Bright wrote:
>
> But it *looks* seductively plausible, yet has insidious errors.

It has, but considering this is a general-purpose AI that knows nothing about programming save from what it can infer it is frighteningly good. Just imagine what programming-specific AI, done as well, could do.

Also since it understands human languages, it's mistakes could presumably be corrected with feedback, much like mistakes from human programmers.