Has it ever been done ? If not why so ?

[Dio]

Symbolics - https://smbx.org/ has a fair bit of tech info on those.

[WiteWulf]

An old game called Abuse uses Lisp for its AI. There's a number of small example games and the like floating about

Sorry to drag the thread off on a tangent, but just wanted to say how much I loved Abuse! It was one of the first multiplayer LAN games I remember in the late 90s that /wasn’t/ a Doom/Hexen/Heretic/Duke Nukem/etc. FPS. It was a side scrolling platform shooter using WASD for movement and the mouse to aim, making for a very fast, fluid game that still ran well on 486s

[zip accident]

You can write recursive functions in c.

[gilesgoat]

You can write recursive functions in c.

OF COURSE YOU CAN !

[tufty]

And tells you "actually is a bit of a shit implementation", it's not efficient as it could.

It was almost certainly talking about tail recursion versus general recursion.

Here's a non tail recursive fib

Code: Select all

(define (fib n)
  (if (<= n 2)
      1
      (+ (fib (- n 1)) (fib (- n 2)))))

That's the classic way of doing a recursive fib. Unfortunately, the 2 recursive calls to fib require pushing context onto the stack, and we will be recalculating values multiple times. If we are calculating (fib 5), our sequence of calls will be :

Code: Select all

(fib 5)
> (fib 4)      ;; 5 - 1
  > (fib 3)    ;; 4 - 1
    > (fib 2)  ;; 3 - 1
    > (fib 1)  ;; 3 - 2
  > (fib 2)    ;; 4 - 2
> (fib 3)      ;; 5 - 2
  > (fib 2)    ;; 3 - 1
  > (fib 1)    ;; 3 - 2

That's not at all efficient, and not bounded in stack usage either. For n, we will have n-2 stack frames pushed - at some point, we will esplode the stack.

A tail recursive version will call the recursive function as the last thing it does. In terms giles can understand, this means that tail recursion is a goto. No stack overhead, it's not even function call, no so bounded memory usage. It is a loop, but it looks like recursion.

Here's the easy to understand version

Code: Select all

(define (fib n)
  (fib-help 1 0 n))

(define (fib-help a b count)
  (if (= count 0)
      b
      (fib-help (+ a b) a (- count 1))))

fib-help, only dealing with 32 bit integers and converted to assembler, looks like this

Code: Select all

fib-help:
  subs  r3, r3, 1
  movpl r4, r0
  addpl r0, r0, r1
  movpl r1, r4
  bpl fib-help
  mov r0, r1
  rts

It's a loop.

[TheHighlander]

Watch this for an exotic combination. Reidrac (maker of many games for many different platforms) is coding a game using Haskell:

https://www.youtube.com/watch?v=PM-O4m0EVz4

There are 60 videos and he codes the entire thing. Actually produces a working game.

That said, I once worked for a company that did end user support for an application written entirely in Prolog.

[skab]

A tail recursive version will call the recursive function as the last thing it does. In terms giles can understand, this means that tail recursion is a goto. No stack overhead, it's not even function call, no so bounded memory usage. It is a loop, but it looks like recursion.

THANK YOU!

I've been wondering for the last 25 years how LISP et al get away without loop constructs. A 1 to 1000 loop would use the stack 1000 times, anything bigger would be impractical. I'm sure I heard about tail recursion in the past, but never connected the dots...

I heavily admire the elegance of LISP, but unfortunately can't wrap my head around the syntax. I understand it in the end, but reading it is exhausting. Apparently my brain (and Giles') is hard-wired for C-style syntax.

I dusted off my barely-ever-used Amiga 500+ and put a PiStorm in it, so I had a lot of fun playing around with AmigaOS once again for the last 6 months. And I now realized that the AmigaOS installer program uses a LISP-style "brackety" syntax. Whoever wrote the installer had a thing for functional programming. Maybe he/she was an Emacs fan?

[zip accident]

I heavily admire the elegance of LISP, but unfortunately can't wrap my head around the syntax. I understand it in the end, but reading it is exhausting. Apparently my brain (and Giles') is hard-wired for C-style syntax.

You can easily write recursion in C. I thought it used the stack, but maybe I was mistaken.

https://en.wikipedia.org/wiki/Tower_of_Hanoi

The following C function demonstrates the recursive solution. It will print out a list of move instructions. Output from a call for 3 disks is shown.

Code: Select all

#include <stdio.h>

void hanoi(int disks, char src, char dest, char temp)
{
    if (disks > 0) {
        hanoi(disks - 1, src, temp, dest);
        printf("move disc from %c to %c\n", src, dest);
        hanoi(disks - 1, temp, dest, src);
    }
}

int main()
{
    hanoi(3, 'A', 'C', 'B');
}

move disc from A to C
move disc from A to B
move disc from C to B
move disc from A to C
move disc from B to A
move disc from B to C
move disc from A to C

Erm, that Wiklpedia code is perhaps not tail recursive, but you certainly can write a tail recursive Towers of Hanoi in c.

[tufty]

I heavily admire the elegance of LISP, but unfortunately can't wrap my head around the syntax.

Oh, lisp syntax can be pretty hard to understand, particularly if, like most of us, you've been indoctrinated with ALGOL since before you first saw a computer. But there's more to it than that, underneath the difficult syntax it's Wordsworth compared to Jane and John and Spot the dog :)

Seriously, though, long functions in any language are painful to read and worse to actually understand (hence the huge number of unspotted bugs in open source code), and lisp code is no exception. It's why refactoring code to readable chunks is so valuable.

[KlaxonCow]

I always heard that LISP was "sold" as "the language for AI" .. with all the neural networking of today .. is still relevant ?

To answer this question specifically, not remotely so.

No programming language is relevant to neural networks. They are their own beast, and just don't work that way. They are not "programmed", they are "trained" instead.

I mean, there is the programming language in which you write your simulation software - I used C++ when I was doing this back in the '90s - but this is not relevant, because what you're coding is just the "virtual state machine" upon which the neural network actually runs.

And, ideally, it'd be hardware anyway. That was the case with the neural network project I was working on. As I say, I coded the simulation software in C++ but the ultimate intent was that, once we'd got the network and its weightings, it'd all be transferred to a dedicated FPGA. Directly in hardware, because it had to have real-time reactions.

It was software in training and testing, because of the flexibility to change the number of layers, the number of neurons, the algorithms, experiment with things. But, once nailed down, it was to be "frozen" and converted into dedicated hardware (and, yes, at that point, losing all of its capability to "learn" anymore but that was a wanted feature, as it was a safety critical system, and total predictability was desirable).

Admittedly, part of that was because this was the '90s and we were using early Pentiums. Today's hardware kicks its arse, of course. Lesser hardware demands better optimisation or it just won't run fast enough for that real-time reaction.

Indeed, with neural networks, it's all about that "immense parallelism". So, truth is, it's all about spooling the calculations to a GPU to calculate the, for example, roughly 1.7 trillion parameters of ChatGPT-4. Yes, 1.7 trillion. Although, you know, when you realise the comparable estimate on the human brain is 1000 trillion axons, it puts even a silly big number like 1.7 trillion in perspective.

An "axon" - a connection weight - is just a floating-point multiplication. That's it. Take the output, multiply by a float, copy it over as an input to every neuron in the next layer. The "multiply by a float" is not the clever bit, of course. It's hooking up that output to every single neuron in the next layer (and doing that for every single output in the current layer, repeated for each layer).

The "neuron" - the heart of the beast - is just "add up all the inputs and put it through a threshold function (typically a sigmoid)". Again, that's it.

The individual bits of a neural network are very simplistic and dumb. A neural network only has one type of "logic gate" - the neuron - that only does the one "dumb" thing.

The "magic" comes from the immense parallelism. The truly insane weighted "data bus" that connects everything in one layer to everything in the next layer.

Indeed, if you want to save a neural network, then you save the connection weights to a file. Oh, you also keep track of how many layers and how many neurons in each layer, but that's to do with putting the connection weights back into the right places. And remember to reapply the same threshold function - but, ah, you're probably just using the standard "sigmoid" like everyone else, o that's just implict.

You don't save anything else, as the rest of it is always the same. Put the connection weights back in the right places and you've recreated the network, where the same input will churn out the same output.

And, indeed, the process of "training" is looking at that output, judging how in error it was from the expected result, then you distribute the error backwards into the network - tweaking the connection weights proportionally to their influence (as earlier neurons have more effect on the output than later neurons, just because they're connected up to more neurons, indirectly) - to tweak the network. So that if you gave it the same input, after modifying the weights, it'd move a little closer to giving you the expected output.

Then you do it again. Then you do it again. And the process of "training" is, in this view, a case of tweaking the network to give the "correct" expected outputs. But in a "rolling average" kind of way, to optimise the weights to simultaneously give the right answers to all inputs.

You "program" it with data. Look at this picture of a dog - the answer you should give is "dog". Look at this picture of a cat - the answer you should give is "cat".

And with each picture and expected output, the network tweaks its response a little towards "the right answer" you gave it. Then you do it again with a different picture and it optimises its response a little towards that "right answer" as well.

Do this thousands of times and you're effectively optimising its response for the whole data set simultaneously. If it gets it horribly wrong, then you make big changes to the connection weights. If it gets it very slightly wrong, then it's only a tiny tweak. If it's dead right - exactly spot on the expected result - do nothing.

The structure of a neural network is basically this "Turing Complete" system, as you can coax ANY output from ANY set of inputs. It's all determined by those connection weights, which you keep tweaking - with "back propogation" - to optimise it a little towards this answer, then this answer, then this answer. Optimising it towards all the answers simultaneously.

So that the network's weights take on the "shape" of the entire data set.

And what's mad is that no-one thought of this. The design of neural networks comes originally from looking at the brain and trying to work out what it's doing. Then we start building (imperfect) simulations of that and - bloody hell - something that looks an awful lot like "cognition" is falling out the other end.

Kind of like Genetic Algorithms. You create a "population" of answers, you mix them up together, you "mutate" them a little sometimes, you test them against a "survival function" and keep what's good and discard what's bad. Repeat, repeat, repeat. And "evolution by natural selection" falls out the other end.

The solution - though it should be noted that this process is far from perfect and has all manner of problems - just falls out the other end.

No-one thought of this. It's just imperfectly copying nature and... oh, bloody hell, it actually works!

Turns out Darwin was right and here's the concrete mathematical proof of that. Just follow his "recipe" and, there you go, "evolution" falls out the other end.

Biologists looked at the brain. "We think it kind of works like this". And a computer scientists says "okay, let's try that" and... oh, bloody hell, it actually seems to be working! This chatbot might not be Einstein, true enough, but it is actually providing perfectly coherent responses. It's actually talking to me, of a certain limited fashion.

And, yes, this is what worries AI experts. Because, like, is there actually some kind of sentience inside there? Fuck knows. Maybe. Maybe not. If there is, then I have no idea how I'd detect that.

So, for all we know, maybe Google Bard actually does have some kind of low level sentience going on in there - no, not "human-like", but maybe "mouse-like" or "spider-like", as we must remember that we humans are not the only animals with some kind of conscious experience going on - but how do we know? How can we tell?

But, yeah, in the realm of neural networks, programming languages are irrelevant. And, in the realm of artificial intelligence, neural networks are the only game in town. I mean, the other stuff - with "expert systems" and such - is an interesting curiosity. But neural networks are the real deal. They are artificial brains (albeit imperfectly so, but that's what they fundamentally are - brain simulators). And, since ChatGPT illustrated it to the world so clearly, you can see that their capabilities make everything else pointless.

Because, in a sense, they already pass a limited version of the Turing Test. You can have perfectly natural coherent conversations with it. Only a few limitations - a few that have been imposed upon it, to make it "woke" in its responses and not deceive anyone with misinformation - give the game away.

Well, put it this way, ChatGPT-4 has 1.7 trilllion parameters. The human brain, by certain estimates, has 1000 trillion. Still a way to go, but they're getting there fast.

I mean, yes, one would hope there's maybe more to this than sheer scale. But, you know, we don't actually know. It might literally just be reaching 1 quadrillion axons and it becomes SkyNet.

As I've touched on before. Is there a barrier there? There doesn't seem to be one there that we can point at. There's nothing that suggests there is one. One should at least entertain the idea that maybe there just isn't one and it's only a matter of scale, and enough "training data" to get there.

How does it work to have a planet with two different intelligent species upon it at the same time? No-one knows. Never happened before.

Put it another way, everyone - in sci-fi - imagined that the aliens would come from space. But maybe, in an abstract sense, they've been here all along, just waiting for us to invent them.