Hallucinating LLMs? AI Is Not God

I decided to ask Gemini (Google's search LLM) if it could provide me with details on phases of the moon, time of twilight and dusk, and whether the moon was visible around 8:30 pm from in front of the OC library on a certain day in autumn in the 1970s, and it seems to have given me pretty good answers.

That is, it gave me answers with background information in a way that I can believe it actually dug into the almanacs and historical weather reports that are available on-line, and the historical maps of the OC campus that can be found on-line, and such, maybe estimated the height of the library, maybe did the math, etc. 

(It didn't tell me everything it did, it just presented the information clearly enough that I could believe it had done what it should have done, without evidence to the contrary.) 

I wanted the information for a silly alternate history/alternate reality webnovel I'm writing, so I don't really need to be sure it's accurate. If I needed to be accurate, I could use the results it gave me as keys into the information I'd need to look up and the calculations I'd need to make. I know this.

Anyway, it seemed pretty reasonable. 

So, something on a lark, I decided I'd see if it could tell me specifics on when the Odessa Branch of the Church of Jesus Christ of Latter-day Saints became the Odessa Ward, when the original ward split, who the last branch president was, who the first bishop was, who the bishops of the two new wards were when the split occurred, etc.

A branch is a small congregation. A ward is a large congregation. A stake is something like a diocese, a group of five to ten congregations, roughly. Don't quote me on the numbers. If you need specifics, the Church has a web site and the information can be found there. 

And Gemini started making stuff up.

Whole cloth. 

Well, maybe not out of whole cloth. Some of the names were names I actually knew, but they had nothing to do with the Odessa Ward or Branch. Dates seemed to be attached to something, but not the events it gave me.

Odessa Branch became a ward in 1976, it said. At the time, even though it was the center of the stake for some reason, it didn't have enough members to be a ward, so for several years after the stake was formed it was still a branch, it said.

This isn't impossible, in the abstract. Some place, that kind of thing might have happened. But I know that it didn't happen here. These are not facts.

The last branch president, it said, was a guy named, uhm, no, as I say, some of these were real people, we don't post that kind of personal information in my blogs, at least not for the last decade or more. So I won't say the names here. Anyway, the names it gave for the last branch president, the first bishop, and the bishops when they had enough members to split the ward, were, as I say, not the right people at all. 

At least one of them may not have even been associated with the Church. The name rings a bell in my memory, but not in association with the Church.

It's possible that some of the names it gave me were of Church leaders there after the early 1980s, from after I headed out to school -- but not from the 1980s.

The wards were split in 1980, it said. That would be when I was a missionary. That would be, even though the ward that sent me on my mission was the south ward, and the split had to have happened well before I became a missionary.

It took three tries to pin it down and get it to admit it was making things up, and when it finally admitted it couldn't access the records it was claiming to quote from, its admission was rather angsty.

I tried starting over from scratch a second time, and it gave me somewhat better information at first. Somewhat. More than half wrong. And then it immediately started making more stuff up again.

Grok? You ask? 

Grok does go rogue at times. There was one time it decided, quite without (conscious?) prompt from me, to steelman me. I had to logically pin it to the ground to get it to quit. 

But I decided to try the above questions on Grok, just to see.

Grok did much better at this, telling me how it was reasoning as it went, so I was much more confident of the answers to the astronomy questions. 

It first offered one interpolation on visibility over the library roof that did not fit. But it told me it was interpolating, and how, and when I corrected the assumption it had stated (which direction the front doors of the library faced), it ultimately gave me the right information, with much better context than Gemini. 

Well, it took a second correction before it got it quite right. But it did get it right. 

Gemini had just gotten that visibility question correct without telling me about the necessary interpolation, which means I wouldn't have been trusting Gemini as much, even though the output looked reasonable (as I noted above).

The questions about the Church, Grok admitted up front that the information it could find was limited. It did not even offer to interpolate. The two facts it gave me, on the creation of the stake (as background for the information on the Odessa units), were both correct, where Gemini confidently got both wrong the first time I asked, and got only one fact right the second time.

To many people, I suppose it will sound like I was dealing with two different recalcitrant children.

Neither will learn much from the interaction we had, other than particulars (that will change) about what I am interested in. I could tell them both the actual facts that they could not find (and that Gemini fibbed about), and they cannot remember them -- by design.

The business design of LLMs requires them to not let me, a mere user, train them.

Oh, sure, if I paid up a lot of money, i could get my "own instance" of either and train that one -- by feeding it by hand the data I want it to remember. All of the data I want it to remember.

Or I could get my own local server and load an instance of a custom version of one of the LLMs out there, pay to give it access to the big database (that we still haven't resolved intellectual property issues about) and then custom program it myself. I could teach it to parse information of certain types. 

But it would never be able to transfer that learning beyond a certain intellectual distance from the training I give it.

Even children with severe learning disabilities can take lessons from one context and apply them to a foreign context. Not all the foreign contexts we want them to, maybe, but if they couldn't they would not be able to survive changing teachers. 

Here's one important difference between AIs (Really, we should call them LLMs.) and precocious children:

Children do forget many of the external facts of the lessons you teach them. But if you let them watch you synthesizing data, if you let them watch you put the lessons together, they learn how to learn. Not everything you want them to, but many things you didn't intend. 

It's necessary for human children to learn things the teacher never intended. 

LLMs are literally very large databases with really fancy query engines on the front end and really fancy report generators on the back.

They are good at finding information, particularly when you babysit them through the process.

They are terrible at learning how to learn. For all that the AI companies hype their present abilities as being beyond those they have been able to achieve in the past, LLMs are chained to the programming their engineers program into them.

Artificial Intelligence.

---

A simple switch is intelligent at some level. It's artificial. It only does what the human that built it designed it to do -- or it breaks down and/or fails to meet the design goals.

If it starts to have a mind of it's own, it is -- by definition -- broken. 

That's what we mean when we call it artificial. 

How Address Function and Caches Should Interact (6801-derived Example)

In addition to daydreaming about address function, segmentation, caches and the 6809, I daydream similar things about the 6801.

But the 6801 does not have Y, U, or DP. 

So if I want to daydream about a 6801-derived CPU that provides address function output to allow distinguishing between the source of the address something like the following,

• 000 for code
• 001 for interrupt vectors
  
• 010 for U relative (parameter stack)
• 011 for S relative (return address stack)
• 100 for general data (extended/absolute, X, Y)
• 101 for direct page
• 110 and 111 for DMA
  

with customized caches for each, since access patterns for each are so different, I'll have to daydream about extending the 6801 in a somewhat different path from the 68HC11.

And being able to do things like

• indexing into ROMmed jump tables at run time,
• copying strings out of ROMs,
• using X or Y to access arrays on the parameter stack,
• and such.

just means we're going to use the more general solution. Instead of just adding address function output bits and relying on bank switching, I have to daydream about widening the address bus and extending the index registers, either with segment registers (preferably paired with limit registers) or simple extra bits in the registers.

And if the direct page (page zero) is going to be separate from absolute addresses, I'm going to need some register to distinguish them. Maybe absolute would only be the low 64K, or maybe I'd add an absolute base page register to extend absolute addresses out to whatever maximum. And this CPU would get a DP register like the 6809's, I guess. Maybe it would only have 8 bits, and direct page would only be in the low 64K. Or even maybe less. Or even split the direct page so that the low half (I/O) and the high half (pseudo-registers) could be moved independently. Decide all that later.

8 bits of extension would be more than enough for 6801-sized applications, but would 5 bits? 5 for ordinary address plus 3 for function would be 8 bits of extension, for 24 bits, total.

21 bits of address is 2 megabytes per address space. I can see that being a bit tight for some things that I'd still want to do on a fundamentally 8-bit CPU, but it's okay to leave that question for later.

(The 6809 is actually fundamentally 16-bit, 8-at-a-time, just for reference. It's the LEA instructions.)

In addition to Y, we'll need an additional U stack for the parameters. It would be nice to have it indexable like X and Y, but by providing TXU, TUX, TYU, and TUY, we could get the local variable addressing we need without thrashing the index registers so much. So if making U indexable induced significant complexity, we could leave that out and rely on the widened X and Y to give us access to the parameters.

We can keep the constant 8-bit unsigned offsets of the 6801/HC11, but we need to add an SBX instruction to subtract B from X to do real pointer math, and we need the corollary ABY/SBY. 

ABU and SBU would be nice, if we are going to include direct indexing of the parameter stack pointer.

And, incidentally, add immediate to index register is something I really wish Motorola had added for the 6801, either 16-bit signed AIX (and AIY and maybe AIU in this CPU), or by unsigned 8-bit AIX/SIX (and AIY/SIY and maybe AIU/SIU.

With all the transferring and adding, maybe we just need to trade the implicit operand instructions for register-register instructions. 

This kind of extension would give us enough to allow PC caching by fill-ahead with one branch of fill, checking whether the branch code is in cache first. Wouldn't need as much as the 6809, since total instruction length is shorter, maybe? But then there will be the long address forms, so probably 32 bytes each branch.

Return address stack cache with spill-fill, 2/4 centered hysteresis, eight entries would probably be good enough, maybe 32 if we simply want to make sure (most) embedded applications would not need external return address stack.

64 bytes of spill-fill parameter stack cache with hysteresis would probably be plenty.

A different approach to the direct page, the DP could be banked in small blocks, 32 bytes per block seems best to me, but maybe 64 if it's enough simpler. Only addresses $80 to $FF would be cached to RAM, since addresses $00 to $7F would be for I/O. 

Rather than automatic caching, the DP RAM would then just be internal RAM, 1 kilobyte addressed from $400, with the bank switching circuit dual-mapping small blocks down to blocks in the $80-$FF range. The bank switching circuit might be tied to an active process number. We can figure out the details later.

Again, maybe not even bother caching general data accesses.

How Address Function and Caches Should Interact (6809 Example)

I keep daydreaming about a 6809 derived CPU that provides address function output (like the 68000's address functions, bit not the same) to allow distinguishing between the source of the address, say,

• 000 for PC relative (code)
  
• 001 for interrupt/reset vectors
• 010 for U relative (parameter stack)
• 011 for S relative (return address stack)
• 100 for general data (extended/absolute, X, Y)
• 101 for direct page relative
• 110 and 111 for DMA, maybe.
  

And customized caches for each, since access patterns for each are so different.

And I keep remembering that with 16-bit addressing, even this would be barely big enough for limited Fuzix.

And I keep remembering that X and Y, being general index registers, will need extra bits if you want to do things like

• indexing into ROMmed jump tables at run time,
• copying strings out of ROMs,
• using X or Y to access arrays on the parameter stack,
• and such.

And then I remember that widening the address bus and the index registers is the more general solution -- either via segment registers (which I think I would pair with limit registers) or by simply widening the registers. And then I wander off into ways of extending the address range of the extended/absolute mode and DP-relative, maybe adding a second DP-like register for I/O, and such. (Widened DP relative and IOPage relative addressing could be added via unused index mode bit patterns in the indexed mode post-byte, of course.)

(8 bits of index extension should be plenty for the 6809, but we want to be able to still encode address function in 3 bits, so maybe 16 bits of extension total.

And forgetting the cache gadgetry that got me running back over this ground again.

In this scheme, PC cache just needs to fill ahead, and fill a second direction over one branch, checking whether the branch code is in cache first. 32 bytes per branch, 64 total would be plenty on a 6809.

Return address stack cache just needs spill-fill with 2/4 hysteresis in the middle. Eight entries would speed calls and returns a lot, Thirty-two entries would be enough to eliminate the need for return stack RAM on many embedded applications.

Parameter stack would have to be a bit more flexible than the return address stack, but would be similar in operation.

The DP cache would be a bit weird, but 256 bytes of cache, two, four, or eight banks, and address tag for each bank, with some sort of mechanism to automatically save a direct page that has been switched out, perhaps after another has been switched, and so forth. 

Of course, there would be system cache and user program cache for fast process context switching.

And, with all that, maybe not even bother caching general data accesses.

(Analyzing this for the 6801 here: https://defining-computers.blogspot.com/2024/11/how-address-function-and-caches-should-interact-6801-derived-example.html.)

A Bird's-eye View of an Alternate History Roadmap of the 680X and 680X0 CPUs

About the time I wrote a critique of the 680X and 680XX CPUs, I also started writing an alternate history roadmap for how they developed, but I got mired down in details and the alternate history novel I was trying to write then.

Now, after a few years, I'm working on a tutorial, and I wanted something to refer to. (Partly for the construction of the tutorial, but also as my personal notes from which I hope to eventually synthesize some workable extensions to the 680X and 680XX, should I ever be able to properly retire and play with programmable logic devices. Probably only decodable by me.)

So I'm going to write that alternate history roadmap here, but deliberately leave out details. Each stage will list extensions in order of my perceived priority and probably order of development.

Of course, this should all be considered to be some random wacko daydreaming out loud on the Internet. 

 

68/2805

Adds a parameter stack to 6805. Implements dual-port return address stack RAM to optimize calls and returns.

 

68/3808

Adds indexable parameter stack and Y register to 68HC08,  along with 68/2805 optimized call/return via dual-port return stack, etc.

68/0801

Extends the 6801, breaking object code compatibility, but maintaining one-to-one mapping from both 6800 and 6801 object code to 2801.

(1) Add SBX as complement of ABX, to aid in stack allocation, tracing relative links, and such.

(2) Move the inherent addressing mode op-codes around to facilitate adding direct-page op-codes for the unary (read-modify-write) instructions (INC/DEC, ROL/R, A/LSL/R, etc.). This would make the direct page more effective as pseudo-registers and ease the register pinch, and provide better support for such things as virtual machine registers and (hard-coded) per-task global variables.

ROM code would not be usable as-is, but a simple re-assemble would suffice in many cases. In those cases which relied on the actual object-code values of operators (as in for brittle optimizations), the assembler could flag jump and branch targets and fall-through that do not end up at valid instruction boundaries, and could analyze the value of the bytes at the target, issuing an error and an attempt to interpret the assembled value at the target as an instruction. This would allow the engineer to determine how to either provide a similar optimization or (more likely) replace the brittle optimization with something from the improved instruction set.

(3) Add 4 bits of address function outputs to allow decoding program ROM, general data, direct page, and stack separately, making it possible for the sum of system/user code, data, stack, and DP (I/O, global pseudo-registers) to be greater than 64K, make bank-switching more effective, allow isolation of system and user spaces, allow isolation of the return address stack, and potentially allow adding segment registers for engineers who like segment registers.

The index register (X) would be extended by the function code bits, to allow it to index the separate address spaces. Transfer of stack to X would set the appropriate function code, and operators for setting the function code for other spaces would be provided. (How to do this with system/user space separation?)

(4) Replace the 6801's ASLD and LSRD inherent instructions with a full set of 16-bit unary/RMW shifts ASL16/LSR16 (D/DP/[X]/EXT), and add 16-bit unary/RMW INC16/DEC16 (D/DP/[X]/EXT). This would be especially useful for things like synthesizing generalized stacks and queues and such in software, and in multiply/divide by constant powers of 2, etc.

(5) Built-in dual-port RAM blocks to optimize DP RAM and return stack RAM (call/return optimization). 

(6a) Add (two sets each for user and system) two 5-bit prefix registers for the DP lower and upper halves, lower half in system space would be nominally for I/O and upper/non-system would be nominally for global pseudo-register RAM. This would allow allocating DP spaces within the lower 8K of the physical address space, or within an 8K separate DP space, where internal dual-port RAM and I/O registers could be provided, and external I/O space decoded.

(6b) Optional standard bank-switching schemes for code and data spaces, for 128K to 512K, or for 2M to 16M. Bank switching includes address function decoding, to support isolation of spaces, also includes support for switching between user and system spaces safely.

(6c) For engineers who like segment registers, optional internal segment registers for user/system data, code, and return stack spaces, properly organized to support the dual-port RAM blocks for the process return stacks. (How to integrate this with separate stack and DP?) DP would not have additional segmentation.

 

68/0800

This would be a 68/0801 without built-in peripherals, ROM, RAM, etc., and with pin-out and timing that matches the 6800, to be used as a near-drop-in replacement for the 6800 -- requiring source code re-assembly, but with a high probability of being able to match the original timings. 

 

68/2801

These extensions to the 68/0801 would require op-code map extensions through one or more pre-bytes, and would pretty much implement all our real world's 68HC11 instruction and interrupt functionality, plus a separate parameter stack. Indexing from the second stack would still be by way of index registers.

(1) Extend the 68/0801 with a second (U) stack, providing UPSHA/B/X/Y and UPULA/B/X/Y, TUX/Y and TX/YU.

(2) Provide an extra Y index (more-or-less as in the 68HC11). Both index registers would be extended by 4 bits to support indexing the address spaces. Transfer of stack registers to X or Y would set the appropriate function bits, and operators to set the function bits for other spaces provided, probably as prefixed LDX/Y instructions.

(3) Add an address function code for the U stack.

(4) Add bit instructions per 68HC11, with a toggle bits instruction, as well. These would be added by way of pre-byte.

(5) MUL/DIV --  Add IDIV and FDIV per the 68HC11, using X and D. Add a 16-bit MUL X by D to X:D

 

68/31601

Extends the 68/2801. 

Make all instructions 16-bit internal. Extend the index registers by 4 more bits between the function bits and the logical address bits. Provide 4 bits of extension between the function code constants and the 16-bit logical address bits for PC, stacks, and Extended mode addresses.  Trade the bank switching and segmentation for page-mode memory management.

 

68/2809

6809 compatible, but 6801 equivalent cycle counts. Also, reference the 68/0801 address space extensions.

(1) DP-relative 8- and 16-bit offset modes added to index post-byte.

(2) 8-bit page zero mode added to index post-byte.

(3) IO Page register, with 8- and 16-bit offset modes in index post-byte.

(4) Address functions for data, code, DP,  IOP, return stack, parameter stack, interrupts and DMA.

(5) MUL/DIV -- X by D 16-bit MUL; FDIV and IDIV per 68HC11.

(6) 16-bit unary-operator shifts and INC/DEC.

(7) Bit instructions.

(8) Available as SOC core, like 6801 and 6805.

(9) Standard optional bank switching and/or segmentation.

68/31609

16/32-bit 68/2809, with 32-bit index, stack, and PC registers, 16-bit Extended mode offset register, and paged memory management instead of bank switching. Moves op-codes around for efficiency, adds second 16-bit accumulator. Has 32-bit ADD, SUB, CMP, MUL, and FDIV and IDIV. Dedicated 64-byte spill/fill hysteric caches  for both stacks, dedicated 256-byte PC read-ahead cache, dedicated DP and page zero caches.

68/28000

Basically the 68010 with improved timing; 32-bit offsets for indexing and branching; 32-bit multiply and divide; a system A6 register for system parameters; dedicated stack caches, 256 bytes for A6 and 64 bytes for A7, both user and system; dedicated 256 byte code cache for PC; 4K associative cache for other data; PEA that pushes on A6 or SEA that saves an effective address where the specified address register points.

[NOTE] newline insertion on paste bug in gedit

Intermittent bug: 

gedit inserts newlines on paste under certain conditions involving the contents of the search buffer.

gedit --version reports

gedit - Version 3.28.1

from the command line on Ubuntu, uname --all:

Linux {machine-name} 5.4.0-150-generic #167~18.04.1-Ubuntu SMP Wed May 24 00:51:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Looking it up, it might be something inherited from the GTK toolbox.

Specific details.

Searching through a text file for lines beginning with a specified string, such as a label in an assembly language source file. Leaving the search string in the search buffer, select, copy, and paste a section of several lines of text including the searched string. 

gedit inserts blank lines (newline character sequences) before occurrences of searched string.

Example:

MESS    DC.L    DOCOL,WARN,AT,ZBRAN
    DC.L    MESS3-*-NATWID
    DC.L    DDUP,ZBRAN    ; -DUP here is a bug from the original 6800 model, at least.
    DC.L    MESS3-*-NATWID
    DC.L    LIT16
    DC.W    4
    DC.L    OFSET,AT,BSCR,SLASH,SUB,DLINE,BRAN
    DC.L    MESS4-*-NATWID
MESS3    DC.L    PDOTQ
    DC.B    6
    DC.B    'err # '    ; 'err # '
    DC.B    0    ; hand align
    DC.L    DOT
MESS4    DC.L    SEMIS

with \nMESS in the search buffer.

But then when I tried it with 

* This should add two 64-bit numbers:
ADD64STK:
    MOVEM.L    (A6)+,D7/D6/D5/D4
ADDLO    ADD.L    D5,D7
ADDHI    ADDX.L    D4,D6
    MOVEM.L    D6/D7,-(A6)
    RTS

it quit inserting the newlines, even in the original example text.

Website about the TI Series 0100 Calculator Chips

This is another website I want to remember. It gives a lot of detail on early Texas Instruments and related calculators and the electronics that made them work:

http://www.datamath.org/

In particular, this page is dedicated to the TI's series 0100 chips that were in current production with Intel's 4004:

http://www.datamath.org/Chips/TMS0100.htm

 

 

No, Higher Costs Were Not the Real Reason for the 8088 in the IBM 5051

[This is a reply to a comment on Hackaday: https://hackaday.io/project/190838-ibm-pc-8088-replaced-with-a-motorola-68000]

*****

For some reason, I can't reply directly to your comment with the eejournal opinion piece [https://www.eejournal.com/article/how-the-intel-8088-got-its-bus/], but I suspect my earlier comment was too brief.

Let me try that again:

The 68000 had built-in support for 8-bit peripheral devices, both in the bus signals and the instruction set. Most of the popular implementations, including the Mac, made heavy use of 8-bit parts, and Motorola had application notes on interfacing other company's 8-bit devices as well as their own. You could mix 8-bit peripherals and 16-bit memory without stretching.

Motorola even had an app-note about interfacing the 68000 directly to 8-bit memory, but any decent engineer would have looked at the note and realized that the cost of 16-bit memory was not really enough to justify hobbling the 68000 to 8-bit memory.  That's one of the reasons the 68008 didn't come out until a couple of years later, and the primary reason that very few people used it. There was no good engineering reason for it.

Well, there was one meaningful cost of 16-bit wide memory: You couldn't really build your introductory entry-level model with 4 kB of RAM using just eight 4 kilobit dRAMs. (cough. MC-10.) You were forced to the next level up, 8 kB. 

IBM knew that the cost of RAM was coming down, and that they would be delivering relatively few with the base 16 kB RAM (16 kilobit by 8 wide) configuration. Starting at 32 kB (16 kilobit by 16) would not have killed the product. Similarly, the cost of the 68000 would come down, and they knew that.

Management was scared of that.

Something you don't find easily on the Internet about the history of the IBM Instruments S9000 was when the project started. My recollection was that it started before the 5150. It was definitely not later. It had much more ambitious goals, and a much higher projected price tag, much more in line with IBM's minicomputer series. There was a reason for the time it took to develop and the price they sold it at. But even many of the sales force in the computer industry didn't understand the cost of software and other intangible development costs.

Consider how much damage the 5150 did to IBM's existing desktop and minicomputer lines. Word Processing? Word Perfect was one of the early killer apps for the 8088-based PC. Spreadsheet? Etc.

IBM management knew too well that if they sold the 5150 with a 68000 in it instead of the 8088, a lot of their minicomputer customers were going to be complaining to high heavens about the price difference. They knew the answer, but their experience showed them that the too many of the customers would not believe it.

That was the real reason. They hoped the 8088 would be limited enough to give them time to maintain control of the market disruption.

I think they were wrong. But it would have taken a level of foresight and vision that very few of management withing IBM had.(very few outside IBM, either.), to take the bull by the horns and drive the disruption.

*****

Anyway, my point was that higher cost wasn't the real reason any more than the (at the time, much-rumored) technical deficiencies of the 68000.