How to run BASIC programs from OptROM

Thinking on how to run BASIC programs from OptROM, I dug up an old Compuserve post from the M100SIG archives.

Fm: Stan Wong 70346,1267
To: Denny Thomas 76701,40

Paul's on the right track about the Basic interpreter. The Basic code never moves but there is a rom call that interprets each tokenized statement. Other rom routines find the next statement to execute. It's been a while but my memory is sort of dim on the subject but each Basic statement in storage is prefixed with the line line followed by the statement tokens followed by a null. Three nulls indicate the end of the program (end-of-statement null followed by a line number of zero).

A small program could switch to the option rom, read a Basic line (tokenized) into ram, switch back to system rom, and then feed it to the interpreter loop. There is no housekeeping since Basic takes care of it. Oops, I forgot, as part of each line I believe that there is also the address of the next statement. That needs to be fixed up also. Anyway, it's a fairly starightforward proposition. If there is any interest I can dive back into my notes and re-figure out what's what (in any case I got all my info from Robert Covington's ROM maps in the DLs).

Seems like that would work. That would sure be something nice to have working for Rex users.

I guess one would have are some initial questions:

• Where to store the line of BASIC code to be run
• How do you get to/from the inner interpreter in a way that gets you back to the OptROM? What's the right entry point?
• What should the "next line" pointer be set to?
• Is there any initial setup/cleanup that we need to do to ensure we enter properly and exit cleanly back to MENU
• Ideally since the idea here is to make a turnkey BASIC program it would be nice to make sure we always return cleanly to menu, never get dumped to BASIC prompt. Or if we are using an environment running from OptROM that the user gets returned to that.
  

Anyway reading some of these old posts it is pretty clear why Idea from URII is so craptastic compared to Thought from SuperROM... Idea is written in BASIC!

Perl Bindings for VirtualT

VirtualT has turned out, unsurprisingly I guess to be a great development tool. I don't think I could have gotten as far on my Forth OptROM as I have without it.

It could be better though. Although it can single step through code it is lacking in breakpoint support or source level debugging. Ken graciously added a breakpoint feature to the CPU Registers window for me, but I need more. Debugging an OptROM containing a Forth VM running on top of a Model T VM is not the walk in the park that it ought to be ;-) .

In addition to debugging I would like to be able to use VirtualT for testing purposes. Once I get a subroutine or module working the way I want I typically create a battery of regression tests for excercising its functionality. This permits me to do serious load testing. It permits me to make significant code changes while knowing I have a safety net in my battery of unit tests. Essentially, the battery of automated unit tests is an executable version of the requirements for the software which I can and do execute on a whim. Typically that whim comes right before I check my code into the main source tree and label it.

I could do that with VirtualT as it is today... if I wanted to write my unit tests in C++, that is. Unfortunately I find languages like C++ completely unsuitable for writing test code in. Some people can do that... not me. Languages like C, Java, C# or even C++ are fantastic for writing your final, fully specified, highly efficient and effective end product. Tests on the other hand I write in whatever language I feel I can be most efficient and effective in writing as a programmer. Well, and the language I have the most fun writing. These days, that language is Perl.

So what would be a good Perl interface to Virtual T?

I would need to be able to create instances of the vm with specific properties, like whether to use Remem or not, BASIC ROM images, Option ROM images, file system images. Anything that is configurable today through the UI needs to be configurable through the API. I need to on a function call be able to load a BASIC, DO, or CO file to/from the virtual machine's memory.

I need access to the machine: reset, run, break, single-step. I need to be able to place breakpoints. I need to be able to read all hardware state, like the E0 register. I need to be able to jump to a specific location. I need to have access to all registers.

I need to be able to dump the display to a png file. I need to be able to feed characters into the keyboard buffer, and hook the character output routines. This permits interacting with the running vm in a way a user would interact with it.

I need to be able to connect to the serial port as a file steam. I need to be able to take direct control of serial hardware flow control input lines to the vm.

It might make sense for this API to be slightly abstract. For example, I may want to implement an instance of the API that talks over a serial link to a monitor program running on a real laptop. Then I could reuse some of my tests on a real computer. The fact is that there may still be some bugs in VirtualT so it makes sense to test on the real thing too.

Such a programming interface would really supercharge my Model T development efforts. As it is now, I am spending hours resolving problems that could be done in 10s of minutes since I am constantly having to do repetitive actions during compile-edit-debug. When your target is a machine with a user community in the 100's any wasted development time is really expensive.

Option ROM Programming Theory

In a comment from a prior post, twospruces (Steve, I guess?) asks whether there are two kinds of rom routines... those compatible with an option rom and those not.

I guess it depends on the trampoline code that you use, and the routine being called. The general idea is that there is code copied from main ROM to high RAM at boot. When code in option rom wants to call the main rom it runs some code like this (from romhd.asm):

StdCall:
    di
    shld    HoldH             ;Caller's hl
    xchg
    shld    HoldD             ;Caller's de
    pop     h                 ;(hl)=Routine to Call
    mov     e,m
    inx     h
    mov     d,m               ;de=Routine to Call
    inx     h                 ;hl=Return Address
    push    h
    lxi     h,EnableOptROM    ;return through EnableOptROM
    push    h
    push    d                 ;Return Address
    lhld    HoldD             ;Caller's de
    xchg
    lhld    HoldH             ;Caller's hl
    ei
    jmp     StdOn

This code saves HL and DE and sets up the stack with some return addresses to get to and from the main ROM code. It doesn't look like it damages any registers in the process. The EnableOptROM code preserves a by saving and restoring the psw.

EnableOptROMImg:
    push    psw
    mvi    a,1
    out    0xe0
    pop    psw
    ret

And this code, org'd at 0x85:

StdOn:
    push    psw
    push    h
    lxi    h,0x26c8
    xthl
    xra    a
OpExit:
    out    0xe0
    ret

This is in the Option ROM, but I guess it must slide into the main ROM right after the out instruction, like is done on a Remem inter-bank jump. Maybe it doesn't have to transition through high RAM?

I assume that the code at 0x26c8 in the main rom restores the a register before making the jump.

So I think you should be able to call any function in main ROM.

That said, all this stuff adds some overhead to every call. I can imagine if you decided to preserve less of the registers and take advantage of instructions like pchl, you could make a lot more lightweight far calls.

Anyway, I suggest using romhd.asm from club100 as a starting point. There is also code to call interrupt handlers in the main ROM.

Making Option ROMs

Repackaging Fig Forth as an Option ROM is an ongoing task...

I have ported the code to as8085 assembler/linker. Unfortunately I am having a problem with the .s19 file generated by the linker. Other than that, I seem to have a proper Option ROM. I based the Option ROM code of of ROMHD.ASM from http://club100.org

What I still need to do is make sure that it is using RAM memory as its user dictionary and for all state variables.

The biggest task left is making sure that all calls to the main ROM go through the far call code. This switches from OptROM to Main ROM, makes call, and gets back to OptROM. This has to be really slow... to keep things snappy I'd like to use Steve Adolph's fast text scroll routines and embed it along with enough of the text generator to get along, so I don't jump to main ROM for every character printed. Plus screen scroll should actually be faster than a regular Model 100/102!

I would like to keep one Fig Forth source code file. That means I need some macro that I can control that decides to either do a far call to the main rom or a regular call, and assembles the right thing automatically. EP3, "the Extensible Perl Preprocessor" looks promising since Perl is my string processing hammer of choice.

Ultimate VirtualT Trace and Command Line

Sorry Ken, I missed your post sometime back about what trace information I would like from VirtualT.

Trace information...

XML or YAML would be good so it's easy to parse.

If I could just get a running trace of:

• address of instruction
• length
• instruction bytes
• register deltas of value changes
• memory address and new/old value if a memory store operation is performed
• state of option rom (selected or not)

that would be perfect. I could use this as input to my disassembler.

For a command line version, it would be nice to be able to supply a .CO file, a main ROM, an option ROM, and maybe some .DO files that get loaded to memory. The .CO file would run. The Stdout would be the characters as written to the screen, or if an option were given, the contents of a .DO file dumped from RAM file system, or the serial I/O.

Oh, and it would be good to be able to limit tracing to a fixed range of code. So, if PC is in the range, then generate instruction trace. So, this would imply a command line option for this.

Multiple Entry Point Instructions

In compiler theory we have the concept of a "basic block." This is a unit of code which is only entered at the beginning and has a single exit at the end of the block. All code is made up of basic blocks.

My disassembler attempts to discover all basic blocks by reverse engineering the call graph. Normally a instruction is an atomic unit... it has only one entry point and one exit. Steve Adolph brought up some BASIC ROM code in which a given multi-byte instruction can be entered at more than one address, and so the code will execute in different ways.

I think my disassembler would do OK but it would pick one path and not the other depending on which call it reached first. As I disassemble I mark a table of addresses that hold the opcodes and postbytes as "visited." Once an address has been disassembled I don't attempt to disassemble it again.

The reason I have the "visited" flags is to prevent the disassembler code from looping forever disassembling the same code. However, since the actual entry point is different it might make more sense to disassemble both paths. Instead of deciding whether to follow a code path by the visited flag, I would instead decide based on whether the specific entry point has been visited as an entry point rather than as an intermediate byte. Since the goal is to produce code which is ready to be assembled, the alternative code would have to appear as a comment. ideally the longer code string would be the dominant one... otherwise the extra bytes would be disassembled as 'db' raw data.

The really difficult things to disassemble, at least automatically, are computed gotos, vector tables and self-modifying code. At least in the ROM there is no self-modifying code. I noticed some areas where RAM100.CO (which runs from RAM) modifies postbytes. The actual instructions themselves are never modified but the postbytes after the opcode are. In this case, the immediate byte indicating the i/o port is calculated and modified directly in the object code at runtime.

Minimal binary file transfer (concept)

From a recent email:

I've been thinking about how to make an absolutely minimal Model 100 file transfer application which requires no client but can transfer a binary file. Basically the idea would be that on the laptop you type

RUN"COM:98N1E"

which accepts and runs a very small program BASIC program that pokes in ML. Real short, fast... just enough code to read in and append raw binary blocks to a RAM file from the COM port and calculate a checksum as it goes. No flow control, the PC side is tuned to send packets at a fixed
rate that the laptop can sustain. No retransmissions supported... at the end of the transfer the decision is made to either keep the saved file or delete it depending on the checksum match. By and large there should be no errors on a local link.

This sidesteps the whole issue of not having a client on the laptop. Since there is no user interface there would be very few dependencies on the ROM so it could be made cross-platform.

On the PC side I would have a really short (probably 20 lines of Perl) command line program that computes the checksums, sends the canned BASIC code and the file. There is a cross-platform serial API for Perl that would work on both Linux and Windows.

The upload to PC would work similarly... RUN"COM:98N1E" on the laptop, and launch the desktop program... it downloads a short BASIC/ML program that can upload one (or all) files from the RAM file system to the desktop.

Comments?