Lost somewhere inside OpenMSX, please save me.

Page 1/2
| 2

By NYYRIKKI

Enlighted (5426)

NYYRIKKI's picture

20-06-2014, 00:16

Hi,

I just noticed that in latest version of OpenMSX the Finnish keyboard is working also on Console mode!!! As this opens up a whole new world, I wanted to try to make my second OpenMSX-script... How ever it seems that I'm just bouncing from one wall to another... There are not too many examples around and the web-manuals make me feel dizzy.

Now I feel I'm pretty much stuck... (and so is OpenMSX while trying to understand me) Anyone want to take a look & give me hint, how to take next few steps? I know that I'm doing wrong things and my code & approach is wrong as this has been purely trial and error one tiny problem at a time... it just happens when you put MSX coder in front of this kind of modern stuff Smile ... I just wanted to learn some basics, but now I've come to a point when I feel that little guiding to right direction wouldn't hurt...

proc typemsx { filename { channel stdout } } {

     if { [ catch {
        # Open the file, and set up to process it in binary mode.
        set fid [open $filename r]
        fconfigure $fid -translation binary -encoding binary
        set addr 0xfbf0

        debug break

        while { ! [ eof $fid ] } {

           #Convert byte from file as parameter good for poke-function (From "?"-type to integer or string... I don't know)
           set s [read $fid 1]
           binary scan $s H*@0 hex

           if {$hex=="" | $hex=="0a"} {} else {

              poke $addr 0x$hex
              incr addr

              if {$addr==0xfc17} {
                 pokew 0xf3f8 $addr      #Yeah... I couldn't get poke16 working
                 pokew 0xf3fa 0xfbf0

		 ###### THIS LOOP KILLS THE OPENMSX #######

                 while  {[peekw 0xf3f8]!=[peekw 0xf3fa]} {
                     debug step
                 }
                 ##########################################

                 set addr 0xfbf0
              }
           }
        }

        if {$addr!=0xfbf0} {
                 pokew 0xf3f8 $addr
                 pokew 0xf3fa 0xfbf0
        }

     } err ] } {
        catch { ::close $fid }
        debug cont
        return -code error $err
     }
     # When we're done, close the file.
     catch { ::close $fid }
     debug cont
     return
 }

proc pokew {addr val} {
	debug write memory       $addr        [expr { $val       & 255}]
	debug write memory [expr {$addr + 1}] [expr {($val >> 8) & 255}]
}
proc peekw {addr} {
	expr {[peek $addr] + 256 * [peek [expr {$addr + 1}]]}
}
Login or register to post comments

By NYYRIKKI

Enlighted (5426)

NYYRIKKI's picture

20-06-2014, 03:25

Hmm...

Now it actually already works, but gives out errors...


proc typemsx { filename } {

        global zzfid2
        global zzbp_id
        # Open the file, and set up to process it in binary mode.
        set zzfid2 [open $filename r]
        fconfigure $zzfid2 -translation binary -encoding binary

        set zzbp_id [debug set_bp 0x38 {} {[handlebreak $zzfid2]}]
}

proc handlebreak {fid} {

        if { ! [ eof $fid ]} {
           if [peekw 0xf3f8]==[peekw 0xf3fa] {
             transdata $fid

             #debug cont
           }
        } else {
             debug remove_bp "bp#1"
             # $zzbp_id
             ::close $fid
        }
}



proc transdata {fid} {
      #for {set i 0xfbf0} {$i < 0xfc17} {incr i}
        
        set addr 0xfbf0

        while { ! [ eof $fid ] && $addr < 0xfc10} {

           #Convert byte from file as parameter good for poke-function
           set hex [ getbyteff $fid ]

           if {$hex=="" | $hex=="0a"} {} else {

              poke $addr 0x$hex
              incr addr

           }
        }

        pokew 0xf3f8 $addr
        pokew 0xf3fa 0xfbf0
 }


proc getbyteff {fid} {
           set s [read $fid 1]
           binary scan $s H*@0 hex
           return $hex
}

proc pokew {addr val} {
	debug write memory       $addr        [expr { $val       & 255}]
	debug write memory [expr {$addr + 1}] [expr {($val >> 8) & 255}]
}
proc peekw {addr} {
	expr {[peek $addr] + 256 * [peek [expr {$addr + 1}]]}
}

By Manuel

Ascended (15979)

Manuel's picture

20-06-2014, 08:35

Can you explain what you are trying to accomplish? It looks a bit like typefromfile, which I created a while ago, see http://sourceforge.net/p/openmsx/openmsx/ci/master/tree/shar...

Some hints:
- the poke function also accepts values in int... so poke 12 13 is fine syntax.
- there already exist 16-bit poke/peek, check poke16 and peek16 functions (they are listed in the manual: http://openmsx.sourceforge.net/manual/commands.html ) EDIT: I see you "couldn't get it working". Not sure, but this should be simple, right? poke16 address 16bitvalue and the value is written at address and address + 1... it really works, the function does the same as what you wrote in pokew, see http://sourceforge.net/p/openmsx/openmsx/ci/master/tree/shar... )
- as for examples, there are many examples in the share/scripts directory of course. For the rest, it's just Tcl (many examples of Tcl are on the web).
- (detail: for strings use 'eq' and 'ne' instead of == and !=)

By NYYRIKKI

Enlighted (5426)

NYYRIKKI's picture

20-06-2014, 10:03

Manuel wrote:

Can you explain what you are trying to accomplish? It looks a bit like typefromfile, which I created a while ago, see http://sourceforge.net/p/openmsx/openmsx/ci/master/tree/share/scripts/_type_from_file.tcl

Don't worry, I know your script, I'm just trying to learn the basics... Most importantly, how to read/write binary files and how to modify MSX memory. This is because next I want to write script that moves BIN-file to MSX memory and one step at a time learn this damn language.

We can also once again end up talking about what is right thing to do and what is not... but when I start to make things my self then I don't really have to put my self against official OpenMSX-team opinions. How ever now that this works even somehow I can give some statistics:

Scenario:
I took Panasonic FS-A1GT (as that is the computer that I typically use) and let it boot with clock chip defaults. As test I tried my routine with MSXMEM2.BAS (gone trough SAVE"MSXMEM2.TXT",A) and my own custom BASIC test file.

Results:

Time to enter MSXMEM2.TXT on my program: 10 sec
Time to enter MSXMEM2.TXT on your program: 417 sec (=~ 7 minutes)

Time to enter my custom test file on my program: 2 sec
Time to enter my custom test file on your program: Could not be measured as it crashed to first Scandinavian character.

... but I know that you are doing it right and I'm doing it wrong. Please don't take this as attitude problem. It is just another way of looking things that works better for me. I'm not trying to fight against you or anything, just take few shortcuts that suit for me.

Quote:

Some hints:
- the poke function also accepts values in int... so poke 12 13 is fine syntax.

but this is where I have really hard time... I can't seem to figure out a proper way of getting data read from BINARY FILE to correct format... Maybe I just don't try hard enough, but hints are welcome.

Quote:

- there already exist 16-bit poke/peek, check poke16 and peek16 functions (they are listed in the manual: http://openmsx.sourceforge.net/manual/commands.html ) EDIT: I see you "couldn't get it working".

Yes, I really tried this, but I only get error:
can't import command "disasm": already exists

Quote:

- as for examples, there are many examples in the share/scripts directory of course. For the rest, it's just Tcl (many examples of Tcl are on the web).

Yes, but for example: "Read byte from binary file -> convert to integer" is something that Google really could't give me example. (If you find better examples, please share) It's kind of weird since I would consider this as a very basic thing for an programming language.

Quote:

- (detail: for strings use 'eq' and 'ne' instead of == and !=)

Ah... ok... thanks.

By NYYRIKKI

Enlighted (5426)

NYYRIKKI's picture

20-06-2014, 11:43

Little less errors:


proc typemsx { filename } {

        global zzbp_id
        global zzfid2
        # Open the file, and set up to process it in binary mode.
        set zzfid2 [open $filename r]
        fconfigure $zzfid2 -translation binary -encoding binary
        set zzbp_id [debug set_bp 0x38 {} {[handlebreak $zzfid2]}]
        after break {debug remove_bp $zzbp_id}
}

proc handlebreak {fid} {

        if { ! [ eof $fid ]} {
           if [peekw 0xf3f8]==[peekw 0xf3fa] {transdata $fid}
        } else {
             debug break
             debug cont
             ::close $fid
        }
}

proc transdata {fid} {
      #for {set i 0xfbf0} {$i < 0xfc17} {incr i}

        set addr 0xfbf0

        while { ! [ eof $fid ] && $addr < 0xfc10} {

           #Convert byte from file as parameter good for poke-function
           set hex [getbyteff $fid]

           if {$hex ne "" && $hex ne "0a"} {
              set temp 0x$hex
              poke $addr $temp
              incr addr

           }
        }

        pokew 0xf3f8 $addr
        pokew 0xf3fa 0xfbf0
 }


proc getbyteff {fid} {
           set s [read $fid 1]
           binary scan $s H*@0 hex
           return $hex
}

proc pokew {addr val} {
	debug write memory       $addr        [expr { $val       & 255}]
	debug write memory [expr {$addr + 1}] [expr {($val >> 8) & 255}]
}
proc peekw {addr} {
	expr {[peek $addr] + 256 * [peek [expr {$addr + 1}]]}
}

Now only errors that I get are:

Error executing callback function "message_callback": can't import command "show_osd": already exists
invalid command name ""
Error executing callback function "message_callback": can't import command "toggle_fps": already exists
invalid command name ""
invalid command name ""
invalid command name ""
.
.
.

(times: size of file /size of buffer)

By wouter_

Champion (421)

wouter_'s picture

20-06-2014, 11:56

I started writing this text before I saw Manuel's and NYYRIKKI's last reply. So there might be some duplicate information. (I'm at work and should actually be working instead of replying to forum posts ;-)

Hi NYYRIKKI,

Great that you're experimenting with openMSX scripting! Tcl can indeed be tricky sometimes, the interaction with the openMSX emulation model also sometimes gives surprising effects (see below). I didn't have time yet to study your script in detail, I'll just give some general info in this reply. I may take another look this evening. But feel free to already ask more specific questions.

One general hint to discover openMSX scripting utilities is to use the 'about' command. E.g. 'about peek' gives you list of possible commands (peek, peek16, vpeek, ...). And 'help peek16' gives more details about that specific command.

I personally find it useful to test short commands e.g. 'peek16 0xf3f8' directly in the openMSX console. Longer sequences I put in a text file and then 'source' that file: source filename (this has the same effect as if you typed all the text from that file). Once a script is finished I put it in the ~/.openMSX/share/scripts/ subdirectory so that it gets executed (sourced) automatically each time openMSX starts.

About your 'pokew' and 'peekw' procs, openMSX already offers the same functionality (with nearly identical implementation as yours). So if 'poke16' and 'peek16' really don't work for you we should investigate why, possibly there's something wrong in the scripts subdirectory of your openMSX installation.

Now about your first script. That approach doesn't work because of the way scripts and emulation currently interact in openMSX. Normally openMSX does 'emulation stuff', and this can get interrupted by a Tcl command. When that happens the _full_ command is executed before resuming emulation. So this means that when you execute e.g. a 'debug step' command in a script that 'step' is not executed immediately. In a way it only sets a flag that has effect as soon as the the script returns control to the emulation part. So executing 'debug step' in a loop has no effect. (And if you write an 'infinite' loop in a script this will indeed hang openMSX ;-)

If you execute 'debug step' commands interactively in the console, then openMSX does get the chance to execute the emulation stuff between the step commands.

About your 2nd script, I believe the error is in this line:

  set zzbp_id [debug set_bp 0x38 {} {[handlebreak $zzfid2]}]

To explain what's wrong I'll give some background info about how Tcl command evaluation works:

  • Every command in Tcl has the form 'cmd param1 param2 ...'. A 'set' command takes 2 parameters (variable name and new value). A 'while' command takes 2 parameters (condition and body). A 'debug' command takes a variable amount of parameters, but in the example above the parts are
    • debug: command name
    • set_bp: sub command
    • 0x38: address
    • {}: condition (empty string is interpreted as 'true', so this sets an unconditional breakpoint)
    • {...}: the action that should be executed when this breakpoint is hit
  • To figure out what characters of the command belong to what part (which parameter), Tcl applies some grouping rules:
    • command and parameters are separated by whitespace
    • but stuff enclosed in {} [] or "" is grouped together (in one parameter)
  • After the grouping but before the actual execution of a command, Tcl also applies some substitution rules:
    • Stuff enclosed in {} is passed unchanged to the command
    • Stuff enclosed in [] is treated as another Tcl command, that command gets executed, and the result of that command is substituted in the original command
    • Variable substitution: $varname is replaced by the content of that variable

(See other Tcl tutorials for a much better explanation of these rules.)

So now let's see what this command does exactly:

set zzbp_id [debug set_bp 0x38 {} {[handlebreak $zzfid2]}]
  • First grouping splits this command in 3 parts
    • set
    • zzfid2
    • [debug set_bp 0x38 {} {[handlebreak $zzfid2]}]
  • Next substitution runs. Substitution has no effect on the first 2 parts. The 3rd part is enclosed in [], so it first gets executed as another command. That command returns a string like 'bp#1', so the whole 3rd parts gets substituted by 'bp#1'
  • Now the 'set' command itself gets executed. This sees two parameters 'zzfid2' and 'bp#1', so it sets the variable 'zzfid2' to the value 'bp#1'.

Above we skipped over the interesting part: the execution of the 'debug ...' command. So let's look at that now:

  • First step is grouping, that gives:
    • debug
    • set_bp
    • 0x38
    • {}
    • {[handlebreak $zzfid2]}
  • Next substitution runs: parts 1 2 and 3 are left unchanged, parts 4 and 5 are enclosed in {} so those are left unchanged as well (only the {} are removed).
  • So the debug command sees 4 parameters: 'set_bp', '0x38', '' and '[handlebreak $zzfid2]'. Note that the last parameter still contains [] and $ characters!

When the breakpoint at address 0x38 triggers, the action gets executed. Following the same execution rules we get:

  • Grouping: the whole command is enclosed in [], so there's only one group
  • Substitution: the first (and only) group is enclosed in [], so first we evaluate the command 'handlebreak $zzfid2'. That command returns an empty string (there is no explicit return statement in the 'handlebreak' proc, so the return value of the proc is the return value of the last executed statement in that proc, and in this case that happens to return an empty string). So the first group is substituted by an empty string.
  • Now the command gets executed, and this rightfully complains that there is no command with the name 'empty-string'.

So the fix is to leave out the 2nd set of []. Like this:

  set zzbp_id [debug set_bp 0x38 {} {handlebreak $zzfid2}]

Note that the {} still 'protect' the 'handlebreak $zzfid2' part from variable substitution. Instead this substitution is now performed each time the breakpoint hits. Usually this behaviour is exactly what you want. In this specific case it doesn't really matter, and the following command would be correct as well ("" only groups but doesn't protect from further substitution):

  set zzbp_id [debug set_bp 0x38 {} "handlebreak $zzfid2"]

As I already said, I didn't try to point out every bug or every possible tweak in your script. But I hope this reply was already useful. And feel free to ask more questions.

By NYYRIKKI

Enlighted (5426)

NYYRIKKI's picture

20-06-2014, 13:36

wouter_ wrote:

About your 'pokew' and 'peekw' procs, openMSX already offers the same functionality (with nearly identical implementation as yours). So if 'poke16' and 'peek16' really don't work for you we should investigate why, possibly there's something wrong in the scripts subdirectory of your openMSX installation.

Found it... They started to work after I renamed "disasm.tcl" and "trainer.tcl" to new extension... I think these must be leftovers from some earlier OpenMSX installation. The other two weird errors disappeared after I got rid of "osd.tcl"

Quote:
  • command and parameters are separated by whitespace
  • but stuff enclosed in {} [] or "" is grouped together (in one parameter)
  • After the grouping but before the actual execution of a command, Tcl also applies some substitution rules:
    • Stuff enclosed in {} is passed unchanged to the command
    • Stuff enclosed in [] is treated as another Tcl command, that command gets executed, and the result of that command is substituted in the original command
    • Variable substitution: $varname is replaced by the content of that variable
  • Ok, I finally start to get some idea of what these things are... Until now for me they all looked like () in BASIC. Tongue

    Quote:
      set zzbp_id [debug set_bp 0x38 {} "handlebreak $zzfid2"]
    

    As I already said, I didn't try to point out every bug or every possible tweak in your script. But I hope this reply was already useful. And feel free to ask more questions.

    YES! This fixed the mystical problems I had. Now I finally have error free routine! This is good start... Now that I have own routine that works I can start experimenting with more advanced ways of handling files... ie. in this case I could read the file ie. in 30-byte blocks... Or then read the whole file first and then extract the data from memory, but I'm afraid it's not so easy as it sounds like. This binary scan and handling it's result (peek / poke / insert / delete data) is total mystery to me ATM. I know that reading stuff as text would be probably best idea, but I'm not trying to take best approach, but the one that is most useful for routines that I'm likely to make in the future.

    By NYYRIKKI

    Enlighted (5426)

    NYYRIKKI's picture

    20-06-2014, 13:56

    Ah, BTW here is the working version:

    
    proc typemsx { filename } {
    
            # Open the file, and set up to process it in binary mode.
            set zzfid2 [open $filename r]
            fconfigure $zzfid2 -translation binary -encoding binary
            set zzbp_id [debug set_bp 0x38 {} "handlebreak $zzfid2"]
            after break "debug remove_bp $zzbp_id"
    }
    
    proc handlebreak {fid} {
    
            if { ! [ eof $fid ]} {
               if [peek16 0xf3f8]==[peek16 0xf3fa] {transdata $fid}
            } else {
                 debug break
                 debug cont
                 ::close $fid
            }
    }
    
    proc transdata {fid} {
    
            set addr 0xfbf0
    
            while { ! [ eof $fid ] && $addr < 0xfc10} {
    
               #Convert byte from file as parameter good for poke-function
               set hex [getbyteff $fid]
    
               if {$hex ne "" && $hex ne "0a"} {
                  poke $addr 0x$hex
                  incr addr
    
               }
            }
    
            poke16 0xf3f8 $addr
            poke16 0xf3fa 0xfbf0
     }
    
    
    proc getbyteff {fid} {
               set s [read $fid 1]
               binary scan $s H*@0 hex
               return $hex
    }
    

    Now I can make long action scripts for MSX! Wink (One of the drawbacks on Manuel's approach was that you could not be sure if the MSX got your keypress.)

    By Manuel

    Ascended (15979)

    Manuel's picture

    20-06-2014, 15:38

    My method relies on the type command. It's a bit slow, because it is sure that the MSX receives the keypress... if it crashes with a scandinavian character, I'd like to know which. However, if the character can't be typed (by the built in type command), it's possible the script can't handle it.

    And of course you are very very very free to do whatever you like with openMSX and scripting! I'm just trying to help...

    More examples of "binary scan" can be found in scripts like _example_tools.tcl. I don't think you have to convert the value at all. I think you can use the value as is (don't use ne and eq for numbers, only for strings), if you use the proper formatString arguments for 'binary scan'. But I'm not experienced with 'binary scan' format strings at all, so that will have to be looked up.

    About your scripts dir: if I were you, I'd uninstall openMSX, remove any old stuff and reinstall it with the installer. Then you can be sure all is clean and working.

    By NYYRIKKI

    Enlighted (5426)

    NYYRIKKI's picture

    20-06-2014, 20:29

    Manuel wrote:

    My method relies on the type command. It's a bit slow, because it is sure that the MSX receives the keypress...

    There must be something wrong with your checks. If I start "type_from_file" when MSX is displaying boot logo the MSX has missed about 1-2 lines of text already before it even gets to BASIC.

    Quote:

    if it crashes with a scandinavian character, I'd like to know which. However, if the character can't be typed (by the built in type command), it's possible the script can't handle it.

    In this case the caracter was "Ä"... I think the main reason why it didn't work was because I tried to enter Scandinavian BASIC-program to a Japanese computer. To enter correct ASCII-code from the file to the program you should make sure CAPS is off, turn KANA on and press "9"... This approach would make the program to be in memory same way as it was in a file and it would work correctly when used on European MSX. How ever as you are now not emulating my computer, but my self the things are not quite that clear... I would probably just drop the "¨", so do conversion like "Ä" -> "A", "Ö" -> "O" etc. At least that would be better than not typing anything.

    Quote:

    More examples of "binary scan" can be found in scripts like _example_tools.tcl.

    ??? I found just one mentioning about binary scan from it and that was not file related at all. Maybe I have wrong version once again...

    Quote:

    I don't think you have to convert the value at all. I think you can use the value as is (don't use ne and eq for numbers, only for strings), if you use the proper formatString arguments for 'binary scan'. But I'm not experienced with 'binary scan' format strings at all, so that will have to be looked up.

    Yes, I'm a bit lost, what format I am using... I thought it was string as adding "0x" in front of it made it ok as a parameter. I think I was a bit too tired, sleepy and eager to get things running when I looked that binary-command parameters... Next time I'll try again on some morning after coffee. Smile

    Quote:

    About your scripts dir: if I were you, I'd uninstall openMSX, remove any old stuff and reinstall it with the installer. Then you can be sure all is clean and working.

    You mean it is not good idea just to extract new packages on top of old ones for 10 years in a row without checking the results? ... are you sure? Wink I think it works quite ok now, but next time I'll do clean install for sure, don't worry. I just need to backup the Finnish MSX tR & other similar custom test stuff first.

    By Manuel

    Ascended (15979)

    Manuel's picture

    20-06-2014, 20:52

    NYYRIKKI wrote:

    There must be something wrong with your checks. If I start "type_from_file" when MSX is displaying boot logo the MSX has missed about 1-2 lines of text already before it even gets to BASIC.

    Ah, it's possible that it doesn't work in that case (the type command wouldn't work either). We'd have to make the type command smarter to fix that.

    Quote:
    Quote:

    if it crashes with a scandinavian character, I'd like to know which. However, if the character can't be typed (by the built in type command), it's possible the script can't handle it.

    In this case the caracter was "Ä"... I think the main reason why it didn't work was because I tried to enter Scandinavian BASIC-program to a Japanese computer. To enter correct ASCII-code from the file to the program you should make sure

    Yes, that's because in the Japanese computer, that character doesn't exist and the type command uses the character mapping to type what it should. And for this character it doesn't know what to type (so which keys to press) to get this particular character, because it is impossible. How should it know which alternative to type?

    Quote:
    Quote:

    More examples of "binary scan" can be found in scripts like _example_tools.tcl.

    ??? I found just one mentioning about binary scan from it and that was not file related at all. Maybe I have wrong version once again...

    I said *like*, there are more uses:

    $ grep "binary scan" share/scripts/*
    share/scripts/_cheat.tcl:	binary scan $mymem c* values
    share/scripts/_example_tools.tcl:	binary scan [debug read_block VRAM $offset $nrbytes] c* myvram
    share/scripts/_scc_toys.tcl:		binary scan [debug read_block "$device SCC" 0 224] c* scc_regs
    share/scripts/_scc_toys.tcl~:		binary scan [debug read_block "$device SCC" 0 224] c* scc_regs
    share/scripts/_showdebuggable.tcl:	binary scan $mem c* values
    share/scripts/_vdp.tcl:	binary scan $binRep B* binStr
    

    They are indeed not about files, but that is not relevant, I think. the command works the same.

    Quote:

    think it works quite ok now, but next time I'll do clean install for sure, don't worry. I just need to backup the Finnish MSX tR & other similar custom test stuff first.

    The installer puts stuff in your Program Files (a.k.a. "System") directory, but you can put your own custom additions in your User directory and the installer will never touch them. See this section of the manual for examples on where the System and User dir are located for several operating systems: http://openmsx.sourceforge.net/manual/setup.html#romlocation

    Page 1/2
    | 2