PlayBASIC V1.64N2 / V164N3 (Work In Progress) Gallery

[kevin]

 PlayBASIC V1.64N3b - Compiler Tweaks

     While messing with the Pooyan example, noticed an issue with one the built in constants "PBCompileMode"  not responding correctly to the compile mode.   Fixing it turned out to be straight forward, but the fix made the keyword searching slower.    While looking over it,  there's obviously some baggage in those old routines, so figured I'd take the opportunity to tweak them up a bit.

    The parser performs a lot of searches trying to ID each token from the source code.   String searching is inherently messy and can be a real black hole in terms of performance, in particular when the keyword count gets high.    The structure of searches already minimize most of the work, but at some point you have to compare the likely strings together.  Even if your only comparing a hand fall each time,  that overhead soon adds up when you might be searching 1000's of times.    You can avoid much of this by adding a hash to represent the string.   The hash is just a little routine that skims the string and builds a 32bit pattern from it.    The pattern is used as further trivial reject during searches,  it's not a 100% match though, more an indicator of a close match.   So if the hashes are the same, you then compare the strings.

    So  result wise it's been pretty interesting.   I'm comparing PB1.64N2 (just because it's sitting on the desk top) with today's edition V1.64N3b.   The test code created with the following snippet.

PlayBASIC Code: [Select]

   for lp =0 to 999
         #print "Variable"+Str$(lp)+" ="+str$(lp)
         #print "Variable"+Str$(lp)+"# ="+str$(float(lp))
   next

   Sync
   WaitKey

 
   To build the test program I'm just  running this example (f7),  clicking on he Console tab and select all the text it spat out.     Then open a new project in the IDE and paste the code fragment 10 times into the main source.   So we end up with a slab of 'do nothing' code that's 20,000'ish lines long.   So there's lots of variable creation and lots of variable look ups.

   PB V1.64N2 results

Code Select Expand


Total Compile Time: 3438 Milliseconds
  Lines Per Second: 6406.34
   Number of Lines: 22025



   PB V1.64N3b results

Code Select Expand


Total Compile Time: 2286 Milliseconds
  Lines Per Second: 9634.73
   Number of Lines: 22025



     So about 30% faster compile time running on my 7 year old single core AMD system..  


Constant / Literal Strings Tweaks

     After picking over the other data types i've been looking at the string cache and not liking what i'm seeing.  There's just an excessive amount of string thrashing when searching for preexisting literals.   It does this so strings that occur more than once, aren't stored in the final program.   But the search was basically a linear search through the cache with some trivial rejects.  So it seemed like a good candidate for hashing the literal strings.   Which seems to pretty well, across a 10,000 line program with 1000 unique (but similar) string fragments.


    PB V1.64N2

Code Select Expand


Total Compile Time: 887 Milliseconds
  Lines Per Second: 11299.9
   Number of Lines: 10023



    PB V1.64N3b

Code Select Expand


Total Compile Time: 638 Milliseconds
  Lines Per Second: 15710.0
   Number of Lines: 10023


   So about 20% faster across this program.

[kevin]

PlayBASIC V1.64N3b - Retail Upgrade Released

    This update includes a fix for some issues with some built in constants (like PBCompileMode) not responding to the compiler settings.    Beyond that, it also includes a number of mostly small tweaks to the variable/constants and literal searching  routines.   These tweaks improve the compile times by around 30% in programs that make heavy usage of lots of unique variables / constants or literals.     Routinely getting a compile performance of 10,000 plus lines a second on 7 year old single core AMD system.


Download

   Download  PlayBASIC V1.64N3b Upgrade  (login required)

[kevin]

 PlayBASIC V1.64N3c - Compiler Tweaks Round #2

    Been off working on some other bits and bobs in the real world lately, but have been using PB for a lot of the grunt work, as such have noticed some oddities in some program disassembles.  The first being that the code didn't seem to screen out redundant moves between the same variable.     So if the code had the line  A = A, it'd drop a move operation for it, even though it's pointless.   Not a big deal for the odd operation, but will certainly add up if such code exists in a tight loops.

   Another oddity was occurring when reading the fields from a typed variable/array/list and storing that back in a variable.    So if you had code like X#=Player.X#,  the code generator would always drop an extra move operation.  Not a big thing in every day code, but can quickly magnify in heavy loops.  There's not a huge gain from this, it's more about stream lining the operations.    This can be seen when running the brute force bench, as there's only a about 1.5 milliseconds gain across the entire test.    It'd be very handy if you cache lots of fields or they're modestly strings.  

[kevin]

PlayBASIC V1.64N3c - Compiler Tweaks Round #3

     Dropped another session into peeking through some dis-assemblies and it would seem that reading from cached type fields also had the same problem where it'd drop an extra move opcode all the time, where it's only needed some of the time.   So tweaking the code generator again and those bogus moves vanish. 

     The tests bellow compare V1.64N3 retail with today's build of V1.64N3c, both from the IDE.  The results are pretty interestingly in that even in such a small and simple test there's some gain to be had, most of that is going to be from the reduced  string thrashing, but ya get that.

PlayBASIC Code: [Select]

   Type Stuff
         x,y#,z$   
   EndType

   Dim me as stuff
   
   
   me=new stuff

   me.x=123
   me.y#=45.66
   me.z$="yeah"
   


   max=10000


   Do
      cls

      frames++
      optexpressions 1

      t=timer()   
      For tests=0 to max
         x= me.x
         y#= me.y
         z$=me.z
      next
      tt1#+=Timer()-t
      
      print tt1#/frames
      
      optexpressions 3
      
      t=timer()   
      For tests=0 to max
         x= me.x
         y#= me.y
         z$=me.z
      next
      tt2#+=Timer()-t
      print tt2#/frames
      
      Sync   
   loop   
   

[kevin]

PlayBASIC V1.64N3c - Retail Upgrade Released

    This update incorporates this weeks compiler and optimizer tweaks.  Since the change only alter code generation, it can be used with V1.64N3 runtimes, which haven't been included. 


Download

   Download  PlayBASIC V1.64N3c Upgrade  (login required)

[kevin]

  PlayBASIC V1.64N3d - Compiler Tweaks Round #4

       Been working on various tweaks to PlayMAPPER this last week, and even though i've been able to rip a few 1000 lines out of it,  the dieted version is still pretty big, weighing in just over 20K lines.   Which seems to build (on average on my system) from V1.64N3 in about 3850 milliseconds.  So almost 4 seconds to a person, which is lightning fast compared to the good old days.   The build time is about 250/350 milliseconds quicker in  the B and C revisions of V1.64N3 since they feature some of newer search optimizations, but there's a lot of function searching and Type Field searching in the program.  And while there is in most programs, it seems to really impact the performance here.

       So thought i'd take a look and see if the same thing that was applied to the constants/variables stuff would apply to everything else starting with Functions  / TypeFields, if should, but just ya never know.    Getting the function stuff working seemed to go without much of a hitch really.   The type fields weren't so pleasant.    But after a few hours of head scratching and good old fashion mind numbing fun,  it seems to be working as normal again.    The net result of what turned out into a few hours,  is the build time is now averaging 2947 Milliseconds for the same project, so a second faster, which is very noticeable improvement to the user.

      This might not sound like a big deal, but such changes not only make the compiler far more able to handle bigger  programs without that rapidly raising the build time.   They're what allow you to Edit->Test->Edit->Test in rapid succession.   For many the process is largely transparent.  Which is how it should be.




   

[kevin]

 PlayBASIC V1.64N3d - Compiler Tweaks Round #5

     Made a few more tweaks the search routines tonight,  but haven't really been able to win any serious time back.  The build time felts really speedy on normal sized projects, which should help with productivity.   Another tweak I've been messing with is screening out some redundant expressions in integer operations.   I think the C revision has some of these, but the D revision includes a few more.   Basically it's trapping expressions with one of the terms is literal zero/one or two.   So an expression A=A+0  would have generate the addition operation, with possible a move operation.  But now it produces nothing.  Some other situations are thing like mults.  Where you might have a A=A*1.  Which will get stripped out.    

     In terms of the runtime performance, Addition is generally quicker than Multiplication.  So knowing this, we can recast Integer expressions that include a 2 * something in them, like the expression A=B*2, this is recast as A=B+B.   Today's build does this for you invisibly.    PB actually includes a number of these types of tweaks today.     In terms of performance,  the bench bellow runs about  1/2 a millisecond quicker (15->20%) over 100K additions.  

PlayBASIC Code: [Select]

   max=10000
   Do
      cls 
      frames++

         t=timer()
            For lp =0 to max
                  A=B*2
                  A=B*2
                  A=B*2
                  A=B*2
                  A=B*2

                  A=B*2
                  A=B*2
                  A=B*2
                  A=B*2
                  A=B*2
            next
         tt1#+=(timer()-t)      
         print tt1#/frames


         t=timer()
            For lp =0 to max
                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B

                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B
            next
         tt2#+=(timer()-t)      
         print tt2#/frames
      


         t=timer()
            For lp =0 to max
                  A=2*B
                  A=2*B
                  A=2*B
                  A=2*B
                  A=2*B


                  A=2*B
                  A=2*B
                  A=2*B
                  A=2*B
                  A=2*B
            next
         tt3#+=(timer()-t)      
         print tt3#/frames


         t=timer()
            For lp =0 to max
                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B

                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B
                  A=B+B
            next
         tt4#+=(timer()-t)      
         print tt4#/frames

      Sync
   loop

     

[kevin]

PlayBASIC V1.64N3d - Retail Upgrade Released

    This update incorporates this weeks compiler and optimizer tweaks (Faster keyword searching, various code generation redundancies built in).  Since these change only alter code generation, it can be used with V1.64N3 runtimes, which haven't been included. 


Download

   Download  PlayBASIC V1.64N3d Upgrade  (login required)

[kevin]

 PlayBASIC V1.64N3d - String Comparisons

    The past week or so i've been noticing some odd behavior in one of the helper tools, which is of course written in PlayBASIC.   The behavior initially seemed to be logic error in one of the larger routines, so at first didn't worry about it, assuming it'd be a quick fix when it came time to look at it.   The mood soon changed after sifting through the code trying to locate the issue.   The particular section seemed to produce irrational results when comparing for inequality between two strings.   The more I looked the more bizarre it seemed.  

    Initially was thinking this was likely to be a by product of some recent change to code generation, where the optimizer may have been breaking this particular sequence of the instructions.  But after firing up the diss-assembler, it was fast becoming apparent the cause was else where.   Meaning the actual comparison might have been broken.  The string engine is one of the oldest parts of the runtime, we're still using the second generation of it from about 7 or 8 years ago.   But it has been tweaked over time.   It's those tweaks where little behavior changes occur.    

    After some serious scratching looking over and over the string functions, it turned out there was an issue with the string <> comparison.   At some point the routine has been updated to support batching groups of characters, but the loop expression logical was faulty can could skip most of the iterations most of the time, making for all sorts odd behavior.   But it seems fine again now.    It's worth noting that this fault occurs after V1.64L learning edition, as those builds don't seem to suffer from it.


     It might be a while before the next release, so in the mean time you can invert the state of the EQUAL. 

PlayBASIC Code: [Select]

   a$="Test"
   b$="Tes2" 

 ; So this.. (Which is a bit iffy in V1.64M -> V1.64N  revisions)
  if    A$ <> B$
     print "<>"
  endif

 ; can be expressed as

  if  not (  A$=B$)
     print "<>"
  endif


; or as
  if  1-(  A$=B$)
     print "<>"
  endif


; or as
  if  A$=B$
  
  else
     print "<>"
  endif


sync
waitkey