News:

Building a 3D Ray Tracer  By stevmjon

Main Menu

PlayBASIC V1.64O (Work In Progress) Gallery

Started by kevin, January 27, 2013, 10:50:31 PM

Previous topic - Next topic

kevin

#45
   PlayBASIC V1.64O Beta 17 - Resource Binding Library

        So another session on and we're a bare bones implementation of data binding.   To handle the binding we use the #AddResource  directive followed by a literal string containing the file name (and path).   It's pretty dumb as this point as it only allows files to be bound individually.   This occurs during compile time currently so any time it sees the #AddResource directive it loads that media into memory.     The file is loaded as is, it doesn't understand the file, it's just being considered as pure chunk of data.

        Binding is one thing, using them other problem.   To help out, i'm throwing together some high level functions to grab/copy resources in particular way.  These will most likely be a as string, as bank, or as image if it's a image media type.   To query a resource we use it's imported filename.  This is a bit iffy, but it fits perfectly into how this embeds, so ya get that.  

        The example code bellow, imports a text file into the program memory (and the final exe when built).   To access the resource data in our program we use LoadResourceAsString$ function.   The query functions use the index of the resource, rather than the name.  There's FindResourceIndex(Name$) to find the index of the media in the table.  

    Example Usage:

PlayBASIC Code: [Select]
   Constant TextFile$="Files/Hello-World.txt"

#AddResource TextFile$

; Find this resources index , indexes range between zero and number of bound resources in the program
index=FindResourceIndex(TextFile$)
if index>-1

; load the text file resource as string
s$=LoadResourceAsString$(Index)

; split the string into this array
dim rows$(0)
s$=replace$(s$,chr$(13),"")
count=splittoarray(S$,chr$(10),Rows$())

; dump it to the screen
for lp =0 to Count
print rows$(lp)
next

endif


Sync
WaitKey





    Example Text File:



---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
Hello World

This is some text that's been loaded into the app and treated as an in memory resource.  

This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  
This is some text that's been loaded into the app and treated as an in memory resource.  

---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
 End of this file  is here
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------





kevin

#46
  PlayBASIC V1.64O Beta 17 - Resource Binding Library #2

       Dropped a few more test functions last night,  was going to add them all as internal bindings, but might split them in two and have the core stuff bound and the helper functions as a library.   Since the helpers are extra bits and aren't always needed.   So far we're got some functions that will load bound materials in as PB media as images and fonts.  Since you can do this manually you can modify the resource before you use it.  Which might be some form of simple decompression, decryption etc etc.      

      In this example we're binding the a text file, PNG and CRF font into EXE memory and then loading them directly on demand, without ever needing to save them out to disc.  

PlayBASIC Code: [Select]
   Constant TextFile$="Files/Hello-World.txt"

#AddResource TextFile$
#AddResource "Files/Uwlogo.png"
#AddResource "Files/Arial36.crf"


; Find this resources index , indexes range between zero and number of bound resources in the program
index=FindResourceIndex(TextFile$)
if index>-1

; load the text file resource as string
s$=LoadResourceAsString$(Index)

; split the string and remove chr 13's
dim rows$(0)
s$=replace$(s$,chr$(13),"")
count=splittoarray(S$,chr$(10),Rows$())

; dump it to the screen
for lp =0 to Count
print rows$(lp)
next

ThisBank=LoadResourceAsBank(Index)

Print ThisBank
For lp =0 to GetBankSize(ThisBank)-1
print PeekBankByte(ThisBank,lp)
#print chr$(PeekBankByte(ThisBank,lp))
next

endif

ThisImage=LoadResourceImage("Files/Uwlogo.png")

drawimage ThisIMage,400,300,false

ThisFont =LoadResourceFont("Files/Arial36.crf")

setfont ThisFont


text 200,200,"YEAH"

Sync
WaitKey





     Today's little task will be to hook into the LoadFont + LoadImage commands so they can find attached resources and automatically load them for memory for you.    The frame work exists now so it should be fairly easy..  Should be.. but you never know.


 Edit #1 - Load Font Resources

      So the first step is up and running.   If you bind a CRF into the exe, the load functions will load it from the memory for you.

PlayBASIC Code: [Select]
        // This directive imports the file data into memory/exe at compile time
#AddResource "Files/Arial36.crf"

// The load statement check if this file is actually a resource, if so it loads it from memory
Arial36=loadNewFont("Files/Arial36.crf",1)
setfont Arial36
text 200,300,"Loading directly from resource:"




      yes, it's that easy !
     


 Edit #2 - Load Image Resources (BMP/PNG/TGA)

      The second step is up and running also.   So if you bind a PNG,BMP,TGA image those images can be loaded directly from the resource buffer in memory.  They never have to be extracted to disc.

PlayBASIC Code: [Select]
        // This directive imports the file data into memory/exe at compile time
#AddResource "Files/uwlogo.png"

// The load statement checks if this file is actually a resource, if so, it loads it from memory
logo=loadNewImage("Files/uwlogo.png",1)

DrawImage Logo,200,300,true

Sync
waitkey



     

kevin

#47
   PlayBASIC V1.64O  Beta 17 - Download

     Here's beta 17, this one includes the completed Resource Binding features.

     Old Beta Removed, Newer versions bellow


kevin


  PlayBASIC V1.64O - Final Documentation Pass

         Switching over to documentation mode, since can't really think of anything else worth adding.  There's already been a considerable doc's update made during this revision, so the main task now is to tack in the important new commands wrap it all up and get it out there.   It would be nice if there was actual beta testing going on that used the new abilities, but when only two people bothered to download Beta17 (bmorris & Atlus), it just makes the release build betas.


kevin

 Slowly Documenting

    Have been dropped various sessions into the doc's where possible, but haven't got that far really..   Made a bunch of updates already, but such changes are soon lost in the scope of the task.   Have added in some more examples and screen shots as they come to mind,  but there's literally 1000's of them so making screen shots for them all isn't really practical from a time perspective.  

    The plan is to release the V1.64O revision at the end of the week, so there's not that much time left for anything  major.


   

kevin

#50
  PlayBASIC V1.64O  - ReadDir Tweaks

   While picking through the doc's found my way to the native ReadDir functions, which can be easy but the implementation isn't particularly pretty.   Which is a by-product of being one of the oldest command sets, so they tend to have a lot of legacy string thrashing in them.    If you haven't worked it out by now, string thrashing is bad news for performance.    Unfortunately this particular command is rather large containing support recursive scanning and sorting options.  Replacing it with the time I've available seems impractical, so tweaking it will have to do for the time being.    I generally don't use it, in favor of rolling my own with the FirstFile / NextFile commands.   You can find various examples of that in the source board..  

    Only had fairly quick look through the routines so far, but it seems that inside some of the scanning loops there's some redundant string operations.   These are operations that could be computed prior to entering the loop in question,  just doing that returned about 5->10% gain, but it's still way slower than doing it manually, when just scanning a simple folder at least.     Recursive scanning can be a bit of nightmare really, since the command executes in the same thread, so your program stops while the searching executes.  If you point it and large nested folder the process could take minutes.   Another reason to do it manually :)   See  GetAllFilesInDir



kevin

#51
 PlayBASIC V1.64O  Beta 18  - Optimizing

    While tuning up the run time builders, noticed some chunks of code that could be better represented, by changing the input format slightly.   The idea seemed simple initially but took most of yesterday to get working again, all because of one command declaration that was in a slightly different format.. Grrr.. But it's working now.  The benefit is we remove a slab of function calls from the start up and some dead functions.    Those changes and the updated pre-processor shaved another 28K off the retail runtime.  So V1.64O is currently the same size as V164N3 (at the moment)

    The plan this afternoon is to continue testing Beta18 and pick though some of the legacy commands looking for any easy performance gains.  So far it all seems stable, but ya never know.   Tempted to add some detection operations to the compiler for finding merge friendly operations, just to see what combinations are prevalent enough to make any difference.   My suspicion is that it would be beneficial, but really depends on the type of code people use.. So one routine might benefit, where as others won't at all...


Edit#1 - Sprite Locals

    While picking though the Sprite commands and noticed  the sprites local commands have legacy string support in them.    The following test just times the average cost of writing each data type to a sprites local data.    The simple stuff like Bytes/Integers etc are about the same, but the string writing around 150 milliseconds faster..   So this code in the V164O beta 17 runs 5 FPS,  while in today's build it's 35fps..  Suspect there's a little more room in it also...  



PlayBASIC Code: [Select]
; Create a Sprite
CreateSprite 1

; Attach a Local data area, 100 bytes long To this sprite
CreateSpriteLocals 1,100


Dim Time#(10)

Max=10000


Do
Cls

Frames++
Index=1
t=timer()
For lp =0 to Max
SpriteLocalByte 1,0,255
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Byte="+Str$(Time#(Index)/frames)
Index++

t=timer()
For lp =0 to Max
SpriteLocalWord 1,0,64000
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Word="+Str$(Time#(Index)/frames)
Index++


t=timer()
For lp =0 to Max
SpriteLocalInt 1,0,1234567
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Int="+Str$(Time#(Index)/frames)
Index++


t=timer()
For lp =0 to Max
SpriteLocalString 1,12,"Hello World"
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke STring="+Str$(Time#(Index)/frames)
Index++

print "Fps:"+str$(fps())

Sync
loop





Edit#2 - Sprite Locals Peek/Poke

     Have completely ripped all the legacy string thrashing from those commands and now it takes about 2->3 milliseconds to read/write 10000 strings into a sprites local bank.   The picture bellow is from this second version of the test.. 

PlayBASIC Code: [Select]
; Create a Sprite
CreateSprite 1

; Attach a Local data area, 100 bytes long To this sprite
CreateSpriteLocals 1,100


Dim Time#(20)

Max=10000


Do
Cls

Frames++
Index=1

gosub Test_Poke_Functions
gosub Test_Peek_Functions


print "Fps:"+str$(fps())

Sync
loop



; -----------------------------------------------------------------------------
; -----------------------------------------------------------------------------
Test_Poke_Functions:
; -----------------------------------------------------------------------------
; -----------------------------------------------------------------------------



t=timer()
For lp =0 to Max
SpriteLocalByte 1,0,255
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Byte="+Str$(Time#(Index)/frames)
Index++

t=timer()
For lp =0 to Max
SpriteLocalWord 1,4,64000
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Word="+Str$(Time#(Index)/frames)
Index++


t=timer()
For lp =0 to Max
SpriteLocalInt 1,8,1234567
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Int="+Str$(Time#(Index)/frames)
Index++


t=timer()
For lp =0 to Max
SpriteLocalFloat 1,12,123.456
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Float="+Str$(Time#(Index)/frames)
Index++


t=timer()
For lp =0 to Max
SpriteLocalString 1,16,"Hello World"
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke STring="+Str$(Time#(Index)/frames)
Index++


print ""
print ""
print ""

return






; -----------------------------------------------------------------------------
; -----------------------------------------------------------------------------
Test_Peek_Functions:
; -----------------------------------------------------------------------------
; -----------------------------------------------------------------------------



t=timer()
For lp =0 to Max
Result=GetSpriteLocalByte(1,0)
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Byte="+Str$(Time#(Index)/frames)
print Result
Index++

t=timer()
For lp =0 to Max
Result=GetSpriteLocalWord(1,4)
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Word="+Str$(Time#(Index)/frames)
print Result
Index++


t=timer()
For lp =0 to Max
Result=GetSpriteLocalInt(1,8)
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Int="+Str$(Time#(Index)/frames)
print Result
Index++


t=timer()
For lp =0 to Max
REsult#=GetSpriteLocalFloat(1,12)
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke Float="+Str$(Time#(Index)/frames)
print Result#
Index++


t=timer()
For lp =0 to Max
Result$=GetSpriteLocalString(1,16)
next
Time#(Index)+=Timer()-t
print "Sprite Local Poke STring="+Str$(Time#(Index)/frames)
print Result$
Login required to view complete source code





kevin

#52
   PlayBASIC V1.64O  Beta 18 - Download

     beta 18 contains updated slibs and a bunch of command set (sprite + file/disc) optimizations.

       Old Beta Removed, Newer versions bellow




kevin

#53
 PlayBASIC V1.64O  Beta 19  - Tweaks + Tuning

       Been picking through the command set while updating the doc's and making the odd fine tune where possible.  While picking through the History noticed there's some commands like GetSpriteRect that weren't in the docs.  This particular command was actually returning the wrong data, so it makes sense to address it there and then.      

       Spent most of yesterday looking for more 'speed up' conditions the compiler can silently make for you without much success.   The problem with such opt's is they have to operate identically, otherwise there's a risk some legacy code will break.  Have added a few more error traps to pick up when LABEL names within expressions.  Since when those things occur it'd just pop a generic Syntax Error.   They need to be a little bit more verbose..


    Here's an example.. There's more On the subject Here


PlayBASIC Code: [Select]
    ; Attempt to create a variable called WOMBAT, will fail
; since WOMBAT key is already defined as LABEL

Wombat = 45

end


; define subroutine called WOMBAT starting point

Wombat:
print "This is the Wombat sub-routine"
return




   
   This is pop's an error on the Wombat = 45 line ,telling us the that Wombat is previously been defined as a label.




EDIT #1 - Popping error between operators and labels

   
PlayBASIC Code: [Select]
    ; operations between labels are illegal, but it'll pop an error
; that WOMBAT is a label


; This would error
a = Wombat * 100

; This would error
a = 100* Wombat



end


; define subroutine called WOMBAT starting point

Wombat:
print "This is the Wombat sub-routine"
return




   It's still possible to sneak one through that, since the expression resolver tends to per-compute stuff out as literals,    eg...   Restore Wombat  



EDIT #2

        Added more traps on label names in structural statements.  Should catch a fair degree of those..  While looking at errors, have customized the command input parameter error statement, so it gives the parameter index  in the error message as well what data type it's expecting.       

eg   
   PositionSprite 1,x

Would spit out

       'PositionSprite' commands parameter #3 is incorrect, expecting 'Integer / Float' Data type


      Another one that you're probably hit upon at some point is a Expecting Equals error.   Theres a number of situations  where the pattern matching is expecting some user variable be followed by an equals token. But you'd even get this generic error when you forget to add some expression..   


eg.

      MyVariable

  Now spits out something along the lines of

    Expecting equals in assignment Eg. 'MyVariable = Expression'


kevin

#54
  PlayBASIC V1.64O  Beta 19  - Prototype Pass #1 Tweaks

      This is a bit of an oldie but a goodie that's a by-product of the skimming nature of the Pass#1 in the compiler.   During Pass#1 the compiler is only looking for Labels and function headers.  These are needed up front so we can work out what input / return parameters things like functions will have before building code. This is what allows  a function or label call to be used before the compiler has actually got to that chunk of code.    Otherwise you wouldn't be able to, you'd have to provide a list of prototypes.  Which can be rather tedious..  

      Eg,
PlayBASIC Code: [Select]
      ; call our function
HelloWorld(100,200)

; the function code
Function HelloWorld(Xpos,Ypos)
print Xpos,Ypos,"Hello World"
EndFunction




     Since the compiler works top to bottom, when it finds the HelloWorld keyword during pass #2, it searches the Function name stack (built during pass#1) for the parameter pattern to match the expression with.    if we didn't do this, we'd need to previously provide a prototype for every function call.  

     The problem with Pass#1 is that it's a dumb skim over the input code at text level.  So it didn't really look at the lines of code, it just searched for the function/sub headers.  All of the real parsing occurs  during pass#2 in the compiler, since everything is tokenized on demand during pass #2.    This creates a catch 22 with things like remarks blocks.   Since the pass#2 was the only remark aware pass, if you wrapped a remark pair around a function say, then the remarks won't have any effect.    Pass #1 would see the function header/footers,  so the function prototype would still exist.  Creating a bit of a WTF moment.

     This afternoon i've been updating the Pass#1 to at least skim for remark blocks.   So far it only supports the /*  to */ remark  blocks, but RemStart to RemEnd


     This example would parse differently in today's build than older versions.   Today's build skips the function block completely, so when you try and compile this code,  TEST() is now undeclared and PB doesn't know what it do with it,  so it errors..

PlayBASIC Code: [Select]
  /*  Multi Line COmment on single line */  print "Some Text"

test(10)

Print "Some more text to fill space "

/*

Function TEst(YEah)


EndFunction
*/



Sync
waitkey






 #If / #EndIF Directives Limitations

       Unfortunately, If you wrap an #IF / #ENDIF around a function /psub in the code and try to toggle this OFF / ON in the code, then this won't work for the same basic reason.    #IF statements and Constants etc only exist during PASS#2,  the Test() function will be picked up during prototyped in PASS#1, but that slab of code gets skipped during pass two.   This creates a situation where PB can't resolve the address to the TEst() function cal.  So you'd get an strange error along those lines.


PlayBASIC Code: [Select]
   Constant  Include_This_Function = NO

test(10)



#if Include_This_Function=yes

; Pass #1 will see this function regardless of the directive,
; since those are only processed during pass #2

Function TEst(YEah)


EndFunction

#endif



Sync
waitkey




 




  Edit #1:  RemStart / RemEnd Supported

PlayBASIC Code: [Select]
  test(10)

remstart
Function TEst(YEah)


EndFunction
remend



Sync
waitkey








kevin

#55
   PlayBASIC V1.64O  Beta 19 - Download

     beta 19 includes the updated pre-pass in the compiler and a bunch of error trapping tweaks that should make like easier.

      Old Beta Removed, Newer versions bellow


ATLUS


kevin


kevin

#58
 Threaded Queued Blits maybe ?

     Since the V1.64N2 revisions we've had threading support for the blit image functions.   This allows the entire blit operation to be pushed off the programs core and unto a second core.   Giving us the ability to write asynchronous rendering routines.  The only limitation is that there's no queue support.  

     Today's little task has been to drop in a way to queue up a series of blits, then perform them all on the second CPU core.. Internally there's a stack of potentially 64 operations.   Which might not sound like much but that's way more than we actually need, considering we're talking about calling functions that are for performing screen sized operations.    Under the current implementation,  the user simply turns on the queue threading mode  via BlitFXThreadIngMode 2 (Start Queue) then calls the blitimage commands they want.    These calls don't render anything, they store the call information on the thread stack.   When we're done preparing our To DO list, we set the  BlitFXThreadIngMode 3 this then launches the render thread.   At least that's the theory,  since it sure doesn't want to work yet..


Edit #1

        Ok, so it's couple hours later and i've finally found the problem with the queue and now it seems to be working, at least on my test machines.   Not too keen on the command names though, so will probably rename them as BlitImage  commands, since they're part of that family.

        The queue is executes in the order they'd saved.  So if you called BlitImageAlphaPostMultColour  when the queue is enabled, it'd stack like this,

PlayBASIC Code: [Select]
    ; call #0
BlitImageAlphaPostMultColour(Screens(FrontBuffer),0,0,$e0a090)
; call #1
BlitImageAlphaPostMultColour(Screens(FrontBuffer),0,0,$e0a090)
; call #2
BlitImageAlphaPostMultColour(Screens(FrontBuffer),0,0,$e0a090)
; call #3
BlitImageAlphaPostMultColour(Screens(FrontBuffer),0,0,$e0a090)
; call #4
BlitImageAlphaPostMultColour(Screens(FrontBuffer),0,0,$e0a090)


 

     So obviously when the thread executes, it'll no call #0, #1, #2, #3 , #4 in that order.   This allows you to set up a list of brute operations and run them externally to our logic.   This is really the whole idea of multiple CPU CORES,  it allows us divide and conquer.   The potentially this that threading can make your program twice as fast, that's assuming a best case scenario where the load can be balanced perfectly, which is just fantasy.  The reality is that you're program is bound to have an uneven load. 

     For example.  if we push off 5 milliseconds worth of rendering onto core #2, but core #1 only has 1 milliseconds worth of work to do before it needs to use the images core #2 is working on.  Then Core #1 ends up sitting around waiting for that task to complete, and we're losing most of the benefit.    The goal should be to lay out our program so we can interleave the two task together in such a way that they either take the same time or core #2 completes it's task long before we need the result. 

     Threading introduces a few no no's, the most immediate important one  would be ownership of image resources.   if you set up a render task and start the thread (run it on core#2), then the images that thread is using are now as no got zone for you're core#1, until the thread has completed it task.    This is because the two CPU's are potentially writing to the same memory at the same time.   This is not possible in single cpu (core) system, where you can get away with this type of logic, but it's won't work on multi core systems.    You can render to the same image, you just have to  make you're not drawing over, or reading from drawn over shared pixels. If you do, you'll wacky unpredictable results.. 

kevin

#59
PlayBASIC V1.64O BETA #20 (Retail Compiler Only Beta) - (Avail for Registered Users ONLY)  (3rd,July,2013)

     PlayBASIC V1.64O  Beta #20 sees the addition of the Thread Queue mode in the blitimage library as well as some changes to those functions names.    

     The blitimage commands are by design big job commands, meaning they do a lot of data processing, making them ideal for threading.   Threading for those who many not know, is executing a 'function' on a different CPU core.   So when we thread a blitimage function, PlayBASIC passes this task from our program onto some other cpu.   In systems that have more than one cpu core, or physical cpu we're thus executing two things at once.   Systems with a single core cpu can still use threading, but the  jobs are executed by the system in a 'one at a time' fashion.   Depending upon the size of the threaded job, this can still be of benefit to a single core system..

     This build of PlayBASIC supports two different types of threading.   We have an immediate single job mode an a queue mode.  We control these using the  BlitIMageThreadMode MODE  command

      Mode states

      0 =  Threading OFF, All blitimage calls are drawn by the programs core immediately
   
      1 =  Push Single Job,  When this mode is enabled, the next blitimage function called will push this task onto a second core and execute it now.  

      2 =  Start Queue.   This setting makes the  blitimage function calls stack up.    They're not executed until you set the Mode to Render QUEUE mode

      3 =  Render Queue.   Fires up a thread that renders the captured list of rendering tasks.  



      Commands

   WaitOnBlitImageThread()    - Waits for a previously started thread to complete it's work.  This is very important when threading, since we don't really know how long the system is going to take to execute our threaded task.  So when we need to use the images the threads are drawing onto/with we need to wait for it.  Other wise you risk the two cores writing to the same data.  Which is BAD :)


   BlitImageThreadMode Mode       - Set threading mode

   mode=GetBlitImageThreadMode()    - Get the current threading mode

   BlitImageThreadPriority(PriorityValue)   - Threads can have a priority (1 to 15), generally the higher the value the more 'important the system should take this request.  


   PriorityValue=GetBlitImageThreadPriority()    - Read the priority value

   State=GetBlitImageThreadStatus()   - Check if Threading is active or not ?   0 = Thread is no longer running.  1 = Running.  


   WaitOnBlitImageThread()         - Wait and clean up threads if need be.  



   SizeOfStack=GetBlitImageThreadQueueSize()           - Check the size (number of blitimage requests) on the blit queue.