PlayBASIC Image Format Development Blog

[kevin]

PlayBASIC Image (PBI) Format Development Blog (Dec 2015 - Feb 2016)
(Last Updated: 29th,Dec,2022)

    This thread documents the mid to final development stages of PlayBASIC's internal Image Format.   The goals for this work really boil down to handful of important requirements, such as a format needs to be lossless and compressed to enable fast loading/saving.    Now given it's designed for use with PlayBASIC and only PlayBASIC,  extensions can be embedded to handle animation, mapping, or even fonts for example.  
   
    Important: The PBI format is very much a work in progress, so things are likely to change and change often until a final spec has been decided upon.

   

FAQ about PlayBASIC Image Format

Q. Is there a size limits a PBI can be ?
       A.  Yes, the current PBI file format spec is 32bit, so image sizes are limited to be being 2^32 pixels wide or high.     Which is the same as most common place format such as TGA/BMP/PNG/JPG and various others.  


Q. How many colours can PBI handle ?
       A.  It's a 32bit format where pixels are stored in 8bit ARGB format.    This gives you a 2^32 possible colour palette, although in reality there's only 2^24 visible colours.  


Q. Can PBI support Alpha Channels ?
       A.  Yes, each pixel is stored as 32bit ARGB format, so each pixel includes 8bits of alpha, 8bits of Red, 8Bits of Blue, 8bits of Green.  


Q. Are PBI's compressed ?
       A.     Yes & No, the internal format uses colour reduction when converting the source image, where the source is broken up into variable colour density's, which can then be palette mapped.     The mapping is done within user controlled zones,  so the conversion algorithm scans the zones, colour reduces them, maps them and drops them out to the PBI.    The goal is that each mapped zone is less data than the original raw 32bit zone.   So the size of the zones you picked when creating your PBI can give very different file sizes.    


Q. How good is the compression ?
       A.  The colour reductions (well data shuffling) let us represent sections of the source image with less bits, that's all.     What this means is that areas that are sparse with very   colours compress very well, where as areas that packed full of unique colours will pack poorly.  

         For example, if you had a zone that's 16x*16y*32bit pixels (1024 bytes), then at PBI's lowest bitrate of 1bit, that zone only takes around 32bytes to store.     If you assume the worse case (each zone is full of unique colours) a zone of 16 colours at this size (16x*16y) would be 192 bytes,  where as 8bit zone with 256 unique colours is 1280bytes.   In the case of an 8bit zone,  the size can be as low as 324 bytes, ranging up to the1280 bytes .    All depends on the number of colours contain within the zone.

         So getting back to the question, the compression gain from a PBI conversion will generally be much much better than storing the file as RAW, but worse than string based compression formats like PNG (which use Deflate compression).   So on disc the file will generally be larger, but this is negated when the PBI's files are packed into a distribution such as RAR/ZIP/7z file for example.    Where all the colour reductions in the PBI files, make then easier for the archiver to compress, making some significant gains in distrubation size.    


Q. Could PBI be used to store photo's ?
       A.   Yes.   But warned though, remember this a lossless format, so the output file size is going to be large.    During testing, I've been using some holiday snaps that are around 4000* 2500 pixels in size.  Saving them as 32bit BMP these are around 40 megabyte images (on disc),  converting to 32bit PBI drops them to around 24-25 megabyte, where as 32bit PNG they weigh in around 19->20 meg.      However if you zip or 7z the PBI's there around 14->15 meg zipped,  and 11-12 meg 7zipped.    


Q. Does PBI support animation ?
       A.  Not in todays build, but that's the plan.


Q. How will I load a PBI into PlayBASIC ?
       A. You load PBI images just like any other image in PlayBASIC, through the   LoadIMage /LoadNewImage commands.  



Q. How fast is PBI loader ?
       A.  Even though it's early days, the initial tests are proving to be very positive showing 1024*768 size images loading in about the same speed as raw bitmap.    Loading a PNG of the same image is about 3->5 times slower than PBI.    So it's looking very positive in terms of runtime performance.



Q. is PBI a windows only format ?
       A. No, the format doesn't haven't any constraints that tie to any particular platform..    


Q. Does PBI support media protection ?
       A. Yes & No, the current design of the format doesn't explicitly try to hide your media, but since it's a PlayBASIC only format,  3rd parties outside of the PlayBASIC community  can't easily rip your media if that's your concern.

               So the obscurity of the format will give you some protection from media rippers.


Q. How do I create PBI's of my existing media ?
       A. The format made it's first appearance in PlayBASIC V1.64P4 around beta 7.   That version comes with a new SLIB  called "SavePBI", which includes two saver functions. There's the basic saver called SavePBI that will estimate the zone size for you.  There's also a secondary version called "SavePBIex" which includes a zone width and height parameters, which means you can compression as you require.    

PlayBASIC Code: [Select]

   // include the PBI saver library
      #include "Savepbi"
      

      // load the file into memory
      SrcFile$="SomeImage.bmp"
      THisIMage=LoadNewIMage(SrcFile$,2)


 
      // Save this image out as PBI  (default zone size)
      DestFile$=replace$(SrcFile$,".bmp",".pbi")

                // SavePBI(FileName$, ImageIndex)
      SavePBI(DestFile$,ThisImage)            

      Sync
      waitkey

Q. Are PBI loader/savers backward compatible  ?
       A. That's the long term goal, but I doubt the initial version of the format will be the final, so there is likely to be some changes made between builds.    



Q. Are the PlayBASIC help files available

     PlayBASIC Help - PBI

[kevin]

   PBI Conversion + Saving running (2016-02-11)

         The PBI load/save library is taking shape after a few weeks of (on/off) work.      The conversion routines zone the input data, reduce to a unique set of colours and then dump it out.  Even at this point the conversion gives us some nice gains with minimal effort.   The pictures bellow demonstrate the how the conversion code tries to merge  zones of best fit for the image.    The first image is a low colour very simple image and other is a normal game styled sprite.   The in the second picture you can see most areas of low colours can be ripped from the image data completely.   This gives us a near 50% saving on the actual image size, now of course that's not going to happen in every image you convert though...

[kevin]

   PBI Conversion/Building & ReLoading Working (2016-02-13)

      Crossed something of a milestone this morning where the code base now includes not only conversion/PBI builder( in memory), but the first edition of the loader into a working and seemingly stable form of the library.     The perfromance of the loader (decoding from memory is pretty good and tends to create the new image in about about 10% of the creation time.    Which means in real world terms means a few milliseconds from once it's loaded into memory to being a ready to use as a surface.  

[kevin]

   PBI Tweaking (2016-02-17)

        Been testing the conversion on more and more images lately while looking for tweaks to the data format.  Found a couple of issues where the updated code base wouldn't work on some of the bigger pictures anymore.   Took the best part of a day to find that the C/C++ compiler was signing a cast when it shouldn't have.    After that was fixed all the original test images worked again.    Then right at the end of yesterday found some really simple new images would fail,  stuff like a checker board pattern.   Thankfully that one a lot easier to track down,  as the merge code was missing some checks to make sure the sections use the same palette.   So in those cases the colours would be wrong in the output.   Since then it's been working pretty well.

         While looking around for some old images remembered the raw Donkey Kong Country Level from one of the old demos.  Which is enture level of the game rendered as an image.   I don't have the image on this computer so i can't recall the dimensions, but was something like a 20 meg picture.  Was surprised when just loading and saving it as PBI drops into the 850K mark.   The surprising thing about it is that PBI doesn't have an tile maping in it as yet.   Sp the gains are just through the variable bit rates.      This was the image i was using test testing Play-Pic To Map.   Which converts the source into a unique block list and level map. (link bellow)  
 
          Th format at this point doesn't have run-length or sequence compression in it, in favor of using the packaging compressor to get the most gain.   I have been testing a sequence compressor with it, but results aren't as good as just zipping them at this point.   z7 can make even bigger gains on the data pool.     Picking that zone size is really important though, too small and too many zones create data overlap, too few and you can end up with 32bit raw zone in the output.  How they compress at the end is anybodies guess though.  So any converion tool is going to need to try a few different sizes and test the output.    Depending on if you want smallest zipped size or smallest raw size(on disc/in memory).  



Related To:
 
     Play-Pic-2-Map V0.04 - Double Packing Pictures (login required)

[kevin]

     PBI - Folder Compression Results: (2016-02-18)

         Since last session I've been setting up a test that compress a folder of pictures rather than just one at a time.   So far I've been able to make gains on virtually every picture I've tried, but's on a very one to one basis.   I was bit worried such gains wouldn't work when applied generally to a random set of pictures, using a common zone size rather than a size that suits each image.    Shouldn't have worried though as across the 25 picture collection ranging from small sprite images through to large pictures,  post compressing the folder with zip gives a 7-> 8 megabyte gain comparing a zipped archive of the images in their original format.    If you 7z the folder of in PBI the gain is around 11-> 12 megabyte.    

         So we're proved there's some easy gain to be had,  but the really interesting thing will be exactly how much gain can be had buy optimizing each image separately.  

[kevin]

PBI -  Smallest Distribution Size -  7z / ZIP Testing  (2016-02-20)

     Seems like most of the work lately has been invested in testing a wider variety of pictures and setting up the test environment code.    The news so far is that no new gremlins have been uncovered with the existing conversion/loader code (on lots and lots of images), and the current test code is being tweaking up to form basis of the conversion program.                    

     Last update, the converter was improved to handle image folders, with a user defined zone size.    The output size of the PBI's will vary a lot,  unless they're all similar sizes, but even then what's in picture dictates the output size.   Generally the PBI will be smaller than the raw input size,  it's typical to see 20->30% reduction, up to 70->90%  reductions (compared to raw 32bit version of image).     The smallest possible PBI currently is 56 bytes.

    Anyway, there's two things we need to think about here,  the PBI files size on disc (in memory) and it's zipped size when we distribute our program (game).     So in order to find this out we needed a way for the tool would search for a good zone combination for us.    

     To compute the best zipped size, we run through the image a bunch of times dropping it into a temp state, then zip each one and then look for the smallest one.   Now what's interesting is that it's rarely the smallest PBI.    By that I mean, the PBI file that's the smallest on disc (in memory)  doesn't necessarily zip the best and if we want smaller distribution sizes, that's not the file we need.     From the limited testing thus far, it appears that a couple of odd sized zones tend to produce good results when zipped.  

     I think when all this is stuffed into a tool, then the tool should remember the previous best settings of each image.   As doing a split test on each image is pretty slow as it's calling an external program Z-zip to do the compression each time for big images (biggest image tested so far was about 3800x by 2000y)..  Can probably find a way to speed this process up in the future, but for now that's not a big concern.  

     Anyway...  It works and works very very well... ;)

[kevin]

PBI - Duplicate Zone Detections:  (2016-02-24)

      2D games often use tiled artwork in them for tile maps, but they also appear in character animations.   These tend to make up the bulk of the artwork in a 2D game.    So what generally happens is that either the artist or  programmer will pass such images set through a block reduction tool (editor/import conversion) reducing it to the lowest set possible.     We wrote such a tool a while back for this purpose (Play-Pic2Map).   But the same type of process could be built into the image format itself.    

      I've been testing doing block reductions in the PBI saver across the image set the last few days, wasn't sure it'd be worth the effort initially so wanted to see how well it'd work before embedding it into the library, thus the scanning was doing in high level PlayBASIC code.   Where the theory is if you can get it fast enough in PB, then it's a none issue in C/C++ (generally :) )  -  The results seem to be pretty good thus far, providing the image has common elements in it.    Sometimes such it's  obvious, but it can also be surprising where repetitions appear.  

      The duplicate blocks are removed from the output stream and represented as only a rect list.   The loader, decode such areas by copying the rect from the source location to the targets.   So a duplicate block costs us 24 bytes in 32bit mode.   In reality most images will be less than 2^16 wide/high so they use a 16 or 8bit structures.   Which sounds like overkill until you think at a big image could have 1000's of the duplicate rects in it.  

     Now not all images contain duplicates though, so those images won't gain anything from such a pre-pass, but it doesn't impact on output size of them either.  They'll be same size as before.      


Gradient Tests:

      Here's an interesting little tid bit I should have noticed before, but apparently never had concerning gradient image compression.    What's a gradient it's a those bands of colour, you often see them in websites backgrounds, or behind logs in the game title screens for example (See ShadeBox command).

  Anyway, if you store these images as art, then they compress pretty well generally.     Vertical/Horizontal gradients can only be a maximum of 2^8 colours (256 colours) if they're 2 colour (interpolating colour A to B),  which I knew, but what I hadn't really thought about was that if you cut such an image down in smaller sections,  there's generally less than 16 unique colours in these zones meaning you can store in a 4bit, even 2bit in places.      So I surprised that the test gradient images compress really well in PBI..  

  Obviously, it'd make more sense to use code, or a bilinear a 2 pixel image, but that's not always possible.

 

[leopardps]

As I understand, you are creating PBI to make for fast(er) loading of images.  Seems like alot of work to accomplish something that is usually only done once (except perhaps in special circumstances).

Since rendering speed seems to always be an issue, I have thought that an internal image format which is optimized for rendering would be handy.  My initial thoughts have been even an RLE format which incorporates a mask for the image would be handy, for instance:

I always assume 32 bit images (ARGB), but palettizing them would work to help with compression (if desired/needed).  So, when analyzing the image to convert to this format, it would FIRST look to the alpha and make a mask 'run' of all contiguous pixels with either alpha = 0 (fully transparent, the RGB part makes no difference in rendering - basically 0 alpha means 'Do Not Render'), and all other alphas.

So, an image that looks like this:
OOOOAAAOOOO
OOOABBBAOOO
OOABOBOBAOO
OOOABBBAOOO
OOOOAAAOOOO
with the 'O' pixels being clear (zero alpha), and the A or B pixels being any other color or alpha (in this case, I colorized them different red & green to help see)

pure RLE format:
4@3A4@
3@A3BA3@
2@AB@B@BA2@
3@A3BA3@
4@3A4@

Alpha-RLE format (RLE just the fully transparent/clear pixels, the rest are raw):
4@AAA4@
3@ABBBA3@
2@AB@B@BA2@
3@ABBBA3@
4@AAA4@

I am using the '@' symbol to designate 'transparent' pixel(s) or otherwise, skip plotting these

in this way, the image render routine can skip the transparent pixels, and then use the fastest method (I assume something like memcpy) to draw the plottable pixels.

I don't know if this would actually speed up rendering of sprites/images with transparency, it might, but I really don't know what the low-level routine looks like for rendering images is so it kinda depends on that (I assume Kevin has extensive knowledge of the actual low-level rendering routines used).  Having the images stored as RLE (actually, Alpha-RLE) internally would help save on memory usage (not disk size though, as PNG or others have much better compression than straight RLE).

Of course, this type of format would probably make other types of image processing slower.... but for basic sprite rendering/animation it might be a bit faster - kind of expands on your suggestions for reducing the plotting of transparent/dead pixels, but on a pixel-by-pixel basis, not just rectangular as you suggest on this thread: http://www.underwaredesign.com/forums/index.php?topic=2548.0

Update:
Was just thinking about this, in conjunction with my other experiments with Video Images, and it hit me that any sort of 'special' rendering routine like this would only work for FX images... the speed up with using Video Images is because of the on-board video card blitter, which is NOT any other routine and cannot be modified/replaced/etc.... so sad...

[kevin]

   PB's image format isn't solely about faster loading, it's about better compression yields across sets of images.   

   There's a provisional RLE mode tucked away in the rendering back end, but it's really just double up as if all you're doing is drawing an static image than converting to a font (CRF) or map is just as good.