PlayBASIC Live: Understanding Video & FX/ AFX Image Types (2017-06-15)

[kevin]

PlayBASIC Live: Understanding Video & FX/ AFX Image Types (2017-06-15)


Today we'll be talking about the two primary Images Types you can use in PlayBASIC programs, which are Video Images and FX & AFX images.   Video images are stored in computers GPU and drawn using the GPU's 2D blitter,   FX & AFX images are stored in your computers main system memory and are manipulated with the systems CPU.

  Both image types have different uses, so it's important the new programmer understand which type they should choose to achieve best performance, making the wrong choices can really have a negative impact upon a programs performance .  

For More About PlayBASIC

https://playbasic.com
https://underwaredesign.com

Watch On YouTube

   Video Transcript

Code Select Expand



hello and welcome to a another very
quick a basic live video I just wanted
to talk about image types in particular
I'm not talking about compressed image
types like there's like JPEG and PNG if
happen those things even play basics on
an image format PVR but once we matter
internally what's happening when we have
or why do we have different types of
images and their purpose within our
programming and to get to the some idea
about this we really have to understand
classic I guess in the story images
could be broken into two different
classifications we have what's the video
image and an effects the effects image
or an eye FX which is an extension of
that now if you know something about how
computers actually a bill you probably
understand this a lot easier other
people now inside your system doesn't
matter what kind of process you have
what kind of mirror you have what kind
of GPU you have which is the video card
the CPU and motherboard have certain
characteristics now the CPU is local to
main system memory so your central
processing unit the main CPU in your
computer the thing that runs programs
has a local bus that can access whatever
whatever Ram you've got in your computer
now 50 is also had things called caches
lots of different types of caches in
fact that's also relevant
but not so much in this kind of
conversation now the CPU and memory are
local to each other so if you can fetch
data from from memory here easy and gets
sent back to CPU and all things are good
and that speed X that read speed as well
as such the writing speed and pretty
much the same or the sake of this
conversation watching both ends at the
same speed you can read and write thus
if you can grab data from these RAM
chips over here and send data back at
the same speed not really true of it but
halved if this is this example our video
hardware is not actually inside this one
but that's external so to get to the GPU
which is another processor in touch
internally with another big chunk of
memory inside inside there in particular
modern systems which have very
complicated architectures now on just in
the jet fuel load so when we have an
image that's a video image it gets
loaded into system memory here and then
gets sent across onto the GPU now the
idea of video images is that once on the
GPU side of the equation they can be
drawn to the screen which is also stored
over here and manipulated your nephew
lations are quite well simple we can
just copy the images around accountant
Li you can't rotate them you can't have
transparency effects and do those kind
of things with them because PB is set up
with what's called direct draw which is
really just a set of rendering features
that that do hardware-accelerated
block image transfers or pleating
because when TV was built which is a way
back in the early 2000s now the uptake
is of hardware accelerated today wasn't
ly thing it was kind of coming was kind
of coming into into vogue in early 2 X 9
but but TV uses a very old version of
director or three for compatibility so
the whole idea actually is not really to
make flashy high-end games the idea is
really to make to learn how to do this
but of course as well as in with things
moved along people wanted to do those
effects we had to implement some way of
doing okay so so video images end up
outside of this circle end up outside
over here on the GPU once they're there
we cannot fetch them at the same speed
because then the image data across to
the GPU at a very high rate of getting
it back is probably about 10 to 20 times
slower and now this is it really depends
upon the system some systems between the
modern systems can do this much better
than our resistance but because TV is
designed to support old computers where
this is come almost completely it's not
impossible to fetch data from from the
GPU side it's incredibly fire probably
20 or 30 times slower than reading
normal memory yeah so you can't even
read a 440 image every frame it takes
some offering I think one example work
where to run an 800 by 600 frame was
taking at least 10 frames to grab it
that's a lot of cycle if not same in a
memory this is because you your video
card is actually external device it yeah
the types of adaptive types of car you
know our bet an animal that used to be
and
systems we don't even have that
architecture at all we just have a CPU
and GPU pretty much on the motherboard
itself liking I'm actually using an
all-in-one computer here at the moment
to make this this little recording so
severe memory so video images live over
here they can be drawn easily wizard
around in other words they can be an
image can be stamped onto another image
which is what drawing does but you can't
liro Tate them or do alpha effects on
them because that requires the CPU to
read the data from the GPU modify it and
send it back which is impractical for
most systems it's kind of bent getting
better now but it probably will never be
all that fantastic on PCs these days
you're expected basically to move
everything across the GPU do everything
in shaders it's not another conversation
completely so okay video images are
loading - onto your video card and then
the idea is we never really try and
modify them we just let them go we just
draw them wherever we need them
but we're kind of limited to what kind
of things we can do with it the other
alternative is going back here is our
effects or or effects image types and
these are a version of the image they
store in through the memory where they
go the image lives over here so the
process can grab the data easily modify
and manipulate any way we like so what
we generally do if you want to have
rotated switches in particular is we
store them as an FX image we use the
processor to do the translation and then
send the data across the screen over
here as long as we're not reading from
the screw
that process is pretty quick now it
could be faster but it's generally
pretty quick that's not going to work if
you try and do things like alpha FX or
or anti Allison sort of people call it
that really exists alpha Channel which
is what the AF X version of these things
is trying to find some other stuff sort
in the help files by the way up here and
about there's actually some bits and
pieces so I think we're stuff this off
is gone are they down here in images you
get what image formats different
permutations these are actually
political load of this the load is
fairly it's a fairly old chunk of code
really probably will find a new version
in you later
when we fully move to the VFX
you may have notice different
permutations of these click load image
command it's kind of in a conversation
early on actually it's a bit off-topic
but we're asking people you know which
are rather optional parameters or which
are rather a command name to actually
set what they're doing and surprisingly
enough people wanted command names that
said what they do so yeah so it's kind
of got some double up with some of those
commands they'll only image can do the
same same job you've got an optional
format parameter at the end it defaults
to 1 which is the video image so that
image when it's loaded will end up being
transferred across to the GPU and stay
there they control it is going to GPU to
draw that image onto another image but
you couldn't
you couldn't actually draw the image
into memory back onto something that's
incision memory there has to be just
copied pixel by pixel back that was the
help files gone I lost it hello
ah sorry back then factory images
highlights permutation stuff yeah well
here we are saying that the built-in
loader is okay that formats like PNG for
example I think a lot of people think
that PNG is just one format but
internally lots of permutations of how
they are stored and we don't support
loading every permutation
it's just crazy famous JPEG we support
sort of the common case JPEG but they're
actually lots of other kind of newer
permutations that are probably won't
light
same goes for bitmaps by the way and
TCGA we don't support every every
version of those just just some common
ones if you want to load lots of
different formats you can use the free
image library which we supply anyway you
just include them of functionality PPI
is our own internal image format which
is would add recently all right
try to get back on course here so we've
got there's two different sort of
categories but images the low installed
video memory which refer to video images
and images that have managed and loaded
into justjust a memory and live over
here in system memory so if you has very
fast readwrite access to them you can
use L for effects in general and all
you're bound by really is how fast that
process that you have is and half of the
RAM is by the way to makes it
so generally the probably the best rule
I can give you I think is actually in
the bottom of this article down here
yeah
tell you somewhere okay so crash course
is prescribed wrong mode when we have we
try to this kind of sprite affects the
whole bunch of them in here
put some animated things in here about
what it what its effect is doing yeah
effects that manipulate any one of these
effects here that manipulates the pixel
of the image is going to perform poorly
when the image is stored in via memory
does the CPU assuming that doing that
modification it can't fetch the data
from that video card at the same speeds
that can over here so what we want to do
is we want to have you mean the image
has to be rotated then happen Espeon and
afx are an effects or a FX image simple
as that we're doing blending effects
we're taking the pixels in the source
image then blending them but what if
we're drawing over the same thing
applies if you take a sprite image here
like a chip image we're just looking out
so if you're trying to draw this image
here to the screen which is NVIDIA
memory and over here the way blending
works is that the to the source pixel is
drawn and a little algorithm is run to
blend the sauce pixel with the
destination pixel so to get the
destination pixel you have to read it
from the
ring pull it back to here do the
blending and then send it back generally
very slow painfully slow on on some
whole systems which is why we don't
recommend people do it
no look not you can't do those effects
you can model if use it very fast just
don't stop going to try and do it
directly to be doing real results are
going to be very painful and you be
going to bang your head girl why's it so
slow it's not P very slow if you
architecture the PC these days most of
the languages are based around that if
Kelly Harbour accelerator you have that
derivative clever use OpenGL which means
everything here is happening on the GPU
side all they use in an up to date
version of direct3d we don't we use a
director or for leaks you can add
abilities reasons the whole point at the
time with it when people side does shift
to using directories efforts to make 2d
games it's used a whole bunch of
problems that didn't exist before
like you had limits on how big if your
images could be because of GPUs are and
very small mouths and every for their
textures and old systems in particular
have really tight-knit GPU cases they
don't have much memory on them at all so
games that that you can do easily with
direct draws you couldn't do easily with
those systems sounds crazy I know these
days that the things have shifted a
lasat GPUs and much more powerful
solution but with Phoebe with we're
still using the classic solution so your
your game will run across everything but
it's not really picking up all hardware
acceleration that could
please ace if you want that you can use
that use something along the lines of
the G to D extension which is in here
somewhere
we are showing the video the other day
pop in here somewhere
as I'll confess a bit on the forums you
know all that kind of stuff so should be
that hard to find
all right got a bit bit sidetracked
getting back to my main point okay we've
got two different classifications and
pitches video images when you like whe
your load image here give it anything
you're loading some ship you mean just
number might put it in swap one it will
devolve we're going to tilt the format
to video so the low use command creates
a chunk of memory on the GPU so not
exactly how it happens but what actually
happens is the so far as loading the
memory the image is decoded and then
it's sent across the GPU and that's what
it says right now that's good for images
that you don't want to be manipulated
you can do pretty fast image copying
with those things it really just comes
down to your systems support of director
all windows 10 windows
I don't aren't particularly that good
with it they tend to sort of a pretty
weak emulation right if we need images
that we're going to rotate we're going
to do things like maybe alpha blaine
over the top of something we need
helpful channel those kind of things
we have to use effects or afx either one
so I fix effects is if you just want an
image that you don't have transparencies
in like alpha channel transparencies
Anthony just got my single mask color
that's a it's transparent so in which
just has to stay if you have opaque
pixels which is solid pixels and the
mask color is the color thing cut out
and removed or as a FX is really the
pixels really have three states you've
got opaque pixels translucent pixel or
you've got transparent pixels which is
just dis means that when I'll go through
the three different states so so in this
in an effects image or any image extra
effects or video what happens is that
the got a sauce pixel we've got the
final pixel here we'll say ah pixel and
what kind of checking as a sort of text
all it is you know oh what your very
crew to the basic so if sauce ticks all
it is not it's not a nice color then you
know right white pixel you know the only
thing is that it stands through the
sourcing edge it's looking for this mask
color if it's not mask color gets drawn
to the destination other people's is
just not not when we have an effect
situation looking at the alpha channel
that work out how this X was drawn if
the Alpha channel is 255
it just writes the sauce pixel right
sauce pixel over destination pixel if
alpha channel is anywhere between 1 and
254 we are blending bling so pixel and
destination capsule right result to the
destination ethyl or over destination
vessel and if a is 0 then we go to
anything just nor this pixel so for
those source pixel have no effect on the
destination the only time the only time
the destination is old is when the alpha
channel is either 255 so it's completely
opaque almost translucent yeah hopefully
but through some stuff there so to load
ship let's say with an alpha channel
we'd have you know ship top it that it
may be in the spot to but we've give it
a format this time a sex format is is 2
and a FX is 8 just double check that
here it is
so if we said steady or if you just go
do like this we could go load effects
image and you can can see straight away
we read the code 3 don't understand that
into the optional parameters there's
also another one to load a FX into image
3 so image 1 will be load as a um each
go back to a picture the day is lined at
memory its decoded a few cents across
the Jeep
you and that's where it lives so for an
effects image same process is kind of
happening where the image loading the
mirroring it's been decoded and then the
image is created and stored in this
memory here so it means that you want to
rotate this image at some point or scale
as those kind of things or do some sort
of manipulation of that image in the
future and if so much the final one
we've got the afx image here same thing
happens it gets load a photo flow theory
you just coded and the afx surface is
created into some every year so if you
want to draw this on to something else
you generally you deal we have an effect
surface in system memory that you're
drawing over so you do all the rendering
in effects images here in RAM and then
we once you're finished copy that image
down across to the GPU to display it Wow
I hope that given some sort of clarity
on on I know probably one of the most
complicated and possibly well actually
it's one of the most misunderstood
things about if a at all
attacker I don't think I've ever seen
anyone who really understands what's
going on which is a shame because
they're often you'll see people going on
why does this work so slightly
well you can't fetch that data I go I
load my image it won't rotate very fast
responds the game din so much and yeah
look at the code there blows the image
has a video image that which means that
if you try and rotate that all the
rotation effects in done by the
processor so it asks video carp isn't
the pixels manipulate them that's in the
back
and that sitting across to the GPU is
very fast that reading them off it is
incredibly fire just thinking like that
probably ten times 20 times on my art
miles Iran test system it's probably
about thirty five times slower than
writing to video memory which is why
we've done if I ever doing like that in
the future we'll probably just go you
know just go completely out in geology
you know we'll still have effects images
we can manipulate images are open system
memory with the main processor when
you're done with them you're sending
across as a texture and then - yeah
you're compositing stuff on that side
which is pretty much how most games
worry these days a modern sort of way of
doing it are those things don't work on
all computers yeah that's what we
haven't done that now you can with GTD
include mess around this is not pretty
interesting stuff but this comes into
what you want what you want to achieve
if you all you want to do is learn how
to code then yeah forget about what it
looks like just get the core mechanics
under your belt first because it doesn't
matter how good the rendering solution
is if you can't write it it your idea is
just an idea okay I hope that's covered
however I wanted to cover it and we all
have to leave it there can hear the dog
scratching around behind me so I just
want to thank thanks for watching this
bit ramble and hopefully you you pick
something happen and I'll catch you some
other time bye




 Related Links:

     - PlayBASIC LIVE - Intro to G2d (openGL for PlayBASIC)
     - Tutorial: Economizing Image Blitting (drawing!)
     - Tutorial: Crash course in Image & Sprites Draw Modes
     - Tutorial: Copying/Drawing Between Image Types

[stevmjon]

hey kev,

this video is an important one to help people with.

years ago, i fell into this trap too. i loaded an image into video memory(i didn't know about FX/AFX then) and blended the alpha layer with the background to get anti-aliased effect, and wondered why it was extremely slow.
now i use both video and fx/afx to draw the screen using multi-pass technique for my platformer game, without any frame rate drop.

keep the video's a rollin, these are a great addition to the help files.

  stevmjon