Build a basic 3D Render Engine

[stevmjon]

well, i got the shadows & reflections to work now.
it is faster than the post above too.

above post = 192 seconds
this post = 114 seconds

i do have a few idea's to get it to render faster.
one idea is instead of testing every object per ray with the shadows/reflection routines (this causes slow down), i can use the 3D transformation stages to calc the scene like the ray (shadow/reflection) is the look direction of a camera then calc min/max values to test which object spans across the screen center? if yes then test this object only. this should save a lot of calculations.

more to do it seems, stevmjon

[kevin]

Was going to suggest this earlier but it'd be ncie to be able to set the render resolution.   so 1/2, 1/4, 1/8 resolution.     If the res is set lower the render still goes back and fills in successive levels of detail, just at those lower resolutions the initial pixel takes up like 2by2 , 4by4 or even 8by8  so the user can get a sense of the scene quicker and exit out if something needs to be tweaked
 

[stevmjon]

oh good idea kev.

i can set up different render modes like you suggest. i will add another drop down menu selection with these settings.
i could even allow to set a different camera screen size too, that is not locked to your current window size.

[stevmjon]

kev, i just added a tweaked version of what you suggested with the preview render resolution.

instead of drawing the screen in stages as you suggested, i decided to draw a smaller preview screen size with all the detail in it. still draws so much faster too.
100% = 114 seconds
50% = 28 seconds
25% = 7 seconds

great idea for this suggestion kev. now i don't have to sit thought slow render times, lol.

i will post a new demo very soon too. i now have all the routines in for boxes, and i am now just setting up a nice default scene to view. i wanna get a bit creative with the boxes.

   stevmjon

[kevin]

  Cool!!!  

 Another thing  I was thinking about was possibly trying a dithering style patterns for the filled preview modes too.   So the user can get a sense of it while something is moving without drawing the while thing every refresh

[stevmjon]

good idea kev.

i can add a new preview mode that is 'solid mode' where all the objects are replaced with a PB shape.
this will stop the delay that the shaded modes have each time when movement stops.

i will delay the next release a bit more to implement these new ideas. if you have any more let me know and i will add them.

i have also added a new array tool too. i will post a pic when i get the panel for it done.

   stevmjon

[stevmjon]

well, time for another update to v0.54 March 2024.

this update was delayed because i wanted to add more tools & features requested by Kev.
i also fixed several bugs i found too.

Features :
> 'array' tool : this lets you clone an object in a grid pattern (good for walls, decking etc).
> 'clone' tool : this lets you clone an object in a given direction (good for stairs and above also).
> 'save / load' files : all the shape types are now supported, and the name of a loaded file appears in the screen title (top).
> 'panels' : a panel pops up with settings for different tools. it can be moved around the screen by click & hold the title. it also remembers the last settings used.
> 'panel values' : made sure that small movements of the mouse always give at least 1 integar value (it used to give zero)
> 'solid mode' : this is a new draw mode for the objects. the objects are converted to polygons and drawn solid. the draw order is artists render method so is far - near order.
  * NOTE: the advantage is the objects are drawn solid and are drawn fast. some polygons may 'pop' in front/behind others because i am using artist render method.
> 'more shapes' : there is ball , cylinder & box shapes(new). all shapes can be rotated & scaled.
> 'render modes' : there is multiple smaller render sizes for a quicker view. 100% , 50% , 25% .

i have also included a save file with this demo with the walkway object i made with this program.

   enjoy, stevmjon

[kevin]

  Brilliant mate !

[stevmjon]

thanks kev. glad you like it.

i didn't originally intend for the program to get this big. been workin on it for about a year now.
not sure if i should add more or start a new project?

i was thinking of re-writing this with a better set up, as i didn't know what data needed to be shared.
a lot is copy/paste with an edit. it is getting harder to add to the code, as i need to adjust it in many area's, so a re-write would make it smaller and easier to grow.

i was thinking of a 3D editor with all polygons, so you can make any shaped object you like?
there is already free stuff that does this, but i am doing this for the challenge and the fun of it.

anyway, if there is any bugs let me know, or any new idea's i can add.

   stevmjon

[kevin]

Steve,

didn't originally intend for the program to get this big. been workin on it for about a year now.not sure if i should add more or start a new project?

   Dude, I wish i had some useful advice,  I really don't know what to tell you ! 

i was thinking of re-writing this with a better set up, as i didn't know what data needed to be shared.
a lot is copy/paste with an edit. it is getting harder to add to the code, as i need to adjust it in many area's, so a re-write would make it smaller and easier to grow.

   Yes sadly this such a problem as programs get bigger... Once the surface area of them grows adding new features / changes existing features become more laborious..  I suffer from this situation a lot these days, trying to change anything in a big code base is tedious.   

    If you want I can pick through and try and give some assistance.  Which really is making libraries out of the various underlying elements.   Which allows for the functionality to be shared/exported between projects easier, but as the surface area of the interfaces get bigger,  around we go again! :)   

i was thinking of a 3D editor with all polygons, so you can make any shaped object you like?
there is already free stuff that does this, but i am doing this for the challenge and the fun of it.

    Anything in that space would be cool..  Ideally something that a user can import and just write their code on top of.   So the editor has a set of libraries a user can import and use to load/show objects in their code. 
 
    I dunno...  but something that helps us move in that next direction is needed..  but as you know building big things a hard !

    Enjoy ! :)

[stevmjon]

just wondering, i know PB uses legacy direct X (direct draw), but is it possible to add an openGL 3D pipeline, so the user can use parallel computing for faster rendering?

the main slowdown is i render brute force (per pixel).

i know legacy direct x had 3D games, so even if the parallel component was just x8 or x16, that would certainly help. did PB FX have any of this? i did see it supported 3D when i read the info about it in the development stage, but i didn't get into it at the time because i wasn't sure how to use it (i didn't see any getting started tutorials). it wasn't beginner friendly i mean. but with the knowledge i have in 3D now, i could certainly take a look at it if that is recommended.

   stevmjon

[kevin]

i know PB uses legacy direct X (direct draw), but is it possible to add an openGL 3D pipeline, so the user can use parallel computing for faster rendering?

 Sorta, as it's not a matter of few late nights translating dd calls for gl calls, which people seem to think.   There's various projects around the forums already ( PB GL library (login required) * PBFX 2D CORE)


 Even if you swapped one for the other; it wouldn't benefit the way you think.  Take a look at this loop. 

 i.e.

  For poly=0 to Number_Of_Polygons

      // read the vertex data

      //  draw this thing
      tri x1,y1,z1,x2,y2,z2,x3,y3,z3 
   
  next 

 
   Imagine the rendering request is be passed off to the GPU to be drawn off the main thread.   So the first time the loop runs the gpu is sitting there doing nothing while we grab some vertex data for the polygon.  When we hit the tri command the call passed this job to the gpu and the loop continues. 


   But what happens on the second loop and it gets to draw the tri ?  It has wait until the last one is drawn before the next one can be processed.. This turns out to be really inefficient


   The point i'm making is these api's effectively want all the data managed on their site of the conversation and as such, a draw request will draw batches of polygons all in one call..


 

PB FX have any of this?

    Yep..way back in 2007... 

   

    it's mostly identical on the surface ( meaning the idea was to have most things run out of the box) , but things like the newer camera modes you'd have to dig through whatever examples still exist for it.. and there's various differences. none of which can recall   

[stevmjon]

i actually did use the G2D - 2D OpenGL library a few years back.
i converted the first 3D engine i made to use this. it worked, but the framerate was almost the same speed. just a few fps faster.

when i tried this on my newer computer it farted. the program would not even run.
my old graphics card was a GTX 970 (above worked) and the newer graphics card is RTX 3070 (not happy Jan).
my new card won't even display 16 bit or 24 bit fullscreen modes. 32 bit only.

since there are the different camera modes, i could try again using them. maybe they would work?

[kevin]

i actually did use the G2D - 2D OpenGL library a few years back.
i converted the first 3D engine i made to use this. it worked, but the framerate was almost the same speed. just a few fps faster.

  Yeah.. can't say I'm terribly surprised as swapping calls just shifts burden from one thing to another.  If the total routine takes 20ms say to do a thing and we switch out a call to something that's only costing a two milliseconds of that total, we're not going to win those 18ms back. 

  If we swapped from  dots -> strips -> triangles..  Obviously drawing tri's removes the burden from the cpu side (the cost per pixel plumets), but a loop like the one above isn't ideal for concurrency.  What I mean is a loop of draw triangle calls isn't just building up and being done by the driver in a queue later in the background; rather we're only winning the overlapping render time of the last call..

  Ie.

  TriC x1,y1,z1 ,x2,y2,z2,x3,y3,z3, colour 
  TriC x1,y1,z1 ,x2,y2,z2,x3,y3,z3, colour

  For lp =1 to 100000
      a=b
  next
  end

  So to draw this we setup the call and call the driver for each TRI.  But this creates two stalls (waits)  we loss time setting it up, and once we call TRI we're not going gaining anything as the next call requires the gpu immediately after.  So we wait fro it finish and setup it's state and call the driver again.. 

  The second call gives up a benefit, as the call returns immediately back to the primary thread, but the rendering is taking place whenever the gpu can do it. So provided we don't do any graphics stuff we end up with an asynchronous result.  So it's rendering while the cpu is doing the for/next thing.   

  So there's this balancing point between the two sides..

when i tried this on my newer computer it farted. the program would not even run.
my old graphics card was a GTX 970 (above worked) and the newer graphics card is RTX 3070 (not happy Jan).
my new card won't even display 16 bit or 24 bit fullscreen modes. 32 bit only.

  aren't drivers wonderful ! :)   

  I've not run into a system where they don't work (given it's GL1) but that doesn't mean much..
 

since there are the different camera modes, i could try again using them. maybe they would work?

  maybe, but given there still 2D..  There's no z buffer / perspective in that stuff.  Those old PBFX demos have a real 3D mode in them (z buffered) it's a matter of worker out how to create meshes through the sprites..  Don't ask me.. I don't remember !
 

[stevmjon]

PB is faster then python for brute force calculations (see pic).
brute force means lots of calculations by the cpu (the main computer chip).

for fun i copy/pasted my code into python (and edit to suit language difference) to compare the speed between both languages with the same code set up (same scene with same camera angle).

PB = 11 seconds
python = 26 seconds

even though PB is using legacy direct x, the brute force calculations are still good.