Resource building

-
As you may notice I like to push my tech to it's limit. Because of that I recently decided to switch to a lot better way of building resources. I took my app that were containing everything and split it on 3 different one.
They run as separate processes which connect with each other using TCP/IP. Comparing to previous setup this solution have a lot of advantages and this is some of them:
  • If one node crash I can just restart it and run whole setup further.
  • Whole node setup is scalable so I can have 100 of nodes.
  • I can run nodes on different machines. 
  • I can dispatch new version of building nodes to different machines automatically.
  • I can build specialized Python nodes for some of work.
  • Application don't need to have all this shitty code of resource compilation.
  • Whole concept became a lot easier to control.
So as you can see there is a lot of improvement :) comparing to previous one.

And here to my mind coming my recent discussion with friend:
By telling only truth you can easily manipulate people.
What you read above is not lie. But it's also not fully representing reality. And about this missing part this post will be.

When I decided to switch to this system everything looked exactly like you read above there is so much possibilities and cool features. In reality there came problems which still following me till this day.

If one node crash I can just restart it and run whole setup further.
Well yes I can but there is question what we want to do with resource that crashed it ? Crash may be unrelated to resource itself: i.e. memory leak, some memory stomping in other compilation code (yes they happening in all code bases and it's f....g hard to find them). On other hand we may try to restart it and run compilation of it again. But you need to do it only on this new node not any other because we would be back to begin.

Whole node setup is scalable so I can have 100 of nodes.
Dreams :] 100 processes running on one machine would kill performance of all nodes. Especially when run on slower machines (like my laptop). There is also issue of dispatching work to all this nodes which is big problem on it's own. I personally right now doing some trivial dependencies building just to use 4 nodes I have now and most of the time I use 1 but well this is still work in progress. Proper resolving of this will be some predictions and scaling amount of internal nodes depending on amount of work.
 
I can run nodes on different machines. 
Yes I can but if I should :) Well I still need to do here some testing to see how it will work. Most of resource building is really simple and probably just communication with different machine by TCP/IP may introduce some delay which probably will take some time. I thought about getting 5-10 Orage PI machines which I would use as cluster. I want to do it for fun to see if I will be able to handle nice scalable amount of machines. And there are also some challenges here f.ex:
  • Collecting of statistics for future improvement and balancing of system.
  • Handling crashes of building nodes and reporting issue.
  • Reconnecting node after loosing network. 
  • Handling power fails (they may corrupt some data).
  • Caching of some files for quicker execution.
  • Handling running out of space.
  • Approximations where to send work (sometimes it may be more optimal to just build data locally than do it remotely). 
And this is just the one that came to my mind right now.
 
I can dispatch new version of building nodes to different machines automatically.
Yes I can :) but I would need to build infrastructure to restart new processes and handle few other stuff and I really don't have too much time for this (there are so many other things to do :/).    
 
I can build specialized Python nodes for some of work.
I still considering this because Idea is cool but maintenance of it would make me staying whole nights. So for now I saying "no" to this idea :/
 
Application don't need to have all this shitty code of resource compilation.
This is good :) but then to separate application to build resources I need sometimes to duplicate code. I want building nodes be as much separated from higher level engine features and code as I can.
 
Whole concept became a lot easier to control. 
Concept became more clear and easier to control but execution just got a lot more complicated and introduced all this issues and problems that I didn't had previously. But well it's our job as programmers to resolve problems (just I have too much of them in my code base right now :( )
 
So yeah. Cool system that you read at begin is still there but it sound probably a lot more messy right now. It also coming with its own set of problems which weren't there before. I don't need to hide them because why? Sadly some of articles/papers skipping this most funny part of our work: 
  • What went wrong ?
  • What may go wrong ?
  • Where whole system failing ?
My release code finished building (on my laptop it's taking so long to rebuild project :( ) so I can return to coding. See you next time.

Greg

Comments

Post a Comment

Popular posts from this blog

Query commands execution

-
So like always start with problem description: I have some pool of command represented as enumerator. Each of command can have unique data that size can be different. I wanted to create system that allow me in easy way iterate over them and execute.  After some time I created this implementation: template<ECommands::enum cmd> bool execCommandTemp(ECommands::ENUM a_cmd, void* a_data) { if (a_cmd == cmd) { SCommand<cmd>::execute((SCommand ::SData*)a_data); return false; } return execCommandTemp<(ECommands::ENUM)(cmd+1)>(a_cmd, a_data); } template<> bool execCommandTemp<ECommands::WRAP>(ECommands::ENUM a_cmd, void* a_data) { return true; } bool execCommand( ECommands::ENUM a_cmd, void* a_data ) { return execCommandTemp<(ECommands::ENUM)0>(a_cmd, a_data); } where SCommand look in example such a way: template <> st...
Read More »

W.U. 0x20

-
Image
And as I said last time new preview is available: As you can see a lot has changed. Right now I mostly focus on gameplay so I do test of AI (some of them you can see on end of movie), changes in fight, items managing, fix some problem with physics and many other things. To say truth I don't remember them all :P So I'm going back to work and till next week.
Read More »

Few words about remake

-
Image
Time is something funny we don't notice when but it pass by so quickly. Few months ago I announced release of this pre-alpha build. Then it was so far away right now only a little bit more then month left. You probably wondering what changed in this time. So below you will find some summary of biggest changes (from my point of view): Component base architecture  Decision on big changes in engine architecture aren't easy. I think that transition from class base approach which looked like that : class CGameObject class CModelObject : public CGameObject class CPuppetObject : public CModelObject class CChcaracterObject: public CPuppetObject  to component base ( like in Unity 3D ) was one of the biggest and most problematic changes that I done in White Rabbit Engine in last years. I needed to change a lot of code, don't break the game that I have and in the same time change my approach to resolving problems.  Level Editor : New com...
Read More »