Tuesday, May 17, 2016

C++ riddle

This time small C++ riddle which took me some time to figure out :)

My editor currently use sockets connection to pass data between Qt UI and "Editor module" in engine. Inside editor module this happen on two threads:
  1. Processing events on editor module side
  2. Receive data and send result of events back to UI. 
Code of two main function execute on them can found bellow:

This worked fine except my editor were crashing sometimes. This happen on random operations : moving, selecting, changing properties. Visual Studio debugger where showing me that crash were happening in this code:
    eventResult->next = m_eventsToFinalize;
while (wrAtomic::compareexchange(&m_eventsToFinalize, eventResult, eventResult->next) == eventResult->next);

Data in eventResult was looking like corrupted event or already deleted class.

What is wrong with this code :D ?

EDIT: There was small mistake in original version of code which should not be there (I fixed this in post). Line :
while (!wrAtomic::compareexchange(&m_eventsToFinalize, eventResult, eventResult->next) == eventResult->next);
should be:
while (wrAtomic::compareexchange(&m_eventsToFinalize, eventResult, eventResult->next) == eventResult->next);

Wednesday, May 11, 2016

My little monster

This story started long ago while I was still student at University of Technology in Wrocław. It was probably year 2006 but I'm not sure right now. With time passing by all previous years slowly starting blending together but well I'm already at this age where this is pardonable.

On one of C++ courses we got exercise to do program of our choice to show that we learned language. I decided that isometric game in SDL software mode will be great idea. And so I started working on it. What a great joy I had when everything finally started working together and I was able to send it to person conducting classes. I got passing marks and this was end of story. 

Well at least for a lot of project this would be truth but in this cases there is a lot more. Some time passed and with friend we decided to make casual game. Code from course project was really useful but software rendering were too slow so I switched whole stuff on OpenGL and some time later we finished "Pirate Crystal's: Adventure of Julia".

Another semester started and in advance C++ course I decided to connect physics to engine which at this point had already some basic 3D functionality. This wasn't easy because this was my first encounter with proper physics implementation. But well all of that couldn't prepare me for what came on other course conducted by Kornel Kisielewicz about shaders. I still remember his comment when he was rejecting my idea of doing normal mapping: but this is standard now. 

This may sound silly but for me this meant a lot. I was hiding in save zone avoiding shaders however I could but then I was informed that I'm far behind what happening in industry. This was crucial moment for me because I quickly get out of my save bubble, learned shaders and decided that I never want to stop learning new stuff and ending in this bubble again.

But what the hell this post is about? This is for sure not post about me but "My little monster". I sometimes hate it, sometimes need break from it sometimes I messing with it but in the end this is my "little monster" which I put a lot effort into and try to make it the best engine I can.

Why Monster ? 

People who done work on bigger projects probably know this feeling of being lost in code. In my case it often happen when I move to new part of code which I never had occasion to work on. Now imagine the same situation happening in your own code base. This is moment when you start understanding how big project you working on really is.

Other thing is that a lot of people seeing personal projects as "toys" they don't treat it seriously when you talk that you work on them. In my case I don't see difference between my personal project and projects I worked on in work. Of course code base is smaller but if you look on how much code is handled by programmer then I'm sure that I handle a lot more in my personal project.

There are also issues which you need to deal with yourselves:
  • Code paths that aren't used quickly became broken. 
  • There is issue of tools that you don't compile when I do some changes in engine.
  • Differences between GPU vendors.
  • Memory management,
  • Threading issue like: timing, race conditions.
  • Multi platform development.
  • Code which you don't even know how worked in first place.
  • Build system.
  • Building of game package (ideal if it was automatic).
  • And a lot more.
I personally don't try hack any of this or just do quick prototype. I try to make proper solution which I would also create if I needed to resolve this issue in work. And this change really a lot. 

This was really visible when I were changing work. The first time when I was doing it and mentioned my project people were like good you do something on your own but what you done in previous job. Things changed last time. When I was showing it to people that interview me they sometimes even skipped some tests and we ended talking how I done stuff or they give me their opinion how they would do something. This project was seen as one of projects that I was working on commercially and I think this was great.

Why Little Monster ? 

Well this is not name of engine because its called White Rabbit Engine. This is name that I using when I end in bed after doing some work for engine and finally realizing what kind of technology I creating. I'm scared of it complexity because sometimes I got lost in my own code. What will future bring I don't know but this project carring ten years of development, all my knowledge and well it still my "little monster". I may sometimes have enough of but but I still like what I do and will carry work on it for a little bit more :)

Friday, April 29, 2016

My bookshelves.

I realize recently that I'm weird programmer. If you would look on my bookshelves you will find there:
  • manga, 
  • fantasy books,
  • movies art books,
  • games art books,
  • books about drawing, 
  • books about anatomy,
  • books about animations,
  • D&D manuals.
What you won't find there is any programming book. My only book kind of in this topic is Showstopper about development of windows NT on my kindle. Which is in the fact not programming book but more story of people who created it and I'm still in middle of reading it :) 

The other book I reading is  "The Animator's Survival Kit: A Manual of Methods, Principles and Formulas for Classical, Computer, Games, Stop Motion and Internet Animators." which is for me really interesting. So far I finding there a lot of interesting advice which some even apply to programming.

And this is weird because as programmer I probably should read some books in my field of work. But I don't fell any push into this direction. I don't wan't to read book about programming language because what for? I tried long time ago and they are boring like hell :/ If I need to learn syntax I can just look into online tutorials.

All new stuff about C++ I learning when I searching online for solutions of problems or discuss with others. Sometimes I spend more time on trying understand small piece of example code than on fixing the initial issue. This happening mostly because I go from one page to another and finding more and more interesting stuff.

I would really love to read programming book which is not about language or algorithms but more about programming itself. I imagine such a book would contain stories of development, tips and tricks how to be better programmer. You know this kind of book which you start reading and before you noticing its ends.

Do you know book like that ? If you know I would really like to know its title.


Sunday, April 24, 2016

Plans, real life and monkey job.

Funny thing about planning is that sometimes its not working out the way you want. This was case this week I planned to work on animations and ended doing small upgrade for resource system. The best thing is that I don't even feel bad about stuff ending this way :D 

How I ended doing that is other story. Because I have policy of moving everything what I can to resource system which I have (this make stuff more consistent) I moved my rendering script to it. Everything would be good if not the fact that render script is initialized on main thread and my resource system never was prepared for request from other thread than game thread. So it was time to do improvement to allow for this :D

This was even more important because there were sometimes crashes at startup of game. I split my task on 3 sub task:

  1. Access to resource data require locking of resource (i.e. meshRes->lock()->getMaterial();)
  2. Separation of resource from data so you could have resource handle and it was still loading 
  3. Moving prepared stuff on multithreading.
First two task were robust :/ required iterating over all my resource types and changing it's interface. This was boring like hell but well programming is not only about interesting stuff.

My mind was blank when I was doing this so I had time to think and came to conclusion:
Some people saying that programming is about creating stuff. But reality is different. We as programmer spend most of our time improving existing stuff, changing them, fixing and redesigning. When we do all of this we make small piece of code which do something new and we once again starting whole thing again. This is also true in personal projects.
Few hour later changes were ready and I could start preparing functions which worked on one thread but were already separated in a way part of it could be moved to other thread. This was more interesting task which resulted in changes for some of resources. Generally my resource system look like this:

class ResourceXYZ
     ResourceId  m_id;
     DataType*  m_privateData;

Till now private data was part which control whole resource but now in some cases I want to hold some data in resource so even when private data change it will still be the same. I.e. LevelID (Yes level is also resource).

Another hour later I was ready. I know that my game worked because I were changes step by step with running game to be sure it at least work. Next step was mutli-threaded code which went smoothly.

Thanks to lock system I would add simple wait for loading of resource if it was still loading. There were small issue with dependencies of compiling GPU programs (they expected other resources to be already loaded) but well thanks to my split on soft and hard references fixing it was also easy.

And so my code working now and my previous 700 ms loading on debug configuration resulted in 600 ms loading now :) There are still few things I want to make:

  • File asynchronous loading with forward loading for better performance.
  • Reloading of resources
  • Cleanup iteration over whole system
But well you cannot have everything in life but some day I will make this all work :) For now I will return to animation topic to push everything forward :)


Thursday, April 14, 2016

After break

I took some break from writing on this blog but I had busy last few months. In this time I done a lot of changes in engine one of biggest "finished" one are changes in file system. But this time I don't to talk about what I done but more about what will come and what I working on right now.

And I do some stuff with which I'm not feel most comfortable: Animation system. So I decided to seek some advice and explain what I do, why and how I would like to improve it. So be free to comment and critique.

I don't fell that system I have right now is what I really want to have.

This is this weird felling when you fell that system that you created is working but you fell that it's limiting you. Because of this felling recently more often I turn of computer and sit down with pen and notebook and think what I really want to do with this system. So far my conclusion is because I'm iterative develop White Rabbit Engine technology I didn't notice till now that what was I searching for was different thing from what I have now. 

What I have now is "decent" system for playing animation using blends trees. And what I was searching for are behavior trees. I already have some idea how to make this system part of existing code. This will require some changes because I want blend trees to be just input nodes for behaviors nodes. Everything so whole system was consistent on all levels.

New movement system
Right now there is simple system which apply on character: orientation from pad/keyboard relative to camera direction and then play walk/run animation. This work but don't look nice when you try to change direction.

Because of that currently I started experimenting with simple system which based on direction vector try to choose the animation which will allow to get root bone position to destiny point. How it will work I don't know this is more R&D. So far I'm still not convinced by result. I will also probably postpone this "task" to first do proper behavior tree which will allow me on easier experimenting.

I am doing animations so if it possible I want to optimize whole workflow as much as I can.

This is for more simpler problem than it may sound. Currently I can create "Animation Database" (AnimDB) resource which contain link to animations resource and blends trees inside. 

Sound good but it's not because I cannot reuse the same blend tree between different AnimDB. This was one of this problem I didn't thought so much when I was doing initial system and if I decide to do different animations for each race in Little Big Adventure: Remake it would require from me supporting multiple copies of the same tree which would be really inefficient. So I need to fix :)

Animations in White Rabbit Engine are used to locomotion of character.

Right now this happening by calculating root bone relative movement and applying this to kinematic physics in each frame. This works nice but you need to be really careful how you do animations because slight foot movement may result in sliding effects of foots.

What I would like to experiment with is possibility to lock one of bones in world space and retrieving root offset from this.This would look like we have animation with marked when left or right foot touch ground and when it do it I lock position of marked bone in world space (b.w.s.) and store local animation position (b.l.a.p) of it. Bone is locked as long as distance between stored (b.l.a.p) and current (b.l.a.p) won't be bigger then some delta. I would use this lock to apply corrections to root offset so even if foot slightly move in animation in engine it would still look good


As you can see I think about a lot of stuff and try to improve a lot of different issues. Right now we redoing assets for Little Big Adventure: Remake. Thanks to that we don't need to put extra effort into changing content. But once I will define whole workflow there will be only place for small improvement. 

There will be no second chance so it would be nice to do it right so I'm going back to my pen and notebook to figure out in my head how I want this whole stuff to work. When whole idea will be in my head coding should be the easiest part.

Wednesday, December 16, 2015

Postmortem - 7 hours of hell

If you remember my 7 hours of hell post  you probably wonder why postmortem.

To give you introduction to this you need to know that I recently done port to Visual Studio 2013 (I used vs2010 till now). This migration was planned for some time already. I even started it in the past. But never finished because rebuilding of external libraries was pain in the a.. . Now I fixed this issue by automated building script and without bigger problems I was able to compile everything with Microsoft compiler ver: 12.0.

If this was good idea I still don't know but for sure I know that I still fixing bugs and issues that shown in meantime. I don't mind it so much because some of stuff I do right now were also in plan for future.

Probably you are curious what my one button plugin postmortem have to do with this changes ? Answer is simple I miss my one button plugin in Visual Studio 2013 ;(

I tried to use plugin without any modification but it didn't work and for now I work without it. I need to say you really can get used to this small improvement in workflow and after time I almost sure that I will bring this feature back into vs2013. Even if this will cost me another 7 hours of work :D


Sunday, November 22, 2015

Generated code

When two months ago I created generator for RTTI code I didn't realize how this one script will change so much in my programming style.

Right now I have already four of generators in my code base:
  1. RTTI - Generate part of my RTTI code.
  2. SID - Generate CStringID with already calculated hash
  3. Pimpl - Generate interface class.
  4. Enum - Generate toStringID, fromStringID functions.
So what changed for me after writing this one generator that make difference ? I try not to do monkey job. To show you how this improve my workflow let's look on generator I wrote today: Generator of Enumerator functions: toStringID, fromStringID.

This is simple problem where for some enumerators we want to have conversion from value to some verbose type. Some people may claim that this is not issue. You write functions once and they work. Later you just need to modify them when you do some changes in enum. This take like about half minute work 

... if we don't do mistake there. Then this is one extra compilation time. In some code base this can be counted in minutes. I would really love to see how much time of our programmers life we spending of stuff like that :) 

In my case I just have one macro: WR_ENUMGEN(EnumType, InvalidValue). Which look like this:
#   define WR_ENUMGEN(E, D)    \
        const CStringID&        toStringID(E a_value);                   \
        E                       fromStringID(const CStringIDa_value);
And is used like this:
    namespace EConstantsBuffers
        enum Enum
        typedef Enum Type;
The rest is generated :] So on top of this one line I just need to generate projects and that's all. No extra programmer steps to do. Thanks to that:

  • Adding/modifying of enumerators is simple
  • Generate code is easy to debug (what may be tricky with some macros).
  • Not slowing down my compilation. 
  • There is almost no place for any mistake  (always in sync with enum) .
  • There is unified code convention between enums.
  • Expanding of functionality need only modification of macro and/or generator.
So I don't know about you but personally with time I will probably use more and more generated code. I don't have a lot of spare time for monkey job I prefers to leave it for computers.


Thursday, November 12, 2015

Tools Design

Variable Set Editor - Main view
Continuation of previous post :] this time with real examples :D

Real tools design 

This may look like joke but this one window took me ~3 days to do. This include all it's features and communication:

Editor <-> Engine.

But this is reality. Creating good tools require time and I wasn't joking in Summary from previous post:
"As you can see creating of good tools is not easy. You often need go back and forth to create something really useful. You don't want to modify everything by scripts in which you can make easily mistake. Good tool will for sure save you a lot of work and return effort that you spend on it." 

Tuesday, November 10, 2015

Few words about tools design

Once again I will return to topic of tools. Mostly because right now this is the part of code I spend most of my time in. 

I will focus in this post on one dialog: Variables Set Editor This one dialog may sound simple but if you start thinking more about it it's really complex tool. 

So let's start with short description what Variables Sets are then lets move what we want achieve and in the end look on real implementation of this problem.

Variables Set

This is really simple system which allow you to specify some kind of properties. Right now I support only values of types: 

  • StringID
  • Float
  • Vec4.
Whole system may be use to store values like: life, attacks parameters, movement speed etc. In my case to this data have also access animation system which may request some kind of information which later may be use to control blends and generally its behavior.

Because this is simple list of variables you want to be able to have some kind of hierarchy. i.e:
NPC object contain [local set] which inherit values from [Clones set] which inherit values from [enemies set]. Change in inherited variables will be apply to all not override values. 

Dialog requirement

Short: You should be edit this stuff in nice UI.


Firs approach
Thanks to Qt designer this was really easy. As you can see on right everything look really nice you have section for name, you see right away type of variable are able to modify value and see what value you inherit :] If you want to add variable you using context menu (under Right Mouse Button). 

Of course this wouldn't be so easy if I ended here:P I forgot my dependencies list so I done small upgrade and finished :] Final result bellow:

Final dialog


As you can see creating of good tools is not easy. You often need go back and forth to create something really useful. You don't want to modify everything by scripts in which you can make easily mistake. Good tool will for sure save you a lot of work and return effort that you spend on it. And the tool you seen on screenshots are for sure not good one.

Saturday, October 24, 2015

7 Hours of hell

This will be story, my story based on true events.

Hour 0

Recently I start using project generator not only to generate makes, projects and solutions but also to generate some code. Because of that from time to time I need to run one python script. This is not big problem but I decided to make my life easier and add button in Visual Studio 2010 (I'm working slowly on progression to new one) which will trigger this task. Sound simple right ?