Tuesday, September 29, 2015

Namespaces hell

Welcome in namespace hell. In this place you will find annoying code and the best part: it is on demand. So lets start another topic base on one of my horror stories which ended with hours spend to straight things up.

But let's start from begin. This words probably don't mean a lot for you: s2d, s3d, world2d. world3d, fs, sound, physics, math, navmesh, utility, resources, game, editor, lba, wre, wr.

Personally I know them really well. All of this are namespaces that I used or using in my code. This is collection from 7 years of development and they are probably not all of them. When I start writing engine I wanted to do stuff right. So I created modules which then I encapsulate in namespaces. This was naive and stupid because I ended with annoying code. Who want to write physics::CPhysics for physics class? This is duplicated information. But well with time I got smarter and removed most of them using simple method : Find and replace all.

This was my first sin. After this operation I left with code where you could find stuff like this:

namesace wre
    class A : public wre::class B

The second sin is my mistake of using prefixes for basics types and functions which were outside of namespace. So I had wrVec3, wrQuat, wrMtx44 and functions with similar convention. Why I call it sin ? Because I ended with inconsistent code standard which is not something good and create really a lot of issue. For what you use prefix for what namespace?

So right now I'm decided to clean all of my past mistakes. There will be only one simple convention. Everything will be in one main namespace wr (stands for White Rabbit) and helpers wr::math. I will cleanup naming convention, remove prefixes and put everything related to engine in this namespace.

If somebody think that namespace is useless in he can always use : 

using namespace wr; 

to remove need of using it explicit. I think that this change will only result in simpler and easier to understand code. But well there are no sure things.

What do you think about namespaces ? You would recommend them or rather burn them using fire ?


Sunday, September 27, 2015

Commit comments

Now that I thought about this is interesting topic:

How your comments for commits look like?

In my case you probably think that they are poor because I do project myself. But here I will surprise you (and myself to) because most of the time I do full summary what I done in given change list. So bellow you will find few of my comments from different commits across last 3 years (I use not modified git logs) :

* 13/11/2012

* 25/02/2013

* 30/12/2013

* 05/09/2014

* 21/09/2015

I really like my current form because its simple and easy to understand. Especially with my tendency to submit multi issue change lists. I need to do something about this because this may save me a lot of troubles later. But sometimes you just starting one thing which require something else and before you notice you have overlapping changes. 

What is yours format of check-in comments. ?


Sunday, September 20, 2015

Animation System Changes: Part 1

After weeks of work (probably I could even say months) and hundred resolved tasks and bugs new animation system finally start working ;] Check final effect:

If you think that it is look no different from my old animations then you are almost right :D Currently this new system have even less functionality than old one. Because of that I needed to disable big part of gameplay.

Was it worth it ? 

Short version: I think that: YES. 

Saturday, September 19, 2015

Playing games and Dreams

Normally I don't play too much games but there are days when I just want to relax and play. Sometimes even a little too much for one day but well this have it's charm (till the next day when you need to wake up).

This week I finished playing PS3 Uncharted games and Zone of the Enders. I feel inspired by their style. I really liked Uncharted 2 graphics would it be cool if Little Big Adventure remake had this level of graphics :D Well I need to return to development I still need do few things to make dreams became true.

Tuesday, September 15, 2015

RTTI: Changes for better

As some of you may know White Rabbit Engine use it's own Run Time Type Information system. And well like with everything else sometimes come time when you need to upgrade stuff. Recently I had another occasion to do iteration over this system.

In this post I will try to give you a little bit insight how m RTTI look is engine, what changed and what are plans for future. But well let's start from begin.

Saturday, September 12, 2015

Weird thoughts

Let's start this from some thoughts about who I'm (yes I will be talking about myself).
I'm not seeing myself as somebody really intelligent I would say that I'm event think that I'm stupid. Funny thing is that with time I see that even then I'm more intelligent than a lot of people :|
Question is where this bring us and why I'm in such a position? I thought about this and my answer is: "learning". 

I like to learn new stuff, discover things that I didn't know about. This is for me something incredible and in current time we can do this really simple. Sadly because of that I see how small my knowledge is and how big the sea of possibilities is. There is always ways to growth. 

On other hands we have people who don't really want to learn. They do it only when they are forced by situation. They think that what they know is enough.

I don't know which way of thinking is good or right. The thing that I know is the fact: I don't want to stop learning. I'm where I'm because of things I done right and also all my mistakes. Thanks to all of them I'm different person today than I was yesterday and tomorrow I will be different person too.


Short Term Memory

Funny thing about doing your own tech is that you sometimes need to do some funny stuff. This week I needed done Short Term Memory Manager. My comment in code describe it like this:
Short Term Memory Manager (STM Manager). STM is memory which you want to use for short one-shot tasks:             Create->Execute->[GetResult]->Destroy It is limited by capacity so you never can use more than you pre-allocated for it. Right now it's use to transfer resource data CPU->GPU and in animations tasks.
For me this is temporary memory which I use for example to load texture data and then passing it to GPU upload (which will remove it). There are also other usage where I think using normal allocation would be a little bit overkill.

So my short term memory manager use my new memory ring buffer which look like this:



After creating this code the rest was easy:

    uint8* CShortTermMemoryMgr::cpuAlloc( uint32 a_Nbytes, uint32 a_alignment )
        // memory is HEADER + ALIGNMENT_MEMORY(ALIGNMENT-1) + HEADER_OFFSET(+1) + a_Nbytes
        uint32 memorySize = sizeof(SHeaderCPU) + a_alignment +a_Nbytes;

        uint8* memory = nullptr;

            CMutexAutoLock lock(m_cpuMutex);

            memory = (uint8*)m_cpuMemory.alloc(memorySize);

            if (!memory)
                if (cpuRecoverMemory(memorySize))
                    memory = (uint8*)m_cpuMemory.alloc(memorySize);
                    WR_ERROR(false, "There is not enough memory for this allocation.");
                    return nullptr;

        // Create memory header
        SHeaderCPU* header = (SHeaderCPU*)memory;
        header->isUse = true;
        header->size = a_Nbytes;

        // Fill offset info
        uint8* data = wrAlignAdress(memory + sizeof(SHeaderCPU)+1, a_alignment);
        data[-1] = static_cast<uint8>(data-memory);

        return data; 

    void CShortTermMemoryMgr::cpuFree( uint8* memory )
        if (memory)
            CMutexAutoLock lock(m_cpuMutex);
            SHeaderCPU* header = (SHeaderCPU*)(memory - memory[-1]);
            header->isUse = false;
    bool CShortTermMemoryMgr::cpuRecoverMemory( uint32 a_memoryNeed )
        const void* memory = nullptr;

        while(m_cpuMemory.getBiggestAllocSize() < a_memoryNeed && (memory = m_cpuMemory.peek()))
            const SHeaderCPU* header = (const SHeaderCPU*)memory;

            if (!header->isUse)

        return m_cpuMemory.getBiggestAllocSize() >= a_memoryNeed;
I added also two helper functions:
    template<class t="">
    T* CShortTermMemoryMgr::cpuAllocT( uint32 a_alignment )
        if (uint8* memory = cpuAlloc(sizeof(T), a_alignment))
            return wrNewExt(memory) T;

        return nullptr;

    template<class t="">
    void CShortTermMemoryMgr::cpuFreeT( T* a_memory )
        if (a_memory)

And this finalized my work :] Personally I really like how this came out. Its simple and pretty efficient but well I'm curious what you think about this? Maybe you have some betters idea how to resolve problems like this?


Sunday, September 6, 2015

Hunting of memory corruption

This is the story of memory corruption hunting and what I learned thanks to it.

So the story started when after one of many big changes in component system I needed to re-save all maps and objects templates :] Because I'm lazy bastard some time ago I created processes which loading each of it and re-save. All this after pressing one "magic" button. Great idea and in the same real stress test how object system works.

Sadly after all this time I didn't use it, whole process failed. And in worst possible way: memory corruption. Everybody who have occasion to deal with at leas one not trivial one know that this are not nice bug. Especially in multi-threaded environment:/ In my case normal corruption wasn't enough and sometimes even my callstack was corrupted. Fun :|

So well my hunt begin because I couldn't left this issue without fix.

1st approach: Debugger
The first approach was the most trivial one check what happening when app crashed. Simple and useful but not in this place. The debugger stop when it's already too late. I tried to find some pattern in memory which is corrupted and nothing there :/ I failed.

2nd approach: Enabling all my memory debugging options. 
Yeah in all this years I accumulate some memory debugging tools like: full tracing what memory is allocated, memory guards, validation if memory is not release second time and full allocation/releasing logging. Final result: another defeat :/ 

Non of this tool helped. I only found that my guard sometimes get replaced from "WREGUARD" to "WREGWARD" :/ and it's happening in TiXML node memory (I using XML right now for storing RAW version of map which is use in editor. But let's return to this topic in next approach).

3rd approach: Checks on memory release (Desperation begin somewhere here)
Microsoft _CrtCheckMemory(..) is so nice thing. But well after I done that I can say: I was stupid. XML and full memory check on memory release. Heeeee heeee heee hee he (laugh of crazy person). I started process and started play Uncharted 2 on PS3 and after more than hour later it was still releasing the first map. 

This as bad idea as idea to use XML for project:/ My To-Do task for replacing it with Jason gain higher priority after this.

4th approach: Decoding of callstack from stack memory.
Yep I was so desperate that I learn how to decode callstack from memory for win32 (In the fact it's really easy :D but well. Right now I know how to decode PowerPC, x86 callstack so only x64 left). 

After reading corrupted stack I learned: nothing more than I already know. Releasing of xml again :/

5th approach: Luck
And this one was in the fact funny because I noticed that very often memory which is released is not managed by my memory tools. So after short check I find out sad truth: Allocating memory in one module and releasing it in another is not the best idea :/

Another funny side story:
To be sure that I removed the issue I installed even Valgrind on my Linux computer but it was crashing in initialization of libGlew. First thought: well probably Valgrind do something that glew crash. Because I didn't want to spend too much time into it I just ignored. 

This was mistake. One simple check: run game without Valgrind would show me that the problem was in my changes that I done on Windows not tool itself. But well :/ Day later I figure out this and will soon run full check of editor using Valgrind (This is one of reason why its worth to have Linux build :) )

This one issue shown me so much problems with my code. After it I properly secured my code for releasing of memory from other module, guess what ? I found more of similar problems :/ 

But well right now everything what I found is fixed but this is not the end. I decided that after animations it will be good occasion to spend some time on improving my memory system: make it quicker, allow better control over it and add more tools for checking stuff (just in case). 

From things I will do I will for sure switch on explicit use of dlmalloc and switch on memory tags stack:
 MemStack memstack(EMemTag::System); int32* memory = wrNew int32 [32]
Which will coexist with my existing system:
 int32* memory = wrNewEx(EMemType:System) int32 [32];
Do you have some recommendations what you would want to see in your memory system? How you would trace stuff like that? Do you have maybe some nice secret tool that can help? Or maybe some nice trick in your slave that could help me with future issues ?


Sunday, August 30, 2015

Return from overseas

Finally I have my stuff from Canada :] There is a lot of them but one of the important ones are my computers. Thanks to that I return to normal programming but well before that I needed to sync my projects.

There was a lot of changes in projects in last three months so I expected problems and well there were :)

  • First is my lack of normal internet connection :/ yep I still waiting for it. So right now I try to use hotspot option in android to be able exchange code between Linux laptop <-> PC.
  • Second was broken Windows build :/ I tried really hard to have working version of windows but without continuous compilation it's hard to switch systems all the time just to check if I didn't broke something. But well hour later the build worked.
  • Third were missing manifest generations (TBD) which allow me to hold plugins to editor in separate folder. Because of that I needed to do some changes manually.
  • My continuous compilation is still dead I need to recover it (I missing it).

There were also things that went well:
  • New project generator works great :D I needed to do some improvement which took like one minute. Right now I'm adding support to vs2013 and I'm really happy how easy it is.
  • Compilation time - better computer and I don't have time almost on anything between pressing compiling and starting tools.
  • After fixing compilation errors everything works again.
  • I started using sharing of keyboard and mouse (I using Synergy) and this allow me on quicker switching between systems :) 
So as you see I'm not laying around and doing nothing. Return of my hardware should be boost for me in development and allow me on quicker iteration time and there is a lot to do :/ But well, such a life.


Sunday, August 16, 2015

Am I good programmer?

After some break in posting I returning with another weird topic :D

Am I good programmer ? 

This is one of this topics that is simple: I'm the best programmer :D so we all came to this nice conclusion and we can end all discussion ...

If only real life was so simple :) Some time ago I spend whole evening with friend discussing if I am good in what I do. Funny thing is that in this discussion I was the person who claimed that I'm still not skilled enough. Because what in the end mean that I'm good programmer?

I think that this is tricky question because of multiple reason:

  1. It depend who you compare to.
  2. It depend what kind of work you do/done.
  3. It depend what you expect being good in.
  4. It depend what you mean by being good.
So lets start in order:
Follow and be followed

After all this year I spend on programming I still seeing that there are people who are better than me and in the same time I see people that are not so good as me. It don't mean that they are good or bad. 

It just happen that in current time we are in this configuration. Few years later situation may change I can be better than all of them. Nothing may change and there exist possibility that I will be the worst of them. This is the reason why we programmers can never stop learning. In this world everything changing and if won't change with it we will be left behind.

Look what I done

This one is tricky because what mean that I done something? This most of the time mean that:
  • This task was assigned to me,
  • There was need for it,
  • There is no similar system in code base,
  • I sacrifice my own time to do something after work.
All of this mean that not everybody can do great, big stuff in work. There is army of people which do small changes, bug fixes and support. This not mean that they are worse than the people who done big systems.

I'm good in what I do

In this world there are specialist, generalists and stupid people like me. 
  • Specialists - are people who are awesome in what they do but they knowledge behind their specialization is often really limited. 
  • Generalists - are people who know a little bit of everything but they don't have tight specialization.
  • There are also this stupid people who want to be specialists in everything. Always trying to have big knowledge in everything they touch.
No mater which kind of people you want to be it won't be easy.

I'm who I'm 

If you see me as good programmer I'm appreciate this. If not then tell me why you think so. I will try my best to fix this. In the end I'm just another programmer who do the best it can to be the best in what he do. Thanks to that I know that I still have infinite possibilities for growth and this is what I love in this work: 

I can learn new stuff all the time so it's never really boring.

This is also reason why in my eyes you need to deserve for being good programmer you don't gain it just because you done something or you have a lot years of experience. You need to prove it to me and do this all the time.