Monday, January 30, 2017

Editor disaster ...

So I'm still waiting for starting of my new job in Unity Technologies so I have some spare time. I use it to see Copenhagen, drink coffee, meet new people and of course as always: coding. 

This time in my coding journey I try to figure out reason behind my slow reaction time of editor. My windows machine handed everything perfectly fine but my a lot slower laptop with Linux gave terrible results :/ How bad? I tried to rotate scene and there was few second delay between action and reaction So really bad. There was no choice but to fix it. 


First like always my setup:

Currently when I start editor I running:
  • Editor UI:
    • Main Thread 
    • Editor event processing thread (TCP/IP connection with Engine)
  • Engine:
    • Main Thread
    • App Thread
    • Worker thread A
    • Worker thread B
    • Editor events processing thread (TCP/IP connection with UI)
  • Resource Builder Server:
    • Main Thread
    • Thread per remote file system connection (TCP/IP connection with Client)
    • Thread for file system server
    • Thread per builder node (TCP/IP connection with Building Node)
    • Dir watch threads (right now running around 2)
  • Resource Builder Node: 
    • Main Thread
So as you can see I have a lot to handle for my Quad Core machine. But well looking on editor whole scene was running properly so I decided to first focus on Editor events processing thread. This was my starting point for my search.

// Well I wish if it was like that but well first I found few issues with compilation of resources on Linux which made me busy for few hours. But then...

I started my investigation by checking my CPU usage and verifying if priorities of threads not affected they performance. As you can guest non of them shown anything weird. My other idea was that maybe I have some mistake in my network code and there is bottleneck. 

Hacked quickly profiles shown me something interesting: my receiving of data taking a lot more time comparing to sending. But after a little bit of looking around and thinking: This is not something weird (at least in my opinion). When we sending data we just filling buffers receiving need more processing to send data. I will need to read about this a little bit more in my free time.

// I also have theory that Linux processing lo connection (on the same computer between processes) slower then windows :( I may be here wrong and will need to verify this (Anybody know something about this topic ? or know some good sources ?)

Well after that came the phase where I tried to poke some pretty random places to collect more information about problem. Affinities added to threads had no effect at all so I gave up on idea that there is some issue with threading. Few other leads also didn't shown anything interesting but what interested me my random breaks in debugger were stuck into exchange of data about changed files with Resource Builder Server. Because I didn't need this code to run editor I decided to run it without it and there was improvement :)

Rest is pretty straight forward because of course mistake was on my side. I tried be smart and created just one connection between Editor UI and Engine which resulted in some gymnastic to be able send and receive asynchronously data both way. Well this didn't make too much sense when I look on it right now and now I just opening two separate connection for sending and receiving. I need also to say that Linux was here really useful: tcpdump gave me enough information to understand that my way of using TCP/IP connection was here the biggest bottleneck.

Well this is not the first time and for sure not the last time when after switching on slower machine there shown some issues which we never seen before. But for now I can finally relax a little bit and move to recovering rest of broken stuff :) I slowly get used to debugging in lldb but I still missing Visual Studio :(

 Greg

No comments:

Post a Comment