Today I was attending CG Linden’s office hours within Second Life where he was mostly talking about software deployment, infrastructure and related topics. It was a quite interesting discussion in which I again learned a bit more.
So here is some summary of what we talked about.
Server deployment
Today and tomorrow the server version 1.21 will be deployed. We talked a bit how this works and it was interesting to learn that they actually use BitTorrent to move their code to all their 6000 servers. This takes approx. 20 mins compared to the 7 hours it took before that (he admitted though that is maybe was not coded optimally).
The new version will mainly consist of database relief code and Havok fixes. The database code hopefully frees up some time to do some coding instead of watching and fixing the database. The Havok fixes should make sims more stable in general.
Regarding stability he was very happy with what the new Havok version brings because when he started there were usually 50 regions down at any given time (with 14,000 regions in total) and now they get annoyed if there are 5 down (with now nearly 19,000 regions). So this sounds like quite an improvement.
We also talked about deployment a bit more and he said with the deployment of the HetGrid Linden Lab can now run different simulator versions mixed on the grid. You notice that by that little popup which appears when you enter a region running a different version from the former one. Unfortunately it does not say yet what the changes really are. One point to note is also that right now all the simulators running on a physical machine (up to 4) need to run the same code base making moving sims around sometimes a bit hard because you need to meet the right software version.
There is some work ongoing though to change that so that different simulator on one machine can also run different versions.
Client deployment and infrastructure
As you might have noticed we got 2 RCs this week simply because there was a bug in the first one which needed to be fixed immediately. In general though he said that the code of the new version should be much saner and stable. The only problem is that when you do refactoring (restructuring) of the code you might screw something up. Usually unit tests and all that helps but the test coverage is not that big yet (unit tests are tests which can run automatically and should be run after every change and esp. before deployment). He also said that when he started (9 months ago) there have been about 5 tests or so and now there are over 200, so there is quite some progress. Still Second Life of course stays a quite comples piece of software.
We also talked a bit about Open Source and he stated again that Linden Lab really wants to be more open. He also said that maybe the choice of subversion as the version control software was not a good one and they are planning to change to mercurial. I asked him about whether they looked at git and he said yes but git is not working that well on Windows. Thus mercurial is the choice for the future (interesting as I remember lots of discussion about why not mercurial back when the Open Source client was released. IIRC the main reason was that Linden Lab employees were more used to CVS/SVN).
He also said that they need to communicate better with the Open Source developers and explain why certain fixes are not making it into the final release. Additionally Linden Lab wants to implement more in the open but certain propietary modules (such as Vivox) make this difficulty. But they are working on this and so there is hope for the future. At least having a „live“ repository for at least reading would be a good thing IMHO.
In general his opinion is that Linden Lab can also afford to work more on infrastructure now. Back in time the lab seemed more to be in survival mode, just quickly fixing bugs but not really working on some infrastructure like tests and developer community because the direct payback wasn’t there. It only counts on the long run. He said though that slowly they are getting there.
So all in all it has been a quite informative office hour and especially one at a good time for me (probably many things have been discussed already at the Open Source Office Hour but that’s usually a bit late for me).
If you want to join, you can find all the information here.
Technorati Tags: secondlife, office hours, lindenlab, cg linden, opensource