This was purely the state of things when I was working there. Most of the people were on reality server. Technology was ported from RS to mental ray, but porting was a nightmare. Most of the people that didn't quit I assume they still are on RS. What you see now in ray 3.6/3.7 was being initiated in 2006, and all the people who did that, eg irradiance particles, bsp2, particle maps etc are gone (you can see all this from linkedin, again
From my perspective mental ray is a finished product that just lives on its past success in being integrated in OEMs. The codebase and architecture is too old to do anything innovative with it in an efficient quick and clean manner. Of course other will believe the opposite and love it (I actually liked it myself - shame on me - before having some terms of comparisons), anyway mine is a purely personal opinion. I just like simplicity, freedom and openness.
Softimage realized all this first, they opened the rendering API very well with XSI 6.5. That was a signal of lack of satisfaction towards mental. They were ahead already with ICE and multithreading, and were actually a profitable company (I think they were doing 120% of their targets last year and this year it would have been way better).
That is why I am a bit sad it is not an independent company. Also a lot Soft people lost their work in the worst time possible, so that is sad too.
If ADSK management keeps their high promises and the company has no financing problem then all should be alright.
But of for any reason ADSK will be in troubles, then they will be forced to cut down on their overlapping or less profitable products.
Also they have no renderer (I have to say they have some cool stuff in showcase, as you mentioned, but that is not for animation) while mental ray is sprayed allover with different integration quality, very very risky & bad. They should think of a safer solution, a unified integration, and keep a very open rendering API & policy at the same time (so that more modern renderers can be integrated properly). A challenge.
On a separate line, just look at the poor implementation of RenderPasses in Maya2009. What a horrible work they've managed to do in two years of R&D is absolutely unbelievable from my point of view.