Project Lambda, Just How Bad A Mistake Is It?

Java 8 stuck in a traffic jam of its own making.
Pretty massive - that is how bad.

Lambdas in Java 8 work OK and are very fast. So, that is not a problem. I showed just how fast they are previously. The problem is that they are very much too complex (as can be seen by profiling their operation). A typical ten thousand moving part machine to crack a nut approach. Why oh why did Oracle not use a hammer?

There are two competing approaches which could have worked. One is invokedynamic and the other is using inner classes. I will show that invokedynamic was not a very good choice but first lets look at the reasons invokedynamic was chosen and why they are wrong.

Some issues with using inner classes (see this lecture

I found the lecture very interesting as it really did appear to be post rationalisation. Further, it was materially incorrect. The lecturer Brian Goetz must have known he was talking nonsense because the alternative is that he is an idiot, which seems unlikely. As someone who has worked on the design of two JVM language compilers I will show here the errors in what he has been saying.

(Incorrect) Reasons Invoke Dynamic Is Better Than Inner Classes:

Non capturing Lambdas
BG argues that if one were to use inner classes, then the class would still need to be generated even when the lambda is non capturing. This he suggests adds a performance cost. He is wrong. A non capturing lambda is stateless; if one were to implement this as a inner class then it would also be stateless and thus all methods in it would be static. Invokestatic is the fastest form of invokation in the JVM. OK, but a lambda can be polymorphic and/or passed around as a first class entity. Again, we can cache instances of stateless inner classes so they performance benefit suggested does not apply. By deliberately not discussing these points BG and others make the inner class approach seem as though it is slower for non capturing lambdas, but this is sophistry.

BG argues that many inner class instances is 'clunky'. I would say so what - clunky is an emotional response and has nothing to do with the actual suitability of the approach. Further, adding a container type to the JVM such that classes and the inner classes can be packaged together would have been a trivial piece of work. Thus, this argument holds no water.

Lock In
BG also argues that using inner classes causes 'implementation lock in' which is a bad thing. Leaving implementation to the runtime avoids this. He is doubly wrong here; firstly, binary lock in is a very good thing (as I will explain later) but more important is the second point that inner classes already exist and so there is no lock in which does not already exist if one continues to use them. Put simply, the posited argument is null and void.

Class Proliferation
The argument over inner class proliferation is also not a strong one because with invokedynamic, inner classes are still required for capturing lambdas. The difference is much less than one might suspect. Indeed, any implementation done with invokedynamic can be replicated using invokevirtual and/or invokestatic because Java is a static typed language. Only in dynamically typed languages does this not hold. 

Why Runtime Implementation Is A Very Bad Idea Indeed
Rather than the delightful notion that creating the implementation at runtime offers many benefits actually it creates enormous problems. One of the greatest benefit tempted for runtime implementation is that the implementation can be changed at any point without having to recompile the source code. This is not a benefit; actually it is a major, huge catastrophe. You might ask why; it is quite simply a disaster for the implementation of tooling. The Java ecosystem has a locked of excellence tooling. Everything from profilers to debuggers need to have some understanding of how the Java system works to be able to present information to the user cleanly. This tooling simply cannot be supported in the face of a runtime implementation which can change at the drop of a hat. This level of ignorance (by the Oracle team) as to the way commercial languages really work in the real world is rather disconcerting.

It gets worse, because no implementation is truly independent. Let me explain, a well tuned Java program in a mission-critical system will be tuned against a particular implementation of the JVM. Despite the Lambda system being defined by standards and interfaces, it cannot not be perfectly independent. Changing the runtime implementation will affect the tuning of mission-critical Java programs. This means that should Oracle change the implementation, lambdas will be rendered much less useful for mission-critical systems. Therefore a binary lockin happens anyway. The extra complexity of runtime resolution does not yield extra flexibility in the real world.

There are even more major problems with the runtime approach. It is so complex with so many classes being called it will cause even more difficulties for tooling. Therefore we have added an enormous burden to the Java ecosystem with no discernible benefit. Indeed I believe it is reasonable to say that the choice of runtime implementation for lambdas will cost the Java community hundreds of millions of dollars. Not only was the runtime implementation not well thought through it is actually highly irresponsible and will damage Java.

Existing and future damage to Java
The reality is that the overly complex, overly ambitious and just plain wrong headed invokedynamic approach is already damaging Java. We can see this in the repeated slipping of the Java eight release date. We can also see it in the large waste of resource on this project which could have been placed elsewhere. Indeed Oracle is now doing the right thing by diverting resource from the lambda implementation into much more important stuff like stability and security.

The wrong approach taken for Java 8 lambdas will also really hinder uptake of Java eight. Says in the security instability of the system will be damaged. A good set of tools which will be stable in the face of the new language feature will be a long time incoming. All these factors combined mean that project lambda in Java 8 will cause Java 7 to be hanging around much longer than it should and could indeed significantly weaken the proposition of Java as a language itself.

There is no evidence that lambdas are a absolutely critical language feature; they were never anything more than a nice to have. But again it gets worse; because the key thing that makes lambdas really useful is type influence. That is missing from Java 8 lambdas. In C++ we have the auto keyword in C# we have the var keyword in Java we have nothing like that. This may means that implementing lambdas in Java requires definition of interfaces for a lot of conditions under which these would not be required in other languages. Further, there is no capture by reference facility forcing the encapsulation of references by object. These "features" are actually crippling for the Java Lambda approach.

In Conclusion
The only reasonable conclusion is that project Lambda has been an unmitigated cock-up with the possible exception of the method references syntax which is quite neat. Java did not need lambdas so at least when they were introduced this should have been done in a clean way with minimal side effects and impact. However the implementation is clunky slow and adds massive complexity to the JVM damages the ecosystem messes up serialisation concepts and has massively delayed the release of Java eight. What a cluster f***!

Lessons Learned
  1. Implement stuff that needs to be done. Don't use a mainstream commercial program as a playground.
  2. Runtime implementation is a bad idea. 
  3. Invokeddynamic in a static language is a bad idea. 
  4. Dynamic anything is a bad idea for stability and performance. 
  5. The power of static languages is in the name static. 
  6. The Java language implementers really need to pull their heads out of their asses before they do anything else as big as project lambda.