When is @uncheckedVariance needed in Scala, and why is it used in GenericTraversableTemplate?

@uncheckedVariance can be used to bridge the gap between Scala's declaration site variance annotations and Java's invariant generics.

scala> import java.util.Comparator    
import java.util.Comparator

scala> trait Foo[T] extends Comparator[T]
defined trait Foo

scala> trait Foo[-T] extends Comparator[T]     
<console>:5: error: contravariant type T occurs in invariant position in type [-T]java.lang.Object with java.util.Comparator[T] of trait Foo
       trait Foo[-T] extends Comparator[T]
             ^

scala> import annotation.unchecked._    
import annotation.unchecked._

scala> trai开发者_StackOverflowt Foo[-T] extends Comparator[T @uncheckedVariance]    
defined trait Foo

This says that java.util.Comparator is naturally contra-variant, that is the type parameter T appears in parameters and never in a return type.

This raises the question: why is it also used in the Scala collections library which doesn't extends from Java interfaces?

trait GenericTraversableTemplate[+A, +CC[X] <: Traversable[X]] extends HasNewBuilder[A, CC[A] @uncheckedVariance]

What are the valid uses for this annotation?

The problem is that GenericTraversableTemplate is used twice: once for mutable collections (where its type parameter should be invariant), and once for immutable collections (where covariance is invariably king).

GenericTraversableTemplate's typechecks assuming either covariance or invariance for the A type parameter. However, when we inherit it in a mutable trait, we have to pick invariance. Conversely, we'd like covariance in an immutable subclass.

Since we can't abstract over the variance annotation (yet ;-)) in GenericTraversableTemplate, so that we could have instantiated it to either one depending on the subclass, we have to resort to casting (@uncheckVariance is essentially a kind-cast). For further reading, I recommend my dissertation (sorry ;-)) or our recent bitrot paper

In my thesis I describe a calculus, Scalina, that has bounds & variance annotations as part of the kind language (an earlier version is also available as a workshop paper). The relevance to this discussion is the next step that I want to take in developing this calculus: build another layer on top of that so that you can abstract over bounds (easy) and variance annotations (makes my head spin). Actually, you wouldn't just tack 1 extra layer on there, but rather generalise your polymorphism constructs so they work at all levels, and make your "attributes" (bounds, variance annotations, required implicit arguments,...) into regular types with special kinds, which are all subject to abstraction.

The "attributes are types" idea is explained nicely by Edsko de Vries in the context of uniqueness types.

Uniqueness Typing Simplified, Edsko de Vries, Rinus Plasmeijer, and David Abrahamson. In Olaf Chitil, Zoltán Horváth and Viktória Zsók (Eds.): IFL 2007, LNCS 5083, pp. 201-218, 2008.

Abstract: We present a uniqueness type system that is simpler than both Clean's uniqueness system and the system we proposed previously. The new type system is straightforward to implement and add to existing compilers, and can easily be extended with advanced features such as higher rank types and impredicativity. We describe our implementation in Morrow, an experimental functional language with both these features. Finally, we prove soundness of the core type system with respect to the call-by-need lambda calculus.

I found another time where @uncheckedVariance is used -- the synthetic method that returns the default value for a parameter of an abstract type:

M:\>scala -Xprint:typer -e "class C { def p[T >: Null](t: T = null) = t }"
[[syntax trees at end of typer]]// Scala source: (virtual file)
package <empty> {
  final object Main extends java.lang.Object with ScalaObject {
    def this(): object Main = {
      Main.super.this();
      ()
    };
    def main(argv: Array[String]): Unit = {
      val args: Array[String] = argv;
      {
        final class $anon extends scala.AnyRef {
          def this(): anonymous class $anon = {
            $anon.super.this();
            ()
          };
          class C extends java.lang.Object with ScalaObject {
            <synthetic> def p$default$1[T >: Null <: Any]: Null @scala.annotation.unchecked.uncheckedVariance = null;
            def this(): this.C = {
              C.super.this();
              ()
            };
            def p[T >: Null <: Any](t: T = null): T = t
          }
        };
        {
          new $anon();
          ()
        }
      }
    }
  }

In Iterable.scala one can read for +C, the type of what tail returns that: "We require that for all child classes of Iterable the variance of the child class and the variance of the C parameter passed to IterableOps are the same. We cannot express this since we lack variance polymorphism. That's why we have to resort at some places to write C[A @uncheckedVariance]."

Now as to what are valid uses for this annotation, let's consider the following code:

class X[+T] {
    var ref_ : Any = null
    def ref:T = ref_.asInstanceOf[T]
    def ref_=(ref: T@uncheckedVariance): Unit = ref_ = ref
}

Without @uncheckedVariance, it wouldn't compile because covariant T appears in contravariant position. Now if Y[+T] extends X[+T] and B extends A then Y[B] extends X[A] and you can write:

val y: Y[B] = null
val x : X[A] = y

Which means you cannot write:

y.ref = new A{}

But you can write, despite x being y:

x.ref = new A{}

That means that when you define Y[B], you most likely don't intend it to be passed for ref some A which lacks the specificities of B, and should some A sneaks its way notheless, you would have a nasty bug. That's why variance is checked by default. Example of code that throws a ClassCastException:

val y = new Y[B]
val x : X[A] = y
x.ref = new A
y.ref.b() // b() in B but not in A

However sometimes you know what you're doing and are absolutely certain that nothing of the sort can happen. Or maybe your documentation explicitly warns the potential user, @uncheckedVariance being already a clear warning. In my experience there are four circumstances in which you may use this annotation.

1. When you are working within a private or protected scope, with more control over what's used and how. Perfectly valid.
2. When you're extending code that is intended to be used that way, or implementing a pattern that explicitly requires it. Also perfectly valid.
3. When you desperately need it because covariance is in your way, the same way you may use setAcessible(true) to bypass a scope limitation. Not valid, but may help, especially when trying to figure things out and as a temporary relief to let the code compile in the process of a complex refactoring.
4. When it makes your API significantly more convenient, with the benefit of covariance at a minor risk of miss-use. No less valid than the Java equivalent, generics don't have to be perfectly safe. For instance if your classes are mostly read from and covariance is great, but sometimes they are written to, and covariance is a pain, it's acceptable for writting functions to require extra checking.