Tuesday, July 30, 2013

Voyage into the Scala reflection land, part 1.

Recently I have played a little with Scala reflection capabilities. Hence I would like to share what I have learned.

Welcome to Scala version 2.10.2 (OpenJDK 64-Bit Server VM, Java 1.7.0_21).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import reflect.runtime.{universe=>ru}
import reflect.runtime.{universe=>ru}
scala> import reflect.runtime.{currentMirror=>mirror}

First step: import the "experimental" reflection library and assign a tag to it (this way it's easy to see in code where this library has been used. Also, import the "root" mirror through which you, like Alice, can enter the reflection world. Another way to obtain the "root" mirror would be by calling:

scala> val otherRootMirror = ru.runtimeMirror(getClass.getClassLoader)
otherRootMirror: reflect.runtime.universe.Mirror = JavaMirror with scala.tools.nsc.interpreter.IMain$TranslatingClassLoader@59a10521 of type class scala.tools.nsc.interpreter.IMain$TranslatingClassLoader with classpath [(memory)] and parent being scala.tools.nsc.util.ScalaClassLoader$URLClassLoader@5a57e77f of type class scala.tools.nsc.util.ScalaClassLoader$URLClassLoader with classpath [file:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/resources.jar,file:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/rt.jar,file:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/jsse.jar,file:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/jce.jar,file:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/charsets.jar,file:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/rhino.jar,file:/usr/local/share/scala/scala-2.10.2/lib/akka-actors.jar,f...

Now, let's do an easy thing: define a case class and inspect its instance with reflection.

scala> case class ReflectDemo(intVal: Integer, stringVal:String) {
     | def multiply(byThat: Int): Int = intVal*byThat
     | def reverse(): String = stringVal.reverse
     | }
defined class ReflectDemo

scala> val rd = ReflectDemo(5, "five")
rd: ReflectDemo = ReflectDemo(5,five)

Let's get the type symbol for the object we are using. The docs about the Scala reflection suggest defining a helper in the following way:

scala> def getTypeTag[T:ru.TypeTag](obj:T) = ru.typeOf[T]
getTypeTag: [T](obj: T)(implicit evidence$1: reflect.runtime.universe.TypeTag[T])reflect.runtime.universe.Type

Now we can get the type tag of our object!

scala> val typeTag = getTypeTag(rd)
typeTag: reflect.runtime.universe.Type = ReflectDemo

It opens a lot of possibilities...

scala> typeTag.
=:=                 asInstanceOf        asSeenFrom          baseClasses         baseType            contains            
declaration         declarations        erasure             exists              find                foreach             
isInstanceOf        map                 member              members             normalize           substituteSymbols   
substituteTypes     takesTypeArgs       termSymbol          toString            typeConstructor     typeSymbol          
weak_<:<            widen               

For example, we can use it to get all the base classes for this type:

scala> typeTag.baseClasses
res10: List[reflect.runtime.universe.Symbol] = List(class ReflectDemo, trait Serializable, trait Serializable, trait Product, trait Equals, class Object, class Any)

We can get the type name:

scala> typeTag.typeSymbol.name
res18: reflect.runtime.universe.Name = ReflectDemo

And all the members (defined in all base classes up to the very generic ones:

scala> typeTag.members
res8: reflect.runtime.universe.MemberScope = Scopes(method equals, method toString, method hashCode, method canEqual, method productIterator, method productElement, method productArity, method productPrefix, method copy$default$2, method copy$default$1, method copy, method reverse, method multiply, constructor ReflectDemo, value stringVal, value stringVal, value intVal, value intVal, method $init$, method $asInstanceOf, method $isInstanceOf, method synchronized, method ##, method !=, method ==, method ne, method eq, constructor Object, method notifyAll, method notify, method clone, method getClass, method wait, method wait, method wait, method finalize, method asInstanceOf, method isInstanceOf, method !=, method ==)

The other method, declarations, seems to omit the generics:

scala> typeTag.declarations
res9: reflect.runtime.universe.MemberScope = SynchronizedOps(value intVal, value intVal, value stringVal, value stringVal, constructor ReflectDemo, method multiply, method reverse, method copy, method copy$default$1, method copy$default$2, method productPrefix, method productArity, method productElement, method productIterator, method canEqual, method hashCode, method toString, method equals)

In any case, we can use one of these to get the list of all the variables defined in the object of this type:

scala> val variables = typeTag.members.filter(!_.isMethod)
variables: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(value stringVal, value intVal)

Now, back to the mirrors and stuff. Let's use the root mirror to get the InstanceMirror for the runtime object which we are inspecting.

scala> val instanceMirror = mirror.reflect(rd)
instanceMirror: reflect.runtime.universe.InstanceMirror = instance mirror for ReflectDemo(5,five)

We can use this mirror to reflect any of the variables belonging to that object, using the Symbols we obtained earlier:

scala> variables.map(m=>instanceMirror.reflectField(m.asTerm))
res12: Iterable[reflect.runtime.universe.FieldMirror] = List(field mirror for ReflectDemo.stringVal (bound to ReflectDemo(5,five)), field mirror for ReflectDemo.intVal (bound to ReflectDemo(5,five)))

Ultimately, we can obtain the values for these variables:

scala> val values = variables.map(m=>instanceMirror.reflectField(m.asTerm).get)
values: Iterable[Any] = List(five, 5)

Also, we can use the same Symbols to get access to the variables' types:

scala> val types = variables.map(_.typeSignature)
types: Iterable[reflect.runtime.universe.Type] = List(String, java.lang.Integer)

Now, if we combine all this, we get the whole definition of the variable's properties:

scala> (variables zip( types zip values)).toMap
res17: scala.collection.immutable.Map[reflect.runtime.universe.Symbol,(reflect.runtime.universe.Type, Any)] = Map(value stringVal -> (String,five), value intVal -> (java.lang.Integer,5))

This can be used, for example, if you get a fancy to create a wrapper for persisting the given object into the database based solely on its structure. You can use the type information to create the table structure and then use the values of the given objects to populate the rows. (I did something like that for someone recently, as an experimental prototype. It did work :) ).

Update: How about creating new instances of classes during reflection? Well, let's try to create a new instance of our ReflectDemo class.

The original input, then, would be the type and the map of variables, where the names point to the values (in principle, we could also reflect the type of every variable and use the type info to check if the variables match the expectations of the constructor...)

scala> val values = Map("intVal"->5, "stringVal"->"five")
values: scala.collection.immutable.Map[String,Any] = Map(intVal -> 5, stringVal -> five)

scala> val rdType = ru.typeOf[ReflectDemo]
rdType: reflect.runtime.universe.Type = ReflectDemo

Let's reflect the class:

scala> val rdClass = mirror.reflectClass(rdType.typeSymbol.asClass)
rdClass: reflect.runtime.universe.ClassMirror = class mirror for ReflectDemo (bound to null)

The result is of the type ClassMirror and it can be used to invoke the constructor for the given class.
We can get all constructors for a class from its Type information:

scala> val constructors = rdType.members.filter(m=>m.isMethod && m.asMethod.isConstructor)
constructors: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(constructor ReflectDemo, method $init$, constructor Object)

In our case, we know we only have one constructor, and we can also access it using the predefined name tag:

scala> val defaultCtor = rdType.member(ru.nme.CONSTRUCTOR)
defaultCtor: reflect.runtime.universe.Symbol = constructor ReflectDemo

Let's have a look which parameters it expects, now that's the piece of magic you just have to know (took me some research :) ) :

scala> val paramsList = defaultCtor.asMethod.paramss
paramsList: List[List[reflect.runtime.universe.Symbol]] = List(List(value intVal, value stringVal))

I suppose that this reflects the fact that one can invoke the constructor with various sets of parameters.
In our case we only have one constructor, so let's to the easy way. If there would be more than one constructor, we would have to use the name and type information from the available values in order to find out the suitable one, which is the exercise I leave for the curious reader (as such things go :) )

scala> val params = paramsList.head

params: List[reflect.runtime.universe.Symbol] = List(value intVal, value stringVal)
scala> val mappedValues = params map (m=>values(m.name.toString))
mappedValues: List[Any] = List(5, five)

Now we are almost there. Use the ClassMirror to reflect the constructor:

scala> val runtimeCtor = rdClass.reflectConstructor(defaultCtor.asMethod)
runtimeCtor: reflect.runtime.universe.MethodMirror = constructor mirror for ReflectDemo.<init>(intVal: java.lang.Integer, stringVal: String): ReflectDemo (bound to null)

MethodMirror provides the method apply(args: Any*) which allows to supply a sequence of any kind of values. However, it won't accept the List, because it expects a Seq, so it will consider the List to be a single parameter, and complain loudly:

scala> runtimeCtor.apply(mappedValues)
java.lang.IllegalArgumentException: wrong number of arguments
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaConstructorMirror.apply(JavaMirrors.scala:444)

But (that's another little piece of magic) one can easily make a sequence from the List:

scala> val newRD = runtimeCtor.apply(mappedValues:_*)
newRD: Any = ReflectDemo(5,five)

Remember the instance of the same class we had in the beginning? Let's compare them and prove that we could recreate the same instance!

scala> rd == newRD
res24: Boolean = true


Well, that's enough for now. Next topic (if I get to that): the specifics of case classes and objects, and the dangers of their instantiating via reflection :) By the way, in this post, the case class could also be the normal class - nothing would have changed except that you'd create it with New. The ways you instantiate (case) objects and (case) classes via reflection, though, are very different, and should be used with some care! To be continued :)


  1. just a word of encouragement to continue....

    1. Thank you, added an update here, the other one is following shortly.