org.datavec.spark.transform.misc

Class SequenceMergeFunction<T>

java.lang.Object

org.datavec.spark.transform.misc.SequenceMergeFunction<T>

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.Function<scala.Tuple2<T,java.lang.Iterable<java.util.List<java.util.List<Writable>>>>,java.util.List<java.util.List<Writable>>>

public class SequenceMergeFunction<T>
extends java.lang.Object
implements org.apache.spark.api.java.function.Function<scala.Tuple2<T,java.lang.Iterable<java.util.List<java.util.List<Writable>>>>,java.util.List<java.util.List<Writable>>>

Spark function for merging multiple sequences, using a SequenceMerge instance.
Typical usage:

 
 JavaPairRDD<SomeKey,List<List<Writable>>> myData = ...;
 SequenceComparator comparator = ...;
 SequenceMergeFunction<String> sequenceMergeFunction = new SequenceMergeFunction<>(new SequenceMerge(comparator));
 JavaRDD<List<List<Writable>>> merged = myData.groupByKey().map(sequenceMergeFunction);
 
 

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SequenceMergeFunction(SequenceMerge sequenceMerge)

Method Summary

Modifier and Type	Method and Description
java.util.List<java.util.List<Writable>>	call(scala.Tuple2<T,java.lang.Iterable<java.util.List<java.util.List<Writable>>>> t2)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SequenceMergeFunction

public SequenceMergeFunction(SequenceMerge sequenceMerge)

Method Detail

call

public java.util.List<java.util.List<Writable>> call(scala.Tuple2<T,java.lang.Iterable<java.util.List<java.util.List<Writable>>>> t2)
                                              throws java.lang.Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function<scala.Tuple2<T,java.lang.Iterable<java.util.List<java.util.List<Writable>>>>,java.util.List<java.util.List<Writable>>>

Throws:

java.lang.Exception