org.datavec.api.io

Class WritableUtils

java.lang.Object

org.datavec.api.io.WritableUtils

public final class WritableUtils
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description
WritableUtils()

Method Summary

Modifier and Type	Method and Description
static <T extends Writable> T	clone(T orig, Configuration conf) Make a copy of a writable object using serialization to a buffer.
static int	decodeVIntSize(byte value) Parse the first byte of a vint/vlong to determine the number of bytes
static void	displayByteArray(byte[] record)
static int	getVIntSize(long i) Get the encoded length if an integer is stored in a variable-length format
static boolean	isNegativeVInt(byte value) Given the first byte of a vint/vlong, determine the sign
static byte[]	readCompressedByteArray(java.io.DataInput in)
static java.lang.String	readCompressedString(java.io.DataInput in)
static java.lang.String[]	readCompressedStringArray(java.io.DataInput in)
static <T extends java.lang.Enum<T>> T	readEnum(java.io.DataInput in, java.lang.Class<T> enumType) Read an Enum value from DataInput, Enums are read and written using String values.
static java.lang.String	readString(java.io.DataInput in)
static java.lang.String[]	readStringArray(java.io.DataInput in)
static int	readVInt(java.io.DataInput stream) Reads a zero-compressed encoded integer from input stream and returns it.
static long	readVLong(java.io.DataInput stream) Reads a zero-compressed encoded long from input stream and returns it.
static void	skipCompressedByteArray(java.io.DataInput in)
static void	skipFully(java.io.DataInput in, int len) Skip len number of bytes in input streamin
static byte[]	toByteArray(Writable... writables) Convert writables to a byte array
static int	writeCompressedByteArray(java.io.DataOutput out, byte[] bytes)
static int	writeCompressedString(java.io.DataOutput out, java.lang.String s)
static void	writeCompressedStringArray(java.io.DataOutput out, java.lang.String[] s)
static void	writeEnum(java.io.DataOutput out, java.lang.Enum<?> enumVal) writes String value of enum to DataOutput.
static void	writeString(java.io.DataOutput out, java.lang.String s)
static void	writeStringArray(java.io.DataOutput out, java.lang.String[] s)
static void	writeVInt(java.io.DataOutput stream, int i) Serializes an integer to a binary stream with zero-compressed encoding.
static void	writeVLong(java.io.DataOutput stream, long i) Serializes a long to a binary stream with zero-compressed encoding.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WritableUtils

public WritableUtils()

Method Detail

readCompressedByteArray

public static byte[] readCompressedByteArray(java.io.DataInput in)
                                      throws java.io.IOException

Throws:

java.io.IOException

skipCompressedByteArray

public static void skipCompressedByteArray(java.io.DataInput in)
                                    throws java.io.IOException

Throws:

java.io.IOException

writeCompressedByteArray

public static int writeCompressedByteArray(java.io.DataOutput out,
                                           byte[] bytes)
                                    throws java.io.IOException

Throws:

java.io.IOException

readCompressedString

public static java.lang.String readCompressedString(java.io.DataInput in)
                                             throws java.io.IOException

Throws:

java.io.IOException

writeCompressedString

public static int writeCompressedString(java.io.DataOutput out,
                                        java.lang.String s)
                                 throws java.io.IOException

Throws:

java.io.IOException

writeString

public static void writeString(java.io.DataOutput out,
                               java.lang.String s)
                        throws java.io.IOException

Throws:

java.io.IOException

readString

public static java.lang.String readString(java.io.DataInput in)
                                   throws java.io.IOException

Throws:

java.io.IOException

writeStringArray

public static void writeStringArray(java.io.DataOutput out,
                                    java.lang.String[] s)
                             throws java.io.IOException

Throws:

java.io.IOException

writeCompressedStringArray

public static void writeCompressedStringArray(java.io.DataOutput out,
                                              java.lang.String[] s)
                                       throws java.io.IOException

Throws:

java.io.IOException

readStringArray

public static java.lang.String[] readStringArray(java.io.DataInput in)
                                          throws java.io.IOException

Throws:

java.io.IOException

readCompressedStringArray

public static java.lang.String[] readCompressedStringArray(java.io.DataInput in)
                                                    throws java.io.IOException

Throws:

java.io.IOException

displayByteArray

public static void displayByteArray(byte[] record)

clone

public static <T extends Writable> T clone(T orig,
                                           Configuration conf)

Make a copy of a writable object using serialization to a buffer.

Parameters:

orig - The object to copy

Returns:

The copied object

writeVInt

public static void writeVInt(java.io.DataOutput stream,
                             int i)
                      throws java.io.IOException

Serializes an integer to a binary stream with zero-compressed encoding. For -120 <= i <= 127, only one byte is used with the actual value. For other values of i, the first byte value indicates whether the integer is positive or negative, and the number of bytes that follow. If the first byte value v is between -121 and -124, the following integer is positive, with number of bytes that follow are -(v+120). If the first byte value v is between -125 and -128, the following integer is negative, with number of bytes that follow are -(v+124). Bytes are stored in the high-non-zero-byte-first order.

Parameters:

stream - Binary output stream

i - Integer to be serialized

Throws:

java.io.IOException

writeVLong

public static void writeVLong(java.io.DataOutput stream,
                              long i)
                       throws java.io.IOException

Serializes a long to a binary stream with zero-compressed encoding. For -112 <= i <= 127, only one byte is used with the actual value. For other values of i, the first byte value indicates whether the long is positive or negative, and the number of bytes that follow. If the first byte value v is between -113 and -120, the following long is positive, with number of bytes that follow are -(v+112). If the first byte value v is between -121 and -128, the following long is negative, with number of bytes that follow are -(v+120). Bytes are stored in the high-non-zero-byte-first order.

Parameters:

stream - Binary output stream

i - Long to be serialized

Throws:

java.io.IOException

readVLong

public static long readVLong(java.io.DataInput stream)
                      throws java.io.IOException

Reads a zero-compressed encoded long from input stream and returns it.

Parameters:

stream - Binary input stream

Returns:

deserialized long from stream.

Throws:

java.io.IOException

readVInt

public static int readVInt(java.io.DataInput stream)
                    throws java.io.IOException

Reads a zero-compressed encoded integer from input stream and returns it.

Parameters:

stream - Binary input stream

Returns:

deserialized integer from stream.

Throws:

java.io.IOException

isNegativeVInt

public static boolean isNegativeVInt(byte value)

Given the first byte of a vint/vlong, determine the sign

Parameters:

value - the first byte

Returns:

is the value negative

decodeVIntSize

public static int decodeVIntSize(byte value)

Parse the first byte of a vint/vlong to determine the number of bytes

Parameters:

value - the first byte of the vint/vlong

Returns:

the total number of bytes (1 to 9)

getVIntSize

public static int getVIntSize(long i)

Get the encoded length if an integer is stored in a variable-length format

Returns:

the encoded length

readEnum

public static <T extends java.lang.Enum<T>> T readEnum(java.io.DataInput in,
                                                       java.lang.Class<T> enumType)
                                                throws java.io.IOException

Read an Enum value from DataInput, Enums are read and written using String values.

Type Parameters:

T - Enum type

Parameters:

in - DataInput to read from

enumType - Class type of Enum

Returns:

Enum represented by String read from DataInput

Throws:

java.io.IOException

writeEnum

public static void writeEnum(java.io.DataOutput out,
                             java.lang.Enum<?> enumVal)
                      throws java.io.IOException

writes String value of enum to DataOutput.

Parameters:

out - Dataoutput stream

enumVal - enum value

Throws:

java.io.IOException

skipFully

public static void skipFully(java.io.DataInput in,
                             int len)
                      throws java.io.IOException

Skip len number of bytes in input streamin

Parameters:

in - input stream

len - number of bytes to skip

Throws:

java.io.IOException - when skipped less number of bytes

toByteArray

public static byte[] toByteArray(Writable... writables)

Convert writables to a byte array