本文整理汇总了Java中org.apache.drill.exec.physical.impl.OutputMutator类的典型用法代码示例。如果您正苦于以下问题：Java OutputMutator类的具体用法？Java OutputMutator怎么用？Java OutputMutator使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
OutputMutator类属于org.apache.drill.exec.physical.impl包，在下文中一共展示了OutputMutator类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
try {
final int estimateRowSize = getEstimatedRecordSize(config.getTypes());
valueVectors = new ValueVector[config.getTypes().length];
batchRecordCount = 250000 / estimateRowSize;
for (int i = 0; i < config.getTypes().length; i++) {
final MajorType type = config.getTypes()[i].getMajorType();
final MaterializedField field = getVector(config.getTypes()[i].getName(), type, batchRecordCount);
final Class vvClass = TypeHelper.getValueVectorClass(field.getType().getMinorType(), field.getDataMode());
valueVectors[i] = output.addField(field, vvClass);
}
} catch (SchemaChangeException e) {
throw new ExecutionSetupException("Failure while setting up fields", e);
}
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:18，代码来源:MockRecordReader.java
示例2: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(final OperatorContext context, final OutputMutator output) throws ExecutionSetupException {
try{
if (hadoopPath != null) {
this.stream = fileSystem.openPossiblyCompressedStream(hadoopPath);
}
this.writer = new VectorContainerWriter(output);
if (isSkipQuery()) {
this.jsonReader = new CountingJsonReader(fragmentContext.getManagedBuffer());
} else {
this.jsonReader = new JsonReader(fragmentContext.getManagedBuffer(), ImmutableList.copyOf(getColumns()), enableAllTextMode, true, readNumbersAsDouble);
}
setupParser();
}catch(final Exception e){
handleAndRaise("Failure reading JSON file", e);
}
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:19，代码来源:JSONRecordReader.java
示例3: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
this.operatorContext = context;
this.outputMutator = output;
familyVectorMap = new HashMap<String, MapVector>();
try {
// Add Vectors to output in the order specified when creating reader
for (SchemaPath column : getColumns()) {
if (column.equals(ROW_KEY_PATH)) {
MaterializedField field = MaterializedField.create(column, ROW_KEY_TYPE);
rowKeyVector = outputMutator.addField(field, VarBinaryVector.class);
} else {
getOrCreateFamilyVector(column.getRootSegment().getPath(), false);
}
}
logger.debug("Opening scanner for HBase table '{}', Zookeeper quorum '{}', port '{}', znode '{}'.",
hbaseTableName, hbaseConf.get(HConstants.ZOOKEEPER_QUORUM),
hbaseConf.get(HBASE_ZOOKEEPER_PORT), hbaseConf.get(HConstants.ZOOKEEPER_ZNODE_PARENT));
hTable = new HTable(hbaseConf, hbaseTableName);
resultScanner = hTable.getScanner(hbaseScan);
} catch (SchemaChangeException | IOException e) {
throw new ExecutionSetupException(e);
}
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:26，代码来源:HBaseRecordReader.java
示例4: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
try {
final int estimateRowSize = getEstimatedRecordSize(config.getTypes());
if (config.getTypes() == null) {
return;
}
valueVectors = new ValueVector[config.getTypes().length];
batchRecordCount = 250000 / estimateRowSize;
for (int i = 0; i < config.getTypes().length; i++) {
final MajorType type = config.getTypes()[i].getMajorType();
final MaterializedField field = getVector(config.getTypes()[i].getName(), type, batchRecordCount);
final Class<? extends ValueVector> vvClass = TypeHelper.getValueVectorClass(field.getType().getMinorType(), field.getDataMode());
valueVectors[i] = output.addField(field, vvClass);
}
} catch (SchemaChangeException e) {
throw new ExecutionSetupException("Failure while setting up fields", e);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:21，代码来源:MockRecordReader.java
示例5: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
try {
final int estimateRowSize = getEstimatedRecordSize();
valueVectors = new ValueVector[fields.length];
int batchSize = config.getBatchSize();
if (batchSize == 0) {
batchSize = 10 * 1024 * 1024;
}
batchRecordCount = Math.max(1, batchSize / estimateRowSize);
batchRecordCount = Math.min(batchRecordCount, Character.MAX_VALUE);
for (int i = 0; i < fields.length; i++) {
final ColumnDef col = fields[i];
final MajorType type = col.getConfig().getMajorType();
final MaterializedField field = MaterializedField.create(col.getName(), type);
final Class<? extends ValueVector> vvClass = TypeHelper.getValueVectorClass(field.getType().getMinorType(), field.getDataMode());
valueVectors[i] = output.addField(field, vvClass);
}
} catch (SchemaChangeException e) {
throw new ExecutionSetupException("Failure while setting up fields", e);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:24，代码来源:ExtendedMockRecordReader.java
示例6: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(final OperatorContext context, final OutputMutator output) throws ExecutionSetupException {
try {
this.output = output;
this.buffer = new byte[100000];
this.in = fs.open(pathToFile);
this.decoder = new PacketDecoder(in);
this.validBytes = in.read(buffer);
this.projectedCols = getProjectedColsIfItNull();
setColumns(projectedColumns);
} catch (IOException io) {
throw UserException.dataReadError(io)
.addContext("File name:", pathToFile.toUri().getPath())
.build(logger);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:18，代码来源:PcapRecordReader.java
示例7: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(final OperatorContext context, final OutputMutator output) throws ExecutionSetupException {
try{
if (hadoopPath != null) {
this.stream = fileSystem.openPossiblyCompressedStream(hadoopPath);
}
this.writer = new VectorContainerWriter(output, unionEnabled);
if (isSkipQuery()) {
this.jsonReader = new CountingJsonReader(fragmentContext.getManagedBuffer(), enableNanInf);
} else {
this.jsonReader = new JsonReader.Builder(fragmentContext.getManagedBuffer())
.schemaPathColumns(ImmutableList.copyOf(getColumns()))
.allTextMode(enableAllTextMode)
.skipOuterList(true)
.readNumbersAsDouble(readNumbersAsDouble)
.enableNanInf(enableNanInf)
.build();
}
setupParser();
} catch (final Exception e){
handleAndRaise("Failure reading JSON file", e);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:25，代码来源:JSONRecordReader.java
示例8: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
this.operatorContext = context;
this.writer = new VectorContainerWriter(output, unionEnabled);
// Default is BsonReader and all text mode will not be honored in
// BsonRecordReader
if (isBsonRecordReader) {
this.bsonReader = new BsonRecordReader(fragmentContext.getManagedBuffer(), Lists.newArrayList(getColumns()),
readNumbersAsDouble);
logger.debug("Initialized BsonRecordReader. ");
} else {
this.jsonReader = new JsonReader.Builder(fragmentContext.getManagedBuffer())
.schemaPathColumns(Lists.newArrayList(getColumns()))
.allTextMode(enableAllTextMode)
.readNumbersAsDouble(readNumbersAsDouble)
.enableNanInf(enableNanInf)
.build();
logger.debug(" Intialized JsonRecordReader. ");
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:21，代码来源:MongoRecordReader.java
示例9: init
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void init(OutputMutator output) throws SchemaChangeException {
MaterializedField mf = MaterializedField.create(field.getName(), type);
@SuppressWarnings("unchecked")
Class<V> valueVectorClass = (Class<V>) TypeHelper.getValueVectorClass(type.getMinorType(), type.getMode());
this.vector = output.addField(mf, valueVectorClass);
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:8，代码来源:AbstractWriter.java
示例10: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(final OperatorContext context, final OutputMutator output) throws ExecutionSetupException {
operatorContext = context;
writer = new VectorContainerWriter(output);
try {
reader = new DataFileReader<>(new FsInput(hadoop, fs.getConf()), new GenericDatumReader<GenericContainer>());
logger.debug("Processing file : {}, start position : {}, end position : {} ", hadoop, start, end);
reader.sync(this.start);
} catch (IOException e) {
throw new ExecutionSetupException(e);
}
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:14，代码来源:AvroRecordReader.java
示例11: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
MaterializedField field = MaterializedField.create(ref, Types.repeated(TypeProtos.MinorType.VARCHAR));
try {
vector = output.addField(field, RepeatedVarCharVector.class);
} catch (Exception e) {
handleAndRaise("Failure in setting up reader", e);
}
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:10，代码来源:DrillTextRecordReader.java
示例12: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
this.operatorContext = context;
this.writer = new VectorContainerWriter(output);
this.jsonReader = new JsonReader(fragmentContext.getManagedBuffer(), Lists.newArrayList(getColumns()), enableAllTextMode, false, readNumbersAsDouble);
}
开发者ID:skhalifa，项目名称:QDrill，代码行数:8，代码来源:MongoRecordReader.java
示例13: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
public void setup(final OperatorContext context, final OutputMutator output) throws ExecutionSetupException {
this.writer = new VectorContainerWriter(output);
String regex = config.pattern;
r = Pattern.compile(regex);
fieldNames = config.fieldNames;
dataTypes = config.dataTypes;
errorOnMismatch = config.errorOnMismatch;
}
开发者ID:cgivre，项目名称:drill-logfile-plugin，代码行数:9，代码来源:LogRecordReader.java
示例14: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
for (int i = 0; i < columnNames.size(); i++) {
final TypeProtos.MajorType type = columnTypes.get(i);
final MaterializedField field = MaterializedField.create(columnNames.get(i), type);
final Class vvClass = TypeHelper.getValueVectorClass(type.getMinorType(), type.getMode());
try {
output.addField(field, vvClass);
} catch (SchemaChangeException e) {
throw new ExecutionSetupException(e);
}
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:14，代码来源:FindLimit0Visitor.java
示例15: setupWriters
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
/**
* Creates writers based on pojo field class types. Ignores static fields.
*
* @param output output mutator
* @return list of pojo writers
*/
@Override
protected List<PojoWriter> setupWriters(OutputMutator output) throws ExecutionSetupException {
List<PojoWriter> writers = new ArrayList<>();
Field[] declaredFields = pojoClass.getDeclaredFields();
for (Field field : declaredFields) {
if (Modifier.isStatic(field.getModifiers())) {
continue;
}
writers.add(initWriter(field.getType(), field.getName(), output));
fields.add(field);
}
return writers;
}
开发者ID:axbaretto，项目名称:drill，代码行数:20，代码来源:PojoRecordReader.java
示例16: setupWriters
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
/**
* Initiates writers based on given schema which contains field name and its type.
*
* @param output output mutator
* @return list of pojo writers
*/
@Override
protected List<PojoWriter> setupWriters(OutputMutator output) throws ExecutionSetupException {
List<PojoWriter> writers = new ArrayList<>();
for (Map.Entry<String, Class<?>> field : schema.entrySet()) {
writers.add(initWriter(field.getValue(), field.getKey(), output));
}
return writers;
}
开发者ID:axbaretto，项目名称:drill，代码行数:15，代码来源:DynamicPojoRecordReader.java
示例17: initWriter
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
/**
* Creates writer based input class type and then initiates it.
*
* @param type class type
* @param fieldName field name
* @param output output mutator
* @return pojo writer
*/
protected PojoWriter initWriter(Class<?> type, String fieldName, OutputMutator output) throws ExecutionSetupException {
PojoWriter writer = PojoWriters.getWriter(type, fieldName, output.getManagedBuffer());
try {
writer.init(output);
return writer;
} catch (SchemaChangeException e) {
throw new ExecutionSetupException("Failure while setting up schema for AbstractPojoRecordReader.", e);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:18，代码来源:AbstractPojoRecordReader.java
示例18: init
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void init(OutputMutator output) throws SchemaChangeException {
MaterializedField mf = MaterializedField.create(fieldName, type);
@SuppressWarnings("unchecked")
Class<V> valueVectorClass = (Class<V>) TypeHelper.getValueVectorClass(type.getMinorType(), type.getMode());
this.vector = output.addField(mf, valueVectorClass);
}
开发者ID:axbaretto，项目名称:drill，代码行数:8，代码来源:AbstractPojoWriter.java
示例19: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(final OperatorContext context, final OutputMutator output) throws ExecutionSetupException {
writer = new VectorContainerWriter(output);
try {
reader = getReader(hadoop, fs);
logger.debug("Processing file : {}, start position : {}, end position : {} ", hadoop, start, end);
reader.sync(this.start);
} catch (IOException e) {
throw new ExecutionSetupException(e);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:13，代码来源:AvroRecordReader.java
示例20: setup
import org.apache.drill.exec.physical.impl.OutputMutator; //导入依赖的package包/类
@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
MaterializedField field = MaterializedField.create(ref.getAsNamePart().getName(), Types.repeated(TypeProtos.MinorType.VARCHAR));
try {
vector = output.addField(field, RepeatedVarCharVector.class);
} catch (Exception e) {
handleAndRaise("Failure in setting up reader", e);
}
}
开发者ID:axbaretto，项目名称:drill，代码行数:10，代码来源:DrillTextRecordReader.java
注：本文中的org.apache.drill.exec.physical.impl.OutputMutator类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论