• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java FieldAssembler类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.avro.SchemaBuilder.FieldAssembler的典型用法代码示例。如果您正苦于以下问题:Java FieldAssembler类的具体用法?Java FieldAssembler怎么用?Java FieldAssembler使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



FieldAssembler类属于org.apache.avro.SchemaBuilder包,在下文中一共展示了FieldAssembler类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: createSchemaFor

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
 * Generate a schema with given schema name and has one field of STRING type
 * for each dimension and one field of LONG type for each metric
 * @param schemaName
 * @param dimensions
 * @param metrics
 * @return
 */
public static Schema createSchemaFor(String schemaName, String[] dimensions,
    String[] metrics) {
  FieldAssembler<Schema> fields;
  RecordBuilder<Schema> record = SchemaBuilder.record(schemaName);
  fields = record.namespace("com.linkedin.thirdeye.join").fields();
  for (String dimension : dimensions) {
    fields = fields.name(dimension).type().nullable().stringType()
        .noDefault();
  }
  for (String metric : metrics) {
    fields = fields.name(metric).type().nullable().longType().longDefault(0);
  }
  Schema schema = fields.endRecord();
  return schema;
}
 
开发者ID:Hanmourang,项目名称:Pinot,代码行数:24,代码来源:AvroTestUtil.java


示例2: initDefaultSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema initDefaultSchema(Schema designSchema) {
    AvroRegistry avroReg = new AvroRegistry();
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    for (SplunkJSONEventField metadataField : SplunkJSONEventField.getMetadataFields()) {
        Schema base = avroReg.getConverter(metadataField.getDataType()).getSchema();
        FieldBuilder<Schema> fieldBuilder = record.name(metadataField.getName());
        if (metadataField.getName().equals(SplunkJSONEventField.TIME.getName())) {
            String datePattern;
            Field designField = designSchema.getField(metadataField.getName());
            if (designField != null) {
                datePattern = designField.getProp(SchemaConstants.TALEND_COLUMN_PATTERN);
            } else {
                datePattern = designSchema.getProp(SchemaConstants.TALEND_COLUMN_PATTERN);
            }
            if (datePattern == null || datePattern.isEmpty()) {
                datePattern = "dd-MM-yyyy";
            }
            fieldBuilder.prop(SchemaConstants.TALEND_COLUMN_PATTERN, datePattern);
        }
        fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
    }
    Schema defaultSchema = record.endRecord();
    return defaultSchema;
}
 
开发者ID:Talend,项目名称:components,代码行数:25,代码来源:TSplunkEventCollectorWriter.java


示例3: createTestSchema5

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema5() {
    FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();

    Schema schema = AvroUtils._int();
    schema = wrap(schema);
    builder = builder.name("ID").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "ID").type(schema).noDefault();

    schema = AvroUtils._string();
    schema = wrap(schema);
    builder = builder.name("NAME").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "NAME").type(schema).noDefault();

    schema = AvroUtils._string();// TODO : fix it as should be object type
    schema = wrap(schema);
    builder = builder.name("RESULTSET").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "RESULTSET").type(schema)
            .noDefault();

    return builder.endRecord();
}
 
开发者ID:Talend,项目名称:components,代码行数:19,代码来源:DBTestUtils.java


示例4: getGranularityBasedSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
@SuppressWarnings("fallthrough")
private Schema getGranularityBasedSchema() {
  FieldAssembler<Schema> assembler =
      SchemaBuilder.record("GenericRecordTimePartition").namespace("gobblin.writer.partitioner").fields();

  // Construct the fields in reverse order
  if (!Strings.isNullOrEmpty(this.writerPartitionSuffix)) {
    assembler = assembler.name(SUFFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
  }
  assembler = assembler.name(this.granularity.toString()).type(Schema.create(Schema.Type.STRING)).noDefault();

  if (!Strings.isNullOrEmpty(this.writerPartitionPrefix)) {
    assembler = assembler.name(PREFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
  }

  Schema schema = assembler.endRecord();
  Collections.reverse(schema.getFields());
  return schema;
}
 
开发者ID:apache,项目名称:incubator-gobblin,代码行数:20,代码来源:TimeBasedWriterPartitioner.java


示例5: schemaFor

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema schemaFor(List<String> fieldNames, List<String> fieldTypes) {
  FieldAssembler<Schema> assembler = SchemaBuilder.record("t").fields();

  for (int i = 0; i < fieldNames.size(); i++) {
    String fieldName = fieldNames.get(i);
    String fieldType = fieldTypes.get(i);

    switch (fieldType) {
      case "string":
        assembler = assembler.optionalString(fieldName);
        break;
      case "float":
        assembler = assembler.optionalFloat(fieldName);
        break;
      case "double":
        assembler = assembler.optionalDouble(fieldName);
        break;
      case "int":
        assembler = assembler.optionalInt(fieldName);
        break;
      case "long":
        assembler = assembler.optionalLong(fieldName);
        break;
      case "boolean":
        assembler = assembler.optionalBoolean(fieldName);
        break;
      case "binary":
        assembler = assembler.optionalBytes(fieldName);
        break;
      default:
        throw new RuntimeException("Unsupported provided field type: " + fieldType);
    }
  }

  return assembler.endRecord();
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:37,代码来源:AvroTranslator.java


示例6: inferSchemaRecord

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema inferSchemaRecord(IndexedRecord in) {
    FieldAssembler<Schema> builder = SchemaBuilder.builder().record(in.getSchema().getName()).fields();
    for (Schema.Field field : in.getSchema().getFields()) {
        Schema fieldSchema = inferSchema(field);
        Object fieldDefault = field.defaultVal();
        if (null == fieldDefault) {
            builder = builder.name(field.name()).type(fieldSchema).noDefault();
        } else {
            builder = builder.name(field.name()).type(fieldSchema).withDefault(fieldDefault);
        }
    }
    return builder.endRecord();
}
 
开发者ID:Talend,项目名称:components,代码行数:14,代码来源:GenericAvroRegistry.java


示例7: createTestSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema() {
    FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();

    Schema schema = AvroUtils._int();
    schema = wrap(schema);
    builder = builder.name("ID").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "ID").type(schema).noDefault();

    schema = AvroUtils._string();
    schema = wrap(schema);
    builder = builder.name("NAME").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "NAME").type(schema).noDefault();

    return builder.endRecord();
}
 
开发者ID:Talend,项目名称:components,代码行数:14,代码来源:PropertiesPreparer.java


示例8: setupDefaultSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private void setupDefaultSchema() {
    AvroRegistry avroReg = new AvroRegistry();
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    for (SplunkJSONEventField metadataField : SplunkJSONEventField.getMetadataFields()) {
        Schema base = avroReg.getConverter(metadataField.getDataType()).getSchema();
        FieldBuilder<Schema> fieldBuilder = record.name(metadataField.getName());
        if (metadataField.getName().equals(SplunkJSONEventField.TIME.getName())) {
            fieldBuilder.prop(SchemaConstants.TALEND_COLUMN_PATTERN, "dd-MM-yyyy");
        }
        fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
    }
    Schema defaultSchema = record.endRecord();
    schema.schema.setValue(defaultSchema);
}
 
开发者ID:Talend,项目名称:components,代码行数:15,代码来源:TSplunkEventCollectorProperties.java


示例9: createSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
 * Create default static schema for indexed record.
 */
private Schema createSchema() {
    AvroRegistry avroReg = new AvroRegistry();
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    addField(record, "FieldString", String.class, avroReg);
    addField(record, "FieldInt", Integer.class, avroReg);
    Schema defaultSchema = record.endRecord();
    return defaultSchema;
}
 
开发者ID:Talend,项目名称:components,代码行数:12,代码来源:TSplunkEventCollectorWriterTestIT.java


示例10: createSchemaWithTimeAsString

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
 * create schema for indexed record with Date presented as String.
 */
private Schema createSchemaWithTimeAsString() {
    AvroRegistry avroReg = new AvroRegistry();
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    addField(record, "FieldString", String.class, avroReg);
    addField(record, "FieldInt", Integer.class, avroReg);
    addField(record, "Description", String.class, avroReg);
    addField(record, "time", String.class, avroReg);
    Schema defaultSchema = record.endRecord();
    return defaultSchema;
}
 
开发者ID:Talend,项目名称:components,代码行数:14,代码来源:TSplunkEventCollectorWriterTestIT.java


示例11: addField

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
 * add field to Schema.
 */
private FieldAssembler<Schema> addField(FieldAssembler<Schema> record, String name, Class<?> type, AvroRegistry avroReg) {
    Schema base = avroReg.getConverter(type).getSchema();
    FieldBuilder<Schema> fieldBuilder = record.name(name);
    fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
    return record;
}
 
开发者ID:Talend,项目名称:components,代码行数:10,代码来源:TSplunkEventCollectorWriterTestIT.java


示例12: createDynamicSchemaWithDatePattern

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
 * Create dynamic schema with date pattern in dynamic column. Required to test parsing of Date presented as String.
 */
private Schema createDynamicSchemaWithDatePattern() {
    FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
    Schema defaultSchema = record.endRecord();
    AvroUtils.setIncludeAllFields(defaultSchema, true);
    AvroUtils.setProperty(defaultSchema, SchemaConstants.TALEND_COLUMN_PATTERN, DATE_PATTERN);
    return defaultSchema;
}
 
开发者ID:Talend,项目名称:components,代码行数:11,代码来源:TSplunkEventCollectorWriterTestIT.java


示例13: getMakeRowSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public Schema getMakeRowSchema(boolean isDynamic) {
    FieldAssembler<Schema> fa = SchemaBuilder.builder().record("MakeRowRecord").fields() //
            .name("Name").type().nullable().stringType().noDefault() //
            .name("ShippingStreet").type().nullable().stringType().noDefault() //
            .name("ShippingPostalCode").type().nullable().intType().noDefault() //
            .name("BillingStreet").type().nullable().stringType().noDefault() //
            .name("BillingState").type().nullable().stringType().noDefault() //
            .name("BillingPostalCode").type().nullable().stringType().noDefault();
    if (isDynamic) {
        fa = fa.name("ShippingState").type().nullable().stringType().noDefault();
    }

    return fa.endRecord();
}
 
开发者ID:Talend,项目名称:components,代码行数:15,代码来源:SalesforceTestBase.java


示例14: createTestSchema2

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema2() {
    FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();

    Schema schema = AvroUtils._int();
    schema = wrap(schema);
    builder = builder.name("ID").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "ID")
            .prop(SchemaConstants.TALEND_COLUMN_IS_KEY, "true").type(schema).noDefault();

    schema = AvroUtils._string();
    schema = wrap(schema);
    builder = builder.name("NAME").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "NAME").type(schema).noDefault();

    return builder.endRecord();
}
 
开发者ID:Talend,项目名称:components,代码行数:15,代码来源:DBTestUtils.java


示例15: createTestSchema4

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema4() {
    FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();

    Schema schema = AvroUtils._string();// TODO : fix it as should be object type
    schema = wrap(schema);
    builder = builder.name("RESULTSET").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "RESULTSET").type(schema)
            .noDefault();

    return builder.endRecord();
}
 
开发者ID:Talend,项目名称:components,代码行数:11,代码来源:DBTestUtils.java


示例16: addField

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private FieldAssembler<Schema> addField(
		FieldAssembler<Schema> fieldAssembler, PredicateObjectMap pom,
		boolean isMap, Schema targetSchema, String predicateShortHand) {
	try{
	if(isMap)
	{
		if(targetSchema == null)
		{
			fieldAssembler = fieldAssembler.name(pom.getPredicate().getId().replaceAll("[^\\w]", "_")).type().unionOf().map().values().unionOf().map().values().stringType().and().stringType().and().nullType().endUnion().and().nullType().endUnion().noDefault();
		}
		else
		{
			fieldAssembler = fieldAssembler.name(pom.getPredicate().getId().replaceAll("[^\\w]", "_")).type().unionOf().map().values().unionOf().map().values(targetSchema).and().type(targetSchema).and().nullType().endUnion().and().nullType().endUnion().noDefault();
		}
		
	}
	else {
		if(targetSchema == null)
		{
			fieldAssembler = fieldAssembler.name(predicateShortHand).type().unionOf().array().items().stringType().and().stringType().and().nullType().endUnion().noDefault();
		}
		else
		{
			fieldAssembler = fieldAssembler.name(predicateShortHand).type().unionOf().array().items(targetSchema).and().type(targetSchema).and().nullType().endUnion().noDefault();
		}
	}
	}
	catch(Exception e)
	{
		LOG.error("Unable to add field: " + predicateShortHand + " for " + pom.getTriplesMap().getSubject().getTemplate().toString(), e);
	}
	
	return fieldAssembler;
}
 
开发者ID:therelaxist,项目名称:spring-usc,代码行数:35,代码来源:AvroKR2RMLRDFWriter.java


示例17: getDateTimeFormatBasedSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema getDateTimeFormatBasedSchema() {
  FieldAssembler<Schema> assembler =
      SchemaBuilder.record("GenericRecordTimePartition").namespace("gobblin.writer.partitioner").fields();

  if (!Strings.isNullOrEmpty(this.writerPartitionPrefix)) {
    assembler = assembler.name(PREFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
  }
  assembler = assembler.name(PARTITIONED_PATH).type(Schema.create(Schema.Type.STRING)).noDefault();
  if (!Strings.isNullOrEmpty(this.writerPartitionSuffix)) {
    assembler = assembler.name(SUFFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
  }

  return assembler.endRecord();
}
 
开发者ID:apache,项目名称:incubator-gobblin,代码行数:15,代码来源:TimeBasedWriterPartitioner.java


示例18: constructAvroSchemaFromPinotSchema

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
 * Constructs an avro schema from a pinot schema
 * @param schema
 * @return
 */
public static Schema constructAvroSchemaFromPinotSchema(com.linkedin.pinot.common.data.Schema schema) {
  Schema avroSchema = null;

  RecordBuilder<Schema> recordBuilder = SchemaBuilder.record("record");
  FieldAssembler<Schema> fieldAssembler = recordBuilder.fields();

  for (FieldSpec fieldSpec : schema.getAllFieldSpecs()) {
    String fieldName = fieldSpec.getName();
    DataType dataType = fieldSpec.getDataType();
    BaseFieldTypeBuilder<Schema> baseFieldTypeBuilder = fieldAssembler.name(fieldName).type().nullable();
    switch (dataType) {
      case BOOLEAN:
        fieldAssembler = baseFieldTypeBuilder.booleanType().noDefault();
        break;
      case DOUBLE:
        fieldAssembler = baseFieldTypeBuilder.doubleType().noDefault();
        break;
      case FLOAT:
        fieldAssembler = baseFieldTypeBuilder.floatType().noDefault();
        break;
      case INT:
        fieldAssembler = baseFieldTypeBuilder.intType().noDefault();
        break;
      case LONG:
        fieldAssembler = baseFieldTypeBuilder.longType().noDefault();
        break;
      case STRING:
        fieldAssembler = baseFieldTypeBuilder.stringType().noDefault();
        break;
      default:
        break;
    }
  }

  avroSchema = fieldAssembler.endRecord();
  LOGGER.info("Avro Schema {}", avroSchema.toString(true));

  return avroSchema;
}
 
开发者ID:linkedin,项目名称:pinot,代码行数:45,代码来源:ThirdeyeAvroUtils.java


示例19: FieldsContextImpl

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public FieldsContextImpl(FieldAssembler<?> fieldAssembler) {
	super();
	this.fieldAssembler = fieldAssembler;
}
 
开发者ID:Talend,项目名称:avro-schema-editor,代码行数:5,代码来源:FieldsContextImpl.java


示例20: fields

import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
@Override
public FieldStartContext fields() {
	FieldAssembler<?> fieldAssembler = recordBuilder.fields();
	return new FieldsContextImpl(fieldAssembler);
}
 
开发者ID:Talend,项目名称:avro-schema-editor,代码行数:6,代码来源:RecordContextImpl.java



注:本文中的org.apache.avro.SchemaBuilder.FieldAssembler类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java GetClusterNodesRequestPBImpl类代码示例发布时间:2022-05-22
下一篇:
Java ContainerEvent类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap