本文整理汇总了Java中org.apache.avro.SchemaBuilder.FieldAssembler类的典型用法代码示例。如果您正苦于以下问题：Java FieldAssembler类的具体用法？Java FieldAssembler怎么用？Java FieldAssembler使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
FieldAssembler类属于org.apache.avro.SchemaBuilder包，在下文中一共展示了FieldAssembler类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: createSchemaFor
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
* Generate a schema with given schema name and has one field of STRING type
* for each dimension and one field of LONG type for each metric
* @param schemaName
* @param dimensions
* @param metrics
* @return
*/
public static Schema createSchemaFor(String schemaName, String[] dimensions,
String[] metrics) {
FieldAssembler<Schema> fields;
RecordBuilder<Schema> record = SchemaBuilder.record(schemaName);
fields = record.namespace("com.linkedin.thirdeye.join").fields();
for (String dimension : dimensions) {
fields = fields.name(dimension).type().nullable().stringType()
.noDefault();
}
for (String metric : metrics) {
fields = fields.name(metric).type().nullable().longType().longDefault(0);
}
Schema schema = fields.endRecord();
return schema;
}
开发者ID:Hanmourang，项目名称:Pinot，代码行数:24，代码来源:AvroTestUtil.java
示例2: initDefaultSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema initDefaultSchema(Schema designSchema) {
AvroRegistry avroReg = new AvroRegistry();
FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
for (SplunkJSONEventField metadataField : SplunkJSONEventField.getMetadataFields()) {
Schema base = avroReg.getConverter(metadataField.getDataType()).getSchema();
FieldBuilder<Schema> fieldBuilder = record.name(metadataField.getName());
if (metadataField.getName().equals(SplunkJSONEventField.TIME.getName())) {
String datePattern;
Field designField = designSchema.getField(metadataField.getName());
if (designField != null) {
datePattern = designField.getProp(SchemaConstants.TALEND_COLUMN_PATTERN);
} else {
datePattern = designSchema.getProp(SchemaConstants.TALEND_COLUMN_PATTERN);
}
if (datePattern == null || datePattern.isEmpty()) {
datePattern = "dd-MM-yyyy";
}
fieldBuilder.prop(SchemaConstants.TALEND_COLUMN_PATTERN, datePattern);
}
fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
}
Schema defaultSchema = record.endRecord();
return defaultSchema;
}
开发者ID:Talend，项目名称:components，代码行数:25，代码来源:TSplunkEventCollectorWriter.java
示例3: createTestSchema5
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema5() {
FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();
Schema schema = AvroUtils._int();
schema = wrap(schema);
builder = builder.name("ID").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "ID").type(schema).noDefault();
schema = AvroUtils._string();
schema = wrap(schema);
builder = builder.name("NAME").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "NAME").type(schema).noDefault();
schema = AvroUtils._string();// TODO : fix it as should be object type
schema = wrap(schema);
builder = builder.name("RESULTSET").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "RESULTSET").type(schema)
.noDefault();
return builder.endRecord();
}
开发者ID:Talend，项目名称:components，代码行数:19，代码来源:DBTestUtils.java
示例4: getGranularityBasedSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
@SuppressWarnings("fallthrough")
private Schema getGranularityBasedSchema() {
FieldAssembler<Schema> assembler =
SchemaBuilder.record("GenericRecordTimePartition").namespace("gobblin.writer.partitioner").fields();
// Construct the fields in reverse order
if (!Strings.isNullOrEmpty(this.writerPartitionSuffix)) {
assembler = assembler.name(SUFFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
}
assembler = assembler.name(this.granularity.toString()).type(Schema.create(Schema.Type.STRING)).noDefault();
if (!Strings.isNullOrEmpty(this.writerPartitionPrefix)) {
assembler = assembler.name(PREFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
}
Schema schema = assembler.endRecord();
Collections.reverse(schema.getFields());
return schema;
}
开发者ID:apache，项目名称:incubator-gobblin，代码行数:20，代码来源:TimeBasedWriterPartitioner.java
示例5: schemaFor
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema schemaFor(List<String> fieldNames, List<String> fieldTypes) {
FieldAssembler<Schema> assembler = SchemaBuilder.record("t").fields();
for (int i = 0; i < fieldNames.size(); i++) {
String fieldName = fieldNames.get(i);
String fieldType = fieldTypes.get(i);
switch (fieldType) {
case "string":
assembler = assembler.optionalString(fieldName);
break;
case "float":
assembler = assembler.optionalFloat(fieldName);
break;
case "double":
assembler = assembler.optionalDouble(fieldName);
break;
case "int":
assembler = assembler.optionalInt(fieldName);
break;
case "long":
assembler = assembler.optionalLong(fieldName);
break;
case "boolean":
assembler = assembler.optionalBoolean(fieldName);
break;
case "binary":
assembler = assembler.optionalBytes(fieldName);
break;
default:
throw new RuntimeException("Unsupported provided field type: " + fieldType);
}
}
return assembler.endRecord();
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:37，代码来源:AvroTranslator.java
示例6: inferSchemaRecord
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema inferSchemaRecord(IndexedRecord in) {
FieldAssembler<Schema> builder = SchemaBuilder.builder().record(in.getSchema().getName()).fields();
for (Schema.Field field : in.getSchema().getFields()) {
Schema fieldSchema = inferSchema(field);
Object fieldDefault = field.defaultVal();
if (null == fieldDefault) {
builder = builder.name(field.name()).type(fieldSchema).noDefault();
} else {
builder = builder.name(field.name()).type(fieldSchema).withDefault(fieldDefault);
}
}
return builder.endRecord();
}
开发者ID:Talend，项目名称:components，代码行数:14，代码来源:GenericAvroRegistry.java
示例7: createTestSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema() {
FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();
Schema schema = AvroUtils._int();
schema = wrap(schema);
builder = builder.name("ID").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "ID").type(schema).noDefault();
schema = AvroUtils._string();
schema = wrap(schema);
builder = builder.name("NAME").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "NAME").type(schema).noDefault();
return builder.endRecord();
}
开发者ID:Talend，项目名称:components，代码行数:14，代码来源:PropertiesPreparer.java
示例8: setupDefaultSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private void setupDefaultSchema() {
AvroRegistry avroReg = new AvroRegistry();
FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
for (SplunkJSONEventField metadataField : SplunkJSONEventField.getMetadataFields()) {
Schema base = avroReg.getConverter(metadataField.getDataType()).getSchema();
FieldBuilder<Schema> fieldBuilder = record.name(metadataField.getName());
if (metadataField.getName().equals(SplunkJSONEventField.TIME.getName())) {
fieldBuilder.prop(SchemaConstants.TALEND_COLUMN_PATTERN, "dd-MM-yyyy");
}
fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
}
Schema defaultSchema = record.endRecord();
schema.schema.setValue(defaultSchema);
}
开发者ID:Talend，项目名称:components，代码行数:15，代码来源:TSplunkEventCollectorProperties.java
示例9: createSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
* Create default static schema for indexed record.
*/
private Schema createSchema() {
AvroRegistry avroReg = new AvroRegistry();
FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
addField(record, "FieldString", String.class, avroReg);
addField(record, "FieldInt", Integer.class, avroReg);
Schema defaultSchema = record.endRecord();
return defaultSchema;
}
开发者ID:Talend，项目名称:components，代码行数:12，代码来源:TSplunkEventCollectorWriterTestIT.java
示例10: createSchemaWithTimeAsString
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
* create schema for indexed record with Date presented as String.
*/
private Schema createSchemaWithTimeAsString() {
AvroRegistry avroReg = new AvroRegistry();
FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
addField(record, "FieldString", String.class, avroReg);
addField(record, "FieldInt", Integer.class, avroReg);
addField(record, "Description", String.class, avroReg);
addField(record, "time", String.class, avroReg);
Schema defaultSchema = record.endRecord();
return defaultSchema;
}
开发者ID:Talend，项目名称:components，代码行数:14，代码来源:TSplunkEventCollectorWriterTestIT.java
示例11: addField
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
* add field to Schema.
*/
private FieldAssembler<Schema> addField(FieldAssembler<Schema> record, String name, Class<?> type, AvroRegistry avroReg) {
Schema base = avroReg.getConverter(type).getSchema();
FieldBuilder<Schema> fieldBuilder = record.name(name);
fieldBuilder.type(AvroUtils.wrapAsNullable(base)).noDefault();
return record;
}
开发者ID:Talend，项目名称:components，代码行数:10，代码来源:TSplunkEventCollectorWriterTestIT.java
示例12: createDynamicSchemaWithDatePattern
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
* Create dynamic schema with date pattern in dynamic column. Required to test parsing of Date presented as String.
*/
private Schema createDynamicSchemaWithDatePattern() {
FieldAssembler<Schema> record = SchemaBuilder.record("Main").fields();
Schema defaultSchema = record.endRecord();
AvroUtils.setIncludeAllFields(defaultSchema, true);
AvroUtils.setProperty(defaultSchema, SchemaConstants.TALEND_COLUMN_PATTERN, DATE_PATTERN);
return defaultSchema;
}
开发者ID:Talend，项目名称:components，代码行数:11，代码来源:TSplunkEventCollectorWriterTestIT.java
示例13: getMakeRowSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public Schema getMakeRowSchema(boolean isDynamic) {
FieldAssembler<Schema> fa = SchemaBuilder.builder().record("MakeRowRecord").fields() //
.name("Name").type().nullable().stringType().noDefault() //
.name("ShippingStreet").type().nullable().stringType().noDefault() //
.name("ShippingPostalCode").type().nullable().intType().noDefault() //
.name("BillingStreet").type().nullable().stringType().noDefault() //
.name("BillingState").type().nullable().stringType().noDefault() //
.name("BillingPostalCode").type().nullable().stringType().noDefault();
if (isDynamic) {
fa = fa.name("ShippingState").type().nullable().stringType().noDefault();
}
return fa.endRecord();
}
开发者ID:Talend，项目名称:components，代码行数:15，代码来源:SalesforceTestBase.java
示例14: createTestSchema2
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema2() {
FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();
Schema schema = AvroUtils._int();
schema = wrap(schema);
builder = builder.name("ID").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "ID")
.prop(SchemaConstants.TALEND_COLUMN_IS_KEY, "true").type(schema).noDefault();
schema = AvroUtils._string();
schema = wrap(schema);
builder = builder.name("NAME").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "NAME").type(schema).noDefault();
return builder.endRecord();
}
开发者ID:Talend，项目名称:components，代码行数:15，代码来源:DBTestUtils.java
示例15: createTestSchema4
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public static Schema createTestSchema4() {
FieldAssembler<Schema> builder = SchemaBuilder.builder().record("TEST").fields();
Schema schema = AvroUtils._string();// TODO : fix it as should be object type
schema = wrap(schema);
builder = builder.name("RESULTSET").prop(SchemaConstants.TALEND_COLUMN_DB_COLUMN_NAME, "RESULTSET").type(schema)
.noDefault();
return builder.endRecord();
}
开发者ID:Talend，项目名称:components，代码行数:11，代码来源:DBTestUtils.java
示例16: addField
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private FieldAssembler<Schema> addField(
FieldAssembler<Schema> fieldAssembler, PredicateObjectMap pom,
boolean isMap, Schema targetSchema, String predicateShortHand) {
try{
if(isMap)
{
if(targetSchema == null)
{
fieldAssembler = fieldAssembler.name(pom.getPredicate().getId().replaceAll("[^\\w]", "_")).type().unionOf().map().values().unionOf().map().values().stringType().and().stringType().and().nullType().endUnion().and().nullType().endUnion().noDefault();
}
else
{
fieldAssembler = fieldAssembler.name(pom.getPredicate().getId().replaceAll("[^\\w]", "_")).type().unionOf().map().values().unionOf().map().values(targetSchema).and().type(targetSchema).and().nullType().endUnion().and().nullType().endUnion().noDefault();
}
}
else {
if(targetSchema == null)
{
fieldAssembler = fieldAssembler.name(predicateShortHand).type().unionOf().array().items().stringType().and().stringType().and().nullType().endUnion().noDefault();
}
else
{
fieldAssembler = fieldAssembler.name(predicateShortHand).type().unionOf().array().items(targetSchema).and().type(targetSchema).and().nullType().endUnion().noDefault();
}
}
}
catch(Exception e)
{
LOG.error("Unable to add field: " + predicateShortHand + " for " + pom.getTriplesMap().getSubject().getTemplate().toString(), e);
}
return fieldAssembler;
}
开发者ID:therelaxist，项目名称:spring-usc，代码行数:35，代码来源:AvroKR2RMLRDFWriter.java
示例17: getDateTimeFormatBasedSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
private Schema getDateTimeFormatBasedSchema() {
FieldAssembler<Schema> assembler =
SchemaBuilder.record("GenericRecordTimePartition").namespace("gobblin.writer.partitioner").fields();
if (!Strings.isNullOrEmpty(this.writerPartitionPrefix)) {
assembler = assembler.name(PREFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
}
assembler = assembler.name(PARTITIONED_PATH).type(Schema.create(Schema.Type.STRING)).noDefault();
if (!Strings.isNullOrEmpty(this.writerPartitionSuffix)) {
assembler = assembler.name(SUFFIX).type(Schema.create(Schema.Type.STRING)).noDefault();
}
return assembler.endRecord();
}
开发者ID:apache，项目名称:incubator-gobblin，代码行数:15，代码来源:TimeBasedWriterPartitioner.java
示例18: constructAvroSchemaFromPinotSchema
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
/**
* Constructs an avro schema from a pinot schema
* @param schema
* @return
*/
public static Schema constructAvroSchemaFromPinotSchema(com.linkedin.pinot.common.data.Schema schema) {
Schema avroSchema = null;
RecordBuilder<Schema> recordBuilder = SchemaBuilder.record("record");
FieldAssembler<Schema> fieldAssembler = recordBuilder.fields();
for (FieldSpec fieldSpec : schema.getAllFieldSpecs()) {
String fieldName = fieldSpec.getName();
DataType dataType = fieldSpec.getDataType();
BaseFieldTypeBuilder<Schema> baseFieldTypeBuilder = fieldAssembler.name(fieldName).type().nullable();
switch (dataType) {
case BOOLEAN:
fieldAssembler = baseFieldTypeBuilder.booleanType().noDefault();
break;
case DOUBLE:
fieldAssembler = baseFieldTypeBuilder.doubleType().noDefault();
break;
case FLOAT:
fieldAssembler = baseFieldTypeBuilder.floatType().noDefault();
break;
case INT:
fieldAssembler = baseFieldTypeBuilder.intType().noDefault();
break;
case LONG:
fieldAssembler = baseFieldTypeBuilder.longType().noDefault();
break;
case STRING:
fieldAssembler = baseFieldTypeBuilder.stringType().noDefault();
break;
default:
break;
}
}
avroSchema = fieldAssembler.endRecord();
LOGGER.info("Avro Schema {}", avroSchema.toString(true));
return avroSchema;
}
开发者ID:linkedin，项目名称:pinot，代码行数:45，代码来源:ThirdeyeAvroUtils.java
示例19: FieldsContextImpl
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
public FieldsContextImpl(FieldAssembler<?> fieldAssembler) {
super();
this.fieldAssembler = fieldAssembler;
}
开发者ID:Talend，项目名称:avro-schema-editor，代码行数:5，代码来源:FieldsContextImpl.java
示例20: fields
import org.apache.avro.SchemaBuilder.FieldAssembler; //导入依赖的package包/类
@Override
public FieldStartContext fields() {
FieldAssembler<?> fieldAssembler = recordBuilder.fields();
return new FieldsContextImpl(fieldAssembler);
}
开发者ID:Talend，项目名称:avro-schema-editor，代码行数:6，代码来源:RecordContextImpl.java
注：本文中的org.apache.avro.SchemaBuilder.FieldAssembler类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论