• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java LogicalTypes类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.avro.LogicalTypes的典型用法代码示例。如果您正苦于以下问题:Java LogicalTypes类的具体用法?Java LogicalTypes怎么用?Java LogicalTypes使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



LogicalTypes类属于org.apache.avro包,在下文中一共展示了LogicalTypes类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: toTypeSchemaDateNullable

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void toTypeSchemaDateNullable() throws Exception {
  Schema schema = AvroUtils.typeFor(DataTypes.DateType);

  assertEquals("Invalid type", Schema.Type.UNION, schema.getType());
  assertEquals("Invalid union size", 2, schema.getTypes().size());

  for (Schema s : schema.getTypes()) {
    assertThat("Invalid union types", s.getType(), anyOf(is(Schema.Type.INT), is(Schema.Type.NULL)));
    if (s.getType().equals(Schema.Type.INT)) {
      assertEquals("Invalid LogicalType", LogicalTypes.date(), s.getLogicalType());
    }
  }

  //System.out.println(schema.toString(true));
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:17,代码来源:TestAvroUtils.java


示例2: toTypeSchemaTimestampNullable

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void toTypeSchemaTimestampNullable() throws Exception {
  Schema schema = AvroUtils.typeFor(DataTypes.TimestampType);

  assertEquals("Invalid type", Schema.Type.UNION, schema.getType());
  assertEquals("Invalid union size", 2, schema.getTypes().size());

  for (Schema s : schema.getTypes()) {
    assertThat("Invalid union types", s.getType(), anyOf(is(Schema.Type.LONG), is(Schema.Type.NULL)));
    if (s.getType().equals(Schema.Type.LONG)) {
      assertEquals("Invalid LogicalType", LogicalTypes.timestampMillis(), s.getLogicalType());
    }
  }

  //System.out.println(schema.toString(true));
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:17,代码来源:TestAvroUtils.java


示例3: toTypeSchemaDecimalNullable

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void toTypeSchemaDecimalNullable() throws Exception {
  // Defaults to precision 10, scale 0
  Schema schema = AvroUtils.typeFor(DataTypes.createDecimalType());

  assertEquals("Invalid type", Schema.Type.UNION, schema.getType());
  assertEquals("Invalid union size", 2, schema.getTypes().size());

  for (Schema s : schema.getTypes()) {
    assertThat("Invalid union types", s.getType(), anyOf(is(Schema.Type.BYTES), is(Schema.Type.NULL)));
    if (s.getType().equals(Schema.Type.BYTES)) {
      assertEquals("Invalid LogicalType", LogicalTypes.decimal(10, 0), s.getLogicalType());
    }
  }

  //System.out.println(schema.toString(true));
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:18,代码来源:TestAvroUtils.java


示例4: getSchema

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
/**
 * Generate a schema from output type and format
 *
 * @param outputType
 * @param outputFormat
 * @return
 */
public static Schema getSchema(TypeConverterProperties.TypeConverterOutputTypes outputType, String outputFormat) {
    Schema result = Schema.create(outputType.getTargetType());
    switch (outputType) {
        case Decimal:
            result = LogicalTypes.decimal(20, 4).addToSchema(result);
            break;
        case Date:
            result = LogicalTypes.date().addToSchema(result);
            break;
        case Time:
            result = LogicalTypes.timeMillis().addToSchema(result);
            break;
        case DateTime:
            result = LogicalTypes.timestampMillis().addToSchema(result);
            break;
    }
    return result;
}
 
开发者ID:Talend,项目名称:components,代码行数:26,代码来源:TypeConverterUtils.java


示例5: convertLogicalType

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
private OriginalType convertLogicalType(LogicalType logicalType) {
  if (logicalType == null) {
    return null;
  } else if (logicalType instanceof LogicalTypes.Decimal) {
    return OriginalType.DECIMAL;
  } else if (logicalType instanceof LogicalTypes.Date) {
    return OriginalType.DATE;
  } else if (logicalType instanceof LogicalTypes.TimeMillis) {
    return OriginalType.TIME_MILLIS;
  } else if (logicalType instanceof LogicalTypes.TimeMicros) {
    return OriginalType.TIME_MICROS;
  } else if (logicalType instanceof LogicalTypes.TimestampMillis) {
    return OriginalType.TIMESTAMP_MILLIS;
  } else if (logicalType instanceof LogicalTypes.TimestampMicros) {
    return OriginalType.TIMESTAMP_MICROS;
  }
  return null;
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:19,代码来源:AvroSchemaConverter.java


示例6: convertOriginalType

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
private LogicalType convertOriginalType(OriginalType annotation, DecimalMetadata meta) {
  if (annotation == null) {
    return null;
  }
  switch (annotation) {
    case DECIMAL:
      return LogicalTypes.decimal(meta.getPrecision(), meta.getScale());
    case DATE:
      return LogicalTypes.date();
    case TIME_MILLIS:
      return LogicalTypes.timeMillis();
    case TIME_MICROS:
      return LogicalTypes.timeMicros();
    case TIMESTAMP_MILLIS:
      return LogicalTypes.timestampMillis();
    case TIMESTAMP_MICROS:
      return LogicalTypes.timestampMicros();
  }
  return null;
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:21,代码来源:AvroSchemaConverter.java


示例7: testDecimalBytes

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testDecimalBytes() throws IOException {
  Schema schema = REFLECT.getSchema(DecimalRecordBytes.class);
  Assert.assertEquals("Should have the correct record name",
      "org.apache.parquet.avro.TestReflectLogicalTypes$",
      schema.getNamespace());
  Assert.assertEquals("Should have the correct record name",
      "DecimalRecordBytes",
      schema.getName());
  Assert.assertEquals("Should have the correct logical type",
      LogicalTypes.decimal(9, 2),
      LogicalTypes.fromSchema(schema.getField("decimal").schema()));

  DecimalRecordBytes record = new DecimalRecordBytes();
  record.decimal = new BigDecimal("3.14");

  File test = write(REFLECT, schema, record);
  Assert.assertEquals("Should match the decimal after round trip",
      Arrays.asList(record),
      read(REFLECT, schema, test));
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:22,代码来源:TestReflectLogicalTypes.java


示例8: testDecimalFixed

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testDecimalFixed() throws IOException {
  Schema schema = REFLECT.getSchema(DecimalRecordFixed.class);
  Assert.assertEquals("Should have the correct record name",
      "org.apache.parquet.avro.TestReflectLogicalTypes$",
      schema.getNamespace());
  Assert.assertEquals("Should have the correct record name",
      "DecimalRecordFixed",
      schema.getName());
  Assert.assertEquals("Should have the correct logical type",
      LogicalTypes.decimal(9, 2),
      LogicalTypes.fromSchema(schema.getField("decimal").schema()));

  DecimalRecordFixed record = new DecimalRecordFixed();
  record.decimal = new BigDecimal("3.14");

  File test = write(REFLECT, schema, record);
  Assert.assertEquals("Should match the decimal after round trip",
      Arrays.asList(record),
      read(REFLECT, schema, test));
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:22,代码来源:TestReflectLogicalTypes.java


示例9: testWriteUUIDMissingLogicalType

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test(expected = ClassCastException.class)
public void testWriteUUIDMissingLogicalType() throws IOException {
  Schema uuidSchema = SchemaBuilder.record(RecordWithUUID.class.getName())
      .fields().requiredString("uuid").endRecord();
  LogicalTypes.uuid().addToSchema(uuidSchema.getField("uuid").schema());

  UUID u1 = UUID.randomUUID();
  UUID u2 = UUID.randomUUID();

  RecordWithUUID r1 = new RecordWithUUID();
  r1.uuid = u1;
  RecordWithUUID r2 = new RecordWithUUID();
  r2.uuid = u2;

  // write without using REFLECT, which has the logical type
  File test = write(uuidSchema, r1, r2);

  // verify that the field's type overrides the logical type
  Schema uuidStringSchema = SchemaBuilder
      .record(RecordWithStringUUID.class.getName())
      .fields().requiredString("uuid").endRecord();

  // this fails with an AppendWriteException wrapping ClassCastException
  // because the UUID isn't converted to a CharSequence expected internally
  read(ReflectData.get(), uuidStringSchema, test);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:27,代码来源:TestReflectLogicalTypes.java


示例10: testReadUUIDArray

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testReadUUIDArray() throws IOException {
  Schema uuidArraySchema = SchemaBuilder.record(RecordWithUUIDArray.class.getName())
      .fields()
      .name("uuids").type().array().items().stringType().noDefault()
      .endRecord();
  LogicalTypes.uuid().addToSchema(
      uuidArraySchema.getField("uuids").schema().getElementType());

  UUID u1 = UUID.randomUUID();
  UUID u2 = UUID.randomUUID();

  GenericRecord r = new GenericData.Record(uuidArraySchema);
  r.put("uuids", Arrays.asList(u1.toString(), u2.toString()));

  RecordWithUUIDArray expected = new RecordWithUUIDArray();
  expected.uuids = new UUID[] {u1, u2};

  File test = write(uuidArraySchema, r);

  Assert.assertEquals("Should convert Strings to UUIDs",
      expected,
      read(REFLECT, uuidArraySchema, test).get(0));
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:25,代码来源:TestReflectLogicalTypes.java


示例11: testReadUUIDList

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testReadUUIDList() throws IOException {
  Schema uuidListSchema = SchemaBuilder.record(RecordWithUUIDList.class.getName())
      .fields()
      .name("uuids").type().array().items().stringType().noDefault()
      .endRecord();
  uuidListSchema.getField("uuids").schema().addProp(
      SpecificData.CLASS_PROP, List.class.getName());
  LogicalTypes.uuid().addToSchema(
      uuidListSchema.getField("uuids").schema().getElementType());

  UUID u1 = UUID.randomUUID();
  UUID u2 = UUID.randomUUID();

  GenericRecord r = new GenericData.Record(uuidListSchema);
  r.put("uuids", Arrays.asList(u1.toString(), u2.toString()));

  RecordWithUUIDList expected = new RecordWithUUIDList();
  expected.uuids = Arrays.asList(u1, u2);

  File test = write(uuidListSchema, r);

  Assert.assertEquals("Should convert Strings to UUIDs",
      expected, read(REFLECT, uuidListSchema, test).get(0));
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:26,代码来源:TestReflectLogicalTypes.java


示例12: testWriteUUIDReadStringSchema

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testWriteUUIDReadStringSchema() throws IOException {
  Schema uuidSchema = record("R",
      field("uuid", LogicalTypes.uuid().addToSchema(Schema.create(STRING))));
  GenericRecord u1 = instance(uuidSchema, "uuid", UUID.randomUUID());
  GenericRecord u2 = instance(uuidSchema, "uuid", UUID.randomUUID());

  Schema stringUuidSchema = Schema.create(STRING);
  stringUuidSchema.addProp(GenericData.STRING_PROP, "String");
  Schema stringSchema = record("R", field("uuid", stringUuidSchema));
  GenericRecord s1 = instance(stringSchema, "uuid", u1.get("uuid").toString());
  GenericRecord s2 = instance(stringSchema, "uuid", u2.get("uuid").toString());

  File test = write(GENERIC, uuidSchema, u1, u2);
  Assert.assertEquals("Should read UUIDs as Strings",
      Arrays.asList(s1, s2), read(GENERIC, stringSchema, test));
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:18,代码来源:TestGenericLogicalTypes.java


示例13: testWriteNullableUUID

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testWriteNullableUUID() throws IOException {
  Schema nullableUuidSchema = record("R",
      optionalField("uuid", LogicalTypes.uuid().addToSchema(Schema.create(STRING))));
  GenericRecord u1 = instance(nullableUuidSchema, "uuid", UUID.randomUUID());
  GenericRecord u2 = instance(nullableUuidSchema, "uuid", UUID.randomUUID());

  Schema stringUuidSchema = Schema.create(STRING);
  stringUuidSchema.addProp(GenericData.STRING_PROP, "String");
  Schema nullableStringSchema = record("R", optionalField("uuid", stringUuidSchema));
  GenericRecord s1 = instance(nullableStringSchema, "uuid", u1.get("uuid").toString());
  GenericRecord s2 = instance(nullableStringSchema, "uuid", u2.get("uuid").toString());

  File test = write(GENERIC, nullableUuidSchema, u1, u2);
  Assert.assertEquals("Should read UUIDs as Strings",
      Arrays.asList(s1, s2), read(GENERIC, nullableStringSchema, test));
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:18,代码来源:TestGenericLogicalTypes.java


示例14: typeFor

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
private static Schema typeFor(DataType dataType, boolean isOptional, int recordCount) {
  LOG.trace("Converting {} to Avro, optional[{}]", dataType, isOptional);

  Schema typeSchema;
  SchemaBuilder.BaseTypeBuilder<Schema> typeBuilder = SchemaBuilder.builder();

  switch (dataType.typeName()) {
    case "binary":
      // bytes
      typeSchema = typeBuilder.bytesType();
      break;
    case "boolean":
      typeSchema = typeBuilder.booleanType();
      break;
    case "date":
      // int (logical)
      typeSchema = LogicalTypes.date().addToSchema(typeBuilder.intType());
      break;
    case "timestamp":
      // long (logical)
      typeSchema = LogicalTypes.timestampMillis().addToSchema(typeBuilder.longType());
      break;
    case "double":
      typeSchema = typeBuilder.doubleType();
      break;
    case "float":
      typeSchema = typeBuilder.floatType();
      break;
    case "integer":
    case "byte":
    case "short":
      typeSchema = typeBuilder.intType();
      break;
    case "long":
      typeSchema = typeBuilder.longType();
      break;
    case "null":
      typeSchema = typeBuilder.nullType();
      break;
    case "string":
      typeSchema = typeBuilder.stringType();
      break;
    case "array":
      ArrayType arrayType = (ArrayType) dataType;
      typeSchema = typeBuilder.array().items(typeFor(arrayType.elementType(), arrayType.containsNull(), recordCount));
      break;
    case "map":
      MapType mapType = (MapType) dataType;
      // Keys must be strings: mapType.keyType()
      typeSchema = typeBuilder.map().values(typeFor(mapType.valueType(), mapType.valueContainsNull(), recordCount));
      break;
    case "struct":
      StructType structType = (StructType) dataType;
      // Nested "anonymous" records
      typeSchema = schemaFor(structType, null, null, recordCount);
      break;
    default:
      if (dataType.typeName().startsWith("decimal")) {
        // byte (logical)
        DecimalType decimalType = (DecimalType) dataType;
        typeSchema = LogicalTypes.decimal(decimalType.precision(), decimalType.scale()).addToSchema(typeBuilder.bytesType());
      } else {
        throw new RuntimeException(String.format("DataType[%s] - DataType unrecognized or not yet implemented",
            dataType));
      }
  }

  if (isOptional && !typeSchema.getType().equals(Schema.Type.NULL)) {
    return SchemaBuilder.builder().nullable().type(typeSchema);
  }

  return typeSchema;
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:74,代码来源:AvroUtils.java


示例15: toTypeSchemaDateNotNullable

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void toTypeSchemaDateNotNullable() throws Exception {
  Schema schema = AvroUtils.typeFor(DataTypes.DateType, false);

  assertEquals("Invalid type", Schema.Type.INT, schema.getType());
  assertEquals("Invalid LogicalType", LogicalTypes.date(), schema.getLogicalType());

  //System.out.println(schema.toString(true));
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:10,代码来源:TestAvroUtils.java


示例16: toTypeSchemaTimestampNotNullable

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void toTypeSchemaTimestampNotNullable() throws Exception {
  Schema schema = AvroUtils.typeFor(DataTypes.TimestampType, false);

  assertEquals("Invalid type", Schema.Type.LONG, schema.getType());
  assertEquals("Invalid LogicalType", LogicalTypes.timestampMillis(), schema.getLogicalType());

  //System.out.println(schema.toString(true));
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:10,代码来源:TestAvroUtils.java


示例17: toTypeSchemaDecimalNotNullable

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void toTypeSchemaDecimalNotNullable() throws Exception {
  Schema schema = AvroUtils.typeFor(DataTypes.createDecimalType(), false);

  assertEquals("Invalid type", Schema.Type.BYTES, schema.getType());
  assertEquals("Invalid LogicalType", LogicalTypes.decimal(10, 0), schema.getLogicalType());

  //System.out.println(schema.toString(true));
}
 
开发者ID:cloudera-labs,项目名称:envelope,代码行数:10,代码来源:TestAvroUtils.java


示例18: getConverter

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Override
public JDBCConverter getConverter(final Field f) {
    final Schema basicSchema = AvroUtils.unwrapIfNullable(f.schema());

    return null == basicSchema.getLogicalType() ? super.getConverter(f) : new JDBCConverter() {

        @Override
        public Object convertToAvro(ResultSet value) {
            int index = f.pos() + 1;
            try {
                if (basicSchema.getLogicalType() == LogicalTypes.date()) {
                    // Snowflake stores the value as the number of days. So it is possible to retrieve that as an
                    // int value instead of converting it to Date first and then to days from milliseconds. If we
                    // convert it to date, Snowflake jdbc shifts the time to 00:00 in current timezone.
                    return value.getInt(index);
                } else if (basicSchema.getLogicalType() == LogicalTypes.timeMillis()) {
                    java.sql.Time time = value.getTime(index);
                    return (time != null) ? (int) time.getTime() : null;
                } else {
                    java.sql.Timestamp timestamp = value.getTimestamp(index);
                    return (timestamp != null) ? timestamp.getTime() : null;
                }
            } catch (SQLException e) {
                throw new ComponentException(e);
            }
        }
    };
}
 
开发者ID:Talend,项目名称:components,代码行数:29,代码来源:SnowflakeAvroRegistry.java


示例19: testDecimalBytesType

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testDecimalBytesType() throws Exception {
  Schema schema = Schema.createRecord("myrecord", null, null, false);
  Schema decimal = LogicalTypes.decimal(9, 2).addToSchema(
      Schema.create(Schema.Type.BYTES));
  schema.setFields(Collections.singletonList(
      new Schema.Field("dec", decimal, null, null)));

  testRoundTripConversion(schema,
      "message myrecord {\n" +
          "  required binary dec (DECIMAL(9,2));\n" +
          "}\n");
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:14,代码来源:TestAvroSchemaConverter.java


示例20: testDecimalFixedType

import org.apache.avro.LogicalTypes; //导入依赖的package包/类
@Test
public void testDecimalFixedType() throws Exception {
  Schema schema = Schema.createRecord("myrecord", null, null, false);
  Schema decimal = LogicalTypes.decimal(9, 2).addToSchema(
      Schema.createFixed("dec", null, null, 8));
  schema.setFields(Collections.singletonList(
      new Schema.Field("dec", decimal, null, null)));

  testRoundTripConversion(schema,
      "message myrecord {\n" +
          "  required fixed_len_byte_array(8) dec (DECIMAL(9,2));\n" +
          "}\n");
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:14,代码来源:TestAvroSchemaConverter.java



注:本文中的org.apache.avro.LogicalTypes类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java ItemStackDataLists类代码示例发布时间:2022-05-22
下一篇:
Java Type类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap