• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java RecordConsumer类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.parquet.io.api.RecordConsumer的典型用法代码示例。如果您正苦于以下问题:Java RecordConsumer类的具体用法?Java RecordConsumer怎么用?Java RecordConsumer使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



RecordConsumer类属于org.apache.parquet.io.api包,在下文中一共展示了RecordConsumer类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testEmptyField

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testEmptyField() {
  MemPageStore memPageStore = new MemPageStore(1);
  ColumnWriteStoreV1 columns = newColumnWriteStore(memPageStore);
  MessageColumnIO columnIO = new ColumnIOFactory(true).getColumnIO(schema);
  final RecordConsumer recordWriter = columnIO.getRecordWriter(columns);
  recordWriter.startMessage();
  recordWriter.startField("DocId", 0);
  recordWriter.addLong(0);
  recordWriter.endField("DocId", 0);
  recordWriter.startField("Links", 1);
  try {
    recordWriter.endField("Links", 1);
    Assert.fail("expected exception because of empty field");
  } catch (ParquetEncodingException e) {
    Assert.assertEquals("empty fields are illegal, the field should be ommited completely instead", e.getMessage());
  }
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:19,代码来源:TestColumnIO.java


示例2: writeTestRecords

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
private MemPageStore writeTestRecords(MessageColumnIO columnIO, int number) {
  MemPageStore memPageStore = new MemPageStore(number * 2);
  ColumnWriteStoreV1 columns = new ColumnWriteStoreV1(
      memPageStore,
      ParquetProperties.builder()
          .withPageSize(800)
          .withDictionaryEncoding(false)
          .build());

  RecordConsumer recordWriter = columnIO.getRecordWriter(columns);
  GroupWriter groupWriter = new GroupWriter(recordWriter, schema);
  for ( int i = 0; i < number; i++ ) {
    groupWriter.write(r1);
    groupWriter.write(r2);
  }
  recordWriter.flush();
  columns.flush();
  return memPageStore;
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:20,代码来源:TestFiltered.java


示例3: testUnannotatedListOfPrimitives

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
@Ignore("Not yet supported")
public void testUnannotatedListOfPrimitives() throws Exception {
  Path test = writeDirect(
      "message UnannotatedListOfPrimitives {" +
          "  repeated int32 list_of_ints;" +
          "}",
      new DirectWriter() {
        @Override
        public void write(RecordConsumer rc) {
          rc.startMessage();
          rc.startField("list_of_ints", 0);

          rc.addInteger(34);
          rc.addInteger(35);
          rc.addInteger(36);

          rc.endField("list_of_ints", 0);
          rc.endMessage();
        }
      });
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:23,代码来源:TestArrayCompatibility.java


示例4: validateSameTupleAsEB

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
/**
 * <ul> steps:
 * <li>Writes using the thrift mapping
 * <li>Reads using the pig mapping
 * <li>Use Elephant bird to convert from thrift to pig
 * <li>Check that both transformations give the same result
 * @param o the object to convert
 * @throws TException
 */
public static <T extends TBase<?,?>> void validateSameTupleAsEB(T o) throws TException {
  final ThriftSchemaConverter thriftSchemaConverter = new ThriftSchemaConverter();
  @SuppressWarnings("unchecked")
  final Class<T> class1 = (Class<T>) o.getClass();
  final MessageType schema = thriftSchemaConverter.convert(class1);

  final StructType structType = ThriftSchemaConverter.toStructType(class1);
  final ThriftToPig<T> thriftToPig = new ThriftToPig<T>(class1);
  final Schema pigSchema = thriftToPig.toSchema();
  final TupleRecordMaterializer tupleRecordConverter = new TupleRecordMaterializer(schema, pigSchema, true);
  RecordConsumer recordConsumer = new ConverterConsumer(tupleRecordConverter.getRootConverter(), schema);
  final MessageColumnIO columnIO = new ColumnIOFactory().getColumnIO(schema);
  ParquetWriteProtocol p = new ParquetWriteProtocol(new RecordConsumerLoggingWrapper(recordConsumer), columnIO, structType);
  o.write(p);
  final Tuple t = tupleRecordConverter.getCurrentRecord();
  final Tuple expected = thriftToPig.getPigTuple(o);
  assertEquals(expected.toString(), t.toString());
  final MessageType filtered = new PigSchemaConverter().filter(schema, pigSchema);
  assertEquals(schema.toString(), filtered.toString());
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:30,代码来源:TestThriftToPigCompatibility.java


示例5: testSimplestMessage

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testSimplestMessage() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProtobuf.InnerMessage.class, readConsumerMock);

  TestProtobuf.InnerMessage.Builder msg = TestProtobuf.InnerMessage.newBuilder();
  msg.setOne("oneValue");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromString("oneValue"));
  inOrder.verify(readConsumerMock).endField("one", 0);

  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:21,代码来源:ProtoWriteSupportTest.java


示例6: testProto3SimplestMessage

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testProto3SimplestMessage() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProto3.InnerMessage.class, readConsumerMock);

  TestProto3.InnerMessage.Builder msg = TestProto3.InnerMessage.newBuilder();
  msg.setOne("oneValue");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromString("oneValue"));
  inOrder.verify(readConsumerMock).endField("one", 0);

  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:21,代码来源:ProtoWriteSupportTest.java


示例7: testRepeatedIntMessage

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testRepeatedIntMessage() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProtobuf.RepeatedIntMessage.class, readConsumerMock);

  TestProtobuf.RepeatedIntMessage.Builder msg = TestProtobuf.RepeatedIntMessage.newBuilder();
  msg.addRepeatedInt(1323);
  msg.addRepeatedInt(54469);

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("repeatedInt", 0);
  inOrder.verify(readConsumerMock).addInteger(1323);
  inOrder.verify(readConsumerMock).addInteger(54469);
  inOrder.verify(readConsumerMock).endField("repeatedInt", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:22,代码来源:ProtoWriteSupportTest.java


示例8: testProto3RepeatedIntMessage

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testProto3RepeatedIntMessage() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProto3.RepeatedIntMessage.class, readConsumerMock);

  TestProto3.RepeatedIntMessage.Builder msg = TestProto3.RepeatedIntMessage.newBuilder();
  msg.addRepeatedInt(1323);
  msg.addRepeatedInt(54469);

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("repeatedInt", 0);
  inOrder.verify(readConsumerMock).addInteger(1323);
  inOrder.verify(readConsumerMock).addInteger(54469);
  inOrder.verify(readConsumerMock).endField("repeatedInt", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:22,代码来源:ProtoWriteSupportTest.java


示例9: testRepeatedInnerMessageMessage_message

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testRepeatedInnerMessageMessage_message() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProtobuf.TopMessage.class, readConsumerMock);

  TestProtobuf.TopMessage.Builder msg = TestProtobuf.TopMessage.newBuilder();
  msg.addInnerBuilder().setOne("one").setTwo("two");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("inner", 0);
  inOrder.verify(readConsumerMock).startGroup();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("one".getBytes()));
  inOrder.verify(readConsumerMock).endField("one", 0);
  inOrder.verify(readConsumerMock).startField("two", 1);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("two".getBytes()));
  inOrder.verify(readConsumerMock).endField("two", 1);
  inOrder.verify(readConsumerMock).endGroup();
  inOrder.verify(readConsumerMock).endField("inner", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:27,代码来源:ProtoWriteSupportTest.java


示例10: testProto3RepeatedInnerMessageMessage_message

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testProto3RepeatedInnerMessageMessage_message() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProto3.TopMessage.class, readConsumerMock);

  TestProto3.TopMessage.Builder msg = TestProto3.TopMessage.newBuilder();
  msg.addInnerBuilder().setOne("one").setTwo("two");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("inner", 0);
  inOrder.verify(readConsumerMock).startGroup();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("one".getBytes()));
  inOrder.verify(readConsumerMock).endField("one", 0);
  inOrder.verify(readConsumerMock).startField("two", 1);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("two".getBytes()));
  inOrder.verify(readConsumerMock).endField("two", 1);
  inOrder.verify(readConsumerMock).endGroup();
  inOrder.verify(readConsumerMock).endField("inner", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:27,代码来源:ProtoWriteSupportTest.java


示例11: testOptionalInnerMessage

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testOptionalInnerMessage() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProtobuf.MessageA.class, readConsumerMock);

  TestProtobuf.MessageA.Builder msg = TestProtobuf.MessageA.newBuilder();
  msg.getInnerBuilder().setOne("one");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("inner", 0);

  inOrder.verify(readConsumerMock).startGroup();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("one".getBytes()));
  inOrder.verify(readConsumerMock).endField("one", 0);
  inOrder.verify(readConsumerMock).endGroup();

  inOrder.verify(readConsumerMock).endField("inner", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:26,代码来源:ProtoWriteSupportTest.java


示例12: testProto3OptionalInnerMessage

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testProto3OptionalInnerMessage() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProto3.MessageA.class, readConsumerMock);

  TestProto3.MessageA.Builder msg = TestProto3.MessageA.newBuilder();
  msg.getInnerBuilder().setOne("one");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("inner", 0);

  inOrder.verify(readConsumerMock).startGroup();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("one".getBytes()));
  inOrder.verify(readConsumerMock).endField("one", 0);
  inOrder.verify(readConsumerMock).endGroup();

  inOrder.verify(readConsumerMock).endField("inner", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:26,代码来源:ProtoWriteSupportTest.java


示例13: writeGroups

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
private void writeGroups(MessageType writtenSchema, MemPageStore memPageStore, Group... groups) {
  ColumnIOFactory columnIOFactory = new ColumnIOFactory(true);
  ColumnWriteStoreV1 columns = newColumnWriteStore(memPageStore);
  MessageColumnIO columnIO = columnIOFactory.getColumnIO(writtenSchema);
  RecordConsumer recordWriter = columnIO.getRecordWriter(columns);
  GroupWriter groupWriter = new GroupWriter(recordWriter, writtenSchema);
  for (Group group : groups) {
    groupWriter.write(group);
  }
  recordWriter.flush();
  columns.flush();
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:13,代码来源:TestColumnIO.java


示例14: testPushParser

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testPushParser() {
  MemPageStore memPageStore = new MemPageStore(1);
  ColumnWriteStoreV1 columns = newColumnWriteStore(memPageStore);
  MessageColumnIO columnIO = new ColumnIOFactory().getColumnIO(schema);
  RecordConsumer recordWriter = columnIO.getRecordWriter(columns);
  new GroupWriter(recordWriter, schema).write(r1);
  recordWriter.flush();
  columns.flush();

  RecordReader<Void> recordReader = columnIO.getRecordReader(memPageStore, new ExpectationValidatingConverter(expectedEventsForR1, schema));
  recordReader.read();

}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:15,代码来源:TestColumnIO.java


示例15: testGroupWriter

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testGroupWriter() {
  List<Group> result = new ArrayList<Group>();
  final GroupRecordConverter groupRecordConverter = new GroupRecordConverter(schema);
  RecordConsumer groupConsumer = new ConverterConsumer(groupRecordConverter.getRootConverter(), schema);
  GroupWriter groupWriter = new GroupWriter(new RecordConsumerLoggingWrapper(groupConsumer), schema);
  groupWriter.write(r1);
  result.add(groupRecordConverter.getCurrentRecord());
  groupWriter.write(r2);
  result.add(groupRecordConverter.getCurrentRecord());
  assertEquals("deserialization does not display the expected result", result.get(0).toString(), r1.toString());
  assertEquals("deserialization does not display the expected result", result.get(1).toString(), r2.toString());
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:14,代码来源:TestColumnIO.java


示例16: testWriteWithGroupWriter

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testWriteWithGroupWriter() {

  final String[] expected = {
      "[DocId]: 10, r:0, d:0",
      "[Links, Forward]: 20, r:0, d:2",
      "[Links, Forward]: 40, r:1, d:2",
      "[Links, Forward]: 60, r:1, d:2",
      "[Links, Backward]: null, r:0, d:1",
      "[Name, Language, Code]: en-us, r:0, d:2",
      "[Name, Language, Country]: us, r:0, d:3",
      "[Name, Language, Code]: en, r:2, d:2",
      "[Name, Language, Country]: null, r:2, d:2",
      "[Name, Url]: http://A, r:0, d:2",
      "[Name, Url]: http://B, r:1, d:2",
      "[Name, Language, Code]: null, r:1, d:1",
      "[Name, Language, Country]: null, r:1, d:1",
      "[Name, Language, Code]: en-gb, r:1, d:2",
      "[Name, Language, Country]: gb, r:1, d:3",
      "[Name, Url]: null, r:1, d:1",
      "[DocId]: 20, r:0, d:0",
      "[Links, Backward]: 10, r:0, d:2",
      "[Links, Backward]: 30, r:1, d:2",
      "[Links, Forward]: 80, r:0, d:2",
      "[Name, Url]: http://C, r:0, d:2",
      "[Name, Language, Code]: null, r:0, d:1",
      "[Name, Language, Country]: null, r:0, d:1"
  };

  ValidatingColumnWriteStore columns = new ValidatingColumnWriteStore(expected);
  MessageColumnIO columnIO = new ColumnIOFactory().getColumnIO(schema);
  RecordConsumer recordWriter = columnIO.getRecordWriter(columns);
  GroupWriter groupWriter = new GroupWriter(recordWriter, schema);
  groupWriter.write(r1);
  groupWriter.write(r2);
  recordWriter.flush();
  columns.validate();
  columns.flush();
  columns.close();
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:41,代码来源:TestColumnIO.java


示例17: testUnannotatedListOfGroups

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
@Ignore("Not yet supported")
public void testUnannotatedListOfGroups() throws Exception {
  Path test = writeDirect(
      "message UnannotatedListOfGroups {" +
          "  repeated group list_of_points {" +
          "    required float x;" +
          "    required float y;" +
          "  }" +
          "}",
      new DirectWriter() {
        @Override
        public void write(RecordConsumer rc) {
          rc.startMessage();
          rc.startField("list_of_points", 0);

          rc.startGroup();
          rc.startField("x", 0);
          rc.addFloat(1.0f);
          rc.endField("x", 0);
          rc.startField("y", 1);
          rc.addFloat(1.0f);
          rc.endField("y", 1);
          rc.endGroup();

          rc.startGroup();
          rc.startField("x", 0);
          rc.addFloat(2.0f);
          rc.endField("x", 0);
          rc.startField("y", 1);
          rc.addFloat(2.0f);
          rc.endField("y", 1);
          rc.endGroup();

          rc.endField("list_of_points", 0);
          rc.endMessage();
        }
      });
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:40,代码来源:TestArrayCompatibility.java


示例18: testRepeatedPrimitiveInList

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testRepeatedPrimitiveInList() throws Exception {
  Path test = writeDirect(
      "message RepeatedPrimitiveInList {" +
          "  required group list_of_ints (LIST) {" +
          "    repeated int32 array;" +
          "  }" +
          "}",
      new DirectWriter() {
        @Override
        public void write(RecordConsumer rc) {
          rc.startMessage();
          rc.startField("list_of_ints", 0);

          rc.startGroup();
          rc.startField("array", 0);

          rc.addInteger(34);
          rc.addInteger(35);
          rc.addInteger(36);

          rc.endField("array", 0);
          rc.endGroup();

          rc.endField("list_of_ints", 0);
          rc.endMessage();
        }
      });

  ListOfInts expected = new ListOfInts(Lists.newArrayList(34, 35,36));
  ListOfInts actual = reader(test, ListOfInts.class).read();
  Assert.assertEquals("Should read record correctly", expected, actual);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:34,代码来源:TestArrayCompatibility.java


示例19: validate

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
private <T extends TBase<?,?>> void validate(T expected) throws TException {
  @SuppressWarnings("unchecked")
  final Class<T> thriftClass = (Class<T>)expected.getClass();
  final MemPageStore memPageStore = new MemPageStore(1);
  final ThriftSchemaConverter schemaConverter = new ThriftSchemaConverter();
  final MessageType schema = schemaConverter.convert(thriftClass);
  LOG.info("{}", schema);
  final MessageColumnIO columnIO = new ColumnIOFactory(true).getColumnIO(schema);
  final ColumnWriteStoreV1 columns = new ColumnWriteStoreV1(memPageStore,
      ParquetProperties.builder()
          .withPageSize(10000)
          .withDictionaryEncoding(false)
          .build());
  final RecordConsumer recordWriter = columnIO.getRecordWriter(columns);
  final StructType thriftType = schemaConverter.toStructType(thriftClass);
  ParquetWriteProtocol parquetWriteProtocol = new ParquetWriteProtocol(recordWriter, columnIO, thriftType);

  expected.write(parquetWriteProtocol);
  recordWriter.flush();
  columns.flush();

  ThriftRecordConverter<T> converter = new TBaseRecordConverter<T>(thriftClass, schema, thriftType);
  final RecordReader<T> recordReader = columnIO.getRecordReader(memPageStore, converter);

  final T result = recordReader.read();

  assertEquals(expected, result);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:29,代码来源:TestParquetReadProtocol.java


示例20: testRepeatedInnerMessageMessage_scalar

import org.apache.parquet.io.api.RecordConsumer; //导入依赖的package包/类
@Test
public void testRepeatedInnerMessageMessage_scalar() throws Exception {
  RecordConsumer readConsumerMock =  Mockito.mock(RecordConsumer.class);
  ProtoWriteSupport instance = createReadConsumerInstance(TestProtobuf.TopMessage.class, readConsumerMock);

  TestProtobuf.TopMessage.Builder msg = TestProtobuf.TopMessage.newBuilder();
  msg.addInnerBuilder().setOne("one");
  msg.addInnerBuilder().setTwo("two");

  instance.write(msg.build());

  InOrder inOrder = Mockito.inOrder(readConsumerMock);

  inOrder.verify(readConsumerMock).startMessage();
  inOrder.verify(readConsumerMock).startField("inner", 0);
  //first inner message
  inOrder.verify(readConsumerMock).startGroup();
  inOrder.verify(readConsumerMock).startField("one", 0);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("one".getBytes()));
  inOrder.verify(readConsumerMock).endField("one", 0);
  inOrder.verify(readConsumerMock).endGroup();

  //second inner message
  inOrder.verify(readConsumerMock).startGroup();
  inOrder.verify(readConsumerMock).startField("two", 1);
  inOrder.verify(readConsumerMock).addBinary(Binary.fromConstantByteArray("two".getBytes()));
  inOrder.verify(readConsumerMock).endField("two", 1);
  inOrder.verify(readConsumerMock).endGroup();

  inOrder.verify(readConsumerMock).endField("inner", 0);
  inOrder.verify(readConsumerMock).endMessage();
  Mockito.verifyNoMoreInteractions(readConsumerMock);
}
 
开发者ID:apache,项目名称:parquet-mr,代码行数:34,代码来源:ProtoWriteSupportTest.java



注:本文中的org.apache.parquet.io.api.RecordConsumer类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java JreMemoryLeakPreventionListener类代码示例发布时间:2022-05-22
下一篇:
Java HighlightInfoHolder类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap