• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java Writer类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.orc.Writer的典型用法代码示例。如果您正苦于以下问题:Java Writer类的具体用法?Java Writer怎么用?Java Writer使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



Writer类属于org.apache.orc包,在下文中一共展示了Writer类的4个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: flush

import org.apache.orc.Writer; //导入依赖的package包/类
private boolean flush(BufferSegment segment, String path, TypeDescription schema)
    {
        Configuration conf = new Configuration();
        try {
            Writer writer = OrcFile.createWriter(new Path(path),
                    OrcFile.writerOptions(conf)
                            .setSchema(schema)
                            .stripeSize(orcFileStripeSize)
                            .bufferSize(orcFileBufferSize)
                            .blockSize(orcFileBlockSize)
                            .compress(CompressionKind.ZLIB)
                            .version(OrcFile.Version.V_0_12));
            VectorizedRowBatch batch = schema.createRowBatch();
            while (segment.hasNext()) {
                String[] contents = segment.getNext();
                int rowCount = batch.size++;
//                    System.out.println("contents : message.getValues() : " + Arrays.toString(contents));
                System.out.println("contents.length : " + contents.length);
                for (int i = 0; i < contents.length; i++) {
                    ((BytesColumnVector) batch.cols[i]).setVal(rowCount, contents[i].getBytes());
                    //batch full
                    if (batch.size == batch.getMaxSize()) {
                        writer.addRowBatch(batch);
                        batch.reset();
                    }
                }
                if (batch.size != 0) {
                    writer.addRowBatch(batch);
                    batch.reset();
                }
                writer.close();
                segment.setFilePath(path);
                System.out.println("path : " + path);
            }
            return true;
        }
        catch (IOException e) {
            e.printStackTrace();
            return false;
        }
    }
 
开发者ID:dbiir,项目名称:paraflow,代码行数:42,代码来源:OrcFlushThread.java


示例2: AWriterImpl

import org.apache.orc.Writer; //导入依赖的package包/类
public AWriterImpl(FileSystem fs, Path path, OrcFile.WriterOptions opts) throws IOException {
  this.fs = fs;
  this.path = path;
  this.conf = opts.getConfiguration();
  this.callback = opts.getCallback();
  this.schema = opts.getSchema();
  if (callback != null) {
    callbackContext = new OrcFile.WriterContext() {

      @Override
      public Writer getWriter() {
        return AWriterImpl.this;
      }
    };
  } else {
    callbackContext = null;
  }
  this.adjustedStripeSize = opts.getStripeSize();
  this.defaultStripeSize = opts.getStripeSize();
  this.version = opts.getVersion();
  this.encodingStrategy = opts.getEncodingStrategy();
  this.compressionStrategy = opts.getCompressionStrategy();
  this.addBlockPadding = opts.getBlockPadding();
  this.blockSize = opts.getBlockSize();
  this.paddingTolerance = opts.getPaddingTolerance();
  this.compress = opts.getCompress();
  this.rowIndexStride = opts.getRowIndexStride();
  this.memoryManager = opts.getMemoryManager();
  buildIndex = rowIndexStride > 0;
  codec = createCodec(compress);
  int numColumns = schema.getMaximumId() + 1;
  if (opts.isEnforceBufferSize()) {
    this.bufferSize = opts.getBufferSize();
  } else {
    this.bufferSize = getEstimatedBufferSize(defaultStripeSize, numColumns, opts.getBufferSize());
  }
  if (version == OrcFile.Version.V_0_11) {
    /* do not write bloom filters for ORC v11 */
    this.bloomFilterColumns = new boolean[schema.getMaximumId() + 1];
  } else {
    this.bloomFilterColumns = OrcUtils.includeColumns(opts.getBloomFilterColumns(), schema);
  }
  this.bloomFilterFpp = opts.getBloomFilterFpp();
  treeWriter = createTreeWriter(schema, streamFactory, false);
  if (buildIndex && rowIndexStride < MIN_ROW_INDEX_STRIDE) {
    throw new IllegalArgumentException("Row stride must be at least " + MIN_ROW_INDEX_STRIDE);
  }

  // ensure that we are able to handle callbacks before we register ourselves
  memoryManager.addWriter(path, opts.getStripeSize(), this);
  // LOG.info("ORC writer created for path: {} with stripeSize: {} blockSize: {}" +
  // " compression: {} bufferSize: {}", path, defaultStripeSize, blockSize,
  // compress, bufferSize);
}
 
开发者ID:ampool,项目名称:monarch,代码行数:55,代码来源:AWriterImpl.java


示例3: OrcEntityProcessor

import org.apache.orc.Writer; //导入依赖的package包/类
OrcEntityProcessor(Writer writer, VectorizedRowBatch batch) {
    this.writer = writer;
    this.batch = batch;
}
 
开发者ID:mojodna,项目名称:osm2orc,代码行数:5,代码来源:OrcWriter.java


示例4: OrcMetaDataWriter

import org.apache.orc.Writer; //导入依赖的package包/类
public OrcMetaDataWriter( Writer writer ) {
  this.writer = writer;
}
 
开发者ID:pentaho,项目名称:pentaho-hadoop-shims,代码行数:4,代码来源:OrcMetaDataWriter.java



注:本文中的org.apache.orc.Writer类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java TIFFDecodeParam类代码示例发布时间:2022-05-23
下一篇:
Java ImmediateExecutor类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap