• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号
登陆 注册
描述文字

Java LoadTestDataGenerator类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.hbase.util.test.LoadTestDataGenerator类的典型用法代码示例。如果您正苦于以下问题：Java LoadTestDataGenerator类的具体用法？Java LoadTestDataGenerator怎么用？Java LoadTestDataGenerator使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



LoadTestDataGenerator类属于org.apache.hadoop.hbase.util.test包，在下文中一共展示了LoadTestDataGenerator类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: prepareForLoadTest

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
protected void prepareForLoadTest() throws IOException {
  LOG.info("Starting load test: dataBlockEncoding=" + dataBlockEncoding +
      ", isMultiPut=" + isMultiPut);
  numKeys = numKeys();
  Admin admin = new HBaseAdmin(conf);
  while (admin.getClusterStatus().getServers().size() < NUM_RS) {
    LOG.info("Sleeping until " + NUM_RS + " RSs are online");
    Threads.sleepWithoutInterrupt(1000);
  }
  admin.close();

  HTableDescriptor htd = new HTableDescriptor(TABLE);
  HColumnDescriptor hcd = new HColumnDescriptor(CF)
    .setCompressionType(compression)
    .setDataBlockEncoding(dataBlockEncoding);
  createPreSplitLoadTestTable(htd, hcd);

  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(CF);
  writerThreads = prepareWriterThreads(dataGen, conf, TABLE);
  readerThreads = prepareReaderThreads(dataGen, conf, TABLE, 100);
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:22，代码来源:TestMiniClusterLoadSequential.java


示例2: prepareForLoadTest

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
protected void prepareForLoadTest() throws IOException {
  LOG.info("Starting load test: dataBlockEncoding=" + dataBlockEncoding +
      ", isMultiPut=" + isMultiPut);
  numKeys = numKeys();
  HBaseAdmin admin = new HBaseAdmin(conf);
  while (admin.getClusterStatus().getServers().size() < NUM_RS) {
    LOG.info("Sleeping until " + NUM_RS + " RSs are online");
    Threads.sleepWithoutInterrupt(1000);
  }
  admin.close();

  HTableDescriptor htd = new HTableDescriptor(TABLE);
  HColumnDescriptor hcd = new HColumnDescriptor(CF)
    .setCompressionType(compression)
    .setDataBlockEncoding(dataBlockEncoding);
  createPreSplitLoadTestTable(htd, hcd);

  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(CF);
  writerThreads = prepareWriterThreads(dataGen, conf, TABLE);
  readerThreads = prepareReaderThreads(dataGen, conf, TABLE, 100);
}
开发者ID:tenggyut，项目名称:HIndex，代码行数:22，代码来源:TestMiniClusterLoadSequential.java


示例3: prepareForLoadTest

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
protected void prepareForLoadTest() throws IOException {
  LOG.info("Starting load test: dataBlockEncoding=" + dataBlockEncoding +
      ", isMultiPut=" + isMultiPut);
  numKeys = numKeys();
  Admin admin = TEST_UTIL.getAdmin();
  while (admin.getClusterMetrics(EnumSet.of(Option.LIVE_SERVERS))
              .getLiveServerMetrics().size() < NUM_RS) {
    LOG.info("Sleeping until " + NUM_RS + " RSs are online");
    Threads.sleepWithoutInterrupt(1000);
  }
  admin.close();

  HTableDescriptor htd = new HTableDescriptor(TABLE);
  HColumnDescriptor hcd = new HColumnDescriptor(CF)
    .setCompressionType(compression)
    .setDataBlockEncoding(dataBlockEncoding);
  createPreSplitLoadTestTable(htd, hcd);

  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(CF);
  writerThreads = prepareWriterThreads(dataGen, conf, TABLE);
  readerThreads = prepareReaderThreads(dataGen, conf, TABLE, 100);
}
开发者ID:apache，项目名称:hbase，代码行数:23，代码来源:TestMiniClusterLoadSequential.java


示例4: prepareForLoadTest

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
protected void prepareForLoadTest() throws IOException {
  LOG.info("Starting load test: dataBlockEncoding=" + dataBlockEncoding +
      ", isMultiPut=" + isMultiPut);
  numKeys = numKeys();
  HBaseAdmin admin = new HBaseAdmin(conf);
  while (admin.getClusterStatus().getServers().size() < NUM_RS) {
    LOG.info("Sleeping until " + NUM_RS + " RSs are online");
    Threads.sleepWithoutInterrupt(1000);
  }
  admin.close();

  int numRegions = HBaseTestingUtility.createPreSplitLoadTestTable(conf,
      TABLE, CF, compression, dataBlockEncoding);

  TEST_UTIL.waitUntilAllRegionsAssigned(numRegions);

  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(CF);
  writerThreads = new MultiThreadedWriter(dataGen, conf, TABLE);
  writerThreads.setMultiPut(isMultiPut);
  readerThreads = new MultiThreadedReader(dataGen, conf, TABLE, 100);
}
开发者ID:daidong，项目名称:DominoHBase，代码行数:22，代码来源:TestMiniClusterLoadSequential.java


示例5: MultiThreadedAction

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedAction(LoadTestDataGenerator dataGen, Configuration conf,
                           TableName tableName,
                           String actionLetter) throws IOException {
  this.conf = conf;
  this.dataGenerator = dataGen;
  this.tableName = tableName;
  this.actionLetter = actionLetter;
  this.connection = HConnectionManager.createConnection(conf);
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:10，代码来源:MultiThreadedAction.java


示例6: getLoadGeneratorInstance

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
private LoadTestDataGenerator getLoadGeneratorInstance(String clazzName) throws IOException {
  try {
    Class<?> clazz = Class.forName(clazzName);
    Constructor<?> constructor = clazz.getConstructor(int.class, int.class, int.class, int.class,
        byte[][].class);
    return (LoadTestDataGenerator) constructor.newInstance(minColDataSize, maxColDataSize,
        minColsPerKey, maxColsPerKey, families);
  } catch (Exception e) {
    throw new IOException(e);
  }
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:12，代码来源:LoadTestTool.java


示例7: getMultiThreadedWriterInstance

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
private MultiThreadedWriter getMultiThreadedWriterInstance(String clazzName
    , LoadTestDataGenerator dataGen) throws IOException {
  try {
    Class<?> clazz = Class.forName(clazzName);
    Constructor<?> constructor = clazz.getConstructor(
      LoadTestDataGenerator.class, Configuration.class, TableName.class);
    return (MultiThreadedWriter) constructor.newInstance(dataGen, conf, tableName);
  } catch (Exception e) {
    throw new IOException(e);
  }
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:12，代码来源:LoadTestTool.java


示例8: getMultiThreadedUpdaterInstance

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
private MultiThreadedUpdater getMultiThreadedUpdaterInstance(String clazzName
    , LoadTestDataGenerator dataGen) throws IOException {
  try {
    Class<?> clazz = Class.forName(clazzName);
    Constructor<?> constructor = clazz.getConstructor(
      LoadTestDataGenerator.class, Configuration.class, TableName.class, double.class);
    return (MultiThreadedUpdater) constructor.newInstance(
      dataGen, conf, tableName, updatePercent);
  } catch (Exception e) {
    throw new IOException(e);
  }
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:13，代码来源:LoadTestTool.java


示例9: getMultiThreadedReaderInstance

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
private MultiThreadedReader getMultiThreadedReaderInstance(String clazzName
    , LoadTestDataGenerator dataGen) throws IOException {
  try {
    Class<?> clazz = Class.forName(clazzName);
    Constructor<?> constructor = clazz.getConstructor(
      LoadTestDataGenerator.class, Configuration.class, TableName.class, double.class);
    return (MultiThreadedReader) constructor.newInstance(dataGen, conf, tableName, verifyPercent);
  } catch (Exception e) {
    throw new IOException(e);
  }
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:12，代码来源:LoadTestTool.java


示例10: MultiThreadedUpdaterWithACL

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedUpdaterWithACL(LoadTestDataGenerator dataGen, Configuration conf,
    TableName tableName, double updatePercent, User userOwner, String userNames)
        throws IOException {
  super(dataGen, conf, tableName, updatePercent);
  this.userOwner = userOwner;
  this.userNames = userNames.split(COMMA);
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:8，代码来源:MultiThreadedUpdaterWithACL.java


示例11: loadData

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
/** Loads data into the table using the multi-threaded writer. */
private void loadData() throws IOException {
  long startKey = 0;
  long endKey = 100000;
  int minColsPerKey = 5;
  int maxColsPerKey = 15;
  int minColDataSize = 256;
  int maxColDataSize = 256 * 3;
  int numThreads = 10;

  // print out the arguments
  System.out.printf("Key range %d .. %d\n", startKey, endKey);
  System.out.printf("Number of Columns/Key: %d..%d\n", minColsPerKey,
      maxColsPerKey);
  System.out.printf("Data Size/Column: %d..%d bytes\n", minColDataSize,
      maxColDataSize);
  System.out.printf("Client Threads: %d\n", numThreads);

  // start the writers
  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(
    minColDataSize, maxColDataSize, minColsPerKey, maxColsPerKey,
    LoadTestTool.DEFAULT_COLUMN_FAMILY);
  MultiThreadedWriter writer = new MultiThreadedWriter(dataGen, conf, TABLE_NAME);
  writer.setMultiPut(true);
  writer.start(startKey, endKey, numThreads);
  System.out.printf("Started loading data...");
  writer.waitForFinish();
  System.out.printf("Finished loading data...");
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:30，代码来源:RestartMetaTest.java


示例12: TimeBoundedMultiThreadedReader

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public TimeBoundedMultiThreadedReader(LoadTestDataGenerator dataGen, Configuration conf,
    TableName tableName, double verifyPercent) throws IOException {
  super(dataGen, conf, tableName, verifyPercent);
  long timeoutMs = conf.getLong(
    String.format("%s.%s", TEST_NAME, GET_TIMEOUT_KEY), DEFAULT_GET_TIMEOUT);
  timeoutNano = timeoutMs * 1000000;
  LOG.info("Timeout for gets: " + timeoutMs);
  String runTimeKey = String.format(RUN_TIME_KEY, this.getClass().getSimpleName());
  this.runTime = conf.getLong(runTimeKey, -1);
  if (this.runTime <= 0) {
    throw new IllegalArgumentException("Please configure " + runTimeKey);
  }
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:14，代码来源:IntegrationTestTimeBoundedRequestsWithRegionReplicas.java


示例13: getLoadGeneratorInstance

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
private LoadTestDataGenerator getLoadGeneratorInstance(String clazzName) throws IOException {
  try {
    Class<?> clazz = Class.forName(clazzName);
    Constructor<?> constructor = clazz.getConstructor(int.class, int.class, int.class, int.class,
        byte[][].class);
    return (LoadTestDataGenerator) constructor.newInstance(minColDataSize, maxColDataSize,
        minColsPerKey, maxColsPerKey, COLUMN_FAMILIES);
  } catch (Exception e) {
    throw new IOException(e);
  }
}
开发者ID:grokcoder，项目名称:pbase，代码行数:12，代码来源:LoadTestTool.java


示例14: loadData

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
/** Loads data into the table using the multi-threaded writer. */
private void loadData() throws IOException {
  long startKey = 0;
  long endKey = 100000;
  int minColsPerKey = 5;
  int maxColsPerKey = 15;
  int minColDataSize = 256;
  int maxColDataSize = 256 * 3;
  int numThreads = 10;

  // print out the arguments
  System.out.printf("Key range %d .. %d\n", startKey, endKey);
  System.out.printf("Number of Columns/Key: %d..%d\n", minColsPerKey,
      maxColsPerKey);
  System.out.printf("Data Size/Column: %d..%d bytes\n", minColDataSize,
      maxColDataSize);
  System.out.printf("Client Threads: %d\n", numThreads);

  // start the writers
  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(
    minColDataSize, maxColDataSize, minColsPerKey, maxColsPerKey, LoadTestTool.COLUMN_FAMILY);
  MultiThreadedWriter writer = new MultiThreadedWriter(dataGen, conf, TABLE_NAME);
  writer.setMultiPut(true);
  writer.start(startKey, endKey, numThreads);
  System.out.printf("Started loading data...");
  writer.waitForFinish();
  System.out.printf("Finished loading data...");
}
开发者ID:grokcoder，项目名称:pbase，代码行数:29，代码来源:RestartMetaTest.java


示例15: MultiThreadedAction

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedAction(LoadTestDataGenerator dataGen, Configuration conf,
                           TableName tableName,
                           String actionLetter) {
  this.conf = conf;
  this.dataGenerator = dataGen;
  this.tableName = tableName;
  this.actionLetter = actionLetter;
}
开发者ID:tenggyut，项目名称:HIndex，代码行数:9，代码来源:MultiThreadedAction.java


示例16: MultiThreadedAction

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedAction(LoadTestDataGenerator dataGen, Configuration conf,
                           TableName tableName,
                           String actionLetter) throws IOException {
  this.conf = conf;
  this.dataGenerator = dataGen;
  this.tableName = tableName;
  this.actionLetter = actionLetter;
  this.connection = (ClusterConnection) ConnectionFactory.createConnection(conf);
}
开发者ID:apache，项目名称:hbase，代码行数:10，代码来源:MultiThreadedAction.java


示例17: loadData

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
/** Loads data into the table using the multi-threaded writer. */
private void loadData() throws IOException {
  long startKey = 0;
  long endKey = 100000;
  int minColsPerKey = 5;
  int maxColsPerKey = 15;
  int minColDataSize = 256;
  int maxColDataSize = 256 * 3;
  int numThreads = 10;

  // print out the arguments
  System.out.printf("Key range %d .. %d\n", startKey, endKey);
  System.out.printf("Number of Columns/Key: %d..%d\n", minColsPerKey,
      maxColsPerKey);
  System.out.printf("Data Size/Column: %d..%d bytes\n", minColDataSize,
      maxColDataSize);
  System.out.printf("Client Threads: %d\n", numThreads);

  // start the writers
  LoadTestDataGenerator dataGen = new MultiThreadedAction.DefaultDataGenerator(
    minColDataSize, maxColDataSize, minColsPerKey, maxColsPerKey,
    HFileTestUtil.DEFAULT_COLUMN_FAMILY);
  MultiThreadedWriter writer = new MultiThreadedWriter(dataGen, conf, TABLE_NAME);
  writer.setMultiPut(true);
  writer.start(startKey, endKey, numThreads);
  System.out.printf("Started loading data...");
  writer.waitForFinish();
  System.out.printf("Finished loading data...");
}
开发者ID:apache，项目名称:hbase，代码行数:30，代码来源:RestartMetaTest.java


示例18: MultiThreadedAction

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedAction(LoadTestDataGenerator dataGen, Configuration conf, byte[] tableName,
    String actionLetter) {
  this.conf = conf;
  this.dataGenerator = dataGen;
  this.tableName = tableName;
  this.actionLetter = actionLetter;
}
开发者ID:daidong，项目名称:DominoHBase，代码行数:8，代码来源:MultiThreadedAction.java


示例19: MultiThreadedWriterBase

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedWriterBase(LoadTestDataGenerator dataGen, Configuration conf,
    TableName tableName, String actionLetter) throws IOException {
  super(dataGen, conf, tableName, actionLetter);
  this.wroteKeys = createWriteKeysQueue(conf);
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:6，代码来源:MultiThreadedWriterBase.java


示例20: MultiThreadedWriter

import org.apache.hadoop.hbase.util.test.LoadTestDataGenerator; //导入依赖的package包/类
public MultiThreadedWriter(LoadTestDataGenerator dataGen, Configuration conf,
    TableName tableName) throws IOException {
  super(dataGen, conf, tableName, "W");
}
开发者ID:fengchen8086，项目名称:ditb，代码行数:5，代码来源:MultiThreadedWriter.java



注：本文中的org.apache.hadoop.hbase.util.test.LoadTestDataGenerator类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。

鲜花
握手
雷人
路过
鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
More+
上一篇：
Java TimeOfDay类代码示例发布时间：2022-05-22
下一篇：
Java Output类代码示例发布时间：2022-05-22
热门推荐
More+
热门话题
More+
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服（服务时间 9:00～18:00）

在线QQ客服
地址：深圳市南山区西丽大学城创智工业园
电邮：jeky_zhao#qq.com
移动电话：139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap