• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java FSDatasetInterface类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface的典型用法代码示例。如果您正苦于以下问题:Java FSDatasetInterface类的具体用法?Java FSDatasetInterface怎么用?Java FSDatasetInterface使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



FSDatasetInterface类属于org.apache.hadoop.hdfs.server.datanode包,在下文中一共展示了FSDatasetInterface类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: addSomeBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
int addSomeBlocks(FSDatasetInterface fsdataset, int startingBlockId) throws IOException {
  int bytesAdded = 0;
  for (int i = startingBlockId; i < startingBlockId+NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0); // we pass expected len as zero, - fsdataset should use the sizeof actual data written
    BlockDataFile.Writer dataOut = ((SimulatedFSDataset.SimulatedBlockInlineChecksumFileWriter) fsdataset
        .writeToBlock(0, b, b, false, false, -1, -1)).getBlockDataFile()
        .getWriter(0);
    assertEquals(0, fsdataset.getFinalizedBlockLength(0,b));
    for (int j=1; j <= blockIdToLen(i); ++j) {
      dataOut.write(new byte[] {(byte)j});
      assertEquals(j, fsdataset.getFinalizedBlockLength(0,b)); // correct length even as we write
      bytesAdded++;
    }
    dataOut.close();
    b.setNumBytes(blockIdToLen(i));
    fsdataset.finalizeBlock(0,b);
    assertEquals(blockIdToLen(i), fsdataset.getFinalizedBlockLength(0,b));
  }
  return bytesAdded;  
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:21,代码来源:TestSimulatedFSDataset.java


示例2: checkBlockDataAndSize

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
void  checkBlockDataAndSize(FSDatasetInterface fsdataset, 
            Block b, long expectedLen) throws IOException { 
  ReplicaToRead replica = fsdataset.getReplicaToRead(0, b);
  InputStream input = replica.getBlockInputStream(null, 0);
  long lengthRead = 0;
  int data;
  int count = 0;
  while ((data = input.read()) != -1) {
    if (count++ < BlockInlineChecksumReader.getHeaderSize()) {
      continue;
    }
    assertEquals(SimulatedFSDataset.DEFAULT_DATABYTE, data);
    lengthRead++;
  }
  assertEquals(expectedLen, lengthRead);
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:17,代码来源:TestSimulatedFSDataset.java


示例3: testInvalidate

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testInvalidate() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  int bytesAdded = addSomeBlocks(fsdataset);
  Block[] deleteBlocks = new Block[2];
  deleteBlocks[0] = new Block(1, 0, 0);
  deleteBlocks[1] = new Block(2, 0, 0);
  fsdataset.invalidate(0,deleteBlocks);
  checkInvalidBlock(deleteBlocks[0]);
  checkInvalidBlock(deleteBlocks[1]);
  long sizeDeleted = blockIdToLen(1) + blockIdToLen(2);
  assertEquals(bytesAdded-sizeDeleted, fsdataset.getDfsUsed());
  assertEquals(fsdataset.getCapacity()-bytesAdded+sizeDeleted,  fsdataset.getRemaining());
  
  
  
  // Now make sure the rest of the blocks are valid
  for (int i=3; i <= NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0);
    assertTrue(fsdataset.isValidBlock(0, b, false));
  }
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:22,代码来源:TestSimulatedFSDataset.java


示例4: addSomeBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
int addSomeBlocks(FSDatasetInterface fsdataset, int startingBlockId) throws IOException {
  int bytesAdded = 0;
  for (int i = startingBlockId; i < startingBlockId+NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0); // we pass expected len as zero, - fsdataset should use the sizeof actual data written
    OutputStream dataOut  = fsdataset.writeToBlock(b, false, false).dataOut;
    assertEquals(0, fsdataset.getLength(b));
    for (int j=1; j <= blockIdToLen(i); ++j) {
      dataOut.write(j);
      assertEquals(j, fsdataset.getLength(b)); // correct length even as we write
      bytesAdded++;
    }
    dataOut.close();
    b.setNumBytes(blockIdToLen(i));
    fsdataset.finalizeBlock(b);
    assertEquals(blockIdToLen(i), fsdataset.getLength(b));
  }
  return bytesAdded;  
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:19,代码来源:TestSimulatedFSDataset.java


示例5: testGetMetaData

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testGetMetaData() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  Block b = new Block(1, 5, 0);
  try {
    assertFalse(fsdataset.metaFileExists(b));
    assertTrue("Expected an IO exception", false);
  } catch (IOException e) {
    // ok - as expected
  }
  addSomeBlocks(fsdataset); // Only need to add one but ....
  b = new Block(1, 0, 0);
  InputStream metaInput = fsdataset.getMetaDataInputStream(b);
  DataInputStream metaDataInput = new DataInputStream(metaInput);
  short version = metaDataInput.readShort();
  assertEquals(FSDataset.METADATA_VERSION, version);
  DataChecksum checksum = DataChecksum.newDataChecksum(metaDataInput);
  assertEquals(DataChecksum.CHECKSUM_NULL, checksum.getChecksumType());
  assertEquals(0, checksum.getChecksumSize());  
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:20,代码来源:TestSimulatedFSDataset.java


示例6: testInvalidate

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testInvalidate() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  int bytesAdded = addSomeBlocks(fsdataset);
  Block[] deleteBlocks = new Block[2];
  deleteBlocks[0] = new Block(1, 0, 0);
  deleteBlocks[1] = new Block(2, 0, 0);
  fsdataset.invalidate(deleteBlocks);
  checkInvalidBlock(deleteBlocks[0]);
  checkInvalidBlock(deleteBlocks[1]);
  long sizeDeleted = blockIdToLen(1) + blockIdToLen(2);
  assertEquals(bytesAdded-sizeDeleted, fsdataset.getDfsUsed());
  assertEquals(fsdataset.getCapacity()-bytesAdded+sizeDeleted,  fsdataset.getRemaining());
  
  
  
  // Now make sure the rest of the blocks are valid
  for (int i=3; i <= NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0);
    assertTrue(fsdataset.isValidBlock(b));
  }
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:22,代码来源:TestSimulatedFSDataset.java


示例7: addSomeBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
int addSomeBlocks(FSDatasetInterface fsdataset, int startingBlockId) throws IOException {
  int bytesAdded = 0;
  for (int i = startingBlockId; i < startingBlockId+NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0); // we pass expected len as zero, - fsdataset should use the sizeof actual data written
    ReplicaInPipelineInterface bInfo = fsdataset.createRbw(b);
    BlockWriteStreams out = bInfo.createStreams(true, 512, 4);
    try {
      OutputStream dataOut  = out.dataOut;
      assertEquals(0, fsdataset.getLength(b));
      for (int j=1; j <= blockIdToLen(i); ++j) {
        dataOut.write(j);
        assertEquals(j, bInfo.getBytesOnDisk()); // correct length even as we write
        bytesAdded++;
      }
    } finally {
      out.close();
    }
    b.setNumBytes(blockIdToLen(i));
    fsdataset.finalizeBlock(b);
    assertEquals(blockIdToLen(i), fsdataset.getLength(b));
  }
  return bytesAdded;  
}
 
开发者ID:cumulusyebl,项目名称:cumulus,代码行数:24,代码来源:TestSimulatedFSDataset.java


示例8: addSomeBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
int addSomeBlocks(FSDatasetInterface fsdataset, int startingBlockId) throws IOException {
  int bytesAdded = 0;
  for (int i = startingBlockId; i < startingBlockId+NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0); // we pass expected len as zero, - fsdataset should use the sizeof actual data written
    OutputStream dataOut  = fsdataset.writeToBlock(0, b, false, false).dataOut;
    assertEquals(0, fsdataset.getFinalizedBlockLength(0,b));
    for (int j=1; j <= blockIdToLen(i); ++j) {
      dataOut.write(j);
      assertEquals(j, fsdataset.getFinalizedBlockLength(0,b)); // correct length even as we write
      bytesAdded++;
    }
    dataOut.close();
    b.setNumBytes(blockIdToLen(i));
    fsdataset.finalizeBlock(0,b);
    assertEquals(blockIdToLen(i), fsdataset.getFinalizedBlockLength(0,b));
  }
  return bytesAdded;  
}
 
开发者ID:iVCE,项目名称:RDFS,代码行数:19,代码来源:TestSimulatedFSDataset.java


示例9: testGetMetaData

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testGetMetaData() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  Block b = new Block(1, 5, 0);
  try {
    assertFalse(fsdataset.metaFileExists(0,b));
    assertTrue("Expected an IO exception", false);
  } catch (IOException e) {
    // ok - as expected
  }
  addSomeBlocks(fsdataset); // Only need to add one but ....
  b = new Block(1, 0, 0);
  InputStream metaInput = fsdataset.getMetaDataInputStream(0,b);
  DataInputStream metaDataInput = new DataInputStream(metaInput);
  short version = metaDataInput.readShort();
  assertEquals(FSDataset.METADATA_VERSION, version);
  DataChecksum checksum = DataChecksum.newDataChecksum(metaDataInput);
  assertEquals(DataChecksum.CHECKSUM_NULL, checksum.getChecksumType());
  assertEquals(0, checksum.getChecksumSize());  
}
 
开发者ID:iVCE,项目名称:RDFS,代码行数:20,代码来源:TestSimulatedFSDataset.java


示例10: addSomeBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
int addSomeBlocks(FSDatasetInterface fsdataset, int startingBlockId) throws IOException {
  int bytesAdded = 0;
  for (int i = startingBlockId; i < startingBlockId+NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0); // we pass expected len as zero, - fsdataset should use the sizeof actual data written
    OutputStream dataOut  = fsdataset.writeToBlock(b, false).dataOut;
    assertEquals(0, fsdataset.getLength(b));
    for (int j=1; j <= blockIdToLen(i); ++j) {
      dataOut.write(j);
      assertEquals(j, fsdataset.getLength(b)); // correct length even as we write
      bytesAdded++;
    }
    dataOut.close();
    b.setNumBytes(blockIdToLen(i));
    fsdataset.finalizeBlock(b);
    assertEquals(blockIdToLen(i), fsdataset.getLength(b));
  }
  return bytesAdded;  
}
 
开发者ID:thisisvoa,项目名称:hadoop-0.20,代码行数:19,代码来源:TestSimulatedFSDataset.java


示例11: injectBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
/**
 * This method is valid only if the data nodes have simulated data
 * @param dataNodeIndex - data node i which to inject - the index is same as for getDataNodes()
 * @param blocksToInject - the blocks
 * @throws IOException
 *              if not simulatedFSDataset
 *             if any of blocks already exist in the data node
 *   
 */
public void injectBlocks(int dataNodeIndex, Block[] blocksToInject) throws IOException {
  if (dataNodeIndex < 0 || dataNodeIndex > dataNodes.size()) {
    throw new IndexOutOfBoundsException();
  }
  FSDatasetInterface dataSet = dataNodes.get(dataNodeIndex).datanode.getFSDataset();
  if (!(dataSet instanceof SimulatedFSDataset)) {
    throw new IOException("injectBlocks is valid only for SimilatedFSDataset");
  }
  SimulatedFSDataset sdataset = (SimulatedFSDataset) dataSet;
  sdataset.injectBlocks(getNameNode().getNamespaceID(), blocksToInject);
  dataNodes.get(dataNodeIndex).datanode.scheduleNSBlockReport(0);
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:22,代码来源:MiniDFSCluster.java


示例12: testStorageUsage

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testStorageUsage() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  assertEquals(fsdataset.getDfsUsed(), 0);
  assertEquals(fsdataset.getRemaining(), fsdataset.getCapacity());
  int bytesAdded = addSomeBlocks(fsdataset);
  assertEquals(bytesAdded, fsdataset.getDfsUsed());
  assertEquals(fsdataset.getCapacity()-bytesAdded,  fsdataset.getRemaining());
  
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:10,代码来源:TestSimulatedFSDataset.java


示例13: testWriteRead

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testWriteRead() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  addSomeBlocks(fsdataset);
  for (int i=1; i <= NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0);
    assertTrue(fsdataset.isValidBlock(0, b, false));
    assertEquals(blockIdToLen(i), fsdataset.getFinalizedBlockLength(0,b));
    checkBlockDataAndSize(fsdataset, b, blockIdToLen(i));
  }
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:11,代码来源:TestSimulatedFSDataset.java


示例14: testGetBlockReport

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testGetBlockReport() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  Block[] blockReport = fsdataset.getBlockReport(0);
  assertEquals(0, blockReport.length);
  int bytesAdded = addSomeBlocks(fsdataset);
  blockReport = fsdataset.getBlockReport(0);
  assertEquals(NUMBLOCKS, blockReport.length);
  for (Block b: blockReport) {
    assertNotNull(b);
    assertEquals(blockIdToLen(b.getBlockId()), b.getNumBytes());
  }
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:13,代码来源:TestSimulatedFSDataset.java


示例15: testInValidBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testInValidBlocks() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  Block b = new Block(1, 5, 0);
  checkInvalidBlock(b);
  
  // Now check invlaid after adding some blocks
  addSomeBlocks(fsdataset);
  b = new Block(NUMBLOCKS + 99, 5, 0);
  checkInvalidBlock(b);
  
}
 
开发者ID:rhli,项目名称:hadoop-EAR,代码行数:12,代码来源:TestSimulatedFSDataset.java


示例16: injectBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
/**
 * This method is valid only if the data nodes have simulated data
 * @param dataNodeIndex - data node i which to inject - the index is same as for getDataNodes()
 * @param blocksToInject - the blocks
 * @throws IOException
 *              if not simulatedFSDataset
 *             if any of blocks already exist in the data node
 *   
 */
public void injectBlocks(int dataNodeIndex, Block[] blocksToInject) throws IOException {
  if (dataNodeIndex < 0 || dataNodeIndex > dataNodes.size()) {
    throw new IndexOutOfBoundsException();
  }
  FSDatasetInterface dataSet = dataNodes.get(dataNodeIndex).datanode.getFSDataset();
  if (!(dataSet instanceof SimulatedFSDataset)) {
    throw new IOException("injectBlocks is valid only for SimilatedFSDataset");
  }
  SimulatedFSDataset sdataset = (SimulatedFSDataset) dataSet;
  sdataset.injectBlocks(blocksToInject);
  dataNodes.get(dataNodeIndex).datanode.scheduleBlockReport(0);
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:22,代码来源:MiniDFSCluster.java


示例17: checkBlockDataAndSize

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
void  checkBlockDataAndSize(FSDatasetInterface fsdataset, 
            Block b, long expectedLen) throws IOException { 
  InputStream input = fsdataset.getBlockInputStream(b);
  long lengthRead = 0;
  int data;
  while ((data = input.read()) != -1) {
    assertEquals(SimulatedFSDataset.DEFAULT_DATABYTE, data);
    lengthRead++;
  }
  assertEquals(expectedLen, lengthRead);
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:12,代码来源:TestSimulatedFSDataset.java


示例18: testWriteRead

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testWriteRead() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  addSomeBlocks(fsdataset);
  for (int i=1; i <= NUMBLOCKS; ++i) {
    Block b = new Block(i, 0, 0);
    assertTrue(fsdataset.isValidBlock(b));
    assertEquals(blockIdToLen(i), fsdataset.getLength(b));
    checkBlockDataAndSize(fsdataset, b, blockIdToLen(i));
  }
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:11,代码来源:TestSimulatedFSDataset.java


示例19: testGetBlockReport

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
public void testGetBlockReport() throws IOException {
  FSDatasetInterface fsdataset = new SimulatedFSDataset(conf); 
  Block[] blockReport = fsdataset.getBlockReport();
  assertEquals(0, blockReport.length);
  int bytesAdded = addSomeBlocks(fsdataset);
  blockReport = fsdataset.getBlockReport();
  assertEquals(NUMBLOCKS, blockReport.length);
  for (Block b: blockReport) {
    assertNotNull(b);
    assertEquals(blockIdToLen(b.getBlockId()), b.getNumBytes());
  }
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre,代码行数:13,代码来源:TestSimulatedFSDataset.java


示例20: injectBlocks

import org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface; //导入依赖的package包/类
/**
 * This method is valid only if the data nodes have simulated data
 * @param dataNodeIndex - data node i which to inject - the index is same as for getDataNodes()
 * @param blocksToInject - the blocks
 * @throws IOException
 *              if not simulatedFSDataset
 *             if any of blocks already exist in the data node
 *   
 */
public void injectBlocks(int dataNodeIndex, Iterable<Block> blocksToInject) throws IOException {
  if (dataNodeIndex < 0 || dataNodeIndex > dataNodes.size()) {
    throw new IndexOutOfBoundsException();
  }
  FSDatasetInterface dataSet = dataNodes.get(dataNodeIndex).datanode.getFSDataset();
  if (!(dataSet instanceof SimulatedFSDataset)) {
    throw new IOException("injectBlocks is valid only for SimilatedFSDataset");
  }
  SimulatedFSDataset sdataset = (SimulatedFSDataset) dataSet;
  sdataset.injectBlocks(blocksToInject);
  dataNodes.get(dataNodeIndex).datanode.scheduleBlockReport(0);
}
 
开发者ID:cumulusyebl,项目名称:cumulus,代码行数:22,代码来源:MiniDFSCluster.java



注:本文中的org.apache.hadoop.hdfs.server.datanode.FSDatasetInterface类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java JCAssert类代码示例发布时间:2022-05-22
下一篇:
Java SetCount类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap