• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java AbortSpec类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec的典型用法代码示例。如果您正苦于以下问题:Java AbortSpec类的具体用法?Java AbortSpec怎么用?Java AbortSpec使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



AbortSpec类属于org.apache.hadoop.hdfs.server.namenode.TestEditLog包,在下文中一共展示了AbortSpec类的8个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testInprogressRecoveryMixed

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/**
 * Test a mixture of inprogress files and finalised. Set up 3 edits 
 * directories and fail the second on the last roll. Verify that reading
 * the transactions, reads from the finalised directories.
 */
@Test
public void testInprogressRecoveryMixed() throws IOException {
  File f1 = new File(TestEditLog.TEST_DIR + "/mixtest0");
  File f2 = new File(TestEditLog.TEST_DIR + "/mixtest1");
  File f3 = new File(TestEditLog.TEST_DIR + "/mixtest2");
  
  List<URI> editUris = ImmutableList.of(f1.toURI(), f2.toURI(), f3.toURI());

  // abort after the 5th roll 
  NNStorage storage = setupEdits(editUris,
                                 5, new AbortSpec(5, 1));
  Iterator<StorageDirectory> dirs = storage.dirIterator(NameNodeDirType.EDITS);
  StorageDirectory sd = dirs.next();
  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  assertEquals(6*TXNS_PER_ROLL, getNumberOfTransactions(jm, 1, true, false));
  
  sd = dirs.next();
  jm = new FileJournalManager(conf, sd, storage);
  assertEquals(5*TXNS_PER_ROLL + TXNS_PER_FAIL, getNumberOfTransactions(jm, 1,
      true, false));

  sd = dirs.next();
  jm = new FileJournalManager(conf, sd, storage);
  assertEquals(6*TXNS_PER_ROLL, getNumberOfTransactions(jm, 1, true, false));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:31,代码来源:TestFileJournalManager.java


示例2: testFinalizeErrorReportedToNNStorage

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
@Test(expected=IllegalStateException.class)
public void testFinalizeErrorReportedToNNStorage() throws IOException, InterruptedException {
  File f = new File(TestEditLog.TEST_DIR + "/filejournaltestError");
  // abort after 10th roll
  NNStorage storage = setupEdits(Collections.<URI>singletonList(f.toURI()),
                                 10, new AbortSpec(10, 0));
  StorageDirectory sd = storage.dirIterator(NameNodeDirType.EDITS).next();

  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  String sdRootPath = sd.getRoot().getAbsolutePath();
  FileUtil.chmod(sdRootPath, "-w", true);
  try {
    jm.finalizeLogSegment(0, 1);
  } finally {
    FileUtil.chmod(sdRootPath, "+w", true);
    assertTrue(storage.getRemovedStorageDirs().contains(sd));
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:19,代码来源:TestFileJournalManager.java


示例3: testReadFromStream

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/** 
 * Test that we can read from a stream created by FileJournalManager.
 * Create a single edits directory, failing it on the final roll.
 * Then try loading from the point of the 3rd roll. Verify that we read 
 * the correct number of transactions from this point.
 */
@Test 
public void testReadFromStream() throws IOException {
  File f = new File(TestEditLog.TEST_DIR + "/readfromstream");
  // abort after 10th roll
  NNStorage storage = setupEdits(Collections.<URI>singletonList(f.toURI()),
                                 10, new AbortSpec(10, 0));
  StorageDirectory sd = storage.dirIterator(NameNodeDirType.EDITS).next();

  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  long expectedTotalTxnCount = TXNS_PER_ROLL*10 + TXNS_PER_FAIL;
  assertEquals(expectedTotalTxnCount, getNumberOfTransactions(jm, 1,
      true, false));

  long skippedTxns = (3*TXNS_PER_ROLL); // skip first 3 files
  long startingTxId = skippedTxns + 1; 

  long numLoadable = getNumberOfTransactions(jm, startingTxId,
      true, false);
  assertEquals(expectedTotalTxnCount - skippedTxns, numLoadable); 
}
 
开发者ID:naver,项目名称:hadoop,代码行数:27,代码来源:TestFileJournalManager.java


示例4: testManyLogsWithCorruptInprogress

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/** 
 * Test that we can load an edits directory with a corrupt inprogress file.
 * The corrupt inprogress file should be moved to the side.
 */
@Test
public void testManyLogsWithCorruptInprogress() throws IOException {
  File f = new File(TestEditLog.TEST_DIR + "/manylogswithcorruptinprogress");
  NNStorage storage = setupEdits(Collections.<URI>singletonList(f.toURI()), 10, new AbortSpec(10, 0));
  StorageDirectory sd = storage.dirIterator(NameNodeDirType.EDITS).next();

  File[] files = new File(f, "current").listFiles(new FilenameFilter() {
      @Override
      public boolean accept(File dir, String name) {
        if (name.startsWith("edits_inprogress")) {
          return true;
        }
        return false;
      }
    });
  assertEquals(files.length, 1);
  
  corruptAfterStartSegment(files[0]);

  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  assertEquals(10*TXNS_PER_ROLL+1, 
               getNumberOfTransactions(jm, 1, true, false));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:28,代码来源:TestFileJournalManager.java


示例5: testReadFromStream

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/** 
 * Test that we can read from a stream created by FileJournalManager.
 * Create a single edits directory, failing it on the final roll.
 * Then try loading from the point of the 3rd roll. Verify that we read 
 * the correct number of transactions from this point.
 */
@Test 
public void testReadFromStream() throws IOException {
  File f = new File(TestEditLog.TEST_DIR + "/filejournaltest1");
  // abort after 10th roll
  NNStorage storage = setupEdits(Collections.<URI>singletonList(f.toURI()),
                                 10, new AbortSpec(10, 0));
  StorageDirectory sd = storage.dirIterator(NameNodeDirType.EDITS).next();

  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  long expectedTotalTxnCount = TXNS_PER_ROLL*10 + TXNS_PER_FAIL;
  assertEquals(expectedTotalTxnCount, getNumberOfTransactions(jm, 1,
      true, false));

  long skippedTxns = (3*TXNS_PER_ROLL); // skip first 3 files
  long startingTxId = skippedTxns + 1; 

  long numLoadable = getNumberOfTransactions(jm, startingTxId,
      true, false);
  assertEquals(expectedTotalTxnCount - skippedTxns, numLoadable); 
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:27,代码来源:TestFileJournalManager.java


示例6: testManyLogsWithCorruptInprogress

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/** 
 * Test that we can load an edits directory with a corrupt inprogress file.
 * The corrupt inprogress file should be moved to the side.
 */
@Test
public void testManyLogsWithCorruptInprogress() throws IOException {
  File f = new File(TestEditLog.TEST_DIR + "/filejournaltest5");
  NNStorage storage = setupEdits(Collections.<URI>singletonList(f.toURI()), 10, new AbortSpec(10, 0));
  StorageDirectory sd = storage.dirIterator(NameNodeDirType.EDITS).next();

  File[] files = new File(f, "current").listFiles(new FilenameFilter() {
      @Override
      public boolean accept(File dir, String name) {
        if (name.startsWith("edits_inprogress")) {
          return true;
        }
        return false;
      }
    });
  assertEquals(files.length, 1);
  
  corruptAfterStartSegment(files[0]);

  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  assertEquals(10*TXNS_PER_ROLL+1, 
               getNumberOfTransactions(jm, 1, true, false));
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:28,代码来源:TestFileJournalManager.java


示例7: testInprogressRecovery

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/**
 * Test that inprogress files are handled correct. Set up a single
 * edits directory. Fail on after the last roll. Then verify that the 
 * logs have the expected number of transactions.
 */
@Test
public void testInprogressRecovery() throws IOException {
  File f = new File(TestEditLog.TEST_DIR + "/inprogressrecovery");
  // abort after the 5th roll 
  NNStorage storage = setupEdits(Collections.<URI>singletonList(f.toURI()),
                                 5, new AbortSpec(5, 0));
  StorageDirectory sd = storage.dirIterator(NameNodeDirType.EDITS).next();

  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  assertEquals(5*TXNS_PER_ROLL + TXNS_PER_FAIL, 
               getNumberOfTransactions(jm, 1, true, false));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:TestFileJournalManager.java


示例8: testInprogressRecoveryAll

import org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec; //导入依赖的package包/类
/** 
 * Test that FileJournalManager behaves correctly despite inprogress
 * files in all its edit log directories. Set up 3 directories and fail
 * all on the last roll. Verify that the correct number of transaction 
 * are then loaded.
 */
@Test
public void testInprogressRecoveryAll() throws IOException {
  File f1 = new File(TestEditLog.TEST_DIR + "/failalltest0");
  File f2 = new File(TestEditLog.TEST_DIR + "/failalltest1");
  File f3 = new File(TestEditLog.TEST_DIR + "/failalltest2");
  
  List<URI> editUris = ImmutableList.of(f1.toURI(), f2.toURI(), f3.toURI());
  // abort after the 5th roll 
  NNStorage storage = setupEdits(editUris, 5, 
                                 new AbortSpec(5, 0),
                                 new AbortSpec(5, 1),
                                 new AbortSpec(5, 2));
  Iterator<StorageDirectory> dirs = storage.dirIterator(NameNodeDirType.EDITS);
  StorageDirectory sd = dirs.next();
  FileJournalManager jm = new FileJournalManager(conf, sd, storage);
  assertEquals(5*TXNS_PER_ROLL + TXNS_PER_FAIL, getNumberOfTransactions(jm, 1,
      true, false));
  
  sd = dirs.next();
  jm = new FileJournalManager(conf, sd, storage);
  assertEquals(5*TXNS_PER_ROLL + TXNS_PER_FAIL, getNumberOfTransactions(jm, 1,
      true, false));

  sd = dirs.next();
  jm = new FileJournalManager(conf, sd, storage);
  assertEquals(5*TXNS_PER_ROLL + TXNS_PER_FAIL, getNumberOfTransactions(jm, 1,
      true, false));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:35,代码来源:TestFileJournalManager.java



注:本文中的org.apache.hadoop.hdfs.server.namenode.TestEditLog.AbortSpec类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java PrincipalEntry类代码示例发布时间:2022-05-22
下一篇:
Java AllocateResponseProtoOrBuilder类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap