• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号
登陆 注册
描述文字

Java TokenScanner类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.eclipse.jdt.internal.corext.dom.TokenScanner类的典型用法代码示例。如果您正苦于以下问题：Java TokenScanner类的具体用法？Java TokenScanner怎么用？Java TokenScanner使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



TokenScanner类属于org.eclipse.jdt.internal.corext.dom包，在下文中一共展示了TokenScanner类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: getSurroundingComment

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private int getSurroundingComment(IScanner scanner) {
  try {
    int start = fLocation.getOffset();
    int end = start + fLocation.getLength();

    int token = scanner.getNextToken();
    while (token != ITerminalSymbols.TokenNameEOF) {
      if (TokenScanner.isComment(token)) {
        int currStart = scanner.getCurrentTokenStartPosition();
        int currEnd = scanner.getCurrentTokenEndPosition() + 1;
        if (currStart <= start && end <= currEnd) {
          return token;
        }
      }
      token = scanner.getNextToken();
    }

  } catch (InvalidInputException e) {
    // ignore
  }
  return ITerminalSymbols.TokenNameEOF;
}
开发者ID:eclipse，项目名称:che，代码行数:23，代码来源:TaskMarkerProposal.java


示例2: getSurroundingComment

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private int getSurroundingComment(IScanner scanner) {
	try {
		int start= fLocation.getOffset();
		int end= start + fLocation.getLength();

		int token= scanner.getNextToken();
		while (token != ITerminalSymbols.TokenNameEOF) {
			if (TokenScanner.isComment(token)) {
				int currStart= scanner.getCurrentTokenStartPosition();
				int currEnd= scanner.getCurrentTokenEndPosition() + 1;
				if (currStart <= start && end <= currEnd) {
					return token;
				}
			}
			token= scanner.getNextToken();
		}

	} catch (InvalidInputException e) {
		// ignore
	}
	return ITerminalSymbols.TokenNameEOF;
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:23，代码来源:TaskMarkerProposal.java


示例3: isValidComment

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private boolean isValidComment(String template) {
	IScanner scanner = ToolFactory.createScanner(true, false, false, false);
	scanner.setSource(template.toCharArray());
	try {
		int next = scanner.getNextToken();
		while (TokenScanner.isComment(next)) {
			next = scanner.getNextToken();
		}
		return next == ITerminalSymbols.TokenNameEOF;
	} catch (InvalidInputException e) {
	}
	return false;
}
开发者ID:eclipse，项目名称:eclipse.jdt.ls，代码行数:14，代码来源:CodeTemplateContextType.java


示例4: StatementAnalyzer

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
public StatementAnalyzer(ICompilationUnit cunit, Selection selection, boolean traverseSelectedNode) throws CoreException {
	super(selection, traverseSelectedNode);
	Assert.isNotNull(cunit);
	fCUnit = cunit;
	fStatus = new RefactoringStatus();
	fScanner = new TokenScanner(fCUnit);
}
开发者ID:eclipse，项目名称:eclipse.jdt.ls，代码行数:8，代码来源:StatementAnalyzer.java


示例5: isValidComment

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private boolean isValidComment(String template) {
  IScanner scanner = ToolFactory.createScanner(true, false, false, false);
  scanner.setSource(template.toCharArray());
  try {
    int next = scanner.getNextToken();
    while (TokenScanner.isComment(next)) {
      next = scanner.getNextToken();
    }
    return next == ITerminalSymbols.TokenNameEOF;
  } catch (InvalidInputException e) {
  }
  return false;
}
开发者ID:eclipse，项目名称:che，代码行数:14，代码来源:CodeTemplateContextType.java


示例6: visit

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
@Override
public boolean visit(LambdaExpression node) {
  Selection selection = getSelection();
  int selectionStart = selection.getOffset();
  int selectionExclusiveEnd = selection.getExclusiveEnd();
  int lambdaStart = node.getStartPosition();
  int lambdaExclusiveEnd = lambdaStart + node.getLength();
  ASTNode body = node.getBody();
  int bodyStart = body.getStartPosition();
  int bodyExclusiveEnd = bodyStart + body.getLength();

  boolean isValidSelection = false;
  if ((body instanceof Block)
      && (bodyStart < selectionStart && selectionExclusiveEnd <= bodyExclusiveEnd)) {
    // if selection is inside lambda body's block
    isValidSelection = true;
  } else if (body instanceof Expression) {
    try {
      TokenScanner scanner = new TokenScanner(fCUnit);
      int arrowExclusiveEnd =
          scanner.getTokenEndOffset(ITerminalSymbols.TokenNameARROW, lambdaStart);
      if (selectionStart >= arrowExclusiveEnd) {
        isValidSelection = true;
      }
    } catch (CoreException e) {
      // ignore
    }
  }
  if (selectionStart <= lambdaStart && selectionExclusiveEnd >= lambdaExclusiveEnd) {
    // if selection covers the lambda node
    isValidSelection = true;
  }

  if (!isValidSelection) {
    return false;
  }
  return super.visit(node);
}
开发者ID:eclipse，项目名称:che，代码行数:39，代码来源:ExtractMethodAnalyzer.java


示例7: StatementAnalyzer

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
public StatementAnalyzer(
    ICompilationUnit cunit, Selection selection, boolean traverseSelectedNode)
    throws CoreException {
  super(selection, traverseSelectedNode);
  Assert.isNotNull(cunit);
  fCUnit = cunit;
  fStatus = new RefactoringStatus();
  fScanner = new TokenScanner(fCUnit);
}
开发者ID:eclipse，项目名称:che，代码行数:10，代码来源:StatementAnalyzer.java


示例8: needsSemicolon

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private boolean needsSemicolon(Expression expression) {
  if ((expression.getParent().getFlags() & ASTNode.RECOVERED) != 0) {
    try {
      TokenScanner scanner = new TokenScanner(getCompilationUnit());
      return scanner.readNext(expression.getStartPosition() + expression.getLength(), true)
          != ITerminalSymbols.TokenNameSEMICOLON;
    } catch (CoreException e) {
      // ignore
    }
  }
  return false;
}
开发者ID:eclipse，项目名称:che，代码行数:13，代码来源:AssignToVariableAssistProposal.java


示例9: isValidComment

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private boolean isValidComment(String template) {
  IScanner scanner = ToolFactory.createScanner(true, false, false, false);
  scanner.setSource(template.toCharArray());
  try {
    int next = scanner.getNextToken();
    while (TokenScanner.isComment(next)) {
      next = scanner.getNextToken();
    }
    return next == ITerminalSymbols.TokenNameEOF;
  } catch (InvalidInputException e) {
    // If there are lexical errors, the comment is invalid
  }
  return false;
}
开发者ID:gwt-plugins，项目名称:gwt-eclipse-plugin，代码行数:15，代码来源:TypeCreator.java


示例10: isValidComment

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private boolean isValidComment(String template) {
	IScanner scanner= ToolFactory.createScanner(true, false, false, false);
	scanner.setSource(template.toCharArray());
	try {
		int next= scanner.getNextToken();
		while (TokenScanner.isComment(next)) {
			next= scanner.getNextToken();
		}
		return next == ITerminalSymbols.TokenNameEOF;
	} catch (InvalidInputException e) {
	}
	return false;
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:14，代码来源:ParameterObjectFactory.java


示例11: visit

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
@Override
public boolean visit(LambdaExpression node) {
	Selection selection= getSelection();
	int selectionStart= selection.getOffset();
	int selectionExclusiveEnd= selection.getExclusiveEnd();
	int lambdaStart= node.getStartPosition();
	int lambdaExclusiveEnd= lambdaStart + node.getLength();
	ASTNode body= node.getBody();
	int bodyStart= body.getStartPosition();
	int bodyExclusiveEnd= bodyStart + body.getLength();

	boolean isValidSelection= false;
	if ((body instanceof Block) && (bodyStart < selectionStart && selectionExclusiveEnd <= bodyExclusiveEnd)) {
		// if selection is inside lambda body's block
		isValidSelection= true;
	} else if (body instanceof Expression) {
		try {
			TokenScanner scanner= new TokenScanner(fCUnit);
			int arrowExclusiveEnd= scanner.getTokenEndOffset(ITerminalSymbols.TokenNameARROW, lambdaStart);
			if (selectionStart >= arrowExclusiveEnd) {
				isValidSelection= true;
			}
		} catch (CoreException e) {
			// ignore
		}
	}
	if (selectionStart <= lambdaStart && selectionExclusiveEnd >= lambdaExclusiveEnd) {
		// if selection covers the lambda node
		isValidSelection= true;
	}

	if (!isValidSelection) {
		return false;
	}
	return super.visit(node);
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:37，代码来源:ExtractMethodAnalyzer.java


示例12: StatementAnalyzer

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
public StatementAnalyzer(ICompilationUnit cunit, Selection selection, boolean traverseSelectedNode) throws CoreException {
	super(selection, traverseSelectedNode);
	Assert.isNotNull(cunit);
	fCUnit= cunit;
	fStatus= new RefactoringStatus();
	fScanner= new TokenScanner(fCUnit);
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:8，代码来源:StatementAnalyzer.java


示例13: getMemberStartOffset

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private int getMemberStartOffset(IMember curr, IDocument document) throws JavaModelException {
	int offset= curr.getSourceRange().getOffset();
	TokenScanner scanner= new TokenScanner(document, curr.getJavaProject());
	try {
		return scanner.getNextStartOffset(offset, true); // read to the first real non comment token
	} catch (CoreException e) {
		// ignore
	}
	return offset;
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:11，代码来源:AddJavaDocStubOperation.java


示例14: getLocationForFirstToken

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private OccurrenceLocation getLocationForFirstToken(ASTNode node) {
	try {
		int nextEndOffset= new TokenScanner(fASTRoot.getTypeRoot()).getNextEndOffset(node.getStartPosition(), true);
		return new OccurrenceLocation(node.getStartPosition(), nextEndOffset - node.getStartPosition(), 0, fDescription);
	} catch (CoreException e) {
		// ignore
	}
	return new OccurrenceLocation(node.getStartPosition(), node.getLength(), 0, fDescription);
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:10，代码来源:BreakContinueTargetFinder.java


示例15: getNextElseOffset

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private int getNextElseOffset(Statement then, ITypeRoot editorInput) {
	int thenEnd= ASTNodes.getExclusiveEnd(then);
	try {
		TokenScanner scanner= new TokenScanner(editorInput);
		return scanner.getNextStartOffset(thenEnd, true);
	} catch (CoreException e) {
		// ignore
	}
	return -1;
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:11，代码来源:JavaSourceHover.java


示例16: needsSemicolon

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private boolean needsSemicolon(Expression expression) {
	if ((expression.getParent().getFlags() & ASTNode.RECOVERED) != 0) {
		try {
			TokenScanner scanner= new TokenScanner(getCompilationUnit());
			return scanner.readNext(expression.getStartPosition() + expression.getLength(), true) != ITerminalSymbols.TokenNameSEMICOLON;
		} catch (CoreException e) {
			// ignore
		}
	}
	return false;
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:12，代码来源:AssignToVariableAssistProposal.java


示例17: JavaTokenComparator

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
/**
 * Creates a token comparator for the given string.
 *
 * @param text the text to be tokenized
 */
public JavaTokenComparator(String text) {
	Assert.isLegal(text != null);

	fText= text;

	int length= fText.length();
	fStarts= new int[length];
	fLengths= new int[length];
	fCount= 0;

	IScanner scanner= ToolFactory.createScanner(true, true, false, false); // returns comments & whitespace
	scanner.setSource(fText.toCharArray());
	int endPos= 0;
	try {
		int tokenType;
		while ((tokenType= scanner.getNextToken()) != ITerminalSymbols.TokenNameEOF) {
			int start= scanner.getCurrentTokenStartPosition();
			int end= scanner.getCurrentTokenEndPosition()+1;
			// Comments and strings should not be treated as a single token, see https://bugs.eclipse.org/78063
			if (TokenScanner.isComment(tokenType) || tokenType == ITerminalSymbols.TokenNameStringLiteral) {
				// Line comments are often commented code, so lets treat them as code. See https://bugs.eclipse.org/216707
				boolean parseAsJava= tokenType == ITerminalSymbols.TokenNameCOMMENT_LINE;
				int dl= parseAsJava ? getCommentStartTokenLength(tokenType) : 0;
				if (dl > 0)
					recordTokenRange(start, dl);
				parseSubrange(start + dl, text.substring(start + dl, end), parseAsJava);
			} else {
				recordTokenRange(start, end - start);
			}
			endPos= end;
		}
	} catch (InvalidInputException ex) {
		// We couldn't parse part of the input. Fall through and make the rest a single token
	}
	// Workaround for https://bugs.eclipse.org/bugs/show_bug.cgi?id=13907
	if (endPos < length) {
		recordTokenRange(endPos, length - endPos);
	}
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion，代码行数:45，代码来源:JavaTokenComparator.java


示例18: JavaTokenComparator

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
/**
 * Creates a token comparator for the given string.
 *
 * @param text the text to be tokenized
 * @param textTokenComparatorFactory a factory to create text token comparators
 */
public JavaTokenComparator(String text, ITokenComparatorFactory textTokenComparatorFactory) {

	fTextTokenComparatorFactory= textTokenComparatorFactory;
	Assert.isLegal(text != null);

	fText= text;

	int length= fText.length();
	fStarts= new int[length];
	fLengths= new int[length];
	fCount= 0;

	IScanner scanner= ToolFactory.createScanner(true, true, false, false); // returns comments & whitespace
	scanner.setSource(fText.toCharArray());
	int endPos= 0;
	try {
		int tokenType;
		while ((tokenType= scanner.getNextToken()) != ITerminalSymbols.TokenNameEOF) {
			int start= scanner.getCurrentTokenStartPosition();
			int end= scanner.getCurrentTokenEndPosition()+1;
			// Comments are treated as a single token (see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=78063)
			if (TokenScanner.isComment(tokenType) || tokenType == ITerminalSymbols.TokenNameStringLiteral) {
				int dl= fTextTokenComparatorFactory == null ? getCommentStartTokenLength(tokenType) : 0;
				if (dl > 0)
					recordTokenRange(start, dl);
				parseText(start + dl, text.substring(start + dl, end));
			} else {
				recordTokenRange(start, end - start);
			}
			endPos= end;
		}
	} catch (InvalidInputException ex) {
		// We couldn't parse part of the input. Fall through and make the rest a single token
	}
	// Workaround for https://bugs.eclipse.org/bugs/show_bug.cgi?id=13907
	if (endPos < length) {
		recordTokenRange(endPos, length - endPos);
	}
}
开发者ID:trylimits，项目名称:Eclipse-Postfix-Code-Completion-Juno38，代码行数:46，代码来源:JavaTokenComparator.java


示例19: getTokenScanner

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
protected TokenScanner getTokenScanner() {
	return fScanner;
}
开发者ID:eclipse，项目名称:eclipse.jdt.ls，代码行数:4，代码来源:StatementAnalyzer.java


示例20: initializeRanges

import org.eclipse.jdt.internal.corext.dom.TokenScanner; //导入依赖的package包/类
private void initializeRanges() throws CoreException {
  fRanges = new HashMap<ASTNode, SourceRange>();
  if (fSelectedNodes.length == 0) return;

  fRanges.put(fSelectedNodes[0], super.computeSourceRange(fSelectedNodes[0]));
  int last = fSelectedNodes.length - 1;
  fRanges.put(fSelectedNodes[last], super.computeSourceRange(fSelectedNodes[last]));

  IScanner scanner = ToolFactory.createScanner(true, false, false, false);
  char[] source = fDocumentPortionToScan.toCharArray();
  scanner.setSource(source);
  fDocumentPortionToScan = null; // initializeRanges() is only called once

  TokenScanner tokenizer = new TokenScanner(scanner);
  int pos = tokenizer.getNextStartOffset(0, false);

  ASTNode currentNode = fSelectedNodes[0];
  int newStart = Math.min(fSelectionStart + pos, currentNode.getStartPosition());
  SourceRange range = fRanges.get(currentNode);
  fRanges.put(
      currentNode,
      new SourceRange(newStart, range.getLength() + range.getStartPosition() - newStart));

  currentNode = fSelectedNodes[last];
  int scannerStart = currentNode.getStartPosition() + currentNode.getLength() - fSelectionStart;
  tokenizer.setOffset(scannerStart);
  pos = scannerStart;
  int token = -1;
  try {
    while (true) {
      token = tokenizer.readNext(false);
      pos = tokenizer.getCurrentEndOffset();
    }
  } catch (CoreException e) {
  }
  if (token == ITerminalSymbols.TokenNameCOMMENT_LINE) {
    int index = pos - 1;
    while (index >= 0 && IndentManipulation.isLineDelimiterChar(source[index])) {
      pos--;
      index--;
    }
  }

  int newEnd =
      Math.max(fSelectionStart + pos, currentNode.getStartPosition() + currentNode.getLength());
  range = fRanges.get(currentNode);
  fRanges.put(
      currentNode, new SourceRange(range.getStartPosition(), newEnd - range.getStartPosition()));
}
开发者ID:eclipse，项目名称:che，代码行数:50，代码来源:SelectionAwareSourceRangeComputer.java



注：本文中的org.eclipse.jdt.internal.corext.dom.TokenScanner类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。

鲜花
握手
雷人
路过
鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
More+
上一篇：
Java AxisDirection类代码示例发布时间：2022-05-22
下一篇：
Java FDBigInteger类代码示例发布时间：2022-05-22
热门推荐
More+
热门话题
More+
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服（服务时间 9:00～18:00）

在线QQ客服
地址：深圳市南山区西丽大学城创智工业园
电邮：jeky_zhao#qq.com
移动电话：139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap