C++ Toker类代码示例

OGeek|极客世界-中国程序员成长平台 › 门户 › 编程› C++›C++教程

原作者: [db:作者] 来自: [db:来源] 收藏邀请

本文整理汇总了C++中Toker类的典型用法代码示例。如果您正苦于以下问题：C++ Toker类的具体用法？C++ Toker怎么用？C++ Toker使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。

在下文中一共展示了Toker类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的C++代码示例。

示例1: main

int main()
{
	std::cout << "-----------------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 1 -" << " " << " I have used Microsoft Visual Studio C++ Console Projects" << std::endl;
	std::cout << "-------------------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 2 -" << " " << " I have used <iostream> for all I/O and new and delete for heap based management " << std::endl;
	std::cout << "----------------------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 3 -" << " " << " I have provided Tokenzier Package, SemiExp package whose .ccp implements an interface named iTokCollection.h" << std::endl;
	std::cout << "------------------------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 4 -" << " " << "Tokenizing Test File into Alphanumeric, Punctuator, Special One Character, Special Two Character, C Style Comments, C++ Comments and Quoted Strings" << std::endl;
	std::cout << "-------------------------------------------------------------------------------------------------------------\n";
	Toker toker;
	int f = dealTokenizer(toker);
	toker.des();
	std::cout << "------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 5 - " << " " << "The getTok() method prodduces one token for each call to it" << std::endl;
	std::cout << "-------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 6 -" << " " << "SemiExp package produces set of tokens by repeatdely calling getTok() until we run into a termination condition " << std::endl;
	std::cout << "--------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 7 - " << " " << "SemiExpressions are terminated on },{,; and : public, :private and :protected " << std::endl;
	std::cout << "---------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 8 - " << " " << "Folding for" << std::endl;
	std::cout << "----------------------------------------------------------------------------------------------\n";
	std::cout << "Output for 5,6,7,8 with same input file passed to Tokennizer with the Comment toggle turned off" << std::endl;
	int g = dealSemiExpressions();
	std::cout << "\n------------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 9 -" << " " << " All the virtual methods in the iTokCollection interface have been implemented by the SemiExp Class " << std::endl;
	std::cout << "------------------------------------------------------------------------------------------------\n";
	std::cout << "Requirement 10 -" << " " << " TestExecutive which illustrates all possible test cases" << std::endl;
	std::cout << "---------------------------------------------------------------------------------------------------\n";
	std::cout << "Please enter any character to continue " << std::endl;

	std::getchar();
	
}

开发者ID:sidmarkjoseph，项目名称:C--piggybank，代码行数:35，代码来源:TestExecutive.cpp

示例2: dealSemiExpressions

int dealSemiExpressions()
{
	Toker tok;
	std::string fileSpec1 = "../Tokenizer/test_tokenizer.txt";
	std::fstream inn(fileSpec1);
	if (!inn.good())
	{
		std::cout << "\n  can't open file " << fileSpec1 << "\n\n";
		return 1;
	}
	tok.attach(&inn);
	tok.setComments(false);
	tok.set("<,>,[,],(,),{,},:,=,+,-,*,\n", "<,<,>,>,:,:,+,+,-,-,=,=,+,=,-,=,*,=,/,=,-,>");
	SemiExp semi(&tok);
	semi.get();

	while (semi.get())
	{
		std::cout << "\n  -- semiExpression --";
		std::string str = semi.show();
		std::cout << str;
	}

	if (semi.length() > 0)
	{
		std::cout << "\n  -- semiExpression --";
		semi.show();
		std::cout << "\n\n";
	}
	return 0;
}

开发者ID:sidmarkjoseph，项目名称:C--piggybank，代码行数:31，代码来源:TestExecutive.cpp

示例3: testSemiExpOnCpp

int testSemiExpOnCpp(std::string fileSpecSemi) {
	Toker tokercpp;
	std::fstream insemicpp(fileSpecSemi);
	if (!insemicpp.good())
	{
		std::cout << "\n  can't open file " << fileSpecSemi << "\n\n";
		return 1;
	}
	tokercpp.attach(&insemicpp);

	SemiExp semi(&tokercpp);
	while (semi.get())
	{
		std::cout << "\n  -- semiExpression --";
		semi.show();
	}

	if (semi.length() > 0)
	{
		std::cout << "\n  -- semiExpression --";
		semi.show();
		std::cout << "\n\n";
	}
	Toker::reset();
	return 0;
}

开发者ID:lovingmage，项目名称:SPA-Platform，代码行数:26，代码来源:TestExecutive.cpp

示例4: AnalyzeFileDepedency

void CodeAnalyzer::AnalyzeFileDepedency(std::string file) {
	Toker toker;
	std::fstream in(file);
	// Check if the filename is valid
	if (!in.good())
	{
		std::cout << "\n can't open file " << file << "\n\n";
		return;
	}

	toker.attach(&in);
	do
	{
		std::string tok = toker.getTok();
		ProcessTok(file, tok);										// Check for match in Final Type Table and perform corresponding action
	} while (in.good());
}

开发者ID:bharanikrishna7，项目名称:code_analyzer，代码行数:17，代码来源:CodeAnalyzer.cpp

示例5: main

///////////////////////////
// Main Test Stub function
int main()
{
	Timer time;
	time.StartClock();

	StringHelper::Title("\n Testing SemiExp class", '=');
	
	Toker toker;
	toker.setVerbose(true);
	std::string fileSpec = "SemiExp.cpp";

	std::fstream in(fileSpec);
	if (!in.good())
	{
		std::cout << "\n Unable to read file \"" << fileSpec << "\"\n\n";
		return 1;
	}
	else
	{
		std::cout << "\n Processing file \"" << fileSpec << "\"\n";
	}
	toker.attach(&in);

	SemiExp semi(&toker);
	semi.setVerbose(true);
	while (semi.get())
	{
		std::cout << "\n -- semiExpression -- at line number " << semi.currentLineCount();
		std::cout << semi.show();
	}
	/*
	May have collected tokens, but reached end of stream
	before finding SemiExp terminator.
	*/
	if (semi.length() > 0)
	{
		std::cout << "\n  -- semiExpression --";
		semi.show(true);
	}
	std::cout << "\n\n\n [Execution Time] : " << time.StopClock() << " ms";
	std::cout << "\n\n ";
	system("pause");
	return 0;
}

开发者ID:bharanikrishna7，项目名称:code_analyzer，代码行数:46，代码来源:SemiExp.cpp

示例6: main1

	int main1() {

		showInit();
		std::ifstream in(fileSpec);
		if (!in.good()) {
			std::cout << "\n  can't open " << fileSpec << "\n\n";
		}
		Toker toker;

		toker.attach(&in);
		std::ofstream myfile;
		tokenizerSetUp(toker);
		while (in.good()) {
			std::string* toks = toker.getToks();
			std::cout << "\n--" << toks[1];
			if (toks[1] != "NewLine") {
				std::cout << "\n" << toks[0];
			}
		}
		std::cout << "\n\n";

		///////////
		cout << "\n-------------------------Semi Exp------------------------";
		Toker toker1;
		std::string fileSpec1 = "/home/malz/workspace/new_workspace/testtok.cpp";
		std::fstream in1(fileSpec1);
		if (!in1.good()) {
			std::cout << "\n  can't open file " << fileSpec1 << "\n\n";
			return 1;
		}
		toker1.attach(&in1);
		SemiExp semi(&toker1);
		tokenizerSetUp(toker1);
		while (semi.get1(true)) {
			std::cout << "\n  -- semiExpression --";
			cout<<semi.show1(true);
		}
		if (semi.length() > 0) {
			std::cout << "\n  -- semiExpression --";
			semi.show();
			std::cout << "\n\n";
		}
		return 0;
	}

开发者ID:malapatiravi，项目名称:CppParser，代码行数:44，代码来源:TestExec.cpp

示例7: dealTokenizer

int dealTokenizer(Toker& toker)
{
	std::cout << "Input for Tokenizer" << std::endl;
	std::string fileSpec = "../Tokenizer/test_tokenizer.txt";
	std::ifstream file("../Tokenizer/test_tokenizer.txt");

	std::string str;
	while (std::getline(file, str))
	{
		std::cout << str << std::endl;
	}

	std::cout << "-----------------------------------------------------------------------------------------\n";

	std::ifstream in(fileSpec);
	if (!in.good())
	{
		std::cout << "\n  can't open " << fileSpec << "\n\n";
		return 1;
	}
	
	toker.attach(&in);
	toker.setComments(true);
	toker.set("<,>,[,],(,),{,},:,=,+,-,*,\n", "<,<,>,>,:,:,+,+,-,-,=,=,+,=,-,=,*,=,/,=,-,>");
	std::cout << "OUPUT \n";
	do
	{
		std::string tok = toker.getTok();
		if (tok == "\n")
		{
			tok = "newline";
		
		}
		std::cout << "\n -- " << tok;
	 
	} while (in.good());
	std::cout << "\n";
	
	return 0;
}

开发者ID:sidmarkjoseph，项目名称:C--piggybank，代码行数:40，代码来源:TestExecutive.cpp

示例8: main

int main()
{
  std::cout << "\n  Testing SemiExp class";
  std::cout << "\n =======================\n";

  Toker toker;
  //std::string fileSpec = "../Tokenizer/Tokenizer.cpp";
  std::string fileSpec = "SemiExp.cpp";

  std::fstream in(fileSpec);
  if (!in.good())
  {
    std::cout << "\n  can't open file " << fileSpec << "\n\n";
    return 1;
  }
  else
  {
    std::cout << "\n  processing file \"" << fileSpec << "\"\n";
  }
  toker.attach(&in);

  SemiExp semi(&toker);
  while(semi.get())
  {
    std::cout << "\n  -- semiExpression -- at line number " << semi.currentLineCount();
    std::cout << semi.show();
  }
  /*
     May have collected tokens, but reached end of stream
     before finding SemiExp terminator.
   */
  if (semi.length() > 0)
  {
    std::cout << "\n  -- semiExpression --";
    semi.show(true);
  }
  std::cout << "\n\n";
  return 0;
}

开发者ID:abhilash215，项目名称:Code-Parser-with-Abstract-Syntax-Tree，代码行数:39，代码来源:SemiExp.cpp

示例9: main

int main()
{
  //std::string fileSpec = "../Tokenizer/Tokenizer.cpp";
  //std::string fileSpec = "../Tokenizer/Tokenizer.h";
  std::string fileSpec = "../Tokenizer/Test.txt";

  std::ifstream in(fileSpec);
  if (!in.good())
  {
    std::cout << "\n  can't open " << fileSpec << "\n\n";
    return 1;
  }
  Toker toker;
  toker.attach(&in);
  while (in.good())
  {
    std::string tok = toker.getTok();
    if (tok == "\n")
      tok = "newline";
    std::cout << "\n -- " << tok;
  }
  std::cout << "\n\n";
  return 0;
}

开发者ID:alarya，项目名称:LexicalScannerCPP，代码行数:24，代码来源:Tokenizer.cpp

示例10: main

int main()
{
	/*std::fstream fIn;
	fIn.open("C:\\Users\thzha_000\Desktop\finaltest.txt", std::ios::in);*/

	//Tokenizer testing
	std::string fileSpec = "../Tokenizer/test_R4.txt";

	std::cout << "---------------------------------------" << "\n";
	std::cout << "---------  Testing Tokenizer ----------" << "\n";
	std::cout << "---------------------------------------" << "\n";
	std::cout << "--------Operating authentic code-------" << "\n";
	std::cout << "Requirements 5 is well shown in this process" << "\n\n";

	std::ifstream in(fileSpec);
	if (!in.good())
	{
		std::cout << "\n  can't open " << fileSpec << "\n\n";
		return 1;
	}
	else
		std::cout << "\n Read file successfull!" << "\n";
	Toker toker;
	toker.attach(&in);
	do
	{
		std::string tok = toker.getTok();
		if (tok == "\n")
			tok = "newline";
		std::cout << "\n -- " << tok;
	} while (in.good());
	std::cout << "\n\n";

	toker.reset();


	// SemiExp testing
	Toker toker2;
	std::cout << "---------------------------------------" << "\n";
	std::cout << "------- Testing SemiExpression --------" << "\n";
	std::cout << "---------------------------------------" << "\n";
	std::cout << "--------Operating authentic code-------" << "\n\n";
	std::cout << "Requirements 6-8 is well shown in this process" << "\n\n";

	std::fstream in2(fileSpec);
	if (!in2.good())
	{
		std::cout << "\n  can't open file " << fileSpec << "\n\n";
		return 1;
	}
	else
		std::cout << "\n Read file successfull!" << "\n";
	toker2.attach(&in2);
	SemiExp semi(&toker2);
	while (semi.get())
	{
		std::cout << "\n  -------------- semiExpression --------------";
		semi.show();
	}

	/*
	May have collected tokens, but reached end of stream
	before finding SemiExp terminator.
	*/
	if (semi.length() > 0)
	{
		std::cout << "\n  -------------- semiExpression --------------";
		semi.show();
		std::cout << "\n\n";
	}

	toker2.reset();

	// Test Requirement 4 

	std::string fileSpec_R4 = "../Tokenizer/test_R4.txt";

	std::vector<std::string> alter_setSpecialSingleChar = { "=", "+" };
	std::vector<std::string> alter_setSpecialDoubleChar = { "++" , "--" };

	std::cout << "------------------- Requiement 4 ------------------" << "\n";
	std::cout << "---------------------------------------------------" << "\n";
	std::cout << "------Using a small piece of text for testing------" << "\n\n";
	Toker toker1;
	std::ifstream in1(fileSpec_R4);
	if (!in1.good())
	{
		std::cout << "\n  can't open " << fileSpec << "\n\n";
		return 1;
	}
	else
		std::cout << "\n Read file successfull!" << "\n";
	toker.attach(&in1);
	do
	{
		std::string tok = toker1.getTok();
		if (tok == "\n")
			tok = "newline";
		std::cout << "\n -- " << tok;
	} while (in1.good());
//.........这里部分代码省略.........

开发者ID:WillZzzz，项目名称:Code-Tokenizer，代码行数:101，代码来源:TestPackage.cpp

示例11: main

void main()
{
	try
	{
	std::stack<PNode<XMLValue>*> *nStack = new std::stack<PNode<XMLValue>*>;
	PNode<XMLValue>* rootnode = new PNode<XMLValue>(XMLValue(XMLValue::TAG,"Document"));
	XMLTree* xtree = new XMLTree(rootnode);
	ParserState* ps = new ParserState;
	
	Toker* pToker = new Toker;
	pToker->setMode(Toker::xml);          //XML parser mode
    XmlParts* pSemi = new XmlParts(pToker);
    Parser* pParser = new Parser(pSemi);

	ElementEnd* pElementendrule = new ElementEnd(nStack,ps);
	ElementStart* pElementstartrule = new ElementStart(nStack,ps);
	ElementSelfClosed* pElementselfcrule = new ElementSelfClosed(nStack,ps);
	Text* ptextrule = new Text(nStack,ps);
	Comment* pCommentrule = new Comment(nStack,ps);
	ProcInstr* pIinstrrule = new ProcInstr(nStack,ps);

	EndFound* pEndact = new EndFound(nStack,xtree);
	StartFound* pStartact = new StartFound(nStack,xtree);
	SelfClosedFound* pSelfClosedact = new SelfClosedFound(nStack,xtree);
	TextFound* pTextact = new TextFound(nStack,xtree);
	CommentFound* pCommentact = new CommentFound(nStack,xtree);
	ProcInstrFound* pInstract = new ProcInstrFound(nStack,xtree);
	
	pElementendrule->addAction(pEndact);
	pElementstartrule->addAction(pStartact);
	pElementselfcrule->addAction(pSelfClosedact);
	ptextrule->addAction(pTextact);
	pCommentrule->addAction(pCommentact);
	pIinstrrule->addAction(pInstract);
	
	pParser->addRule(pElementendrule);
	pParser->addRule(pElementstartrule);
	pParser->addRule(pElementselfcrule);
	pParser->addRule(ptextrule);
	pParser->addRule(pCommentrule);
	pParser->addRule(pIinstrrule);
	    

	nStack->push(rootnode);

	if(pParser)
    {
		if(! pToker->attach("..\\LectureNote.xml"))
        {
	      std::cout << "\n  could not open file "  << std::endl;
          return;
        }
     }

	while(pParser->next())
       pParser->parse();
	
	//display the tree and the rebuilt xml
	XMLDisplay disply(xtree);
	//generate the string of tree and rebuilt xml
	disply.buildTreeDisplay();
	disply.rebuildXML();
	//attach the output file to the display object
	disply.AttachFile("..\\output.txt");
	//display them
	std::cout<<"=======================Parser Tree====================="<<std::endl;
	std::cout<<disply.OutputTree();
	
      std::cout << "\n\n";

	std::cout<<"=====================Rebuild XML====================="<<std::endl;
	std::cout<<disply.OutputXML();
	
	delete pEndact;
	delete pStartact;
	delete pSelfClosedact;
	delete pTextact;
	delete pCommentact;
	delete pInstract;
	
	delete pElementendrule;
	delete pElementstartrule;
	delete pElementselfcrule;
	delete ptextrule;
	delete pCommentrule;
	delete pIinstrrule;

	delete pToker;
	delete pSemi;
	delete pParser;

	delete ps;
	delete nStack;
	
	xtree->remove(xtree->getroot());
	std::vector< PNode<XMLValue>* > deletednodes = xtree->getdeletednodes();
	for(size_t i = 0; i < deletednodes.size(); ++i)
		delete deletednodes[i];
	delete xtree;

//.........这里部分代码省略.........

开发者ID:zhuzhu9910，项目名称:AcademicProjects，代码行数:101，代码来源:XMLDisplay.cpp

示例12: main

int main(int argc, char* argv[])
{
	std::cout << "\n  Testing Tokenizer class\n "
		<< std::string(25, '=') << std::endl;
	std::cout
		<< "\n  Note that comments and quotes are returned as single tokens\n\n";

	// collecting tokens from a string

	Toker t_fromStr("tokens from a string: -> int x; /* a comment */", false);
	std::string tok;
	do
	{
		tok = t_fromStr.getTok();
		std::cout << "  " << tok;
	} while (tok != "");
	std::cout << "\n\n";

	// collecting tokens from files, named on the command line

	if (argc < 2)
	{
		std::cout
			<< "\n  please enter name of file to process on command line\n\n";
		return 1;
	}

	for (int i = 1; i<argc; ++i)
	{
		std::cout << "\n  Processing file " << argv[i];
		std::cout << "\n  " << std::string(16 + strlen(argv[i]), '-') << "\n";

		try
		{
			Toker t;
			t.setMode(Toker::xml);        // comment out to show tokenizing for code
			//    t.setSingleCharTokens("<>");  // will return same results as above for XML

			if (!t.attach(argv[i]))
			{
				std::cout << "\n    can't open file " << argv[i] << "\n\n";
				continue;
			}
			t.returnComments();  // remove this statement to discard comment tokens
			std::string temp;
			do
			{
				temp = t.getTok();
				std::cout << "  ln: " << t.lines() << ", br lev: "
					<< t.braceLevel() << ", tok size: " << std::setw(3) << temp.length() << " -- ";
				if (temp != "\n")
					std::cout << temp << std::endl;
				else
					std::cout << "newline\n";
			} while (temp != "");
		}
		catch (std::exception& ex)
		{
			std::cout << "  " << ex.what() << "\n\n";
		}
	}
}

开发者ID:karantech9，项目名称:XML-Document-Object-Model，代码行数:62，代码来源:Tokenizer.cpp

示例13: main

int main()
{
	{
		Toker toker;
		//std::string fileSpec = "../Tokenizer/Tokenizer.cpp";
		//std::string fileSpec = "../../Tokenizer/test.txt";
		//std::string fileSpec = "../../SemiExp/SemiExp.h";
		std::string fileSpec = "../../Tokenizer/test_cases.txt";

		std::fstream in(fileSpec);
		if (!in.good())
		{
			std::cout << "\n  can't open file " << fileSpec << "\n\n";
			return 1;
		}

		std::cout << "\n File: " << fileSpec << "\n\n";
		toker.attach(&in);

		SemiExp semi(&toker);
		while (semi.get())
		{
			std::cout << "\n  -- semiExpression --";
			semi.show(true);
		}

		if (semi.length() > 0)
		{
			std::cout << "\n  -- semiExpression --";
			semi.show();
			std::cout << "\n\n";
		}
	}

	std::cout << "\n -----Testing ITokcollection functions -----------\n";
	try {

		Test::test0();

		Test::test1();

		Test::test2();

		Test::test3();

		Test::test4();

		Test::test5();

		Test::test6();

		Test::test7();

		Test::test8();

	}
	catch (std::exception& ex)
	{
		std::cout << "\n\n  " << ex.what() << "\n\n";
		return 1;
	}
	return 0;
}

开发者ID:alarya，项目名称:LexicalScannerCPP，代码行数:63，代码来源:Executive.cpp

示例14: main

int main()
{
  Helper::Title("Testing Tokenizer");

  //std::string fileSpec = "../Tokenizer/Tokenizer.cpp";
  //std::string fileSpec = "../Tokenizer/Tokenizer.h";
  std::string fileSpec = "../Tokenizer/Test.txt";//from here

  try
  {
    std::ifstream in(fileSpec);
    if (!in.good())
    {
      std::cout << "\n  can't open " << fileSpec << "\n\n";
      return 1;
    }

    {
      Toker toker;
      toker.returnComments();
      toker.attach(&in);
      std::cout << "\n  current line count = " << toker.currentLineCount();
      do
      {
        std::string tok = toker.getTok();
        if (tok == "\n")
          tok = "newline";
        std::cout << "\n -- " << tok;
      } while (in.good());
      std::cout << "\n  current line count = " << toker.currentLineCount();
    }
 
    putline();
    Helper::title("Testing change of special characters");
    std::string newSpecialChars = "., :, +, +=, \n { }";

    Toker toker;
    toker.returnComments();
    toker.setSpecialTokens(newSpecialChars);
    in.clear();

    in.seekg(std::ios::beg);
    toker.attach(&in);
    std::cout << "\n  new special tokens: " << newSpecialChars;
    do
    {
      std::string tok = toker.getTok();
      if (tok == "\n")
        tok = "newline";
      std::cout << "\n -- " << tok;//check if tok is in type table
    } while (in.good());// if there , get filename and say this file is dependent on this file
    std::cout << "\n  current line count = " << toker.currentLineCount();
  }
  catch (std::logic_error& ex)
  {
    std::cout << "\n  " << ex.what();
  }
  std::cout << "\n\n";//here
  return 0;
}

开发者ID:sguvvala，项目名称:ParallelDependancyAnalyser，代码行数:60，代码来源:Tokenizer.cpp

示例15: tokenizerSetUp

	void tokenizerSetUp(Toker& toker) {
		toker.setSpecialCharPairs("<<");
		toker.setSpecialCharPairs("<=");
		toker.setSpecialCharPairs(">=");
		toker.setSpecialCharPairs(">>");
		toker.setSpecialCharPairs("::");
		toker.setSpecialCharPairs("++");
		toker.setSpecialCharPairs("--");
		toker.setSpecialCharPairs("==");
		toker.setSpecialCharPairs("+=");
		toker.setSpecialCharPairs("-=");
		toker.setSpecialCharPairs("*=");
		toker.setSpecialCharPairs("/=");
		toker.setSpecialCharPairs("&&");
		toker.setSpecialSingleChars(">");
		toker.setSpecialSingleChars("<");
		toker.setSpecialSingleChars("[");
		toker.setSpecialSingleChars("]");
		toker.setSpecialSingleChars("(");
		toker.setSpecialSingleChars(")");
		toker.setSpecialSingleChars("{");
		toker.setSpecialSingleChars("}");
		toker.setSpecialSingleChars(":");
		toker.setSpecialSingleChars(";");
		toker.setSpecialSingleChars("=");
		toker.setSpecialSingleChars("+");
		toker.setSpecialSingleChars("-");
		toker.setSpecialSingleChars("*");
		toker.setSpecialSingleChars("\n");
		toker.setTokenTypesFlag("white_spaces");
		toker.setTokenTypesFlag("CppComment-Req4.5");
		toker.setTokenTypesFlag("C Comment-Req4.4");
		toker.setTokenTypesFlag("Punctuator-Req4");
		toker.setTokenTypesFlag("Punctuator-Req-4.2");
		toker.setTokenTypesFlag("character_single_quote_string_Req4.6");
		toker.setTokenTypesFlag("character string-Req4.6");
		toker.setTokenTypesFlag("character_double_quote_string_Req4.6");
		toker.setTokenTypesFlag("Special_two_characters_Req4.3");
		toker.setTokenTypesFlag("Special_Single_character_Req4.3");
		toker.setTokenTypesFlag("Alphanum_Req4");
		toker.setTokenTypesFlag(" ");
	}

开发者ID:malapatiravi，项目名称:CppParser，代码行数:42，代码来源:TestExec.cpp

注：本文中的Toker类示例由纯净天空整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。

鲜花

握手

雷人

路过

鸡蛋

该文章已有0人参与评论

请发表评论

全部评论

专题导读

More+

10-27 六六分期app的软件客服如何联系？(六六分期

11-06 可心卡盟:win10系统火狐flash插件崩溃怎么

11-06 亲亲特价:怎么删除回收站图标

11-06 济南大学虚拟社区:鲁大师节能降温的具体办

11-06 xlueops.exe:无线网络安装向导

11-06 女斗合众国:win7系统cf与主机连接不稳定怎

11-06 0xc000022-[cf烟雾头]cf怎么调烟雾头

11-06 qizideyouhuo:应用程序无法正常启动0xc0000

11-06 ipz-185:win7系统vcf文件怎么打开

11-06 傻哥蹦迪:win10系统s4怎么打开usb调试

11-06 八神浩树gtaste:回收站清空了怎么恢复

11-06 妖尾之黑色守护:win10系统电脑没有1440x900

11-06 校园至尊魔王小说:win7系统浏览网页时字体

11-06 女斗合众国:win10系统访问共享文件夹提示请

11-06 tokyo hot n0654:恢复win7系统默认字体一招

11-06 雨酷仙境:设置win7系统转移临时文件夹腾出

11-06 阿穆纳伊之杖:win7系统开始菜单在右边还原

11-06 tunespotting:win10系统火狐flash插件总是

11-06 甘尔葛分析师：计谋网站seo关键词暴涨有什

11-06 蔡贵霖: 计谋网站seo关键词暴涨有什么秘密

11-06 博益网首页:ao3网页版进入不了解决方法

11-06 漏斗子专栏: 网站数据分析小白易懂精华篇

11-06 见证双虹怎么做:win7系统开启telnet命令的

11-06 颾狐蝶蜋:系统资源不足无法完成请求的服务

11-06 国光中学校歌:提交网站到alexa查询详细步骤

11-06 西安有情天:静态网页和动态网页的区别

11-06 红木雅尚斋:外部链接构造对网站的好处

11-06 前官礼遇：防止域名劫持–增强域安全性的10

11-06 密传二转答案: 中文分词算法有哪些

11-06 金泉家园邮编:百度快照劫持的表现及应对方

C++ Tol类代码示例发布时间：2022-05-31

C++ Tokens类代码示例发布时间：2022-05-31

剪的笔顺,诠释剪的笔画,认识剪的部首

1 六六分期app的软件客服如何联系？(六六分期

六六分期app的软件客服如何联系？不知道吗？加qq群【895510560】即可！标题：六六分期

阅读：18222|2023-10-27

2 可心卡盟:win10系统火狐flash插件崩溃怎么

今天小编告诉大家如何处理win10系统火狐flash插件总是崩溃的问题，可能很多用户都不知

阅读：9658|2022-11-06

3 亲亲特价:怎么删除回收站图标

今天小编告诉大家如何对win10系统删除桌面回收站图标进行设置，可能很多用户都不知道

阅读：8173|2022-11-06

4 济南大学虚拟社区:鲁大师节能降温的具体办

今天小编告诉大家如何对win10系统电脑设置节能降温的设置方法，想必大家都遇到过需要

阅读：8543|2022-11-06

5 xlueops.exe:无线网络安装向导

我们在使用xp系统的过程中,经常需要对xp系统无线网络安装向导设置进行设置，可能很多

阅读：8450|2022-11-06

6 女斗合众国:win7系统cf与主机连接不稳定怎

今天小编告诉大家如何处理win7系统玩cf老是与主机连接不稳定的问题，可能很多用户都不

阅读：9375|2022-11-06

7 0xc000022-[cf烟雾头]cf怎么调烟雾头

电脑对日常生活的重要性小编就不多说了，可是一旦碰到win7系统设置cf烟雾头的问题，很

阅读：8420|2022-11-06

8 qizideyouhuo:应用程序无法正常启动0xc0000

我们在日常使用电脑的时候，有的小伙伴们可能在打开应用的时候会遇见提示应用程序无法

阅读：7855|2022-11-06

9 ipz-185:win7系统vcf文件怎么打开

今天小编告诉大家如何对win7系统打开vcf文件进行设置，可能很多用户都不知道怎么对win

阅读：8405|2022-11-06

10 傻哥蹦迪:win10系统s4怎么打开usb调试

今天小编告诉大家如何对win10系统s4开启USB调试模式进行设置，可能很多用户都不知道怎

阅读：7392|2022-11-06

客服电话

电子邮件

C++ Toker类代码示例

示例1: main

示例2: dealSemiExpressions

示例3: testSemiExpOnCpp

示例4: AnalyzeFileDepedency

示例5: main

示例6: main1

示例7: dealTokenizer

示例8: main

示例9: main

示例10: main

示例11: main

示例12: main

示例13: main

示例14: main

示例15: tokenizerSetUp

请发表评论

全部评论

上一篇：

下一篇：

librespeed/speedtest: Self-hosted Speedt

CVE-2022-34965

avehtari/BDA_m_demos: Bayesian Data Anal

四维彩超怎么看性别？四维看男孩女孩诀窍

个体户条例实施细则，2020 最新个体工商户

剪的笔顺,诠释剪的笔画,认识剪的部首

六六分期app的软件客服如何联系？(六六分期

florent37/ViewAnimator: A fluent Android

florent37/Shrine-MaterialDesign2: implem

CVE-2020-36276

SimpleSoftwareIO/simple-sms: Send and re

关于我们

产品与服务

解决方案

139-2527-9053