• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java ListCollector类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.flink.api.common.functions.util.ListCollector的典型用法代码示例。如果您正苦于以下问题:Java ListCollector类的具体用法?Java ListCollector怎么用?Java ListCollector使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



ListCollector类属于org.apache.flink.api.common.functions.util包,在下文中一共展示了ListCollector类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testApply_withDocumentThrowingAnExceptionWithRawDataIncluded

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided an iterable holding a document which misses a required field (with raw data included)
 */
@Test
public void testApply_withDocumentThrowingAnExceptionWithRawDataIncluded() throws Exception {
	
	String content = "{\"field1\": {\"field2\":\"test-value\"}}";
	FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput", 
			new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
	fieldCfg.addAggregationMethod(ContentAggregator.COUNT);

	AggregatorConfiguration cfg = new AggregatorConfiguration();
	cfg.setOutputElement("output");
	cfg.setRaw(true);
	cfg.addFieldAggregation(fieldCfg);

	List<JSONObject> values = new ArrayList<>();
	values.add(new JSONObject(content));
	
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	
	new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
	Assert.assertEquals(1, result.size());
	JSONObject resultObject  = result.get(0);
	Assert.assertEquals("{\"output\":{},\"raw\":[{\"field1\":{\"field2\":\"test-value\"}}]}",resultObject.toString());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:29,代码来源:WindowedJsonContentAggregatorTest.java


示例2: testApply_withDocumentThrowingAnException

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided an iterable holding a document which misses a required field (raw must not be included)
 */
@Test
public void testApply_withDocumentThrowingAnException() throws Exception {
	
	String content = "{\"field1\": {\"field2\":\"test-value\"}}";
	FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput", 
			new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
	fieldCfg.addAggregationMethod(ContentAggregator.COUNT);

	AggregatorConfiguration cfg = new AggregatorConfiguration();
	cfg.setOutputElement("output");
	cfg.addFieldAggregation(fieldCfg);

	List<JSONObject> values = new ArrayList<>();
	values.add(new JSONObject(content));
	
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	
	new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
	Assert.assertEquals(1, result.size());
	JSONObject resultObject  = result.get(0);
	Assert.assertEquals("{\"output\":{}}",resultObject.toString());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:28,代码来源:WindowedJsonContentAggregatorTest.java


示例3: testApply_withDocumentValidValuesRawNotIncluded

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided an iterable holding a document which shows valid values (raw not included)
 */
@Test
public void testApply_withDocumentValidValuesRawNotIncluded() throws Exception {
	
	String content = "{\"field1\": {\"field2\":{\"field3\":\"test\"}}}";
	FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput", 
			new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
	fieldCfg.addAggregationMethod(ContentAggregator.COUNT);

	AggregatorConfiguration cfg = new AggregatorConfiguration();
	cfg.setOutputElement("output");

	cfg.addFieldAggregation(fieldCfg);

	List<JSONObject> values = new ArrayList<>();
	values.add(new JSONObject(content));
	
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	
	new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
	Assert.assertEquals(1, result.size());
	JSONObject resultObject  = result.get(0);
	Assert.assertEquals("{\"output\":{\"fieldOutput\":{\"COUNT\":1}}}",resultObject.toString());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:29,代码来源:WindowedJsonContentAggregatorTest.java


示例4: testApply_withDocumentValidValuesOptionalIncluded

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided an iterable holding a document which shows valid values (optional included)
 */
@Test
public void testApply_withDocumentValidValuesOptionalIncluded() throws Exception {
	
	String content = "{\"field1\": {\"field2\":{\"field3\":\"test\"}}}";
	FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput", 
			new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
	fieldCfg.addAggregationMethod(ContentAggregator.COUNT);

	AggregatorConfiguration cfg = new AggregatorConfiguration();
	cfg.setOutputElement("output");
	cfg.addFieldAggregation(fieldCfg);
	cfg.addOptionalField("counter", WindowedJsonContentAggregator.OPTIONAL_FIELD_TYPE_TOTAL_MESSAGE_COUNT);

	List<JSONObject> values = new ArrayList<>();
	values.add(new JSONObject(content));
	
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	
	new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
	Assert.assertEquals(1, result.size());
	JSONObject resultObject  = result.get(0);
	Assert.assertEquals("{\"output\":{\"fieldOutput\":{\"COUNT\":1}},\"counter\":1}",resultObject.toString());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:29,代码来源:WindowedJsonContentAggregatorTest.java


示例5: executeOnCollections

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Override
protected List<T> executeOnCollections(List<T> inputData, RuntimeContext ctx, ExecutionConfig executionConfig) throws Exception {
	FlatMapFunction<T, T> function = this.userFunction.getUserCodeObject();
	
	FunctionUtils.setFunctionRuntimeContext(function, ctx);
	FunctionUtils.openFunction(function, this.parameters);
	
	ArrayList<T> result = new ArrayList<T>(inputData.size());
	ListCollector<T> collector = new ListCollector<T>(result);

	for (T element : inputData) {
		function.flatMap(element, collector);
	}
	
	FunctionUtils.closeFunction(function);
	
	return result;
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:19,代码来源:FilterOperatorBase.java


示例6: computeOuterJoin

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@SuppressWarnings("unchecked, rawtypes")
private List<Tuple4<String, String, String, Object>> computeOuterJoin(ResettableMutableObjectIterator<Tuple2<String, String>> input1,
																	  ResettableMutableObjectIterator<Tuple2<String, Integer>> input2,
																	  OuterJoinType outerJoinType) throws Exception {
	input1.reset();
	input2.reset();
	AbstractMergeOuterJoinIterator iterator =
			createOuterJoinIterator(
					outerJoinType, input1, input2, serializer1, comparator1, serializer2, comparator2,
					pairComp, this.memoryManager, this.ioManager, PAGES_FOR_BNLJN, this.parentTask
			);

	List<Tuple4<String, String, String, Object>> actual = new ArrayList<>();
	ListCollector<Tuple4<String, String, String, Object>> collector = new ListCollector<>(actual);
	while (iterator.callWithNextKey(new SimpleTupleJoinFunction(), collector)) ;
	iterator.close();

	return actual;
}
 
开发者ID:axbaretto,项目名称:flink,代码行数:20,代码来源:AbstractSortMergeOuterJoinIteratorITCase.java


示例7: testFilterEmbeddingsOnClosingColumn

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testFilterEmbeddingsOnClosingColumn() throws Exception {
  GradoopId a = GradoopId.get();
  GradoopId b = GradoopId.get();

  Embedding embedding = new Embedding();
  embedding.add(a);
  embedding.add(b);

  List<Embedding> result = new ArrayList<>();
  new AdoptEmptyPaths(1, 0).flatMap(embedding, new ListCollector<>(result));
  assertTrue(result.isEmpty());


  new AdoptEmptyPaths(1, 1).flatMap(embedding, new ListCollector<>(result));
  assertEquals(1, result.size());
}
 
开发者ID:dbs-leipzig,项目名称:gradoop,代码行数:18,代码来源:AdoptEmptyPathsTest.java


示例8: testJoin

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Tests the join of the edge with an Embedding
 * (m, e0, n) x edge
 *
 * @param edge the edge the join is performed with
 * @param distinctVertices distinct vertex columns of the base embedding
 * @param distinctEdges distinct edge columns of the base embedding
 * @param closingColumn closing column
 * @param isResult if true it is expected that the join yields exactly one result, 0 otherwise
 * @throws Exception
 */
private void testJoin(Embedding edge, List<Integer> distinctVertices,
  List<Integer> distinctEdges, int closingColumn, boolean isResult) throws Exception {

  Embedding base = new Embedding();
  base.add(m);
  base.add(e0);
  base.add(n);

  EdgeWithTiePoint edgeTuple = new EdgeWithTiePoint(edge);

  CreateExpandEmbedding op = new CreateExpandEmbedding(distinctVertices, distinctEdges, closingColumn);

  List<ExpandEmbedding> results = new ArrayList<>();
  op.join(base, edgeTuple, new ListCollector<>(results));

  assertEquals(isResult ? 1:0, results.size());

  if (isResult) {
    assertEquals(base, results.get(0).getBase());
    assertArrayEquals(new GradoopId[]{edge.getId(1)}, results.get(0).getPath());
    assertEquals(edge.getId(2), results.get(0).getEnd());
  }
}
 
开发者ID:dbs-leipzig,项目名称:gradoop,代码行数:35,代码来源:CreateInitialExpandEmbeddingTest.java


示例9: testStructureBasedProximity

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
    public void testStructureBasedProximity() throws Exception {
        String wikiDocStr = getFileContents("RecommendationPairExtractorTest/article.xml.in");

        DocumentProcessor dp = new DocumentProcessor();
        WikiDocument doc = dp.processDoc(wikiDocStr);

        List<RecommendationPair> list = new ArrayList<>();

        RecommendationPairExtractor rpe = new RecommendationPairExtractor();
        rpe.collectLinkPairsBasedOnStructure(doc, new ListCollector<>(list));

//        System.out.println(list);

        for(RecommendationPair pair: list) {
            assertEquals("Proximity should be paragraph level only", LinkPosition.CPI_PARAGRAPH_LEVEL, pair.getDistance(), 0);
        }
    }
 
开发者ID:wikimedia,项目名称:citolytics,代码行数:19,代码来源:RecommendationPairExtractorTest.java


示例10: executeOnCollections

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Override
protected List<T> executeOnCollections(List<T> inputData, RuntimeContext ctx, boolean mutableObjectSafeMode) throws Exception {
	FlatMapFunction<T, T> function = this.userFunction.getUserCodeObject();
	
	FunctionUtils.setFunctionRuntimeContext(function, ctx);
	FunctionUtils.openFunction(function, this.parameters);
	
	ArrayList<T> result = new ArrayList<T>(inputData.size());
	ListCollector<T> collector = new ListCollector<T>(result);
	
	for (T element : inputData) {
		function.flatMap(element, collector);
	}
	
	FunctionUtils.closeFunction(function);
	
	return result;
}
 
开发者ID:citlab,项目名称:vs.msc.ws14,代码行数:19,代码来源:FilterOperatorBase.java


示例11: testFlatMap_withNullString

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
 * null as input to string parameter
 */
@Test
public void testFlatMap_withNullString() throws Exception {
	@SuppressWarnings("unchecked")
	ListCollector<JSONObject> resultCollector = Mockito.mock(ListCollector.class);
	new StringToJsonObject().flatMap(null, resultCollector);
	Mockito.verify(resultCollector, Mockito.never()).collect(Mockito.anyObject());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:12,代码来源:StringToJsonObjectTest.java


示例12: testFlatMap_withEmptyString

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
 * an empty string as input to string parameter
 */
@Test
public void testFlatMap_withEmptyString() throws Exception {
	@SuppressWarnings("unchecked")
	ListCollector<JSONObject> resultCollector = Mockito.mock(ListCollector.class);
	new StringToJsonObject().flatMap("", resultCollector);
	Mockito.verify(resultCollector, Mockito.never()).collect(Mockito.anyObject());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:12,代码来源:StringToJsonObjectTest.java


示例13: testFlatMap_withInvalidJSONString

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
 * an invalid json string as input to string parameter
 */
@Test
public void testFlatMap_withInvalidJSONString() throws Exception {
	@SuppressWarnings("unchecked")
	ListCollector<JSONObject> resultCollector = Mockito.mock(ListCollector.class);
	new StringToJsonObject().flatMap("{ test", resultCollector);
	Mockito.verify(resultCollector, Mockito.never()).collect(Mockito.anyObject());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:12,代码来源:StringToJsonObjectTest.java


示例14: testFlatMap_withValidInput

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
 * valid input
 */
@Test
public void testFlatMap_withValidInput() throws Exception {
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	new StringToJsonObject().flatMap("{\"test\":\"value\"}", resultCollector);
	Assert.assertEquals(result.size(), 1);
	Assert.assertEquals("{\"test\":\"value\"}", result.get(0).toString());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:13,代码来源:StringToJsonObjectTest.java


示例15: testApply_withNullWindowValues

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(org.apache.flink.streaming.api.windowing.windows.TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided null as input to iterable parameter
 */
@Test
public void testApply_withNullWindowValues() throws Exception {
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	new WindowedJsonContentAggregator("id", new AggregatorConfiguration()).apply(
			Mockito.mock(TimeWindow.class), null, resultCollector);
	Assert.assertTrue(result.isEmpty());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:13,代码来源:WindowedJsonContentAggregatorTest.java


示例16: testApply_withEmptyWindowValues

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(org.apache.flink.streaming.api.windowing.windows.TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided an empty list as input to iterable parameter
 */
@Test
public void testApply_withEmptyWindowValues() throws Exception {
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	new WindowedJsonContentAggregator("id", new AggregatorConfiguration()).apply(
			Mockito.mock(TimeWindow.class), new ArrayList<>(), resultCollector);
	Assert.assertTrue(result.isEmpty());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:13,代码来源:WindowedJsonContentAggregatorTest.java


示例17: testApply_withValidDocuments

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
 * Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
 * being provided a set of documents and groupby settings
 */
@Test
public void testApply_withValidDocuments() throws Exception {
	
	String content1 = "{\"field1\": {\"field2\":\"test-value\"}, \"field3\":\"test-value-2\"}";
	String content2 = "{\"field1\": {\"field2\":\"test-value4\"}, \"field3\":\"test-value-3\"}";
	FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput", 
			new JsonContentReference(new String[]{"field1","field2"}, JsonContentType.STRING, true));
	fieldCfg.addAggregationMethod(ContentAggregator.COUNT);

	AggregatorConfiguration cfg = new AggregatorConfiguration();
	cfg.setOutputElement("output");
	cfg.addFieldAggregation(fieldCfg);
	cfg.addGroupByField(new JsonContentReference(new String[]{"field1", "field2"}, JsonContentType.STRING, true));
	cfg.addGroupByField(new JsonContentReference(new String[]{"field3"}, JsonContentType.STRING, true));
	cfg.addOptionalField("counter", WindowedJsonContentAggregator.OPTIONAL_FIELD_TYPE_TOTAL_MESSAGE_COUNT);

	List<JSONObject> values = new ArrayList<>();
	values.add(new JSONObject(content1));
	values.add(new JSONObject(content2));
	values.add(new JSONObject(content2));
	
	List<JSONObject> result = new ArrayList<>();
	ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
	
	new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
	Assert.assertEquals(1, result.size());
	JSONObject resultObject  = result.get(0);
	String expected = "{\"output\":{\"test-value4\":{\"test-value-3\":{\"fieldOutput\":{\"COUNT\":2}}},\"test-value\":{\"test-value-2\":{\"fieldOutput\":{\"COUNT\":1}}}},\"counter\":3}";
	Assert.assertEquals(expected,resultObject.toString());
}
 
开发者ID:ottogroup,项目名称:flink-operator-library,代码行数:35,代码来源:WindowedJsonContentAggregatorTest.java


示例18: testCollectHashTags

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testCollectHashTags() throws Exception {
    ArrayList<Tuple2<String, Integer>> list = new ArrayList<>();
    Collector<Tuple2<String, Integer>> collector = new ListCollector<>(list);
    TwitterStreamProcessing.collectHashtags(collector, "\"hashtags\":[{\"text\":\"NadoSincronizado\",\"indices\":[0,17]},{\"text\":\"MEX\",\"indices\":[18,22]}]");

    assertThat(list, hasItem(Tuple2.of("NadoSincronizado", 1)));
    assertThat(list, hasItem(Tuple2.of("MEX", 1)));
}
 
开发者ID:godatadriven,项目名称:flink-streaming-xke,代码行数:10,代码来源:TwitterStreamProcessingTest.java


示例19: processFromInstances

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
public List<EvaluationResult> processFromInstances() throws Exception {
  BufferedReader reader =
    new BufferedReader(new FileReader(config.getInstancesFile()));
  ArffLoader.ArffReader arff = new ArffLoader.ArffReader(reader);
  Instances instances;
  instances = arff.getData();
  instances.setClassIndex(instances.numAttributes() - 1);
  ArrayList<EvaluationResult> evaluationResults = new ArrayList<>();
  //wrap
  Collector<EvaluationResult> c = new ListCollector<>(evaluationResults);
  process(c, instances);
  return evaluationResults;
}
 
开发者ID:ag-gipp,项目名称:mathosphere,代码行数:14,代码来源:WekaLearner.java


示例20: testEmbeddingFormat

import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testEmbeddingFormat() throws Exception{
  GradoopId a = GradoopId.get();
  GradoopId b = GradoopId.get();

  Embedding embedding = new Embedding();
  embedding.add(a);
  embedding.add(b);

  List<Embedding> result = new ArrayList<>();
  new AdoptEmptyPaths(1, -1).flatMap(embedding, new ListCollector<>(result));

  assertTrue(result.get(0).getIdList(2).isEmpty());
  assertEquals(b, result.get(0).getId(3));
}
 
开发者ID:dbs-leipzig,项目名称:gradoop,代码行数:16,代码来源:AdoptEmptyPathsTest.java



注:本文中的org.apache.flink.api.common.functions.util.ListCollector类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java ObjectWithFieldsConverter类代码示例发布时间:2022-05-22
下一篇:
Java AgentWorkgroups类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap