本文整理汇总了Java中org.apache.flink.api.common.functions.util.ListCollector类的典型用法代码示例。如果您正苦于以下问题：Java ListCollector类的具体用法？Java ListCollector怎么用？Java ListCollector使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
ListCollector类属于org.apache.flink.api.common.functions.util包，在下文中一共展示了ListCollector类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: testApply_withDocumentThrowingAnExceptionWithRawDataIncluded
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided an iterable holding a document which misses a required field (with raw data included)
*/
@Test
public void testApply_withDocumentThrowingAnExceptionWithRawDataIncluded() throws Exception {
String content = "{\"field1\": {\"field2\":\"test-value\"}}";
FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput",
new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
fieldCfg.addAggregationMethod(ContentAggregator.COUNT);
AggregatorConfiguration cfg = new AggregatorConfiguration();
cfg.setOutputElement("output");
cfg.setRaw(true);
cfg.addFieldAggregation(fieldCfg);
List<JSONObject> values = new ArrayList<>();
values.add(new JSONObject(content));
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
Assert.assertEquals(1, result.size());
JSONObject resultObject = result.get(0);
Assert.assertEquals("{\"output\":{},\"raw\":[{\"field1\":{\"field2\":\"test-value\"}}]}",resultObject.toString());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:29，代码来源:WindowedJsonContentAggregatorTest.java
示例2: testApply_withDocumentThrowingAnException
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided an iterable holding a document which misses a required field (raw must not be included)
*/
@Test
public void testApply_withDocumentThrowingAnException() throws Exception {
String content = "{\"field1\": {\"field2\":\"test-value\"}}";
FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput",
new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
fieldCfg.addAggregationMethod(ContentAggregator.COUNT);
AggregatorConfiguration cfg = new AggregatorConfiguration();
cfg.setOutputElement("output");
cfg.addFieldAggregation(fieldCfg);
List<JSONObject> values = new ArrayList<>();
values.add(new JSONObject(content));
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
Assert.assertEquals(1, result.size());
JSONObject resultObject = result.get(0);
Assert.assertEquals("{\"output\":{}}",resultObject.toString());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:28，代码来源:WindowedJsonContentAggregatorTest.java
示例3: testApply_withDocumentValidValuesRawNotIncluded
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided an iterable holding a document which shows valid values (raw not included)
*/
@Test
public void testApply_withDocumentValidValuesRawNotIncluded() throws Exception {
String content = "{\"field1\": {\"field2\":{\"field3\":\"test\"}}}";
FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput",
new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
fieldCfg.addAggregationMethod(ContentAggregator.COUNT);
AggregatorConfiguration cfg = new AggregatorConfiguration();
cfg.setOutputElement("output");
cfg.addFieldAggregation(fieldCfg);
List<JSONObject> values = new ArrayList<>();
values.add(new JSONObject(content));
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
Assert.assertEquals(1, result.size());
JSONObject resultObject = result.get(0);
Assert.assertEquals("{\"output\":{\"fieldOutput\":{\"COUNT\":1}}}",resultObject.toString());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:29，代码来源:WindowedJsonContentAggregatorTest.java
示例4: testApply_withDocumentValidValuesOptionalIncluded
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided an iterable holding a document which shows valid values (optional included)
*/
@Test
public void testApply_withDocumentValidValuesOptionalIncluded() throws Exception {
String content = "{\"field1\": {\"field2\":{\"field3\":\"test\"}}}";
FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput",
new JsonContentReference(new String[]{"field1","field2","field3"}, JsonContentType.STRING, true));
fieldCfg.addAggregationMethod(ContentAggregator.COUNT);
AggregatorConfiguration cfg = new AggregatorConfiguration();
cfg.setOutputElement("output");
cfg.addFieldAggregation(fieldCfg);
cfg.addOptionalField("counter", WindowedJsonContentAggregator.OPTIONAL_FIELD_TYPE_TOTAL_MESSAGE_COUNT);
List<JSONObject> values = new ArrayList<>();
values.add(new JSONObject(content));
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
Assert.assertEquals(1, result.size());
JSONObject resultObject = result.get(0);
Assert.assertEquals("{\"output\":{\"fieldOutput\":{\"COUNT\":1}},\"counter\":1}",resultObject.toString());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:29，代码来源:WindowedJsonContentAggregatorTest.java
示例5: executeOnCollections
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Override
protected List<T> executeOnCollections(List<T> inputData, RuntimeContext ctx, ExecutionConfig executionConfig) throws Exception {
FlatMapFunction<T, T> function = this.userFunction.getUserCodeObject();
FunctionUtils.setFunctionRuntimeContext(function, ctx);
FunctionUtils.openFunction(function, this.parameters);
ArrayList<T> result = new ArrayList<T>(inputData.size());
ListCollector<T> collector = new ListCollector<T>(result);
for (T element : inputData) {
function.flatMap(element, collector);
}
FunctionUtils.closeFunction(function);
return result;
}
开发者ID:axbaretto，项目名称:flink，代码行数:19，代码来源:FilterOperatorBase.java
示例6: computeOuterJoin
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@SuppressWarnings("unchecked, rawtypes")
private List<Tuple4<String, String, String, Object>> computeOuterJoin(ResettableMutableObjectIterator<Tuple2<String, String>> input1,
ResettableMutableObjectIterator<Tuple2<String, Integer>> input2,
OuterJoinType outerJoinType) throws Exception {
input1.reset();
input2.reset();
AbstractMergeOuterJoinIterator iterator =
createOuterJoinIterator(
outerJoinType, input1, input2, serializer1, comparator1, serializer2, comparator2,
pairComp, this.memoryManager, this.ioManager, PAGES_FOR_BNLJN, this.parentTask
);
List<Tuple4<String, String, String, Object>> actual = new ArrayList<>();
ListCollector<Tuple4<String, String, String, Object>> collector = new ListCollector<>(actual);
while (iterator.callWithNextKey(new SimpleTupleJoinFunction(), collector)) ;
iterator.close();
return actual;
}
开发者ID:axbaretto，项目名称:flink，代码行数:20，代码来源:AbstractSortMergeOuterJoinIteratorITCase.java
示例7: testFilterEmbeddingsOnClosingColumn
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testFilterEmbeddingsOnClosingColumn() throws Exception {
GradoopId a = GradoopId.get();
GradoopId b = GradoopId.get();
Embedding embedding = new Embedding();
embedding.add(a);
embedding.add(b);
List<Embedding> result = new ArrayList<>();
new AdoptEmptyPaths(1, 0).flatMap(embedding, new ListCollector<>(result));
assertTrue(result.isEmpty());
new AdoptEmptyPaths(1, 1).flatMap(embedding, new ListCollector<>(result));
assertEquals(1, result.size());
}
开发者ID:dbs-leipzig，项目名称:gradoop，代码行数:18，代码来源:AdoptEmptyPathsTest.java
示例8: testJoin
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Tests the join of the edge with an Embedding
* (m, e0, n) x edge
*
* @param edge the edge the join is performed with
* @param distinctVertices distinct vertex columns of the base embedding
* @param distinctEdges distinct edge columns of the base embedding
* @param closingColumn closing column
* @param isResult if true it is expected that the join yields exactly one result, 0 otherwise
* @throws Exception
*/
private void testJoin(Embedding edge, List<Integer> distinctVertices,
List<Integer> distinctEdges, int closingColumn, boolean isResult) throws Exception {
Embedding base = new Embedding();
base.add(m);
base.add(e0);
base.add(n);
EdgeWithTiePoint edgeTuple = new EdgeWithTiePoint(edge);
CreateExpandEmbedding op = new CreateExpandEmbedding(distinctVertices, distinctEdges, closingColumn);
List<ExpandEmbedding> results = new ArrayList<>();
op.join(base, edgeTuple, new ListCollector<>(results));
assertEquals(isResult ? 1:0, results.size());
if (isResult) {
assertEquals(base, results.get(0).getBase());
assertArrayEquals(new GradoopId[]{edge.getId(1)}, results.get(0).getPath());
assertEquals(edge.getId(2), results.get(0).getEnd());
}
}
开发者ID:dbs-leipzig，项目名称:gradoop，代码行数:35，代码来源:CreateInitialExpandEmbeddingTest.java
示例9: testStructureBasedProximity
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testStructureBasedProximity() throws Exception {
String wikiDocStr = getFileContents("RecommendationPairExtractorTest/article.xml.in");
DocumentProcessor dp = new DocumentProcessor();
WikiDocument doc = dp.processDoc(wikiDocStr);
List<RecommendationPair> list = new ArrayList<>();
RecommendationPairExtractor rpe = new RecommendationPairExtractor();
rpe.collectLinkPairsBasedOnStructure(doc, new ListCollector<>(list));
// System.out.println(list);
for(RecommendationPair pair: list) {
assertEquals("Proximity should be paragraph level only", LinkPosition.CPI_PARAGRAPH_LEVEL, pair.getDistance(), 0);
}
}
开发者ID:wikimedia，项目名称:citolytics，代码行数:19，代码来源:RecommendationPairExtractorTest.java
示例10: executeOnCollections
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Override
protected List<T> executeOnCollections(List<T> inputData, RuntimeContext ctx, boolean mutableObjectSafeMode) throws Exception {
FlatMapFunction<T, T> function = this.userFunction.getUserCodeObject();
FunctionUtils.setFunctionRuntimeContext(function, ctx);
FunctionUtils.openFunction(function, this.parameters);
ArrayList<T> result = new ArrayList<T>(inputData.size());
ListCollector<T> collector = new ListCollector<T>(result);
for (T element : inputData) {
function.flatMap(element, collector);
}
FunctionUtils.closeFunction(function);
return result;
}
开发者ID:citlab，项目名称:vs.msc.ws14，代码行数:19，代码来源:FilterOperatorBase.java
示例11: testFlatMap_withNullString
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
* null as input to string parameter
*/
@Test
public void testFlatMap_withNullString() throws Exception {
@SuppressWarnings("unchecked")
ListCollector<JSONObject> resultCollector = Mockito.mock(ListCollector.class);
new StringToJsonObject().flatMap(null, resultCollector);
Mockito.verify(resultCollector, Mockito.never()).collect(Mockito.anyObject());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:12，代码来源:StringToJsonObjectTest.java
示例12: testFlatMap_withEmptyString
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
* an empty string as input to string parameter
*/
@Test
public void testFlatMap_withEmptyString() throws Exception {
@SuppressWarnings("unchecked")
ListCollector<JSONObject> resultCollector = Mockito.mock(ListCollector.class);
new StringToJsonObject().flatMap("", resultCollector);
Mockito.verify(resultCollector, Mockito.never()).collect(Mockito.anyObject());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:12，代码来源:StringToJsonObjectTest.java
示例13: testFlatMap_withInvalidJSONString
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
* an invalid json string as input to string parameter
*/
@Test
public void testFlatMap_withInvalidJSONString() throws Exception {
@SuppressWarnings("unchecked")
ListCollector<JSONObject> resultCollector = Mockito.mock(ListCollector.class);
new StringToJsonObject().flatMap("{ test", resultCollector);
Mockito.verify(resultCollector, Mockito.never()).collect(Mockito.anyObject());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:12，代码来源:StringToJsonObjectTest.java
示例14: testFlatMap_withValidInput
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link StringToJsonObject#flatMap(String, org.apache.flink.util.Collector)} being provided
* valid input
*/
@Test
public void testFlatMap_withValidInput() throws Exception {
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new StringToJsonObject().flatMap("{\"test\":\"value\"}", resultCollector);
Assert.assertEquals(result.size(), 1);
Assert.assertEquals("{\"test\":\"value\"}", result.get(0).toString());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:13，代码来源:StringToJsonObjectTest.java
示例15: testApply_withNullWindowValues
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(org.apache.flink.streaming.api.windowing.windows.TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided null as input to iterable parameter
*/
@Test
public void testApply_withNullWindowValues() throws Exception {
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", new AggregatorConfiguration()).apply(
Mockito.mock(TimeWindow.class), null, resultCollector);
Assert.assertTrue(result.isEmpty());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:13，代码来源:WindowedJsonContentAggregatorTest.java
示例16: testApply_withEmptyWindowValues
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(org.apache.flink.streaming.api.windowing.windows.TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided an empty list as input to iterable parameter
*/
@Test
public void testApply_withEmptyWindowValues() throws Exception {
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", new AggregatorConfiguration()).apply(
Mockito.mock(TimeWindow.class), new ArrayList<>(), resultCollector);
Assert.assertTrue(result.isEmpty());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:13，代码来源:WindowedJsonContentAggregatorTest.java
示例17: testApply_withValidDocuments
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
/**
* Test case for {@link WindowedJsonContentAggregator#apply(TimeWindow, Iterable, org.apache.flink.util.Collector)}
* being provided a set of documents and groupby settings
*/
@Test
public void testApply_withValidDocuments() throws Exception {
String content1 = "{\"field1\": {\"field2\":\"test-value\"}, \"field3\":\"test-value-2\"}";
String content2 = "{\"field1\": {\"field2\":\"test-value4\"}, \"field3\":\"test-value-3\"}";
FieldAggregationConfiguration fieldCfg = new FieldAggregationConfiguration("fieldOutput",
new JsonContentReference(new String[]{"field1","field2"}, JsonContentType.STRING, true));
fieldCfg.addAggregationMethod(ContentAggregator.COUNT);
AggregatorConfiguration cfg = new AggregatorConfiguration();
cfg.setOutputElement("output");
cfg.addFieldAggregation(fieldCfg);
cfg.addGroupByField(new JsonContentReference(new String[]{"field1", "field2"}, JsonContentType.STRING, true));
cfg.addGroupByField(new JsonContentReference(new String[]{"field3"}, JsonContentType.STRING, true));
cfg.addOptionalField("counter", WindowedJsonContentAggregator.OPTIONAL_FIELD_TYPE_TOTAL_MESSAGE_COUNT);
List<JSONObject> values = new ArrayList<>();
values.add(new JSONObject(content1));
values.add(new JSONObject(content2));
values.add(new JSONObject(content2));
List<JSONObject> result = new ArrayList<>();
ListCollector<JSONObject> resultCollector = new ListCollector<>(result);
new WindowedJsonContentAggregator("id", cfg).apply(null,values,resultCollector);
Assert.assertEquals(1, result.size());
JSONObject resultObject = result.get(0);
String expected = "{\"output\":{\"test-value4\":{\"test-value-3\":{\"fieldOutput\":{\"COUNT\":2}}},\"test-value\":{\"test-value-2\":{\"fieldOutput\":{\"COUNT\":1}}}},\"counter\":3}";
Assert.assertEquals(expected,resultObject.toString());
}
开发者ID:ottogroup，项目名称:flink-operator-library，代码行数:35，代码来源:WindowedJsonContentAggregatorTest.java
示例18: testCollectHashTags
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testCollectHashTags() throws Exception {
ArrayList<Tuple2<String, Integer>> list = new ArrayList<>();
Collector<Tuple2<String, Integer>> collector = new ListCollector<>(list);
TwitterStreamProcessing.collectHashtags(collector, "\"hashtags\":[{\"text\":\"NadoSincronizado\",\"indices\":[0,17]},{\"text\":\"MEX\",\"indices\":[18,22]}]");
assertThat(list, hasItem(Tuple2.of("NadoSincronizado", 1)));
assertThat(list, hasItem(Tuple2.of("MEX", 1)));
}
开发者ID:godatadriven，项目名称:flink-streaming-xke，代码行数:10，代码来源:TwitterStreamProcessingTest.java
示例19: processFromInstances
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
public List<EvaluationResult> processFromInstances() throws Exception {
BufferedReader reader =
new BufferedReader(new FileReader(config.getInstancesFile()));
ArffLoader.ArffReader arff = new ArffLoader.ArffReader(reader);
Instances instances;
instances = arff.getData();
instances.setClassIndex(instances.numAttributes() - 1);
ArrayList<EvaluationResult> evaluationResults = new ArrayList<>();
//wrap
Collector<EvaluationResult> c = new ListCollector<>(evaluationResults);
process(c, instances);
return evaluationResults;
}
开发者ID:ag-gipp，项目名称:mathosphere，代码行数:14，代码来源:WekaLearner.java
示例20: testEmbeddingFormat
import org.apache.flink.api.common.functions.util.ListCollector; //导入依赖的package包/类
@Test
public void testEmbeddingFormat() throws Exception{
GradoopId a = GradoopId.get();
GradoopId b = GradoopId.get();
Embedding embedding = new Embedding();
embedding.add(a);
embedding.add(b);
List<Embedding> result = new ArrayList<>();
new AdoptEmptyPaths(1, -1).flatMap(embedding, new ListCollector<>(result));
assertTrue(result.get(0).getIdList(2).isEmpty());
assertEquals(b, result.get(0).getId(3));
}
开发者ID:dbs-leipzig，项目名称:gradoop，代码行数:16，代码来源:AdoptEmptyPathsTest.java
注：本文中的org.apache.flink.api.common.functions.util.ListCollector类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论