在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称:flink开源软件地址:https://gitee.com/apache/flink开源软件介绍:Apache FlinkApache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities. Learn more about Flink at https://flink.apache.org/ Features
Streaming Examplecase class WordWithCount(word: String, count: Long)val text = env.socketTextStream(host, port, '\n')val windowCounts = text.flatMap { w => w.split("\\s") } .map { w => WordWithCount(w, 1) } .keyBy("word") .window(TumblingProcessingTimeWindow.of(Time.seconds(5))) .sum("count")windowCounts.print() Batch Examplecase class WordWithCount(word: String, count: Long)val text = env.readTextFile(path)val counts = text.flatMap { w => w.split("\\s") } .map { w => WordWithCount(w, 1) } .groupBy("word") .sum("count")counts.writeAsCsv(outputPath) Building Apache Flink from SourcePrerequisites for building Flink:
git clone https://github.com/apache/flink.gitcd flinkmvn clean package -DskipTests # this will take up to 10 minutes Flink is now installed in NOTE: Maven 3.3.x can build Flink, but will not properly shade away certain dependencies. Maven 3.1.1 creates the libraries properly.To build unit tests with Java 8, use Java 8u51 or above to prevent failures in unit tests that use the PowerMock runner. Developing FlinkThe Flink committers use IntelliJ IDEA to develop the Flink codebase.We recommend IntelliJ IDEA for developing projects that involve Scala code. Minimal requirements for an IDE are:
IntelliJ IDEAThe IntelliJ IDE supports Maven out of the box and offers a plugin for Scala development.
Check out our Setting up IntelliJ guide for details. Eclipse Scala IDENOTE: From our experience, this setup does not work with Flinkdue to deficiencies of the old Eclipse version bundled with Scala IDE 3.0.3 ordue to version incompatibilities with the bundled Scala version in Scala IDE 4.4.1. We recommend to use IntelliJ instead (see above) SupportDon’t hesitate to ask! Contact the developers and community on the mailing lists if you need any help. Open an issue if you found a bug in Flink. DocumentationThe documentation of Apache Flink is located on the website: https://flink.apache.orgor in the Fork and ContributeThis is an active open-source project. We are always open to people who want to use the system or contribute to it.Contact us if you are looking for implementation tasks that fit your skills.This article describes how to contribute to Apache Flink. AboutApache Flink is an open source project of The Apache Software Foundation (ASF).The Apache Flink project originated from the Stratosphere research project. |
请发表评论