• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

OrphicAbyss/MastodonFlink

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

OrphicAbyss/MastodonFlink

开源软件地址(OpenSource Url):

https://github.com/OrphicAbyss/MastodonFlink

开源编程语言(OpenSource Language):

Java 79.9%

开源软件介绍(OpenSource Introduction):

Mastodon + Flink

Test project for using the Mastodon streaming API as a source in Apache Flink.

Building Fat JAR

You need to have Maven installed or to use an IDE like IntelliJ IDEA

Run the maven package lifecycle with the build-jar profile enabled.

mvn package -P build-local-jar

This will build a 'Fat JAR' containing all the dependencies which can then be used to test the pipeline.

If you have a Apache Flink cluster to deploy to you can use:

mvn package -P build-jar

To create a JAR with only the dependencies required (no Flink dependencies).

Testing

You need:

  • Create a Mastodon account on an instance
  • To create a development token (keep this secret)
java -jar  net.gluonporridge.LiveStream --instance "Name Of Instance" --accessToken "API Access Token Here"

This will start up a local Flink instance and run the pipeline.

It will connect to Mastodon and start streaming in real time.

By default the pipeline is set to use the Federated Timeline. New Toots will be shown as they are streamed in.

The output will look like:

=============
2017-11-29T21:31:59.622Z
hackaday ([email protected]) [14490]
RT @NeuroTinker: Our @kickstarter campaign is now live! NeuroBytes https://apoil.org/tags/neuron simulators and https://apoil.org/tags/curricula combine https://apoil.org/tags/neuroscience and…  source: https://twitter.com/hackaday/status/935984248430796800
Tags: neuroscience, curricula, neuron 
=============

Output To MongoDB

Going to store full records in MongoDB for use in a test interface.




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap