• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

textileio/go-foldersync: A sample-app of file syncing using Threads V2 and IPFS ...

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称:

textileio/go-foldersync

开源软件地址:

https://github.com/textileio/go-foldersync

开源编程语言:

Go 100.0%

开源软件介绍:

Folder-sync app

This sample-app allows peers to join a shared folder, where each user will own a folder named as their username. The local peer can drop new files to its folder, where a filewatcher will automatically detect new files and sync to other joined peers.

Design

The main components of the applications are:

  • A Store using new Threads V2.
  • IPFS Lite to download files.

The Store will hold all files metadata, which is a JSON document derived from the following structure:

type userFolder struct {
	ID    core.EntityID
	Owner string
	Files []file
}

type file struct {
	ID               string
	FileRelativePath string
	CID              string

	IsDirectory bool
	Files       []file
}

In summary, each folder is represented by a userFolder which is owned by a user. This folder has a list of file which has file metadata such as:

  • A unique uuid
  • A relative path from the root of the shared folder: userXXX/file.txt
  • The CID of the file, which can be fetched out-of-band with IPFS Lite.

The original intention is to allow arbitrary nested folders and files, but currently it has only one level support for files. It's on the plans to enable folder discovery to allow nested trees.

Currently, when a new file is discovered by a peer, it will try to fetch it using an ipfslite client, which will leveraged connected peers. By default, the ipfslite peer doesn't boostrap a DHT, but it can be enabled in a single line. Future versions will make this optional through flags. In the same vein, we can provide other sources from which fetch files such as the Filecoin network.

Usage

When a peer wants to bootstrap a new shared folder, to invite others, it has to run: go run main.go util.go client.go -name bob. This will create a new Store with an underlying new Thread, and will print an invitation link that other peers can use to join the shared folder.

Joining peers will run: go run main.go util.go client.go -name alice -inviteLink xxxxxxx to join the shared folder. The invite link has format: <thread-addr>?<follow-key>&<read-key>.

Current flags are:

  • name: to setup the peer name, which will be his sharedfolder name
  • repo: path to the repo that will hold data of Store
  • sharedFolder: path to where will be the shared folder containing all peer folders with their files
  • inviteLink: if provided is used to join an existing sharedFolder

Flags have reasonable defaults.

Tests

This app now has a pretty heavy test setup, where tunning a parameter simulates:

  • A number N of peers joining a shared folder
  • How many random files will each peer generate
  • How big each random file will be
  • How many core peers will be used to join the sharedFolder. This means, X peers will be used as invitation links by others in a round-robin fashion possibly to avoid DoSing all joining the same peer. May also be used to test how this impact syncing perf.
  • How many seconds provide to the peers to sync before asserting convergence.

After the sync timeout triggers, the test will assert if all Store of peers are the same, and if all the files were fetched (that to say, each peer shared folder contains all peers folders and their files).

Take into consideration that tests configuration should be tuned to avoid timeouts and high overload, since all peers are running on the same host.




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap