• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

crustio/crust-smanager: A storage manager integrated with Crust, IPFS and sWorke ...

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称:

crustio/crust-smanager

开源软件地址:

https://github.com/crustio/crust-smanager

开源编程语言:

TypeScript 99.3%

开源软件介绍:

crust-smanager

sManager (Storage Manager) is a file picking bot which continuously picking and handling files from Crust Network. Node operators can customize this module to implement their own file handling strategy. sManager maintains a local database to help making decision on pulling files.

Local Database

The local database stores below information:

  1. File Records: The files metadata(tips, size, replicas count, expire time etc.) on Crust Network.
  2. File and Owner Relationship: sManager also maintains the relationship between a file and an on-chain account. This information will help making better pulling decision.
  3. Chain Metadata: e.g. the block and time on chain.
  4. Pin Records: The pin history of files.
  5. Cleanup Records: The files needs to removed from local filesystem, normally this is triggered when a file expires on Crust Network.

Checkout Db Schema for the schema details.

Components

sManager was designed to have serveral tasks running independently. Tasks are either scheduled by the block event or by configured intervals. Each task plays as an actor which consumes/produces some information and communicate with other tasks through the db or applicaion context.

sManager follows the Fails Early priciple which means it will shutdown on any unexpected error. To support this priciple, tasks are designed to be recoverable after application restarts.

Below are a list of components that sManager has implemented.

Indexers

Indexers extract information into the local database from various data sources. Currently sManager has implemented below indexers:

  1. Chain Database Indexer: indexes file records from the Crust Network on-chain database.
  2. Chain Event Indexer: indexes file records by listening latest chain event.
  3. Chain Time Indexer: a simple indexer which push the latest block height and it's timestamp to the config table.

Simple Tasks

Simple tasks are speciualized tasks which runs periodly. Currently sManager has implemented below tasks:

  1. Group Info Updater: Update sworker identity information from sworker api.
  2. Ipfs Gc: Schedule ipfs gc periodly.
  3. Telemetry Reporting: Report smanager statistics information to the telemetry server.
  4. Pull Scheduler: Schedule file pulling based on configured strategey.
  5. Seal Status Updater: Update sealing status periodly.
  6. File Retry Task: Retry pulling if possible.
  7. File Cleanup Task: Cleanup deleted files from local filesystem.

Usage

  1. Clone repo
git clone https://github.com/crustio/crust-smanager.git
  1. Installing It's recommended to use volta as the node version manager. Please follow the volta docs to install it.
cd crust-smanager && npm i
  1. Debug
npm run dev
  1. Run in Prod
npm run build
npm start

It's recommended to run sManager using Docker with the restart=always restart policy.

A daemon guard should be configured if you want to run sManager natively without docker. Tools like pm2 and nodemon could be used.

Configuration

Checkout smanager-config.example.json

Those config items will be configured in the sManager configuration setup process. The meaning of each item is as follows:

  • chain.account: your member account

  • chain.endPoint: your chain endpoint

  • sworker.endPoint: your sWorker endpoint

  • ipfs.endPoint: your IPFS endpoint

  • dataDir: the directory of the database of sManager

  • scheduler.minSrdRatio: a minimum ratio of SRD that one node can start to accept storage orders.

    For example, if the ratio is 30, then your node will start to accept storage order once the ratio of SRD capacity is higher than 30%

  • scheduler.strategy.dbFilesWeight: how much bandwidth of this node will be used to fetch and store the history storage orders(Storage orders in the past four months).

  • scheduler.strategy.newFilesWeight: how much bandwidth of this node will be used to fetch and store the newest storage orders.




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
crustio/ipfs-w3auth-pinning-service: IPFS remote pinning service by using Crust发布时间:2022-06-22
下一篇:
ipfs/go-ipfs-example-plugin发布时间:2022-06-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap