• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

mgsloan/store: Fast binary serialization in Haskell

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

mgsloan/store

开源软件地址(OpenSource Url):

https://github.com/mgsloan/store

开源编程语言(OpenSource Language):

Haskell 100.0%

开源软件介绍(OpenSource Introduction):

store

The 'store' package provides efficient binary serialization. There are a couple features that particularly distinguish it from most prior Haskell serialization libraries:

  • Its primary goal is speed. By default, direct machine representations are used for things like numeric values (Int, Double, Word32, etc) and buffers (Text, ByteString, Vector, etc). This means that much of serialization uses the equivalent of memcpy.

    We have plans for supporting architecture independent serialization - see #36 and #31. This plan makes little endian the default, so that the most common endianness has no overhead.

    • Another way that the serialization behavior can vary is if integer-simple is used instead of GHC's default of using GMP. Integer serialized with the integer-simple flag enabled are not compatible with those serialized without the flag enabled.
  • Instead of implementing lazy serialization / deserialization involving multiple input / output buffers, peek and poke always work with a single buffer. This buffer is allocated by asking the value for its size before encoding. This simplifies the encoding logic, and allows for highly optimized tight loops.

  • store can optimize size computations by knowing when some types always use the same number of bytes. This allows us to compute the byte size of a Vector Int32 by just doing length v * 4.

It also features:

  • Optimized serialization instances for many types from base, vector, bytestring, text, containers, time, template-haskell, and more.

  • TH and GHC Generics based generation of Store instances for datatypes.

  • TH generation of testcases.

  • Utilities for streaming encoding / decoding of Store encoded messages, via the store-streaming package.

Gotchas

Store is best used for communication between trusted processes and local caches. It can certainly be used for other purposes, but the builtin set of instances have some gotchas to be aware of:

  • Store's builtin instances serialize in a format which depends on machine endianness.

  • Store's builtin instances trust the data when deserializing. For example, the deserialization of Vector will read the vector's length from the first 8 bytes. It will then allocate enough memory to store all the elements. Malicious or malformed input could cause allocation of large amounts of memory. See issue #122.

  • Serialization may vary based on the version of datatypes. For example, Text serialized from text < 2 will not be compatible with Text from text >= 2, because the internal representation switched from UTF-16 to UTF-8.

Blog posts




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap