• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号
登陆 注册
描述文字

wanglingsong/JsonSurfer: A streaming JsonPath processor in Java

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称：

wanglingsong/JsonSurfer

开源软件地址：

https://github.com/wanglingsong/JsonSurfer

开源编程语言：

Java 99.5%

开源软件介绍：

JsonSurfer - Let's surf on Json!

Join the chat at https://gitter.im/jsurfer/JsonSurfer Coverage Status Maven Central

Why JsonSurfer

  • Streaming

    No need to deserialize entire json into memory.

  • JsonPath

    Selectively extract json data by the power of JsonPath.

  • Stoppable

    JsonSurfer is built on stoppable SAX-like interface that allows the processor to stop itself if necessary.

  • Non-Blocking

    JsonSurfer is event-driven and offers non-blocking parser interface.

  • Binary format

    Support multiple binary data formats including Avro, CBOR, Protobuf, Smile and Ion.

Getting started

What is JsonPath?

  • Supported JsonPath operator in JsonSurfer:
Operator Description
$ root
@ current node
* wildcard
.. recursive descent
.<name> child
['<name>' (, '<name>')] child/children
[<number> (, <number>)] index/indices
[start:end] array slice
[?(<expression>)] filter expression
  • JsonSurfer is available in cetral maven repository.

JsonSurfer has drivers for most of popular json libraries including: Gson, Jackson, FastJson and JsonSimple. Choose one and add to your POM.

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-gson</artifactId>
    <version>1.6.3</version>
</dependency>

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-jackson</artifactId>
    <version>1.6.3</version>
</dependency>

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-fastjson</artifactId>
    <version>1.6.3</version>
</dependency>

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-jsonsimple</artifactId>
    <version>1.6.3</version>
</dependency>

Usage:

Create your JsonSurfer:

  • JsonSurfer has flexible constructor. You can create yourself or pick a prebuilt one according the json library you used:
  1. Gson
        // use gson parser and use gson provider use to deserialize json into gson model i.e.com.google.gson.JsonElement
        JsonSurfer surfer = new JsonSurfer(GsonParser.INSTANCE, GsonProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferGson.INSTANCE;
  1. Jackson
        JsonSurfer surfer = new JsonSurfer(JacksonParser.INSTANCE, JacksonProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferJackson.INSTANCE;
  1. JsonSimple
        // use json-simple parser and json-simple provider to deserialize json into json-simple model i.e.org.json.simple.JSONObject or org.json.simple.JSONArray
        JsonSurfer surfer = new JsonSurfer(JsonSimpleParser.INSTANCE, JsonSimpleProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferJsonSimple.INSTANCE;
  1. Fastjson
        JsonSurfer surfer = new JsonSurfer(FastJsonParser.INSTANCE, FastJsonProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferFastJson.INSTANCE;

Collect value by JsonPath

        Collector collector = surfer.collector(sample);
        ValueBox<String> box1 = collector.collectOne("$.store.book[1].category", String.class);
        ValueBox<Object> box2 = collector.collectOne("$.store.book[2].isbn");
        ValueBox<Collection<Object>> box3 = collector.collectAll("$.store.book[*]");
        collector.exec(); // make sure exec() invoked before getting value from boxes
        box1.get();
        box2.get();
        box3.get();

【DEPRECATED】 Collect the first matched value and stop immediately

        JsonSurfer jsonSurfer = JsonSurferGson.INSTANCE;
        Object singleResult = jsonSurfer.collectOne(sample, "$.store.book[0]");

【DEPRECATED】 Colllect every matched value into a collection

        JsonSurfer jsonSurfer = JsonSurferGson.INSTANCE;
        Collection<Object> multipleResults = jsonSurfer.collectAll(sample, "$.store.book[*]");

"Surfing" in Json and collecting matched value in the listeners

        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.book[*]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Reuse listener binding.

SurfingConfiguration is thread-safe as long as your listeners are stateless.

        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        SurfingConfiguration config = surfer.configBuilder()
                .bind("$.store.book[*]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .build();        
        surfer.surf(sample1, config);
        surfer.surf(sample2, config);

Compiled JsonPath

JsonPath object is immutable and can be reused safely.

Tips: Most of JsonSurfer API have two version: accepting raw JsonPath string or JsonPath object. The latter is always faster than the former without compiling JsonPath.

        JsonPath compiledPath = JsonPathCompiler.compile("$..book[1,3]['author','title']");
        String value = surfer.collectOne(read("sample.json"), String.class, compiledPath);

JsonPath Filters

  • Filter operators
Operator Description
== equal
< less than
> greater than

You can use logical operators '&&' and '||' to create more complex filter expression. For example:

$.store.book[?(@.price < 10 || @.category && @.isbn && @.price>10)].volumes[?(@.chapter == 1)]

Resolver API:

  • Limitation: Wildcard and Recursive Descent are NOT supported.
  • As of 1.2.6, JsonSurfer provides another way of processing json. You can directly resolve value with JsonPath from a well-built DOM like HashMap or even POJO:
        Book book = new Book();
        book.setAuthor("Leo");
        book.setCategory("Fiction");
        book.setPrice(100.0d);
        book.setTitle("JsonSurfer is great!");
        System.out.print(compile("$.author").resolve(book, new PoJoResolver()));

which prints "Leo".

        List<String> list = Arrays.asList("foo", "bar");
        HashMap<String, Object> map = new HashMap<String, Object>();
        map.put("list", list);
        System.out.println(compile("$.list[1]").resolve(map, JavaCollectionProvider.INSTANCE));

which prints "bar".

  • If you want to process POJO with full JsonPath feature, you can convert the POJO into binary format and then surfer on it.

Binaray format (Jackson only)

By importing Jackson binary format backend, JsonSurfer is capable to surfer with multiple binary object representation formats such as Avro, CBOR, Protobuf(A known bug to be fixed in Jackson 2.9.6), Smile and Ion.

For example, if you want to surfer with CBOR data, firstly, CBOR format backend need to be imported as dependency.

    <dependency>
        <groupId>com.fasterxml.jackson.dataformat</groupId>
        <artifactId>jackson-dataformat-cbor</artifactId>
        <version>${jackson.version}</version>
    </dependency>

Then create a JsonSurfer with CBOR-backed JacksonParser and surfer as usual

    surfer = new JsonSurfer(new JacksonParser(new CBORFactory()), provider);

Find more examples here: https://github.com/jsurfer/JsonSurfer/blob/master/jsurfer-all/src/test/java/org/jsfr/json/JacksonParserTest.java

Share data among processors

Since JsonSurfer emit data in the way of callback, it would be difficult if one of your processing depends one another. Therefore a simple transient map is added for sharing data among your processors. Following unit test shows how to use it:

        surfer.configBuilder().bind("$.store.book[1]", new JsonPathListener() {
            @Override
            public void onValue(Object value, ParsingContext context) {
                context.save("foo", "bar");
            }
        }).bind("$.store.book[2]", new JsonPathListener() {
            @Override
            public void onValue(Object value, ParsingContext context) {
                assertEquals("bar", context.load("foo", String.class));
            }
        }).bind("$.store.book[0]", new JsonPathListener() {
            @Override
            public void onValue(Object value, ParsingContext context) {
                assertNull(context.load("foo", String.class));
            }
        }).buildAndSurf(read("sample.json"));

Control parsing

  • How to pause and resume parsing.
    SurfingConfiguration config = surfer.configBuilder()
            .bind("$.store.book[0]", new JsonPathListener() {
                @Override
                public void onValue(Object value, ParsingContext context) {
                    LOGGER.info("The first pause");
                    context.pause();
                }
            })
            .bind("$.store.book[1]", new JsonPathListener() {
                @Override
                public void onValue(Object value, ParsingContext context) {
                    LOGGER.info("The second pause");
                    context.pause();
                }
            }).build();
    ResumableParser parser = surfer.getResumableParser(read("sample.json"), config);
    assertFalse(parser.resume());
    LOGGER.info("Start parsing");
    parser.parse();
    LOGGER.info("Resume from the first pause");
    assertTrue(parser.resume());
    LOGGER.info("Resume from the second pause");
    assertTrue(parser.resume());
    LOGGER.info("Parsing stopped");
    assertFalse(parser.resume());

Java 8 Streams API support

As of 1.4, JsonSurfer can create an iterator from Json and JsonPath. Matched value can be pulled from the iterator one by one without loading entire json into memory.

    Iterator iterator = surfer.iterator(read("sample.json"), JsonPathCompiler.compile("$.store.book[*]"));

Java8 user can also convert the iterator into a Stream

    Stream<Object> targetStream = StreamSupport.stream(
          Spliterators.spliteratorUnknownSize(iterator, Spliterator.ORDERED),
          false);

Functions implmented by Streams API

    public static Stream<Object> toStream(String json, String path) {
        Iterator<Object> iterator = JsonSurferJackson.INSTANCE.iterator(json, JsonPathCompiler.compile(path));
        return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, Spliterator.ORDERED), false);
    }

    public static void main(String[] s) throws Exception {
        String json = "[1,3,5,7,11]";
        // Count
        System.out.println(toStream(json, "$[*]").count());
        // Max
        toStream(json, "$[*]").mapToInt(o -> ((LongNode) o).asInt()).max().ifPresent(System.out::println);
        // Min
        toStream(json, "$[*]").mapToInt(o -> ((LongNode) o).asInt()).min().ifPresent(System.out::println);
        // Average
        toStream(json, "$[*]").mapToDouble(o -> ((LongNode) o).asDouble()).average().ifPresent(System.out::println);
    }

Non-Blocking parsing

As of 1.4, JsonSurfer support non-blocking parsing for JacksonParser. You can achieve 100% non-blocking JSON processing with JsonSurfer in a NIO application. Let's take a Vertx request handler as an example:

    Vertx vertx = Vertx.vertx();
    HttpServer server = vertx.createHttpServer(new HttpServerOptions());
    JsonSurfer surfer = JsonSurferJackson.INSTANCE;
    SurfingConfiguration config = surfer.configBuilder()
            .bind("$[*]", (JsonPathListener) (value, context) -> {
                // Handle json
                System.out.println(value);
            }).build();
    server.requestHandler(request -> {
        NonBlockingParser parser = surfer.createNonBlockingParser(config);
        request.handler(buffer -> {
            byte[] bytes = buffer.getBytes();
            System.out.println("Received " + bytes.length + " bytes");
            parser.feed(bytes, 0, bytes.length);
        });
        request.endHandler(aVoid -> {
            parser.endOfInput();
            System.out.println("End of request");
            request.response().end();
        });
    }).listen(8080);

Examples

Sample Json:

{
    "store": {
        "book": [
            {
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
            },
            {
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99
            },
            {
                "category": "fiction",
                "author": "Herman Melville",
                "title": "Moby Dick",
                "isbn": "0-553-21311-3",
                "price": 8.99
            },
            {
                "category": "fiction",
                "author": "J. R. R. Tolkien",
                "title": "The Lord of the Rings",
                "isbn": "0-395-19395-8",
                "price": 22.99
            }
        ],
        "bicycle": {
            "color": "red",
            "price": 19.95
        }
    },
    "expensive": 10
}
JsonPath Result
$.store.book[*].author Find the authors of all books
$..author All authors
$.store.* All things in store
$.store..price The price of everything in the store
$..book[2] The third book
$..book[0,1] The first two books
$.store.book[?(@.price==8.95)] Filter all books whose price equals to 8.95
$.store.book[?(@.category=='fiction')]             Filter all books which belong to fiction category                  
$.store.book[?(@.author=~/tolkien/i)] All books matching regex


鲜花
握手
雷人
路过
鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
More+
上一篇：
vmware-archive/vsphere-storage-for-kubernetes: vSphere Storage for Kubernetes发布时间：2022-07-09
下一篇：
factbook/factbook.json: World Factbook Country Profiles in JSON - Free Open Pub ...发布时间：2022-07-08
热门推荐
More+
热门话题
More+
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服（服务时间 9:00～18:00）

在线QQ客服
地址：深圳市南山区西丽大学城创智工业园
电邮：jeky_zhao#qq.com
移动电话：139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap