• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号
登陆 注册
描述文字

Java Record类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.kitesdk.morphline.api.Record类的典型用法代码示例。如果您正苦于以下问题：Java Record类的具体用法？Java Record怎么用？Java Record使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



Record类属于org.kitesdk.morphline.api包，在下文中一共展示了Record类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: executePipeline

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void executePipeline(
    final @Mocked MorphlineUtils.Pipeline pipeline,
    final @Mocked Command morphline
) throws Exception {

  final Record inputRecord = new Record();

  final Record outputRecord = new Record();
  outputRecord.put("field1", "value1");

  new Expectations() {{
    morphline.process(inputRecord); result = true;
    pipeline.getCollector().getRecords(); result = Lists.newArrayList(outputRecord);
  }};

  List<Record> outputList = MorphlineUtils.executePipeline(pipeline, inputRecord);
  assertEquals(Lists.newArrayList(outputRecord), outputList);
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:20，代码来源:TestMorphlineUtils.java


示例2: convertToRowValidValue

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowValidValue(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("field1", "one");

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, false))
  );

  new Expectations() {{
    RowUtils.toRowValue("one", DataTypes.StringType); result = "success";
  }};

  assertEquals("Invalid conversion", "success", MorphlineUtils.convertToRow(schema, record).get(0));
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:19，代码来源:TestMorphlineUtils.java


示例3: convertToRowValidNullValue

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowValidNullValue(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("field1", null);

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, true))
  );

  assertEquals("Invalid conversion", null, MorphlineUtils.convertToRow(schema, record).get(0));

  new Verifications() {{
    RowUtils.toRowValue(any, (DataType) any); times = 0;
  }};
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:19，代码来源:TestMorphlineUtils.java


示例4: convertToRowInvalidNullValue

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowInvalidNullValue(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("field1", null);

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, false))
  );

  try {
    MorphlineUtils.convertToRow(schema, record);
    fail("Did not throw a RuntimeException");
  } catch (Exception e) {
    assertThat(e.getMessage(), JUnitMatchers.containsString("DataType cannot contain 'null'"));
  }

  new Verifications() {{
    RowUtils.toRowValue(any, (DataType) any); times = 0;
  }};
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:24，代码来源:TestMorphlineUtils.java


示例5: convertToRowInvalidTypeNotNullable

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowInvalidTypeNotNullable(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("field1", "one");

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, false))
  );

  new Expectations() {{
    RowUtils.toRowValue("one", DataTypes.StringType); result = new RuntimeException("Conversion exception");
  }};

  try {
    MorphlineUtils.convertToRow(schema, record);
    fail("Did not throw a RuntimeException");
  } catch (Exception e) {
    assertThat(e.getMessage(), JUnitMatchers.containsString("Error converting Field"));
  }
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:24，代码来源:TestMorphlineUtils.java


示例6: convertToRowInvalidTypeNullable

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowInvalidTypeNullable(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("field1", "one");

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, true))
  );

  new Expectations() {{
    RowUtils.toRowValue("one", DataTypes.StringType); result = new RuntimeException("Conversion exception");
  }};

  try {
    MorphlineUtils.convertToRow(schema, record);
    fail("Did not throw a RuntimeException");
  } catch (Exception e) {
    assertThat(e.getMessage(), JUnitMatchers.containsString("Error converting Field"));
  }
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:24，代码来源:TestMorphlineUtils.java


示例7: convertToRowMissingColumnNotNullable

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowMissingColumnNotNullable(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("foo", "one");

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, false))
  );

  try {
    MorphlineUtils.convertToRow(schema, record);
    fail("Did not throw a RuntimeException");
  } catch (Exception e) {
    assertThat(e.getMessage(), JUnitMatchers.containsString("Error converting Record"));
  }

  new Verifications() {{
    RowUtils.toRowValue(any, (DataType) any); times = 0;
  }};
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:24，代码来源:TestMorphlineUtils.java


示例8: convertToRowMissingColumnNullable

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void convertToRowMissingColumnNullable(
    final @Mocked RowUtils utils
) throws Exception {

  Record record = new Record();
  record.put("foo", "one");

  StructType schema = DataTypes.createStructType(Lists.newArrayList(
      DataTypes.createStructField("field1", DataTypes.StringType, true))
  );

  try {
    MorphlineUtils.convertToRow(schema, record);
    fail("Did not throw a RuntimeException");
  } catch (Exception e) {
    assertThat(e.getMessage(), JUnitMatchers.containsString("Error converting Record"));
  }

  new Verifications() {{
    RowUtils.toRowValue(any, (DataType) any); times = 0;
  }};
}
开发者ID:cloudera-labs，项目名称:envelope，代码行数:24，代码来源:TestMorphlineUtils.java


示例9: detectParser

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
private Parser detectParser(Record record) {
  if (!hasAtLeastOneMimeType(record)) {
    return null;
  }
  String mediaTypeStr = (String) record.getFirstValue(Fields.ATTACHMENT_MIME_TYPE); //ExtractingParams.STREAM_TYPE);
  assert mediaTypeStr != null;
  
  MediaType mediaType = parseMediaType(mediaTypeStr).getBaseType();
  Parser parser = mediaTypeToParserMap.get(mediaType); // fast path
  if (parser != null) {
    return parser;
  }
  // wildcard matching
  for (Map.Entry<MediaType, Parser> entry : mediaTypeToParserMap.entrySet()) {
    if (isMediaTypeMatch(mediaType, entry.getKey())) {
      return entry.getValue();
    }
  }
  if (LOG.isDebugEnabled()) {
    LOG.debug("No supported MIME type parser found for " + Fields.ATTACHMENT_MIME_TYPE + "=" + mediaTypeStr);
  }
  return null;
}
开发者ID:europeana，项目名称:search，代码行数:24，代码来源:SolrCellBuilder.java


示例10: doProcess

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Override
protected boolean doProcess(Record record) {
  Collection<Map.Entry> entries = new ArrayList<Map.Entry>(record.getFields().asMap().entrySet());
  for (Map.Entry<String, Collection<Object>> entry : entries) {
    String key = entry.getKey();
    if (schema.getFieldOrNull(key) == null) {
      LOG.debug("Sanitizing unknown Solr field: {}", key);
      Collection values = entry.getValue();
      if (renameToPrefix != null) {
        record.getFields().putAll(renameToPrefix + key, values);
      }
      values.clear(); // implicitly removes key from record
    }
  }
  return super.doProcess(record);
}
开发者ID:europeana，项目名称:search，代码行数:17，代码来源:SanitizeUnknownSolrFieldsBuilder.java


示例11: doProcess

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Override
protected boolean doProcess(Record record) {
  try {
    List outputValues = record.get(outputFieldName);
    for (Object value : record.get(inputFieldName)) {
      reader.setValue(value.toString());
      TokenStream tokenStream = analyzer.tokenStream("content", reader);
      tokenStream.reset();
      while (tokenStream.incrementToken()) {
        if (token.length() > 0) { // incrementToken() updates the token!
          String tokenStr = new String(token.buffer(), 0, token.length());
          outputValues.add(tokenStr);
        }
      }
      tokenStream.end();
      tokenStream.close();
    }
  } catch (IOException e) {
    throw new MorphlineRuntimeException(e);
  }
  
  // pass record to next command in chain:
  return super.doProcess(record);
}
开发者ID:europeana，项目名称:search，代码行数:25，代码来源:TokenizeTextBuilder.java


示例12: testLoadSolrBasic

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void testLoadSolrBasic() throws Exception {
  //System.setProperty("ENV_SOLR_HOME", testSolrHome + "/collection1");
  morphline = createMorphline("test-morphlines/loadSolrBasic");    
  //System.clearProperty("ENV_SOLR_HOME");
  Record record = new Record();
  record.put(Fields.ID, "id0");
  record.put("first_name", "Nadja"); // will be sanitized
  startSession();
  Notifications.notifyBeginTransaction(morphline);
  assertTrue(morphline.process(record));
  assertEquals(1, collector.getNumStartEvents());
  Notifications.notifyCommitTransaction(morphline);
  Record expected = new Record();
  expected.put(Fields.ID, "id0");
  assertEquals(Arrays.asList(expected), collector.getRecords());
  assertEquals(1, queryResultSetSize("*:*"));
  Notifications.notifyRollbackTransaction(morphline);
  Notifications.notifyShutdown(morphline);
}
开发者ID:europeana，项目名称:search，代码行数:21，代码来源:SolrMorphlineTest.java


示例13: testTokenizeText

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void testTokenizeText() throws Exception {
  morphline = createMorphline("test-morphlines" + File.separator + "tokenizeText");
  for (int i = 0; i < 3; i++) {
    Record record = new Record();
    record.put(Fields.MESSAGE, "Hello World!");
    record.put(Fields.MESSAGE, "\[email protected] #%()123");
    Record expected = record.copy();
    expected.getFields().putAll("tokens", Arrays.asList("hello", "world", "foo", "bar.com", "123"));
    collector.reset();
    startSession();
    Notifications.notifyBeginTransaction(morphline);
    assertTrue(morphline.process(record));
    assertEquals(1, collector.getNumStartEvents());
    Notifications.notifyCommitTransaction(morphline);
    assertEquals(expected, collector.getFirstRecord());
  }
}
开发者ID:europeana，项目名称:search，代码行数:19，代码来源:SolrMorphlineTest.java


示例14: assertTweetEquals

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
private void assertTweetEquals(GenericData.Record expected, Record actual, int i) {
    Preconditions.checkNotNull(expected);
    Preconditions.checkNotNull(actual);
//    System.out.println("\n\nexpected: " + toString(expected));
//    System.out.println("actual:   " + actual);
    String[] fieldNames = new String[] { 
        "id", 
        "in_reply_to_status_id", 
        "in_reply_to_user_id", 
        "retweet_count",
        "text", 
        };
    for (String fieldName : fieldNames) {
      assertEquals(
          i + " fieldName: " + fieldName, 
          expected.get(fieldName).toString(), 
          actual.getFirstValue(fieldName).toString());
    }
  }
开发者ID:europeana，项目名称:search，代码行数:20，代码来源:SolrMorphlineZkAvroTest.java


示例15: doProcess

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Override
protected boolean doProcess(Record record) {
    ListIterator iter = record.get(fieldName).listIterator();
    while (iter.hasNext()) {
        String[] segments = iter.next().toString().split(separator);
        iter.remove();
        for (int i = 0; i < segments.length; i++) {
            if (i < newFields.size()) {
                record.put(newFields.get(i), trimIfNeeded(segments[i]));
            } else {
                if (!dropUndeclaredField) {
                   record.put(String.valueOf(i), trimIfNeeded(segments[i]));
                }
            }
        }
    }
    return super.doProcess(record);
}
开发者ID:sequenceiq，项目名称:sequenceiq-samples，代码行数:19，代码来源:SplitBuilder.java


示例16: testAddValues

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void testAddValues() throws Exception {
  morphline = createMorphline("test-morphlines/addValues");    
  Record record = new Record();
  record.put("first_name", "Nadja");
  Record expected = new Record();
  expected.put("first_name", "Nadja");
  expected.put("source_type", "text/log");
  expected.put("source_type", "text/log2");
  expected.put("source_host", "123");
  expected.put("name", "Nadja");
  expected.put("names", "@{first_name}");
  expected.put("pids", 456);
  expected.put("pids", "hello");
  processAndVerifySuccess(record, expected);
}
开发者ID:kite-sdk，项目名称:kite-examples，代码行数:17，代码来源:ExampleMorphlineTest.java


示例17: testGrokSyslogNgCisco

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void testGrokSyslogNgCisco() throws Exception {
  morphline = createMorphline("test-morphlines/grokSyslogNgCisco");
  Record record = new Record();
  String msg = "<179>Jun 10 04:42:51 www.foo.com Jun 10 2013 04:42:51 : %myproduct-3-mysubfacility-251010: " +
      "Health probe failed for server 1.2.3.4 on port 8083, connection refused by server";
  record.put(Fields.MESSAGE, msg);
  assertTrue(morphline.process(record));
  Record expected = new Record();
  expected.put(Fields.MESSAGE, msg);
  expected.put("cisco_message_code", "%myproduct-3-mysubfacility-251010");
  expected.put("cisco_product", "myproduct");
  expected.put("cisco_level", "3");
  expected.put("cisco_subfacility", "mysubfacility");
  expected.put("cisco_message_id", "251010");
  expected.put("syslog_message", "%myproduct-3-mysubfacility-251010: Health probe failed for server 1.2.3.4 " +
      "on port 8083, connection refused by server");
  assertEquals(expected, collector.getFirstRecord());
  assertNotSame(record, collector.getFirstRecord());      
}
开发者ID:kite-sdk，项目名称:kite-examples，代码行数:21，代码来源:ExampleMorphlineTest.java


示例18: testGrokSyslogNgCiscoWithoutSubFacility

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
public void testGrokSyslogNgCiscoWithoutSubFacility() throws Exception {
    morphline = createMorphline("test-morphlines/grokSyslogNgCisco");
    Record record = new Record();
    String msg = "<179>Jun 10 04:42:51 www.foo.com Jun 10 2013 04:42:51 : %myproduct-3-mysubfacility-251010: " +
        "Health probe failed for server 1.2.3.4 on port 8083, connection refused by server";
    record.put(Fields.MESSAGE, msg);
    assertTrue(morphline.process(record));
    Record expected = new Record();
    expected.put(Fields.MESSAGE, msg);
    expected.put("cisco_message_code", "%myproduct-3-251010");
    expected.put("cisco_product", "myproduct");
    expected.put("cisco_level", "3");
//    expected.put("cisco_subfacility", "mysubfacility");
    expected.put("cisco_message_id", "251010");
    expected.put("syslog_message", "%myproduct-3-mysubfacility-251010: Health probe failed for server 1.2.3.4 " +
        "on port 8083, connection refused by server");
    assertEquals(expected, collector.getFirstRecord());
    assertNotSame(record, collector.getFirstRecord());      
  }
开发者ID:kite-sdk，项目名称:kite-examples，代码行数:20，代码来源:ExampleMorphlineTest.java


示例19: testGrokEmail

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void testGrokEmail() throws Exception {
  morphline = createMorphline("test-morphlines/grokEmail");
  Record record = new Record();
  byte[] bytes = Files.toByteArray(new File(RESOURCES_DIR + "/test-documents/email.txt"));
  record.put(Fields.ATTACHMENT_BODY, bytes);
  assertTrue(morphline.process(record));
  Record expected = new Record();
  String msg = new String(bytes, "UTF-8"); //.replaceAll("(\r)?\n", "\n");
  expected.put(Fields.MESSAGE, msg);
  expected.put("message_id", "[email protected]");
  expected.put("date", "Wed, 6 Feb 2012 06:06:05 -0800");
  expected.put("from", "[email protected]");
  expected.put("to", "[email protected]");
  expected.put("subject", "WEDNESDAY WEATHER HEADLINES");
  expected.put("from_names", "Foo Bar <[email protected]>@xxx");
  expected.put("to_names", "'Weather News Distribution' <[email protected]>");    
  expected.put("text", 
      "Average 1 to 3- degrees above normal: Mid-Atlantic, Southern Plains.." +
      "\nAverage 4 to 6-degrees above normal: Ohio Valley, Rockies, Central Plains");
  assertEquals(expected, collector.getFirstRecord());
  assertNotSame(record, collector.getFirstRecord());      
}
开发者ID:kite-sdk，项目名称:kite-examples，代码行数:24，代码来源:ExampleMorphlineTest.java


示例20: testExtractJsonPathsFlattened

import org.kitesdk.morphline.api.Record; //导入依赖的package包/类
@Test
public void testExtractJsonPathsFlattened() throws Exception {
  morphline = createMorphline("test-morphlines/extractJsonPathsFlattened");    
  File file = new File(RESOURCES_DIR + "/test-documents/arrays.json");
  InputStream in = new BufferedInputStream(new FileInputStream(file));
  Record record = new Record();
  record.put(Fields.ATTACHMENT_BODY, in);
  
  startSession();
  assertEquals(1, collector.getNumStartEvents());
  assertTrue(morphline.process(record));    
  
  assertEquals(1, collector.getRecords().size());
  List expected = Arrays.asList(1, 2, 3, 4, 5, 10, 20, 100, 200);
  assertEquals(1, collector.getRecords().size());
  assertEquals(expected, collector.getFirstRecord().get("/price"));
  assertEquals(expected, collector.getFirstRecord().get("/price/[]"));
  assertEquals(Arrays.asList(), collector.getFirstRecord().get("/unknownField"));

  in.close();
}
开发者ID:kite-sdk，项目名称:kite-examples，代码行数:22，代码来源:ExampleMorphlineTest.java



注：本文中的org.kitesdk.morphline.api.Record类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。

鲜花
握手
雷人
路过
鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
More+
上一篇：
Java NotIdentifiableEventImpl类代码示例发布时间：2022-05-22
下一篇：
Java CompileUnit类代码示例发布时间：2022-05-22
热门推荐
More+
热门话题
More+
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服（服务时间 9:00～18:00）

在线QQ客服
地址：深圳市南山区西丽大学城创智工业园
电邮：jeky_zhao#qq.com
移动电话：139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap