• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java RecordMetaData类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.datavec.api.records.metadata.RecordMetaData的典型用法代码示例。如果您正苦于以下问题:Java RecordMetaData类的具体用法?Java RecordMetaData怎么用?Java RecordMetaData使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



RecordMetaData类属于org.datavec.api.records.metadata包,在下文中一共展示了RecordMetaData类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testCodecReaderMeta

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Test
public void testCodecReaderMeta() throws Exception {
    File file = new ClassPathResource("fire_lowres.mp4").getFile();
    SequenceRecordReader reader = new CodecRecordReader();
    Configuration conf = new Configuration();
    conf.set(CodecRecordReader.RAVEL, "true");
    conf.set(CodecRecordReader.START_FRAME, "160");
    conf.set(CodecRecordReader.TOTAL_FRAMES, "500");
    conf.set(CodecRecordReader.ROWS, "80");
    conf.set(CodecRecordReader.COLUMNS, "46");
    reader.initialize(new FileSplit(file));
    reader.setConf(conf);
    assertTrue(reader.hasNext());
    List<List<Writable>> record = reader.sequenceRecord();
    assertEquals(500, record.size()); //500 frames

    reader.reset();
    SequenceRecord seqR = reader.nextSequence();
    assertEquals(record, seqR.getSequenceRecord());
    RecordMetaData meta = seqR.getMetaData();
    //        System.out.println(meta);
    assertTrue(meta.getURI().toString().endsWith("fire_lowres.mp4"));

    SequenceRecord fromMeta = reader.loadSequenceFromMetaData(meta);
    assertEquals(seqR, fromMeta);
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:27,代码来源:CodecReaderTest.java


示例2: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
/**
 * @see #loadFromMetaData(RecordMetaData)
 */
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<Record> ret = new ArrayList<>();

    for (RecordMetaData rmd : recordMetaDatas) {
        if (!(rmd instanceof RecordMetaDataJdbc)) {
            throw new IllegalArgumentException(
                "Invalid metadata; expected RecordMetaDataJdbc instance; got: " + rmd);
        }
        QueryRunner runner = new QueryRunner();
        String request = ((RecordMetaDataJdbc) rmd).getRequest();

        try {
            Object[] item = runner
                .query(this.conn, request, new ArrayHandler(), ((RecordMetaDataJdbc) rmd).getParams().toArray());
            ret.add(new org.datavec.api.records.impl.Record(toWritable(item), rmd));
        } catch (SQLException e) {
            throw new IllegalArgumentException("Could not execute statement \"" + request + "\"", e);
        }
    }
    return ret;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:26,代码来源:JDBCRecordReader.java


示例3: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {

    List<Record> out = new ArrayList<>();
    for (RecordMetaData metaData : recordMetaDatas) {
        URI uri = metaData.getURI();

        String fileAsString;
        try {
            fileAsString = FileUtils.readFileToString(new File(uri.toURL().getFile()));
        } catch (IOException e) {
            throw new RuntimeException("Error reading URI file", e);
        }

        List<Writable> writables = readValues(uri, fileAsString);
        out.add(new org.datavec.api.records.impl.Record(writables, metaData));
    }

    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:21,代码来源:JacksonRecordReader.java


示例4: testsBasicMeta

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Test
public void testsBasicMeta() throws Exception {
    //As per testBasic - but also loading metadata
    RecordReader rr2 = new CSVRecordReader(0, ',');
    rr2.initialize(new FileSplit(new ClassPathResource("iris.txt").getTempFileFromArchive()));

    RecordReaderMultiDataSetIterator rrmdsi = new RecordReaderMultiDataSetIterator.Builder(10)
                    .addReader("reader", rr2).addInput("reader", 0, 3).addOutputOneHot("reader", 4, 3).build();

    rrmdsi.setCollectMetaData(true);

    int count = 0;
    while (rrmdsi.hasNext()) {
        MultiDataSet mds = rrmdsi.next();
        MultiDataSet fromMeta = rrmdsi.loadFromMetaData(mds.getExampleMetaData(RecordMetaData.class));
        assertEquals(mds, fromMeta);
        count++;
    }
    assertEquals(150 / 10, count);
}
 
开发者ID:deeplearning4j,项目名称:deeplearning4j,代码行数:21,代码来源:RecordReaderMultiDataSetIteratorTest.java


示例5: testSplittingCSVMeta

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Test
public void testSplittingCSVMeta() throws Exception {
    //Here's the idea: take Iris, and split it up into 2 inputs and 2 output arrays
    //Inputs: columns 0 and 1-2
    //Outputs: columns 3, and 4->OneHot
    RecordReader rr2 = new CSVRecordReader(0, ',');
    rr2.initialize(new FileSplit(new ClassPathResource("iris.txt").getTempFileFromArchive()));

    RecordReaderMultiDataSetIterator rrmdsi = new RecordReaderMultiDataSetIterator.Builder(10)
                    .addReader("reader", rr2).addInput("reader", 0, 0).addInput("reader", 1, 2)
                    .addOutput("reader", 3, 3).addOutputOneHot("reader", 4, 3).build();
    rrmdsi.setCollectMetaData(true);

    int count = 0;
    while (rrmdsi.hasNext()) {
        MultiDataSet mds = rrmdsi.next();
        MultiDataSet fromMeta = rrmdsi.loadFromMetaData(mds.getExampleMetaData(RecordMetaData.class));
        assertEquals(mds, fromMeta);
        count++;
    }
    assertEquals(150 / 10, count);
}
 
开发者ID:deeplearning4j,项目名称:deeplearning4j,代码行数:23,代码来源:RecordReaderMultiDataSetIteratorTest.java


示例6: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<Record> out = new ArrayList<>();
    for (RecordMetaData meta : recordMetaDatas) {
        URI uri = meta.getURI();
        File f = new File(uri);

        List<Writable> next;
        try (DataInputStream dis = new DataInputStream(new BufferedInputStream(new FileInputStream(f)))) {
            next = record(uri, dis);
        }
        out.add(new org.datavec.api.records.impl.Record(next, meta));
    }
    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:16,代码来源:BaseImageRecordReader.java


示例7: testMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Test
public void testMetaData() throws IOException {

    ClassPathResource cpr = new ClassPathResource("/testimages/class0/0.jpg");
    File parentDir = cpr.getFile().getParentFile().getParentFile();
    //        System.out.println(f.getAbsolutePath());
    //        System.out.println(f.getParentFile().getParentFile().getAbsolutePath());
    ParentPathLabelGenerator labelMaker = new ParentPathLabelGenerator();
    ImageRecordReader rr = new ImageRecordReader(32, 32, 3, labelMaker);
    rr.initialize(new FileSplit(parentDir));

    List<List<Writable>> out = new ArrayList<>();
    while (rr.hasNext()) {
        List<Writable> l = rr.next();
        out.add(l);
        assertEquals(2, l.size());
    }

    assertEquals(6, out.size());

    rr.reset();
    List<List<Writable>> out2 = new ArrayList<>();
    List<Record> out3 = new ArrayList<>();
    List<RecordMetaData> meta = new ArrayList<>();

    while (rr.hasNext()) {
        Record r = rr.nextRecord();
        out2.add(r.getRecord());
        out3.add(r);
        meta.add(r.getMetaData());
        //            System.out.println(r.getMetaData() + "\t" + r.getRecord().get(1));
    }

    assertEquals(out, out2);

    List<Record> fromMeta = rr.loadFromMetaData(meta);
    assertEquals(out3, fromMeta);
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:39,代码来源:TestImageRecordReader.java


示例8: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<Record> out = new ArrayList<>();

    for (Record fileContents : super.loadFromMetaData(recordMetaDatas)) {
        INDArray transform = tfidfVectorizer.transform(fileContents);

        org.datavec.api.records.impl.Record record = new org.datavec.api.records.impl.Record(
                        new ArrayList<>(Collections.<Writable>singletonList(new NDArrayWritable(transform))),
                        new RecordMetaDataURI(fileContents.getMetaData().getURI(), TfidfRecordReader.class));

        if (appendLabel)
            record.getRecord().add(fileContents.getRecord().get(fileContents.getRecord().size() - 1));
        out.add(record);
    }

    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:19,代码来源:TfidfRecordReader.java


示例9: loadSequenceFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<SequenceRecord> loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<SequenceRecord> out = new ArrayList<>();
    for (RecordMetaData meta : recordMetaDatas) {
        File f = new File(meta.getURI());

        List<List<Writable>> list = loadData(f, null);
        out.add(new org.datavec.api.records.impl.SequenceRecord(list, meta));
    }

    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:13,代码来源:BaseCodecRecordReader.java


示例10: testNativeCodecReaderMeta

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Ignore
@Test
public void testNativeCodecReaderMeta() throws Exception {
    File file = new ClassPathResource("fire_lowres.mp4").getFile();
    SequenceRecordReader reader = new NativeCodecRecordReader();
    Configuration conf = new Configuration();
    conf.set(CodecRecordReader.RAVEL, "true");
    conf.set(CodecRecordReader.START_FRAME, "160");
    conf.set(CodecRecordReader.TOTAL_FRAMES, "500");
    conf.set(CodecRecordReader.ROWS, "80");
    conf.set(CodecRecordReader.COLUMNS, "46");
    reader.initialize(new FileSplit(file));
    reader.setConf(conf);
    assertTrue(reader.hasNext());
    List<List<Writable>> record = reader.sequenceRecord();
    assertEquals(500, record.size()); //500 frames

    reader.reset();
    SequenceRecord seqR = reader.nextSequence();
    assertEquals(record, seqR.getSequenceRecord());
    RecordMetaData meta = seqR.getMetaData();
    //        System.out.println(meta);
    assertTrue(meta.getURI().toString().endsWith("fire_lowres.mp4"));

    SequenceRecord fromMeta = reader.loadSequenceFromMetaData(meta);
    assertEquals(seqR, fromMeta);
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:28,代码来源:CodecReaderTest.java


示例11: testNextRecord

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Test
public void testNextRecord() throws Exception {
    try (JDBCRecordReader reader = getInitializedReader("SELECT * FROM Coffee")) {
        Record r = reader.nextRecord();
        List<Writable> fields = r.getRecord();
        RecordMetaData meta = r.getMetaData();
        assertNotNull(r);
        assertNotNull(fields);
        assertNotNull(meta);
        assertEquals(new Text("Bolivian Dark"), fields.get(0));
        assertEquals(new Text("14-001"), fields.get(1));
        assertEquals(new DoubleWritable(8.95), fields.get(2));
        assertEquals(RecordMetaDataJdbc.class, meta.getClass());
    }
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:16,代码来源:JDBCRecordReaderTest.java


示例12: testNextRecordAndRecover

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Test
public void testNextRecordAndRecover() throws Exception {
    try (JDBCRecordReader reader = getInitializedReader("SELECT * FROM Coffee")) {
        Record r = reader.nextRecord();
        List<Writable> fields = r.getRecord();
        RecordMetaData meta = r.getMetaData();
        Record recovered = reader.loadFromMetaData(meta);
        List<Writable> fieldsRecovered = recovered.getRecord();
        assertEquals(fields.size(), fieldsRecovered.size());
        for (int i = 0; i < fields.size(); i++) {
            assertEquals(fields.get(i), fieldsRecovered.get(i));
        }
    }
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:15,代码来源:JDBCRecordReaderTest.java


示例13: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
/**
 * Load multiple records from the given a list of {@link RecordMetaData} instances<br>
 *
 * @param recordMetaDatas Metadata for the records that we want to load from
 * @return Multiple records for the given RecordMetaData instances
 * @throws IOException If I/O error occurs during loading
 */
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<Record> records = sequenceRecordReader.loadFromMetaData(recordMetaDatas);
    for (Record record : records)
        record.setRecord(transformProcess.execute(record.getRecord()));
    return records;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:15,代码来源:TransformProcessSequenceRecordReader.java


示例14: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<Record> out = new ArrayList<>();

    for (RecordMetaData meta : recordMetaDatas) {
        URI uri = meta.getURI();

        File f = new File(uri);
        List<Writable> list = loadFromFile(f);
        out.add(new org.datavec.api.records.impl.Record(list, meta));
    }

    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:15,代码来源:FileRecordReader.java


示例15: nextRecord

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
/**
 * Return next Record.
 *
 * @return
 */
@Override
public Record nextRecord() {
    List<Writable> next = next();
    URI uri = (locations == null || locations.length < 1 ? null : locations[splitIndex]);
    RecordMetaData meta = new RecordMetaDataLine(this.lineIndex - 1, uri, SVMLightRecordReader.class); //-1 as line number has been incremented already...
    return new org.datavec.api.records.impl.Record(next, meta);
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:13,代码来源:SVMLightRecordReader.java


示例16: loadSequenceFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<SequenceRecord> loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<SequenceRecord> out = new ArrayList<>();
    for (RecordMetaData meta : recordMetaDatas) {
        File next = new File(meta.getURI());
        URI uri = next.toURI();
        String fileContents = FileUtils.readFileToString(next, charset.name());
        List<List<Writable>> sequence = loadSequence(fileContents, uri);
        out.add(new org.datavec.api.records.impl.SequenceRecord(sequence, meta));
    }
    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:13,代码来源:RegexSequenceRecordReader.java


示例17: nextRecord

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public Record nextRecord() {
    List<Writable> next = next();
    URI uri = (locations == null || locations.length < 1 ? null : locations[splitIndex]);
    RecordMetaData meta = new RecordMetaDataLine(this.lineIndex - 1, uri, RegexLineRecordReader.class); //-1 as line number has been incremented already...
    return new org.datavec.api.records.impl.Record(next, meta);
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:8,代码来源:RegexLineRecordReader.java


示例18: loadFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    List<Record> list = super.loadFromMetaData(recordMetaDatas);

    for (Record r : list) {
        String line = r.getRecord().get(0).toString();
        r.setRecord(parseLine(line));
    }

    return list;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:12,代码来源:RegexLineRecordReader.java


示例19: nextRecord

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public Record nextRecord() {
    List<Writable> next = next();
    URI uri = (locations == null || locations.length < 1 ? null : locations[splitIndex]);
    RecordMetaData meta = new RecordMetaDataLine(this.lineIndex - 1, uri, LineRecordReader.class); //-1 as line number has been incremented already...
    return new org.datavec.api.records.impl.Record(next, meta);
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:8,代码来源:LineRecordReader.java


示例20: loadSequenceFromMetaData

import org.datavec.api.records.metadata.RecordMetaData; //导入依赖的package包/类
@Override
public List<SequenceRecord> loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException {
    Set<Integer> toLoad = new LinkedHashSet<>();
    for (RecordMetaData recordMetaData : recordMetaDatas) {
        if (!(recordMetaData instanceof RecordMetaDataIndex)) {
            throw new IllegalArgumentException("Expected RecordMetaDataIndex; got: " + recordMetaData);
        }
        long idx = ((RecordMetaDataIndex) recordMetaData).getIndex();
        if (idx >= original.size()) {
            throw new IllegalStateException(
                            "Cannot get index " + idx + " from collection: contains " + original + " elements");
        }
        toLoad.add((int) idx);
    }

    List<SequenceRecord> out = new ArrayList<>();
    Iterator<? extends Collection<? extends Collection<Writable>>> iter = original.iterator();
    int i = 0;
    while (iter.hasNext()) {
        Collection<? extends Collection<Writable>> c = iter.next();
        if (!toLoad.contains(i++)) {
            continue;
        }
        List<List<Writable>> record = toList(c);
        SequenceRecord r = new org.datavec.api.records.impl.SequenceRecord(record,
                        new RecordMetaDataIndex(i - 1, null, CollectionSequenceRecordReader.class));
        out.add(r);
    }
    return out;
}
 
开发者ID:deeplearning4j,项目名称:DataVec,代码行数:31,代码来源:CollectionSequenceRecordReader.java



注:本文中的org.datavec.api.records.metadata.RecordMetaData类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java IdUserGroup类代码示例发布时间:2022-05-22
下一篇:
Java GeneralizedCounter类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap