• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java IOUtil类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中com.hankcs.hanlp.corpus.io.IOUtil的典型用法代码示例。如果您正苦于以下问题:Java IOUtil类的具体用法?Java IOUtil怎么用?Java IOUtil使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



IOUtil类属于com.hankcs.hanlp.corpus.io包,在下文中一共展示了IOUtil类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testExtract

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
public void testExtract() throws Exception
{
    List<File> fileList = FolderWalker.open(FOLDER);
    Map<String, String> phraseMap = new TreeMap<String, String>();
    int i = 0;
    for (File file : fileList)
    {
        System.out.print(++i + " / " + fileList.size() + " " + file.getName() + " ");
        String path = file.getAbsolutePath();
        List<String> phraseList = MutualInformationEntropyPhraseExtractor.extract(IOUtil.readTxt(path), 3);
        System.out.print(phraseList);
        for (String phrase : phraseList)
        {
            phraseMap.put(phrase, file.getAbsolutePath());
        }
        System.out.println();
    }
    BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream("data/phrase.txt")));
    for (Map.Entry<String, String> entry : phraseMap.entrySet())
    {
        bw.write(entry.getKey() + "\t" + entry.getValue());
        bw.newLine();
    }
    bw.close();
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:26,代码来源:TestPhrase.java


示例2: load

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
private static boolean load(String path)
{
    String binPath = path + Predefine.BIN_EXT;
    if (loadBin(binPath)) return true;
    CONVERT = new char[Character.MAX_VALUE + 1];
    for (int i = 0; i < CONVERT.length; i++)
    {
        CONVERT[i] = (char) i;
    }
    IOUtil.LineIterator iterator = new IOUtil.LineIterator(path);
    while (iterator.hasNext())
    {
        String line = iterator.next();
        if (line == null) return false;
        if (line.length() != 3) continue;
        CONVERT[line.charAt(0)] = CONVERT[line.charAt(2)];
    }
    logger.info("正在缓存字符正规化表到" + binPath);
    IOUtil.saveObjectTo(CONVERT, binPath);

    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:23,代码来源:CharTable.java


示例3: loadBin

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
private static boolean loadBin(String path)
{
    try
    {
        ObjectInputStream in = new ObjectInputStream(IOUtil.newInputStream(path));
        CONVERT = (char[]) in.readObject();
        in.close();
    }
    catch (Exception e)
    {
        logger.warning("字符正规化表缓存加载失败,原因如下:" + e);
        return false;
    }

    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:17,代码来源:CharTable.java


示例4: saveDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
static boolean saveDat(String path, TreeMap<String, Integer> map)
{
    try
    {
        DataOutputStream out = new DataOutputStream(new BufferedOutputStream(IOUtil.newOutputStream(path)));
        Collection<Integer> freqList = map.values();
        out.writeInt(freqList.size());
        for (int freq : freqList)
        {
            out.writeInt(freq);
        }
        trie.save(out);
        out.close();
    }
    catch (Exception e)
    {
        logger.log(Level.WARNING, "在缓存" + path + "时发生异常", e);
        return false;
    }

    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:23,代码来源:CoreBiGramMixDictionary.java


示例5: saveDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
/**
 * 保存dat到磁盘
 * @param map
 * @return
 */
static boolean saveDat(TreeMap<String, Character> map)
{
    try
    {
        DataOutputStream out = new DataOutputStream(IOUtil.newOutputStream(path + Predefine.VALUE_EXT));
        out.writeInt(map.size());
        for (Character character : map.values())
        {
            out.writeChar(character);
        }
        out.close();
    }
    catch (Exception e)
    {
        logger.warning("保存值" + path + Predefine.VALUE_EXT + "失败" + e);
        return false;
    }
    return trie.save(path + Predefine.TRIE_EXT);
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:25,代码来源:JapanesePersonDictionary.java


示例6: loadDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
private EnumItem<NR>[] loadDat(String path)
{
    byte[] bytes = IOUtil.readBytes(path);
    if (bytes == null) return null;
    NR[] nrArray = NR.values();
    int index = 0;
    int size = ByteUtil.bytesHighFirstToInt(bytes, index);
    index += 4;
    EnumItem<NR>[] valueArray = new EnumItem[size];
    for (int i = 0; i < size; ++i)
    {
        int currentSize = ByteUtil.bytesHighFirstToInt(bytes, index);
        index += 4;
        EnumItem<NR> item = new EnumItem<NR>();
        for (int j = 0; j < currentSize; ++j)
        {
            NR nr = nrArray[ByteUtil.bytesHighFirstToInt(bytes, index)];
            index += 4;
            int frequency = ByteUtil.bytesHighFirstToInt(bytes, index);
            index += 4;
            item.labelMap.put(nr, frequency);
        }
        valueArray[i] = item;
    }
    return valueArray;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:27,代码来源:NRDictionary.java


示例7: saveDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
private boolean saveDat(String path, EnumItem<NS>[] valueArray)
{
    try
    {
        DataOutputStream out = new DataOutputStream(IOUtil.newOutputStream(path));
        out.writeInt(valueArray.length);
        for (EnumItem<NS> item : valueArray)
        {
            out.writeInt(item.labelMap.size());
            for (Map.Entry<NS, Integer> entry : item.labelMap.entrySet())
            {
                out.writeInt(entry.getKey().ordinal());
                out.writeInt(entry.getValue());
            }
        }
        out.close();
    }
    catch (Exception e)
    {
        logger.warning("保存失败" + e);
        return false;
    }
    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:25,代码来源:NSDictionary.java


示例8: load

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
public boolean load(String path)
{
    trie = new BinTrie<Byte>();
    if (loadDat(path + Predefine.TRIE_EXT)) return true;
    String line = null;
    try
    {
        BufferedReader bw = new BufferedReader(new InputStreamReader(IOUtil.newInputStream(path)));
        while ((line = bw.readLine()) != null)
        {
            trie.put(line, null);
        }
        bw.close();
    }
    catch (Exception e)
    {
        logger.warning("加载" + path + "失败," + e);
    }
    if (!trie.save(path + Predefine.TRIE_EXT)) logger.warning("缓存" + path + Predefine.TRIE_EXT + "失败");
    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:22,代码来源:CommonStringDictionary.java


示例9: loadDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
private EnumItem<NT>[] loadDat(String path)
{
    byte[] bytes = IOUtil.readBytes(path);
    if (bytes == null) return null;
    NT[] values = NT.values();
    int index = 0;
    int size = ByteUtil.bytesHighFirstToInt(bytes, index);
    index += 4;
    EnumItem<NT>[] valueArray = new EnumItem[size];
    for (int i = 0; i < size; ++i)
    {
        int currentSize = ByteUtil.bytesHighFirstToInt(bytes, index);
        index += 4;
        EnumItem<NT> item = new EnumItem<NT>();
        for (int j = 0; j < currentSize; ++j)
        {
            NT tag = values[ByteUtil.bytesHighFirstToInt(bytes, index)];
            index += 4;
            int frequency = ByteUtil.bytesHighFirstToInt(bytes, index);
            index += 4;
            item.labelMap.put(tag, frequency);
        }
        valueArray[i] = item;
    }
    return valueArray;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:27,代码来源:NTDictionary.java


示例10: saveDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
private boolean saveDat(String path, EnumItem<NT>[] valueArray)
{
    try
    {
        DataOutputStream out = new DataOutputStream(IOUtil.newOutputStream(path));
        out.writeInt(valueArray.length);
        for (EnumItem<NT> item : valueArray)
        {
            out.writeInt(item.labelMap.size());
            for (Map.Entry<NT, Integer> entry : item.labelMap.entrySet())
            {
                out.writeInt(entry.getKey().ordinal());
                out.writeInt(entry.getValue());
            }
        }
        out.close();
    }
    catch (Exception e)
    {
        logger.warning("保存失败" + e);
        return false;
    }
    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:25,代码来源:NTDictionary.java


示例11: saveDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
static boolean saveDat(String path, AhoCorasickDoubleArrayTrie<Pinyin[]> trie, Set<Map.Entry<String, Pinyin[]>> entrySet)
{
    try
    {
        DataOutputStream out = new DataOutputStream(IOUtil.newOutputStream(path + Predefine.BIN_EXT));
        out.writeInt(entrySet.size());
        for (Map.Entry<String, Pinyin[]> entry : entrySet)
        {
            Pinyin[] value = entry.getValue();
            out.writeInt(value.length);
            for (Pinyin pinyin : value)
            {
                out.writeInt(pinyin.ordinal());
            }
        }
        trie.save(out);
        out.close();
    }
    catch (Exception e)
    {
        logger.warning("缓存值dat" + path + "失败");
        return false;
    }

    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:27,代码来源:PinyinDictionary.java


示例12: load

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
static boolean load(String path)
{
    trie = new DoubleArrayTrie<String>();
    if (loadDat(path + ".bi" + Predefine.BIN_EXT)) return true;
    TreeMap<String, String> map = new TreeMap<String, String>();
    for (String line : IOUtil.readLineListWithLessMemory(path))
    {
        String[] param = line.split(" ");
        if (param[0].endsWith("@"))
        {
            continue;
        }
        String dependency = param[1];
        map.put(param[0], dependency);
    }
    if (map.size() == 0) return false;
    trie.build(map);
    if (!saveDat(path, map)) logger.warning("缓存" + path + Predefine.BIN_EXT + "失败");
    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:21,代码来源:BigramDependencyModel.java


示例13: saveDat

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
static boolean saveDat(String path, TreeMap<String, String> map)
{
    Collection<String> dependencyList = map.values();
    // 缓存值文件
    try
    {
        DataOutputStream out = new DataOutputStream(IOUtil.newOutputStream(path +  ".bi" + Predefine.BIN_EXT));
        out.writeInt(dependencyList.size());
        for (String dependency : dependencyList)
        {
            out.writeUTF(dependency);
        }
        if (!trie.save(out)) return false;
        out.close();
    }
    catch (Exception e)
    {
        logger.warning("保存失败" + e);
        return false;
    }
    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:23,代码来源:BigramDependencyModel.java


示例14: saveTxtTo

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
@Override
public boolean saveTxtTo(String path)
{
    if (trie.size() == 0) return true;  // 如果没有词条,那也算成功了
    try
    {
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(IOUtil.newOutputStream(path), "UTF-8"));
        Set<Map.Entry<String, Item>> entries = trie.entrySet();
        for (Map.Entry<String, Item> entry : entries)
        {
            bw.write(entry.getValue().toString());
            bw.newLine();
        }
        bw.close();
    }
    catch (Exception e)
    {
        logger.warning("保存到" + path + "失败" + e);
        return false;
    }

    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:24,代码来源:DictionaryMaker.java


示例15: saveTxtTo

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
@Override
public boolean saveTxtTo(String path)
{
    try
    {
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(IOUtil.newOutputStream(path)));
        bw.write(toString());
        bw.close();
    }
    catch (Exception e)
    {
        logger.warning("在保存转移矩阵词典到" + path + "时发生异常" + e);
        return false;
    }
    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:17,代码来源:TMDictionaryMaker.java


示例16: save

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
/**
 * 保存词典
 * @param path
 * @return 是否成功
 */
public boolean save(String path)
{
    try
    {
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(IOUtil.newOutputStream(path)));
        for (Map.Entry<String, String> entry : trie.entrySet())
        {
            bw.write(entry.getKey());
            bw.write(separator);
            bw.write(entry.getValue());
            bw.newLine();
        }
        bw.close();
    }
    catch (Exception e)
    {
        logger.warning("保存词典到" + path + "失败");
        return true;
    }
    return false;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:27,代码来源:StringDictionary.java


示例17: saveNGramToTxt

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
/**
 * 保存NGram词典
 *
 * @param path
 * @return
 */
public boolean saveNGramToTxt(String path)
{
    try
    {
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(IOUtil.newOutputStream(path)));
        for (Map.Entry<String, Integer> entry : trie.entrySet())
        {
            bw.write(entry.getKey() + " " + entry.getValue());
            bw.newLine();
        }
        bw.close();
    }
    catch (Exception e)
    {
        logger.warning("在保存NGram词典到" + path + "时发生异常" + e);
        return false;
    }

    return true;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:27,代码来源:NGramDictionaryMaker.java


示例18: fix

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
public static boolean fix(String path)
{
    StringBuilder sbOut = new StringBuilder();
    for (String line : IOUtil.readLineListWithLessMemory(path))
    {
        if (line.trim().length() == 0)
        {
            sbOut.append(line);
            sbOut.append('\n');
            continue;
        }
        String[] args = line.split("\t");
        for (int i = 10 - args.length; i > 0; --i)
        {
            line += "\t_";
        }
        sbOut.append(line);
        sbOut.append('\n');
    }
    return IOUtil.saveTxt(path + ".fixed.txt", sbOut.toString());
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:22,代码来源:CoNLLFixer.java


示例19: loadSentenceList

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
public static LinkedList<CoNLLSentence> loadSentenceList(String path)
{
    LinkedList<CoNLLSentence> result = new LinkedList<CoNLLSentence>();
    LinkedList<CoNllLine> lineList = new LinkedList<CoNllLine>();
    for (String line : IOUtil.readLineListWithLessMemory(path))
    {
        if (line.trim().length() == 0)
        {
            result.add(new CoNLLSentence(lineList));
            lineList = new LinkedList<CoNllLine>();
            continue;
        }
        lineList.add(new CoNllLine(line.split("\t")));
    }

    return result;
}
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:18,代码来源:CoNLLLoader.java


示例20: convert2Document

import com.hankcs.hanlp.corpus.io.IOUtil; //导入依赖的package包/类
public static Document convert2Document(File file)
    {
//        try
//        {
            Document document = Document.create(IOUtil.readTxt(file.getPath()));
            if (document != null)
            {
                return document;
            }
            else
            {
                System.exit(-1);
            }
//        }
//        catch (IOException e)
//        {
//            e.printStackTrace();
//        }
        return null;
    }
 
开发者ID:priester,项目名称:hanlpStudy,代码行数:21,代码来源:CorpusLoader.java



注:本文中的com.hankcs.hanlp.corpus.io.IOUtil类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java VCFContigHeaderLine类代码示例发布时间:2022-05-23
下一篇:
Java NativeJavaArray类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap