本文整理汇总了Java中no.priv.garshol.duke.Processor类的典型用法代码示例。如果您正苦于以下问题:Java Processor类的具体用法?Java Processor怎么用?Java Processor使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
Processor类属于no.priv.garshol.duke包,在下文中一共展示了Processor类的7个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: setup
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
@Before
public void setup() throws CorruptIndexException, IOException {
listener = new TestUtils.TestListener();
List<Property> props = new ArrayList();
GeopositionComparator comp = new GeopositionComparator();
comp.setMaxDistance(100);
props.add(new PropertyImpl("ID"));
props.add(new PropertyImpl("LOCATION", comp, 0.3, 0.9));
config = new ConfigurationImpl();
// Define database as LuceneDatabase (not default anymore)
config.addDatabase(new LuceneDatabase());
config.setProperties(props);
config.setThreshold(0.85);
config.setMaybeThreshold(0.8);
processor = new Processor(config, true);
processor.addMatchListener(listener);
}
开发者ID:enricopal,项目名称:STEM,代码行数:19,代码来源:GeoSearchingTest.java
示例2: setup
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
@Before
public void setup() throws CorruptIndexException, IOException {
listener = new TestUtils.TestListener();
Levenshtein comp = new Levenshtein();
List<Property> props = new ArrayList();
props.add(new PropertyImpl("ID"));
props.add(new PropertyImpl("NAME", comp, 0.3, 0.8));
props.add(new PropertyImpl("EMAIL", comp, 0.3, 0.8));
config = new ConfigurationImpl();
config.setProperties(props);
config.setThreshold(0.85);
config.setMaybeThreshold(0.8);
source1 = new InMemoryDataSource();
source2 = new InMemoryDataSource();
config.addDataSource(1, source1);
config.addDataSource(2, source2);
processor = new Processor(config, true);
processor.addMatchListener(listener);
}
开发者ID:enricopal,项目名称:STEM,代码行数:23,代码来源:RecordLinkTest.java
示例3: setup
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
@Before
public void setup() throws IOException {
listener = new TestUtils.TestListener();
Levenshtein comp = new Levenshtein();
List<Property> props = new ArrayList();
props.add(new PropertyImpl("ID"));
props.add(new PropertyImpl("NAME", comp, 0.3, 0.8));
props.add(new PropertyImpl("EMAIL", comp, 0.3, 0.8));
config = new ConfigurationImpl();
config.setProperties(props);
config.setThreshold(0.85);
config.setMaybeThreshold(0.8);
processor = new Processor(config, true);
processor.addMatchListener(listener);
}
开发者ID:enricopal,项目名称:STEM,代码行数:17,代码来源:DeduplicatorTest.java
示例4: subStitch
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
@Override
protected Map<B, Collection<C>> subStitch(Collection<B> filteredBaseElements,
Collection<C> filteredCandidateElements, Map<B, Collection<C>> stitches) {
// Set Datasources
DataSource baseDataSource, candidateDataSource;
baseDataSource = generateBaseDataSource(filteredBaseElements);
candidateDataSource = generateCandidateDataSource(filteredCandidateElements);
config.addDataSource(1, candidateDataSource); // NOTE: Read comment below
config.addDataSource(2, baseDataSource); // NOTE: Read comment below
// IMPORTANT NOTICE:
// Duke's linking process takes its data source number 1 and indexes it, then traverses its
// data source number 2 to look for matches from the first one. This means that Duke's
// candidates group is its data source 1 and its base group is its data source 2.
// As in this stitcher we are considering that the base data source is our first group and
// the candidate data source is our second group, these needs to be swapped to comply with
// Duke's behaviour. This explains why in the two previous lines our "candidateDataSource"
// is set as data source number 1 and our "baseDataSource" is set as data source number 2.
// Validate configuration before starting linking
config.validate(); // Will throw a RuntimeException if not correct
@SuppressWarnings("unused")
long startTime, stopTime;
startTime = System.currentTimeMillis();
try {
Processor proc = new Processor(config);
proc.addMatchListener(new DukeStitcherMatchListener(stitches));
proc.link();
proc.close();
} catch (IOException e) {
return null;
}
stopTime = System.currentTimeMillis();
return stitches;
}
开发者ID:HewlettPackard,项目名称:loom,代码行数:40,代码来源:DukeStitcher.java
示例5: TestFileListener
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
/**
* Creates a test file listener.
*/
public TestFileListener(String testfile, Configuration config,
boolean debug, Processor processor,
boolean showmatches, boolean pretty)
throws IOException {
InMemoryLinkDatabase testdb = new InMemoryLinkDatabase();
//testdb.setDoInference(true);
LinkDatabaseUtils.loadTestFile(testfile, testdb);
init(testdb, config, debug, processor, showmatches, pretty);
}
开发者ID:enricopal,项目名称:STEM,代码行数:13,代码来源:TestFileListener.java
示例6: init
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
private void init(LinkDatabase linkdb, Configuration config,
boolean debug, Processor processor,
boolean showmatches, boolean pretty) {
this.golddb = linkdb;
this.dukedb = new InMemoryLinkDatabase();
//((InMemoryLinkDatabase) this.dukedb).setDoInference(true);
this.idprops = config.getIdentityProperties();
this.props = config.getProperties();
this.debug = debug;
this.processor = processor;
this.database = processor.getDatabase();
this.linkage = !config.isDeduplicationMode();
this.showmatches = showmatches;
this.pretty = pretty;
}
开发者ID:enricopal,项目名称:STEM,代码行数:16,代码来源:TestFileListener.java
示例7: DukeController
import no.priv.garshol.duke.Processor; //导入依赖的package包/类
public DukeController(Properties props) {
this.status = "Initialized, inactive";
String configfile = get(props, "duke.configfile");
try {
// setting up logger
String loggerclass = get(props, "duke.logger-class", null);
if (loggerclass != null) {
logger = (Logger) ObjectUtils.instantiate(loggerclass);
logger.debug("DukeController starting up");
}
// loading configuration
Configuration config = ConfigLoader.load(configfile);
this.processor = new Processor(config, false);
this.linkdb = makeLinkDatabase(props);
processor.addMatchListener(new LinkDatabaseMatchListener(config, linkdb));
processor.addMatchListener(this);
batch_size = get(props, "duke.batch-size", 40000);
error_factor = get(props, "duke.error-wait-skips", 6);
// add loggers
if (logger != null) {
processor.setLogger(logger);
if (linkdb instanceof RDBMSLinkDatabase)
((RDBMSLinkDatabase) linkdb).setLogger(logger);
}
} catch (Throwable e) {
// this means init failed, and we need to clean up so that we can try
// again later. unfortunately, we don't know what failed, so we need
// to be careful
if (processor != null)
try {
processor.close();
} catch (Exception e2) {
if (logger != null)
logger.error("Couldn't close processor", e2);
}
if (linkdb != null)
linkdb.close();
throw new DukeException(e); // we failed, so signal that
}
}
开发者ID:enricopal,项目名称:STEM,代码行数:45,代码来源:DukeController.java
注:本文中的no.priv.garshol.duke.Processor类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论