org.kitesdk.data.Datasets.load()方法的使用及代码示例

本文整理了Java中org.kitesdk.data.Datasets.load()方法的一些代码示例，展示了Datasets.load()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Datasets.load()方法的具体详情如下：
包路径：org.kitesdk.data.Datasets
类名称：Datasets
方法名：load

Datasets.load介绍

[英]Load a Dataset or View for the given URI.

URIs must begin with dataset: or view:. The remainder of the URI is implementation specific, depending on the dataset scheme.

If you use a dataset URI, load returns the unfiltered dataset. If you use a view URI, load returns a View configured to read a subset of the dataset.
[中]加载给定URI的数据集或视图。
URI必须以数据集或视图开头。URI的其余部分是特定于实现的，具体取决于数据集方案。
如果使用数据集URI，load将返回未筛选的数据集。如果使用视图URI，load将返回配置为读取数据集子集的视图。

代码示例

代码示例来源：origin: apache/flume

@Override
public Dataset run() {
return Datasets.load(datasetUri);
}
});

代码示例来源：origin: apache/nifi

private View load(ProcessContext context, FlowFile file) {
String uri = context.getProperty(KITE_DATASET_URI)
.evaluateAttributeExpressions(file)
.getValue();
return Datasets.load(uri, Record.class);
}
}

代码示例来源：origin: apache/nifi

return Datasets.load(uri).getDataset().getDescriptor().getSchema();
} else if ("resource".equals(uri.getScheme())) {
try (InputStream in = Resources.getResource(uri.getSchemeSpecificPart()).openStream()) {

代码示例来源：origin: apache/flume

private SavePolicy(Context context) {
String uri = context.getString(CONFIG_KITE_ERROR_DATASET_URI);
Preconditions.checkArgument(uri != null, "Must set "
+ CONFIG_KITE_ERROR_DATASET_URI + " when " + CONFIG_FAILURE_POLICY
+ "=save");
if (Datasets.exists(uri)) {
dataset = Datasets.load(uri, AvroFlumeEvent.class);
} else {
DatasetDescriptor descriptor = new DatasetDescriptor.Builder()
.schema(AvroFlumeEvent.class)
.build();
dataset = Datasets.create(uri, descriptor, AvroFlumeEvent.class);
}
nEventsHandled = 0;
}

代码示例来源：origin: kite-sdk/kite

@Override
public Dataset run() {
return Datasets.load(uri);
}
});

代码示例来源：origin: org.apache.flume.flume-ng-sinks/flume-dataset-sink

@Override
public Dataset run() {
return Datasets.load(datasetUri);
}
});

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets.>
load("dataset:unknown:/tmp/data/test", Record.class);
}
});

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets
.>load("dataset:hbase:" + zk + "/nosuchdataset", Object.class);
}
}

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets.>
load("dataset:hdfs://" + hdfsAuth + "/tmp/data/ns/nosuchdataset",
Object.class);
}
});

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets.>
load("dataset:hdfs://" + hdfsAuth + "/test", Object.class);
}
});

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets.>
load("dataset:hdfs://" + hdfsAuth + "/tmp/data/nosuchnamespace/test",
Object.class);
}
});

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets.>
load("dataset:file:/tmp/data/ns/nosuchdataset", Record.class);
}
});

代码示例来源：origin: kite-sdk/kite

@Override
public void run() {
Dataset ds = Datasets.>
load("dataset:file:/test", Record.class);
}
});

代码示例来源：origin: kite-sdk/kite

@SuppressWarnings("deprecation")
private static View load(JobContext jobContext) {
Configuration cOnf= Hadoop.JobContext.getConfiguration.invoke(jobContext);
Class type = getType(jobContext);
String outputUri = conf.get(KITE_OUTPUT_URI);
return Datasets.>load(outputUri, type);
}

代码示例来源：origin: org.kitesdk/kite-data-mapreduce

@SuppressWarnings("deprecation")
private static View load(JobContext jobContext) {
Configuration cOnf= Hadoop.JobContext.getConfiguration.invoke(jobContext);
Class type = getType(jobContext);
String outputUri = conf.get(KITE_OUTPUT_URI);
return Datasets.>load(outputUri, type);
}

代码示例来源：origin: kite-sdk/kite

@Test(expected=DatasetIOException.class)
public void testCannotFindHDFS() throws Exception {
// do not set the default configuration that the loader will use
Datasets.load("dataset:hdfs:/tmp/datasets/ns/strings");
}
}

代码示例来源：origin: org.apache.nifi/nifi-kite-processors

private View load(ProcessContext context, FlowFile file) {
String uri = context.getProperty(KITE_DATASET_URI)
.evaluateAttributeExpressions(file)
.getValue();
return Datasets.load(uri, Record.class);
}
}

代码示例来源：origin: kite-sdk/kite

@Test
public void testLoadWithoutType() {
Dataset expected = mock(Dataset.class);
when(repo.load("ns", "test", GenericRecord.class)).thenReturn(expected);
URI datasetUri = new URIBuilder(repoUri, "ns", "test").build();
Dataset ds = Datasets.load(datasetUri);
verify(repo).load("ns", "test", GenericRecord.class);
verifyNoMoreInteractions(repo);
verifyNoMoreInteractions(expected);
Assert.assertEquals(expected, ds);
}

代码示例来源：origin: kite-sdk/kite

@Test
public void testLoadStringUri() {
Dataset expected = mock(Dataset.class);
when(repo.load("ns", "test", Object.class)).thenReturn(expected);
URI datasetUri = new URIBuilder(repoUri, "ns", "test").build();
Dataset ds = Datasets.>
load(datasetUri, Object.class);
verify(repo).load("ns", "test", Object.class);
verifyNoMoreInteractions(repo);
verifyNoMoreInteractions(expected);
Assert.assertEquals(expected, ds);
}

代码示例来源：origin: kite-sdk/kite

@Test
public void testMissingHDFSAuthority() {
try {
Datasets.load("dataset:hdfs:/tmp/data/ns/test", Object.class);
Assert.fail("Shouldn't be able to connect to HDFS");
} catch (DatasetIOException e) {
Assert.assertTrue("Should have helpful error message",
e.getMessage().contains("make sure the default hdfs URI is configured"));
}
}