本文整理汇总了Python中pyspark.sql.Row类的典型用法代码示例。如果您正苦于以下问题:Python Row类的具体用法?Python Row怎么用?Python Row使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
在下文中一共展示了Row类的4个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Python代码示例。
示例1: test_convert_row_to_dict
def test_convert_row_to_dict(self):
row = Row(l=[Row(a=1, b='s')], d={"key": Row(c=1.0, d="2")})
self.assertEqual(1, row.asDict()['l'][0].a)
df = self.sc.parallelize([row]).toDF()
df.registerTempTable("test")
row = self.sqlCtx.sql("select l, d from test").head()
self.assertEqual(1, row.asDict()["l"][0].a)
self.assertEqual(1.0, row.asDict()['d']['key'].c)
开发者ID:uncleGen,项目名称:ps-on-spark,代码行数:8,代码来源:tests.py
示例2: test_convert_row_to_dict
def test_convert_row_to_dict(self):
row = Row(l=[Row(a=1, b='s')], d={"key": Row(c=1.0, d="2")})
self.assertEqual(1, row.asDict()['l'][0].a)
df = self.sc.parallelize([row]).toDF()
with self.tempView("test"):
df.createOrReplaceTempView("test")
row = self.spark.sql("select l, d from test").head()
self.assertEqual(1, row.asDict()["l"][0].a)
self.assertEqual(1.0, row.asDict()['d']['key'].c)
开发者ID:JingchengDu,项目名称:spark,代码行数:10,代码来源:test_types.py
示例3: _create_row
def _create_row(fields, values):
row = Row(*values)
row.__fields__ = fields
return row
开发者ID:Bekbolatov,项目名称:spark,代码行数:4,代码来源:types.py
示例4: StructField
StructField("pix6",DoubleType(),True),
StructField("pix7",DoubleType(),True),
StructField("pix8",DoubleType(),True),
StructField("pix9",DoubleType(),True),
StructField("pix10",DoubleType(),True),
StructField("pix11",DoubleType(),True),
StructField("pix12",DoubleType(),True),
StructField("pix13",DoubleType(),True),
StructField("pix14",DoubleType(),True),
StructField("pix15",DoubleType(),True),
StructField("pix16",DoubleType(),True),
StructField("label",DoubleType(),True)
])
pen_raw = sc.textFile("first-edition/ch08/penbased.dat", 4).map(lambda x: x.split(", ")).map(lambda row: [float(x) for x in row])
dfpen = sqlContext.createDataFrame(pen_raw.map(Row.fromSeq(_)), penschema)
def parseRow(row):
d = {("pix"+str(i)): row[i-1] for i in range(1,17)}
d.update({"label": row[16]})
return d
dfpen = sqlContext.createDataFrame(pen_raw.map(parseRow), penschema)
va = VectorAssembler(outputCol="features", inputCols=dfpen.columns[0:-1])
penlpoints = va.transform(dfpen).select("features", "label")
pensets = penlpoints.randomSplit([0.8, 0.2])
pentrain = pensets[0].cache()
penvalid = pensets[1].cache()
penlr = LogisticRegression(regParam=0.01)
开发者ID:AkiraKane,项目名称:first-edition,代码行数:30,代码来源:ch08-listings.py
注:本文中的pyspark.sql.Row类示例由纯净天空整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论