scala - How to do distributed Principal Components Analysis + Kmeans using Apache Spark? -

-

i need run principal components analysis , k-means clustering on large-ish dataset (around 10 gb) spread out on many files. want use apache spark since it's known fast , distributed.

i know spark supports pca , pca + kmeans.

however, haven't found example demonstrates how many files in distributed manner.


Comments

Post a Comment

Popular posts from this blog

xslt - Substring before throwing error -

-
i've below xml <?xml version="1.0" encoding="utf-8"?> <body> <p>industrial drawing: creative composition</p> <p>industrial drawing: creative<fn> <fnn>4</fnn> <fnt> <p>ftn1"</p> </fnt> </fn> composition </p> </body> and below xsl. <xsl:template match="p"> <xsl:choose> <xsl:when test="contains(substring-before(./text(),' '),'article')"> <xsl:text>sect3</xsl:text> <xsl:value-of select="./text()"/> </xsl:when> <xsl:when test="contains(substring-before(./b/text(),' '),'section')"> <xsl:text> sect 2</xsl:text> <xsl:value-of select="./text()"/> </xsl:when> <xsl:w...
Read more

Google Cloud Bigtable Durability/Availability Guarantees -

-
i google provide guidelines on durability , availability guarantees provided cloud bigtable service. here understanding far: the fact minimum cluster requires 3 nodes suggests that, @ least within zone, data highly durable , replicated 3 nodes. however, this answer googler states "cloud bigtable doesn’t replicate data" — directly contradicting quote on cloud bigtable homepage claims "is built replicated storage strategy". it? replicated or not? , if so, how many copies kept? the fact clusters can set within particular zone suggests availability of cluster tied directly availability of zone. if want have highly available bigtable-based data storage, best practice set independent clusters across multiple zones , handle synchronisation of writes across clusters myself? there no information on whether bigtable clusters across zones independent or not. if set clusters across multiple zones, , 1 zone goes down, expect clusters in other zones carry on work...
Read more

Android M doze mode and "native" posix socket freezing up -

-
do not know if considering android m floating around since not much. experiencing strange behaviour app. using native posix tcp client sockets make connections. these sockets working until "doze" or "app standby" modes not activate...cause once either phone reset can not allow app work once again. solution clear app data or reinstall app on phone. still have not investigated errno of subsequent connections, after doze mode, sure sound strange complete app reinstall necessary in order fix posix tcp connections work once again. tomorrow able post few more details regarding issue. has encountered similar problem on android m after doze had prevented network access? thank you!
Read more