scala - How to do distributed Principal Components Analysis + Kmeans using Apache Spark? -

-

i need run principal components analysis , k-means clustering on large-ish dataset (around 10 gb) spread out on many files. want use apache spark since it's known fast , distributed.

i know spark supports pca , pca + kmeans.

however, haven't found example demonstrates how many files in distributed manner.


Comments

Post a Comment

Popular posts from this blog

xslt - Substring before throwing error -

-
i've below xml <?xml version="1.0" encoding="utf-8"?> <body> <p>industrial drawing: creative composition</p> <p>industrial drawing: creative<fn> <fnn>4</fnn> <fnt> <p>ftn1"</p> </fnt> </fn> composition </p> </body> and below xsl. <xsl:template match="p"> <xsl:choose> <xsl:when test="contains(substring-before(./text(),' '),'article')"> <xsl:text>sect3</xsl:text> <xsl:value-of select="./text()"/> </xsl:when> <xsl:when test="contains(substring-before(./b/text(),' '),'section')"> <xsl:text> sect 2</xsl:text> <xsl:value-of select="./text()"/> </xsl:when> <xsl:w...
Read more

javascript - gulp-nodemon - nodejs restart after file change - Error: listen EADDRINUSE events.js:85 -

-
i'm running nodejs 0.12.3 - gulp 3.10.8 , gulp-nodemon 2.0.3. gulpfile.js : gulp.task('serve', ['build'], function() { gulp.start('fb-flo'); nodemon({ script: paths.server, env: { 'node_env': 'development' }, // env: { 'node_env': 'production' }, execmap: { 'js': 'node_modules/babel/bin/babel-node --stage 1' //es7 cote server }, delay: '0ms', watch: ['src/shared/', 'src/server/'], ignore: ['src/shared/components'], tasks: function (changedfiles) { var tasks = []; changedfiles.foreach(function (file) { if (path.extname(file) === '.jsx' && !~tasks.indexof('bundlejs')) tasks.push('bundlejs'); if (path.extname(file) === '.less' && !~tasks.indexof('compileless')) tasks.push('compileless'); }); return tasks; } ...
Read more

oracle - Changing start date for system jobs related to automatic statistics collections in 11g -

-
i have oracle database 11g enterprise edition release 11.2.0.1.0 - 64bit production running on windows 2008 r2 somehow got next run dates scheduled jobs set dates in year 2016. jobs created able reset start_date in turn caused next_run_date reset. jobs related oracle automatic statistics collections unable dms_scheduler.set_attribute work changing start_date values. i tried resetting weekly window's dates per oracle document id 1450173.1 using sql following , did in fact reset values see within dba_scheduler_windows had no change dba_scheduler_jobs begin dbms_scheduler.set_attribute ( name => '"sys"."monday_window"', attribute => 'start_date', value => '10-jun-15 10.00.00.000000000 pm us/central'); end; / the job names seeing issue on are: rse$clean_recoverable_script ora$autotask_clean purge_log sm$clean_auto_split_merge bsln_maintain_stats_job mgmt_stats_config_job some of lack schedule names if try dbms_schedul...
Read more