amazon web services - Error with Data Pipeline backup when I transfer my data from DynamoDb to S3 -

-

i have backup dynamodb table s3 when launch service receive error after 3 attempts:

private.com.amazonaws.amazonserviceexception: user: arn:aws:sts::769870455028:assumed-role/datapipelinedefaultresourcerole/i-3678d99c not authorized perform: elasticmapreduce:modifyinstancegroups (service: amazonelasticmapreduce; status code: 400; error code: accessdeniedexception; request id: 9065ea77-0f95-11e5-8f35-39a70915a1ef) @ private.com.amazonaws.http.amazonhttpclient.handleerrorresponse(amazonhttpclient.java:1077) @ private.com.amazonaws.http.amazonhttpclient.executeonerequest(amazonhttpclient.java:725) @ private.com.amazonaws.http.amazonhttpclient.executehelper(amazonhttpclient.java:460) @ private.com.amazonaws.http.amazonhttpclient.execute(amazonhttpclient.java:295) @ private.com.amazonaws.services.elasticmapreduce.amazonelasticmapreduceclient.invoke(amazonelasticmapreduceclient.java:1391) @ private.com.amazonaws.services.elasticmapreduce.amazonelasticmapreduceclient.modifyinstancegroups(amazonelasticmapreduceclient.java:785) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:57) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:606) @ private.com.amazonaws.services.datapipeline.retrier.retryproxy.invokeinternal(retryproxy.java:36) @ private.com.amazonaws.services.datapipeline.retrier.retryproxy.invoke(retryproxy.java:48) @ com.sun.proxy.$proxy33.modifyinstancegroups(unknown source) @ amazonaws.datapipeline.cluster.emrutil.acquirecorenodes(emrutil.java:325) @ amazonaws.datapipeline.activity.abstractclusteractivity.resizeifrequired(abstractclusteractivity.java:47) @ amazonaws.datapipeline.activity.abstracthiveactivity.runactivity(abstracthiveactivity.java:113) @ amazonaws.datapipeline.objects.abstractactivity.run(abstractactivity.java:16) @ amazonaws.datapipeline.taskrunner.taskpoller.executeremoterunner(taskpoller.java:132) @ amazonaws.datapipeline.taskrunner.taskpoller.executetask(taskpoller.java:101) @ amazonaws.datapipeline.taskrunner.taskpoller$1.run(taskpoller.java:77) @ private.com.amazonaws.services.datapipeline.poller.pollworker.executework(pollworker.java:76) @ private.com.amazonaws.services.datapipeline.poller.pollworker.run(pollworker.java:53) @ java.lang.thread.run(thread.java:745)

how can backup? have error?

edit: new policy

{ "version": "2012-10-17", "statement": [ { "effect": "allow", "action": [ "s3:", "dynamodb:", "ec2:describe*", "elasticmapreduce:describe*", "elasticmapreduce:listinstance*", "elasticmapreduce:addjobflowsteps", "elasticmapreduce:", "rds:describe", "datapipeline:", "cloudwatch:", "redshift:describeclusters", "redshift:describeclustersecuritygroups", "sdb:", "sns:", "sqs:" ], "resource": [ "" ] } ]

this new exception :

error during job, obtaining debugging information... examining task id: task_1434014832347_0001_m_000008 (and more) job job_1434014832347_0001 examining task id: task_1434014832347_0001_m_000013 (and more) job job_1434014832347_0001 examining task id: task_1434014832347_0001_m_000005 (and more) job job_1434014832347_0001 examining task id: task_1434014832347_0001_m_000034 (and more) job job_1434014832347_0001 examining task id: task_1434014832347_0001_m_000044 (and more) job job_1434014832347_0001 examining task id: task_1434014832347_0001_m_000004 (and more) job job_1434014832347_0001 task failures(4): ----- task id: task_1434014832347_0001_m_000002 url: http://ip-10-37-138-149.eu-west-1.compute.internal:9026/taskdetails.jsp?jobid=job_1434014832347_0001&tipid=task_1434014832347_0001_m_000002 ----- diagnostic messages task: error: java heap space failed: execution error, return code 2 org.apache.hadoop.hive.ql.exec.mr.mapredtask mapreduce jobs

datapipeline agent (taskrunner) running on emr cluster trying resize emr cluster , failing. resource role passed emr cluster not have permissions invoke following api amazonelasticmapreduce::modifyinstancegroups.

i looked @ defaultresourcerolepolicy, created using wizard in console, (http://docs.aws.amazon.com/datapipeline/latest/developerguide/dp-iam-roles.html ) these allowed policies emr: "elasticmapreduce:describe*", "elasticmapreduce:listinstance*", "elasticmapreduce:addjobflowsteps"

and found not allow modifyinstancegroups.
please update resource role policy allow that. e.g.,"elasticmapreduce:*"

thx reporting bug. in meanwhile, work on fixing console wizard generated default resource role policy.

aravind r.


Comments

Post a Comment

Popular posts from this blog

javascript - gulp-nodemon - nodejs restart after file change - Error: listen EADDRINUSE events.js:85 -

-
i'm running nodejs 0.12.3 - gulp 3.10.8 , gulp-nodemon 2.0.3. gulpfile.js : gulp.task('serve', ['build'], function() { gulp.start('fb-flo'); nodemon({ script: paths.server, env: { 'node_env': 'development' }, // env: { 'node_env': 'production' }, execmap: { 'js': 'node_modules/babel/bin/babel-node --stage 1' //es7 cote server }, delay: '0ms', watch: ['src/shared/', 'src/server/'], ignore: ['src/shared/components'], tasks: function (changedfiles) { var tasks = []; changedfiles.foreach(function (file) { if (path.extname(file) === '.jsx' && !~tasks.indexof('bundlejs')) tasks.push('bundlejs'); if (path.extname(file) === '.less' && !~tasks.indexof('compileless')) tasks.push('compileless'); }); return tasks; } ...
Read more

Fatal Python error: Py_Initialize: unable to load the file system codec. ImportError: No module named 'encodings' -

-
i trying make simple python program opening list of webpages user manually download reports site. don't have previous experience preparing exe files.. , i'm in learning process python coding. of done on windows 7 x64 this python code: #!c:/python34/python.exe -u splinter import * import time import os import csv #---------------------------------- raporty = [] open('../raporty.csv', newline='') csvfile: contents = csv.reader(csvfile, delimiter=' ', quotechar='|') row in contents: r = ', '.join(row) r = r.replace(',','') raporty.append(r) #--not implemented yet zmienne = [] open('../zmienne.csv', newline='') csvfile: contents = csv.reader(csvfile, delimiter=' ', quotechar='|') row in contents: r = ', '.join(row) r = r.replace(',','') zmienne.append(r) print("start") browser = browser(...
Read more

javascript - oscilloscope of speaker input stops rendering after a few seconds -

-
the following script reads audio user's microphone , renders oscilloscope on html canvas. the source taken example of mozilla developer network: visualizations web audio api and here fiddle: http://jsfiddle.net/b7j8pktp/ mozgetusermedia (note: code has no fork mechanism different browsers: works firefox) it works fine few seconds , stops rendering. whereas works totally stable: http://mdn.github.io/voice-change-o-matic/ the problem can reduced following code. microphone activation icon (next the address bar in firefox) disappears after 5 seconds: navigator.mozgetusermedia({audio: true}, function() {}, function() {} ); ( http://jsfiddle.net/b7j8pktp/2/ ) this known bug in firefox. take stream getusermedia call , hook window so: navigator.mozgetusermedia({audio: true}, function(stream) { window.stream = stream; // rest of code }, function err() { // handle error }); hopefully can fixed soon. problem we're f...
Read more