Skip to content
This repository was archived by the owner on Sep 16, 2022. It is now read-only.
This repository was archived by the owner on Sep 16, 2022. It is now read-only.

Google cloud Storage 308 Error while writing file #60

@chaitu1124

Description

@chaitu1124

Hi All,
I tried to pull the report from Double Click Search API and put it into Google Cloud Storage by using google app engine(Python).

The following code is used to pull the report and place in google cloud storage.

If I hard code the report_run_id in ds_check_status_report it is writing the file properly but when I try to pull the report_run_id from bigquery and pass then it is throwing 308 error.

class ds_check_status_report(webapp.RequestHandler):

    _apptitle      = None
    _projectid     = None
    _projectnumber = None
   
    def get(self):
        cfg=appsettings()
##      report_run_id="kdajdsjflsjfl"  
        result=dsbqfuns._dsbqquery()
        result1=json.dumps(result['stacktrace'])
        res=json.loads(result1)
        self.response.write(res['rows'][0]['f'][0]['v'])
        dsutils.poll_report(res['rows'][0]['f'][0]['v'])

        


def poll_report(report_id):
  """Poll the API with the reportId until the report is ready, up to ten times.

  Args:
    service: An authorized Doublelcicksearch service.
    report_id: The ID DS has assigned to a report.
  """
  print "Enter into poll_report"
  cfg=appsettings()
  creds = create_credentials(cfg._client_id , cfg._client_secret, cfg._refresh_token)
  service = get_service(creds)
  try:
    request = service.reports().get(reportId=report_id)
    json_data = request.execute()
    if json_data['isReportReady']:
      print('The report is ready.')

      # For large reports, DS automatically fragments the report into multiple
      # files. The 'files' property in the JSON object that DS returns contains
      # the list of URLs for file fragment. To download a report, DS needs to
      # know the report ID and the index of a file fragment.
      for i in range(len(json_data['files'])):
        print('Downloading fragment ' + str(i) + ' for report ' + report_id)
        download_files(service, report_id, str(i)) # See Download the report.
        
    else:
      print('Report is not ready. I will try again.')

  except:
    exc_type, exc_value, exc_traceback = sys.exc_info()
    print exc_type, exc_value, exc_traceback 


def download_files(service, report_id, report_fragment):
  """Generate and print sample report.

  Args:
    service: An authorized Doublelcicksearch service.
    report_id: The ID DS has assigned to a report.
    report_fragment: The 0-based index of the file fragment from the files array.
  """
  print "Enter into download_files", report_id
  filename="/awstogcs/DoubleClickSearch_Campaign"+report_id+"_"+report_fragment+".csv"
  write_retry_params = _gcs.RetryParams(backoff_factor=1.1)
  gcs_file=_gcs.open(filename, 'w', content_type='text/plain',retry_params=write_retry_params)
  request = service.reports().getFile(reportId=report_id, reportFragment=report_fragment)
  gcs_file.write(request.execute())
  gcs_file.close()

Error Message:

INFO     2017-06-26 13:23:37,417 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO     2017-06-26 13:23:37,710 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO     2017-06-26 13:23:37,976 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO     2017-06-26 13:23:38,157 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO     2017-06-26 13:23:38,440 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_Campa

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions