Skip to content

Warrior - issue & Dunfell takes longer time to generate SPDX #29

Closed
@chaithubk

Description

@chaithubk

I am planning to use meta-spdxscanner layer to generate the SPDX docs in my poky warrior distro. I have run the fossology/fossology:3.5.0 docker image and have installed fossdriver. I am noticing this error with do_spdx()


ERROR: linux-libc-headers-5.0-r0 do_spdx: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_spdx(d)
     0003:
File: '/home/yocto/output/poky_warrior/poky/meta-spdxscanner/classes/fossdriver-host.bbclass', lineno: 137, function: do_spdx
     0133:    bb.note(' run fossdriver ...... ')
     0134:    if not os.path.isfile(tar_name):
     0135:        bb.warn(info['pn'] + "has no source, do nothing")
     0136:        return
 *** 0137:    invoke_fossdriver(tar_name,sstatefile)
     0138:    if get_cached_spdx(sstatefile) != None:
     0139:        write_cached_spdx( info,sstatefile,cur_ver_code )
     0140:        ## CREATE MANIFEST(write to outfile )
     0141:        create_manifest(info,sstatefile)
File: '/home/yocto/output/poky_warrior/poky/meta-spdxscanner/classes/fossdriver-host.bbclass', lineno: 237, function: invoke_fossdriver
     0233:    if (Reuse(server, tar_file, "Software Repository", tar_file, "Software Repository").run()  != True):
     0234:        bb.note("This OSS has not been scanned. So upload it to fossology server.")
     0235:        i = 0
     0236:        while i < 5:
 *** 0237:            if (Upload(server, tar_file, "Software Repository").run() != True):
     0238:                bb.warn("%s Upload failed, try again!" %  tar_file)
     0239:                time.sleep(delaytime)
     0240:                i += 1
     0241:            else:
File: '/home/fossdriver/fossdriver/tasks.py', lineno: 73, function: run
     0069:        logging.info("Upload complete, {} upload ID number is {}".format(self.filePath, newUploadNum))
     0070:
     0071:        # and wait until upload finishes unpacking
     0072:        logging.info("Waiting for upload {} to unpack".format(newUploadNum))
 *** 0073:        self.server.WaitUntilAgentIsDone(newUploadNum, "ununpack", pollSeconds=5)
     0074:        self.server.WaitUntilAgentIsDone(newUploadNum, "adj2nest", pollSeconds=5)
     0075:
     0076:        return True
     0077:
File: '/home/fossdriver/fossdriver/server.py', lineno: 457, function: WaitUntilAgentIsDone
     0453:            - agent: name of agent to check for.
     0454:            - pollSeconds: number of seconds to wait between polling. Defaults to 10.
     0455:        """
     0456:        # FIXME consider adding a max # of tries before returning
 *** 0457:        jobNum = self._getMostRecentAgentJobNum(uploadNum, agent)
     0458:        while not self._isJobDoneYet(jobNum):
     0459:            time.sleep(pollSeconds)
File: '/home/fossdriver/fossdriver/server.py', lineno: 297, function: _getMostRecentAgentJobNum
     0293:            - agent: name of agent to check for.
     0294:        Returns job ID number or -1 if not found.
     0295:        """
     0296:        # FIXME given _getJobsForUpload, currently retrieves just first page
 *** 0297:        jobs = self._getJobsForUpload(uploadNum)
     0298:        if jobs is None or jobs == []:
     0299:            return -1
     0300:        # will be returned in reverse chrono order, so we can just loop through
     0301:        # and stop on the first one we come to
File: '/home/fossdriver/fossdriver/server.py', lineno: 280, function: _getJobsForUpload
     0276:        # response format changed from XML to JSON on or around 3.5.0
     0277:        # see https://github.com/fossology/fossdriver/issues/17
     0278:        if self.IsAtLeastVersion("3.5.0"):
     0279:            # parse json
 *** 0280:            jobData = fossdriver.parser.parseJSONShowJobsData(results.content)
     0281:            return jobData
     0282:        else:
     0283:            # decode and parse XML
     0284:            decodedContent = fossdriver.parser.decodeAjaxShowJobsData(results.content)
File: '/home/fossdriver/fossdriver/parser.py', lineno: 209, function: parseJSONShowJobsData
     0205:    jobData = []
     0206:    js = json.loads(content.decode('utf-8'))
     0207:    showJobsData = js.get("showJobsData", [])
     0208:    for sjd in showJobsData:
 *** 0209:        jd = sjd.get("job", None)
     0210:        if jd is not None:
     0211:            jq = jd.get("jobQueue", None)
     0212:            if jq is not None:
     0213:                for k, v in jq.items():
Exception: AttributeError: 'str' object has no attribute 'get'

ERROR: linux-libc-headers-5.0-r0 do_spdx: 'str' object has no attribute 'get'
ERROR: linux-libc-headers-5.0-r0 do_spdx: Function failed: do_spdx
ERROR: Logfile of failure stored in: /home/yocto/output/poky_warrior/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/linux-libc-headers/5.0-r0/temp/log.do_spdx.27653
ERROR: Task (/home/yocto/output/poky_warrior/poky/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.0.bb:do_spdx) failed with exit code '1'

Can you please help me to solve this. ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions