DBREF lib alternative solution?

[tmushayahama]

There is a lib being used called @geneontology/dbxrefs. it seemed like a hack lib as it is relying on GitHub as a api. Sometimes it doesnt work, maybe too many calls, any better solutions

For the curious : Sample code

see the URL and the line of the error is dbxrefs.filter if GitHub says no dbxref, it throws an error that filter of undefined

// can not use the GO URL at the moment since not https and would block the integration by 3rd parties..
// let URL = "http://current.geneontology.org/metadata/db-xrefs.yaml";
let URL = "https://raw.githubusercontent.com/geneontology/go-site/master/metadata/db-xrefs.yaml";

let dbxrefs;

let dbReady = false;
let dbError = false;

function getURL(database, entityType, id) {
    let db = dbxrefs.filter(elt => (elt.database.toLowerCase() == database.toLowerCase()) || (elt.synonyms && elt.synonyms.includes(database)));
    if (!db || db.length == 0) { console.warn("@geneontology/dbxrefs: database ", database, " not found for requested id ", id); return undefined; }
    ...

cc @kltm @pkalita-lbl and maybe @sierra-moxon might no if there is an api for this

[kltm]

@tmushayahama Yeah, honestly, GitHub never works well as a fileserver. I would try
https://current.geneontology.org/metadata/db-xrefs.yaml
instead--it's being delivered on the AWS CloudFront CDN.

[pkalita-lbl]

I can't even find a repository where the @geneontology/dbxrefs source code lives. npm view says that it was last published by Laurent-Philippe on March 1, 2021. I wonder if he never committed the code to a repository? Or at least not one we have access to?

[tmushayahama]

Any solution to this, it dies occasionally. Maybe since I am developing and refreshing too often, my IP gets some soft ban or reach rate limit. (I just wait for like a minute) Should we just temporarily create a file called dbrefs.json inhouse constants and copy its current GitHub contents. Or maybe just make the dbxref library function inhouse and provide the error checking when fetching to GitHub, it is only a tiny function. Or if there is a proper api available

cc @kltm @pkalita-lbl

[kltm]

Does this not fit the criteria? http://current.geneontology.org/metadata/db-xrefs.json
The CDN will not soft-block you.

[tmushayahama]

Something to think about

Slow Initial Load Due to DB-Xrefs Dependencies

Current Behavior

The component's initialization is currently blocked waiting for http://current.geneontology.org/metadata/db-xrefs.json to load. This is because getTermUrl is used throughout the code to resolve only links during the initial load. This dependency makes the component's performance directly tied to the db-xrefs API response time.

Code Example

async initDBXrefs() {
    try {
        await this.dbxrefService.init();
        while (!this.dbxrefService.isReady()) {
            await new Promise(resolve => setTimeout(resolve, 100));
        }
        if (this.dbxrefService.hasError()) {
            throw new Error('Failed to initialize dbxrefs');
        }
        this.dbXrefsReady = true;
    } catch (error) {
        console.error('Failed to initialize dbxrefs:', error);
        this.dbXrefsReady = false;
    }
}

async componentWillLoad() {
    await this.initDBXrefs();
    this.initCytoscape();
}

Potential Solutions

1. inhouse xrefs since the links doesn't change much, right (tie the xrefs with widget release)
2. Lazy load the URLs by calling getTermUrl only when needed, but how
3. Update the render after receiving the db-xrefs data (would require double render)

Note: Just to note, if the api fails, it will load the widget without clickable links

cc @pkalita-lbl @kltm

[kltm]

@tmushayahama Out of curiosity, what is the load time for https://current.geneontology.org/metadata/db-xrefs.json versus the rest of the app? Could it just be loaded initially in parallel? I'm a little surprised that it's an issue, as it's a file out of AWS's CDN and should hopefully not be too laggy...

[vanaukenk]

@tmushayahama @kltm @pkalita-lbl

Should we keep this ticket open?

If so, please add assignees, labels, and let me know what project it should be in.

Thanks.