Create designs for revised workflow of .xls import

[BibhaT]

Initial issue: https://github.com/orgs/younginnovations/projects/3?pane=issue&itemId=61369915

New workflow: IATI Publisher Import Workflow.pdf

Explanatory video for the same: https://youtu.be/fbrz8th02so

Things to consider: (as commented by Emma in #1439
For the download step - for info - I had already started thinking about how we simplify exporting data as a future improvement here: #1526 (there are a lot of steps currently)

I'm happy to progress with this design, and happy to help with instruction text for the interface if you need. The main thing I think is important to make clear to users is when an import is about to overwrite existing data in IATI Publisher, which can be factored into the in-tool instructions at the "change data and upload" steps in the workflow.

[asminashrestha]

Hi, @emmajclegg

We have created three variations of the revised workflow for the bulk import and within-activity import, each divided into four sections:

1. A section for users to choose the element to be imported
2. A section to select if the importing element exists or not
3. A section for the actual import process
4. A section to inform users about the download option available in our system

Variation 1: This variation features separate pages for each functionality, offering clear separation between steps.

Bulk Import: View Here

Within Activity: View Here

Variation 2: This variation focuses on distinct visual grouping of sections within the workflow, while maintaining a unified flow.

Bulk Import: View Here

Within Activity: View Here

Variation 3: This variation keeps all the functionalities on a single page, allowing users to complete the workflow without navigating through multiple pages.

Bulk Import: View Here

Within Activity: View Here

These screens are still a work in progress, and we would greatly appreciate your feedback.
Please take a look at these options and share your thoughts.

[emmajclegg]

Hi @asminashrestha - thanks for these, it's definitely useful to see options.

Variations 1 and 2 look more intuitive than 3 (where I think it's too complicated having all the steps on the same page)

My favourite is variation 1, as I think it's logical to read the steps left-to-right. That said, I think it makes sense to swap the "Upload your file" block with the "Templates" block, as the workflow would be: Choose an action >> Download template or existing data >> Upload your file , if I'm interpreting it correctly?

Under "Choose an Action", we'll need to make it clear that users can update existing data and add new data at the same time by selecting the "Update existing..." option, but we can include that in the subtext.

I'm highlighting what I'm referring to here, in case it's not clear:

Any questions - or different views - let me know.

[asminashrestha]

@emmajclegg Thank you for your feedback! I completely see where you're coming from regarding the left-to-right flow. I’d like to clarify that our intention wasn’t to present these elements as sequential steps but rather as individual instances, each serving a distinct role in the workflow.
Additionally, in some cases, users might not always download and then upload—these actions can vary depending on their needs at the time.
With that in mind, we thought placing the main action at the center would create a more flexible and user-friendly experience.
However, what you’re trying to say makes complete sense if we look into these as step wise series of actions. We’ll look into your suggestions and work accordingly.

[emmajclegg]

Ok, thanks for the thoughts @asminashrestha.

A potential problem that comes to mind is users trying to upload a random data file (i.e. one not in the IATI Publisher template format), which I expect some people will do regardless of the guidance we provide. We sometimes see this in support queries. So I don't think it's a bad thing to make the template more obvious in a left-to-right flow.

I agree the upload feature is the main action and you've made this obvious from its size on screen.

[asminashrestha]

@emmajclegg Thanks for your feedback. As you suggested we have decided to move forward with the left-to-right approach for the design. While doing so, we also realized that the previous design didn’t make it clear that the section with download and upload options changes when the user clicks 'Add New' or 'Update Existing'. To resolve this, we’ve created an updated design that not only incorporates the left-to-right flow but also provides a more logical and intuitive user experience.
Additionally, we have created a screen to handle the case where an activity has no results.

Here are the links:
Bulk Import: View Link
Within Activity: View Link
No Result Case: View Link

[emmajclegg]

Hi @asminashrestha - ok thanks, I think this looks fine.

[praweshsth]

@emmajclegg please check the links above to see the full workflow design

[emmajclegg]

@praweshsth @asminashrestha - thanks.

I've added comments in Figma for the "Bulk import" and "download flow".

Mainly,

• can we keep in mind the work we did on this issue (Publishing steps and progress are unclear to users through the Background Tasks box #1423) when it comes to the background task box (users are much more likely to miss information in the bottom-right of the screen and the more we can move messages centre-screen the better. Unnecessary user clicks should also be removed)
• I've experienced file imports taking a long time - any more than a few seconds requires a visible progress bar so that the user knows that processing is ongoing and the screen hasn't frozen

Also - I'd like to review all on screen text so let me know when the best point is to do this (and whether collaborating directly on the GitHub file is easier than comments in Figma?). We'll be reviewing English text across the tool ahead of translation, though this may still be a while off with the dependencies on automated string extraction and reintegration

[asminashrestha]

Hi @emmajclegg We have updated some of the designs in the flow as you have suggested. View link
Also, We think communicating through figma would be easier for the texts.

[emmajclegg]

Hi @asminashrestha - I've added comments to the "bulk import" Figma, largely around text. I think questions and high-level comments are great on there, but I do think more mundane text review (typos, etc.) would be easiest for me to do by committing changes directly to the GitHub code, for your team to review and merge into the development branch. (I mentioned this on the call with Prawesh and Bibha this morning - it would save both of us time and it's how we would do this kind of task within ODS). Do let me know if that's an option for future?

The same comments will apply to the "within activity" and "no results case" flows - i.e.

• can we make sure page titles are consistent
• can we make sure "basic activity data", "transactions" and "results" are written consistently throughout (these are the most intuitive way to refer to these three elements of the Standard)
• if sub-text is not providing new information, can we remove it

Any questions, let me know

[emmajclegg]

@asminashrestha - following my Figma comment, can I check if this page will still be at the end of the data import flow?

I think this page works well for displaying errors, and it then requires user action to select activities and do the final click to import.

Given that, I don't understand why we need a "click to continue" on this page:

The user has selected the xlsx data file they want to import, they should be shown on screen that the file is being processed, then - when processing is complete - they should automatically move to the "Add / Update All Activity" page above, surely?

[asminashrestha]

@emmajclegg Yes, the Add/Update activity page will be at the end of the flow.

Since this is a background process which means that the fetching process will take a long time, therefore, we're trying to give users the freedom to carry on with their other tasks while this is running in the background. So, when this process finishes, we first need the user to know about the process being finished rather than abruptly taking them to the other section that is the list with activities being imported. Hence, we need a 'click to continue' button here so that we can show users a summary of how many errors are present in the imported activity and help them make a conscious decision about moving forward with the process. After clicking the "Click to continue" users can then move to the "Add/Update activity" page where users can see the full list of errors.

[emmajclegg]

Ok, @asminashrestha - for the sake of progressing this, let's keep the ongoing tasks box where it was (bottom-right) and make sure we have a clear "Click to continue" when user action is required. It's hard for me to comment further on messaging at the moment, as the current design draft is talking about publishing data rather than importing.

Since this is a background process which means that the fetching process will take a long time

Similar to the discussion on bulk publishing (#1480), we do need to set limits and keep the user informed in these cases:
a) how big of a data import file is too big for IATI Publisher to successfully import? (and how do we inform the user of this before they attempt the action)
b) can we be more specific about what a "long time" is in user messaging? e.g. "this may take a few minutes", or "this could take up to x minutes for files larger than y"...

[PG-Momik]

@emmajclegg

a) how big of a data import file is too big for IATI Publisher to successfully import? (and how do we inform the user of this before they attempt the action)

• We are currently unsure of the maximum file size that IATI Publisher can handle for imports, as we haven't conducted stress tests to determine the capacity for importing large XLSX files.

b) can we be more specific about what a "long time" is in user messaging? e.g. "this may take a few minutes", or "this could take up to x minutes for files larger than y"...

• In this context, a "long time" refers to any task that takes longer than usual, like when you quickly save a form. For actions that may take a bit longer, we run them in the background so users can continue using the application without being stuck waiting for the task to finish.
• When it comes to user messaging, we can’t specify an exact duration like "this could take up to x minutes for files larger than y" because import times vary based on the payload. Instead, we could use the more general "this may take a few minutes"

---

We should address the import and publish size limits in a separate issue. We’ve conducted stress testing for publishing, we haven’t yet done the same for importing files. Focusing on that now would divert us from our current priorities, which are: 1) Successfully publish 100 files and 2) Ensuring an easy and smooth import process. But if it’s absolutely crucial, we’ll prioritize it and find a max limit for imports.

---

cc: @BibhaT @asminashrestha

[asminashrestha]

@emmajclegg We have now moved the ongoing tasks box to the bottom right, following your suggestion.

[emmajclegg]

@PG-Momik - yes, as discussed, let's not spend a lot of energy investigating the limits of bulk importing at this point. My last comments were related to the workflow and whether we really expect the user to be navigating away from the import screen because of the processing time. Either way, if it becomes an obvious issue during testing, we can revisit it then.

@asminashrestha - do you have all the feedback you need now on the import workflow design? I've no further comments

[asminashrestha]

Yes, Thanks @emmajclegg