Add progress indicator for software file uploads

[lukeheath]

Goal

User story
As an IT admin,
I want to see the upload progress for software packages without timeout
so that I can deploy large software (e.g. Microsoft Excel over 1GB) to my fleet.

Context

• Requestor(s): @lukeheath
• Product designer: @marko-lisica

Changes

Product

• UI changes: Figma link
• CLI changes: Figma link
• Other: Work with CS team to increase the server timeout limit to 15 minutes for all manged cloud customers and update best practice reference architecture.
• Reference documentation changes: Update best practice reference architecture.
  • UPDATE: @noahtalerman: Updates to best practice Terraform are in the PR here. No updates needed to best practice reference architecture

Engineering

• Usage documentation changes: No guide needed
• Database schema migrations: No migrations
• Load testing: No load testing needed

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Risk level: Low
• Risk description: Frontend only, low risk

Manual testing steps

1. Test progress bar is present when uploading an installer

• Upload large installers over 1gb or throttle your connection

2. Test native browser warning modal displays alert requiring approval to close

• While the installer is uploading, attempt to close the browser window or navigate to another tab and confirm alert

3. Test size limit has been increased to 3GB and larger files are not allowed

• Upload an installer larger than 500mb (the old size limit) and ensure it succeeds
• Upload an installer larger than 3GB and ensure you get an error message

4. Test 2min limit timeout has been removed

• Throttle your internet connection and upload a large file. Watch inspector to ensure the upload does not get terminated due to any time out limitations. Ensure it lasts beyond the old 2min limit

GitOps - test changes in file size limitation increase

• Use Fleetctl to upload an installer larger than 500MB and ensure it succeeds
• Use Fleetctl to upload an installer larger than 3GB and ensure it fails with the appropriate error message

Testing notes

Gitops and the 3gb file limit increase were tested as part of this ticket #22704

Confirmation

1. Engineer (@____): Added comment to user story confirming successful completion of QA.
2. QA (@PezHub ): Added comment to user story confirming successful completion of QA.

[noahtalerman]

Thanks for tracking this @lukeheath.

I spoke with @spokanemac and he mentioned this is happening because the file upload is timing out. Not notifying the user there is a file upload maximum, or telling them when it gets reached, is a bug that I'll file separately.

I don't think max file size is the issue. The expected max file size is 500 MB and, looking at the bug here, it looks like you uploaded a 204 MB package.

Rather, I think we're not showing the right error message when the user hits the 2 minute timeout.

I think it makes sense to fix that wrong error message bug and then follow up w/ this progress indicator improvement separately.

[marko-lisica]

Converting issue description to a story format and moving original issue description here:

Problem

When uploading large software files, the indeterminate loading spinner doesn't make clear how far along my progress is, or if the file upload has stalled. After waiting several minutes, mine closed and I got an error saying it couldn't complete.

I spoke with @spokanemac and he mentioned this is happening because the file upload is timing out. Not notifying the user there is a file upload maximum, or telling them when it gets reached, is a bug that I'll file separately.

UPDATE: I think that it's not the file size maximum. The maximum is 500 MB (noahtalerman 2024-07-11)

What have you tried?

Upload Google Chrome universal installer.

Potential solutions

1. Determinate loading indicator that shows actual file upload progress.

What is the expected workflow as a result of your proposal?

[noahtalerman]

Hey @georgekarrv heads up that it looks like we missed estimating this one before today's sprint kickoff.

I think we missed it because it didn't have the #g-mdm label?

I left it in the ready for spec column. Do you think we can estimate this one at the next standup and bring it into the current sprint if we have room?

cc @lukeheath

[lukeheath]

I started to self-assign before I realized the scope has grown quite a bit.

@ghernandez345 I put together a draft PR proposing this which may or may not be a helpful starting point: #19086. Up to you!

[georgekarrv]

Hey team! Please add your planning poker estimate with Zenhub @ghernandez345 @gillespi314

[lukeheath]

As we prepare for app library next sprint we're escalating this to P2 so that it can be prioritized over bugs. Some of the packages that will be uploaded are very large, and on a less-than-stellar connection can take several minutes to finish. If we just using a loading spinner without indicating process, it appears broken after a couple of minutes of spinning, even if it's still in progress.

[PezHub]

QA Notes:

• Progress bar is now present
• Closing browser windows throws warning and behaves as expected

*GitOps, removing the 2min timeout, and the new 3GB file size limit will be tested once ready (it was a late addition to the ticket)

[PezHub]

QA Notes - increase file size upload limit & remove 2min timeout:

• 2.49GB MSOffice pkg upload completed successfully
• file over 3gb threw the expected error
  
• confirmed 2min timeout has been removed. I throttled my connection and uploaded a large file and it continued to upload for 53min before I finally killed it

[PezHub]

I'm going to pass this ticket as good and continue testing the Gitops workflow here #22704

[lukeheath]

@PezHub @georgekarrv Please add manual testing steps to this story.

[PezHub]

Manual testing steps have been added

[marko-lisica]

Hey @zayhanlon, can CS team help us with updates specified in "Product" section?

Work with CS team to increase the server timeout limit to 15 minutes for all manged cloud customers and update best practice reference architecture.

It's also required for Fleet-maintained apps too. #20308

[noahtalerman]

Hey @zayhanlon just giving you another ping!

I tracked this as a CS request here: #23286

We want to increase the server timeout limit to 15 mins for all managed cloud customers and Fleet's best practice reference architecture for self-hosted instance.

Why? So that users on slow Wi-Fi don't sit in front of a progress bar for 5 minutes just to see the software upload fail. My current understanding is that the timeout is 5 mins.

[zayhanlon]

@noahtalerman @marko-lisica i'm sorry i missed this!

we took the new issue into our sprint today - it will be implemented during the next deploy (so when we schedule 4.59 sometime next week)

[noahtalerman]

Waiting until we bump the server timeout limit to 15 mins to close this story. More context in a separate issue #20308 (comment).

[noahtalerman]

Hey @zayhanlon now that 4.59 is shipped, reminder to bump the server timeout limit for cloud customers and update the reference architecture docs: #23286

[zayhanlon]

@noahtalerman thanks! we will take care of it on cloud deploy tomorrow

[noahtalerman]

@zayhanlon, it looks like the cloud upgrade issue is still open but just checking, did the cloud deploy w/ the server timeout limit bump happen?

[zayhanlon]

@noahtalerman yes it did! last thursday night

[noahtalerman]

We bumped the timeout to 15 mins for managed cloud customers ✅

PR to update best practice Terraform is here: #23939

[noahtalerman]

PR to update best practice Terraform is here: #23939

PR is merged.

Closing this story.

[fleet-release]

Progress bar, like moon,
Guides through vast data seas,
Fleet's upload journey blooms.