Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/data export #815

Open
wants to merge 23 commits into
base: dev
Choose a base branch
from
Open

Fix/data export #815

wants to merge 23 commits into from

Conversation

kasimhelil
Copy link
Contributor

This resolves #780

  • Added exported file name for all pages with table,
    • Appended all when all grid data is exported (assuming I will get export the current page only work)
  • Added **sortType** props that help us to pass a customized sort function for each column
    • e.g dob column on flightpax, biographic column on vetting, and timestamp column on audit log
  • On vetting, export departure or arrival time for timer column (since timer is not exportable)
  • Disabled hits column on vetting page (I need to fined a cleaner way to export hits data; I will create a ticket)
  • Hit counts can be exported only from Hits tab grid (on both flight and flightpx)

@kasimhelil kasimhelil requested a review from KMnOFour June 10, 2021 15:52
@KMnOFour KMnOFour added the bug 🐞 Something isn't working label Jun 10, 2021
@KMnOFour
Copy link
Contributor

KMnOFour commented Jun 10, 2021

Gonna break this into separate comments cause it's getting long.

The first issue is a critical performance issue on the vetting page on initial load. This must be resolved before this update can go in. It renders the page unresponsive until the garbage collector kicks in. Possibly the new export parsing code is getting rerun on every dom refresh. Will debug later tonight.

Images are from devtools perfmonitor, no throttling, cache enabled.

Below, I am getting cpu spikes at 100% for about 20 seconds in Chrome (top pink arrow) when I load the Vetting page, and the jsheap shoots from 20mb to 350mb or so (middle pink arrow). For comparison, the little spike at the black arrow is from loading the Flights page.

image

For comparison, I ran it again in the dev branch (below). Flight page load is the black arrow, pink is the Vetting page. We don't have hard perf requirements to meet, but the load should look more or less like this.

image

@KMnOFour
Copy link
Contributor

KMnOFour commented Jun 10, 2021

Export column issues:

  1. The countdown timer data is very important. I like that we included the eta/etd dates in some places but I think we also need a solution to show the timer data. It should show everywhere it's used in a table.
  2. FlightPax - the document numbers and PNR Rec Locator need to be converted to text. In excel at least, leading zeroes get truncated if the entire ID is numeric, values in the same column get aligned differently depending on whether they begin with a digit or letter, and some numeric IDs get formatted as exponents. We can massage data like
    0005, 32940002
    as
    ="0005", ="32940002"
    to prevent losing the leading zeroes and accidental conversion to exponent format and ensure all values in a column are aligned the same way
  3. FlightPax - PNR Record Loc. column got renamed to Co-traveler ID.
  4. Paxdetail Attachments - Actions col should be removed, export filename should be updated.
  5. Vetting page probably needs a separate ticket. We need:
  • the flight info and the biographic info, each broken out into their individual columns (dest/origin airports, dob, nationality, etc).
  • hits column data needs to show (will have to spec out how to display it in the ticket)
  • need to show both arrival and departure times and probably the flight direction as well. We'll discuss later.
  1. Boolean field values should be converted to yes/no or true/false (Rules, manage users, where else?)
  2. Code Editor IATA, ICAO, and ISO Numeric codes should be converted to text
  3. Lookout lanes - Order col is renamed to Ord.
  4. Tools Watchlist - document number should be text.
  5. Manage Users - boolean values should be yes/no
  6. Language editor - old filename

We should discuss Monday whether to add a separate export button just for Excel or do the text conversion for the existing one.

@kasimhelil
Copy link
Contributor Author

Export column issues:

  1. The countdown timer data is very important. I like that we included the eta/etd dates in some places but I think we also need a solution to show the timer data. It should show everywhere it's used in a table.
  2. FlightPax - the document numbers and PNR Rec Locator need to be converted to text. In excel at least, leading zeroes get truncated if the entire ID is numeric, values in the same column get aligned differently depending on whether they begin with a digit or letter, and some numeric IDs get formatted as exponents. We can massage data like
    0005, 32940002
    as
    ="0005", ="32940002"
    to prevent losing the leading zeroes and accidental conversion to exponent format and ensure all values in a column are aligned the same way
  3. FlightPax - PNR Record Loc. column got renamed to Co-traveler ID.
  4. Paxdetail Attachments - Actions col should be removed, export filename should be updated.
  5. Vetting page probably needs a separate ticket. We need:
  • the flight info and the biographic info, each broken out into their individual columns (dest/origin airports, dob, nationality, etc).
  • hits column data needs to show (will have to spec out how to display it in the ticket)
  • need to show both arrival and departure times and probably the flight direction as well. We'll discuss later.
  1. Boolean field values should be converted to yes/no or true/false (Rules, manage users, where else?)
  2. Code Editor IATA, ICAO, and ISO Numeric codes should be converted to text
  3. Lookout lanes - Order col is renamed to Ord.
  4. Tools Watchlist - document number should be text.
  5. Manage Users - boolean values should be yes/no
  6. Language editor - old filename

We should discuss Monday whether to add a separate export button just for Excel or do the text conversion for the existing one.

  • We won't export the countdown timer until we find a way to keep counting down the time after exporting the data. For now, we will export the arrival and departure time only.
  • All the other issues are fixed

@KMnOFour
Copy link
Contributor

^ Agreed. So the only piece left is nailing down the performance issue which is on my todo list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐞 Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants