-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[Odometer] Image stitching #83556
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Julesssss
merged 28 commits into
Expensify:main
from
software-mansion-labs:jakubkalinski0/Odometer_Add_merged_image_flow
Mar 13, 2026
Merged
[Odometer] Image stitching #83556
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
40425d1
Initial implementation of image stitching for Odometer
jakubkalinski0 3f840aa
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 5fbbcaf
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 d1f1717
improvement: moved imageStitching feature from hooks to lib
jakubkalinski0 e32a3ee
feature: when only 1 odometer image exist set it as a receipt
jakubkalinski0 b021cc8
improvement: added stitchLayout to make code DRY
jakubkalinski0 a7959e1
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 81b3f51
fix: added catch error handling to image stitching
jakubkalinski0 93bec81
fix: added revokeObjectURL of a stitched image
jakubkalinski0 dff2791
improvement: added error to display for user when navigateToNextPage …
jakubkalinski0 c77e153
improvement: small adjustments to comments in imageStitching
jakubkalinski0 bb6b774
fix: added file cleanup to native image stitching
jakubkalinski0 d61057a
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 df136c7
fix: added async to navigateToNextPage
jakubkalinski0 c831f96
fix: remove fallback to receipt source to avoid receipt flash on odom…
jakubkalinski0 f74f113
fix: preserve filename for camera-captured odometer image on mWeb
jakubkalinski0 ea7bd64
improvement: move error handling from stitchOdometerImages to caller
jakubkalinski0 ce37aed
fix: hide replace button for stitched odometer receipt
jakubkalinski0 d5f5d5d
improvement: scale images to match shared edge length when stitching
jakubkalinski0 d36ad3a
fix: use contain resize mode for stitched odometer receipt preview
jakubkalinski0 4b0ed83
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 c78936a
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 e3810a0
fix: added XYWH to cspell.json as an acceptable word
jakubkalinski0 18e5d32
added translations
jakubkalinski0 729c537
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 e77527f
Merge branch 'main' into jakubkalinski0/Odometer_Add_merged_image_flow
jakubkalinski0 4976987
fix: use isOdometerDistanceRequest util instead of filename-based odo…
jakubkalinski0 072e926
chore: prettier run
jakubkalinski0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -843,6 +843,7 @@ | |
| "xmlgateway", | ||
| "Xours", | ||
| "Xtheirs", | ||
| "XYWH", | ||
| "yalc", | ||
| "Yapl", | ||
| "YAPL", | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| import {Skia} from '@shopify/react-native-skia'; | ||
| import RNFS from 'react-native-fs'; | ||
| import Log from '@libs/Log'; | ||
| import type {FileObject} from '@src/types/utils/Attachment'; | ||
| import calculateStitchLayout from './stitchLayout'; | ||
|
|
||
| async function stitchOdometerImages(image1: FileObject | string | undefined, image2: FileObject | string | undefined): Promise<FileObject | null> { | ||
| const source1 = typeof image1 === 'string' ? image1 : (image1?.uri ?? null); | ||
| const source2 = typeof image2 === 'string' ? image2 : (image2?.uri ?? null); | ||
|
|
||
| if (!source1 || !source2) { | ||
| return null; | ||
| } | ||
|
|
||
| let skImage1 = null; | ||
| let skImage2 = null; | ||
| let surface = null; | ||
| let snapshot = null; | ||
|
|
||
| try { | ||
| const [buffer1, buffer2] = await Promise.all([fetch(source1).then((r) => r.arrayBuffer()), fetch(source2).then((r) => r.arrayBuffer())]); | ||
|
|
||
| skImage1 = Skia.Image.MakeImageFromEncoded(Skia.Data.fromBytes(new Uint8Array(buffer1))); | ||
| skImage2 = Skia.Image.MakeImageFromEncoded(Skia.Data.fromBytes(new Uint8Array(buffer2))); | ||
|
|
||
| if (!skImage1 || !skImage2) { | ||
| throw new Error('Failed to decode odometer images'); | ||
| } | ||
|
|
||
| const {width, height, img1Dest, img2Dest} = calculateStitchLayout(skImage1.width(), skImage1.height(), skImage2.width(), skImage2.height()); | ||
|
|
||
| surface = Skia.Surface.MakeOffscreen(width, height); | ||
| if (!surface) { | ||
| throw new Error('Failed to create Skia surface'); | ||
| } | ||
|
|
||
| const canvas = surface.getCanvas(); | ||
| const paint = Skia.Paint(); | ||
| canvas.drawImageRect(skImage1, Skia.XYWHRect(0, 0, skImage1.width(), skImage1.height()), Skia.XYWHRect(img1Dest.x, img1Dest.y, img1Dest.w, img1Dest.h), paint); | ||
| canvas.drawImageRect(skImage2, Skia.XYWHRect(0, 0, skImage2.width(), skImage2.height()), Skia.XYWHRect(img2Dest.x, img2Dest.y, img2Dest.w, img2Dest.h), paint); | ||
| surface.flush(); | ||
|
|
||
| snapshot = surface.makeImageSnapshot(); | ||
| const base64 = snapshot.encodeToBase64(); | ||
|
|
||
| // Delete any previously stitched files before creating a new one | ||
| try { | ||
| const tempDirContents = await RNFS.readDir(RNFS.TemporaryDirectoryPath); | ||
| const oldStitchedFiles = tempDirContents.filter((f) => f.name.startsWith('stitched_odometer_') && f.name.endsWith('.jpg')); | ||
| await Promise.all(oldStitchedFiles.map((f) => RNFS.unlink(f.path))); | ||
jakubkalinski0 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| } catch (error) { | ||
| Log.warn('stitchOdometerImages (native) failed to clean up old stitched files', {error}); | ||
| } | ||
|
|
||
| const filename = `stitched_odometer_${Date.now()}.jpg`; | ||
| const tempPath = `${RNFS.TemporaryDirectoryPath}/${filename}`; | ||
| await RNFS.writeFile(tempPath, base64, 'base64'); | ||
|
|
||
| return {uri: `file://${tempPath}`, name: filename, type: 'image/jpeg'}; | ||
| } finally { | ||
| skImage1?.dispose?.(); | ||
| skImage2?.dispose?.(); | ||
| snapshot?.dispose?.(); | ||
| surface?.dispose?.(); | ||
| } | ||
| } | ||
|
|
||
| export default stitchOdometerImages; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| import type {FileObject} from '@src/types/utils/Attachment'; | ||
| import calculateStitchLayout from './stitchLayout'; | ||
|
|
||
| // Tracks the single active stitched blob URL so that we can revoke it on the next call so at most one blob URL exists at a time | ||
| let previousBlobUrl: string | null = null; | ||
|
|
||
| function stitchOdometerImages(image1: FileObject | string | undefined, image2: FileObject | string | undefined): Promise<FileObject | null> { | ||
| const source1 = typeof image1 === 'string' ? image1 : (image1?.uri ?? null); | ||
| const source2 = typeof image2 === 'string' ? image2 : (image2?.uri ?? null); | ||
|
|
||
| if (!source1 || !source2) { | ||
| return Promise.resolve(null); | ||
| } | ||
|
|
||
| const loadImage = (src: string): Promise<HTMLImageElement> => | ||
| new Promise((resolve, reject) => { | ||
| const img = new window.Image(); | ||
| img.onload = () => resolve(img); | ||
| img.onerror = reject; | ||
| img.src = src; | ||
| }); | ||
|
|
||
| return Promise.all([loadImage(source1), loadImage(source2)]).then(([img1, img2]) => { | ||
| const {width, height, img1Dest, img2Dest} = calculateStitchLayout(img1.width, img1.height, img2.width, img2.height); | ||
|
|
||
| const offscreenCanvas = document.createElement('canvas'); | ||
| offscreenCanvas.width = width; | ||
| offscreenCanvas.height = height; | ||
| const ctx = offscreenCanvas.getContext('2d'); | ||
| if (!ctx) { | ||
| throw new Error('Failed to get canvas context'); | ||
| } | ||
|
|
||
| ctx.drawImage(img1, img1Dest.x, img1Dest.y, img1Dest.w, img1Dest.h); | ||
| ctx.drawImage(img2, img2Dest.x, img2Dest.y, img2Dest.w, img2Dest.h); | ||
|
|
||
| return new Promise<FileObject | null>((resolve, reject) => { | ||
| offscreenCanvas.toBlob((blob) => { | ||
| if (!blob) { | ||
| reject(new Error('Canvas toBlob returned null')); | ||
| return; | ||
| } | ||
| if (previousBlobUrl) { | ||
| URL.revokeObjectURL(previousBlobUrl); | ||
| } | ||
| const uri = URL.createObjectURL(blob); | ||
| previousBlobUrl = uri; | ||
| resolve({uri, name: 'stitched_odometer.jpg', type: 'image/jpeg'}); | ||
| }, 'image/jpeg'); | ||
| }); | ||
| }); | ||
| } | ||
|
|
||
| export default stitchOdometerImages; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| type ImageRect = {x: number; y: number; w: number; h: number}; | ||
|
|
||
| type StitchLayout = { | ||
| width: number; | ||
| height: number; | ||
| horizontal: boolean; | ||
| img1Dest: ImageRect; | ||
| img2Dest: ImageRect; | ||
| }; | ||
|
|
||
| /** | ||
| * Calculates the layout for stitching two odometer images into a single combined image. | ||
| * | ||
| * Stitching rules: | ||
| * - Images are merged horizontally (side-by-side) by default | ||
| * - If either image is landscape, images are merged vertically (stacked) | ||
| * - The smaller image is resized so the shared edge matches the length of the larger image's shared edge | ||
| */ | ||
|
|
||
| function calculateStitchLayout(w1: number, h1: number, w2: number, h2: number): StitchLayout { | ||
| const horizontal = !(w1 > h1 || w2 > h2); | ||
|
|
||
| if (horizontal) { | ||
| // Side-by-side: scale both images to the same height so the shared vertical edges match | ||
| const targetH = Math.max(h1, h2); | ||
| const scaledW1 = Math.round((w1 * targetH) / h1); | ||
| const scaledW2 = Math.round((w2 * targetH) / h2); | ||
| return { | ||
| width: scaledW1 + scaledW2, | ||
| height: targetH, | ||
| horizontal: true, | ||
| img1Dest: {x: 0, y: 0, w: scaledW1, h: targetH}, | ||
| img2Dest: {x: scaledW1, y: 0, w: scaledW2, h: targetH}, | ||
| }; | ||
| } | ||
|
|
||
| // Stacked: scale both images to the same width so the shared horizontal edges match | ||
| const targetW = Math.max(w1, w2); | ||
| const scaledH1 = Math.round((h1 * targetW) / w1); | ||
| const scaledH2 = Math.round((h2 * targetW) / w2); | ||
| return { | ||
| width: targetW, | ||
| height: scaledH1 + scaledH2, | ||
| horizontal: false, | ||
| img1Dest: {x: 0, y: 0, w: targetW, h: scaledH1}, | ||
| img2Dest: {x: 0, y: scaledH1, w: targetW, h: scaledH2}, | ||
| }; | ||
| } | ||
|
|
||
| export default calculateStitchLayout; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.