Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Conditionally bundling assets based on flavor make tests to fail #150296

Closed
AngeloAvv opened this issue Jun 15, 2024 · 8 comments · Fixed by #150461
Closed

Conditionally bundling assets based on flavor make tests to fail #150296

AngeloAvv opened this issue Jun 15, 2024 · 8 comments · Fixed by #150461
Assignees
@andrewkolos
Labels
found in release: 3.22 Found to occur in 3.22 found in release: 3.23 Found to occur in 3.23 has reproducible steps The issue has been confirmed reproducible and is ready to work on P1 High-priority issues at the top of the work list r: fixed Issue is closed as already fixed in a newer version team-tool Owned by Flutter Tool team tool Affects the "flutter" command-line tool. See also t: labels. triaged-tool Triaged by Flutter Tool team

Comments

@AngeloAvv
Copy link

AngeloAvv commented Jun 15, 2024 •
edited
Loading

Steps to reproduce

  1. Create a new Flutter project (no need to create native flavors as this also fails with normal projects too)
  2. Add a new asset image and register it under pubspec.yaml as a flavor asset
  3. Create your app with an Image that references the image asset you just added to your project
  4. Write your UI tests asserting the image will be there
  5. Run your tests (with or without the --flavor param)
  6. Tests are failing: Unable to load asset

Expected results

  1. Create a new Flutter project (no need to create native flavors as this also fails with normal projects too)
  2. Add a new asset image and register it under pubspec.yaml as a flavor asset
  3. Create your app with an Image that references the image asset you just added to your project
  4. Write your UI tests asserting the image will be there
  5. Run your tests (with or without the --flavor param)
  6. Tests are passing and assets are properly rendered under UI tests

Actual results

Unable to load assets, no matter if you pass or not the flavor parameter.

Also, if you don't configure conditional imports under pubspec.yaml, tests work like a charm:

  1. Create a new Flutter project (no need to create native flavors as this also fails with normal projects too)
  2. Add a new asset image and register it under pubspec.yaml
  3. Create your app with an Image that references the image asset you just added to your project
  4. Write your UI tests asserting the image will be there
  5. Run your tests (with or without the --flavor param)
  6. Tests are passing

Code sample

Code sample 
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatelessWidget {
  const MyHomePage({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: Center(
        child: Image.asset('assets/flavor/image.png'),
      ),
    );
  }
}
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

import 'package:flavor_assets_test/main.dart';

void main() {
  testWidgets('Expect Image', (tester) async {
    await tester.pumpWidget(const MyApp());

    expect(find.byType(Image), findsOneWidget);
  });
}

name: flavor_assets_test
description: "A new Flutter project."
publish_to: 'none'
version: 1.0.0+1

environment:
  sdk: '>=3.4.1 <4.0.0'
dependencies:
  flutter:
    sdk: flutter
dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^3.0.0
flutter:
  uses-material-design: true

  assets:
    - path: assets/flavor/
      flavors:
        - flavor

Screenshots or Video

No response

Logs

Logs (with flavor argument) 
fvm flutter test --flavor flavor
00:00 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:01 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:02 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:03 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:04 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:04 +0: Expect Image                                                                                                                                                          00:04 +0: Expect Image                                                                                                                                                                                 
══╡ EXCEPTION CAUGHT BY IMAGE RESOURCE SERVICE ╞════════════════════════════════════════════════════
The following assertion was thrown resolving an image codec:
Unable to load asset: "assets/flavor/image.png".
The asset does not exist or has empty data.

When the exception was thrown, this was the stack:
#0      PlatformAssetBundle.load (package:flutter/src/services/asset_bundle.dart:332:7)
#1      PlatformAssetBundle.loadBuffer (package:flutter/src/services/asset_bundle.dart:359:36)
#2      AssetBundleImageProvider._loadAsync (package:flutter/src/painting/image_provider.dart:754:33)
#3      AssetBundleImageProvider.loadImage (package:flutter/src/painting/image_provider.dart:714:14)
#4      ImageProvider.resolveStreamForKey.<anonymous closure> (package:flutter/src/painting/image_provider.dart:520:39)
#5      ImageCache.putIfAbsent (package:flutter/src/painting/image_cache.dart:377:22)
#6      ImageProvider.resolveStreamForKey (package:flutter/src/painting/image_provider.dart:517:81)
#7      ScrollAwareImageProvider.resolveStreamForKey (package:flutter/src/widgets/scroll_aware_image_provider.dart:104:19)
#8      ImageProvider.resolve.<anonymous closure> (package:flutter/src/painting/image_provider.dart:366:9)
#9      ImageProvider._createErrorHandlerAndKey.<anonymous closure> (package:flutter/src/painting/image_provider.dart:479:24)
#10     SynchronousFuture.then (package:flutter/src/foundation/synchronous_future.dart:43:39)
#11     ImageProvider._createErrorHandlerAndKey (package:flutter/src/painting/image_provider.dart:476:9)
#12     ImageProvider.resolve (package:flutter/src/painting/image_provider.dart:363:5)
#13     _ImageState._resolveImage (package:flutter/src/widgets/image.dart:1111:16)
#14     _ImageState.didChangeDependencies (package:flutter/src/widgets/image.dart:1061:5)
#15     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:5636:11)
#16     ComponentElement.mount (package:flutter/src/widgets/framework.dart:5463:5)
...     Normal element mounting (28 frames)
#44     Element.inflateWidget (package:flutter/src/widgets/framework.dart:4340:16)
#45     MultiChildRenderObjectElement.inflateWidget (package:flutter/src/widgets/framework.dart:6904:36)
#46     MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:6916:32)
...     Normal element mounting (338 frames)
#384    Element.inflateWidget (package:flutter/src/widgets/framework.dart:4340:16)
#385    MultiChildRenderObjectElement.inflateWidget (package:flutter/src/widgets/framework.dart:6904:36)
#386    MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:6916:32)
...     Normal element mounting (463 frames)
#849    Element.inflateWidget (package:flutter/src/widgets/framework.dart:4340:16)
#850    Element.updateChild (package:flutter/src/widgets/framework.dart:3843:20)
#851    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#852    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#853    ProxyElement.update (package:flutter/src/widgets/framework.dart:5816:5)
#854    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#855    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#856    StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:5650:11)
#857    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#858    StatefulElement.update (package:flutter/src/widgets/framework.dart:5673:5)
#859    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#860    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#861    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#862    ProxyElement.update (package:flutter/src/widgets/framework.dart:5816:5)
#863    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#864    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#865    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#866    ProxyElement.update (package:flutter/src/widgets/framework.dart:5816:5)
#867    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#868    _RawViewElement._updateChild (package:flutter/src/widgets/view.dart:291:16)
#869    _RawViewElement.update (package:flutter/src/widgets/view.dart:378:5)
#870    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#871    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#872    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#873    StatelessElement.update (package:flutter/src/widgets/framework.dart:5563:5)
#874    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#875    RootElement._rebuild (package:flutter/src/widgets/binding.dart:1581:16)
#876    RootElement.update (package:flutter/src/widgets/binding.dart:1559:5)
#877    RootElement.performRebuild (package:flutter/src/widgets/binding.dart:1573:7)
#878    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#879    BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2905:19)
#880    AutomatedTestWidgetsFlutterBinding.drawFrame (package:flutter_test/src/binding.dart:1418:19)
#881    RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:443:5)
#882    SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1392:15)
#883    SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1313:9)
#884    AutomatedTestWidgetsFlutterBinding.pump.<anonymous closure> (package:flutter_test/src/binding.dart:1273:9)
#887    TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#888    AutomatedTestWidgetsFlutterBinding.pump (package:flutter_test/src/binding.dart:1260:27)
#889    WidgetTester.pumpWidget.<anonymous closure> (package:flutter_test/src/widget_tester.dart:602:22)
#892    TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#893    WidgetTester.pumpWidget (package:flutter_test/src/widget_tester.dart:599:27)
#894    main.<anonymous closure> (file:///Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart:8:18)
#895    testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:29)
<asynchronous suspension>
#896    TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

Image provider: AssetImage(bundle: null, name: "assets/flavor/image.png")
Image key: AssetBundleImageKey(bundle: PlatformAssetBundle#161dc(), name: "assets/flavor/image.png",
  scale: 1.0)
════════════════════════════════════════════════════════════════════════════════════════════════════
00:04 +0 -1: Expect Image [E]                                                                                                                                                                          
  Test failed. See exception logs above.
  The test description was: Expect Image
  

To run this test again: /Users/angeloavv/fvm/versions/3.22.1/bin/cache/dart-sdk/bin/dart test /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart -p vm --plain-name 'Expect Image'
00:04 +0 -1: Some tests failed.
Logs (without flavor argument) 
 fvm flutter test
Resolving dependencies...
Downloading packages...
  collection 1.18.0 (1.19.0 available)
  flutter_lints 3.0.2 (4.0.0 available)
  leak_tracker 10.0.4 (10.0.5 available)
  leak_tracker_flutter_testing 3.0.3 (3.0.5 available)
  lints 3.0.0 (4.0.0 available)
  material_color_utilities 0.8.0 (0.12.0 available)
  meta 1.12.0 (1.15.0 available)
  test_api 0.7.0 (0.7.2 available)
  vm_service 14.2.1 (14.2.4 available)
Got dependencies!
9 packages have newer versions incompatible with dependency constraints.
Try `flutter pub outdated` for more information.
00:00 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:01 +0: loading /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart                                                                                      00:01 +0: Expect Image                                                                                                                                                          00:01 +0: Expect Image                                                                                                                                                                                 
══╡ EXCEPTION CAUGHT BY IMAGE RESOURCE SERVICE ╞════════════════════════════════════════════════════
The following assertion was thrown resolving an image codec:
Unable to load asset: "assets/flavor/image.png".
The asset does not exist or has empty data.

When the exception was thrown, this was the stack:
#0      PlatformAssetBundle.load (package:flutter/src/services/asset_bundle.dart:332:7)
#1      PlatformAssetBundle.loadBuffer (package:flutter/src/services/asset_bundle.dart:359:36)
#2      AssetBundleImageProvider._loadAsync (package:flutter/src/painting/image_provider.dart:754:33)
#3      AssetBundleImageProvider.loadImage (package:flutter/src/painting/image_provider.dart:714:14)
#4      ImageProvider.resolveStreamForKey.<anonymous closure> (package:flutter/src/painting/image_provider.dart:520:39)
#5      ImageCache.putIfAbsent (package:flutter/src/painting/image_cache.dart:377:22)
#6      ImageProvider.resolveStreamForKey (package:flutter/src/painting/image_provider.dart:517:81)
#7      ScrollAwareImageProvider.resolveStreamForKey (package:flutter/src/widgets/scroll_aware_image_provider.dart:104:19)
#8      ImageProvider.resolve.<anonymous closure> (package:flutter/src/painting/image_provider.dart:366:9)
#9      ImageProvider._createErrorHandlerAndKey.<anonymous closure> (package:flutter/src/painting/image_provider.dart:479:24)
#10     SynchronousFuture.then (package:flutter/src/foundation/synchronous_future.dart:43:39)
#11     ImageProvider._createErrorHandlerAndKey (package:flutter/src/painting/image_provider.dart:476:9)
#12     ImageProvider.resolve (package:flutter/src/painting/image_provider.dart:363:5)
#13     _ImageState._resolveImage (package:flutter/src/widgets/image.dart:1111:16)
#14     _ImageState.didChangeDependencies (package:flutter/src/widgets/image.dart:1061:5)
#15     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:5636:11)
#16     ComponentElement.mount (package:flutter/src/widgets/framework.dart:5463:5)
...     Normal element mounting (28 frames)
#44     Element.inflateWidget (package:flutter/src/widgets/framework.dart:4340:16)
#45     MultiChildRenderObjectElement.inflateWidget (package:flutter/src/widgets/framework.dart:6904:36)
#46     MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:6916:32)
...     Normal element mounting (338 frames)
#384    Element.inflateWidget (package:flutter/src/widgets/framework.dart:4340:16)
#385    MultiChildRenderObjectElement.inflateWidget (package:flutter/src/widgets/framework.dart:6904:36)
#386    MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:6916:32)
...     Normal element mounting (463 frames)
#849    Element.inflateWidget (package:flutter/src/widgets/framework.dart:4340:16)
#850    Element.updateChild (package:flutter/src/widgets/framework.dart:3843:20)
#851    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#852    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#853    ProxyElement.update (package:flutter/src/widgets/framework.dart:5816:5)
#854    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#855    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#856    StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:5650:11)
#857    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#858    StatefulElement.update (package:flutter/src/widgets/framework.dart:5673:5)
#859    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#860    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#861    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#862    ProxyElement.update (package:flutter/src/widgets/framework.dart:5816:5)
#863    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#864    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#865    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#866    ProxyElement.update (package:flutter/src/widgets/framework.dart:5816:5)
#867    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#868    _RawViewElement._updateChild (package:flutter/src/widgets/view.dart:291:16)
#869    _RawViewElement.update (package:flutter/src/widgets/view.dart:378:5)
#870    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#871    ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:5512:16)
#872    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#873    StatelessElement.update (package:flutter/src/widgets/framework.dart:5563:5)
#874    Element.updateChild (package:flutter/src/widgets/framework.dart:3827:15)
#875    RootElement._rebuild (package:flutter/src/widgets/binding.dart:1581:16)
#876    RootElement.update (package:flutter/src/widgets/binding.dart:1559:5)
#877    RootElement.performRebuild (package:flutter/src/widgets/binding.dart:1573:7)
#878    Element.rebuild (package:flutter/src/widgets/framework.dart:5203:7)
#879    BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2905:19)
#880    AutomatedTestWidgetsFlutterBinding.drawFrame (package:flutter_test/src/binding.dart:1418:19)
#881    RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:443:5)
#882    SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1392:15)
#883    SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1313:9)
#884    AutomatedTestWidgetsFlutterBinding.pump.<anonymous closure> (package:flutter_test/src/binding.dart:1273:9)
#887    TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#888    AutomatedTestWidgetsFlutterBinding.pump (package:flutter_test/src/binding.dart:1260:27)
#889    WidgetTester.pumpWidget.<anonymous closure> (package:flutter_test/src/widget_tester.dart:602:22)
#892    TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#893    WidgetTester.pumpWidget (package:flutter_test/src/widget_tester.dart:599:27)
#894    main.<anonymous closure> (file:///Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart:8:18)
#895    testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:29)
<asynchronous suspension>
#896    TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

Image provider: AssetImage(bundle: null, name: "assets/flavor/image.png")
Image key: AssetBundleImageKey(bundle: PlatformAssetBundle#b2f74(), name: "assets/flavor/image.png",
  scale: 1.0)
════════════════════════════════════════════════════════════════════════════════════════════════════
00:01 +0 -1: Expect Image [E]                                                                                                                                                                          
  Test failed. See exception logs above.
  The test description was: Expect Image
  

To run this test again: /Users/angeloavv/fvm/versions/3.22.1/bin/cache/dart-sdk/bin/dart test /Users/angeloavv/StudioProjects/flavor_assets_test/test/widget_test.dart -p vm --plain-name 'Expect Image'
00:01 +0 -1: Some tests failed.

Flutter Doctor output

Doctor output 
 fvm flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel stable, 3.22.1, on macOS 14.5 23F79 darwin-arm64, locale it-IT)
    ! Warning: `dart` on your path resolves to /opt/homebrew/Cellar/dart/3.2.3/libexec/bin/dart, which is not inside your current Flutter SDK checkout at /Users/angeloavv/fvm/versions/3.22.1. Consider adding /Users/angeloavv/fvm/versions/3.22.1/bin to the front of your path.
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2024.1)
[✓] IntelliJ IDEA Community Edition (version 2024.1.3)
[✓] Connected device (5 available)
[✓] Network resources

! Doctor found issues in 1 category.
@darshankawar darshankawar added the in triage Presently being triaged by the triage team label Jun 17, 2024
@darshankawar
Copy link
Member

Thanks for the detailed report @AngeloAvv

  • Write your UI tests asserting the image will be there
  • Run your tests (with or without the --flavor param)
  • Tests are failing: Unable to load asset

Since you are trying to run flutter test, according to #132985, the use case you are trying to achieve currently supports flutter run and flutter build only, so it may not be applicable to flutter test that you can check.

Also, /cc @andrewkolos for thoughts on the issue report.

@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Jun 17, 2024
@andrewkolos
Copy link
Contributor

Thanks for filing this (and thanks for tagging me)!

I think it's fair to call this a bug (or surprising behavior at the very least). If --flavor is supported in flutter test, then flavor-conditional asset bundling should work for it as well. I'll try to patch this.

@andrewkolos andrewkolos added a: assets Packaging, accessing, or using assets P1 High-priority issues at the top of the work list team-tool Owned by Flutter Tool team triaged-tool Triaged by Flutter Tool team and removed waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds labels Jun 18, 2024
@darshankawar darshankawar removed the in triage Presently being triaged by the triage team label Jun 18, 2024
@andrewkolos
Copy link
Contributor

andrewkolos commented Jun 18, 2024 •
edited
Loading

I looked at the code that builds and writes the asset bundle for flutter test, and it looks like we already pass along the flavor. I got confused, because I originally thought I was able to repro this, but I must have made a mistake when setting up my test project. I attempted to repro again using a fresh project, and I was unable to.

@darshankawar would you mind trying to repro this?

@andrewkolos andrewkolos added in triage Presently being triaged by the triage team and removed a: assets Packaging, accessing, or using assets P1 High-priority issues at the top of the work list team-tool Owned by Flutter Tool team triaged-tool Triaged by Flutter Tool team labels Jun 18, 2024
@AngeloAvv
Copy link
Author

Ok, @andrewkolos this looks weird. I was trying to share with you a repo to reproduce the code, so that's what I did.

  1. I ran the tests one more time, they were failing both with or without the flavor arg
  2. Init a .git repo
  3. Run the tests again: for some reason this time "flutter test" did a pub get under the hood which also cleaned something, tests were working fine with both parameters

I wasn't able to reproduce the bug anymore!

So here's what I did.

  1. I ran flutter clean
  2. Run flutter test without flavor param, tests failed
  3. Run flutter test with flavor param, tests are failing again

But if you do the opposite

  1. I ran flutter clean
  2. Run flutter test with flavor param, tests pass
  3. Run flutter test without flavor param, tests are passing

Looks like the assets files are not updated or something like that every time you switch from one run to another.
Here's a repo with a Flutter project I created for you where I'm able to reproduce the bug

@andrewkolos
Copy link
Contributor

Ah okay, this could be a caching bug. I will take a second look.

@andrewkolos
Copy link
Contributor

Yeah, it looks like we don't take --flavor into account when invalidating the asset bundle used in tests. We only look at the asset files and pubspec:

flutter/packages/flutter_tools/lib/src/commands/test.dart

Line 709 in 5e448f4

bool _needRebuild(Map<String, AssetBundleEntry> entries) {

@andrewkolos andrewkolos added P1 High-priority issues at the top of the work list team-tool Owned by Flutter Tool team triaged-tool Triaged by Flutter Tool team and removed in triage Presently being triaged by the triage team labels Jun 18, 2024
@andrewkolos andrewkolos self-assigned this Jun 18, 2024
@andrewkolos andrewkolos mentioned this issue Jun 18, 2024
Merged
9 tasks
@darshankawar
Copy link
Member

Here's a repo with a Flutter project I created for you where I'm able to reproduce the bug

Using this repo and running the test, it throws the reported error log.

stable, master flutter doctor -v 
[!] Flutter (Channel stable, 3.22.2, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.22.2 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 761747bfc5 (33 hours ago), 2024-06-05 22:15:13 +0200
    • Engine revision edd8546116
    • Dart version 3.4.3
    • DevTools version 2.34.3
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[!] Flutter (Channel master, 3.23.0-13.0.pre.262, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.23.0-13.0.pre.262 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6a8f70f11e (2 hours ago), 2024-06-18 00:12:10 -0400
    • Engine revision 78fdd06af5
    • Dart version 3.5.0 (build 3.5.0-272.0.dev)
    • DevTools version 2.36.0
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/dhs/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13C100
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] IntelliJ IDEA Ultimate Edition (version 2021.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 65.1.4
    • Dart plugin version 213.7228

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.29.0

[✓] Connected device (3 available)
    • Darshan's iphone (mobile) • 21150b119064aecc249dfcfe05e259197461ce23 • ios
      • iOS 15.3.1 19D52
    • macOS (desktop)           • macos                                    •
      darwin-x64     • macOS 12.2.1 21D62 darwin-x64
    • Chrome (web)              • chrome                                   •
      web-javascript • Google Chrome 109.0.5414.119

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.
      
[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

@darshankawar darshankawar added has reproducible steps The issue has been confirmed reproducible and is ready to work on found in release: 3.22 Found to occur in 3.22 found in release: 3.23 Found to occur in 3.23 tool Affects the "flutter" command-line tool. See also t: labels. labels Jun 19, 2024
@auto-submit auto-submit bot closed this as completed in 8e4f704 Jun 20, 2024
@AngeloAvv
Copy link
Author

Thank you @andrewkolos @darshankawar

flutteractionsbot pushed a commit to flutteractionsbot/flutter that referenced this issue Jun 21, 2024
@andrewkolos
[CLI tool] in flutter test, consider --flavor when validating the…
9e7dc34 
… cached asset bundle (flutter#150461)

Fixes flutter#150296

**Context.** `flutter test` has its own code path for writing flutter app [assets](https://docs.flutter.dev/ui/assets/assets-and-images). flutter#132985 introduced [flavor-conditional asset bundling ](https://docs.flutter.dev/deployment/flavors#conditionally-bundling-assets-based-on-flavor), which lets users control which assets get bundled based on `--flavor`. `--flavor` is supported in `flutter test`.

**Bug and fix.** `--flavor` isn't considered when deciding whether we need to rebuild this asset bundle:

https://github.com/flutter/flutter/blob/5e448f4ce57723ac0792ae822ebac69df3188ba1/packages/flutter_tools/lib/src/commands/test.dart#L709

This PR address this by writing the value of `--flavor` to a file in the build directory and checking that when validating the cached asset bundle.
@andrewkolos andrewkolos mentioned this issue Jun 21, 2024
Open
5 tasks
@darshankawar darshankawar added the r: fixed Issue is closed as already fixed in a newer version label Jun 25, 2024
sigurdm pushed a commit to sigurdm/flutter that referenced this issue Jun 26, 2024
@andrewkolos @sigurdm
[CLI tool] in flutter test, consider --flavor when validating the…
b0afd20 
… cached asset bundle (flutter#150461)

Fixes flutter#150296

**Context.** `flutter test` has its own code path for writing flutter app [assets](https://docs.flutter.dev/ui/assets/assets-and-images). flutter#132985 introduced [flavor-conditional asset bundling ](https://docs.flutter.dev/deployment/flavors#conditionally-bundling-assets-based-on-flavor), which lets users control which assets get bundled based on `--flavor`. `--flavor` is supported in `flutter test`.

**Bug and fix.** `--flavor` isn't considered when deciding whether we need to rebuild this asset bundle:

https://github.com/flutter/flutter/blob/5e448f4ce57723ac0792ae822ebac69df3188ba1/packages/flutter_tools/lib/src/commands/test.dart#L709

This PR address this by writing the value of `--flavor` to a file in the build directory and checking that when validating the cached asset bundle.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andrewkolos andrewkolos

Labels
found in release: 3.22 Found to occur in 3.22 found in release: 3.23 Found to occur in 3.23 has reproducible steps The issue has been confirmed reproducible and is ready to work on P1 High-priority issues at the top of the work list r: fixed Issue is closed as already fixed in a newer version team-tool Owned by Flutter Tool team tool Affects the "flutter" command-line tool. See also t: labels. triaged-tool Triaged by Flutter Tool team
Projects
None yet
Milestone
No milestone
3 participants
@AngeloAvv @andrewkolos @darshankawar