Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starfield support (PSA) #232

Open
hexabits opened this issue Aug 31, 2023 · 58 comments
Open

Starfield support (PSA) #232

hexabits opened this issue Aug 31, 2023 · 58 comments

Comments

@hexabits
Copy link
Member

hexabits commented Aug 31, 2023

I'm making this issue primarily as a PSA to anyone hoping for or expecting quick turnaround time for Starfield support. TL;DR - I can quickly get files loading in NifSkope, but nothing would be visible in the viewport.

Starfield introduces a number of issues for existing tooling. The actual geometry is now external to the NIF. This happens in at least two ways:

BSWeakReferenceNode

A list of many NIF references via their BSResourceID (file hash, extension, dir hash), plus transforms to place them into the root scene. This can be dozens or hundreds of separate NIFs loaded in one file. This is similar to BSDistantObjectInstancedNode introduced in FO76.

BSGeometry

Holds no geometry itself, includes a 41 character filepath (some kind of hash) to a .mesh file which stores the geometry. BSGeometry can reference more than one .mesh (up to 4).


NifSkope and nifxml can be updated for the format changes. However, there will be no visible geometry in virtually all of the NIFs without a refactor to NifSkope's serialization, rendering, and data management.

// 010 Binary Template

typedef struct BSResourceID {
    uint fileHash <format=hex>;
    char extension[4];
    uint dirHash <format=hex>;
};
typedef struct UnknownWeafRefStruct {
    Matrix44 transform;
    BSResourceID resourceID;
    uint unkInt1;
    SSTRING material;
};

// BSWeakReferenceNode
typedef struct {
    NiNode node <open=true>;
    uint numWeakRefs;
    struct {
        BSResourceID resourceID;
        uint numTransforms;
        Matrix44 transforms[numTransforms] <optimize=false>;
        uint numUnk1;
        struct {
            uint unkInt1;
            uint dirHash;
            uint fileHash;
            string mat; // mat\0
        } unkMaterialStruct[numUnk1];
    } weakRef[numWeakRefs] <optimize=false>;
    uint unkInt;
    uint numUnk2;
    UnknownWeafRefStruct unkStruct[numUnk2] <optimize=false>;
} BSWeakReferenceNode <name="BSWeakReferenceNode">;

Making this block functional (i.e. visible) in NifSkope may take an overhaul to the rendering and scenegraph. The solution I foresee is adding an actual Scenegraph pane. This will hold many loaded NIFs, and selecting a NIF will make its data active in the Block List. The NifModel will almost certainly need to be unloaded because of how much memory it takes up so that there is still only one NifModel at a time. This means the geometry will have to be loaded and cached for non-active NIFs.

Such a refactor will also be beneficial to BSDistantObjectInstancedNode and things such as loading more than one BTO/BTR file (e.g. loading an entire map's terrain).

// 010 Binary Template
// BSGeometry
typedef struct {
    NiAVObject base;
    NiBound bound;
    if ( stream > 130 )
        float boundMinMax[6];
    Ref skin;
    Ref shaderProperty;
    Ref alphaProperty;
    if ( stream <= 155 ) {
        VertexDesc vertexDesc;
    }
    else {
        // Null-terminated array
        local int i <hidden=true> = 0;
        for (i = 0; i < 4; i++)
        {
            byte test <hidden=true>;
            if (test)
            {
                struct {
                    uint triSize;
                    uint numVerts;
                    uint flags;          // Often 64
                    uint meshNameLength; // Always(?) 41
                    char meshName[meshNameLength] <optimize=false>;
                } meshStruct <optimize=false, open=true>;
            }
        }
    }
} BSGeometry <name="BSGeometry", open=true>; // Abstract (bsver 155 and below)

The null-terminated array is awful but that is how they implemented it. It is thankfully encapsulable in nifxml.

For rendering, another shape class will need to be introduced, since the others are heavily coupled with NifModel. Though a workaround in lieu of visible meshes would be to provide actions to export/import the .mesh to/from another format for editing in Blender etc.


Other bookkeeping:

Dummy bones (flat root bones) in NIFs seem to have been replaced by:

// 010 Binary Template
// SkinAttach
typedef struct {
    NiExtraData data;
    uint numBones;
    SSTRING bone[numBones] <optimize=false>;
} SkinAttach;

Adjusting bones via dummy bones has been replaced by:

typedef struct {
    NiExtraData data;
    uint numTranslations;
    struct {
        SSTRING boneName;
        Vec3 translation;
    } boneTranslations[numTranslations];
} BoneTranslations;
@hexabits
Copy link
Member Author

hexabits commented Sep 5, 2023

All files in all Meshes BA2 reading correctly. This doesn't mean much until .mesh unknowns are figured out. It is fully readable but there are buffers with unknown uses.

image

@hexabits
Copy link
Member Author

hexabits commented Sep 6, 2023

image

I have .mesh loading rudimentarily into a new Shape class, but we're still figuring out .mesh unknowns.

@obeyandre
Copy link

https://github.com/fo76utils/ce2utils/blob/main/src/meshfile.cpp

@DigitArt89
Copy link

DigitArt89 commented Sep 14, 2023

4
1
2
3
I also managed to decode the geometry. but I still don’t understand how TBNs are encrypted. And the order of reading the data is not the same as the order of reading the vertex data
in 4 bytes the vectors x y z are transmitted.
as I understand it, these are vectors converted to RGBA

@DigitArt89
Copy link

DigitArt89 commented Sep 14, 2023

and so, I managed to extract TBN using the recent Zenimax TES Online project. TBN in File.mesh are shaded in a two-dimensional coordinate system in the same way as it is done in GR2 2byte per vector.
00 40 00 00 02 CC 00 C0 23 81 00 C0 (Half_Float Type) just in File.mesh it is encrypted in the Short_Signed type which needs to be converted to Float
6

I really don’t have any idea about the two-dimensional calculation method, maybe someone knows how to convert 3 into 2 vectors or how it works in general?

@Jarmanprops
Copy link

Doing God's work as per usual fellas and it is much appreciated!

Any ballpark idea on when we might have an updated NifSkope to open these assets?

@hexabits
Copy link
Member Author

hexabits commented Sep 15, 2023

and so, I managed to extract TBN using the recent Zenimax TES Online project. TBN in File.mesh are shaded in a two-dimensional coordinate system in the same way as it is done in GR2 2byte per vector. 00 40 00 00 02 CC 00 C0 23 81 00 C0 (Half_Float Type) just in File.mesh it is encrypted in the Short_Signed type which needs to be converted to Float 6

I really don’t have any idea about the two-dimensional calculation method, maybe someone knows how to convert 3 into 2 vectors or how it works in general?

TBN are R10G10B10A2, the sign of the bitangent is two of the bits in the leftover 4 bits between the two buffers.

Anyway, I've been slowly pushing things to my branch. I haven't pushed the new Shape class for rendering .mesh, I'm still working on gltf import/export.

@hexabits
Copy link
Member Author

https://github.com/fo76utils/ce2utils/blob/main/src/meshfile.cpp

As of this comment this decoding is still incomplete FYI. I provided them the full .mesh but it doesn't look to be implemented in their code yet.

@DigitArt89
Copy link

и вот мне удалось извлечь TBN с помощью недавнего проекта Zenimax TES Online. TBN в File.mesh заштрихованы в двумерной системе координат так же, как это сделано в GR2 по 2 байта на вектор. 00 40 00 00 02 CC 00 C0 23 81 00 C0 (Тип Half_Float) просто в File.mesh он зашифрован в типе Short_Signed, который нужно преобразовать в Float6
Я правда понятия не имею о двумерном методе расчета, может быть кто-нибудь знает, как преобразовать 3 в 2 вектора или как это вообще работает?

TBN — это R10G10B10A2, знак битангенса — это два бита в оставшихся 4 битах между двумя буферами.

В любом случае, я медленно продвигал вещи в свою ветку. Я еще не внедрил новый класс Shape для рендеринга .mesh, я все еще работаю над импортом/экспортом gltf.

how to convert RGBA to float for vector values?

@hexabits
Copy link
Member Author

hexabits commented Sep 19, 2023

OK I have basic gltf export working with the NiNode structure intact (allows transforming meshes without affecting their vertex data, which then eases re-import):

image

But I may still take a day or two to sort out:

  1. Mesh LODs. I can export these as separate meshes in gltf, but I realized that for large NIFs, all the LOD would be visible on import and look very ugly, and be annoying to hide every non-LOD0 mesh. I need to figure out what is best for user workflows while still exporting LOD.
  2. Weights and joints for skinned meshes. I should have all the data necessary to do so, I just haven't tackled it yet.
  3. Using gltf extras to encompass arbitrary NIF data, theoretically a NIF could be entirely encapsulated by the gltf format. For example, the gltf could contain data to recreate arbitrary NiObjects like BSXFlags on gltf import. This would mean gltf could create fully functional NIFs when importing into NifSkope.
  4. Importing. I only have this half implemented at the moment.

@Caseter
Copy link

Caseter commented Sep 19, 2023

OK I have basic gltf export working with the NiNode structure intact (allows transforming meshes without affecting their vertex data, which then eases re-import):

image

But I may still take a day or two to sort out:

  1. Mesh LODs. I can export these as separate meshes in gltf, but I realized that for large NIFs, all the LOD would be visible on import and look very ugly, and be annoying to hide every non-LOD0 mesh. I need to figure out what is best for user workflows while still exporting LOD.
  2. Weights and joints for skinned meshes. I should have all the data necessary to do so, I just haven't tackled it yet.
  3. Using gltf extras to encompass arbitrary NIF data, theoretically a NIF could be entirely encapsulated by the gltf format. For example, the gltf could contain data to recreate arbitrary NiObjects like BSXFlags on gltf import. This would mean gltf could create fully functional NIFs when importing into NifSkope.
  4. Importing. I only have this half implemented at the moment.

Would you be willing to share this verion at all? I know it's not fully functional for proper use, but if you've got nif files exporting/importing to Blender thats all some of us need to get started playing around with textures in Substance Painter.

Thanks for all the work either way.

@hexabits
Copy link
Member Author

Would you be willing to share this verion at all? I know it's not fully functional for proper use, but if you've got nif files exporting/importing to Blender thats all some of us need to get started playing around with textures in Substance Painter.

Thanks for all the work either way.

@Caseter

I understand the feeling believe me but the longer NifSkope goes without a second update after this hypothetical update, the more people are exporting gltf that aren't fully complete (such as the LOD and gltf extras I mentioned) and the less complete the data will be on import back into NIF. So I'm trying to shove as much data into gltf using gltf extras as I can so that people don't have to keep re-exporting to gltf to get more complete data later.

I will try to get a release out within the next 48 hours regardless, I know this has been a long process and I apologize. Last weekend I was on vacation for several days and then I've been slow on progress this week so I know people are getting antsy.

Also I haven't yet integrated DirectXMesh lib so the end of .mesh can't be generated yet, I am doing that as soon as I make export a little more complete. If it ends up taking too long, I'm just going to not allow import for the initial release.

@DuckersMcQuack
Copy link

Would you be willing to share this verion at all? I know it's not fully functional for proper use, but if you've got nif files exporting/importing to Blender thats all some of us need to get started playing around with textures in Substance Painter.
Thanks for all the work either way.

@Caseter

I understand the feeling believe me but the longer NifSkope goes without a second update after this hypothetical update, the more people are exporting gltf that aren't fully complete (such as the LOD and gltf extras I mentioned) and the less complete the data will be on import back into NIF. So I'm trying to shove as much data into gltf using gltf extras as I can so that people don't have to keep re-exporting to gltf to get more complete data later.

I will try to get a release out within the next 48 hours regardless, I know this has been a long process and I apologize. Last weekend I was on vacation for several days and then I've been slow on progress this week so I know people are getting antsy.

Also I haven't yet integrated DirectXMesh lib so the end of .mesh can't be generated yet, I am doing that as soon as I make export a little more complete. If it ends up taking too long, I'm just going to not allow import for the initial release.

Is your accomplishment good enough to export X mesh as a stl? As i really want to have a few models/meshes 3d printed. And simply just a complete model as vertices is enough for me :P

@hexabits
Copy link
Member Author

hexabits commented Sep 22, 2023

I've sorted all the skinned mesh issues on export, so I'm trying to prepare a release. The skinning had issue after issue and took longer than expected so I've decided to release with export-only for now and work on import after initial release.

image

2023-09-21.20-08-24.mp4
2023-09-21.20-52-44.mp4
2023-09-21.23-44-43.mp4

Also, the later two videos show an LOD helper script written by Inaki for this.

@hexabits
Copy link
Member Author

hexabits commented Sep 22, 2023

Also I will document this with a guide/etc. but FYI stuff rigged on the human actors will need to copy/paste the COM branch of the human skeleton.nif into the mesh NIF before export. At least for the time being until I automate this. The creatures have a COM_Twin skeleton inside them and do not need this step.

@hexabits
Copy link
Member Author

I've put up a very early release, please read README_GLTF at least. https://github.com/hexabits/nifskope/releases

@DuckersMcQuack
Copy link

For the ranger badge, how did beth deal with that one? Does it have a complete model somewhere? Or is it literally just bumpmap textures that makes it 3d in-game? And do you guys know the technique to align the normals properly to make out the rest of the "model" that can be made solid and export as stl for instance? O tried all sorts of unwrappings yesterday, but neither made it "complete"

@liguokai527
Copy link

Thank you, sir, for your hard work. This is fantastic! Finally, we can view and export models in Nifskope, which is of great significance for MOD authors in texture redesign tasks. It's great. However, I'd like to inquire if there is currently a way to modify the models. Simply modifying textures has become tiresome for people. We urgently need a convenient method to modify models in the game and export them to the NIF format with .mesh files in the correctly named folders, facilitating model replacement work for mod authors. I believe this is what players and mod authors all around the world have been eagerly anticipating day and night. Thanks again for your efforts, and I look forward to seeing a version of Nifskope soon that allows modification and import/export of models.

@NerdSneeze
Copy link

Hello! First I would like to thank everyone who has and is currently working on Nifskope for all of your hard work. You guys have made it possible for me to finally put my foot in the door for a career in something I truly enjoy. Secondly I would like to state that I am having issues with the current version of Nifskope and I apologize if this has been stated before but I'm not familiar with the way this website works. For Starfield it seems that about 75% of the meshes are visible when I open the file but the files that I'm looking to edit are seemingly invisible. When I open the naked_f.nif everything seems to be working correctly, but when I open the naked_m.nif nothing shows up. Is this issue something I am doing wrong or is this something the developers are currently working to fix?

@GabGone
Copy link

GabGone commented Sep 26, 2023

Hi there, Sorry if i write here but discord channel invitation is not working anymore, and the readme seciton of troubleshooting is empty... I've downloaded the software, then I added Starfield data folder, but when I try to add archives they are just being ignored (while fallout 4 ones are all listed visible) and when I extracted the geometry folder and put it in data, loading nif files still gives issues. What am I doing wrong? Thanks for attention and for the modding tools guys!

---EDIT----

Problem solved. I'm an idiot. I downloaded the latest version, then unzipped the old one. Sorry for bothering you, I'll keep the message just in case there is some other genius like me around. And I kinda hope so... would make me feel better not being alone :/

@Coverop
Copy link

Coverop commented Sep 28, 2023

I'm having problem with importing outfits upperbodies
I have followed the instruction of copying Skeleton NiNode and paste It on the cloth model... but upperbodies meshes refuse to be imported in Blender.

I'm using Blender 3.4

image
image

I tried to tweak import settings but no luck.
Help?

@GabGone
Copy link

GabGone commented Sep 28, 2023

So, I did a few test before answering.
Ecliptig legs, they work.
Marine armor plates addon, it works.
UC Navy pouches addon, it works.
Constellation body, it works.
UC police works.
Neon security body, it doesn't work.
Try with some other meshes just to make sure your process is right. The above mentioned, for instance: if they work for me, they should work for you too. I'm on blender 3.6

@hexabits
Copy link
Member Author

hexabits commented Sep 29, 2023

@Coverop @GabGone Sorry, I've covered it on multiple Discord servers but failed to really mention it on GitHub, as I found out about a while after release. Currently stuff with Havok Cloth bones doesn't export correctly. These bones do not exist in the COM skeleton because they likely get added by Havok at runtime.

The only info I have is the bone index/weight plus the inverse bone matrix for the Havok bone which isn't enough to reconstruct it in the full armature in Blender. I am currently finding the best way to fail on these meshes still. They will simply render incorrectly in Blender because I can't put them in the correct place in the armature, but the underlying data should all be fine for re-import.

@hexabits
Copy link
Member Author

Also, there was a secondary issue with Bethesda including close to 0.0 vertex weights where they should have been 0.0, and I've also figured this out now. Everyone should just not import skinned meshes until I update the release. Due to the way glTF works with joints and weights, this ends up messing up a few of the weights in the first vertex group, which was also the reason Guess Original Bind Pose had to be unchecked. You should no longer have to do this after I update the release.

@GabGone
Copy link

GabGone commented Sep 29, 2023

Thanks for the answer. I've been looking for a discord page about nifskope, but I've only found expired invitations. Would you please post a working link?

@hexabits
Copy link
Member Author

Thanks for the answer. I've been looking for a discord page about nifskope, but I've only found expired invitations. Would you please post a working link?

There is no official server anymore, I closed it years ago. I did remove the stuff that no longer exists from the app and readmes. Did I miss something somewhere? (I'm aware the old niftools page still has a Discord widget though)

@hexabits
Copy link
Member Author

I have updated the release (same link): https://github.com/hexabits/nifskope/releases/tag/v2.0.dev9

Please read the 2023-09-29 notes.

@Protoferium
Copy link

Hello!
I am having a small issue, when i export a model into nifskope it appears as invisible? and when i export it into blender it only is the bones, its likely a problem on my end but i wanted to see if anyone else had this problem?

@GabGone
Copy link

GabGone commented Oct 4, 2023

Latest version was working fine on me. Can you please tell us which niff is giving you the issue? So far, I can confirm we're able to:

  • copy branches of niff to make mash ups, but it seems that you can't move/scale branches (still testing) on armors
  • copy branches from one niff to another to make mash ups on weapons. Scales and transformations on XYZ are saved
  • export to blender, by following hexabits instruction, should work. I'm gonna try again this evening and let you know.

@Protoferium
Copy link

Latest version was working fine on me. Can you please tell us which niff is giving you the issue? So far, I can confirm we're able to:

  • copy branches of niff to make mash ups, but it seems that you can't move/scale branches (still testing) on armors
  • copy branches from one niff to another to make mash ups on weapons. Scales and transformations on XYZ are saved
  • export to blender, by following hexabits instruction, should work. I'm gonna try again this evening and let you know.

I am having issue with a large portion of the file "Starfield - Meshes01.ba2" (i have tried the laser turret, starborn helmets, and Armillary) when i import them the scene resets but doesn't show the model, however when i do click "0.Ninode" it dose show the models rig, both for models from the archive and BAE.

I am using 2.0 dev9 that is within the "NifSkope_2_0_2023-09-29-x64" file.

I do not know if it effects anything, but I have both nifskope and starfield on a external SSD.

@hexabits
Copy link
Member Author

hexabits commented Oct 5, 2023

Did you add

Starfield - FaceMeshes
Starfield - Meshes01
Starfield - Meshes02
Starfield - MeshesPatch 

BA2s to your Resources > Archives tab?

Also, I recommend putting MeshesPatch at the top of the archive list, so it's loaded from first.

@Protoferium
Copy link

Did you add

Starfield - FaceMeshes
Starfield - Meshes01
Starfield - Meshes02
Starfield - MeshesPatch 

BA2s to your Resources > Archives tab?

Also, I recommend putting MeshesPatch at the top of the archive list, so it's loaded from first.

I didn't, and when i did it worked! thank you =D

@bartrv
Copy link

bartrv commented Oct 5, 2023

Thank you so much for your work on this!
TLDR: 1. Are you aware of a reliable NifScope -> 3dsMax->nif pipeline? 2. would the FBX SDK cause more problems than it would solve?
...
I've been trying to get a 3ds Max workflow ironed out. I can import from the ba2's, export to glTF, import into Blender, Export as FBX, import into Max, export as .Nif. Upon reloading the exported nif into nifscope the skeleton is disassociated with the mesh and the reference export/root nodes are missing... the geometry (all 3 LOD) seem to re-import fine, aside from the scale, though I think that is a blender export setting.
image

Are you aware of a better pipeline? (I'm using 2021 - no glTF i/o)
Have you looked into the Autodesk fbx SDK? It's supported by 3ds Max, Maya, Blender, and Lightwave. The docs claim it is able to handle the nodes that gltF doesn't natively support, also it claims a built in collada and OBJ i/o component.
(VS2017-2022)https://www.autodesk.com/developer-network/platform-technologies/fbx-sdk-2020-3-4
or
(VS2012-2017)https://www.autodesk.com/developer-network/platform-technologies/fbx-sdk-2020-0

I'm willing to be a test subject for 3ds Max i/o if need be.

@GabGone
Copy link

GabGone commented Oct 6, 2023

Hi there. ATM, there is a working blender plugin on the nexus
https://www.nexusmods.com/starfield/mods/4360

Using this in combo with nifskope should work.

I'm gonna test tonight and tomorrow again, I'll tell the workflow I'm using.

  1. Open niff in nifskope, see which geometry I want to edit
  2. Open the geometry in Blender, edit, export (in the tool menu, not from "File"
  3. Rename the geometry, so you do not overwrite anything. I usually change the latest two hex number, increasing
  4. Change the geometry in the nifskope line, writing the new file name, and save it
  5. Close nifskope, copy & paste folder in starfield installation data\geometry[correct path] (nifskope do not see documents)
  6. Open again the nif in nifskope, now you should see the edited geometry loaded
  7. DO NOT MIX DIFFERENT PART FROM DIFFERENT NIFs IN ONE GEOMETRY, if you want to do mashups, do it in nifskope copying branches, or it will fuck up all body weights.

Let me explain, you you want to make a new nif with marine spacesuit arms and security armor chest, first you edit the arm, you can import them together so you can see clipping to fix, etc, but when you export, export two different geometries, load and save two different nifs, then copy one branch into another nif and all will be fine. If you export arms and body in the geometry of the body, chest body weights will be fine, while arms body weight will be all wrong.

  1. Now that you have exported your mashup, remove the geometry folder from installation folder, it appears it crashes the game, at least to me
  2. open the game, you should be able to see your new mesh.

I've already made my first new mashup last night, still have to publish it because I want to do a minor fix, but this workflow should be fine.

@Gilibran
Copy link

Gilibran commented Oct 10, 2023

Could it be that there are still parts (values?) NIFskope cant identify and therefore they dont showup under the 0Ninode or child list?

I'm trying to figure out ship snappoints but cant find anything in the NIF that points to snappoints!?

What I did find out, I flipped a ship module in NIFskope and replaced an existing shipmodule with this flipped NIF.

What happened was it showed up in game like i wanted but inherited the snappoints of the NIF I replaced. So the NIF i edited originally had snappoints aft and top, after replacing it had the snappoints fore and aft like the module i replaced.

This does make me question if the snappoints are in the NIF like they are in FO4, I watched Kingaths tutorial on FO4 snappoints and howto add them using NIFskope to see if that gave me a lead!?

I did find a number of markers related to snappoints \meshes\markers\shipmarkers: shipsnapnodemarker_bottom01 / shipsnapbehaviorhelper01 etc.

Again though no link between these and the NIF of the modules!?

I'm new to this so I am learning as I go.

Thanks in advance

P.s. using NIFskope 2.0 Dev 9

@hexabits
Copy link
Member Author

I'm trying to figure out ship snappoints but cant find anything in the NIF that points to snappoints!?

Because they're no longer in the NIFs, they are in the ESM.

@Gilibran
Copy link

I'm trying to figure out ship snappoints but cant find anything in the NIF that points to snappoints!?

Because they're no longer in the NIFs, they are in the ESM.

And we cant do much with the ESM yet :-(

I'll get the ESM explorer I saw on Nexus tomorrow, see if I can at least locate them

:-) Thank you very much for this info

@luxzg
Copy link

luxzg commented Nov 13, 2023

Sorry for bumping this thread, but I have no idea where else to ask. Is there any plans to support seeing textures in NifSkope for Starfield? I use NifSkope for finding nice models, do a quick preview, do placeholders for snaps, but I really miss seeing textures. Some models look great in theory, but once I see them in game they don't fit my idea, so waste of 30 minutes. If anyone has a timeline, workaround, or any pointers at all please share. If it involves exporting to Blender that's fine, it's still quicker than adding to game.

@fo76utils
Copy link

For now, there is a fork here that can display Starfield textures with some limitations (only the first layer of the material is rendered), and it makes a few fixes/improvements to Fallout 76 rendering as well. It also shows detailed material properties like texture paths, although editing them is not possible, only material paths (BSLightingShaderProperty block names) can be changed.

@luxzg
Copy link

luxzg commented Nov 15, 2023

For now, there is a fork here that can display Starfield textures with some limitations (only the first layer of the material is rendered), and it makes a few fixes/improvements to Fallout 76 rendering as well. It also shows detailed material properties like texture paths, although editing them is not possible, only material paths (BSLightingShaderProperty block names) can be changed.

This sounds as enough for my needs. But... is it possible to provide a ZIP with binaries? It had been years since I compiled anything, and all I see is source. Would be most grateful! Doesn't have to be proper github release, any downloadable ZIP is fine

@fo76utils
Copy link

This sounds as enough for my needs. But... is it possible to provide a ZIP with binaries?

Binary packages are generated automatically by GitHub on any update to the source code. If you open the most recent successful workflow run under actions, they are available as "artifacts" (build-linux and build-win) in .zip format. Downloading those files requires signing in to GitHub.

@luxzg
Copy link

luxzg commented Nov 15, 2023

This sounds as enough for my needs. But... is it possible to provide a ZIP with binaries?

Binary packages are generated automatically by GitHub on any update to the source code. If you open the most recent successful workflow run under actions, they are available as "artifacts" (build-linux and build-win) in .zip format. Downloading those files requires signing in to GitHub.

Ah ok, found them, didn't know that even existed. Anyway, I get error about shaders on start. I open NIF and can see eg connection points but no model displayed (I can see data though).I've tried disabling shaders, no error in that csae on startup, but still don't see the model either. Seems it pulled all settings from dev version posted here earlier, one I've been using so far. Sorry if this isn't helpful :-/ If you point me in the right direction I can try to debug it and get you logs or whatever. W11 Pro btw.

@fo76utils
Copy link

Anyway, I get error about shaders on start.

Can you post the shader compilation error message?

@luxzg
Copy link

luxzg commented Nov 16, 2023

Anyway, I get error about shaders on start.

Can you post the shader compilation error message?

Sure:

There were errors during shader compilation

stf_default.frag:

ERROR: 0:446: 'variable indexing sampler array' : not supported with this profile: none
ERROR: 0:446: '' : compilation terminated 
ERROR: 2 compilation errors.  No code generated.


stf_default.prog:

depends on shader stf_default.frag which was not compiled successful

I was adding "Connection points" in the dev9, and that renders fine (but without textures) on the version from post above.

In your version it looks like this (after dismissing shader error):

image

@fo76utils
Copy link

Does it fix the error if you replace this line at the beginning of stf_default.frag:
#version 130
with one of these, either:
#version 150 compatibility
or:
#version 400 compatibility

@luxzg
Copy link

luxzg commented Nov 16, 2023

Does it fix the error if you replace this line at the beginning of stf_default.frag: #version 130 with one of these, either: #version 150 compatibility or: #version 400 compatibility

Yes, both get rid of the shader error. But I still don't see the mesh or anything in the preview window, only little green dots where the connection points should be.

@fo76utils
Copy link

Does the mesh disappear if you use the modified stf_default.frag file with the original dev9 build of NifSkope (or vice versa, the old dev9 shader with my fork)?

@luxzg
Copy link

luxzg commented Nov 17, 2023

When I move your file to original dev9 build (I moved only stf_default.frag), then model is rendered in dark grey, while usually it's almost white.

But when I move original dev9 version of stf_default.frag to your build, I still don't see the model.

Can I do anything else? And does the change of color indicate anything?

@fo76utils
Copy link

fo76utils commented Nov 17, 2023

I can reproduce the issue by running on an AMD GPU (instead of Nvidia) with the open source drivers. In fact, on Linux, even the original dev9 version freezes now on trying to render anything, but the Windows build works with Wine and has the same problems as you reported (it does display Fallout 76 models, however). Now that I can reproduce the issue, I will look into fixing it, and ideally the Linux AMD driver freeze as well that is not specific to my fork. Edit: the latter was fixed by reducing anti-aliasing, so it was not necessarily a NifSkope bug.

@fo76utils
Copy link

There is an updated build available now that tries to work around the issue.

@luxzg
Copy link

luxzg commented Nov 18, 2023

There is an updated build available now that tries to work around the issue.

Sorry, no luck. I don't get the shader error anymore, but don't see the actual model in the view.

I have AMD CPU 5800H and Nvidia RTX3060 (laptop). September build of dev9 works fine, but without textures. But in yours I simply don't get model to show at all. Windows 11 with latest patches, if it helps at all.

Are you using Linux to run NifSkope? IDK, maybe if it works for you I can spin up a VM. Just let me know which OS you use so I can install something that's as close to what you have.

@fo76utils
Copy link

fo76utils commented Nov 18, 2023

But in yours I simply don't get model to show at all.

Are models from Fallout 76 or other older games visible?

Are you using Linux to run NifSkope?

I use mainly that for development, and previously tested on an Nvidia desktop GPU, which works on both Linux and Windows. With AMD integrated video, the updated build works correctly on Linux native (the previous version did have the shader compilation error and invisible models). The Windows version shows Starfield models on both native Windows and with Wine, although it does have different issues (corrupted textures in all games on native, and broken normals with Wine).

Edit: the texture bug on Windows is now patched.

@luxzg
Copy link

luxzg commented Nov 18, 2023

Thanks, I will test on Monday as I'm currently traveling and can't access my PC. Also, don't have any models to test right now except Starfield :-/ Edit: Sorry, forgot to ask which Linux OS/version you used?

@luxzg
Copy link

luxzg commented Nov 21, 2023

Edit: the texture bug on Windows is now patched.

OK, I'm probably doing something messy, now I was running through settings, and now NifSkope just crashes (force close, no error on screen) when I try to open Starfield NIF. "dev 9" from September still works. I need to try and clean my installation and settings, and see if it works after that. Btw, do I have to enable anything special in settings for textures to show up? Or should the defaults be fine? That's what I was looking into when suddenly it started crashing on anything.

EDIT: OIK, erased registry, extracted just your build, opens now without crash, textures and do skinning are on, but it still crashes. Back to dev9 from hex.

EDIT: Also, your "audo detect archives" does not work, while one from hex works fine. Wouldn't notice otherwise, but having to reinstall NifSkope and setup path to Starfield I noticed it didn't work, now I was setting up hex's build and that one works fine.

@fo76utils
Copy link

OK, I'm probably doing something messy, now I was running through settings, and now NifSkope just crashes (force close, no error on screen) when I try to open Starfield NIF.

Does the previous build (the one with the commit message "Restore pixel storage mode...") also crash?

Also, your "audo detect archives" does not work

I cannot reproduce this issue, but in any case having Games and Paths (the first two tabs under resources) correctly set up should be enough.

@luxzg
Copy link

luxzg commented Nov 23, 2023

Does the previous build (the one with the commit message "Restore pixel storage mode...") also crash?
I cannot reproduce this issue, but in any case having Games and Paths (the first two tabs under resources) correctly set up should be enough.

Sorry, I was really busy finishing my last mod update. Anyway, yes, that version crashes, and todays' version from 6h ago crashes. Version from 6h ago worked, but then I opened Settings, and added game path (auto-detect worked this time) and the archives (also worked autodetecting). After saving these settings it started crashing. dev9 build from september still works, so I keep using that.

Thanks for your efforts, too bad I'm having issues with your builds :(

@fo76utils
Copy link

I can reproduce the crash now, it seems to be caused by changes that have been made to the material database format in the latest update to the game. While I am not sure if I can fix the invisible models that seem to occur with certain GPUs/drivers, I will look into updating the material code over the weekend, particularly since that issue also affects tools like mat_info and nif_info in ce2utils.

@luxzg
Copy link

luxzg commented Nov 28, 2023

https://github.com/fo76utils/nifskope/releases

NifSkope_UazTbWhHKB

This works now!
I got the file here (for Windows):
https://github.com/fo76utils/nifskope/actions/runs/6997990181

Your ce2utils also work as a charm, if anyone needs a link:
https://github.com/fo76utils/ce2utils

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests