Skip to content

Home

Jump to bottom
Candoran2 edited this page Oct 28, 2021 · 25 revisions

NifTools XML format

This project is used to describe the Nif, Kf, and Kfm file formats used by Niftools projects. These XML files are central to our efforts to understand the file formats and create tools that support as many games as possible. The XML files also contain our human-readable documentation of the format, serves as the source for our HTML version of the file format specification.

Several Niftools projects use these format description internally to read and write files.

NifLib is a C++ library for reading and writing nif files. Python scripts read the XML file and generate some of the source code used by Niflib to read and write Nif files.

Pyffi is a pure python implementation that allows dynamic reading of the formats to expose a read/write library. Additional tools enable modification, sanitising, regex replacement and more on the files.

NifSkope uses these files directly using an internal file library to read the files. Almost any new discoveries about the format can be enabled in NifSkope by editing the nif.xml file included in its install directory.

The Basics

If you've never heard of XML before, you may want to check out a few of these informational links borrowed from Wikipedia:

If you've ever worked with HTML, XML will look very familiar to you. Basically, instead of working with tags like # <B>, <IMG>, and <TABLE>, you will be working with special tags designed to describe the Nif file format such as <basic>, <compound>, and <niobject>.

Every XML file starts with a basic heading. Following is the heading for the NifTools XML format. It doesn't really do anything important other than tell you that this document is in the NifTools XML format, and it has to be there:

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE niftoolsxml>
    <niftoolsxml version="0.7.0.0">
        'XML File contents go here'
    </niftoolsxml>

Also, like in HTML, plain text can appear between the start and end version of a tag. Similarly, plain text enclosed in NifTools XML tags contains the human readable documentation about what this Nif structure does. This is completely plain text... no HTML tags are allowed. Here's an example:

    <field name="Name" type="string">
        Name of this object. This is also the name of the action associated with this file.
        For instance, if the original Nif file is called `demon.nif` and this
        animation file contains an attack sequence, then the file would be called
        `demon_attack1.kf` and this field would contain the string `attack1`.
    </field>

There are also certain characters which are not allowed in XML plain text. These must be replaced with special codes. Some examples:

Character Replace with this
" &quot;
< &lt;
> &gt;
& &amp;

Some attributes may accept lists of things (for example integers or strings). In this case, they are space-separated. So an example of an attribute with a list of strings would look like this:

<sometag attr="string1 string2 string3" />

Organization

The Nif XML file is divided into sections based around the five main tags. These are; <version>, <basic>, <enum>, <bitflags>, <bitfield>, <compound>, and <niobject>. At the top of each section is an XML comment. Just like in HTML, a comment is text that is ignored by the program reading the XML file.

For example: <!--This text is for informational use and will be ignored by parsers-->

Tags

This section will introduce each tag and explain how to use it. Tags which appear inside other tags will be grouped under the tag which they are most likely to appear inside of

<niftoolsxml>

The <niftoolsxml> tag is the root tag, and everything else in the file goes inside it. There is exactly one of these tags in a NifTools XML document. This is similar to the way the <HTML> tag works for HTML documents.

Attributes:

Name Format Description
version #.#.#.# Used to specify the version of the NifTools XML format being used.

Tags Which can appear inside this one:

<version>, <module>, <token>, <basic>, <enum>, <bitflags>, <bitfield>, <compound>, and <niobject>

Example:

    <niftoolsxml version="0.7.0.0"></niftoolsxml>

<version>

The <version> tag is a hint to NifSkope about which versions of the Nif format we currently support. Basically, if you try to open a file of a version that doesn't have a tag, NifSkope will tell you that that version is not supported. Adding a version tag makes that error message go away so you can start figuring out any other changes you might have to make so that all the files from that new version number can be opened. For informational purposes, each version tag contains description text which lists the games that are known to use that version. When that <version> is the default for a game, that game will be surrounded by double curly brackets {{}}. For expansive explanation/discussion, see #69.

Attributes:

Name Format Description
id V#_#_#_#, but can be any text Unique identifier for this combination of version, user version and bsver. Generally speaking, this is V, followed by the num attribute (but with the dots replaced by underscores). If user version and bsver are specified for this tag, they may follow after a double underscore, or a game abbreviation if that is more appropriate.
num #.#.#.# The Nif version to be flagged as supported by NifSkope.
supported true/false False if the XML does not fully support reading of this version.
user list of integers User version for this version id.
bsver list of integers Bethesda-specific extra user version for this version id.
custom true/false If this attribute is true, it denotes that it is not an official Gamebryo version and has customization of one or more blocks. If user or bsver is specified, this is assumed to be true.
ext list of strings Any custom NIF extensions associated with this version (e.g. .bto and .btr files can both be read as nifs).

Tags Which can appear inside this one:

None.

Example:

    <version num="20.0.0.4">Civilization IV, Oblivion</version>

<module>

The <module> tag defines modules, which can be used for organizing various other tags. They do not express any functional differences, but may be used to organize generated code. For more discussion about the rationale behind modules, see issue 72.

Attributes:

Name Format Description
name text Name of the module.
priority integer Used for explicit ordering of the modules. Lower numbers are older/broader categories.
depends list of strings Modules which this module depends on.
custom true/false Whether blocks in this module are non-standard.

Tags which can appear inside this one

None.

Example:

    <module name="NiParticle" priority="50" depends="NiMain NiAnimation" />

<token>

The <token> tag can be used for simple text replacement on attributes within the XML, to make it more humanly readable. Tokens which use other tokens as their replacement string must be specified before the tokens they use, because these replacements are done sequentially - so the first tokens are evaluated and replaced, which puts the new tokens in, and only then are those new tokens evaluated and replaced.

Attributes:

Name Format Description
name text This is the name of this token. It needs to be used as the tag for the children of this token (rather than the standard tags).
attrs list of strings The attributes where this token is used.

Tags which can appear inside this one:

Whatever was used for the name atrribute of the token.

Example:

    <token name="operator" attrs="length width">
        <operator token="#ADD#" string="+" />
        <operator token="#SUB#" string="-" />
        <operator token="#MUL#" string="*" />
        <operator token="#DIV#" string="/" />
    </token>

<token name>

NOTE: token name IS NOT A REAL TAG. Rather, here in the documentation it is a stand-in for the name which you give a token, which determines what the tag is called that you can use inside the token.

Attributes:

Name Format Description
token text Text that this token will replace.
string text Text that it will be replaced by.

Tags which can appear inside this one:

None

Example:

    <operator token="#ADD#" string="+" />

<basic>

The <basic> tag specifies a new simple data type, one that holds just one piece of data. For example, a number, a string, etc. The main reason to add a new basic data type would be to allow NifSkope or Niflib to treat it differently. At this point most basic data types that you need should already have been created and it's probably best to ask if it's necessary to create a new one. You should know what the basic data types are, however, because you will refer to them often in other tags. For expansive explanation/discussion of the tags, see #76.

Attributes:

Name Format Description
name text This is the name of this basic data type. It is used to refer to this type in other tags.
boolean true/false Whether it is a data type that can be used as a boolean.
integral true/false This determines whether it is a data type that can be added (bit-wise, i.e. floats do not have integral=true).
countable true/false This determines whether a basic data type can be used as a count (e.g. for array size). Only data types that store unsigned integers can be used as a count. Is implied to be false if integral is false.
generic true/false Optional. Whether fields of this type use the template attribute. If not specified, defaults to false.
size integer How large this data type is in bytes.
convertible list of strings What other basic types this converts to without losing data.

Tags Which can appear inside this one:

None.

Example:

    <basic name="short" integral="true" countable="false" size="2" convertible="int int64">
        A signed 16-bit integer.
    </basic>

<enum>

The <enum> tag specifies a data type which consists of a list of options. Each <enum> tag will contain two or more <option> tags which specify the choices available in this collection. Each option corresponds to an integer value which is given a name and can have a description.

Attributes:

Name Format Description
name text This is the name of this enum data type. It is used to refer to this type in other tags and will be the name of the C++ enum in Niflib.
storage text This specifies the data type used to store the enum value in the Nif file. Generally, this will be uint, ushort, or byte. It must exist as the name of a <basic> tag somewhere else in the file.
module text Optional. Which module this belongs to.
prefix text Optional. Used to be able to separate out and access the values of the option of every enum option without the names conflicting.
ver1 version ID Optional. The first Nif file version ID where this variable appears. If absent, it is assumed that this variable appears in the earliest Nif files.
versions list of version IDs Optional. This specifies which versions this element can appear in.

Tags Which can appear inside this one:

<option>

Example:

    <enum name="ChoiceType" storage="uint">
        Specifies a type of choice.
        <option value="0" name="CHOICE_YES">An affirmative choice.</option>
        <option value="1" name="CHOICE_NO">A negative choice.</option>
        <option value="2" name="CHOICE_MAYBE">A noncommittal choice.</option>
    </enum>

<option>

The <option> tag describes a choice that is part of an enum data type. Enums store the choice as a number, so each <option> tag contains that number and a name so people can understand what the number means. It can also contain descriptive text explaining what the option is for. The name should be in all capital letters with spaces replaced by underscores(_) because it will be used as a constant name in Niflib.

Attributes:

Name Format Description
value integer This is the number that is stored in the Nif file when this choice is selected. There should be only one <option> tag per number.
name text This is the short name of this choice. It should be in all capital letters with no spaces and should start with an abbreviated form of the enum name. It must be unique; no choice name can be the same as any other choice name anywhere in the XML file.

Tags Which can appear inside this one:

None.

Example:

   <option value="0" name="ANIMAL_DRAGON">Indicates that this object is a dragon.</option>

<bitfield>

The <bitfield> tag denotes a data type spans 1 or more bytes, but which actually contains sub-fields, each of which can have their own data type and size (including sizes which are not whole bytes).

Attributes:

Name Format Description
name text This is the name of this bitfield data type. It is used to refer to this type in other tags.
storage text This specifies the data type used to store the value of the bitfield in the Nif file. Generally, this will be uint, ushort, or byte. It must exist as the name of a <basic> tag somewhere else in the file.
versions list of version IDs Optional. This specifies which versions this element can appear in.

Tags Which can appear inside this one:

<member>

Example:

    <bitfield name="TimeControllerFlags" storage="ushort">
        Flags for NiTimeController
        <member width="1" pos="0" mask="0x0001" name="Anim Type" type="AnimType" />
        <member width="2" pos="1" mask="0x0006" name="Cycle Type" type="CycleType" default="CYCLE_CLAMP" />
        <member width="1" pos="3" mask="0x0008" name="Active" type="bool" default="1" />
        <member width="1" pos="4" mask="0x0010" name="Play Backwards" type="bool" />
        <member width="1" pos="5" mask="0x0020" name="Manager Controlled" type="bool" />
        <member width="1" pos="6" mask="0x0040" name="Compute Scaled Time" type="bool" default="1" />
        <member width="1" pos="7" mask="0x0080" name="Forced Update" type="bool" />
    </bitfield>

<member>

The <member> tag is used to describe a part of a bitfield data type. It specifies exactly which bits of the bitfield belong to a specific sub-field, what that sub-field should be named and what kind of data are stored in there.

Attributes:

Name Format Description
width integer This is the number of bits that this member takes up in the bitfield.
pos integer This is the bit where this sub-field starts in the bitfield, counting from the right, where the right-most bit is bit 0.
mask hexadecimal, preceded by 0x This is a hexadecimal representation of all the bits in the bitfield belonging to this member. It is obtained from the binary representation of the same thing, where a 1 means that the bit in that location belongs to the member. The mask can also be inferred from the width and pos attributes.
name text This is the name of this member.
type text This is the name of the type which this member uses. This type can normally be larger than what would fit in the bits covered by this member. In the example below, the "CycleType" type is used in a member which covers only two bits, even though CycleType is normally stored as a uint (which takes up 32 bits).
default depends on type Optional. Default value of this option

Example:

    <member width="2" pos="1" mask="0x0006" name="Cycle Type" type="CycleType" default="CYCLE_CLAMP" />

<bitflags>

The <bitflags> tag can be seen in two different ways.

  1. It is specialization of an <enum> which allows for the type to hold multiple options instead of just one. The syntax is identical except for the tag name, so please refer to the above <enum> documentation. What differs is the semantics of the <option>, specifically the bit attribute. The integral value of each option is not the value but 2^bit.
  2. It is a specialization of the <bitfield>, which uses the <option> tag instead of member. width is implied to be 1 and type is implied to be "bool" for every option, and bit is the equivalent of the pos attribute.

Example:

    <!-- BitFlagsType can hold the value 0, 1, 2, 4 and also 3 (A+B), 5 (A+C), 6 (B+C), 7 (A+B+C)  -->
    <bitflags name="BitflagsType" storage="uint">
        Specifies a type of choice.
        <option bit="0" name="CHOICE_A" /> <!-- 2^0 = 1 -->
        <option bit="1" name="CHOICE_B" /> <!-- 2^1 = 2 -->
        <option bit="2" name="CHOICE_C" /> <!-- 2^2 = 4 -->
    </bitflags>

<compound>

The <compound> tag defines a new data type which contains several instances of other data types. Compounds are analogous to C/C++ structs or classes in that they contain several variables in a certain order. These other data types can be specified in preceding <basic>, <enum>, or other <compound> tags. Basically, a compound data type is like a list of data that will be found in a Nif file in the order given. It is useful to create a compound when a complex part of another data type is repeated many times, such as in an array. The compound tag should contain a plain text description in addition to any <field> tags to document its purpose.

Attributes:

Name Format Description
name text This is the name of this compound data type. It is used to refer to this type in other tags and will be used for the name of the C++ struct for Niflib. The first letter of each word should be capitalized and it should contain no spaces or underscores.
generic true/false Optional. Whether fields of this type use the template attribute. If not specified, defaults to false.
size integer Optional. Size of this compound in bytes, only used if that remains constant in all situations.
convertible list of other block names Optional. Much like the convertible in the basic tag, this lists which types this type can be converted to without losing information (e.g. ByteVector3 can be converted to Vector3).
versions list of version IDs Optional. This specifies which versions this element can appear in.
ver1 version ID Optional. The first Nif file version ID where this variable appears. If absent, it is assumed that this variable appears in the earliest Nif files.
ver2 version ID Optional. The last Nif file version that this Nif object is known to appear in. If absent, it is assumed that this Nif object can appear in the most recent Nif files.
module text Optional. Which module this belongs to.

Tags Which can appear inside this one:

<field>

Example:

    <compound name="LODRange">
        The distance range where a specific level of detail applies.
        <field name="Near Extent" type="float">Beginning of range.</field>
        <field name="Far Extent" type="float">End of Range.</field>
    </compound>

<field>

The <field> tag is used to add a variable to a <compound> or <niobject> tag. Like a declaration within a class or struct in C++, this new variable has a name and a storage type. A description should also be added between the beginning and end tag to describe the purpose of this attribute. These tags are considered sequentially and should be placed in the same order that these pieces of data appear in a Nif file.

The <field> tag also has a few extra features that make it a bit more complex than other tags. Not only can it be used to specify a single piece of data, but it can be used to define a 1 or 2 dimensional array of the same data as well. The size of these arrays can be fixed, or can come from a preceding variable by name. Most programmers will recognize this concept.

A variable can also have a template type specified. A template is a way, in C++, to have a single generic set of code apply to many different types. The biggest example of this in the Nif format is the Ref type, which points to another object somewhere else in the file. It's important to know what kind of object can be referenced here, so in addition to specifying the type, Ref, we also specify the template, say "NiNode." There are few other examples of templates, and you probably won't have to worry too much about them.

The Nif format also changes with time. Some variables disappear or move to another location, while new ones crop up all the time, often between existing ones. To take care of this, the field tag has attributes to specify the beginning and end version where that variable appears. Also, if you add a variable with the same name and type, but different version spans, it is considered to be the same variable for the purposes of Niflib. Only the order it is written to the file changes.

There are also many times when the existence of one variable is conditional on the value of a preceding one. To allow for this, a syntax using boolean operators can refer back to previous variables by name. If the condition ends up true, the variable is read or written from the file, if not it isn't.

Finally, to make things easier on everyone, it is often a good idea to have default settings. This way, new objects created with Niflib or NifSkope can be used right away without lengthy, repetitive setup routines. The field tag allows for this as well.

Attributes:

Name Format Description
name text This is the name of the variable/attribute to add. It is used to refer to this variable in subsequent tags and will be used for the name of the C++ struct for Niflib. The first letter of each word should be capitalized and separated by spaces.
suffix text Optional. Used in order to (be able to) separate fields in the same block with the same names, but different types.
type text This refers to the name of a <basic>, <compound>, <enum>, or <bitflags> tag. It is the data type that will be used for this variable/attribute and determines the size of the data on the disk as well as the operations that will be available from either NifSkope or Niflib. If this is part of a template compound, the type “TEMPLATE” can be used to make this variable take on the dynamic type specified in the variable that creates a template.
arg expression Optional. This passes the value of the expression down to whatever compound this field is, which can then use #ARG# in any of its evaluated attributes (cond, arr1, arr2 and even arg itself) to access the value.
calc expression Optional. Hint on how to calculate this field using other fields in the same block.
arr1 integer/var name This is the optional first array dimension for this variable/attribute. It can either be an integer or the name of a preceding count type variable.
arr2 integer/var name This is the optional second array dimension for this variable/attribute. arr1 must be specified for arr2 to be given a value. It can either be an integer or the name of a preceding count type variable.
template type name This is the optional template type for this variable/attribute. It can be the name of a <basic>, <enum>, <bitfield>, <bitflags>, <compound>, or <niobject> tag.
cond expression Optional. If this expression evaluates to true, this data is assumed to be present in the file, otherwise it is not. The expressions follow C/C++ format and can include names of preceding variables.
ver1 version number Optional. The first Nif file version where this variable appears. If absent, it is assumed that this variable appears in the earliest Nif files.
ver2 version number Optional. The last Nif file version where this variable appears. If absent, it is assumed that this variable appears in the most recent Nif files.
vercond expression Optional. Conditional expression using only global variables (e.g. for the nif format, version, user version and bs version) and logical operators. If this expression evaluates to true, this data is assumed to be present in the file, otherwise it is not.
binary true/false Optional. Whether this type (typically an array of bytes) should be treated as a binary data blob. Implies each array member should not be treated individually (e.g. editor UI).
range value:value Optional. If this field is a number, which range is valid. Formatted as min_value:max_value.
excludeT name of block type Optional. Means this field is not present if the owning block is this type.
onlyT name of block type Optional. Means this field is only present if the owning block is this type.

Tags Which can appear inside this one:

<default>

Example:

    <field name="Triangles" type="Triangle" arr1="Num Triangles" cond="Num Strips == 0">
        The triangles.
    </field>

<default>

The default tag is used to modify the default values of a field based on what type the parent block is and the nif version. This is mostly used in (potentially abstract) parent block types which have different default values when used in different blocks, but always the same field type in that place. Tags later in the same field take priority.

Attributes:

Name Format Description
type text For which block types this default should be used.
value text The value that should be used as the default.
versions list of version IDs Optional. For which nif versions this default should be used.

<niobject>

The <niobject> tag defines a new Nif objects. These are the core building blocks of a Nif file. Like compounds, they can contain <field> tags which insert new variables, but unlike compounds, they can inherit from each other. When one Nif object inherits from another, it gains all its variables, and any new variables added appear below these. For example, NiObjectNET has the Name variable, and NiAVObject inherits from NiObjectNET. While the <niobject> tag for NiAVObject makes no mention of the Name variable, all NiAVObject objects start with the Name variable that they inherited from NiObjectNET. Sometimes a <niobject> will have no <field> tags in it at all; all its variables coming from its ancestors.

The name of a <niobject> tag should be taken directly from a Nif file whenever possible. These are non-abstract types... in other words you're allowed to use these types in a Nif file. Abstract types, on the other hand, are only used as ancestors, and can not actually appear in Nif files. Whenever possible, the names of these should be determined from the SSG command in Morrowind or Oblivion. If we must invent a name ourselves for an abstract type, we previx it with a capital A.

Determining the proper inheritence structure of Nif objects can be one of the fastest ways to figure out what they are. An object that consists of several hundred bytes of information could be as simple as adding one new value on the end of an existing object. It also means that, in Niflib, the new object will inherit all the special functions that make it possible to work with that object. So figuring out the way Nif objects inherit from each other is very important.

Attributes:

Name Format Description
name text This is the name of this compound data type. It is used to refer to this type in other tags and will be used for the name of the C++ struct for Niflib. The first letter of each word should be capitalized and it should contain no spaces or underscores.
abstract true/false This determines whether this Nif object is abstract. If absent, false is assumed.
inherit Nif object name The name of the Nif object that this one inherits from. All Nif objects inherit from another except NiObject, which is the basis for all other Nif objects.
ver1 version ID Optional. The earliest Nif file version that this Nif object is known to appear in. If absent, it is assumed that this Nif object can appear in the earliest Nif files.
ver2 version ID Optional. The last Nif file version that this Nif object is known to appear in. If absent, it is assumed that this Nif object can appear in the most recent Nif files.
versions list of version IDs Optional. Which versions this object is known to appear in.
module text Optional. Which module this belongs to.

Tags Which can appear inside this one:

<field>

Example:

    <niobject name="NiObject" abstract="1">
        Abstract object type.
    </niobject>

Expressions

There are various attributes which can have expressions. These are: arr1, arr2, cond, vercond, arg and calc. For more extensive discussion of the expression grammar, see issue 73.

Predefined tokens

There are certain strings which, when used in the correct attributes, have a special meaning. They are formatted like tokens, but are not defined in the xml. Instead, these play a special role in the format:

#ARG#

#ARG# is currently used in field attributes arr1, arr2, cond and arg. This token is used to access data from the owning field/compound, specifically whatever it passed down in the arg attribute. A simplified hypothetical example is shown below:

    <struct name="VertexData">
        <field name="Vertex" type="Vector3"/>
	<field name="Vertex Normal" type="ByteVector3" cond="#ARG#" />
    </struct>

    <niobject name="NiVertexData">
        <field name="Num Vertices" type="ushort">How many vertices there are</field>
        <field name="Has Normals">Whether there are normals stored in the vertices</field>
        <field name="VertexArray" type="VertexData" arr1="Num Vertices" arg="Has Normals"></field>
    </niobject>

In the above example, the "Has Normals" field is passed down to all the VertexData structs within the VertexArray field of the NiVertexData compound. This way, the VertexData struct knows whether the "Vertex Normal" field should be present or not.

#T#

This token is currently only used in type and template attributes of field (not the typical expression attributes). I believe, like #ARG#, it can be used to obtain data from the owning field, and it will be filled with whatever type the template attribute of the owning field is set to.

#LEN[...]#

Expression to get the arr1 of the field within the square brackets.

... #THEN# ... #ELSE# ...

Currently only used in calc attributes. A simple if-statement, where the first part (before #THEN#) is evaluated to true/false and then the second or third part, respectively, gives the value of the element.

Clone this wiki locally